Positional-only parameters for Python

Arguments can be passed to Python functions by position or by keyword—generally both. There are times when API designers may wish to restrict some function parameters to only be passed by position, which is harder than some think it should be in pure Python. That has led to a PEP that is meant to make the situation better, but opponents say it doesn't really do that; it simply replaces one obscure mechanism with another. The PEP was assigned a fairly well-known "BDFL delegate" (former BDFL Guido van Rossum), who has accepted it, presumably for Python 3.8.

Background

Since Python 1.0 or so, parameters to Python functions can be passed as positional arguments or as keyword arguments. For example:

    def fun(a, b, c=None):
        ...

    fun(1, 2)
    fun(1, 2, 3)
    fun(a=1, b=2, c=3)
    fun(c=3, a=1, b=2)

The function fun() takes two positional arguments and one optional argument, which defaults to None. All four of the invocations of the function shown above are legal as well. The only restriction is that positional arguments, those without a "keyword=", must all come before any keyword arguments when calling a function. So "fun(a=1, 2)" will raise a SyntaxError.

As can be seen above, any of the parameters can be passed as a keyword argument, even if the function author did not expect them to be. That means that changing the parameter name down the road, due to refactoring or for more clarity, say, may cause callers to fail if they are using the old name. That places more of a burden to come up with a "meaningful" name, even when there may be no real reason to do so (e.g. min(arg1, arg2)).

Positional-only parameters

It would be nice if library authors could indicate which parameter names are meant to be used only by positional arguments. In fact, some CPython builtins and standard library functions written in C are already able to specify and enforce positional-only arguments. Looking at help(pow) in the Python interpreter will show the function signature as follows:

    pow(x, y, z=None, /)

The "/" is a documentation convention that originated in the Python "Argument Clinic", which is a preprocessor to generate argument-handling code for CPython builtins. The "/" separates positional-only arguments from those that can be either positional or keyword (though the convention is not used in the pow() entry in the online documentation). Trying to call pow() in any of the following ways will lead to a TypeError being raised:

    pow(2, 4, z=5)
    pow(x=5, y=7)

PEP 570 ("Python Positional-Only Parameters") seeks to make that convention an actual part of the language syntax. As the PEP notes, using *args can accomplish the same goal, but it obscures the function's "true" signature. In a function definition, *args acts as a tuple that collects up any positional arguments that have not been consumed by earlier positional parameters. (The documentation for function-parameter syntax in Python is a bit scattered, as pointed out by this helpful blog post, which summarizes that information.)

Using a symbol to separate different kinds of parameters is an already-established precedent in Python. In 2006, PEP 3102 ("Keyword-Only Arguments") described using "*" in parameter lists to indicate that any following parameters must be specified as keywords. As noted in that PEP, one could emulate the * by using a dummy parameter (e.g. *dummy) to collect up any remaining positional arguments, but if that isn't an empty tuple, the function has been called incorrectly. Rather than force users to add a dummy parameter and test it for emptiness, * was added to have the same effect.

    def fun(a, b, *dummy, kword=None):
        if dummy:
	    raise TypeError
	...

    # becomes:

    def fun(a, b, *, kword=None):
        ...

PEP 570 extends that idea to a certain extent. As described in the "Specification" section, it would work as follows:

    def name(positional_or_keyword_parameters, *, keyword_only_parameters):

    def name(positional_only_parameters, /, positional_or_keyword_parameters,
             *, keyword_only_parameters):

There are some performance benefits to positional-only parameters, so providing pure-Python functions with a way to specify them would be helpful. In addition, since different Python implementations make their own choices about what language to use for standard library functions, inconsistencies can arise. Pure-Python implementations of standard library functions cannot exactly match the behavior of C-based functions due to the lack of positional-only parameters. Alternatives to CPython should not have to jump through hoops to emulate positional-only parameters simply because they have a chosen a pure-Python implementation for some standard library function.

Another thing to consider, according to the PEP, is consistency for subclasses. If a base class defines a method using one parameter name that is intended to be positional and a subclass uses a different name, calls using the parent class's parameter name as a keyword argument will fail for the subclass, as the PEP's example shows. Adding a positional-only parameter will remove that problem. Beyond that, there is a corner case that can be cleaned up:

    def fun(name, **kwords):
        return 'name' in kwords       # always False

    def fun(name, /, **kwords):
        return 'name' in kwords       # True for fun(a, name=foo)

This corner case also plays out in other scenarios. If a function uses a name for a parameter, that precludes callers from using it as a keyword argument elsewhere in the argument list. An example using the str.format_map() builtin:

    def fun(fmt, **kwords):
        fmt.format_map(kwords)

    fun('format: {fmt}', fmt='binary')  # TypeError because fmt is reused

If fun() could be defined using the proposed syntax, the "reuse" of fmt would not cause an error:

    def fun(fmt, /, **kwords):
        ...

In a long discussion on the Python Discourse instance (which, incidentally, demonstrates the deficiencies of Discourse's unthreaded discussion, at least for me), the idea was hashed out. Much of the objection turned out to be the use of "/", it seems. That bit of syntax was seen as ugly and/or unnecessary, but its heritage goes a fair ways back. It originated from Van Rossum in a 2012 python-ideas post; he pointed out that / is kind of the opposite of * (which is used to mark keyword-only parameters) in some contexts (e.g. Python arithmetic). No one, including Van Rossum, is entirely happy with using /, but no one has come up with anything less ugly—at least in his view.

That's not for lack of trying. The use case has been discussed multiple times along the way but, even just this time, there were suggestions for using a decorator-based approach, repurposing Python 2 tuple-unpacking parameters, or using double-underscore prefixes on parameters to mark them as positional-only. All of those were rejected for various reasons, which are described in the PEP. As might be guessed, the reasons are often not entirely convincing to the proponents of those ideas.

No change desired

Either leaving things as they are or, perhaps, even changing the C-based functions to accept all arguments as keywords were suggested in the discussion. Raymond Hettinger said that the / notation used in the runtime documentation for builtins has been a failure. He is strongly opposed to adding it as real syntax, at least in part because it will be difficult to teach. He is concerned that it is a fairly minor problem being solved:

Pablo Galindo Salgado, who has been shepherding the PEP, unsurprisingly disagreed with Hettinger's complaints. There are valid problems that would be solved with the new syntax, Galindo Salgado said. In addition, the help() output would become more useful because it would always correspond with what can be used in a def statement, unlike the situation today.

Steve Dower had a more sweeping idea. He would like to see all parameters allowed as keyword arguments:

But Van Rossum did not agree:

He is not particularly swayed by the "hard to teach" argument: "there are tons of advanced features that beginners don’t need to be taught". He also noted that adding the syntax that help() uses will help remove that confusion. Van Rossum made it clear that he believes the function's author should be in control of how the function is called, while Dower had the opposite view.

Along the way, Galindo Salgado pointed to a bug report from 2010 that could have been solved with positional-only parameters. In addition, Serhiy Storchaka created a work-in-progress pull request with a patch to change all positional-only parameters in the standard library once the new syntax is adopted. Van Rossum is not inclined to go quite that far, but does want to adopt the obvious cases.

As part of Storchaka's look into the standard library, he found a simple change that would automatically fix most or all of the corner-case variety of problems. As described in a bug report, the change would simply allow a keyword argument that duplicated a positional parameter name to be placed into **kwords. So the fmt example above would simply start working without the change in syntax. It is a fairly fundamental change to how the **kwords parameters work, however, so Van Rossum would like to see it get its own PEP and to discuss it separately.

The worries from Hettinger and others about teaching the new feature did lead Van Rossum to request that a new "How To Teach This" section be added to the PEP. That section is a draft for an addition to the "More on Defining Functions" section of the Python Tutorial. Steering committee member Carol Willing suggested adding some documentation helping to guide users on choosing between the various parameter types, along the lines of a blog post from Trey Hunner.

In truth, there was not much suspense about the outcome of this PEP. Fairly early on, Van Rossum tipped his hand in support of the PEP—well before the bulk of the thread was posted. It does seem like a useful addition to the language and one that can largely be ignored by those who don't need it. For those who do need it, though, it can make things a lot easier for certain types of functions.