Why not real anonymous functions?

For whatever reason I often find myself writing code like this:

def returns_a_func(this_var_gets_used):
    def _inner_func(*args, **kwargs):
        """Do some stuff in here"""

    return _inner_func

Or like this:

def some_useful_func(*args, **kwargs):
    """Does useful and interesting things, I promise"""

this_one_takes_a_func_arg(
    "foo",
    42,
    some_useful_func,
)

Maybe I’m mistaken but these kinds of patterns seem to arise a lot in functional-style code. The problem is that it feels needlessly verbose and repetitive. It’s not very economical from a purely syntactic standpoint.

I realize that functions are first-order objects in Python. I think my hangup is that they are semantically first-order objects but the Python syntax doesn’t reflect that fact. And of course I realize there is lambda but its usefulness is quite limited and its syntax is frankly a bit cumbersome.

So here’s my question: why not real anonymous functions a la JavaScript or Rust? Has this been proposed before? I did a quick search of PEPs and was not able to find anything.

As the language adopts more functional constructs it seems like this would be quite useful.

I’m not sure how difficult it would be to implement in the parser, but it seems like it would be relatively unambiguous to implement syntax like this:

def returns_a_func(this_var_gets_used):
    return (*args, **kwargs) => (  # are the parens actually useful/necessary here?
        """Do some stuff in here"""
    )

Similarly, like this:

this_one_takes_a_func_arg(
    "foo",
    42,
    (*args, **kwargs) => (
        """Does useful and interesting things, I promise"""
    ),
)

I supposed the biggest problem is knowing how to delimit the anonymous function. Languages with brackets don’t have this problem but it’s slightly more difficult with only whitespace, although it still seems unambiguous to me in most circumstances. Perhaps I’m also missing some trickiness around the scope of the parameters, although it seems like lambda would have the same problem.

Anyway, this is mostly just some idle questioning, but it has bothered me enough over the years to bring it up here. If it’s actually interesting and unobjectionable I’d be willing to consider drafting a PEP.

1 Like

Python has a lambda keyword that does this job.

this_one_takes_a_func_arg(
    "foo",
    42,
    lambda *args, **kwargs: """Does useful and interesting things, I promise""",
)

EDIT: Just saw your edit and that you’re aware, but then I’m not sure what you’re asking for that’s different.

3 Likes

If you’re asking “why not multiline lambda syntax” then there are years if not decades of previous iterations of this exact question that you can find on the mailing list and then these forums more recently. You’ll note that despite this long history, there are still no multiline lambdas, so that’s probably unlikely to change this time either.

Here’s a blog post from Guido in 2006 explaining it from a design rather than technical standpoint: Language Design Is Not Just Solving Puzzles

10 Likes

The functionality of lambda is severely restricted.

That’s useful context but it’s also sort of a non-answer. An opinion that may have been justified in 2006 may no longer be so in 2024, especially in light of developments in other languages. Python itself has adopted many functional constructs in the meantime. (Try to imagine using match in Python back in 2006!) So it seems worth revisiting at the least.

5 Likes

may”. You are the person asking, so if you think something has changed, go through the existing discussion, collect the arguments and show that they no longer apply.

6 Likes

Yes, that is very true. But what HAS changed? What new information are you bringing to the discussion? What other developments in what other languages have falsified previous statements?

4 Likes

When announcing Python 1 30 years ago, Guido gave ‘it is readable’ as a primary design goal and feature. This means naming intermediate objects when appropriate, even though this is not ‘economical’ with letters. The latter is not a goal of Python design.

Named functions have a name that is unique within its context and which can be printed in tracebacks. Non-nested functions can be tested. Lambda expressions work best when obvious enough to not need testing and when unlikely to be a direct cause of an exception. In practice, they seem expressive enough to cover most of the cases where they are most appropriate.

10 Likes

Given that a lambda defines an expression, how exactly do you imagine the syntax looking when someone wants to embed a multi-line lambda as part of a more complex expression, in turn used inside e.g. an if or for statement?

1 Like

Side note: When crossing between different languages, it’s actually quite interesting what the needs are for different features. I work a lot in JavaScript (which has inline functions of this style), and make extensive use of them for higher-order functions, for example (anonymized from the original):

const cached_gather_some_data = cache_data(function generate_data(x, y, z) {
    //... actual code which might be costly ...
    return some_data;
});

And that’s an extremely important thing to do! But in Python, I have an even better way to write this:

@cache_data
def cached_gather_some_data(x, y, z):
    # ... similarly expensive code
    return some_data

It technically is still the same thing (there’s a caching function which receives a function as an argument, does the cache management, and calls the original function when needed), but I can use a regular named function in Python thanks to decorator syntax.

8 Likes

Definitely true, although this does not prevent other languages from supporting anonymous function literals.

Also true, but one of the justifications for avoiding anoynmous function literals is that nested functions are a suitable replacement. So that’s not necessarily a good argument against anonymous function literals.

This is an opinion. A perfectly valid one, but I disagree. Also I think it’s interesting that all discussions about anonymous functions or function literals are framed in terms of lambdas. Lambdas are not the only solution to this problem. It’s just the one that Python happens to have chosen for a severely restricted implementation of anonymous functions.

This is fair and aligns with “explicit is better than implicit”. However, the economy I’m talking about isn’t really in terms of number of characters. It’s more in terms of syntactic parity. The fact that anonymous function literals are not supported as expressions in their own right seems inelegant to me.

A lot of people are asking “what has changed?” I think this is a strange line of questioning and merely serves as a rather tired defense of the status quo. What has changed is… everything, all the time. The world around us is constantly changing. We all encounter new problems and new ways of solving them. It’s reasonable to want to apply what we have learned and to approach old problems with new eyes.

My own opinion, informed by my experience with Rust and other languages, is that anonymous function literals are an elegant and useful construct. Personally I would love to use them in Python, especially as I gravitate more and more towards functional paradigms.

There may be perfectly good reasons that Python does not currently, and maybe will never, support anonymous function literals. But the question can and should be asked without any further justification.

1 Like

Sure! You just might not get answers you like. :joy:

9 Likes

Many of which have been enumerated before. These are precisely the objections which you’re dismissing because “everything changes all the time”. Maybe if you picked one or two of those, and explained precisely why they don’t apply any more, that might be a better argument than simply “everything changes, so let’s propose something that’s been rejected before, one more time”.

There may be perfectly good reasons why Python should now support anonymous functions. But if no-one ever puts together a proposal that actually addresses the reasons why this idea has failed so many times before, we’ll never know.

8 Likes

The problem is that people have asked it. Then asked it again. And again. And again. At some point, it doesn’t need to be asked more. As a maintainer, it’s not very fun to keep having to say no over and over in slightly different ways. The way you frame it, there’s a large imbalance in the effort and attention that’s required by the asker versus the maintainers. And since it seems like you don’t accept all the variations of “no” that have been provided, it’s not clear how this discussion can reach a conclusion.

5 Likes

In the past, the question was asked: Should Python have multiline anonymous functions? And the answer was: No.

You are now asking: Should Python have multiline anonymous functions? And we’re asking you why the answer shouldn’t be an automatic No.

Of course things change. That is a fact of life. But some things also do not change, and we need to know the difference. (If I may be al ittle facetious for a moment: “Everything has changed” does not mean that 1+1 is no longer 2.) So… WHAT has changed that is relevant to this question?

If you ask the question without further justification, all you’ll get is the same answer from previously. Which is: lambda and def are fine, we don’t need anything else. To get a different answer, you’ll have to add information that wasn’t known previously.

4 Likes

Can you please point to some specific references where the idea of anonymous functions has been explicitly proposed and rejected? I searched this forum as well as all of the PEPs before making my original post and there are exactly zero references to anonymous functions. As far as I can tell, the entire discussion up to this point has been circumscribed to “multi-line lambdas” which is one particular way to implement anonymous functions but is not coextensive with all possible implementations. Maybe I have missed something; truly I’m not trying to be obtuse.

I would ask the same question here: where have anonymous functions been explicitly proposed and rejected? Again, it seems like there has been a lot of discussion about multi-line lambdas. I don’t see many if any references to other potential implementations of anonymous functions. And most of the objections to multi-line lambdas seem to boil down to how ugly they would be.

There are several things that have changed. One of the most salient is the ascendency of functional paradigms in a wide variety of languages in the past few years, including Python itself. Programmers from a wide variety of backgrounds (e.g. Rust, JavaScript, Java) are familiar with anonymous functions and might reasonable expect to be able to use them in Python. Python explicitly supports a wide variety of functional constructs. It added structural pattern matching which would have been unimaginable only a few years ago. So it is reasonable to ask: why not anonymous functions? So far the only objection seems to boil down to “Python doesn’t do that because we’ve never done that.” I’m failing to see a stronger argument.

I think that probably the main objection is that Python would have a very difficult time delineating inline function expressions without introducing either parentheses or brackets. And that’s a valid objection! But Python is clearly open to introducing new kinds of syntax and other constructs, so why is this so touchy?

It’s not my job to find all the previous references. And I already linked to one from 2006. I’m sure you can find more if you want to put more effort into the proposal. Also, it’s not this specific one that’s “touchy”, literally every syntax change goes through this.

6 Likes

What’s the difference?

Wasn’t it because the parser changed (in CPython?), that it was finally possible to add this feature which would not have been possible to have with the former parser?

I’m inclined to move this to the Help category rather than Ideas at this point. Your actual question was “has this been proposed before”, the answer of which is “yes”. And you seemed to be looking for more resources than your “quick search” turned up.

You also framed it as “idle questioning” rather than a serious proposal that requires maintainer attention.

The Help category is a great place for the community to help you find more resources about this.

4 Likes

Sorry, I sincerely don’t mean to imply that you need to go digging up references for me. But I also ask you to give me the benefit of the doubt and assume I actually did some due diligence before posting this question to begin with and did not find any concrete proposals for anonymous functions outside of the existing lambda syntax. Not here, not in PEPs, and only very limited references on other forums. It’s entirely possible I’ve missed something.

That’s a strong reference given that it’s from BDFL. However, it’s also reasonable to ask whether an opinion held in 2006 is still relevant today.

Perfectly fair and reasonable.

2 Likes