Nim: RFC: Lambda expression (with inferred placeholders)

[drslump]

This is something I implemented in a toy language some years ago and I really enjoyed the results (at a personal scale mind you), so I wanted to share it in case it's useful for Nim.

This proposal seeks to introduce a more ergonomic syntax hopefully better suited for functional style programming than proc expressions or the do notation. It seeks to be very lightweight so it's the natural go to solution for what it works well. Its limited scope is to ensure it's not abused to create complex anonymous functions that become hard to test and might also help the compiler with optimizations.

The syntax is simply "\" expr. The \ character has some convenient features in my opinion, it's very obvious when reading code yet lightweight, has a strongly stablished association with escaping stuff and, perhaps more importantly, resembles λ.

Semantics of the expression is escaping, similarly to its common use inside string literals, just that in this case escapes an expression so it's not computed at that point but wrapped in a lambda for future use. In a sense it converts an expression into a deferred computation.

The next trick it does is that it works without explitic placeholders, inferring which values are missing from the, potentially incomplete, expression. Given how Nim handles operators and since it's statically typed this seems complicated, so no idea if it's even possible to implement.
Ideally the incomplete expression would add a placeholder node to any potentially missing value position deferring typing, once it gets attached as a callback it would check how many arguments are required by the callback and instantiate it filling in the placeholders in some prioritized order, disambiguating prefix/binary operators, and producing and error only if arguments > placeholders.

Some contrived examples to clarify the idea:

lst.map \ $
# lst.map( proc (x: int): string = $x )
lst.map \ + 2
# lst.map( proc (x: int): int = x + 2 )
lst.filter \ 3 <   # arguable code style though
# lst.map( proc(x: int): bool = 3 < x )
lst.reduce(\ +, 0)
# lst.reduce( proc(a,b: int): int = a+b, 0 )
lst.sort \ cmp(.lower, .lower)
# lst.sort( proc(a, b: string): int = cmp(a.lower, b.lower) )

Although I can imagine that a first implementation would require explicit placeholders (i.e. \ _ + 2) and then those examples offer little advantage over the current templates in sequtils for instance. There is however a key difference in that these lambda expressions can be used anywhere that accepts a callback, no need to create specific templates for it, it's just a callable from the user's perspective. Library authors can of course leverage it though and easily inline the expression on a macro if desired, but from the user's point of view it's fully transparent if it's being used on a proc or a macro.

Additionally they are clearly demarcated by the \ character, which once learned you can quickly understand that the computation is being delegated. Moreover, my intuition is that this could replace many uses of an anonymous proc or a do notation, oftentimes they just need to apply a simple operation over a value, hence making those a bit of a code smell that the code is perhaps not well structured.

Unlike the fat arrow sugar => this goes all the way in on terseness by eliding the declaration, it's a specific tool for a common problem, in a sense it's similar to a regex. In my opinion though, if all it ends up doing is replacing the => macro with a different syntax then it might not be worth it, it should feel as a language feature regardless of how it's implemented.

Note also that the \ character is currently a valid operator so using it for this could break some code. I couldn't say how popular of an operator it is though.

[mratsim]

I'm not sure about the final syntax but I always thought the "it" in mapIt and the a and b in foldl just came out of nowhere.

Basically you have to check the documentation to know how to use them. I also would like to see some effort on template {.inject.} syntax.

Note that for advanced uses on non-public templates, the current syntax is very useful though.

[dom96]

Let's compare to =>:

lst.map \ $
lst.map(x => $x)
# lst.map( proc (x: int): string = $x )
lst.map \ + 2
lst.map(x => x + 2)
# lst.map( proc (x: int): int = x + 2 )
lst.filter \ 3 <   # arguable code style though
lst.filter(x => 3 < x)
# lst.map( proc(x: int): bool = 3 < x )
lst.reduce(\ +, 0)
lst.reduce((a, b) => a+b, 0)
# lst.reduce( proc(a,b: int): int = a+b, 0 )
lst.sort \ cmp(.lower, .lower)
lst.sort((a, b) => cmp(a.lower, b.lower))
# lst.sort( proc(a, b: string): int = cmp(a.lower, b.lower) )

I consider => clearer with little extra typing.

[awr1]

=> is more familiar to users of other languages, too, e.g. JS. As I'm worried that any discussion of lambda syntax might open up the possibility of deprecating other forms, I feel that do-notation should be kept around alongside any new lambda form.

[RSDuck]

it's already possible to implement this partially using macros. My implementation is very shallow, but it covers the base idea:

import macros, sequtils, strutils

macro `\`(e: untyped): untyped =
    e.expectKind nnkPar
    e[0].expectKind {nnkPrefix, nnkCommand, nnkAccQuoted}

    let param = newIdentNode("x")

    var
        call = nnkCall.newTree(e[0][0], param)

    for i in 1..<e[0].len:
        call.add(e[0][i])

    nnkLambda.newTree(
        newEmptyNode(), newEmptyNode(), newEmptyNode(), 
        nnkFormalParams.newTree(bindSym"auto", newIdentDefs(param, bindSym"auto", newEmptyNode())), 
        newEmptyNode(), newEmptyNode(), 
        call)

echo [2, 3, 5, 6].map(\(+ 10))
echo [10, 15, 3, 1].map(\(* x))

[Araq]

I'm not sure about the final syntax but I always thought the "it" in mapIt and the a and b in foldl just came out of nowhere.

I think it's a lovely quirk and makes Nim stand out. We should strive to have very few of these "lovely quirks" though...

[drslump]

Thank you guys for the input! some minor remarks about the feedback following.

... I always thought the "it" in mapIt and the a and b in foldl just came out of nowhere.

Totally, it's the main motivation behind this RFC, expose clearly and conveniently to the user what's the expected semantics of the expression in a way that works uniformly in the language.

I consider => clearer with little extra typing

Indeed, just a thought, familiarity is not the same as simplicity. In my opinion \ has the potential to be simpler if it's properly implemented otherwise it'll be utterly confusing which would make opting for the familiarity of => objectively better.

I feel that do-notation should be kept around alongside any new lambda form

Agree, if anything this kind of lambda expression could deprecate proc expressions. Although the do notation is quite special and the best solution for its use cases is still out there :)

it's already possible to implement this partially using macros.

Cool! only with explicit placeholders and using parenthesis though. The rules for unary operators make them bind stronger than binary ones, so even using \= as operator to reduce the binding precedence won't work because it's unary. At that point it starts being confusing and the familiarity of => makes that a superior solution.

[dom96]

As I'm worried that any discussion of lambda syntax might open up the possibility of deprecating other forms, I feel that do-notation should be kept around alongside any new lambda form.

I disagree and in fact I'm planning to push for the complete removal of the do notation. I've always felt it didn't fit the language and I still do to this day. Procedure expressions and => are enough.

[timotheecour]

@drslump @dom96 @mratsim
I have a better proposal, see this PR https://github.com/nim-lang/Nim/pull/8679 ; it has all the advantages of => syntax (familiarity, no magic it), and none of the disadvantages (lack of type inference, inefficiency due to function pointer indirection)

[cooldome]

I think you all forget that in Nim you don't need a lambda at all to pass a piece of code as the argument. Other languages simply don't have such functionality and that why they tend to wrap everything in lambdas. IMO, Nim way is: use template instead and avoid lambda. It gives good performance and syntax is better than all of you have proposed.

[Clyybber]

@cooldome I dont think the syntax is better. it in mapIt is a magic variable with arbitrary naming.

[drslump]

I'm retracting this proposal to focus on https://github.com/nim-lang/Nim/issues/8678. Thanks for the feedback!