Elixir: Support literal anonymous functions in pipe

[josevalim]

The goal is to support the following syntaxes:

string
|> &Regex.replace(~r/foo/, &1, "bar")
|> fn x -> Regex.replace(~r/bar/, x, "baz") end

I have originally argued against this extension because, if we add the syntax above, the following is not true:

replace_foo = &Regex.replace(~r/foo/, &1, "bar")
replace_bar = fn x -> Regex.replace(~r/bar/, x, "baz") end

string
|> replace_foo
|> replace_bar

I.e. I cannot simply replace the right-side of |> by a variable. However, this was a red herring because we can never replace the right side of |> by a variable in Elixir. Therefore, the only issue with this approach is the ambiguity between |> var, which means piping to a local function unless you write it as |> var.(). We could address this ambiguity by requiring parenthesis if you are trying to pipe to an existing variable.

Tasks

Those can be submitted as distinct PRs:

• [ ] Support fn in pipelines (it should raise if the function expects more than one argument)
• [ ] Support &local/1 and &Mod.remote/1 in pipelines (raise if arity is not 1)
• [ ] Support & in pipelines (it should raise if there is a nested & or &2)
• [ ] Warn if piping to a |> var and there is such var in scope (either var() or var.() should be used)

[QuinnWilton]

We could address this ambiguity by requiring parenthesis if you are trying to pipe to an existing variable.

I'm a big fan of this compromise. In my experience, the requirement of calling anonymous functions in a pipe with .() is a very common source of confusion for people who are new to the language.

[fuelen]

this piping does make sense only for anonymous functions with arity 1, so we can think about this as a sugar around case :)

|> case do
  nil -> "None"
  string -> "String: #{string}"
end
|> ...

equal to

|> fn
  nil -> "None"
  string -> "String: #{string}"
end
|> ...

---

|> case(do: (x -> Regex.replace(~r/foo/, x, "bar")))
|> case(do: (x -> Regex.replace(~r/bar/, x, "baz")))

[josevalim]

Good point @fuelen! That could even be used to implement this functionality itself. i.e. fn could be rewritten to case. Handling & is a bit more complicated but can be used around the same lines.

[hauleth]

@fuelen actually Macro.pipe/3 supports position as a 3rd argument, so it can be translated to insert the argument directly where it is needed and remove need for both case and fn.

[hauleth]

@josevalim the question is how should:

foo |> &foo(bar(&1))

behave.

[fertapric]

I am not sure if @hauleth meant this, but I think there might be a possible conflict with local functions. For example:

defmodule Foo do
  def foo do
    bar = fn x -> x + 2 end

    1 |> bar()
  end

  def bar(x), do: x + 3
end

[josevalim]

@hauleth

foo |> &foo(bar(&1))

Is the same as:

case foo do
  x -> foo(bar(x))
end

There is no ambiguity.

[josevalim]

@fertapric your example is not ambiguous too. foo |> var() will call var(foo). foo |> var.() will call var.(foo). The only ambiguity is foo |> var, which is why I propose we warn in those cases (in case there exists such a var).

[fertapric]

Warn if piping to a |> var and there is such var in scope (either var() or var.() should be used)

The conflict comes from allowing the var() syntax. For what I understand, this would work:

my_fun = fn x -> x + 2 end
1 |> my_fun()

But what if there is also a my_fun/1 function defined locally in the same module? Which one would it pick, the anonymous function or the local function? 🙂

[josevalim]

@fertapric as I said above, my_fun() will always be a local module function, never the anonymous function. my_fun.() is the anonymous function call. This ambiguity exists today, before this PR, and it is already addressed this way. :)

[Qqwy]

As you might be aware I am a huge supporter of this addition! :heart:

One of the problems I came across when I was attempting to implement this in user-space was that currently & has a lower parsing precedence than |> which means that when e.g. used like this:

a |> &b |> c |> d

is converted to

a |> &( b |> c |> d)

_(here, a, b, c and d are arbitrary expressions)_

This is a problem when c or d also is a capture, like

a |> &b |> &c |> d

which would be expanded to

a |> &(b |> &(c |> d))

which ends up being a nested capture.

So is the precedence of & w.r.t. |> going to change? Or is there another way this problem will be tackled?

[josevalim]

We can't change the precedence without making it a breaking change. We will have to recommend everyone to use parens. We can probably detect this by seeing if the root of the &is a pipe.

[hauleth]

If it would be up to me, then I would allow only one thing: &1 (or similar) as a top-level placeholder. So foo |> bar() would be the same as foo |> bar(&1). This would be partially "special case", but as it is mentioned above, we already can use case as such syntax, but this would allow using more Erlang functions in the pipelines without the temporary variables. And as &1 would be allowed only as a top level placeholder (so foo |> bar(baz(&1)) would be an error) there would be no ambiguity in case of foo |> bar(&1, &(&1 + &1)).

[josevalim]

There is ambiguity however with & value |> bar(123, &1), which is already valid today.

[hauleth]

I can agree on any other "placeholder token". I would use _ as a placeholder, but I believe that it would not be accepted as in most cases it has different meaning and can be used if the call in pipeline is a macro.

EDIT: Ok, I have found one token that could be used that is currently meaningless ^_.

[josevalim]

Although I guess in this case the behaviour is still clear: &1 will always apply to the closest &. The precedence issues mentioned by @Qqwy discourages me a bit on the initially proposed syntax, so something that doesn't require a leading & seems to be better.

The option of going with &1 or picking another placeholder may be the best way to go. We could pick &_ or ... or similar but I am a bit worried about introducing another token for this. Since we are unsure, perhaps it is best to make a small library out of this and have people try it out, which I believe is something @Qqwy was already planning on doing.

For now, it is probably best to close this. Thanks everyone for the input!

[Qqwy]

I don't actually believe that altering the relative precedence would be a breaking change since older Elixir-versions would balk at a literal anonymous function in a pipe anyway.

(But do prove me wrong with a counter-example :slightly_smiling_face: )

---

I am indeed planning on writing a small library that does this in user-space. We'll see how easy (or how convoluted) it will end up being to write without being able to e.g. alter the relative precedence in Elixir's parsing.

[josevalim]

(But do prove me wrong with a counter-example 🙂 )

This compiles fine today: & &1 |> foo(). It will fail if you change the precedence.

[josevalim]

I am indeed planning on writing a small library that does this in user-space. We'll see how easy (or how convoluted) it will end up being to write without being able to e.g. alter the relative precedence in Elixir's parsing.

My proposal is to go along @hauleth's suggestion and use a different separator without requiring a leading &. So no precedence issues.

[Qqwy]

For posterity: The library I promised to create that implements this has now been released.
I believe that it is fully backwards-compatible (let me know if you find a mistake) and I hope that we might still add this functionality to Elixir proper in the future.