Elixir: somewhat confusing syntax error when using `with` macro

[22a]

Environment

• Elixir & Erlang versions (elixir --version):

Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.6.4 (compiled with OTP 20)

• Operating system: macOS 10.13.3 (17D102)

Current behaviour

It is semantically incorrect to place a newline between the with "keyword" and the first <- expression.

this works:

defmodule Foo do
  def bar do
    with str <- 5 |> Integer.to_string,
         num <- str |> String.to_integer
    do
      IO.inspect str
      IO.inspect num
    end
  end
end

whereas this does not work:

defmodule Foo do
  def bar do
    with
         str <- 5 |> Integer.to_string,
         num <- str |> String.to_integer
    do
      IO.inspect str
      IO.inspect num
    end
  end
end

and presents the following error:

Erlang/OTP 20 [erts-9.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

** (SyntaxError) foo.ex:6: syntax error before: do
    (elixir) lib/code.ex:677: Code.require_file/2

Expected behaviour

The syntax error message isn't as helpful as some of the other Elixir error messages. I might expect this error message to give me a little more information about what I was doing wrong instead of just saying that something was wrong before do on line 6.

I'm not certain if there's anything that can be improved here or if this is in any way actionable but nonetheless I wanted to open an issue in case anyone else makes the same whitespace/newline sensitive mistake.

[josevalim]

We can check if the do is after a newline or a ; and raise accordingly. We can do it in the tokenizer.

[whatyouhide]

We can check if the do is after a newline or a ; and raise accordingly. We can do it in the tokenizer.

@josevalim you mean in general? Because this compiles fine:

defmodule Foo do
  def foo()
  do
    :ok
  end
end

[josevalim]

@whatyouhide good catch. Now I am not sure if there is something we could do. The tokenizer definitely does not have enough information. We could start warning and forbid do at the beginning of a line but then it means we would no longer support some of those cases and I think some developers rely on this idiom quite a lot.

[22a]

Perhaps this example will illustrate my confusion a little better:

This works:

defmodule Foo do
  def bar do
    with str <- 5 |> Integer.to_string,
         num <- str |> String.to_integer do
      IO.inspect str
      IO.inspect num
    end
  end
end

whereas this:

defmodule Foo do
  def bar do
    with
         str <- 5 |> Integer.to_string,
         num <- str |> String.to_integer do
      IO.inspect str
      IO.inspect num
    end
  end
end

results in this error:

Erlang/OTP 20 [erts-9.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

** (SyntaxError) foo.ex:5: syntax error before: do
    (elixir) lib/code.ex:677: Code.require_file/2

the issue being that the error message is not really indicating that the newline on line 3 is causing the syntax error

[whatyouhide]

Oh the error seems to be that there can't be a newline after with then. @josevalim maybe this is something we can take care of in the tokenizer?

[josevalim]

@whatyouhide the newline after with is fine. It is treated as the variable with. The issue is that in the next line we are trying to use a do without having a call to attach it too. You could remove the with altogether and it won't affect the error.

[whatyouhide]

Oh, got it now. Not sure how to improve it then :\

[tiagodavi]

Hi Guys. I would like to know how I can help here. I've already read the main page about "contributing", but what should I do exactly to show that I would be working in this particular issue or any of others ? Are there people working on the same issue at the same time and the team decide what the best solution is ? How does it work exactly to start "contributing" ?

[josevalim]

@tiagodavi typically you just say you are going to tackle it or you send a PR. Note we haven't necessarily agreed on a solution for this particular issue, so it may be that the solution you provide won't be accepted. If we do not accept it though, we will let you know about the reason. :)

[tiagodavi]

@josevalim Thank you for informing me.

[josevalim]

Unfortunately I don't believe there is much we can do here. As explained above, the issue is not related to with at all, because with by itself in its own line is fine. I believe this is one of the rare and unfortunate cases we can't do much to improve an error message. Apologies.