Elixir: Support Erlang 19 new features

[josevalim]

Changes that can be done at any time:

• [x] Support new Erlang typespec map syntax
• [x] No longer include __info__(:compile)[:time] in IEx.Helpers.i/1 info (this attribute has been removed)

Changes that require exclusive Erlang 19 support (therefore we must wait until we drop Erlang 18):

• [x] Use maps:take/2
• [x] Use io_lib:scan_format/1 in Logger
• [x] Use :binary.split/3 with :trim_all in String.Break.split/1
• [x] Add Port.monitor and Port.demonitor
• [x] Remove proper otp_release checks from the codebase

[whatyouhide]

I think that regarding dialyzer we only need to support %{...}, right?

[josevalim]

@whatyouhide typespecs now also make the distinction between required and optional keys, no?

[whatyouhide]

@josevalim yes, now if you do %{key_type => value_type} it means that that key-value pair is required, but there's no change in syntax there. This also changes the behaviour or %{} which now means empty map (with %{...} being "any map").

[josevalim]

@whatyouhide Oh, but there is new Erlang syntax! I have updated the issue to make it clearer. In a nutshell, we should be able to support everything an Erlang developer can write.

[whatyouhide]

@josevalim yeah, agreed. To recap more clearly, about changes in maps and typespecs:

• now KeyType := ValueType pairs are allowed in Erlang maps and they denote "associations" (kv pairs) that must be present. This will syntactically translate to just key_type => value_type in Elixir.
• the ... shorthand stands for the association any() => any(). This will be used as is in Elixir, ....
• #{} now means "empty map" (not "_any_ map") and #{...} should be used to denote any map. This will apply to Elixir as is.

[ericmj]

I will work on the typespec changes after we have Elixir working with the release candidate.

[developerworks]

Erlang/OTP 19 [erts-8.0] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [systemtap]


=INFO REPORT==== 19-May-2016::14:00:24 ===
    application: logger
    exited: {{shutdown,{failed_to_start_child,'Elixir.GenEvent',undef}},
             {'Elixir.Logger.App',start,[normal,[]]}}
    type: temporary
Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)

[josevalim]

@developerworks those have been fixed in master and in the v1.2 branch.

[ericmj]

PR for new map typespecs: https://github.com/elixir-lang/elixir/pull/4662.

[michalmuskala]

I got a compilation error on 19 with phoenix:

== Compilation error on file web/router.ex ==
** (CompileError) web/router.ex: internal error in v3_core;
crash reason: {case_clause,
    {'EXIT',
        {badarg,
            [{erl_anno,anno_info,[-1],[{file,"erl_anno.erl"},{line,360}]},
             {v3_core,record_anno,2,[{file,"v3_core.erl"},{line,2365}]},
             {v3_core,expr,2,[{file,"v3_core.erl"},{line,547}]},
             {v3_core,safe,2,[{file,"v3_core.erl"},{line,1549}]},
             {v3_core,expr,2,[{file,"v3_core.erl"},{line,536}]},
             {v3_core,safe,2,[{file,"v3_core.erl"},{line,1549}]},
             {v3_core,'-safe_list/2-anonymous-0-',2,
                 [{file,"v3_core.erl"},{line,1564}]},
             {lists,foldr,3,[{file,"lists.erl"},{line,1276}]}]}}}

  in function  compile:'-select_passes/2-anonymous-2-'/2 (compile.erl, line 521)
  in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line 306)
  in call from compile:fold_comp/3 (compile.erl, line 332)
  in call from compile:internal_comp/4 (compile.erl, line 316)
  in call from compile:'-do_compile/2-anonymous-0-'/2 (compile.erl, line 166)
    (stdlib) lists.erl:1338: :lists.foreach/2
    (phoenix) expanding macro: Phoenix.Router.__before_compile__/1
    web/router.ex:1: Famitool.Api.Router (module)

[josevalim]

@michalmuskala I believe Phoenix uses line: -1 in a couple places. We need to drop that in favor of line: 0. If that doesn't solve the warnings, we need to use generated: true (master only).

[josevalim]

/cc @chrismccord

[msimcoe]

Erlang/OTP v19.0 is available and I have installed it but, I am not sure how to make it work with IEx v1.3.0 when Erlang/OTP 18 is still installed on my os x machine. At the moment I am really just interested in getting the debugger to work. Sorry if this is the wrong place to bring this up. I'm just starting to learn Elixir and I am working the Dave Thomas's book. Any help you can give would be greatly appreciated.

[josevalim]

@msimcoe while this is not the correct place to bring it up, we are glad to help (as I believe others will end-up finding this exact same issue). For future reference, Stack Overflow or the Elixir Forum is a good place to ask questions.

Now, to the answer, Elixir v1.3.0 should just work with Erlang v19.0. All you need to do is to make sure the desired Erlang installation is in the PATH. You can check that by running which erl. If you installed Erlang from two different sources, you may need to uninstall or unlink it from one of them.

Have fun!

[josevalim]

Btw, after reading the whole changelog for Erlang 19, I have updated the list of changes we are applying to Elixir.

[msimcoe]

Thank you for the reply. I uninstalled ERL v 18 and reinstalled ERL v19 and everything appears to work fine now with IEx using ERL v19. Thanks.

[arsdehnel]

is there a fix I can apply locally to resolve the compilation error in @michalmuskala's comment? or do i just need to revert back to 18 for now as the best course of action?

[michalmuskala]

@arsdehnel as far as I know, the newest phoenix release from the 1.1 branch and phoenix 1.2 should be free from this issue.

[josevalim]

And you should also update phoenix_html to 2.6.

[arsdehnel]

Thanks I updated to 1.2 and it's working again now.

[zmackie]

PR for removing compile time info: #5196

[fishcakez]

There is also maps:get/3 we can use for Map.get/2,3, that arrived during OTP 18 but not for 18.0.

[Ch4s3]

maps:get/3 looks really useful. Would this be an appropriate place to stark working on the standard lib?

[josevalim]

@Ch4s3 we already provide something like maps:get/3, this is just a question of using those functions internally for performance. And we cannot work on those yet, not while we support earlier OTP versions.

[Ch4s3]

ahh, makes sense.

[Mogar21]

Is this error still an issue? I am getting this error with phoenix, Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false]
IEx 1.4.2

== Compilation error on file web/router.ex ==
** (CompileError) web/router.ex: internal error in v3_core;
crash reason: {case_clause,
{'EXIT',
{badarg,
[{erl_anno,anno_info,[-1],[{file,"erl_anno.erl"},{line,360}]},
{v3_core,record_anno,2,[{file,"v3_core.erl"},{line,2410}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,539}]},
{v3_core,safe,2,[{file,"v3_core.erl"},{line,1593}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,528}]},
{v3_core,safe,2,[{file,"v3_core.erl"},{line,1593}]},
{v3_core,'-safe_list/2-anonymous-0-',2,
[{file,"v3_core.erl"},{line,1608}]},
{lists,foldr,3,[{file,"lists.erl"},{line,1276}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,538}]},
{v3_core,safe,2,[{file,"v3_core.erl"},{line,1593}]},
{v3_core,'-safe_list/2-anonymous-0-',2,
[{file,"v3_core.erl"},{line,1608}]},
{lists,foldr,3,[{file,"lists.erl"},{line,1276}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,538}]},
{v3_core,safe,2,[{file,"v3_core.erl"},{line,1593}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,528}]},
{v3_core,safe,2,[{file,"v3_core.erl"},{line,1593}]},
{v3_core,'-safe_list/2-anonymous-0-',2,
[{file,"v3_core.erl"},{line,1608}]},
{lists,foldr,3,[{file,"lists.erl"},{line,1276}]},
{v3_core,expr,2,[{file,"v3_core.erl"},{line,652}]},
{v3_core,exprs,2,[{file,"v3_core.erl"},{line,512}]}]}}}

in function compile:'-select_passes/2-anonymous-2-'/2 (compile.erl, line 544)
in call from compile:'-internal_comp/4-anonymous-1-'/2 (compile.erl, line 329)
in call from compile:fold_comp/3 (compile.erl, line 355)
in call from compile:internal_comp/4 (compile.erl, line 339)
in call from compile:'-do_compile/2-anonymous-0-'/2 (compile.erl, line 177)
in call from compile:'-do_compile/2-anonymous-1-'/1 (compile.erl, line 190)
(stdlib) lists.erl:1338: :lists.foreach/2
(phoenix) expanding macro: Phoenix.Router.__before_compile__/1
web/router.ex:1: TodoApi.Router (module)
(elixir) lib/kernel/parallel_compiler.ex:117: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/1

[josevalim]

It is fixed in Elixir master as well as more recent Phoenix versions.

[josevalim]

This is open for contributions as master now requires OTP 19+.

[kelvinst]

@josevalim, where exactly in Logger you intend to use io_lib:scan_format/1?

Is it on Logger.Utils.inspect/4? The only reference to using the default erlang io.format/2 I found is there.

[josevalim]

@kelvinst yes!

[josevalim]

To be clear, I am not sure if it is possible to use it. Investigating it is part of the challenge. :)

[kelvinst]

Well, ok, I'll try it a little bit, I get back to you with a PR if any success. ;)

[kelvinst]

@josevalim or anyone else

I am trying to figure out how to solve this last bitstring optimization here, but I fear it's beyond my comprehension (punintentend)

Any links about what was done in OTP 19 and how you thought it can be used to optimize the comprehensions compiler output?

[josevalim]

@kelvinst in theory it means those two clauses: https://github.com/elixir-lang/elixir/blob/2d1e00264e7ea00a982092d20248029a4849b964/lib/elixir/src/elixir_erl_for.erl#L248-L252

can be written as:

comprehension_expr({bin, _, []}, Expr) ->
  {inline, Expr};

and that's it. If it works, it is true. if it doesn't, it doesn't . :)

[kelvinst]

Nice! I'll try it here! Do you think we need to add some tests to it? Or the ones we have are good enough?

[josevalim]

The ones we have are good enough.