Elixir: Command `recompile` unstable

[mguimas]

Environment

$ elixir --version
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.9.4 (compiled with Erlang/OTP 22)

Current behavior

The command recompile called in succession can give unstable results, as in the following example:

iex(14)> recompile
Compiling 2 files (.ex)
Generated my_app app
:ok
iex(15)> recompile
Compiling 2 files (.ex)
warning: function inc_element/1 is unused
  lib/my_app/box/element/load_element.ex:406

warning: function inc_element/2 is unused
  lib/my_app/box/element/load_element.ex:401

warning: function lk_xpto/1 is unused
  lib/my_app/box/element/load_element.ex:388

warning: function lk_xpto/3 is unused
  lib/my_app/box/element/load_element.ex:393

warning: function set_thing_xpto/3 is unused
  lib/my_app/box/element/load_element.ex:397


== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:356: undefined function lk_pls/1
    (elixir) src/elixir_locals.erl:108: :elixir_locals."-ensure_no_undefined_local/3-lc$^0/1-0-"/2
    (elixir) src/elixir_locals.erl:109: anonymous fn/3 in :elixir_locals.ensure_no_undefined_local/3
** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(15)> recompile
Compiling 2 files (.ex)
** (exit) exited in: :gen_server.call(#PID<0.478.0>, :remote_references, 30000)
    ** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
    (stdlib) gen_server.erl:223: :gen_server.call/3
    (mix) lib/mix/compilers/elixir.ex:334: Mix.Compilers.Elixir.each_file/4
    (elixir) lib/kernel/parallel_compiler.ex:397: Kernel.ParallelCompiler.wait_for_messages/7
    (elixir) lib/kernel/parallel_compiler.ex:136: Kernel.ParallelCompiler.spawn_workers/3
    (mix) lib/mix/compilers/elixir.ex:191: Mix.Compilers.Elixir.compile_manifest/9
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.all.ex:73: Mix.Tasks.Compile.All.run_compiler/2
    (mix) lib/mix/tasks/compile.all.ex:53: Mix.Tasks.Compile.All.do_compile/4
iex(15)> recompile
Compiling 2 files (.ex)
warning: function inc_element/1 is unused
  lib/my_app/box/element/load_element.ex:406

warning: function inc_element/2 is unused
  lib/my_app/box/element/load_element.ex:401

warning: function lk_xpto/1 is unused
  lib/my_app/box/element/load_element.ex:388

warning: function lk_xpto/3 is unused
  lib/my_app/box/element/load_element.ex:393

warning: function set_thing_xpto/3 is unused
  lib/my_app/box/element/load_element.ex:397


== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(15)> recompile
Compiling 2 files (.ex)
** (MatchError) no match of right hand side value: nil
    (elixir) lib/kernel/parallel_compiler.ex:319: anonymous fn/3 in Kernel.ParallelCompiler.without_definition/2
    (elixir) lib/enum.ex:1948: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir) lib/kernel/parallel_compiler.ex:318: Kernel.ParallelCompiler.without_definition/2
    (elixir) lib/kernel/parallel_compiler.ex:300: Kernel.ParallelCompiler.spawn_workers/7
    (elixir) lib/kernel/parallel_compiler.ex:136: Kernel.ParallelCompiler.spawn_workers/3
    (mix) lib/mix/compilers/elixir.ex:191: Mix.Compilers.Elixir.compile_manifest/9
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.all.ex:73: Mix.Tasks.Compile.All.run_compiler/2
iex(15)> recompile
Compiling 2 files (.ex)

== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(15)> recompile
Compiling 2 files (.ex)

== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(15)> recompile
Compiling 2 files (.ex)

== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(15)>
nil
iex(16)> recompile
Compiling 2 files (.ex)

== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(16)> recompile
Compiling 2 files (.ex)

== Compilation error in file lib/my_app/box/element/load_element.ex ==
** (CompileError) lib/my_app/box/element/load_element.ex:13: cannot define module MyAppStore.box.element.Loadelement because it is currently being defined in lib/my_app/box/element/load_element.ex:13

** (exit) shutdown: 1
    (mix) lib/mix/tasks/compile.all.ex:59: Mix.Tasks.Compile.All.do_compile/4
    (mix) lib/mix/tasks/compile.all.ex:24: anonymous fn/1 in Mix.Tasks.Compile.All.run/1
    (mix) lib/mix/tasks/compile.all.ex:40: Mix.Tasks.Compile.All.with_logger_app/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (mix) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix) lib/mix/task.ex:331: Mix.Task.run_task/3
    (iex) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1
iex(16)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
$ iex -S mix
Erlang/OTP 22 [erts-10.7.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Config MyApp Store
Config MyApp Store (dev)
Compiling 2 files (.ex)
Interactive Elixir (1.9.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>

Unfortunately I cannot reproduce the issue (the project is too complex for that), so I hope this output may shed some light on what needs to be fixed.

Expected behavior

The command recompile should work as expected without failures.

[josevalim]

Thank you. We will leave it open for a while to see if others see the same issue but unfortunately, unless someone can provide a way to reproduce this, there isn’t much we can do.

[pera]

Similar issue here after calling recompile:

Erlang/OTP 22 [erts-10.6.4] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.10.1 (compiled with Erlang/OTP 22)

** (MatchError) no match of right hand side value: nil
    (elixir 1.10.1) lib/kernel/parallel_compiler.ex:427: anonymous fn/3 in Kernel.ParallelCompiler.without_definition/2
    (elixir 1.10.1) lib/enum.ex:2111: Enum."-reduce/3-lists^foldl/2-0-"/3
    (elixir 1.10.1) lib/kernel/parallel_compiler.ex:426: Kernel.ParallelCompiler.without_definition/2
    (elixir 1.10.1) lib/kernel/parallel_compiler.ex:299: Kernel.ParallelCompiler.spawn_workers/7
    (elixir 1.10.1) lib/kernel/parallel_compiler.ex:144: Kernel.ParallelCompiler.spawn_workers/3
    (mix 1.10.1) lib/mix/compilers/elixir.ex:195: Mix.Compilers.Elixir.compile_manifest/9
    (mix 1.10.1) lib/mix/task.ex:330: Mix.Task.run_task/3
    (mix 1.10.1) lib/mix/tasks/compile.all.ex:76: Mix.Tasks.Compile.All.run_compiler/2
    (mix 1.10.1) lib/mix/tasks/compile.all.ex:56: Mix.Tasks.Compile.All.do_compile/4
    (mix 1.10.1) lib/mix/tasks/compile.all.ex:27: anonymous fn/2 in Mix.Tasks.Compile.All.run/1
    (mix 1.10.1) lib/mix/tasks/compile.all.ex:43: Mix.Tasks.Compile.All.with_logger_app/2
    (mix 1.10.1) lib/mix/task.ex:330: Mix.Task.run_task/3
    (mix 1.10.1) lib/mix/tasks/compile.ex:96: Mix.Tasks.Compile.run/1
    (mix 1.10.1) lib/mix/task.ex:330: Mix.Task.run_task/3
    (iex 1.10.1) lib/iex/helpers.ex:104: IEx.Helpers.recompile/1

After ^C^C and restarting iex everything was back to normal.

[josevalim]

Hi @pera, do you have any way to reproduce the issue, so we can take a look at it? Thanks.

[pera]

Hey @josevalim, unfortunately no: I just restarted iex -S mix which did the recompile by itself without problems, and every recompile during that session ran successfully as well. By the way this was not the first time I had an issue with recompile throwing some error within "parallel_compiler.ex" (I don't know if it was the same exact error though).

If this happens again, is there any way I could "dump" some BEAM debugging info that could be useful to you? Thank you!

[josevalim]

When you said it actually fixed automatically the next time, you made me wonder if it was a race condition and not a file system issue, which helped me spot it. Thanks!