Elixir: recompile() does not load newly compiled module

[mpoeter]

Environment

• Elixir 1.5.1 & Erlang 20
• Operating system: Ubuntu 17.04

Current behavior

I have an umbrella app with 2 embedded apps. I start the app with iex -S mix in the root directory of the umbrella app. When I change the source code of some module of one of the apps and call recompile I get a very inconsistent behaviour - sometimes I get an output that some files get recompiled and that the function returns :ok, but sometimes the function simply returns :noop. To make matters worse, sometimes the changes are actually picked up, even though recompile returned :noop. But at some point, recompile will always return noop and no changes will be picked up any more.

I digged through the Mix/Elixir code to get and idea of how recompile works, so I can provide as much information as possible. I can confirm that the manifest file gets updated, as well as the corresponding beam; I decompiled the updated beam file into erlang code and I can confirm that it contains the changed source.

When I call :code.module_status for the changed module it returns :modified. Calliing :code.soft_purge for the module returns true, so (assuming that I understand the erlang documentation correctly) the module is not currently used by any process and can therefore be purged. When I then call :code.delete for the module, and call some function of from that module the newly beam file is picked up.

I did not yet find how recompile is supposed to load the updated modules...

Expected behavior

recompile compiles all the changed files and loads the updated modules.

I am aware that this is probably very difficult to reproduce for someone else. Even on my machine I can not always consistently reproduce this behavior in a deterministic way, but sooner or later I always run into it. I am happy to do some more digging and debugging on my side, but right now I am at point where I could use some input from the experts! :-)

[josevalim]

Thanks for report @mpoeter. Could it be that the issue is related with running in the umbrella root and with some caches?

It may very likely be that, after you compile a given project once, you will never compile it again unless you restart IEx. This should be easy to verify:

1. Start IEx in the umbrella root
2. Change a file in one of the applications
3. recompile should return :ok
4. Change another file in the same application
5. recompile should return :noop

It may even be a more drastic scenario in that only the first call to recompile works, regardless if it compiled something or not.

[mpoeter]

I just tried the following steps:

1. Start IEx in the umbrella root
2. Add a new function foo that returns :foo to some module
3. Call recompile -> returns :noop
4. Try to call the just added function foo -> the function exists and returns :foo
5. Change foo to return :bar instead
6. Call recompile -> actually compiles some files and returns :ok
7. Call foo again -> still returns :foo

I repeated the same steps and experienced the exact same behavior, except that this time the second call to recompile (step 6) also returned :noop.

Then I tried the same in one of the 2 apps instead of the umbrella root, and I experienced the same behavior with regards to the return value of recompile - sometimes it returns :noop, sometimes I see some output that it actually compiles some files and returns :ok, but a call the modified function at least always returned the new value.

I again repeated the same steps and this time I experienced exactly the same behavior as in the umbrella app, i.e., the updated module did not get loaded and the function always return the old value.

No idea what is going on, but at least I can confirm now that this phenomenon is not limited to umbrella apps.

[josevalim]

Thank you. The next step then is for us to try to reproduce it locally.

[josevalim]

I cannot reproduce this behaviour. Here is what I did:

$ mix new foo --umbrella
$ cd foo/apps
$ mix new bar
$ iex -S mix

Then after IEx started, I added def world, do: :hello to "apps/bar/lib/bar.ex". I invoked recompile and it returns :ok and I could invoke the function. Then I changed the function to def world, do: :hello! and recompile also returned :ok.

Can you reproduce the issue using the steps above? And, if you can, can you make sure there is nothing in your editor or in your tooling automatically triggering recompilation that would cause recompile to then misbehave?

Thank you.

[mpoeter]

Your guess regarding the tooling was dead right!
I am using VS Code and had an Elixir Linter plugin which apparently performed silent recompilations in the background. At least I could no longer reproduce this behavior after disabling this plugin.
I am not quite sure why this would cause such issues with recompile, but the main point is that it works now!
I would not have thought of that - thank you very much!

[OvermindDL1]

Yeah those plugins call mix compile to get an errors to report back, so it would recompile the file.

[tmepple]

Hi @mpoeter, you can work around this issue by setting "elixirLinter.mixEnv": "test" as detailed here: https://github.com/PotterDai/vscode-elixirc-mix-linter/issues/2. This way the linter calls mix compile in the test environment which doesn't affect iex> recompile() which by default runs in the dev environment. Hope this helps!

[mpoeter]

Yes, this helps. Thanks!

[wangbus]

recompile() for new projects in MIX_ENV=dev is dead in the water. Tested with Elixir 1.4.5, 1.5.0, 1.5.2 on Erlang 20.1. Works fine with MIX_ENV=test and using the exact same config/config.exs

[josevalim]

@wangbus I have just checked locally and it works fine. Please make sure there is nothing compiling your app in the background, such as an editor, a Phoenix app or a dependency. If the issue still persists please open up a new issue with a project that reproduces the bug.

[wangbus]

@josevalim Looks like you were right. Another dependency was recompiling. Thanks for the hint and for your quick response!

[LoranceChen]

@mpoeter +1. With @josevalim's simple foo, bar example, Emacs alchemist also have the same problem. besides, I can upgrade success with recompile() after edit code in vi (not install elixir relative plugin).

[wangbus]

I actually never got this fixed in the later versions. Opting to not use vim-elixir at the moment. Not sure why this causes issues.

[LoranceChen]

@wangbus, thanks for your effort! Besides, from a refer toipc, hot reload could be work by setting with a different environment on MIX_ENV=test iex -S mix.
hope the tips will easy for others as a temporary solution.