Elixir: Mix does not recompile file on local dependency change

[fbergero]

Environment

• Elixir & Erlang/OTP versions (elixir --version):
  Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.6.5 (compiled with OTP 20)

• Operating system:
  Linux de1572f61a8c 4.13.0-43-generic #48~16.04.1-Ubuntu SMP Thu May 17 12:56:46 UTC 2018 x86_64 GNU/Linux

Current behavior

First I create a application a that depends on b. b application is a local dependency:

mix new a
cd a
mix new b

Now in a/mix.exs I put the dependency {:b, path: "./b"}. The b app defines a struct

 defstruct [:b1, :b2]

and a uses it like

IO.inspect %B{b1: 1, b2: 2}

which when executed prints (as expected)

%B{b1: 1, b2: 2}

Now if I remove the b2 field from the B struct, a does not gets recompiled and prints

%{__struct__: B, b1: 1, b2: 2}

Expected behavior

I would expect mix to recompile a (failing), since one of its dependency defining a struct that a uses (b) changed.

Just saving the file that uses the B struct forces a recompile failing with (which should be the expected behaviour)

== Compilation error in file lib/a.ex ==
** (KeyError) key :b2 not found in: %B{b1: 1}
    (stdlib) :maps.update(:b2, 2, %B{b1: 1})
    lib/b.ex:6: anonymous fn/2 in B.__struct__/1
    (elixir) lib/enum.ex:1899: Enum."-reduce/3-lists^foldl/2-0-"/3

[fbergero]

I couldn't check if this is a problem of the local dependency or if it also happens with hex/git deps.

[uesteibar]

If that's fine, I'm having a look into this one :)

[josevalim]

@uesteibar please go ahead!

[uesteibar]

I dug a little bit in the issue and after some tests the only solution I came up with is to force compile when a local dependency is modified. Not sure if this is the best solution, you can have a look at https://github.com/uesteibar/elixir/commit/63eea812667c226d3088f52820743e600a9e69a9.

If the approach is fine, I'll clean it up and open a PR tomorrow.

[josevalim]

@uesteibar the first question I have is: is the b application even recompiled? Or not really?

The reason I am asking is because we already have code for handling stale local deps. So we need to know why it isn't work. It is either because we aren't recompiling b before we compile a or because we are not seeing those changes from a.

[uesteibar]

@josevalim Good point. We do not recompile b indeed.

On this line https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/compilers/elixir.ex#L84 when compiling b, it does always compile it if needed.

On the lines you linked to it does detect B as stale local dep. However, it's not compiling a.

I'll continue this path tomorrow, thanks for the help! :)

[OvermindDL1]

Just as a side note, I test my protocol_ex library compiler with another project that loads it from a relative location on the filesystem and it always picks up the changes without issue, though it's not using structs defined from it as one difference.

[josevalim]

We were supposed to recompile b as a path dependency. That may be the root

issue.

José Valimwww.plataformatec.com.br
http://www.plataformatec.com.br/Founder and Director of R&D

[fbergero]

B is recompiled. The issue is that A does not get recompiled after B is compiled.

[uesteibar]

Exactly as @fbergero said. Sorry if my explanation wasn't clear. I'll give it a fresh look tomorrow morning.

[fbergero]

@uesteibar recompiling the whole app seems like an overkill. If this is the only solution is ok but recompiling the files that actually use B should do the trick right?

[uesteibar]

I run some tests and it seems this is only happening when a file depends in a local file only for a struct.

for:

# lib/a.ex
defmodule A do
  def test do
    IO.inspect(%B{b1: 4, b2: 2})
  end
end

# b/lib/b.ex
defmodule B do
  defstruct [:b1, :b2]
end

it does not compile a.ex when changing something on b.ex.

However, for:

# lib/a.ex
defmodule A do
  def test do
    IO.inspect(%B{b1: 4, b2: 2})
  end

  def test_fun, do: B.test()
end

# b/lib/b.ex
defmodule B do
  defstruct [:b1, :b2]

  def test, do: true
end

it works perfectly.

It's fixed by passing the result of stale_local_deps/2 as structs to update_stale_entries/5: https://github.com/uesteibar/elixir/commit/68b04fd70cd4258a035db6bb0f358983edbd4f93

[josevalim]

Great job @uesteibar! That fix is perfect. Now we just need a test, you can reuse this test. Note we already check for compile time dependencies, we need to check for struct dependencies too.

[uesteibar]

Thanks @josevalim! I was trying to figure out how to write the test for this in compile.elixir_test.exs but I couldn't manage to get the local dependency fixture to work properly. I'll go it this way instead, would you rather have an extra test or add this check to the same one?

[josevalim]

@uesteibar it is probably time for us to break this test into 3, one for runtime, another for struct and another for compile time :+1: