Elixir: Elixir 1.5.0 Causing Invalid Macro will never be called in Dialyzer

[jdemaris]

Environment

Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.5.0

Mac OSX 10.12.5 (Sierra)

Current behavior

Created a Mix project with a single module: test_macro.ex and added dialyxir 0.5 to the mix.exs

defmodule TestMacro do
  defmacrop good_macro(id) do
    quote do
      {:good, {:good_macro, unquote(id)}}
    end
  end

  def run() do
    good_macro("Not So Bad")
  end
end

When I run it, it works:

➜  test_macro iex -S mix
Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.5.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> TestMacro.run()
{:good, {:good_macro, "Not So Bad"}}
iex(2)>

When I run dialyzer, it fails:

➜  test_macro mix dialyzer
Checking PLT...
[:compiler, :elixir, :kernel, :logger, :stdlib]
PLT is up to date!
Starting Dialyzer
dialyzer args: [check_plt: false,
 init_plt: '/Users/jdemaris/Projects/playground/test_macro/_build/dev/dialyxir_erlang-20.0_elixir-1.5.0_deps-dev.plt',
 files_rec: ['/Users/jdemaris/Projects/playground/test_macro/_build/dev/lib/test_macro/ebin'],
 warnings: [:unknown]]
done in 0m1.08s
lib/test_macro.ex:2: Function 'MACRO-good_macro'/2 will never be called
done (warnings were emitted)

Expected behavior

Dialyzer should pass, since it obviously does call the macro.

Notes

I opened this with dialyxir as well since I am not sure which side the issue might be on, but since this was working on OTP20 with Elixir 1.4.5, it seems like an unexpected change in Elixir, not an issue specifically with dialyxir or dialyzer.

[muhifauzan]

It might be related. I'm having a problem when I try to build PLT from source using this command on lib/:

dialyzer --build_plt --apps elixir --output_plt .dialyzer.plt

This is not the case when building PLT on v1.4.5. This is the erl_crash.dump

[josevalim]

@muhifauzan you need to have Elixir in your path when building the plt. Does dialyzer support the -pa and -pz option? If so, pass them pointing to Elixir's ebin directory.

[josevalim]

Thanks for the report @jdemaris ! This one was quite a challenge. :)

[josevalim]

Fixed in 3a1611f54e1a0db9c2888d4949dcedb6c84544c5.

[muhifauzan]

@josevalim Elixir is already on the PATH. It's under /usr/local/bin/

$ which elixir
/usr/local/bin/elixir

I already pass the Elixir's ebin directory to to dialyzer using -pa. Same result. I tried to build it using Elixir's ebin from the source, same result. Building PLT in another folder doesn't change the outcome. This is not the case when building PLT from Elixir v1.4.5. I do the build on different isolated environment using the same setup. I will try to open a new issue if that's necessary.

[josevalim]

Can you please post the full command that you are running from the Elixir directory? It is not about Elixir being in the OS PATH but about its beams being available in Erlang code path.

[jdemaris]

Thank you for the fix @josevalim!

[muhifauzan]

This is the code that I run on the setup process. I removed the unnecessary code:

ERLANG_VERSION=20.0.1
ELIXIR_VERSION=1.5.0
DIALYXIR_PLT_CORE_PATH=${HOME}/.dialyxir_plt_core
DIALYZER_PLT=${DIALYXIR_PLT_CORE_PATH}/dialyxir_erlang-${ERLANG_VERSION}.plt
ELIXIR_PLT=${DIALYXIR_PLT_CORE_PATH}/dialyxir_erlang-${ERLANG_VERSION}_elixir-${ELIXIR_VERSION}.plt

mkdir /tmp/erlang-build
cd /tmp/erlang-build

git clone -b OTP-${ERLANG_VERSION} --single-branch --depth 1 https://github.com/erlang/otp.git .
export ERL_TOP=/tmp/erlang-build
export PATH=${ERL_TOP}/bin:${PATH}
export CPPFlAGS="-D_BSD_SOURCE ${CPPFLAGS}"
./otp_build autoconf
./configure --prefix=/usr \
  --sysconfdir=/etc \
  --mandir=/usr/share/man \
  --infodir=/usr/share/info \
  --without-javac \
  --without-wx \
  --without-debugger \
  --without-observer \
  --without-jinterface \
  --without-cosEvent\
  --without-cosEventDomain \
  --without-cosFileTransfer \
  --without-cosNotification \
  --without-cosProperty \
  --without-cosTime \
  --without-cosTransactions \
  --without-et \
  --without-gs \
  --without-ic \
  --without-megaco \
  --without-orber \
  --without-percept \
  --without-typer \
  --enable-threads \
  --enable-shared-zlib \
  --enable-ssl=dynamic-ssl-lib \
  --enable-hipe
make -j4 && make install
cd $HOME
rm -rf /tmp/erlang-build
update-ca-certificates --fresh

mkdir /tmp/elixir-build
cd /tmp/elixir-build

git clone -b v${ELIXIR_VERSION} --single-branch --depth 1 https://github.com/elixir-lang/elixir.git .
make && make install
mix local.hex --force
mix local.rebar --force
mkdir -p $DIALYXIR_PLT_CORE_PATH
dialyzer --build_plt --apps erts kernel stdlib crypto mnesia
dialyzer --build_plt --apps lib/*/ebin --output_plt $ELIXIR_PLT
cd $HOME
rm -rf /tmp/elixir-build

cd $HOME

The code above works fine when using ERLANG_VERSION=20.0.1 and ELIXIR_VERSION=1.4.5.

Notes

This code runs on Alpine Linux v3.6 using 2 core processor and 2GB memory. I start the environment clean for each Elixir version

[josevalim]

@muhifauzan you are not passing the -pa and -pz options I have mentioned above. Try this:

dialyzer --build_plt -pa lib/elixir/ebin --apps lib/*/ebin --output_plt $ELIXIR_PLT

[muhifauzan]

@josevalim I'm sorry. I forgot to mention. I do exactly as you suggest. That code above is the original automation script with unnecessary code removed. I do a couple of test. The first one is using the code as you just suggest. The second one, I pass the Elixir's ebin from /usr/lib/elixir/lib/elixir/ebin to -pa. The third one, I execute the code you suggest from the home directory passing Elixir source's ebin to -pa and other options aligned. But all of these scripts fail when building the Elixir's PLT. I don't really understand the error message it produce. Searching on the web doesn't provide useful result. If you need the error message, I will provide that later. I'm not near the computer at the moment.

And again. Doing all of these tests using Elixir 1.4.5 with Erlang 20.0.1 works fine.

[josevalim]

I was able to reproduce it. Once you pass -pa, you get a different error message which reveals the real bug. Pull request #6401 should fix it. I also have changed the CI script to build the plt at least for Elixir, which will protect us from such bugs in the future.

[josevalim]

@muhifauzan merged, thanks for the report!
:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:

[muhifauzan]

@josevalim thank you for the fix!