Elixir: PLT generation takes 40+ mins on Elixir 1.7

Created on 26 Jul 2018 · 14Comments · Source: elixir-lang/elixir

Environment

Elixir & Erlang/OTP versions (elixir --version):
- Erlang 20.3.8.2 + Elixir 1.7.0
- Erlang 21.0.3 + Elixir 1.7.0
Operating system:
- OS X 10.13.6
- Ubuntu 14.04

Current behavior

I'm upgrading a sample phoenix app to Elixir 1.7.0 and finding the PLT generation takes much longer than it does on 1.6.6. I still have yet to actually get a PLT build to complete (either locally or on Travis) and I've increased my TravisCI timeout to 40 minutes. Here is the log output from mix dialyzer --plt

Checking PLT...
[:asn1, :compiler, :connection, :cowboy, :cowlib, :crowdfundr, :crypto,
 :db_connection, :decimal, :ecto, :eex, :elixir, :ex_unit, :gettext, :kernel,
 :logger, :mime, :phoenix, :phoenix_ecto, :phoenix_html, :phoenix_pubsub, :plug,
 :poison, :poolboy, :postgrex, :public_key, :ranch, :runtime_tools, :ssl,
 :stdlib]
Finding suitable PLTs
Looking up modules in dialyxir_erlang-20.3.8.2_elixir-1.7.0_deps-test.plt
Looking up modules in dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Looking up modules in dialyxir_erlang-20.3.8.2.plt
Finding applications for dialyxir_erlang-20.3.8.2.plt
Finding modules for dialyxir_erlang-20.3.8.2.plt
Creating dialyxir_erlang-20.3.8.2.plt
Looking up modules in dialyxir_erlang-20.3.8.2.plt
Removing 3 modules from dialyxir_erlang-20.3.8.2.plt
Checking 11 modules in dialyxir_erlang-20.3.8.2.plt
Adding 152 modules to dialyxir_erlang-20.3.8.2.plt
Finding applications for dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Finding modules for dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Copying dialyxir_erlang-20.3.8.2.plt to dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Looking up modules in dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Checking 163 modules in dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt
Adding 230 modules to dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt

It appears to get bogged down both locally and on Travis on Adding 230 modules to dialyxir_erlang-20.3.8.2_elixir-1.7.0.plt. On this step, beam.smp process uses 100% of one of my cpu cores, but doesn't use any other cores.

Here's my travis build: https://travis-ci.org/aaronrenner/crowdfundr/builds/408340349
And here's the branch with the Elixir 1.7.0 upgrade: https://github.com/aaronrenner/crowdfundr/tree/ar-upgrade-to-elixir-1.7

I've also tried this locally with OTP 21.0.3 and I'm still seeing the same issue. I'm super excited to try Elixir 1.7.0, so any help in figuring this out would be appreciated!

Expected behavior

I know it takes a while to generate PLTs, but I'm thinking it should be 10 -15 mins max.

Source

aaronrenner

👍1

Most helpful comment

v1.7.1 is out, enjoy your dialyzer warnings. :)

josevalim on 26 Jul 2018

❤13 😄2

All 14 comments

We didn't change anything on this area, so my guess is rather that dialyzer is getting stuck somewhere? As a starting point, I would suggest adding each application individually and find each one is responsible for a big chunk of the slow down.

Can you reproduce it locally?

josevalim on 26 Jul 2018

I can confirm that I have the same problem.
Additional information: the problem did not occur for me in 1.7.0-rc.1

amatalai on 26 Jul 2018

Interesting. The changes between versions were very minor. Here is the log:

https://github.com/elixir-lang/elixir/compare/v1.7.0-rc.1...v1.7.0

I would try reverting bfdaf3f8d1b4a4d74583dec15f64130d4c9db276, b6d3f29b69747e863f3ce2515edb7d3d07d9bb7a and 04c45b031a8e766969b865015b7ba5c1b4ba04d7 one by one and see if one of them fixes the issue.

josevalim on 26 Jul 2018

Additionally, I can confirm that the same is happening for the newly generated apps with dialyxir as only dependency. Minimum example: https://github.com/amatalai/dialyzer_issue

I can check these commits or do git bisect, but unfortunately I do not have time for it right now, maybe later today.

amatalai on 26 Jul 2018

👍2

I am able to generate PLTs with d410466
But I have been stuck for around 10 min now using 04c45b0

Elixir 1.7.0-rc.1 (04c45b0) (compiled with Erlang/OTP 21)
hans@hans-XPS-15:~/dialyzer_issue$ mix dialyzer
==> dialyxir
Compiling 2 files (.erl)
Compiling 55 files (.ex)
Generated dialyxir app
==> dialyzer_issue
Compiling 2 files (.ex)
Generated dialyzer_issue app
Finding suitable PLTs
Checking PLT...
[:compiler, :elixir, :kernel, :logger, :stdlib]
Looking up modules in dialyxir_erlang-21.0_elixir-1.7.0-rc.1_deps-dev.plt
Looking up modules in dialyxir_erlang-21.0_elixir-1.7.0-rc.1.plt
Finding applications for dialyxir_erlang-21.0_elixir-1.7.0-rc.1.plt
Finding modules for dialyxir_erlang-21.0_elixir-1.7.0-rc.1.plt
Checking 414 modules in dialyxir_erlang-21.0_elixir-1.7.0-rc.1.plt