Elixir: Mix hangs when compiling code inside a Docker container

[jakubpawlowicz]

I've stumbled upon this issue when building a Docker container containing Erlang 17 + Elixir 1.0.4 + Phoenix 0.13.1. If you check out https://github.com/jakubpawlowicz/elixir-docker-issue and run docker-compose build then it hangs at step 15 - RUN mix compile:

Step 15 : RUN mix compile
 ---> Running in 6e4208f2275c
==> fs (compile)
Compiled src/fs_app.erl
Compiled src/fs.erl
Compiled src/fs_sup.erl
Compiled src/sys/inotifywait_win32.erl
Compiled src/sys/inotifywait.erl
Compiled src/sys/fsevents.erl
Compiled src/fs_event_bridge.erl
Compiled src/fs_server.erl

However it works fine when I attach to the container and run mix compile from CLI.

One workaround I found is to run mix compile with SMP turned off, see https://github.com/jakubpawlowicz/elixir-docker-issue/blob/master/Dockerfile#L29:

elixir --erl "-smp disable" /usr/local/elixir/bin/mix compile

My setup:

• OS X 10.10
• VirtualBox 4.3.26
• Boot2Docker 1.6
• docker-compose 1.2

Couldn't verify using Elixir master as I haven't figure out how to compile it inside Docker image yet.

I wonder if it's because mix compile uses Kernel.ParallelCompiler but then it does not explain why it works fine from CLI. Could be an issue with Docker not playing well with parallel commands?
Any further ideas?

[josevalim]

Maybe you can elixir -e "IO.inspect :erlang.system_info :schedulers_online" on both cases and print the result. There is nothing special about the parallel compiler, so i am really suspicious about what could be happening here.

[josevalim]

Also, I have heard reports of people running Elixir with Docker without any issues.

[jakubpawlowicz]

Thanks @josevalim. It gives 4 (in CLI and Dockerfile) and 1 when I disable SMP.

I kinda wondered why no one raised it before, but though it would be helpful to someone.
I'll try other combinations to rule out some options.

[josevalim]

@jakubpawlowicz any news here? All signs point this is not a bug in Elixir per se. Just in the mailing list someone posted a working docker setup with a phoenix app.

[jakubpawlowicz]

@josevalim I've retested it with centos:6 - no difference, but the setup posted by Saša Jurić at https://groups.google.com/forum/#!searchin/elixir-lang-talk/docker/elixir-lang-talk/IEcAtb9XHlU/mHx9A0Xx1_gJ works fine however it's based on Ubuntu not CentOS.

I think I'll close it for now, maybe someone would be able to add their 0.02$, otherwise we always have a working workaround.

[stevegraham]

+1 mix hangs for me too inside a docker container when started with docker run. I can docker exec to a running container no problem.

[sporto]

I'm having the same issue with docker compose, Docker just hangs on RUN mix do compile

Boot2Docker-cli version: v1.7.1
Docker version 1.7.1
docker-compose version: 1.3.1

I'm using this base image https://registry.hub.docker.com/u/marcelocg/phoenix/dockerfile/

[jakubpawlowicz]

I still stick to the workaround above.

[michaldarda]

I can confirm that this issue happens when using CentOS7, everything works on ubuntu

[stevegraham]

Great workaround @jakubpawlowicz. Works like a charm.

[simonewebdesign]

We had a similar issue, and the solution was to upgrade Elixir to the latest stable version (v1.1.1).

[eidge]

Having the same hanging issue with compile on elixir v 1.1.1 with ubuntu 14.04 both the image and the host os.

[josevalim]

Can you run elixir -e "IO.inspect :erlang.system_info :schedulers_online" and let us know what it returns? Can you reproduce it consistently in every repo or does it happen only in some particular applications?

[eidge]

It's my first application, haven't tried any other. Out of phoenix.new I get the compiler to hang.

> elixir -e "IO.inspect :erlang.system_info :schedulers_online"
** (UndefinedFunctionError) undefined function: IO.inspect/0
    (elixir) IO.inspect()
    (stdlib) erl_eval.erl:669: :erl_eval.do_apply/6
    (elixir) lib/code.ex:168: Code.eval_string/3

[josevalim]

The command did not work properly because you had a new line where you
should not. Please run everything in a single line!

On Sunday, November 22, 2015, Hugo Ribeira [email protected] wrote:

It's my first application, haven't tried any other. Out of phoenix.new I
get the compiler to hang.

elixir -e "IO.inspect :erlang.system_info :schedulers_online"
** (UndefinedFunctionError) undefined function: IO.inspect/0
(elixir) IO.inspect()
(stdlib) erl_eval.erl:669: :erl_eval.do_apply/6
(elixir) lib/code.ex:168: Code.eval_string/3

—
Reply to this email directly or view it on GitHub
https://github.com/elixir-lang/elixir/issues/3342#issuecomment-158775393
.

_José Valimwww.plataformatec.com.br
http://www.plataformatec.com.br/Founder and Director of R&D_

[eidge]

The output is 4. Is there a way to set the maximum number of workers to 1 like with

elixir --erl "-smp disable" /usr/local/elixir/bin/mix compile but globally?

That way I can set it in the container as a workaround.

[josevalim]

Do not set it globally, it means your app would leverage only one core in production. It is better to pass it explicitly, and only during compile.

[edevil]

I too have this problem. Disabling SMP made it work as well but it is strange.

[edevil]

If you use as command bash -c "mix compile 1>&1" it also works for some reason. There is something funky about stdout.

[ivoanjo]

The 1>&1 solution in the above comment worked for me. Thanks @edevil !

[edevil]

If you guys are curious, the problem is described in https://github.com/edevil/docker-erlang-bug#explanation and it is not present in the upcoming Erlang R19.

[jakubpawlowicz]

Good news, thanks @edevil!

[ivoanjo]

Indeed, awesome work! :+1:

[abhurtun]

You can also use tini (https://github.com/krallin/tini) or dumb-init (https://github.com/Yelp/dumb-init) to manage PIDs inside containers when using Erlang this would resolve the issue

[scouten]

FWIW I do not believe this issue has been fixed in R19. I've just encountered the same problem using Elixir 1.3.2 (which uses Erlang R19.0.5, I think). The 1>&1 trick "fixed" the problem.

¡Muchas gracias, @edevil!

[josevalim]

@scouten Elixir does not use any particular Erlang version, it uses the one in your system. So you can run elixir -v and let us know the version reported to be double sure you are on Erlang 19.

[scouten]

@josevalim: Sorry to have let this linger. I just checked and it looks like we are on Erlang 19. I see this when I run elixir --version inside the Docker image:

Erlang/OTP 19 [erts-8.0] [source-6dc93c1] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.3.2

[letops]

So... I have been trying the solutions everyone wrote in here, but I still have this issue. I am trying to do a wercker build, but I cannot test this by doing a local build (it freezes my laptop).

I created a docker image and published it to dockerhub under the name of grvtylabs/phoenix:1.2.1 using this Dockerfile. I also created the wercker configuration and published into this gist

The problem... My wercker pipelines are freezing or they just fail. I have been trying to find the erl_crash.dump, but even if I change the environment variable that sets the location, it is not being written.
I tried the solution to just redirect the stderr and stdout to a dump file (it actually helped to make things faster but it was still freezing or failing). I also tried to tell mix to do everything in a single thread (after this all my builds seem to freeze for 1 hour or more so I have to manually abort them) using elixir --erl "-smp disable" /usr/local/elixir/bin/mix compile

Does anyone knows something that might help me? (I will continue testing and try to get the erl_crash and the stdout)

[stevegraham]

This is fixed for me using Erlang 19 built with asdf.

[letops]

I fixed it, apparently it was a wercker problem when using yes | before executing mix. I added this following the instructions detailed in this tutorial. According to that post, mix gets a "permission-happy", but I wasn't able to find any documentation on the yes | command.

Once I removed it, everything was working as a clock.

If you happen to have problems with wercker and docker, I'll leave my files right here:
Dockerfile
Wercker.yml

Thanks for everything guys!!

[michalmuskala]

Using yes | with Erlang is generally a bad idea. Erlang buffers STDIN - this means that as long as there is input, Erlang will read it an save for further processing. But yes | provides us with an infinite input on STDIN (!) - Erlang will still happily start buffering it. If you're lucky and your command finishes before you run out of memory all is fine. But for commands that take a bit longer it's easy to run out of memory and crash.

[letops]

@michalmuskala that makes so much sense!! And it actually explains all the problems I had when trying to run elixir in wercker. Thank you so much!! (I was about to do some testing using yes | and I bet it would become a disaster haha)