Elixir: Compilation of fs suddenly fails with rebar_abort

[ospaarmann]

Environment

• Elixir v 1.6.4, Erlang 20.3.4
• Operating system: Mac OS 10.13.4

Current behavior

The compilation of the deps of a project suddenly started to fail without any major code change or upgrade to Elixir or Erlang.

My deps:

  certifi 2.0.0
  connection 1.0.4
  cowboy 2.2.0
  cowlib 2.1.0
  db_connection 1.1.3
  earmark 1.2.4
  ex_doc 0.18.3
  excoveralls 0.8.1
  exjsx 4.0.0
  fs 0.9.2
  grpc 0.3.0-alpha.2
  gun 1.0.0-pre.5
  hackney 1.11.0
  idna 5.1.0
  jsx 2.8.3
  metrics 1.0.1
  mimerl 1.0.2
  mix_test_watch 0.5.0
  morphix 0.2.1
  poison 3.1.0
  poolboy 1.5.1
  protobuf 0.5.4
  ranch 1.4.0
  retry 0.8.1
  ssl_verify_fun 1.1.1
  unicode_util_compat 0.3.1
  uuid 1.1.8

Terminal output:

$: mix deps.compile
==> protobuf
Compiling 21 files (.ex)
Generated protobuf app
==> connection
Compiling 1 file (.ex)
Generated connection app
==> fs (compile)
Compiled src/fs_app.erl
Compiled src/fs.erl
Compiled src/fs_server.erl
Compiled src/fs_event_bridge.erl
Compiled src/fs_sup.erl
Compiled src/sys/fsevents.erl
Compiled src/sys/inotifywait_win32.erl
Compiled src/sys/inotifywait.erl
Compiling c_src/mac/cli.c
In file included from c_src/mac/cli.c:2:
In file included from c_src/mac/cli.h:4:
In file included from c_src/mac/common.h:4:
In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:43:
In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:72:
/usr/local/include/Block.h:16:3: error: Never include this file directly. Use <lzma.h> instead.
#       error Never include this file directly. Use <lzma.h> instead.
        ^
/usr/local/include/Block.h:93:2: error: unknown type name 'lzma_check'
        lzma_check check;
        ^
/usr/local/include/Block.h:148:2: error: unknown type name 'lzma_vli'
        lzma_vli compressed_size;
        ^
/usr/local/include/Block.h:172:2: error: unknown type name 'lzma_vli'
        lzma_vli uncompressed_size;
        ^
/usr/local/include/Block.h:200:2: error: unknown type name 'lzma_filter'
        lzma_filter *filters;
        ^
/usr/local/include/Block.h:217:20: error: use of undeclared identifier 'LZMA_CHECK_SIZE_MAX'
        uint8_t raw_check[LZMA_CHECK_SIZE_MAX];
                          ^
/usr/local/include/Block.h:231:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int3;
        ^
/usr/local/include/Block.h:232:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int4;
        ^
/usr/local/include/Block.h:233:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int5;
        ^
/usr/local/include/Block.h:234:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int6;
        ^
/usr/local/include/Block.h:235:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int7;
        ^
/usr/local/include/Block.h:236:2: error: unknown type name 'lzma_vli'
        lzma_vli reserved_int8;
        ^
/usr/local/include/Block.h:237:2: error: unknown type name 'lzma_reserved_enum'
        lzma_reserved_enum reserved_enum1;
        ^
/usr/local/include/Block.h:238:2: error: unknown type name 'lzma_reserved_enum'
        lzma_reserved_enum reserved_enum2;
        ^
/usr/local/include/Block.h:239:2: error: unknown type name 'lzma_reserved_enum'
        lzma_reserved_enum reserved_enum3;
        ^
/usr/local/include/Block.h:240:2: error: unknown type name 'lzma_reserved_enum'
        lzma_reserved_enum reserved_enum4;
        ^
/usr/local/include/Block.h:261:2: error: unknown type name 'lzma_bool'
        lzma_bool ignore_check;
        ^
/usr/local/include/Block.h:263:2: error: unknown type name 'lzma_bool'
        lzma_bool reserved_bool2;
        ^
/usr/local/include/Block.h:264:2: error: unknown type name 'lzma_bool'
        lzma_bool reserved_bool3;
        ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
ERROR: compile failed while processing /Users/olespaarmann/Sites/exdgraph/deps/fs: rebar_abort
==> ex_dgraph
** (Mix) Could not compile dependency :fs, "/Users/olespaarmann/.mix/rebar compile skip_deps=true deps_dir="/Users/olespaarmann/Sites/exdgraph/_build/dev/lib"" command failed. You can recompile this dependency with "mix deps.compile fs", update it with "mix deps.update fs" or clean it with "mix deps.clean fs"

What I tried so far:

$ rm -rf deps
$ rm -rf _build
$ mix deps.get
$ mix deps.compile

• Restarting Mac
• Starting Xcode to see if everything is working
• Upgrading homebrew
• Installing Xcode Command Line Tools because Brew said they are missing with xcode-select --install
• Everything above again

Via https://github.com/synrc/fs/issues/44:

$ locate /CoreServices.h
$ export CFLAGS="-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include"

• Make sure no global version of rebar is installed.
• Run mix local.rebar

Nothing worked so far.

Expected behavior

It should compile...

[OvermindDL1]

Where is your /usr/local/include/Block.h file coming from? That doesn't look right...

EDIT: Also, this should be in the fs repo yes?

[ospaarmann]

To be honest I have no idea. Maybe it was put there by homebrew?

[ospaarmann]

@OvermindDL1 thanks for your help so far.

I opened an issue in the fs repo: https://github.com/synrc/fs/issues/57

I also start to get an idea. It seems that migrating from one mac to another keeps the homebrew installations but creates chaos. brew doctor reveals a lot of issues. I will investigate further.

[ospaarmann]

Running brew doctor revealed: "Unbrewed header files were found in /usr/local/include."

Deleting these files did the trick for me.

[OvermindDL1]

Deleting these files did the trick for me.

Awesome! I'm quite curious what dropped those in there though... ^.^;

[ospaarmann]

This issue put me in the right direction: https://github.com/rbenv/ruby-build/issues/1169

I investigated with colleagues and we've found the solution.
First, problem - it's migration tool from old mac to new not correctly supports brew and libraries installed by brew. After migration, I've ran brew doctor and saw full log with .h files problems and broken symlink.
After that, I began resolve each problem with library and dependencies, and when I clean my brew doctor output, I can install ruby successfully.
So, maybe we could try it too