Linuxbrew-core: numpy failed to build on Red Hat Enterprise Linux Server release 7.3 (Maipo)

[danfulop]

Please fill out the issue checklist below, and provide all of the requested information.

Please always follow these steps:

• [x] Confirmed this is a problem with brew installing one, specific formula and not every time you run brew? If it's a general brew problem please file this issue at https://github.com/Linuxbrew/brew/issues/new.
• [ ] If you have a macOS system and can replicate the problem on macOS with Homebrew, please report the issue to Homebrew/core and follow their issue template from your macOS system. If you are unsure, please report the issue to Linuxbrew.
• [x] Ran brew update and retried your prior step?
• [x] Ran brew doctor, fixed all issues and retried your prior step?
• [x] Ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
• [ ] If brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

To help us debug your issue please explain:

• What you were trying to do (and why)

I was trying to reinstall Numpy because of an error I got running multiQC (http://multiqc.info/).

• What happened (include command output)

I couldn't replicate the original error when running multiQC. I subsequently got a different error relating to the JinJa2 minimum required version and fixed it separately.

This is the command run and the output:
command: brew install numpy
output:

==> Using Homebrew-provided fortran compiler.
This may be changed by setting the FC environment variable.
==> Downloading https://files.pythonhosted.org/packages/bf/2d/005e45738ab07a26e621c9c12dc97381f372e06678adf7dc3356a69b5960/numpy-1.13.3.zip
Already downloaded: /data/home/dfulop/.cache/Homebrew/numpy-1.13.3.zip
==> Downloading https://files.pythonhosted.org/packages/58/a5/0dc93c3ec33f4e281849523a5a913fa1eea9a3068acfa754d44d88107a44/nose-1.3.7.tar.gz
Already downloaded: /data/home/dfulop/.cache/Homebrew/numpy--nose-1.3.7.tar.gz
==> python -c import setuptools... --no-user-cfg install --prefix=/data/home/dfulop/.linuxbrew/Cellar/numpy/1.13.3/libexec/nose --single-version-externally-
==> python setup.py build --fcompiler=gnu95 --parallel=8 install --prefix=/data/home/dfulop/.linuxbrew/Cellar/numpy/1.13.3 --single-version-externally-manag
Last 15 lines from /data/home/dfulop/.cache/Homebrew/Logs/numpy/02.python:
gcc-5: numpy/core/src/multiarray/iterators.c
gcc-5: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/scalartypes.c
gcc-5: build/src.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_templ.c
gcc-5: numpy/core/src/multiarray/nditer_constr.c
gcc-5: numpy/core/src/multiarray/alloc.c
gcc-5: numpy/core/src/private/mem_overlap.c
gcc-5: numpy/core/src/multiarray/array_assign.c
gcc-5 -pthread -shared -L/data/home/dfulop/.linuxbrew/lib build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/alloc.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/arrayobject.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/arraytypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_scalar.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_array.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/buffer.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/calculation.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/compiled_base.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/common.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert_datatype.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/conversion_utils.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ctors.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_strings.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busday.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busdaycal.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/descriptor.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/dtype_transfer.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/einsum.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/flagsobject.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/getset.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/hashdescr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/item_selection.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/iterators.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/lowlevel_strided_loops.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/mapping.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_templ.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_api.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_constr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_pywrap.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/number.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/numpyos.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/refcount.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/sequence.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/shape.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/scalarapi.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/scalartypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/temp_elide.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/usertypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ucsnarrow.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/vdot.o build/temp.linux-x86_64-2.7/numpy/core/src/private/mem_overlap.o build/temp.linux-x86_64-2.7/numpy/core/src/private/ufunc_override.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/cblasfuncs.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/python_xerbla.o -L/usr/lib64/atlas -L/data/home/dfulop/.linuxbrew/lib -L/data/home/dfulop/.linuxbrew/opt/openssl/lib -L/data/home/dfulop/.linuxbrew/opt/sqlite/lib -L/data/home/dfulop/.linuxbrew/Cellar/python/2.7.14/lib -Lbuild/temp.linux-x86_64-2.7 -lnpymath -lnpysort -ltatlas -ltatlas -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/numpy/core/multiarray.so
/data/home/dfulop/.linuxbrew/bin/ld: cannot find -ltatlas
/data/home/dfulop/.linuxbrew/bin/ld: cannot find -ltatlas
collect2: error: ld returned 1 exit status
/data/home/dfulop/.linuxbrew/bin/ld: cannot find -ltatlas
/data/home/dfulop/.linuxbrew/bin/ld: cannot find -ltatlas
collect2: error: ld returned 1 exit status
error: Command "gcc-5 -pthread -shared -L/data/home/dfulop/.linuxbrew/lib build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/alloc.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/arrayobject.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/arraytypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_scalar.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/array_assign_array.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/buffer.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/calculation.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/compiled_base.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/common.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/convert_datatype.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/conversion_utils.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ctors.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_strings.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busday.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/datetime_busdaycal.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/descriptor.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/dtype_transfer.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/einsum.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/flagsobject.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/getset.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/hashdescr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/item_selection.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/iterators.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/lowlevel_strided_loops.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/mapping.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/methods.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/multiarraymodule.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_templ.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_api.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_constr.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/nditer_pywrap.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/number.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/numpyos.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/refcount.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/sequence.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/shape.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/scalarapi.o build/temp.linux-x86_64-2.7/build/src.linux-x86_64-2.7/numpy/core/src/multiarray/scalartypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/temp_elide.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/usertypes.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/ucsnarrow.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/vdot.o build/temp.linux-x86_64-2.7/numpy/core/src/private/mem_overlap.o build/temp.linux-x86_64-2.7/numpy/core/src/private/ufunc_override.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/cblasfuncs.o build/temp.linux-x86_64-2.7/numpy/core/src/multiarray/python_xerbla.o -L/usr/lib64/atlas -L/data/home/dfulop/.linuxbrew/lib -L/data/home/dfulop/.linuxbrew/opt/openssl/lib -L/data/home/dfulop/.linuxbrew/opt/sqlite/lib -L/data/home/dfulop/.linuxbrew/Cellar/python/2.7.14/lib -Lbuild/temp.linux-x86_64-2.7 -lnpymath -lnpysort -ltatlas -ltatlas -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/numpy/core/multiarray.so" failed with exit status 1

gist-logs: https://gist.github.com/anonymous/a6a68a8b210f827f272e00bac25de9f4

• What you expected to happen

I expected Numpy to install without any issue.

Just in case, here's the output of brew config:

HOMEBREW_VERSION: 1.3.8-2-g3aecdc0
ORIGIN: https://github.com/Linuxbrew/brew
HEAD: 3aecdc02948da03bb9feab52643eecfc02d903d5
Last commit: 24 hours ago
Core tap ORIGIN: https://github.com/Linuxbrew/homebrew-core
Core tap HEAD: 3113f3d6626f04182df0dbeb53645bd6774aa0ce
Core tap last commit: 15 hours ago
HOMEBREW_PREFIX: /data/home/dfulop/.linuxbrew
HOMEBREW_REPOSITORY: /data/home/dfulop/.linuxbrew/Homebrew
HOMEBREW_CELLAR: /data/home/dfulop/.linuxbrew/Cellar
HOMEBREW_CACHE: /data/home/dfulop/.cache/Homebrew
HOMEBREW_CELLAR: /data/home/dfulop/.linuxbrew/Cellar
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_GIT_CONFIG_FILE: /data/home/dfulop/.linuxbrew/Homebrew/.git/config
HOMEBREW_PREFIX: /data/home/dfulop/.linuxbrew
HOMEBREW_REPOSITORY: /data/home/dfulop/.linuxbrew/Homebrew
CPU: octa-core 64-bit nehalem
Homebrew Ruby: 2.3.3 => /data/home/dfulop/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3/bin/ruby
Clang: N/A
Git: 2.15.0 => /data/home/dfulop/.linuxbrew/bin/git
Curl: 7.56.1 => /data/home/dfulop/.linuxbrew/opt/curl/bin/curl
Perl: /usr/bin/perl
Python: /data/home/dfulop/.linuxbrew/opt/python/libexec/bin/python => /data/home/dfulop/.linuxbrew/Cellar/python/2.7.14/bin/python2.7
Ruby: /usr/bin/ruby
Java: openjdk version "1.8.0_111"
Kernel: Linux 3.10.0-514.el7.x86_64 x86_64 GNU/Linux
OS: Red Hat Enterprise Linux Server release 7.3 (Maipo)
Host glibc: 2.17
/usr/bin/gcc: 4.8.5
glibc: 2.23
gcc: 5.4.0
xorg: 20170115

• Step-by-step reproduction instructions (by running brew install commands)

Formula additions or changes

To get formulae added or changed in Linuxbrew please file a Pull Request.
To get formulae added or changed that is a dependency of a formula in Linuxbrew/homebrew-core submit a Pull Request to https://github.com/Linuxbrew/homebrew-core/compare
To get formulae added or changed that is not a dependency of a formula in Linuxbrew/homebrew-core submit a Pull Request to https://github.com/Linuxbrew/homebrew-extra/compare
We will close issues requesting formulae changes.

[iMichka]

The error is:

/data/home/dfulop/.linuxbrew/bin/ld: cannot find -ltatlas

I guess that this is not installed on your computer. On mac, I thing accelerate is installed by default, which is the mac implementation, which numpy will pick up.

I think we can try with openblas, which is equivalent (of course, performance may be different, but for most use cases it is fine).

Could you try to add this to the formula:
depends_on "onpenblas" unless OS.mac?

[danfulop]

Hi @iMichka. Yes, thanks for extracting the actual error for clarity. I had noticed that that is the error. Sorry for not making it more clear.

This is odd because I thought I had brew installed Numpy on this cluster without a problem about 1 month ago. I guess the gist-log would show if that was the case or not...

I have 2 questions:

1. Can I install the ltatlas dependency via brew or another way. E.g., perhaps by brew re-installing ld?
2. I'm not sure how to add depends_on "onpenblas" unless OS.mac to the numpy brew formula.

[iMichka]

It is not really about ld; it is just ld looking for atlas, but not finding it. There were updates to brew meanwhile and some dependencies may now be filtered out properly, as we can not rely on people having everything installed. So stuff that worked before may be broken now (but it is quite rare).

To add this line, just run brew edit numpy. This will open the formula in vim and you can add that line below the other depends_on calls. Then try to install again.

Once you are done you can use brew update-reset to cleanup after your change.

[danfulop]

Awesome! Will try and report back.

[danfulop]

@iMichka Nope, openblas didn't work and it's still complaining about not finding ltatlas. Here's the gist for the output using openblas: https://gist.github.com/danfulop/15c6091c4550b7fe5175ff007db6c8e2

[danfulop]

Thinking that one solution would be to install ATLAS and searching for whether it has a formula, I bumped into this issue thread that I'm now reading: https://github.com/Linuxbrew/legacy-linuxbrew/issues/1075

[danfulop]

There apparently is no formula for ATLAS, as brew install atlas didn't work: Error: No available formula with the name "atlas"

Another potential solution could be to install the SciPy suite (including Numpy) using Conda, but:

1. I don't know if it will include ATLAS
2. I haven't figured out a way to make Conda and Linuxbrew play nice

[sjackman]

Check if numpy has an option to disable ATLAS (perhaps --without-atlas) and instead use OpenBLAS (perhaps --with-openblas).

[Char-Aznable]

I had the same issue before. In my case, I had to customize a 'site.cfg' file to tell the numpy installer where to find the BLAS/ATLAS/OpenBLAS library and then it worked:

diff --git a/Formula/numpy.rb b/Formula/numpy.rb
index bd00667135..1aab2d245a 100644
--- a/Formula/numpy.rb
+++ b/Formula/numpy.rb
@@ -32,6 +32,13 @@ class Numpy < Formula
   end

   def install
+    config = <<~EOS
+      [DEFAULT]
+      library_dirs = #{HOMEBREW_PREFIX}/lib
+      include_dirs = #{HOMEBREW_PREFIX}/include
+    EOS
+    Pathname("site.cfg").write config
+
     Language::Python.each_python(build) do |python, version|
       dest_path = lib/"python#{version}/site-packages"
       dest_path.mkpath

Actually, these lines are already present in the scipy formula 'scipy.rb'

[iMichka]

@Char-Aznable Would you be willing to submit a pull request for this? Seems to be the right solution, so we can just use openblas to start with :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[iMichka]

Fixed by #5326