Nixpkgs: Numpy-1.17.{2-4} segfaults tests when built with openblas

Created on 15 Dec 2019  路  1Comment  路  Source: NixOS/nixpkgs

Describe the bug
I am getting a segfault in the check phase of numpy-1.17.4 with openblas, specifically when running test_ufunc.py on the following commit

{
  "url": "https://github.com/NixOS/nixpkgs.git",
  "rev": "20c33be286abd80e41537609843fc50f14a15560",
  "date": "2019-12-12T12:18:55-05:00",
  "sha256": "1j8yw337hdafisdzj1c2s847ag2gcic9633gvsicyiipxiyi4a1h",
  "fetchSubmodules": false
}
The same thing happens when building numpy-1.17.2 from my unstable nixpkgs branch. **To Reproduce** You can use the above pin (created Dec 12th, 2019) and simply drop into a nix-shell with something like
nix-shell -p '(python3.withPackages (ps: [(ps.numpy.override({ blas = openblas;}))]))'
**Expected behavior** Nix-shell compiles numpy **Results**
Logs of the error encountered
../core/tests/test_ufunc.py::TestUfunc::test_axes_argument <- ../../nix/store/4x1gk3nnwd00qlsdrbf8iqpmqc4d74yy-python3.7-numpy-1.17.4/lib/python3.7/site-packages/numpy/core/tests/test_ufunc.py PASSED [ 31%]
../core/tests/test_ufunc.py::TestUfunc::test_axis_argument <- ../../nix/store/4x1gk3nnwd00qlsdrbf8iqpmqc4d74yy-python3.7-numpy-1.17.4/lib/python3.7/site-packages/numpy/core/tests/test_ufunc.py PASSED [ 31%]
../core/tests/test_ufunc.py::TestUfunc::test_keepdims_argument <- ../../nix/store/4x1gk3nnwd00qlsdrbf8iqpmqc4d74yy-python3.7-numpy-1.17.4/lib/python3.7/site-packages/numpy/core/tests/test_ufunc.py Fatal Python error: Segmentation fault

Current thread 0x00007ffff76511c0 (most recent call first):
  File "/nix/store/4x1gk3nnwd00qlsdrbf8iqpmqc4d74yy-python3.7-numpy-1.17.4/lib/python3.7/site-packages/numpy/core/tests/test_ufunc.py", line 851 in test_keepdims_argument
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/python.py", line 170 in pytest_pyfunc_call
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 86 in 
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/python.py", line 1429 in runtest
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 125 in pytest_runtest_call
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 86 in 
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 201 in 
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 229 in from_call
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 201 in call_runtest_hook
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 176 in call_and_report
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 95 in runtestprotocol
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/runner.py", line 80 in pytest_runtest_protocol
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 86 in 
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/main.py", line 258 in pytest_runtestloop
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 86 in 
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/main.py", line 237 in _main
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/main.py", line 193 in wrap_session
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/main.py", line 230 in pytest_cmdline_main
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 86 in 
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/manager.py", line 92 in _hookexec
  File "/nix/store/zdyl9vn3v3wgcdysmbq2f89xb3274bi1-python3.7-pluggy-0.13.0/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/nix/store/7w0sngig0zynlial6l4rfxpkm3gq32x2-python3.7-pytest-5.2.4/lib/python3.7/site-packages/_pytest/config/__init__.py", line 90 in main
  File "/nix/store/4x1gk3nnwd00qlsdrbf8iqpmqc4d74yy-python3.7-numpy-1.17.4/lib/python3.7/site-packages/numpy/_pytesttester.py", line 205 in __call__
  File "", line 1 in 
/nix/store/04fcnjjk5qgrcll1zljb83c0gc6gs3in-stdenv-linux/setup: line 1301:  3025 Segmentation fault      (core dumped) /nix/store/c2n0xp0j3nacr1l17lmrdzwp16ljsvll-python3-3.7.5/bin/python3.7 -c 'import numpy; numpy.test("fast", verbose=10)'
builder for '/nix/store/x4s200kb31n9dp6i00y8216nsw4jk22h-python3.7-numpy-1.17.4.drv' failed with exit code 139
cannot build derivation '/nix/store/4rqdnziyskkszsbzkp80p9xs334mjamv-python3-3.7.5-env.drv': 1 dependencies couldn't be built
error: build of '/nix/store/4rqdnziyskkszsbzkp80p9xs334mjamv-python3-3.7.5-env.drv' failed

Metadata

  • system: "x86_64-linux"
  • host os: Linux 4.19.84, NixOS, 19.09.1300.08e503bac6d (Loris)
  • multi-user?: yes
  • sandbox: yes
  • version: nix-env (Nix) 2.3
  • channels(stites): "home-manager"
  • channels(root): "nixos-19.09.1300.08e503bac6d, unstable-20.03pre202088.e89b21504f3"
  • nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Maintainer information:

attribute:
  - python3Packages.numpy
  - openblas

@ttuegel @fridh

bug

Most helpful comment

The standard numpy in nixpkgs is already build against openblas. However, it is built against openblasCompat, which uses the 32 bit integer interface. openblas uses the 64 bit integer interface. This is common problem with Fortran libraries. Both, the program linking the library and the library itself, must be built with the same convention (see ILP64 vs LP64)

>All comments

The standard numpy in nixpkgs is already build against openblas. However, it is built against openblasCompat, which uses the 32 bit integer interface. openblas uses the 64 bit integer interface. This is common problem with Fortran libraries. Both, the program linking the library and the library itself, must be built with the same convention (see ILP64 vs LP64)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

edolstra picture edolstra  路  3Comments

lverns picture lverns  路  3Comments

chris-martin picture chris-martin  路  3Comments

matthiasbeyer picture matthiasbeyer  路  3Comments

ayyess picture ayyess  路  3Comments