Nixpkgs: pymc3 does not run under nixos

[idontgetoutmuch]

Describe the bug
import pymc3 as pm fails
To Reproduce

nix-shell --pure -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/19.09.tar.gz -p 'python37.withPackages(ps: with ps; [ numpy pymc3 ])'

[nix-shell:~]$ python
Python 3.7.4 (default, Jul  8 2019, 18:31:06) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymc3 as pm

You can find the C code in this temporary file: /run/user/1000/theano_compilation_error_8381jg5y
Traceback (most recent call last):
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 81, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 105, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/pymc3/__init__.py", line 5, in <module>
    from .distributions import *
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/pymc3/distributions/__init__.py", line 1, in <module>
    from . import timeseries
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/pymc3/distributions/timeseries.py", line 1, in <module>
    import theano.tensor as tt
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/__init__.py", line 110, in <module>
    from theano.compile import (
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/compile/__init__.py", line 12, in <module>
    from theano.compile.mode import *
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/compile/mode.py", line 11, in <module>
    import theano.gof.vm
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/gof/vm.py", line 674, in <module>
    from . import lazylinker_c
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 140, in <module>
    preargs=args)
  File "/nix/store/d7wqg2xi972hahp9g7fwk7njsvv58pwi-python3-3.7.4-env/lib/python3.7/site-packages/theano/gof/cmodule.py", line 2396, in compile_str
    (status, compile_stderr.replace('\n', '. ')))
Exception: Compilation failed (return status=1): impure path `/run/user/1000/ccJEQmOq.o' used in link. collect2: error: ld returned 1 exit status. 

Expected behavior
The package should be imported and be available to use.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Metadata
Please run nix run nixpkgs.nix-info -c nix-info -m and paste the result.

[sundials@CUDA:~]$ nix run nixpkgs.nix-info -c nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.69, NixOS, 19.09.git.latest0 (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.2.2`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[veprbl]

Apparently the wrapper is causing problems here as well. (#73034)
cc @twhitehead

[twhitehead]

Responded in #73034. Sorry for the delay. :smiley:

[OmnipotentEntity]

This is failing because of a problem with the Theano package. I'm looking into a solution.

Essentially, on first import, attempts to compile some code and throw it into your home directory under ~/.theano

Nix is catching this because the temp directory it is using is outside of the pure area, /run/user/1000 in this case. You can work around it by setting the $TMPDIR variable to the current directory, and importing once, it will then put the compiled files into your home directory, and it will work, even across uninstalling and reinstalling the program.

But of course, this isn't a very good solution and certainly isn't the NixOS Way(TM). It would be better if it built these files on install and kept them in the store. I'm looking into doing just that.

[OmnipotentEntity]

cc: @bcdarwin

I'm having some trouble with forcing the compilation to occur during the patchPhase. Here's what I have:

  postPatch = ''
    substituteInPlace theano/configdefaults.py \
      --replace 'StrParam(param, is_valid=warn_cxx)' 'StrParam('\'''${cxx_compiler}'\''', is_valid=warn_cxx)' \
      --replace 'rc == 0 and config.cxx != ""' 'config.cxx != ""' \
      --replace "home = os.getenv('HOME')" "home = os.getenv('NIX_THEANO_HOME')"
  '' + stdenv.lib.optionalString cudaSupport ''
    substituteInPlace theano/configdefaults.py \
      --replace 'StrParam(get_cuda_root)' 'StrParam('\'''${cudatoolkit}'\''')'
  '' + stdenv.lib.optionalString cudnnSupport ''
    substituteInPlace theano/configdefaults.py \
      --replace 'StrParam(default_dnn_base_path)' 'StrParam('\'''${cudnn}'\''')'
  '' + ''
    TMPDIR=$out/theano
    NIX_THEANO_HOME=$out/theano
    HOME=$out/theano
    python -c "import theano"
  '';

As you can see, I'm trying to prevent theano from looking at HOME (but it does anyway, and I haven't figured out why yet), but beyond that, I'm also getting a compilation error:

You can find the C code in this temporary file: /nix/store/q2nph5dnaagqvf82cky1ixsqiz71pv5w-python3.7-Theano-1.0.4/theano/theano_compilation_error_57yanbeq
Traceback (most recent call last):
  File "/build/Theano-1.0.4/theano/gof/lazylinker_c.py", line 81, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/build/Theano-1.0.4/theano/gof/lazylinker_c.py", line 105, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/build/Theano-1.0.4/theano/__init__.py", line 110, in <module>
    from theano.compile import (
  File "/build/Theano-1.0.4/theano/compile/__init__.py", line 12, in <module>
    from theano.compile.mode import *
  File "/build/Theano-1.0.4/theano/compile/mode.py", line 11, in <module>
    import theano.gof.vm
  File "/build/Theano-1.0.4/theano/gof/vm.py", line 674, in <module>
    from . import lazylinker_c
  File "/build/Theano-1.0.4/theano/gof/lazylinker_c.py", line 140, in <module>
    preargs=args)
  File "/build/Theano-1.0.4/theano/gof/cmodule.py", line 2396, in compile_str
    (status, compile_stderr.replace('\n', '. ')))
Exception: Compilation failed (return status=1): /nix/store/q2nph5dnaagqvf82cky1ixsqiz71pv5w-python3.7-Theano-1.0.4/theano/.theano/compiledir_Linux-4.19.80-x86_64-with-glibc2.2.5--3.7.5-64/lazylinker_ext/mod.cpp:2:10: fatal error: theano_mod_helper.h: No such file or directory.  #include "theano_mod_helper.h".           ^~~~~~~~~~~~~~~~~~~~~. compilation terminated..
builder for '/nix/store/p90hbc1clf828pbrq8wazqwrb478yxc0-python3.7-Theano-1.0.4.drv' failed with exit code 1

I can't figure out why the environment would be so much different that it can't find a quoted include. I thought about using an environment variable to manually tell it where the include is, but I'm not aware of an environment variable to override the -iquote behavior. All of the usual suspects only override -I/-isystem behavior.

[stale[bot]]

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the
   related code and @ mention them in a comment.
2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
   irc.freenode.net.

[idontgetoutmuch]

I can't see me working with pymc3 for the next 6 months - I guess I can always change its label if I do start working with it.

[twhitehead]

Just a note that the MacOS issues with Theano was resolved in #76327, so, if that was the only issue, this should be working now.

[veprbl]

@twhitehead There were two issues open initially, and the macOS issue #73034 was closed back then.

Would be nice to know if this one still reproduces on NixOS 20.03

[bcdarwin]

On NixOS:

$ nix-shell -p "python3.withPackages (pkgs: [ pkgs.pymc3 ])" -I nixpkgs=~/nixpkgs
$ python3 -c 'import pymc3'
You can find the C code in this temporary file: /run/user/1234/theano_compilation_error_cefl4ub_
Traceback (most recent call last):
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 81, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 105, in <module>
    actual_version, force_compile, _need_reload))
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.211, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/pymc3/__init__.py", line 5, in <module>
    from .distributions import *
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/pymc3/distributions/__init__.py", line 1, in <module>
    from . import timeseries
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/pymc3/distributions/timeseries.py", line 2, in <module>
    import theano.tensor as tt
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/__init__.py", line 110, in <module>
    from theano.compile import (
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/compile/__init__.py", line 12, in <module>
    from theano.compile.mode import *
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/compile/mode.py", line 11, in <module>
    import theano.gof.vm
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/gof/vm.py", line 674, in <module>
    from . import lazylinker_c
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/gof/lazylinker_c.py", line 140, in <module>
    preargs=args)
  File "/nix/store/4q2lr2k4jxflf8c85f61jrqgjayg0hnx-python3-3.7.7-env/lib/python3.7/site-packages/theano/gof/cmodule.py", line 2396, in compile_str
    (status, compile_stderr.replace('\n', '. ')))
Exception: Compilation failed (return status=1): impure path `/run/user/1234/ccr71vvo.o' used in link. collect2: error: ld returned 1 exit status. 
>>>