Xla: ImportError: libifport.so.5: cannot open shared object file: No such file or directory

[CalebEverett]

🐛 Bug

I've started a Debian GNU/Linux 9 Stretch + Pytorch/XLA gcp vm. When I ssh into it and start jupyter, I can't import sklearn.

To Reproduce

1. Start a Debian GNU/Linux 9 Stretch + Pytorch/XLA gcp vm with machine type n1-highmem-16
2. ssh into it
3. activate the torch-xla-1.7 environment
4. start a jupyter notebook
5. import sklearn

/anaconda3/envs/torch-xla-1.7/lib/python3.6/site-packages/scipy/sparse/linalg/isolve/iterative.py in <module>
      8 import numpy as np
      9 
---> 10 from . import _iterative
     11 
     12 from scipy.sparse.linalg.interface import LinearOperator

ImportError: libifport.so.5: cannot open shared object file: No such file or directory

[zcain117]

This happens also without jupyter:

(torch-xla-1.7) zcain@zcain-sanity-check:~$ conda activate torch-xla-nightly
(torch-xla-nightly) zcain@zcain-sanity-check:~$ python3
Python 3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/sklearn/__init__.py", line 76, in <module>
    from .base import clone
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/sklearn/base.py", line 16, in <module>
    from .utils import _IS_32BIT
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/sklearn/utils/__init__.py", line 20, in <module>
    from .validation import (as_float_array,
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/sklearn/utils/validation.py", line 21, in <module>
    from .fixes import _object_dtype_isnan
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/sklearn/utils/fixes.py", line 18, in <module>
    from scipy.sparse.linalg import lsqr as sparse_lsqr  # noqa
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/scipy/sparse/linalg/__init__.py", line 113, in <module>
    from .isolve import *
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/scipy/sparse/linalg/isolve/__init__.py", line 6, in <module>
    from .iterative import *
  File "/anaconda3/envs/torch-xla-nightly/lib/python3.6/site-packages/scipy/sparse/linalg/isolve/iterative.py", line 10, in <module>
    from . import _iterative
ImportError: libifport.so.5: cannot open shared object file: No such file or directory

This happens on torch-xla-nightly, torch-xla-1.7, torch-xla-1.6

[zcain117]

I'm trying now using the docker image rather than the conda env.

sudo docker run -it gcr.io/tpu-pytorch/xla@sha256:30c4d4f76d07deb18d37fd014d04de20d97f5ffb2f05b40488673332b8f912ad
(pytorch) root@64d0ee9b9e98:/# python
Python 3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
>>> import scipy
>>>
>>> sklearn.__version__
'0.23.1'
>>> scipy.__version__
'1.5.0'

No errors

I also tried out this Kaggle notebook that imports scipy and sklearn and did not see any errors there either.

So this looks to be specific to our conda environments

[taylanbil]

running conda install numpy resolves this for me, you can try this workaround for now.

[zcain117]

This should be fixed now if you make a new VM. I tried all 3 pytorch conda envs on new VM and was able to import sklearn on all of them

Aside from making a new VM, you can try to repair your conda env - specifically here is the version of numpy that I think you would need if you're using a VM from before December 15 2020 and seeing this error:

numpy                     1.19.2           py36h54aff64_0  
numpy-base                1.19.2           py36hfa32c7d_0

So I think you could run something like conda install numpy=1.19.2 if that's easier than creating a new VM