Shap: ModuleNotFoundError after importing shap

[emahnovets]

HI,

I've faced an issue, when got an error after importing shap module:

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

It works file on my local machine (macOs 10.14.2, python 3.6.5), but fails in docker container (python:3.6-jessie, python 3.6.8)

Steps to reproduce:

pip install shap
python -c "import shap" //ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

Output of pip show shap:

Name: shap
Version: 0.27.0
Summary: A unified approach to explain the output of any machine learning model.
Home-page: http://github.com/slundberg/shap
Author: Scott Lundberg
Author-email: [email protected]
License: MIT
Location: /usr/local/lib/python3.6/site-packages
Requires: tqdm, pandas, ipython, numpy, scipy, matplotlib, scikit-learn
Required-by: 

Can anyone take a look?

[WilliamRaynaut]

Greetings. I get the exact same thing in a fresh venv with the following versions:

absl-py==0.6.1
astor==0.7.1
backcall==0.1.0
cycler==0.10.0
decorator==4.3.0
gast==0.2.1
grpcio==1.17.1
h5py==2.9.0
ipython==7.2.0
ipython-genutils==0.2.0
jedi==0.13.2
Keras-Applications==1.0.6
Keras-Preprocessing==1.0.5
kiwisolver==1.0.1
Markdown==3.0.1
matplotlib==3.0.2
numpy==1.15.4
pandas==0.23.4
parso==0.3.1
pexpect==4.6.0
pickleshare==0.7.5
prompt-toolkit==2.0.7
protobuf==3.6.1
psycopg2==2.7.6.1
ptyprocess==0.6.0
Pygments==2.3.1
pyparsing==2.3.0
python-dateutil==2.7.5
pytz==2018.9
scikit-learn==0.20.2
scipy==1.2.0
shap==0.27.0
six==1.12.0
tensorboard==1.12.2
tensorflow==1.12.0
termcolor==1.1.0
tqdm==4.29.0
traitlets==4.3.2
wcwidth==0.1.7
Werkzeug==0.14.1
xgboost==0.81

Additionally, call to shap.TreeExplainer fails with a similar ImportError: numpy.core.multiarray failed to import...

In the same venv, simply importing numpy.core.multiarray succeeds, but importing numpy.core._multiarray_umath fails.

Any help appreciated...

[emahnovets]

@WilliamRaynaut I've just found some dirty hack:

pip install shap==0.23.0
pip install -I shap

After that it works fine, but it's a hack and we shouldn't use it. ⚠️

[WilliamRaynaut]

Thanks !
I just tried it on a new venv though, and I get the exact same ImportErrors...

[slundberg]

Is this related to #381 (which seemed to be a general numpy issue)?

You can also always try a conda install.

[WilliamRaynaut]

The solution in #96 to install with pip install git+https://github.com/slundberg/shap.git solved this for me.

I spotted the message C extension was not built during install! at any call to TreeExplainer, which led me to believe that 0.27.0 had a problem with _cext. Directly installing 0.27.2 from the git seems to solve the issue.

[slundberg]

@WilliamRaynaut I can't think what would have changed since 0.27.0 to fix this, but glad it is working. Just post if it comes back.

[ghost]

Had the same issue. Updating Numpy to 1.16 solved it.

[faizan1990]

I already have numpy 1.16.4, but I am still getting
ImportError: No module named 'shap'

[slundberg]

@faizan1990 something probably wen't wrong in the install process. If you see some error message during the install that could help point in the right direction. (you also just try a conda install instead of pip)

[faizan1990]

Hi @slundberg, that's correct! I have documented the steps that worked for me:

My environment:
Running Jupyter notebooks in a Linux Docker container over a Mac

Steps to fix the problem:

1. Shell into container (docker exec -it {container-name} /bin/bash)
2. Run pip3 install {missing-package-name}
3. Restart {container-name}
4. Shift + Enter same cell again
5. Kernel should die temporarily. Wait for it to come back
6. Shift + Enter same cell again and it should work as expected

All the best!

[luisaporrasm]

Hi, I am getting the message: C extension was not built during install!

Any help is appreciated!!!

[ivan-marroquin]

Hi,

I have python 3.6.5 64 bits on a windows machine. I installed numpy 1.15.4 and shap 0.34.0.

When I import shap package, I get this error message:

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

During the installation using pip, I got these messages:
Collecting shap==0.34.0
Using cached https://files.pythonhosted.org/packages/07/81/da8afdb67523f342600c1c2d360ac4e989eb845fbb75b1d29b9f5cf1dc29/shap-0.34.0-cp36-cp36m-win_amd64.whl
Requirement already satisfied: tqdm>4.25.0 in c:\temp\python\python_3.6.5\lib\site-packages (from shap==0.34.0)
Requirement already satisfied: pandas in c:\temp\python\python_3.6.5\lib\site-packages (from shap==0.34.0)
Requirement already satisfied: scipy in c:\temp\python\python_3.6.5\lib\site-packages (from shap==0.34.0)
Requirement already satisfied: numpy in c:\temp\python\python_3.6.5\lib\site-packages (from shap==0.34.0)
Requirement already satisfied: scikit-learn in c:\temp\python\python_3.6.5\lib\site-packages (from shap==0.34.0)
Requirement already satisfied: python-dateutil>=2.5.0 in c:\temp\python\python_3.6.5\lib\site-packages (from pandas->shap==0.34.0)
Requirement already satisfied: pytz>=2011k in c:\temp\python\python_3.6.5\lib\site-packages (from pandas->shap==0.34.0)
Requirement already satisfied: six>=1.5 in c:\temp\python\python_3.6.5\lib\site-packages (from python-dateutil>=2.5.0->pandas->shap==0.34.0)
Installing collected packages: shap
Successfully installed shap-0.34.0
You are using pip version 9.0.3, however version 20.0.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

Any suggestions?

Kind regards,

Ivan

[ivan-marroquin]

Hi,

I tried the following: (1) downloaded the zip file and uncompressed it, and (2) run "python setup.py install".

When I import shap, I don't get the reported numpy error message.

Hope this helps,

Ivan

[slundberg]

@ivan-marroquin glad that worked. The first problem looks like an issue with the numpy that was installed, and to debug would need to know the line that caused the error.

[ivan-marroquin]

Hi @slundberg . thanks for the quick response. I did the following: (1) Open a DOS window console, (2) run python, and (3) type the command "import shap"

If you need more information, please let me know.

Ivan

[slundberg]

My guess is that is was a package issue outside of SHAP.