Faiss: GPU issue when installing from conda
Summary
I install Faiss from conda (GPU version)
And I got ImportError: No module named 'swigfaiss'
Could you guys help me out?
Did I forget anything?
Platform
OS: Ubuntu
Faiss version:
Faiss compilation options:
Running on :
- [ ] CPU
- [x] GPU
Reproduction instructions
hminle
All 20 comments
I install faiss:
faiss-gpu: 1.2.1-py35_cuda9.0.176_1 pytorch
my cuda version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85
I managed to overcome swigfaiss issue, but got this one
It does not recognize my GPU
hminle
on 20 Mar 2018
@hminle , could you list all the items under $YOUR_CONDA_HOME/lib/python3.6/site-packages/faiss?
ailzhang
on 20 Mar 2018
Here is all the items
hminle
on 20 Mar 2018
Inside that folder, could you try 'python; import faiss' and let me know what happens?
ailzhang
on 20 Mar 2018
it's show the same issue
hminle
on 20 Mar 2018
That looks weird, since .so file it tries to import is there.
Could you try rm -rf $YOUR_CONDA_HOME/lib/python3.6/site-packages/faiss*, and then reinstall the faiss-gpu package again?
ailzhang
on 20 Mar 2018
nothing happens, it's the same issue, fall back to CPU version.
I installed pytorch, and check the GPU, so I can make sure that my configuration for GPU usage is right.
Could you find out if there is another system requirement or env variables?
hminle
on 20 Mar 2018
It does require mkl and numpy. That's all I can think of.
Could you paste your conda list here?
ailzhang
on 20 Mar 2018
here you are:
# packages in environment at /home/hminle/miniconda2/envs/pydata:
#
# Name Version Build Channel
absl-py 0.1.11 <pip>
astor 0.6.2 <pip>
bleach 1.5.0 <pip>
bleach 2.1.3 py35_0
ca-certificates 2017.08.26 h1d4fec5_0
certifi 2018.1.18 py35_0
cffi 1.11.5 py35h9745a5d_0
chardet 3.0.4 <pip>
cuda91 1.0 h4c16780_0 pytorch
cudatoolkit 8.0 3
cycler 0.10.0 py35hc4d5149_0
dbus 1.12.2 hc3f9b76_1
decorator 4.2.1 py35_0
entrypoints 0.2.3 py35h48174a2_2
enum34 1.1.6 <pip>
expat 2.2.5 he0dffb1_0
faiss-gpu 1.2.1 py35_cuda9.0.176_1 pytorch
flake8 3.5.0 <pip>
fontconfig 2.12.4 h88586e7_1
freetype 2.8 hab7d2ae_1
gast 0.2.0 <pip>
glib 2.53.6 h5d9569c_2
gmp 6.1.2 h6c8ec71_1
grpcio 1.10.0 <pip>
gst-plugins-base 1.12.4 h33fb286_0
gstreamer 1.12.4 hb53b477_0
h5py 2.7.1 py35h8d53cdc_0
hdf5 1.10.1 h9caa474_1
html5lib 0.9999999 <pip>
html5lib 1.0.1 py35h2f9c1c0_0
icu 58.2 h9c2bf20_1
idna 2.6 <pip>
intel-openmp 2018.0.0 hc7b2577_8
ipykernel 4.8.2 py35_0
ipython 6.2.1 py35hd850d2a_1
ipython_genutils 0.2.0 py35hc9e07d0_0
ipywidgets 7.1.2 py35_0
jedi 0.11.1 py35_0
jinja2 2.10 py35h480ab6d_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py35h4395190_0
jupyter 1.0.0 py35_4
jupyter_client 5.2.2 py35_0
jupyter_console 5.2.0 py35h4044a63_1
jupyter_core 4.4.0 py35ha89e94b_0
kiwisolver 1.0.1 py35hcb1117a_0
libedit 3.1 heed3624_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 7.2.0 h7cc24e2_2
libgfortran-ng 7.2.0 h9f7466a_2
libpng 1.6.34 hb9fc6fc_0
libsodium 1.0.15 hf101ebd_0
libstdcxx-ng 7.2.0 h7a57d05_2
libtiff 4.0.9 h28f6b97_0
libxcb 1.12 hcd93eb1_4
libxml2 2.9.7 h26e45fe_0
Markdown 2.6.11 <pip>
markupsafe 1.0 py35h4f4fcf6_1
matplotlib 2.2.0 py35hbc4b006_0
mccabe 0.6.1 <pip>
mistune 0.8.3 py35_0
mkl 2018.0.1 h19d6760_4
nbconvert 5.3.1 py35hc5194e3_0
nbformat 4.4.0 py35h12e6e07_0
ncurses 6.0 h9df7e31_2
networkx 2.1 <pip>
notebook 5.4.0 py35_0
numpy 1.14.2 py35hdbf6ddf_0
numpy 1.14.1 <pip>
olefile 0.45.1 py35_0
opencv-python 3.4.0.12 <pip>
openssl 1.0.2n hb7f436b_0
pandas 0.22.0 py35hf484d3e_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py35h1565a15_1
parso 0.1.1 py35h1b200a3_0
pcre 8.41 hc27e229_1
pexpect 4.4.0 py35_0
pickleshare 0.7.4 py35hd57304d_0
pillow 5.0.0 py35h3deb7b8_0
Pillow 5.0.0 <pip>
pip 9.0.1 py35_5
portalocker 1.2.0 <pip>
prompt_toolkit 1.0.15 py35hc09de7a_0
protobuf 3.5.2 <pip>
psutil 5.4.3 <pip>
ptyprocess 0.5.2 py35h38ce0a3_0
pycodestyle 2.3.1 <pip>
pycparser 2.18 py35h61b3040_1
pyflakes 1.6.0 <pip>
pygments 2.2.0 py35h0f41973_0
pyparsing 2.2.0 py35h041ed72_1
pyqt 5.6.0 py35h0e41ada_5
python 3.5.4 hc3d631a_27
python-dateutil 2.6.1 py35h90d5b31_1
pytorch 0.3.1 py35_cuda9.1.85_cudnn7.0.5_2 [cuda91] pytorch
pytz 2018.3 py35_0
PyWavelets 0.5.2 <pip>
pyzmq 17.0.0 py35h14c3975_0
qt 5.6.2 hd25b39d_14
qtconsole 4.3.1 py35h4626a06_0
readline 7.0 ha6073c6_4
requests 2.18.4 <pip>
scikit-image 0.13.1 <pip>
scikit-learn 0.19.1 py35hbf1f462_0
scipy 1.0.0 py35hcbbe4a2_0
send2trash 1.5.0 py35_0
setuptools 38.5.1 py35_0
simplegeneric 0.8.1 py35_2
sip 4.18.1 py35h9eaea60_2
six 1.11.0 py35h423b573_1
six 1.11.0 <pip>
sqlite 3.22.0 h1bed415_0
tensorboard 1.6.0 <pip>
tensorflow-gpu 1.4.0 <pip>
tensorflow-tensorboard 0.4.0 <pip>
termcolor 1.1.0 <pip>
terminado 0.8.1 py35_1
testpath 0.3.1 py35had42eaf_0
tk 8.6.7 hc745277_3
torchvision 0.2.0 py35heaa392f_1 pytorch
tornado 5.0 py35_0
traitlets 4.3.2 py35ha522a97_0
urllib3 1.22 <pip>
wcwidth 0.1.7 py35hcd08066_0
webencodings 0.5.1 py35hb6cf162_1
Werkzeug 0.14.1 <pip>
wheel 0.30.0 py35hd3883cf_1
widgetsnbextension 3.1.4 py35_0
xz 5.2.3 h55aa19d_2
zeromq 4.2.3 h439df22_3
zlib 1.2.11 ha838bed_2
Hi, weirdly I cannot repro the error on my side.
Could you try 'from swigfaiss_gpu import *' in $YOUR_CONDA_HOME/lib/python3.6/site-packages/faiss? Since this is the key to make it work.
ailzhang
on 20 Mar 2018
It cannot find the package
hminle
on 20 Mar 2018
Why you import swigfaiss_gpu instead of faiss?
hminle
on 20 Mar 2018
If you take a look at __init__.py, that's what it does for import faiss
ailzhang
on 20 Mar 2018
Also could you do 'ls -alt .' in that folder? We need to figure out why it's not picking up the file.
ailzhang
on 20 Mar 2018
ah @ailzhang I found the problem, your package uses libculas.so.9.0. But I install libculas.so.9.1
hminle
on 20 Mar 2018
I fix the issue by installing CUDA 9.0 instead of the latest version of CUDA. You guys will upgrade your package for CUDA 9.1 any time soon?
hminle
on 20 Mar 2018
Cool! yea we only did it for cuda90 for this release. We will likely include cuda91 in the next release.
ailzhang
on 20 Mar 2018
@ailzhang thank you for your support, I will close this issue, by the way, I think you should specify cuda version for your package's releases (like Pytorch).
hminle
on 20 Mar 2018
@hminle It's shown in the name py35_cuda9.0. But you are right, we probably want to make it clear somewhere in the README as well.
ailzhang
on 20 Mar 2018
Related issues
0DF0Arc
路
3Comments
Ljferrer
路
3Comments
cherryPotter
路
3Comments
mylyu
路
3Comments
wwmmqq
路
3Comments
Most helpful comment
@hminle It's shown in the name py35_cuda9.0. But you are right, we probably want to make it clear somewhere in the README as well.