Gensim: numpy==1.7.1 and gensim

[vlad17]

On a bare environment with numpy==1.7.1 installed, gensim installs successfully but doesn't import. See the following transcript:

vlad@vladT450s:~$ pip --version
pip 1.5.4 from /usr/lib/python3/dist-packages (python 3.4)

vlad@vladT450s:~$ cd /tmp/

vlad@vladT450s:/tmp$ virtualenv .
Using base prefix '/usr'
New python executable in ./bin/python3
Also creating executable in ./bin/python
Installing setuptools, pip, wheel...done.

vlad@vladT450s:/tmp$ . bin/activate

(tmp)vlad@vladT450s:/tmp$ pip install numpy==1.7.1
Collecting numpy==1.7.1
  Downloading numpy-1.7.1.tar.gz (2.8MB)
    100% |████████████████████████████████| 2.8MB 251kB/s 
Building wheels for collected packages: numpy
  Running setup.py bdist_wheel for numpy
  Stored in directory: 
/home/vlad/.cache/pip/wheels/85/29/07/a25278230bd2788be0d5bfecca36422df8a1b1607d285d4cdf
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.7.1

(tmp)vlad@vladT450s:/tmp$ pip install gensim
Collecting gensim
Collecting six>=1.5.0 (from gensim)
  Using cached six-1.10.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.3 in ./lib/python3.4/site-packages (from gensim)
Collecting smart-open>=1.2.1 (from gensim)
Collecting scipy>=0.7.0 (from gensim)
  Using cached scipy-0.16.1.tar.gz
Collecting requests (from smart-open>=1.2.1->gensim)
  Using cached requests-2.8.1-py2.py3-none-any.whl
Collecting bz2file (from smart-open>=1.2.1->gensim)
Collecting boto>=2.32 (from smart-open>=1.2.1->gensim)
  Using cached boto-2.38.0-py2.py3-none-any.whl
Collecting httpretty==0.8.6 (from smart-open>=1.2.1->gensim)
Building wheels for collected packages: scipy
  Running setup.py bdist_wheel for scipy
  Stored in directory: /home/vlad/.cache/pip/wheels/84/d5/73/ad1d93891b39dcc3197feb26d56ae16eb004110379c215184b
Successfully built scipy
Installing collected packages: scipy, gensim
Successfully installed gensim-0.12.2 scipy-0.16.1

(tmp)vlad@vladT450s:/tmp$ python
Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gensim
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/lib/python3.4/site-packages/gensim/__init__.py", line 6, in <module>
    from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization
  File "/tmp/lib/python3.4/site-packages/gensim/models/__init__.py", line 13, in <module>
    from .word2vec import Word2Vec
  File "/tmp/lib/python3.4/site-packages/gensim/models/word2vec.py", line 100, in <module>
    from gensim.models.word2vec_inner import train_sentence_sg, train_sentence_cbow, FAST_VERSION
  File "__init__.pxd", line 864, in init gensim.models.word2vec_inner (./gensim/models/word2vec_inner.c:9265)
ValueError: numpy.ufunc has the wrong size, try recompiling

I was able to replicate this in two environments, Springdale 7 and Ubuntu 14.04

[gojomo]

This scipy issue with a similar error suggests the problem may be another alternate numpy present: https://github.com/scipy/scipy/issues/3448

Is there a reason not to use the Python 3 venv support (rather than virtualenv)? Is there a chance you're using a pre-1.7 virtualenv, and thus the environment may also inherit global packages (unless using the --no-site-packages option)?

As it only says 'collecting gensim' rather than giving any indication it's rebuilt, might something be persisting from an install/build from before numpy-1.7.1 was added?

[vlad17]

In my other platform, venv is unavailable. As I mentioned before, I don't actually have this problem anymore (https://groups.google.com/forum/#!topic/gensim/3NmLzXf9VX0). It can obviously be resolved by upgrading numpy. I opened the issue for posterity.

Your worry about the 'collecting gensim' point is probably moot - I had a line before where I called pip install gensim but terminated it during the stage where I was collecting the scipy package (I forgot to install BLAS on this environment). I didn't include this in the transcript because I felt it'd be too confusing. My apologies for the confusion.

[highintothesky]

I had the same problem using sudo pip install, but I fixed it by removing the pip installed gensim, and then reinstalling using sudo easy_install --upgrade gensim. for some reason this works, I can import without a problem.

[piskvorky]

Closing this -- issue most likely caused by mixing multiple NumPy installation (global + local). Solution is to use Python virtual environments.