Zipline: "ValueError: numpy.dtype has the wrong size, try recompiling"

[ishandutta2007]

The full stack trace:

  File "run_algo.py", line 17, in <module>
    from zipline.utils import parse_args, run_pipeline
  File "/Library/Python/2.7/site-packages/zipline/__init__.py", line 25, in <module>
    from . import data
  File "/Library/Python/2.7/site-packages/zipline/data/__init__.py", line 1, in <module>
    from . import loader
  File "/Library/Python/2.7/site-packages/zipline/data/loader.py", line 25, in <module>
    import pandas as pd
  File "/Library/Python/2.7/site-packages/pandas/__init__.py", line 7, in <module>
    from pandas import hashtable, tslib, lib
  File "pandas/src/numpy.pxd", line 157, in init pandas.hashtable (pandas/hashtable.c:23654)
ValueError: numpy.dtype has the wrong size, try recompiling

I uninstalled and reinstalled numpy(through pip), but it didn't help.
Is there anything else I am missing ?
The tutorial also doesn't mention that before running "python run_algo.py -f dual_moving_average.py --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o dma.pickle" I should move to a directory where run_algo.py and dual_moving_average.py are present

[ssanderson]

Hi Ishan,

That error is telling you that you need to recompile pandas, not numpy. Pandas uses numpy's headers to access their C API, so if you install numpy, then install pandas (using the currently-installed numpy headers), then upgrade numpy, pandas may no longer function properly. Can you try running pip install --upgrade pandas?

[ishandutta2007]

Hey Scott didn't make much difference . Exacltly the same error on trying the following

sudo pip uninstall numpy 
sudo pip uninstall pandas   
sudo pip install --upgrade pandas
chmod 777 run_algo.py  
chmod 777 dual_moving_average.py  
sudo python run_algo.py -f dual_moving_average.py --symbols AAPL --start 2011-1-1 --end 2012-1-1 -o dma.pickle

[ssanderson]

If you just open a Python shell and do import pandas, what happens? Also, it looks like you're running from the system Python on OSX? You might want to consider running from inside a virtualenv.

[ishandutta2007]

In [3]: import pandas
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-d6ac987968b6> in <module>()
----> 1 import pandas

/Library/Python/2.7/site-packages/pandas/__init__.py in <module>()
      5 
      6 try:
----> 7     from pandas import hashtable, tslib, lib
      8 except ImportError as e:  # pragma: no cover
      9     module = str(e).lstrip('cannot import name ')  # hack but overkill to use re

pandas/src/numpy.pxd in init pandas.tslib (pandas/tslib.c:73721)()

ValueError: numpy.dtype has the wrong size, try recompiling

[ishandutta2007]

This worked (ref: http://stackoverflow.com/questions/17709641/valueerror-numpy-dtype-has-the-wrong-size-try-recompiling):

sudo pip uninstall pandas
sudo pip uninstall numpy
sudo easy_install --upgrade numpy
sudo pip install pandas

[ssanderson]

ok. For what it's worth, using sudo with pip and working in the system python is generally considered bad practice (largely for reasons like what you're experiencing). I'd highly recommend using virtualenv or something like it (e.g. conda envs).

[monajalal]

@ishandutta2007 after 30 minutes of trying everything else on OSX Yosemite for Python 2.7, your solution worked! Thanks!

[monajalal]

Also this error happened for me when I was trying to import nltk package!

[chelladurai89]

Unble to uninstall the numpy. But upgrading the numpy worked for me 👍

sudo easy_install --upgrade numpy