zipline incompatible with pandas 0.18 - cannot import name data, cannot import name mask_missing

[edendekker]

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: mac os elcapitan
• Python Version: 2.7.11
• Python Bitness: 64
• How did you install Zipline: pip
• Python packages:
  backports.ssl-match-hostname==3.4.0.2
  bcolz==1.0.0
  beautifulsoup4==4.3.2
  Bottleneck==1.0.0
  certifi==14.5.14
  click==6.6
  contextlib2==0.5.3
  cyordereddict==1.0.0
  Cython==0.24
  decorator==4.0.9
  Django==1.8.2
  docutils==0.11
  gnureadline==6.3.3
  ipython==2.2.0
  Jinja==1.2
  Jinja2==2.7.3
  ladon==0.8.9
  Logbook==0.12.5
  MarkupSafe==0.23
  matplotlib==1.4.2
  mock==1.0.1
  networkx==1.11
  nltk==3.2.1
  nose==1.3.4
  numexpr==2.5.2
  numpy==1.11.0
  pandas==0.18.1
  patsy==0.4.0
  psycopg2==2.5.3
  Pygments==1.6
  pyparsing==2.0.3
  python-dateutil==2.5.3
  pytz==2016.4
  pyzmq==14.3.1
  QSTK==0.2.8
  requests==2.10.0
  scipy==0.17.0
  selenium==2.53.2
  six==1.10.0
  Sphinx==1.2.2
  SQLAlchemy==1.0.12
  statsmodels==0.6.1
  toolz==0.7.4
  tornado==4.0.1
  vboxapi==1.0
  web.py==0.37
  zipline==0.9.0

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• What did you expect to happen? a successful import of zipline
• What happened instead?
  Traceback (most recent call last):
  File "", line 1, in
  File "/usr/local/lib/python2.7/site-packages/zipline/init.py", line 19, in
  from . import data
  ImportError: cannot import name data

Here is how you can reproduce this issue on your machine:

Reproduction Steps

Kalel:DoctorFate e$ sudo pip install zipline
The directory '/Users/e/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/e/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting zipline
Downloading zipline-0.9.0.tar.gz (981kB)
100% |████████████████████████████████| 983kB 981kB/s
Requirement already satisfied (use --upgrade to upgrade): pip>=7.1.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): setuptools>18.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): Logbook>=0.12.5 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): pytz>=2015.4 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.9.2 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): scipy>=0.15.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): pandas>=0.16.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): patsy>=0.4.0 in /Library/Python/2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): statsmodels>=0.6.1 in /Library/Python/2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.4.2 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): six>=1.10.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): requests>=2.9.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.22.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): cyordereddict>=0.2.2 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): bottleneck>=1.0.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.4.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): decorator>=4.0.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): networkx>=1.9.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): numexpr>=2.4.6 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): bcolz>=0.12.1 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): click>=4.0.0 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): toolz>=0.7.4 in /usr/local/lib/python2.7/site-packages (from zipline)
Requirement already satisfied (use --upgrade to upgrade): sqlalchemy>=1.0.8 in /usr/local/lib/python2.7/site-packages (from zipline)
Installing collected packages: zipline
Running setup.py install for zipline ... done
Successfully installed zipline-0.9.0
Kalel:DoctorFate e$ python
Python 2.7.11 (default, Jan 22 2016, 08:29:18)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import numpy
import pandas
import zipline
/usr/local/lib/python2.7/site-packages/pandas/io/data.py:35: FutureWarning:
The pandas.io.data module is moved to a separate package (pandas-datareader) and will be removed from pandas in a future version.
After installing the pandas-datareader package (https://github.com/pydata/pandas-datareader), you can change the import from pandas.io import data, wb to from pandas_datareader import data, wb.
FutureWarning)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/site-packages/zipline/init.py", line 26, in
from . algorithm import TradingAlgorithm
File "/usr/local/lib/python2.7/site-packages/zipline/algorithm.py", line 106, in
from zipline.history import HistorySpec
File "/usr/local/lib/python2.7/site-packages/zipline/history/init.py", line 21, in
from . import history_container
File "/usr/local/lib/python2.7/site-packages/zipline/history/history_container.py", line 27, in
from zipline.utils.munge import ffill, bfill
File "/usr/local/lib/python2.7/site-packages/zipline/utils/munge.py", line 15, in
from pandas.core.common import mask_missing
ImportError: cannot import name mask_missing

What steps have you taken to resolve this already?

google
uninstall reinstall

Anything else?

Nope

Sincerely,
Eddie

[JimCB]

I am having the same issue

[richafrank]

Thanks for reporting this @edendekker ! Looks like something has changed in pandas 0.18 which zipline isn't expecting. Our test matrix is only on pandas 0.16 and 0.17 currently.

I'm going to rename this issue to reflect the 0.18 incompatibility more generally, as I expect we'll work to solve any 0.18 issues all at once.

[twiecki]

This issue also makes current zipline incompatible with pyfolio which is already on pandas 0.18.2.

[wheeler18]

pandas 0.17.1 np110py27_0 also does not work!

[moongraber]

Hi

I have the same issue
running
-windows 10
-python 2.7 64 bit
-I did "conda install -c Quantopian zipline" (it worked but it downgraded pandas from 0.18 to 0.17)

then I try "from zipline.api import history" I get:

File "D:Anaconda64bitlibsite-packageszipline__init__.py", line 37, in
from . import data

ImportError: cannot import name data

[richafrank]

The latest release of zipline isn't compatible with pandas 0.18 yet, but the latest nightly build should be. You can install it from our ci channel:

conda install -c Quantopian -c https://conda.anaconda.org/Quantopian/label/ci zipline

[moongraber]

thanks Richard

I tried just now:

conda install -c Quantopian -c https://conda.anaconda.org/Quantopian/label/ci zipline

I got a downgrade for pandas, maybe the nightly run didn't update this? or maybe I am doing something wrong?

I got this:
The following packages will be DOWNGRADED due to dependency conflicts:

bottleneck:        1.1.0-np111py27_0             --> 1.0.0-np110py27_1 
numexpr:           2.6.1-np111py27_0             --> 2.6.0-np110py27_0 
numpy:             1.11.1-py27_1                 --> 1.10.4-py27_2     
pandas:            0.18.1-np111py27_0            --> 0.17.1-np110py27_0
scipy:             0.18.1-np111py27_0            --> 0.17.1-np110py27_1

Proceed ([y]/n)?

[richafrank]

Oh, sorry about that - looks like the order of the channels in the command matters (makes sense).

Try:

conda install -c https://conda.anaconda.org/Quantopian/label/ci -c Quantopian zipline

[moongraber]

thanks Richard now it has indeed updated zipline with pandas 0.18.1

I tried to run run the buyapple.py example

seems to work fine except when it tries to run the algo:
results = algo.run(data)

Traceback (most recent call last):

File "", line 1, in
results = algo.run(data)

File "C:Users....Anaconda2libsite-packagesziplinealgorithm.py", line 697, in run
for perf in self.get_generator():

File "C:Users....Anaconda2libsite-packagesziplinealgorithm.py", line 596, in get_generator
return self._create_generator(self.sim_params)

File "C:Users....Anaconda2libsite-packagesziplinealgorithm.py", line 567, in _create_generator
self.initialize(_self.initialize_args, *_self.initialize_kwargs)

File "C:Users....Anaconda2libsite-packagesziplinealgorithm.py", line 451, in initialize
self._initialize(self, _args, *_kwargs)

TypeError: initialize() got an unexpected keyword argument 'identifiers'

[richafrank]

The identifiers keyword-only arg to TradingAlgorithm was replaced in zipline 1.0. Under the new system, you zipline ingest your bundle of data once and then zipline run simulations with that data multiple times.

Looks like the docstring for TradingAlgorithm still mentions this arg - we should update that.

As this issue is about pandas 0.18, I'm going to close it as fixed. Feel free to open new issues!

[nme01]

Could you also update Zipline on PyPI?

[ProjectRamo]

Are you sure this is a pandas incompatibility?

I was trying to do exactly the same thing as you guys (running pandas, zipline on anaconda).
Nothing mentioned was helping. So I tried running the python interpreter in the command line and just importing zipline. That's it. Not importing pandas, not running anything else first.

I got this error first:

Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import zipline as zp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/__init__.py", line 42, in <module>
    from .utils.run_algo import run_algorithm
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/utils/run_algo.py", line 18, in <module>
    from zipline.data.bundles.core import load
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/data/bundles/__init__.py", line 2, in <module>
    from . import quandl  # noqa
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/data/bundles/quandl.py", line 18, in <module>
    from . import core as bundles
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/data/bundles/core.py", line 23, in <module>
    from zipline.assets.asset_db_migrations import downgrade
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/assets/asset_db_migrations.py", line 3, in <module>
    from alembic.migration import MigrationContext
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/alembic/__init__.py", line 8, in <module>
    from . import op  # noqa
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/alembic/op.py", line 6, in <module>
    util.create_module_class_proxy(Operations, globals(), locals())
AttributeError: 'module' object has no attribute 'create_module_class_proxy'

Which is explained here:

This stack overflow explains the error

After that initial error, I continue to get the error you pointed out every time i try to import zipline:

>>> import zipline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/miniconda2/lib/python2.7/site-packages/zipline/__init__.py", line 37, in <module>
    from . import data
ImportError: cannot import name data

Maybe there is a collision with Anaconda?