Zipline: Debugging custom bundle in IDE (PyCharm)

[danielkiedrowski]

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: MacOS
• Python Version: Python 3.5.0
• Python Bitness: 64
• How did you install Zipline: pip
• Python packages:

alembic==0.9.9
asn1crypto==0.24.0
attrs==17.4.0
autobahn==18.4.1
Automat==0.6.0
bcolz==0.12.1
Bottleneck==1.2.1
certifi==2018.4.16
cffi==1.11.5
chardet==3.0.4
click==6.7
constantly==15.1.0
contextlib2==0.5.5
cryptography==2.2.2
cyordereddict==1.0.0
Cython==0.28.2
dateparser==0.7.0
decorator==4.3.0
empyrical==0.4.2
hyperlink==18.0.0
idna==2.6
incremental==17.5.0
intervaltree==2.1.0
Logbook==1.3.3
lru-dict==1.1.6
Mako==1.0.7
MarkupSafe==1.0
multipledispatch==0.5.0
networkx==1.11
numexpr==2.6.4
numpy==1.14.3
pandas==0.18.1
pandas-datareader==0.5.0
patsy==0.5.0
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycparser==2.18
pyOpenSSL==17.5.0
PyScaffold==3.0.3
python-binance==0.6.9
python-dateutil==2.7.2
python-editor==1.0.3
pytz==2018.4
regex==2018.2.21
requests==2.18.4
requests-file==1.4.3
requests-ftp==0.3.1
scipy==1.0.1
service-identity==17.0.0
six==1.11.0
sortedcontainers==1.5.10
SQLAlchemy==1.2.7
statsmodels==0.8.0
tables==3.4.3
toolz==0.9.0
Twisted==18.4.0
txaio==2.10.0
tzlocal==1.5.1
urllib3==1.22
zipline==1.2.0
zope.interface==4.5.0

Now that you know a little about me, let me tell you about the issue I am
having:

I am trying to create my own bundle for a crypto exchange. I have now all my functions and am trying to ingest this. I am running into errors, but I find it really hard to debug, since I am using the CLI to do the ingestion, I can't set breakpoints in my code.

I tried to manually run the register function in my own IDE, but nothing happens. So maybe some more stuff happens before and after extensions.py is executed when I try to ingest my data.

What is a good way to get the ingestion process running in my IDE so I can debug my code properly?

Sincerely,
kie

[freddiev4]

Hi @danielkiedrowski thanks for posting. Maybe try posting your errors here and I'll see if I can be of help.

Aside from that, there's a couple of alternatives here:

• You could do is git clone this repo, and then install your local copy using:

  pip install -e zipline/
  

  and then you can edit core.py in zipline/data/bundles/core.py to set breakpoints

• If you're registering your bundle from ~/.zipline/extensions.py you should be able to step through pdb and find the line where your errors are thrown if that's the case; if you are not familiar with pdb already, here's a nice Tutorial

• Edit the files in /path/to/your/zipline-installation/data/bundles/core.py to set break points (definitely do not recommend doing this though)

[richafrank]

Also, you should be able to run the same command from PyCharm by debugging the zipline.__main__.py script.

Try selecting that file and choosing Debug '__main__'. You can edit the PyCharm Debug configuration to pass to it your parameters like you would from the command line.

[danielkiedrowski]

Thank you @richafrank! That worked out of the box.