Zipline: Can't connect to Yahoo - Errors: Loader: failed to cache the new benchmark returns
Dear Zipline Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment
- Operating System: Linux hostname 2.6.18-238.12.1.el5 #1 SMP Tue May 31 13:22:04 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
- Python Version: Python 3.5.1
- Python Bitness: 32
- How did you install Zipline: pip
- Python packages:
alembic==0.9.1
appdirs==1.4.3
bcolz==0.12.1
Bottleneck==1.3.0.dev0
click==6.7
contextlib2==0.5.5
cyordereddict==1.0.0
Cython==0.25.2
decorator==4.0.11
empyrical==0.2.2
intervaltree==2.1.0
Logbook==1.0.0
lru-dict==1.1.6
Mako==1.0.6
MarkupSafe==1.0
multipledispatch==0.4.9
networkx==1.11
numexpr==2.6.2
numpy==1.12.1
packaging==16.8
pandas==0.18.1
pandas-datareader==0.3.0.post0
patsy==0.4.1
pyparsing==2.2.0
python-dateutil==2.6.0
python-editor==1.0.3
pytz==2017.2
requests==2.13.0
requests-file==1.4.2
requests-ftp==0.3.1
scipy==0.19.0
setuptools-scm==1.15.5
six==1.10.0
sortedcontainers==1.5.7
SQLAlchemy==1.1.9
statsmodels==0.8.0
tables==3.4.2
toolz==0.8.2
zipline==1.1.0
Now that you know a little about me, let me tell you about the issue I am
having:
Description of Issue
I just ran the buyapple.py example that described on http://www.zipline.io/beginner-tutorial.html, but got some errors, can you help to check ?
/home/kevinyuan/dev/zipline> bin/zipline ingest Downloading Bundle: quantopian-quandl [####################################] 100%
Writing data to /home/kevinyuan/.zipline/data/quantopian-quandl/2017-05-01T16;36;49.048566.
/home/kevinyuan/dev/zipline> bin/zipline run -f examples/buyapple.py -s 2000-1-1 -e 2001-12-31 [2017-05-01 16:33:28.023652] INFO: Loader: Cache at /home/kevinyuan/.zipline/data/^GSPC_benchmark.csv does not have data from 1990-01-02 00:00:00+00:00 to 2017-04-27 00:00:00+00:00. Downloading benchmark data for '^GSPC'.
[2017-05-01 16:33:28.076307] ERROR: Loader: failed to cache the new benchmark returns Traceback (most recent call last):
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1240, in do_open
h.request(req.get_method(), req.selector, req.data, headers)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1083, in request
self.send_request(method, url, body, headers)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1128, in send_request
self.endheaders(body)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1079, in endheaders
self._send_output(message_body)
File "/share/dev/tools/lib/python3.5/http/client.py", line 911, in _sendoutput
self.send(msg)
File "/share/dev/tools/lib/python3.5/http/client.py", line 854, in send
self.connect()
File "/share/dev/tools/lib/python3.5/http/client.py", line 1237, in connect
serverhostname=server_hostname)
File "/share/dev/tools/lib/python3.5/ssl.py", line 376, in wrap_socket
context=self)
File "/share/dev/tools/lib/python3.5/ssl.py", line 747, in __init_
self.do_handshake()
File "/share/dev/tools/lib/python3.5/ssl.py", line 983, in do_handshake
self.sslobj.do_handshake()
File "/share/dev/tools/lib/python3.5/ssl.py", line 628, in do_handshake
self._sslobj.dohandshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/data/loader.py", line 247, in ensure_benchmark_data
last_date,
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/data/benchmarks.py", line 59, in get_benchmark_returns
squeeze=True, # squeeze tells pandas to make this a Series
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/parsers.py", line 562, in parser_f
return read(filepath_or_buffer, kwds)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/parsers.py", line 301, in read
compression=kwds.get('compression', None))
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/common.py", line 308, in get_filepath_or_buffer
req = urlopen(str(filepath_or_buffer))
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 162, in urlopen
return opener.open(url, data, timeout)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 465, in open
response = self.open(req, data)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 483, in _open
'_open', req)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 443, in _callchain
result = func(*args)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1283, in https_open
context=self.context, check_hostname=self.check_hostname)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1242, in doopen
raise URLError(err)
urllib.error.URLError:
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1240, in doopen
h.request(req.get_method(), req.selector, req.data, headers)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1083, in request
self.send_request(method, url, body, headers)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1128, in send_request
self.endheaders(body)
File "/share/dev/tools/lib/python3.5/http/client.py", line 1079, in endheaders
self._send_output(message_body)
File "/share/dev/tools/lib/python3.5/http/client.py", line 911, in _sendoutput
self.send(msg)
File "/share/dev/tools/lib/python3.5/http/client.py", line 854, in send
self.connect()
File "/share/dev/tools/lib/python3.5/http/client.py", line 1237, in connect
serverhostname=server_hostname)
File "/share/dev/tools/lib/python3.5/ssl.py", line 376, in wrap_socket
context=self)
File "/share/dev/tools/lib/python3.5/ssl.py", line 747, in __init_
self.do_handshake()
File "/share/dev/tools/lib/python3.5/ssl.py", line 983, in do_handshake
self.sslobj.do_handshake()
File "/share/dev/tools/lib/python3.5/ssl.py", line 628, in do_handshake
self._sslobj.dohandshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "bin/zipline", line 11, in
sys.exit(main())
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/core.py", line 722, in call
return self.main(args, *kwargs)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/core.py", line 535, in invoke
return callback(args, *kwargs)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/main.py", line 97, in _
return f(args, **kwargs)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, *kwargs)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/main.py", line 240, in run
environ=os.environ,
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/utils/run_algo.py", line 132, in run
env = TradingEnvironment(asset_db_path=connstr)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/finance/trading.py", line 101, in _init_
self.bm_symbol,
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/data/loader.py", line 164, in load_market_data
trading_day,
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/data/loader.py", line 247, in ensure_benchmark_data
last_date,
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/zipline/data/benchmarks.py", line 59, in get_benchmark_returns
squeeze=True, # squeeze tells pandas to make this a Series
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/parsers.py", line 562, in parser_f
return read(filepath_or_buffer, kwds)
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/parsers.py", line 301, in _read
compression=kwds.get('compression', None))
File "/home-tahoe-n2/kevinyuan/dev/zipline/lib/python3.5/site-packages/pandas/io/common.py", line 308, in get_filepath_or_buffer
req = _urlopen(str(filepath_or_buffer))
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 162, in urlopen
return opener.open(url, data, timeout)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 465, in open
response = self.open(req, data)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 483, in open
'_open', req)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 443, in _callchain
result = func(args)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1283, in https_open
context=self.context, check_hostname=self.check_hostname)
File "/share/dev/tools/lib/python3.5/urllib/request.py", line 1242, in doopen
raise URLError(err)
urllib.error.URLError:
kevinyuan
All 55 comments
Hi @kevinyuan. I see you've tried installing with pip. Did you check to make sure you have all the necessary dependencies? (Check zipline.io)
Also have you tried using a Python 3.4 conda/virtualenv? As we don't support Python 3.5 yet.
freddiev4
on 9 May 2017
I get a similar error when creating an empty TradingAlgorithm(). I traced it down to the Yahoo! URL for the benchmark returns no longer works.
The code generates the following URL:
https://ichart.finance.yahoo.com/table.csv?a=11&s=%5EGSPC&b=29&e=15&d=4&g=d&f=2017&c=1989
which does not work.
This is generated from format_yahoo_index_url() in zipline/data/benchmarks.py.
pbharrin
on 17 May 2017
@pbharrin I've just been able to reproduce this issue today as well. Will look into how to fix this/provide an improved solution at some point.
freddiev4
on 17 May 2017
Same here today. Not sure how to fix the problem.
ywang412
on 17 May 2017
All right, now that I'm at my laptop ... it looks like this is an issue coming from Yahoo and their ichart/Finance API.
I believe we have some benchmark data that is cached/downloaded but when those benchmarks are not found then we make additional requests to Yahoo for those benchmarks.
Because we use Yahoo to download our benchmark data, and there is an issue with the API, we cannot make those requests successfully. In regards to providing a more robust solution, at this very moment I don't have any ideas yet. Will just have to wait on the API issue to get resolved for now.
freddiev4
on 17 May 2017
Where we call our get_benchmark_returns function is in zipline.data.loader.ensure_benchmark_data:
"""
Ensure we have benchmark data for `symbol` from `first_date` to `last_date`
Parameters
----------
symbol : str
The symbol for the benchmark to load.
first_date : pd.Timestamp
First required date for the cache.
last_date : pd.Timestamp
Last required date for the cache.
now : pd.Timestamp
The current time. This is used to prevent repeated attempts to
re-download data that isn't available due to scheduling quirks or other
failures.
trading_day : pd.CustomBusinessDay
A trading day delta. Used to find the day before first_date so we can
get the close of the day prior to first_date.
We attempt to download data unless we already have data stored at the data
cache for `symbol` whose first entry is before or on `first_date` and whose
last entry is on or after `last_date`.
If we perform a download and the cache criteria are not satisfied, we wait
at least one hour before attempting a redownload. This is determined by
comparing the current time to the result of os.path.getmtime on the cache
path.
"""
Thank you, Freddie. I didn't include more dates in my back testing. Not sure why it requires download again. I wonder if the cache is saved somewhere on my disk so I can read it like a CSV.
ywang412
on 17 May 2017
https://forums.yahoo.net/t5/Yahoo-Finance-help/Is-Yahoo-Finance-API-broken/td-p/250503/page/3 it is officially gone. response from Yahoo finance team. Yahoo discontinued the free service.
ywang412
on 18 May 2017
then how can we use zipline? :(
ywang412
on 18 May 2017
Hi @ywang412. zipline is one of many friends affected by these changes, so while we're considering solutions, we'll also see what pandas-datareader et al decide to do. We're not sure on the timeframe, but discussion continues on what changes to make. If you have any opinions on the matter feel free to share them 😃
freddiev4
on 18 May 2017
Here is a solution: Google finance through the Pandas datareader.
- no api key needed
- no shady scraping needed
What am I missing?
Calling this with "^GSPC" throws an exception, not sure of the symbol needed for the SPY500, SPY works.
Update 1.0: Due to licensing agreements Google cannot provide data for ^GSPC. SPY is not 100% ^GSPC because SPY pays dividends. Some will actually argue that SPY is a better benchmark as it is something you can actually trade.
pbharrin
on 18 May 2017
@pbharrin : Could you please share your patch to replace yahoo loader with google one? Thanks!
tibkiss
on 18 May 2017
quocble
on 18 May 2017
Hey Peter @pbharrin thanks for your suggestion. I think we've mentioned this in our discussion at some point; there's some other things we'll need to talk about as well (e.g. like SPY vs ^GSPC).
@quocble If I remember correctly, I looked through that WIKI dataset and there are no tickers like SPY, ^GSPC or other ETFs tracking the S&P
freddiev4
on 18 May 2017
Here is my patch:
https://github.com/zipline-live/zipline/blob/ph_live/zipline/data/benchmarks.py
It doesn't require any new libraries or api-keys. This uses the SPY data from Google Finance.
It is using the Close of SPY which does not include dividends. For some reason Google Finance does not return Adj. Close.
This is now failing 4 tests from tests.test_examples.ExamplesTests:
test_example_4_olmar
test_example_3_dual_ema_talib
test_example_1_momentum_pipeline
test_example_0_buyapple
This is because the benchmark is slightly different.
pbharrin
on 18 May 2017
Thank you, pbharrin! Is https://github.com/zipline-live/ different from https://github.com/quantopian/zipline? One is maintained by the community and the other one by the institution? The push histories are quite different.
ywang412
on 18 May 2017
Yes https://github.com/zipline-live/ is a project we are working on to allow you to live-trade from Zipline. (Like Quantopian, but you can trade any broker you want, any security you want, provided someone writes the code to interact with that broker.) People have expressed interest in using Zipline-live for exchanges in other countries and even Bitcoin.
We forked from Zipline recently so the code base is very similar. If the guys at Quantopian want to use the fix I have proposed then I can make a formal pull request from a Zipline branch. Otherwise you can just overwrite the original benchmarks.py with my benchmarks.py in Zipline and it will work until they figure out a long-term solution.
pbharrin
on 18 May 2017
pbarrin, thx for google SPY patch, I had to add
import pandas_datareader.data as web
got a benchmark warning, but the test Buy_AAPL script seemed to run ok
Downloading benchmark data for '^GSPC'.
[2017-05-18 21:44:29.118015] WARNING: Loader: Still don't have expected data after redownload!
[2017-05-18 21:44:50.681792] INFO: Performance: Simulated 334 trading days out of 334.
[2017-05-18 21:44:50.681929] INFO: Performance: first open: 2016-01-04 14:31:00+00:00
[2017-05-18 21:44:50.682005] INFO: Performance: last close: 2017-05-01 20:00:00+00:00
AAPL algo_volatility algorithm_period_return \
edboyle01
on 18 May 2017
My suspicion is that after Verizon acquired Yahoo (a SV based company), they're not so inclined to give the data for free. Google will likely provide their data for free.
quocble
on 19 May 2017
@pbharrin : Thanks for the patch, Peter!
tibkiss
on 19 May 2017
The patch works! Thanks!
clizarralde
on 21 May 2017
Just wanted to add a note that Google Finance seems to be missing some prices dating back to 2009 (8/11/2009 and 2/2/2012) so with this patch longer term backtests may still crash.
GarrettHoffman
on 23 May 2017
Until the Google-download fix is merged, can we have some option to skip benchmarking so that at least the algorithm is executed?
kennell
on 28 May 2017
I would like to second @kennell's request for an option to skip benchmarking. Zipline is useless to me right now.
rosstripi
on 31 May 2017
Hi everyone. There currently is no option to skip benchmarking without making code changes to zipline. I think that trying to introduce the option to disable benchmarking would take just as much time as trying to swap out yahoo-related code with google; considering we've already made progress on the swap I'd like to suggest using @pbharrin's patch (linked above) until we've merged this fix, as it seems like that's working for people currently.
Once we've merged the yahoo-google swap, we will start working on a way to turn off benchmarking. We're rethinking how we deal with benchmarking in general, so as to avoid an issue like this in the future, such as if google were to discontinue their historical data API.
freddiev4
on 31 May 2017
Instead of spending the time to rewrite the rest of the code to remove or make the benchmark optional, I would suggest at the "benchmarks.py" piece to have a choice to either a. import a benchmark csv manually (very easy to obtain) OR b. automatic update (via quandl or some other provider) would be a better and faster solution.
yiorgosn
on 5 Jun 2017
I've just merged #1812 which swaps out Yahoo with Google
freddiev4
on 5 Jun 2017
Is anyone still having issues running backtests when running the latest zipline master (that are related to benchmarks/Yahoo)?
freddiev4
on 7 Jun 2017
I was getting this on latest master
11:40:28 worker.1 | (key, self.obj._get_axis_name(axis)))
11:40:28 worker.1 | [2017-06-07 11:40:28,115: WARNING/PoolWorker-1] KeyError: 'the label [2017-06-06 00:00:00+00:00] is not in the [index]'
11:40:28 worker.1 | [2017-06-07 11:40:28,115: WARNING/PoolWorker-1] During handling of the above exception, another exception occurred:
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] Traceback (most recent call last):
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] File "/Users/quoc/Projects/alphaoracle/alpha_tasks/tasks/gen_factors.py", line 121, in run
11:40:28 worker.1 | run_algorithm(start=pd.Timestamp(start, tz="UTC"), end=pd.Timestamp(end, tz="UTC"), capital_base=10000, initialize=make_initialize(f['name'], f['hash'], func, args), before_trading_start=before_trading_start, bundle=zipline_db)
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/utils/run_algo.py", line 360, in run_algorithm
11:40:28 worker.1 | environ=environ,
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/utils/run_algo.py", line 179, in _run
11:40:28 worker.1 | overwrite_sim_params=False,
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/algorithm.py", line 700, in run
11:40:28 worker.1 | for perf in self.get_generator():
11:40:28 worker.1 | [2017-06-07 11:40:28,116: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 230, in transform
11:40:28 worker.1 | handle_benchmark(normalize_date(dt))
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 190, in handle_benchmark
11:40:28 worker.1 | benchmark_source.get_value(date)
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/sources/benchmark_source.py", line 73, in get_value
11:40:28 worker.1 | return self._precalculated_series.loc[dt]
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1311, in __getitem__
11:40:28 worker.1 | return self._getitem_axis(key, axis=0)
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1481, in _getitem_axis
11:40:28 worker.1 | self._has_valid_type(key, axis)
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1418, in _has_valid_type
11:40:28 worker.1 | error()
11:40:28 worker.1 | [2017-06-07 11:40:28,117: WARNING/PoolWorker-1] File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1405, in error
@quocble Could you tell me what command/code you ran, that resulted in that error?
freddiev4
on 7 Jun 2017
My code basically runs one pipeline eg RSI, and I use it to collect data on the factor. The error I encountered was in SESSION_END, so I suspect, it's not related to my code itself but something related to the last tick.
trade_simulation.py (228)
elif action == SESSION_END:
# End of the session.
if emission_rate == 'daily':
handle_benchmark(normalize_date(dt))
execute_order_cancellation_policy()
yield self._get_daily_message(dt, algo, algo.perf_tracker)
zipline_db = os.environ.get("ZIPLINE_DB", 'quantopian-quandl')
run_algorithm(start=pd.Timestamp(start, tz="UTC"), end=pd.Timestamp(end, tz="UTC"), capital_base=10000, initialize=make_initialize(f['name'], f['hash'], func, args), before_trading_start=before_trading_start, bundle=zipline_db)
def initialize(context):
set_benchmark(None)
context.algo_name = algo_name
context.hash = hash
attach_pipeline(make_pipeline(hash, f.factor(*args)), 'my_pipeline')
def make_pipeline(hash, func):
# dollar_volume = AverageDollarVolume(window_length=10).top(1000)
logger.info("make pipeline " + hash)
return Pipeline(
columns={
hash: func,
},
# screen=dollar_volume
)
def before_trading_start(context, data):
try:
context.pipeline_data = pipeline_output('my_pipeline')
except:
print ("Err occur in pipeline on %s" % context.get_datetime())
return
@quocble when you run your code, what is your end_date?
freddiev4
on 8 Jun 2017
The patch works, thanks!!!
mellertson
on 10 Jun 2017
@pbharrin @FreddieV4 @mellertson
The following questions may sound stupid, but please bear with me.
(1a) If I change the benchmarks.py file with your google function, do I have to recompile zipline or something? Because I see there is a benchmarks.pyc file too and I am thinking zipline use the pyc?
(1b) If I have to recompile (or do anything else) could you please explain or give a link to that?
(2) Could I past the get_benchmark_returns function in my python script of the algo (instead of changing the benchmarks.py) and get it to work?
thanks.
reMAJ
on 10 Jun 2017
If you just delete the benchmarks.pyc file, Python will automatically re-compile the *.pyc file.
And if you paste the get_benchmark_return function into your algo, it won't work correctly. You need to copy the file into the zipline/data folder of the zipline package. The zipline package is installed into one of multiple directories, based on your operating system and the way you installed zipline.
For myself, I'm running Linux and used the following command to install zipline: sudo pip2 install zipline, so it was installed into /usr/local/lib/python2.7/dist-packages/zipline. I copied the benchmarks.py file into the directory /usr/local/lib/python2.7/dist-packages/zipline/data. You'll need to figure out which directory to copy the benchmarks.py file into. I've included two commands you can run, depending on your operating system, to find where the benchmarks.py file is on your system.
Windows:
cd c:\
dir /s benchmark.py
Linux or Mac OS X:
cd /
sudo find / -name benchmarks.py
The latest merged benchmarks.py doesn't have these two lines in it.
if symbol == "^GSPC":
symbol = "spy"
The two-line snippet was in pbharrin's patch.
Without the snippet, the code will give "unable to read url error".
_utils.RemoteDataError: Unable to read URL: http://www.google.com/finance/historical?q=%5EGSPC&startdate=Dec+29%2C+1989&enddate=Jun+07%2C+2017&output=csv
ywang412
on 11 Jun 2017
@FreddieV4
Here's what you asked for. This time I re-ran the buy_apple example.. I ran this on 6/11.
(lexikon) quoc@MacBook-Pro-5:~/Projects/alphaoracle/alpha_tasks$ zipline run -f ../zipline/zipline/examples/buyapple.py -s 2017-01-01 -e 2017-06-09
[2017-06-12 00:19:44.974160] INFO: Loader: Cache at /Users/quoc/.zipline/data/SPY_benchmark.csv does not have data from 1990-01-02 00:00:00+00:00 to 2017-06-08 00:00:00+00:00.
[2017-06-12 00:19:44.974471] INFO: Loader: Downloading benchmark data for 'SPY' from 1989-12-29 00:00:00+00:00 to 2017-06-08 00:00:00+00:00
[2017-06-12 00:19:46.478470] WARNING: Loader: Still don't have expected data after redownload!
[2017-06-12 00:19:46.478771] INFO: Loader: Cache at /Users/quoc/.zipline/data/treasury_curves.csv does not have data from 1990-01-02 00:00:00+00:00 to 2017-06-08 00:00:00+00:00.
[2017-06-12 00:19:46.478927] INFO: Loader: Downloading treasury data for 'SPY'.
Traceback (most recent call last):
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1395, in _has_valid_type
error()
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1390, in error
(key, self.obj._get_axis_name(axis)))
KeyError: 'the label [2017-06-09 00:00:00+00:00] is not in the [index]'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/quoc/anaconda/envs/lexikon/bin/zipline", line 11, in <module>
load_entry_point('zipline==1.1.0+201.g0840ea1', 'console_scripts', 'zipline')()
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/__main__.py", line 97, in _
return f(*args, **kwargs)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/__main__.py", line 240, in run
environ=os.environ,
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/utils/run_algo.py", line 179, in _run
overwrite_sim_params=False,
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/algorithm.py", line 709, in run
for perf in self.get_generator():
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 230, in transform
handle_benchmark(normalize_date(dt))
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/gens/tradesimulation.py", line 190, in handle_benchmark
benchmark_source.get_value(date)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/zipline/sources/benchmark_source.py", line 75, in get_value
return self._precalculated_series.loc[dt]
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1296, in __getitem__
return self._getitem_axis(key, axis=0)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1466, in _getitem_axis
self._has_valid_type(key, axis)
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1403, in _has_valid_type
error()
File "/Users/quoc/anaconda/envs/lexikon/lib/python3.6/site-packages/pandas/core/indexing.py", line 1390, in error
(key, self.obj._get_axis_name(axis)))
KeyError: 'the label [2017-06-09 00:00:00+00:00] is not in the [index]'
@ywang412 the default symbol that is passed in is SPY in the latest merge. If you've pip installed the latest master branch of zipline, or conda installed a recent build of zipline, then the requests should be going out to Google Finance now.
In the URL you posted, GSPC is the symbol being searched for. Did you clone zipline and then pip install that local directory?
freddiev4
on 13 Jun 2017
@quocble I believe that error is being raised because of this line in particular. Adding a better error message before it's caught in pandas might be a good solution for that.
freddiev4
on 13 Jun 2017
Benchmark missing yesterday's or T-1 data, a quick band-aid solution. Open file ....\zipline\gens\tradesimulation.py replace in handle_benchmark() line 189 with this:
[new line] algo.perf_tracker.all_benchmark_returns[date] = 1
[replacing line:] algo.perf_tracker.all_benchmark_returns[date] = benchmark_source.get_value(date)
I found this solution here: https://stackoverflow.com/questions/41079890/zipline-bundle-yesterdays-data
yiorgosn
on 14 Jun 2017
@FreddieV4 I see that Pyfolio is still calling the Yahoo data from pandas_datareader, any idea when this will be updated?
pbharrin
on 14 Jun 2017
Hey Peter @pbharrin, it looks like https://github.com/quantopian/pyfolio/pull/386 is currently open so if the author continues working on that then hopefully that'll get merged. If the author doesn't have the time to continue work on it, I can open a PR/build on theirs.
freddiev4
on 14 Jun 2017
Hi there, I just installed zipline using the latest github repo, and while trying to run the buyapple.py example, I am running into a similar issue as @quocble but complaining at the beginning of the dataset instead of at the end.
Does anyone have any suggestions for me?
$ zipline run -f algo.py --start 2000-1-1 --end 2014-1-1
[2017-06-19 18:11:48.740235] WARNING: Loader: Refusing to download new benchmark data because a download succeeded at 2017-06-19 17:13:54+00:00.
Traceback (most recent call last):
File "/env-p2/bin/zipline", line 9, in
load_entry_point('zipline==1.1.0+203.g7dea3889', 'console_scripts', 'zipline')()
File "/env-p2/lib/python2.7/site-packages/click/core.py", line 722, in __call__
return self.main(args, *kwargs)
File "/env-p2/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/env-p2/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/env-p2/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "/env-p2/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(args, *kwargs)
File "/env-p2/lib/python2.7/site-packages/zipline/__main__.py", line 97, in _
return f(args, *kwargs)
File "/env-p2/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, *kwargs)
File "/env-p2/lib/python2.7/site-packages/zipline/__main__.py", line 240, in run
environ=os.environ,
File "/env-p2/lib/python2.7/site-packages/zipline/utils/run_algo.py", line 179, in _run
overwrite_sim_params=False,
File "/env-p2/lib/python2.7/site-packages/zipline/algorithm.py", line 709, in run
for perf in self.get_generator():
File "/env-p2/lib/python2.7/site-packages/zipline/gens/tradesimulation.py", line 230, in transform
handle_benchmark(normalize_date(dt))
File "/env-p2/lib/python2.7/site-packages/zipline/gens/tradesimulation.py", line 190, in handle_benchmark
benchmark_source.get_value(date)
File "/env-p2/lib/python2.7/site-packages/zipline/sources/benchmark_source.py", line 75, in get_value
return self._precalculated_series.loc[dt]
File "/env-p2/lib/python2.7/site-packages/pandas/core/indexing.py", line 1296, in __getitem__
return self._getitem_axis(key, axis=0)
File "/env-p2/lib/python2.7/site-packages/pandas/core/indexing.py", line 1466, in _getitem_axis
self._has_valid_type(key, axis)
File "/env-p2/lib/python2.7/site-packages/pandas/core/indexing.py", line 1403, in _has_valid_type
error()
File "/env-p2/lib/python2.7/site-packages/pandas/core/indexing.py", line 1390, in error
(key, self.obj._get_axis_name(axis)))
KeyError: 'the label [2000-01-03 00:00:00+00:00] is not in the [index]'
Environment: Mac OS X 10.12.5, Python 2.7.10
lacabra
on 19 Jun 2017
Hi @lacabra the reason for that is because we only have benchmark data from Google going back to 2001, so running a backtest before that causes that error.
EDIT: I misspoke on that. I'll need to re-investigate as I remember seeing that error before as well. There's a different answer other than the one I just stated above. What happens if you change your start date to 2002-02-01?
freddiev4
on 19 Jun 2017
@FreddieV4 you're absolutely right, setting a later start date did indeed fix the error, thank you very much!
lacabra
on 19 Jun 2017
How did you guys get the Google data? Edit extensions.py?
lionelyoung
on 21 Jun 2017
You can use pandas_datareader.data, here's the docs. They have a pretty
good example.
http://pandas-datareader.readthedocs.io/en/latest/remote_data.html
Best regards,
Mike Ellertson
On Wed, Jun 21, 2017 at 7:37 AM, Lionel Young notifications@github.com
wrote:
How did you guys get the Google data? Edit extensions.py?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/quantopian/zipline/issues/1776#issuecomment-310080747,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHu1f8MiQxFvX8G-BS28zVXBNi7qTR9ks5sGRy2gaJpZM4NNMzb
.
--
Best regards,
Mike Ellertson
Email: [email protected]
mellertson
on 21 Jun 2017
Solution for me using YAHOO... quick and dirty
install pandas_datareader
install fix_yahoo_finance from here: https://pypi.python.org/pypi/fix-yahoo-finance
patch Benchmarks.py with:
import pandas as pd
from six.moves.urllib_parse import urlencode
import pandas_datareader as pdr #NEW
import fix_yahoo_finance as yf #NEW
yf.pdr_override()#NEW
def get_benchmark_returns(symbol, start_date, end_date):
print('NEW')
df = pdr.data.get_data_yahoo(symbol, start=start_date, end=end_date)
df.to_csv('{}_D1.csv'.format(symbol))
return pd.read_csv('{}_D1.csv'.format(symbol),
parse_dates=['Date'],
index_col='Date',
usecols=["Adj Close", "Date"],
squeeze=True, # squeeze tells pandas to make this a Series
# instead of a 1-column DataFrame
).sort_index().tz_localize('UTC').pct_change(1).iloc[1:]
my Setup: Winx64 / Anaconda / Python 3.5
alabaster (0.7.10)
alembic (0.9.2)
astroid (1.4.9)
audioread (2.1.5)
Babel (2.4.0)
bcolz (0.12.1)
beautifulsoup4 (4.6.0)
bleach (1.5.0)
boto (2.47.0)
Bottleneck (1.2.1)
bs4 (0.0.1)
bz2file (0.98)
certifi (2017.4.17)
chardet (3.0.4)
click (6.7)
colorama (0.3.9)
contextlib2 (0.5.5)
cycler (0.10.0)
cyordereddict (1.0.0)
Cython (0.25.2)
decorator (4.0.11)
docutils (0.13.1)
empyrical (0.2.2)
entrypoints (0.2.2)
fix-yahoo-finance (0.0.18)
gensim (2.1.0)
h5py (2.7.0)
html5lib (0.999)
idna (2.5)
imagesize (0.7.1)
intervaltree (2.1.0)
ipykernel (4.6.1)
ipython (6.1.0)
ipython-genutils (0.2.0)
ipywidgets (6.0.0)
isort (4.2.14)
jedi (0.10.2)
Jinja2 (2.9.6)
joblib (0.11)
jsonschema (2.6.0)
jupyter-client (5.0.1)
jupyter-core (4.3.0)
Keras (2.0.5)
lazy-object-proxy (1.2.2)
librosa (0.5.1)
Logbook (1.0.0)
lru-dict (1.1.6)
Mako (1.0.6)
MarkupSafe (1.0)
matplotlib (2.0.2)
mistune (0.7.4)
multipledispatch (0.4.9)
multitasking (0.0.4)
nbconvert (5.2.1)
nbformat (4.3.0)
networkx (1.11)
nltk (3.2.4)
notebook (5.0.0)
numexpr (2.6.2)
numpy (1.13.0+mkl)
numpydoc (0.6.0)
olefile (0.44)
pandas (0.18.1)
pandas-datareader (0.4.0)
pandocfilters (1.4.1)
path.py (10.3.1)
patsy (0.4.1)
pep8 (1.7.0)
pickleshare (0.7.4)
Pillow (4.1.1)
pip (9.0.1)
prompt-toolkit (1.0.14)
protobuf (3.3.0)
psutil (5.2.2)
pyflakes (1.5.0)
Pygments (2.2.0)
pylint (1.6.4)
pyparsing (2.1.4)
PyQt4 (4.11.4)
python-dateutil (2.6.0)
python-editor (1.0.3)
pytz (2017.2)
PyYAML (3.12)
pyzmq (16.0.2)
QtAwesome (0.4.4)
qtconsole (4.3.0)
QtPy (1.2.1)
requests (2.18.1)
requests-file (1.4.2)
requests-ftp (0.3.1)
resampy (0.1.5)
rope-py3k (0.9.4.post1)
scikit-learn (0.18.1)
scipy (0.19.1)
seaborn (0.7.1)
setuptools (36.0.1)
simplegeneric (0.8.1)
six (1.10.0)
smart-open (1.5.3)
snowballstemmer (1.2.1)
sortedcontainers (1.5.7)
sparsesvd (0.2.2)
sphinx (1.6.2)
sphinxcontrib-websupport (1.0.1)
spyder (3.1.4)
SQLAlchemy (1.1.11)
statsmodels (0.8.0)
tables (3.4.2)
tensorflow-gpu (1.1.0)
testpath (0.3)
tflearn (0.3.1)
Theano (0.9.0)
toolz (0.8.2)
tornado (4.5.1)
traitlets (4.3.2)
urllib3 (1.21.1)
wcwidth (0.1.7)
Werkzeug (0.12.2)
wheel (0.29.0)
widgetsnbextension (2.0.0)
win-unicode-console (0.5)
wrapt (1.10.10)
zipline (1.1.0)
edmunch
on 29 Jun 2017
@edmunch great! Now it works!
Before I tried your approach, I had the following error, just in case anybody has the same issue:
Downloading benchmark data for '^GSPC'.
ERROR: Loader: failed to cache the new benchmark returns
Traceback....
.....
python36\lib\urllib\request.py, line 1320, in do_open raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>
saitam1
on 1 Jul 2017
Hi there. This should be fixed in the latest release :). Going to close this. Feel free to reopen if you see this issue again
freddiev4
on 8 Jul 2017
Hello all,
I appear to be encountering the 'the label [timestamp] is not in the [index]' error:
hello world
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-1-dc544426e571> in <module>()
19 initialize=initialize,
20 handle_data=handle_data,
---> 21 bundle='quantopian-quandl',
22 )
/Library/Python/2.7/site-packages/zipline/utils/run_algo.pyc in run_algorithm(start, end, initialize, capital_base, handle_data, before_trading_start, analyze, data_frequency, data, bundle, bundle_timestamp, default_extension, extensions, strict_extensions, environ)
358 print_algo=False,
359 local_namespace=False,
--> 360 environ=environ,
361 )
/Library/Python/2.7/site-packages/zipline/utils/run_algo.pyc in _run(handle_data, initialize, before_trading_start, analyze, algofile, algotext, defines, data_frequency, capital_base, data, bundle, bundle_timestamp, start, end, output, print_algo, local_namespace, environ)
177 ).run(
178 data,
--> 179 overwrite_sim_params=False,
180 )
181
/Library/Python/2.7/site-packages/zipline/algorithm.pyc in run(self, data, overwrite_sim_params)
707 try:
708 perfs = []
--> 709 for perf in self.get_generator():
710 perfs.append(perf)
711
/Library/Python/2.7/site-packages/zipline/gens/tradesimulation.pyc in transform(self)
228 # End of the session.
229 if emission_rate == 'daily':
--> 230 handle_benchmark(normalize_date(dt))
231 execute_order_cancellation_policy()
232
/Library/Python/2.7/site-packages/zipline/gens/tradesimulation.pyc in handle_benchmark(date, benchmark_source)
188 def handle_benchmark(date, benchmark_source=self.benchmark_source):
189 algo.perf_tracker.all_benchmark_returns[date] = \
--> 190 benchmark_source.get_value(date)
191
192 def on_exit():
/Library/Python/2.7/site-packages/zipline/sources/benchmark_source.pyc in get_value(self, dt)
73
74 def get_value(self, dt):
---> 75 return self._precalculated_series.loc[dt]
76
77 def get_range(self, start_dt, end_dt):
/Library/Python/2.7/site-packages/pandas/core/indexing.pyc in __getitem__(self, key)
1294 return self._getitem_tuple(key)
1295 else:
-> 1296 return self._getitem_axis(key, axis=0)
1297
1298 def _getitem_axis(self, key, axis=0):
/Library/Python/2.7/site-packages/pandas/core/indexing.pyc in _getitem_axis(self, key, axis)
1464
1465 # fall thru to straight lookup
-> 1466 self._has_valid_type(key, axis)
1467 return self._get_label(key, axis=axis)
1468
/Library/Python/2.7/site-packages/pandas/core/indexing.pyc in _has_valid_type(self, key, axis)
1401 raise
1402 except:
-> 1403 error()
1404
1405 return True
/Library/Python/2.7/site-packages/pandas/core/indexing.pyc in error()
1388 "key")
1389 raise KeyError("the label [%s] is not in the [%s]" %
-> 1390 (key, self.obj._get_axis_name(axis)))
1391
1392 try:
KeyError: 'the label [2013-01-02 00:00:00+00:00] is not in the [index]'
My environment:
OSX 10.10.5, Python 2.7. Outputs of pip list as follows:
Package Version
-------------------------------------- -----------
alembic 0.9.5
altgraph 0.10.2
appnope 0.1.0
backports-abc 0.4
backports.ssl-match-hostname 3.4.0.2
bcolz 0.12.1
bdist-mpkg 0.5.0
bonjour-py 0.3
Bottleneck 1.2.1
certifi 2017.7.27.1
chardet 3.0.4
click 6.7
contextlib2 0.5.5
cyordereddict 1.0.0
Cython 0.26.1
daemonize 2.3.1
decorator 4.1.2
Django 1.8
empyrical 0.3.2
et-xmlfile 1.0.0
functools32 3.2.3.post2
gnureadline 6.3.3
idna 2.6
intervaltree 2.1.0
ipykernel 4.3.1
ipython 4.1.2
ipython-genutils 0.1.0
ipywidgets 4.1.1
jdcal 1.0
Jinja2 2.8
jsonschema 2.5.1
jupyter 1.0.0
jupyter-client 4.2.2
jupyter-console 4.1.1
jupyter-core 4.1.0
Logbook 1.1.0
lru-dict 1.1.6
macholib 1.5.1
Mako 1.0.7
MarkupSafe 1.0
matplotlib 1.3.1
mistune 0.7.2
modulegraph 0.10.4
mpmath 0.19
multipledispatch 0.4.9
mysql-connector-python 2.0.3
nbconvert 4.1.0
nbformat 4.0.1
networkx 1.11
nose 1.3.7
notebook 4.1.0
numexpr 2.6.4
numpy 1.13.1
openpyxl 1.8.6
pandas 0.18.1
pandas-datareader 0.5.0
path.py 8.1.2
patsy 0.4.1
pexpect 4.0.1
pickleshare 0.6
pip 9.0.1
ptyprocess 0.5.1
py2app 0.7.3
pycrypto 2.6.1
Pygments 2.1.3
pyobjc-core 2.5.1
pyobjc-framework-Accounts 2.5.1
pyobjc-framework-AddressBook 2.5.1
pyobjc-framework-AppleScriptKit 2.5.1
pyobjc-framework-AppleScriptObjC 2.5.1
pyobjc-framework-Automator 2.5.1
pyobjc-framework-CFNetwork 2.5.1
pyobjc-framework-Cocoa 2.5.1
pyobjc-framework-Collaboration 2.5.1
pyobjc-framework-CoreData 2.5.1
pyobjc-framework-CoreLocation 2.5.1
pyobjc-framework-CoreText 2.5.1
pyobjc-framework-DictionaryServices 2.5.1
pyobjc-framework-EventKit 2.5.1
pyobjc-framework-ExceptionHandling 2.5.1
pyobjc-framework-FSEvents 2.5.1
pyobjc-framework-InputMethodKit 2.5.1
pyobjc-framework-InstallerPlugins 2.5.1
pyobjc-framework-InstantMessage 2.5.1
pyobjc-framework-LatentSemanticMapping 2.5.1
pyobjc-framework-LaunchServices 2.5.1
pyobjc-framework-Message 2.5.1
pyobjc-framework-OpenDirectory 2.5.1
pyobjc-framework-PreferencePanes 2.5.1
pyobjc-framework-PubSub 2.5.1
pyobjc-framework-QTKit 2.5.1
pyobjc-framework-Quartz 2.5.1
pyobjc-framework-ScreenSaver 2.5.1
pyobjc-framework-ScriptingBridge 2.5.1
pyobjc-framework-SearchKit 2.5.1
pyobjc-framework-ServiceManagement 2.5.1
pyobjc-framework-Social 2.5.1
pyobjc-framework-SyncServices 2.5.1
pyobjc-framework-SystemConfiguration 2.5.1
pyobjc-framework-WebKit 2.5.1
pyOpenSSL 0.13.1
pyparsing 2.0.1
python-dateutil 2.6.1
python-editor 1.0.3
pytz 2017.2
pyzmq 15.2.0
qtconsole 4.2.0
requests 2.18.4
requests-file 1.4.2
requests-ftp 0.3.1
scikit-learn 0.17.1
scipy 0.19.1
setuptools 36.5.0
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.10.0
sortedcontainers 1.5.7
SQLAlchemy 1.1.14
statsmodels 0.8.0
sympy 1.0
tables 3.4.2
terminado 0.6
toolz 0.8.2
tornado 4.3
traitlets 4.2.1
Twisted 13.2.0
urllib3 1.22
websocket-client 0.32.0
ws4py 0.3.4
xattr 0.6.4
xlrd 0.9.4
zipline 1.1.1
zope.interface 4.1.1
The code which I'm trying to execute:
from pandas import Timestamp
from zipline import run_algorithm
from zipline.api import order, symbol
def initialize(context):
print("hello world")
def handle_data(context, data):
order(symbol('AAPL'), 1)
results = run_algorithm(
Timestamp('2013', tz='UTC'),
Timestamp('2015', tz='UTC'),
capital_base=10000,
initialize=initialize,
handle_data=handle_data,
bundle='quantopian-quandl',
)
Any pointers on how to resolve the above would be sincerely appreciated.
Cheers.
rohan-
on 16 Sep 2017
Hello maintainers,
One client and I are having the same "is not in the [index]" error. We are working on Windows. I tried with both Python 3.5 and 2.7.
It worked one time, then it stopped working.
pauljherrera
on 30 Sep 2017
Playing around with Zipline at home. Custom bundle. Same error "is not in the [index]" error.
Windows 10
Python 3.5
Pandas 0.18.0
zipline 1.1.1
jtromans
on 3 Oct 2017
Hi everyone. The reason for this is because Google has limited users to about 251 days worth of data per request, so you can't run backtests over a year. There is a fix currently being worked on for zipline.
There are duplicates of this issue so I'm just going to direct everyone to this issue: https://github.com/quantopian/zipline/issues/1965
freddiev4
on 4 Oct 2017
I fixed it by loading my own data bundle with minutely data but now I'm getting a different error:
sqlalchemy.exc.InvalidRequestError: Could not reflect: requested table(s) not available in Engine(sqlite:///C:\Users\blablabla/.zipline\data\csv\2017-10-03T21;52;47.125141\assets-6.sqlite): (equity_symbol_mappings, asset_router, futures_root_symbols, futures_exchanges, version_info, equity_supplementary_mappings, equities, futures_contracts)
jtromans
on 4 Oct 2017
Related issues
tonyyuandao
·
3Comments
mosfiqur-rahman
·
4Comments
sekuri
·
3Comments
marketneutral
·
3Comments
ghost
·
8Comments
Most helpful comment
Until the Google-download fix is merged, can we have some option to skip benchmarking so that at least the algorithm is executed?