Zipline: Can zipline do futures backtest now?

[manzihong]

I looked the source code and see some definition about futures, but I couldn't find any examples how zipline backtest it, can it deal with futures now? If yes, can anyone send me a really simple example so I can learn something since I'm really new to this. Thank you very much.

[ssanderson]

Zipline does support Futures now, though it's still a somewhat experimental API. @jfkirk @StewartDouglas is there a good place to look for resources on this?

[jfkirk]

@manzihong Futures are supported by Zipline now but, as @ssanderson said, the API is still somewhat experimental and we have not yet written full documentation for the new features.

In brief, you can provide pricing information for Futures the same way you would provide it for Equities. You can then define those assets as Futures by creating a TradingEnvironment and providing metadata to the futures_data argument of TradingEnvironment.write_data .

Leaving this issue open and self-assigning until full documentation has been written.

[degiere]

I'm trying to put together a simple futures example. Haven't been able to get the point value multiplier to affect returns and ending portfolio value though:
https://gist.github.com/degiere/e68b95ef2ef03418c20478177bd05f6d#file-zipline-futures-py

Any suggestions for what I'm doing wrong?

[StewartDouglas]

hey @degiere, thanks for flagging this. It looks like there's a bug where, as you said, the multiplier does not correctly feed through to the portfolio value. I'm hoping to dig into this and come up with a fix in the near future.

[degiere]

@StewartDouglas great thanks. I didn't dive in too deep. If I see a quick fix I'll try and commit also if you don't beat me to it.

[degiere]

@StewartDouglas looks like the run method on TradingAlgorithm calls _write_and_map_id_index_to_sids and writes the futures symbol into equities_identifiers, and the sid gets incremented by 1:

metadata = {
    55555: {
        'symbol': 'ES1',
        'root_symbol': 'ES1',
        'multiplier': 50.0,
        'tick_size': 12.5,
        'name': 'E-Mini S&P 500',
        'asset_type': 'future'
        }
}

backtest = TradingAlgorithm(initialize=initialize,
                            handle_data=handle_data,
                            futures_metadata=metadata)

env = backtest.trading_environment
af = env.asset_finder
print af.retrieve_asset(55555)
print af.lookup_future_symbol('ES1')
# Future(55555 [ES1])
# Future(55555 [ES1])

perf = backtest.run(data)

print {sid for source in backtest.sources for sid in source.sids}
# {55556}

env = backtest.trading_environment
af = env.asset_finder
print af.retrieve_asset(55555)
print af.retrieve_asset(55556)

# Future(55555 [ES1])
# Equity(55556 [ES1])

Probably it's just being treated as an equity after this point. If I figure out a workaround I'll post it. Not sure if I have a strong enough grasp of the code yet to fix it as is. Hope this helps.

[osierra]

It seems you can avoid the conversion to equity by passing a DataPortal to TradingAlgorithm.run() instead of a DataFrame or Panel:

perf = backtest.run(data=dp.DataPortal(
        backtest.asset_finder,
        backtest.trading_calendar,
        first_trading_day=future_reader.first_trading_day,
        future_daily_reader=future_reader,
    ))

[freddiev4]

Gonna x-ref this issue: https://github.com/quantopian/zipline/issues/603 and close this one 🙂

[moonlight16]

@osierra how do you build a 'future_reader'

[osierra]

I used an equity pricer along the following lines (where ta is an instance of zipline.TradingAlgorithm).

 import pandas as pd
 import zipline.data.us_equity_pricing as ep

 data = pd.Panel({0: time_series})

 future_reader = ep.PanelBarReader(
         ta.trading_calendar,
         data,
         ta.sim_params.data_frequency
     )

[moonlight16]

@osierra Thanks. Would you mind posting all of your code to show as an example. It seems there's been some changes to the interfaces since this thread was first started, and I'm still hitting other errors.

[prediction-labs]

Tried the code above and I'm getting this error. Any idea ?
/Users/danielbara/.pyenv/versions/sucden/lib/python2.7/site-packages/zipline/assets/asset_writer.pyc in _load_data(self, equities, futures, exchanges, root_symbols, equity_supplementary_mappings)
706 (exchanges, 'exchange'),
707 (root_symbols, 'root_symbol')]:
--> 708 if id_col in df.columns:
709 df.set_index(id_col, inplace=True)
710

AttributeError: 'dict' object has no attribute 'columns'