Zipline: Trading Calendar API

[SigmaZipline]

Hi,

I'm trying to switch between calendars in zipline when running an algorithm through run_algorithm-method, but can't seem to get that working. I have tried to initiate get_calendar("LSE") in initialize-function, but it does not respond - it reverts to "NYSE" which is the default.

Anyone that knows how I can easily switch between calendars?

I have read the docs and read the few posts concerning trading calendar, but can't seem to get anywhere.

Any help is much appreciated.

[freddiev4]

Hi @I-Larsson so it looks like the issue is that in run_algo.py, trading calendar isn't actually being passed in; get_calendar() only gives you an instance of a specific calendar (it's not the behavior you're expecting).

So, the calendar you want to use needs to be passed in via keyword to run_algo.py where it creates an instance of TradingAlgorithm. The calendar you'd want to use would ideally be passed in via the CLI in __main__.py.

Then in algorithm.py, where it says:

    # If a schedule has been provided, pop it. Otherwise, use NYSE.
    self.trading_calendar = kwargs.pop(
        'trading_calendar',
        get_calendar("NYSE")
    )

The calendar you set would be stored in trading_calendar.

I can get around to fixing this at some point but otherwise if you'd like to open a PR to fix this on your own feel free to do that as well 😃

Info on installing a zipline dev environment can be found in the Development Guidelines

[deltahedgetech]

hi guys,

this is what I've done.

in backtest -
if __name__ == '__main__':

start = makeTS("2017-5-04"); end = makeTS("2017-5-04")  # this can go anywhere before the TradingAlgorithm

# load the bundle
bundle_data = load('quandl_ftse350', os.environ, None)
cal = bundle_data.equity_daily_bar_reader.trading_calendar.all_sessions
pipeline_loader = USEquityPricingLoader(bundle_data.equity_daily_bar_reader, bundle_data.adjustment_reader)
choose_loader = make_choose_loader(pipeline_loader)
env = TradingEnvironment(bm_symbol='^FTSE', exchange_tz='Europe/London',trading_calendar=get_calendar("LSE"),
                         asset_db_path=parse_sqlite_connstr(bundle_data.asset_finder.engine.url))

data = DataPortal(
    env.asset_finder, get_calendar("LSE"),
    first_trading_day=bundle_data.equity_minute_bar_reader.first_trading_day,
    equity_minute_reader=bundle_data.equity_minute_bar_reader,
    equity_daily_reader=bundle_data.equity_daily_bar_reader,
    adjustment_reader=bundle_data.adjustment_reader,
)


# the actual running of the backtest happens in the TradingAlgorithm object
bt_start = time()
perf = TradingAlgorithm(
    env=env,
    get_pipeline_loader=choose_loader,
    trading_calendar=get_calendar("LSE"),
    sim_params=create_simulation_parameters(
        start=start,
        end=end,
        capital_base=CAPITAL_BASE,
        trading_calendar=get_calendar("LSE"),
        data_frequency='daily'
    ),
    **{
        'initialize': initialize,
        'handle_data': handle_data,
        'before_trading_start': None,
        'analyze': None,
    }
).run(data, overwrite_sim_params=False,)
bt_end = time()

also needed to add some libraries to extension.py and tweaked the code in exchange_calendar_lse.py to handle royal events holidays

Timestamp('2002-06-03', tz='UTC'),
Timestamp('2002-06-04', tz='UTC'),
Timestamp('2011-04-29', tz='UTC'),
Timestamp('2012-06-04', tz='UTC'),
Timestamp('2012-06-05', tz='UTC')

]

@property
def adhoc_holidays(self):
return list(chain(
Royals,
))

Thanks Stuart

[freddiev4]

Some work being done on this branch (not finished)

Also this open PR #1800

[SigmaZipline]

FreddieV4: Thank you for the clarification.
deltahedgetech: The algo seems to be responding to the calendar now with that method - thank you.

[ivanlen]

I am trying to implement Bitcoin data in Zipline.
My problems arise from the fact that Bitcoin does not have weekends or holidays in it calendar. So I don't know how to implement a proper calendar or actually no calendar.
Do you have any advise. Thank you!

[freddiev4]

Hi @ivanlen if you want to implement another calendar you could look at exchange_calendar_nyse.py or trading_calendar.py as examples and then modify it from there

[JimmyWuMadchester]

Hi @FreddieV4, I'm trying to get a custom calendar for bitcoin as well. However, I couldn't find out how to include weekends looking at exchange_calendar_nyse.py and trading_calendar.py. Any ideas how to include weekends?
OK I managed to do this by passing a weekmask to CustomBusinessDay() which includes the weekends.

@I-Larsson, I managed to specify the calendar for my custom bundle via ~\.zipline\extension.py .

from zipline.data.bundles import register
from zipline.data.bundles.viacsv import viacsv

coinSym = {
    "BTC",
}

register(
    'csv',              # name this whatever you like
    viacsv(coinSym),    # function
    "COIN",                 # calendar_name, use the custom one crated.
)

Then in the calendar_utils.py, you can make sure the calendar_name is mapped to the custom_calendar.py stored under the folder zipline\utils\calendars\.

In calendar_utils.py

from zipline.utils.calendars.coin_calendar import CoinCalendar

_default_calendar_factories = {
    'NYSE': NYSEExchangeCalendar,
    'CME': CMEExchangeCalendar,
    'ICE': ICEExchangeCalendar,
    'CFE': CFEExchangeCalendar,
    'BMF': BMFExchangeCalendar,
    'LSE': LSEExchangeCalendar,
    'TSX': TSXExchangeCalendar,
    'us_futures': QuantopianUSFuturesCalendar,
    'COIN': CoinCalendar,
}
_default_calendar_aliases = {
    'NASDAQ': 'NYSE',
    'BATS': 'NYSE',
    'CBOT': 'CME',
    'COMEX': 'CME',
    'NYMEX': 'CME',
    'ICEUS': 'ICE',
    'NYFE': 'ICE',
}

Once you have the calendars ready or using the default ones, you can switch it by specifying the exchange metadata in bundle files.
example in quandl.py with line data['exchange'] = 'QUANDL'

metadata_frame : pd.DataFrame
        A dataframe with the following columns:
          symbol: the asset's symbol
          name: the full name of the asset
          start_date: the first date of data for this asset
          end_date: the last date of data for this asset
          auto_close_date: end_date + one day
          exchange: the exchange for the asset; this is always 'quandl'
        The index of the dataframe will be used for symbol->sid mappings but
        otherwise does not have specific meaning.
    """
    raw_iter = _fetch_raw_metadata(api_key, cache, retries, environ)

    def item_show_func(_, _it=iter(count())):
        'Downloading page: %d' % next(_it)

    with maybe_show_progress(raw_iter,
                             show_progress,
                             item_show_func=item_show_func,
                             label='Downloading WIKI metadata: ') as blocks:
        data = pd.concat(blocks, ignore_index=True).rename(columns={
            'dataset_code': 'symbol',
            'name': 'asset_name',
            'oldest_available_date': 'start_date',
            'newest_available_date': 'end_date',
        }).sort_values('symbol')

    data = data[~data.symbol.isin(excluded_symbols)]
    # cut out all the other stuff in the name column
    # we need to escape the paren because it is actually splitting on a regex
    data.asset_name = data.asset_name.str.split(r' \(', 1).str.get(0)
    data['exchange'] = 'QUANDL'
    data['auto_close_date'] = data['end_date'] + pd.Timedelta(days=1)
    return data

[freddiev4]

This issue should be fixed by this PR #1800 🙂

Feel free to re-open if you have other issues