Gluon-ts: Usage of DateSplitter with max_history

[MaximilianPavon]

Hi,
I am trying to use the DateSplitter together with max_history in order to make the test data shorter and would love to get some help.

The documentation here says the following about max_history:

If given, all entries in the test-set have a max-length of max_history. 
This can be sued to produce smaller file-sizes.

But when I set this value to e.g. 3 weeks of hourly data (max_history = 504) I am getting AssertionErrors from this line, here is the relevant code block:

def split(self, items: List[DataEntry]) -> TrainTestSplit:
    split = TrainTestSplit()

    for item in map(TimeSeriesSlice.from_data_entry, items):

        train = self._train_slice(item)
        test = self._trim_history(self._test_slice(item))

        split._add_train_slice(train)

        assert len(test) - len(train) >= getattr(self, "prediction_length")
        split._add_test_slice(test)

    return split

So the length of the test slice minus the length of the train slice is not greater or equal to the defined prediction length.

When debugging this code line by line, the problem seems to originate from the function _trim_history(), implemented a few lines above:

def _trim_history(self, item: TimeSeriesSlice) -> TimeSeriesSlice:
    if getattr(self, "max_history") is not None:
        return item[: -getattr(self, "max_history")]
    else:
        return item

Currently _trim_history() disregards the last max_history data points from the test slice. Considering that the test slice is exactly prediction_length data points longer than the training slice, this is bound to run into the mentioned AssertionError.

Is this the expected behaviour of this parameter or am I doing something wrong?

Here is some example code to replicate this:

import gluonts
from gluonts.dataset.field_names import FieldName
from gluonts.dataset.split import DateSplitter
import numpy as np
import pandas as pd

print(f"gluonts version: {gluonts.__version__}")  # gluonts version: 0.5.2

prediction_length = 72

start_date = pd.Timestamp("2020-01-01", freq="1H")
split_date = start_date + 30 * pd.Timedelta("1D")
print(f"start_date: {start_date}\nsplit_date: {split_date}")
# start_date: 2020-01-01 00:00:00
# split_date: 2020-01-31 00:00:00

toy_data = [
    {
        FieldName.START: start_date,
        FieldName.TARGET: np.arange(0, 10000),
        "item": "my_item",
        # FieldName.ITEM_ID unfortunately does not work, since the TimeSeriesSlice expects 'item' and not 'item_id':
        # https://github.com/awslabs/gluon-ts/blob/18ab342a3abd7cfb1b6460dacfffd42b64fbefde/src/gluonts/dataset/split/splitter.py#L100
    }
]


for max_history in (None, 1, 2, 10, 20, 30):
    try:
        splitter = DateSplitter(prediction_length=prediction_length, split_date=split_date, max_history=max_history)
        train_validation_split = splitter.split(items=toy_data)
        print(f"\tNo problems when splitting with max_history={max_history}")
    except AssertionError as err:
        print(f"\tWhoopsie, something went wrong when splitting with max_history={max_history}")

#   No problems when splitting with max_history=None
#   Whoopsie, something went wrong when splitting with max_history=1
#   Whoopsie, something went wrong when splitting with max_history=2
#   Whoopsie, something went wrong when splitting with max_history=10
#   Whoopsie, something went wrong when splitting with max_history=20
#   Whoopsie, something went wrong when splitting with max_history=30

[PascalIversen]

Hey @MaximilianPavon,
thank you for the detailed report and sorry for leaving this open for so long. You are using the max_history parameter as intended but there are two bugs:

1. _trim_history needs to return item[-getattr(self, "max_history"):]
   instead of item[:-getattr(self, "max_history")]
2. the assertion needs be guarded:

if getattr(self, "max_history") is None:
    assert len(test) - len(train) >= getattr(self, "prediction_length")

[lostella]

@MaximilianPavon the fix for this was merge to master, and will be included in the next release 0.6.0 (coming soon)

[MaximilianPavon]

Thanks a lot @PascalIversen and @lostella!
We managed to get around this issue by over-writing the problematic functions in a similar way, but still had to ditch the use of the DateSplitter due to https://github.com/awslabs/gluon-ts/issues/997.