Sktime: Options for ensembling forecasters

[ngupta23]

Is your feature request related to a problem? Please describe.

Currently ensemble forecasting model only supports mean reduction as outlined here. Can we add more options similar to this.

Describe the solution you'd like

Provide options to combine the forecasts using

• [ ] Median
• [ ] Min/Max
• [ ] Weighted (currently defaults to mean which is equal weighting, but user could provide custom weighting for each model)

[ngupta23]

@mloning, I can add this if you are OK with the overall idea.

[mloning]

@ngupta23 yes, sounds good to me! What do you have in mind in terms of design? Adding a keyword argument to the constructor would perhaps be the simplest. Eventually, we could abstract the aggregation algorithm further.

[ngupta23]

I am thinking it would be similar to this with an option for weighted

ensemble_func: {'mean' (default), 'median', 'weighted', 'min', 'max'}
        Function to aggregate `base_learners` predictions
weights: Optional[Dict{<name>: <weight>}]
        None (default) will assume equal weight (i.e. same as mean)

[mloning]

@ngupta23 sorry for the late reply. Sounds good to me, would welcome a PR!

[aiwalter]

@ngupta23 I like your idea! I am asking myself if we then would also be able to tune those parameters of EnsembleForecaster also via ForecastingGridSearchCV? Maybe even tuning together with the other hyperparameters of the models contained in the EnsembleForecaster? This would however increase the number of fits quite a lot.

@mloning what do you think?

[fkiraly]

Remembering that we have already some architecture for weighted forecasting ensembles (see PR #333), I was wondering whether it would make sense to involve @magittan in the discussion to see how we integrate this with the current ensembling sub-module?

I think it would make a lot of sense to come up with a joint/consolidated design for forecasting ensembles that's flexible and composable - given that there already is one for weighted ensembles and on-line ensembles.

[fkiraly]

(might make sense to set up a phone call where @magittan gives a quick walkthrough through architecture of the weighted ensemble module?)

[fkiraly]

PS, @ngupta23: Min/Max is usually not very robust (extremely outlier sensitive), high/low quantiles are usually better

[mloning]

@aiwalter as long as we add the aggregation function to the constructor of the ensemble, tuning should work 🙂

[magittan]

Sure, would be happy to participate in a phone call for what we have now for online-ensembling of forecasters.

[ngupta23]

PS, @ngupta23: Min/Max is usually not very robust (extremely outlier sensitive), high/low quantiles are usually better

Yeah, I personally was more interested in the median (instead of mean) for the same reason. The only reason I added min/max there was due to the compatibility with hcrystalball. I would be OK if you decide not to include min/max.

Adding option for quantiles would be good in lieu of min/max.