Sktime: Forecasting wish list

[mloning]

For the API design proposal, see this wiki entry.

Forecasters

Atomic

• [x] NaiveForecaster (strategies={"last", "mean", "seasonal_last"}), see #227
• [x] PolynomialTrendForecaster (linear, polynomial, locally linear, smoothing splines), see this chapter
• [x] [ThetaForecaster](https://github.com/M4Competition/M4methods/blob/master/4Theta%20method.R), see #198
• [x] ExponentialSmoothing, see #198
• [x] ARIMA, see #412
• [x] AutoARIMA, see #104
• [x] Automatic Exponential Smoothing, see ETS in R, source code, see https://github.com/alan-turing-institute/sktime/pull/350, statsmodels: https://www.statsmodels.org/dev/examples/notebooks/generated/ets.html

Reduction

• [x] ReducedRegressionForecaster, see #218
• [x] ReducedTimeSeriesRegressionForecaster, see #218

Composition

• [x] TransformedTargetForecaster (pipeline with target transformations, multiple steps)
• [x] EnsembleForecaster (see e.g. sklearn's VotingRegressor)
• [x] StackingForecaster
• [ ] ColumnEnsembler, multiple univariate forecasters on multivariate time series (forecasting trend and residuals separately, adding/multiplying results together)
• [ ] FeatureUnion to get multiple transforms from a single time series (univariate to multivariate)
• [ ] Concatenator (or splicer) to get single series from segments of series in a pipelines with prior annotation and splicing according to annotations
• [ ] ensemble over steps in forecasting horizon ("direct strategy") which fits a separate model for each step ahead
• [ ] other ensemble methods (weighted averages, etc)
• [x] Recursive strategy (basically https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.RegressorChain.html#sklearn.multioutput.RegressorChain)
• [x] Direct strategy (basically https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputRegressor.html)
• [ ] DirRec strategy #226
• [ ] support for models that inherently support multi-output (see e.g. https://machinelearningmastery.com/multi-output-regression-models-with-python/)
• [ ] Featurizers

Interfaces

• [x] statsmodels
• [x] pmdarima
• [ ] fbprophet
• [ ] gluonts

Transformers

• [x] Detrender (series to series, meta-estimator that works with any forecaster)
• [x] Deseasonaliser
• [x] FittedParamExtractor for regression/classification
• [x] BoxCoxTransformer (see pmdarima's implementation)
• [x] Wrapper for sklearn's non-fittable transformers, so that we can apply them to single series (e.g. FunctionTransformer or scalers)
• [ ] TimeSeriesDecomposer (e.g. using LOESS seasonal-trend-residual decomposition) returning multivariate series

Model selection

Temporal CV

• [x] SlidingWindowSplitter
• [x] ManualWindowSplitter
• [x] SingleWindowSplitter
• [x] temporal_train_test_split

Tuning

• [x] ForecastingGridSearchCV

Enhancements

• [x] in-sample predictions
• [ ] prediction intervals
• [ ] dynamic (using previously predicted values) vs static predictions
• [ ] exogenous variables, including pipelines, multivariate transformers, etc
• [x] simplify FH class into separate utility functions
• [ ] refactor ForecastingGridSearchCV and update_predict(), try to factor out separate cross-validation function

[big-o]

Due to the ThetaForecaster's dependency on SES, #198 also (re-)implements the ExpSmoothingForecaster in the new framework.

[hweiTPR]

Is there a way to tune both sklearn & sktime hyperparameters at the same time, when using regressors from sklearn? If not. It could be really useful to have it.

[mloning]

Hi @hweiTPR, yes you can do something like this:

regressor_param_grid = {"n_estimators": [100, 200, 300]}
forecaster_param_grid = {"window_length": [10, 12, 14]}
regressor = GridSearchCV(RandomForestRegressor(), param_grid=regressor_param_grid)
forecaster = RecursiveRegressionForecaster(regressor)
cv = SlidingWindowSplitter(initial_window=100)
gscv = ForecastingGridSearchCV(forecaster, cv=cv, param_grid=forecaster_param_grid)
gscv.fit(y_train)

print(gscv.best_params_, gscv.best_forecaster_.regressor_.best_params_)
>>> {'window_length': 12} {'n_estimators': 100}

[hweiTPR]

That looks great! Would be helpful to have this in the documentation!

[mloning]

@hweiTPR I mention it already in the forecasting tutorial below cell 27: https://github.com/alan-turing-institute/sktime/blob/master/examples/01_forecasting.ipynb but without showing the code! Would appreciate a PR if you'd like to add it! 🙂

[huayicodes]

@mloning (switching to my personal account) I totally missed that sentence. Happy to contribute. I have another question: what can I put for "score" when using ForecastingGridSearchCV? I tried smape_loss, but it threw me an error.

[mloning]

This should work:

from sktime.forecasting.model_selection import ForecastingGridSearchCV
from sklearn.ensemble import RandomForestRegressor
from sktime.forecasting.compose import RecursiveRegressionForecaster
from sktime.forecasting.model_selection import SlidingWindowSplitter
from sktime.performance_metrics.forecasting import sMAPE
from sklearn.model_selection import GridSearchCV
from sktime.datasets import load_airline

y_train = load_airline()

regressor_param_grid = {"n_estimators": [100, 200, 300]}
forecaster_param_grid = {"window_length": [10, 12, 14]}
regressor = GridSearchCV(RandomForestRegressor(), param_grid=regressor_param_grid)
forecaster = RecursiveRegressionForecaster(regressor)
cv = SlidingWindowSplitter(initial_window=100)
gscv = ForecastingGridSearchCV(forecaster, cv=cv, param_grid=forecaster_param_grid, scoring=sMAPE())
gscv.fit(y_train)
print(gscv.cv_results_)

This is a deviation from scikit-learn, as we encapsulated loss functions as classes rather than functions, that makes it easier to attach additional information that's needed when running GSCV. This functionality also still needs some improvements to handle multi-step forecasting horizons.

[NikeNano]

Is any one working on an interface to fb prophet? Would be happy to look in to it otherwise!

[mloning]

Hi @NikeNano that would be great! It's probably best to create a separate issue for it and link it here!