Sktime: Cannot access feature_importances_ attribute for RISE estimator

[KoolSkotcheCoding]

Describe the bug

Cannot access feature_importances_ attribute for RISE estimator without error.

To Reproduce

from sklearn.model_selection import train_test_split
from sktime.datasets import load_arrow_head
from sktime.classification.frequency_based import RandomIntervalSpectralForest

X, y = load_arrow_head(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)

rise = RandomIntervalSpectralForest(n_estimators=10)
rise.fit(X_train, y_train)
rise.score(X_test, y_test)
rise.feature_importances_

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-26-4a0a11fa4729> in <module>
      5 rise.score(X_test, y_test)
      6 
----> 7 rise.feature_importances_

~/.local/lib/python3.6/site-packages/sklearn/ensemble/_forest.py in feature_importances_(self)
    453 
    454         all_importances = np.mean(all_importances,
--> 455                                   axis=0, dtype=np.float64)
    456         return all_importances / np.sum(all_importances)
    457 

<__array_function__ internals> in mean(*args, **kwargs)

~/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py in mean(a, axis, dtype, out, keepdims)
   3371 
   3372     return _methods._mean(a, axis=axis, dtype=dtype,
-> 3373                           out=out, **kwargs)
   3374 
   3375 

~/.local/lib/python3.6/site-packages/numpy/core/_methods.py in _mean(a, axis, dtype, out, keepdims)
    158             is_float16_result = True
    159 
--> 160     ret = umr_sum(arr, axis, dtype, out, keepdims)
    161     if isinstance(ret, mu.ndarray):
    162         ret = um.true_divide(

ValueError: setting an array element with a sequence.

Expected behavior

rise.feature_importances_ should be a pandas dataframe.

Additional context

Versions

[mloning]

Hi @KoolSkotcheCoding, thanks for raising the issue!

We haven't implemented feature importances for RISE yet, there is an experimental feature importance graph method for the TimeSeriesForestClassifier in sktime.classification.compose.

We need to override the method we inherit from sklearn and raise a NotImplementedError - would appreciate a PR!

[xuyxu]

Would it be better if we could compute the feature_importances_, which enables one to find the most discriminative timestamps (if I understand correctly)

A little thought on implementation:
Since RISE uses the decision tree in scikit-learn as the base estimator, which is already equipped with the property feature_importances_, the implementation of feature_importances_ for RISE simply needs to add another function decorated by @property.

Any discussion is welcomed : )

[mloning]

Hi @AaronX121, yes you're right about the property decorator! We have an implementation for time series forest here.

Compared to scikit-learn, the main complication is that we need to keep track of the intervals too when aggregating feature importances. More details in section 4.4 in the original paper.