Autogluon: Use AutoGluon in Kaggle Kernels

[mli]

A majority of recent Kaggle competitions require to submit via notebooks.

We could use !pip install autogluon, but got the a bunch of errors, even just importing the tabular task. For example

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-17-6f7d1b4fed2f> in <module>
----> 1 from autogluon import TabularPrediction as task

/opt/conda/lib/python3.7/site-packages/autogluon/__init__.py in <module>
     15 logger.setLevel(logging.ERROR)
     16 
---> 17 from .utils.try_import import *
     18 try_import_mxnet()
     19 

/opt/conda/lib/python3.7/site-packages/autogluon/utils/__init__.py in <module>
     18 from .util_decorator import classproperty
     19 from .custom_process import CustomProcess
---> 20 from .openml_download import *
     21 

/opt/conda/lib/python3.7/site-packages/autogluon/utils/openml_download.py in <module>
      1 import os
      2 import numpy as np
----> 3 from sklearn.impute import SimpleImputer
      4 from sklearn.model_selection import train_test_split
      5 

/opt/conda/lib/python3.7/site-packages/sklearn/impute/__init__.py in <module>
      2 
      3 from ._base import MissingIndicator, SimpleImputer
----> 4 from ._knn import KNNImputer
      5 
      6 __all__ = [

/opt/conda/lib/python3.7/site-packages/sklearn/impute/_knn.py in <module>
      7 from ._base import _BaseImputer
      8 from ..utils.validation import FLOAT_DTYPES
----> 9 from ..metrics import pairwise_distances_chunked
     10 from ..metrics.pairwise import _NAN_METRICS
     11 from ..neighbors._base import _get_weights

/opt/conda/lib/python3.7/site-packages/sklearn/metrics/__init__.py in <module>
      5 
      6 
----> 7 from ._ranking import auc
      8 from ._ranking import average_precision_score
      9 from ._ranking import coverage_error

/opt/conda/lib/python3.7/site-packages/sklearn/metrics/_ranking.py in <module>
     33 from ..utils.sparsefuncs import count_nonzero
     34 from ..exceptions import UndefinedMetricWarning
---> 35 from ..preprocessing import label_binarize
     36 from ..preprocessing._label import _encode
     37 

/opt/conda/lib/python3.7/site-packages/sklearn/preprocessing/__init__.py in <module>
      6 from ._function_transformer import FunctionTransformer
      7 
----> 8 from ._data import Binarizer
      9 from ._data import KernelCenterer
     10 from ._data import MinMaxScaler

/opt/conda/lib/python3.7/site-packages/sklearn/preprocessing/_data.py in <module>
     34 from ._csr_polynomial_expansion import _csr_polynomial_expansion
     35 
---> 36 from ._encoders import OneHotEncoder
     37 
     38 BOUNDS_THRESHOLD = 1e-7

/opt/conda/lib/python3.7/site-packages/sklearn/preprocessing/_encoders.py in <module>
      8 from ..base import BaseEstimator, TransformerMixin
      9 from ..utils import check_array
---> 10 from ..utils.fixes import _argmax
     11 from ..utils.validation import check_is_fitted
     12 

ImportError: cannot import name '_argmax' from 'sklearn.utils.fixes' (/opt/conda/lib/python3.7/site-packages/sklearn/utils/fixes.py)

Several competitions don't allow use Internet, which means we cannot pip install. Instead, we should submit a PR to add autogluon at https://github.com/Kaggle/docker-python/blob/master/Dockerfile#L129 and

[Innixma]

Regarding the notebook error, I don't think this is caused by an issue in AutoGluon. Have you tried restarting the kernel after pip installing to ensure the correct versions are loaded? Additionally, can you do a pip freeze after the pip install + kernel restart to ensure the proper sklearn version is being used?

Regarding adding AutoGluon to the Kaggle Dockerfile, we should at least wait until modularization occurs, and I feel that there will be a large amount of work to get it compatible with the Docker, as we don't necessarily support the latest versions of all the packages we use, and we may have dependency version conflicts with some of the many other packages in this Dockerfile build, which is a bit concerning.

[jwmueller]

@mli
I was able to use kaggle kernels with multiple older versions of autogluon, c.f. https://www.kaggle.com/jonaswm/covid-ag
so maybe seemingly-incorrect version of dependency can be corrected via: !pip install scikit-learn<=0.22
as a temporary fix?

Longer term it seems like we should consider removing from .openml_download import *.

@tlienart Is it ok to remove that statement to prevent this error from happening when users import autogluon if they have improper sklearn version installed? AFAIK the functions in openml_download are solely used in your unit-test/tutorial, so they don't need to be imported when autogluon is imported?

[tlienart]

sounds good

[mseeger]

@jwmueller So then Thibaut's tutorial would import load_and_split_openml_data how? By using the absolute path?

Is the problem that nothing in utils must depend on sklearn? Seems to make sense. In the beginning, I had this load_and_split_openml_data somewhere next to the bayesopt searcher code.

We could also move it back into the tutorial if you like, since the unit test which calls it is deactivated anyway (it never really worked, but not because of openml).

[jwmueller]

@mseeger
I agree if the unit-test is not being used it would be nice to move this function to tutorials/ instead. But I didn't mean that nothing in utils/ can depend on sklearn, this is unavoidable as utils/tabular/ uses sklearn all over. I suspect the error here is due to incompatible sklearn version present, which is certainly not the fault of the openml_downloads functions.

What I do think though is "optional" imports from sklearn should be limited to narrow scope (ie. those that aren't core autogluon functionality, eg. just for fetching the data in your tutorial). I'd say nearly half the issues I encounter when trying to use AutoGluon-Tabular with other python packages are due to incompatible sklearn-version dependencies. So it's always pleasant if AutoGluon can mostly work with more versions of sklearn beyond what is officially listed in setup.py (ie. doesn't break upon autogluon-import but maybe only breaks at the moment where a sklearn-version-specific method is called in autogluon)

[salil-gtm]

Can someone suggest a temporary fix to run AutoGluon in Kaggle Kernels ? Tried reverting to older versions as mentioned in the kernel link shared by @jwmueller
!pip install mxnet==1.6.0 autogluon==0.0.6
Still getting :
ImportError: cannot import name 'isStringType' from 'PIL._util' (/opt/conda/lib/python3.7/site-packages/PIL/_util.py)