Autogluon: Tutorial example crash

[ZhangSanMo]

Working Env: Windows/Python 3.7.3/Jupyter NoteBook

Runnig Code:

predictor_age = task.fit(train_data=train_data, output_directory="agModels-predictAge", label=age_column)
performance = predictor_age.evaluate(test_data)

Error Info:

Beginning AutoGluon training ...
Preprocessing data ...
Here are the first 10 unique label values in your data:  [25 23 46 55 36 51 33 18 43 41]
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-29-423783e2f952> in <module>
----> 1 predictor_age = task.fit(train_data=train_data, output_directory="agModels-predictAge", label="age")
      2 performance = predictor_age.evaluate(test_data)

C:\ProgramData\Miniconda3\lib\site-packages\autogluon\task\tabular_prediction\tabular_prediction.py in fit(train_data, label, tuning_data, output_directory, problem_type, eval_metric, hyperparameter_tune, feature_prune, holdout_frac, num_bagging_folds, stack_ensemble_levels, hyperparameters, time_limits, num_trials, search_strategy, search_options, nthreads_per_trial, ngpus_per_trial, dist_ip_addrs, visualizer, verbosity, **kwargs)
    288                       hyperparameter_tune=hyperparameter_tune, feature_prune=feature_prune,
    289                       holdout_frac=holdout_frac, num_bagging_folds=num_bagging_folds, stack_ensemble_levels=stack_ensemble_levels,
--> 290                       hyperparameters=hyperparameters, time_limit=time_limits_orig, verbosity=verbosity)
    291         return TabularPredictor(learner=learner)

C:\ProgramData\Miniconda3\lib\site-packages\autogluon\utils\tabular\ml\learner\default_learner.py in fit(self, X, X_test, scheduler_options, hyperparameter_tune, feature_prune, holdout_frac, num_bagging_folds, stack_ensemble_levels, hyperparameters, time_limit, verbosity)
     48         time_preprocessing_start = time.time()
     49         logger.log(20, 'Preprocessing data ...')
---> 50         X, y, X_test, y_test = self.general_data_processing(X, X_test)
     51         time_preprocessing_end = time.time()
     52         time_preprocessing = time_preprocessing_end - time_preprocessing_start

C:\ProgramData\Miniconda3\lib\site-packages\autogluon\utils\tabular\ml\learner\default_learner.py in general_data_processing(self, X, X_test)
    100 
    101         if self.problem_type is None:
--> 102             self.problem_type = self.get_problem_type(X[self.label])
    103 
    104         # Gets labels prior to removal of infrequent classes

C:\ProgramData\Miniconda3\lib\site-packages\autogluon\utils\tabular\ml\learner\abstract_learner.py in get_problem_type(y)
    428                 reason = "dtype of label-column == int and many unique label-values observed"
    429         else:
--> 430             raise NotImplementedError('label dtype', unique_vals.dtype, 'not supported!')
    431         logger.log(25, "AutoGluon infers your prediction problem is: %s  (because %s)" % (problem_type, reason))
    432         logger.log(25, "If this is wrong, please specify `problem_type` argument in fit() instead (You may specify problem_type as one of: ['%s', '%s', '%s'])\n" % (BINARY, MULTICLASS, REGRESSION))

NotImplementedError: ('label dtype', dtype('int64'), 'not supported!')

Ps.
When I set problem_type='regression', it works well.

[Innixma]

Thanks for your issue submission. I'll take a look into this.

It seems you were able to get AutoGluon working on Windows, is that correct? I'd be very interested to know any difficulties you had and the steps you took to get it running.

Best,
Nick

[Innixma]

I've looked into this and I believe it is likely a defect specific to Windows, on Mac and Linux it properly works on int64:

import numpy as np
import pandas as pd

dtypes_to_test = ['int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8', 'float64', 'float32', 'float16', 'object', 'str']

for dtype in dtypes_to_test:
    series = pd.Series(data=[1, 2, 3], dtype=dtype)
    if series.dtype == 'int':
        print('Y %s is recognized as int' % series.dtype)
    else:
        print('N %s is not recognized as int' % series.dtype)
    if np.issubdtype(series.dtype, np.integer):
        print('Y %s is recognized as subtype of np.integer' % series.dtype)
    else:
        print('N %s is not recognized as subtype of np.integer' % series.dtype)
    if series.dtype == 'float':
        print('Y %s is recognized as float' % series.dtype)
    else:
        print('N %s is not recognized as float' % series.dtype)
    if np.issubdtype(series.dtype, np.floating):
        print('Y %s is recognized as subtype of np.floating' % series.dtype)
    else:
        print('N %s is not recognized as subtype of np.floating' % series.dtype)
    if series.dtype == 'object':
        print('Y %s is recognized as subtype of object' % series.dtype)
    else:
        print('N %s is not recognized as object' % series.dtype)

I get:

Y int64 is recognized as int
Y int64 is recognized as subtype of np.integer
N int64 is not recognized as float
N int64 is not recognized as subtype of np.floating
N int64 is not recognized as object
N int32 is not recognized as int
Y int32 is recognized as subtype of np.integer
N int32 is not recognized as float
N int32 is not recognized as subtype of np.floating
N int32 is not recognized as object
N int16 is not recognized as int
Y int16 is recognized as subtype of np.integer
N int16 is not recognized as float
N int16 is not recognized as subtype of np.floating
N int16 is not recognized as object
N int8 is not recognized as int
Y int8 is recognized as subtype of np.integer
N int8 is not recognized as float
N int8 is not recognized as subtype of np.floating
N int8 is not recognized as object
N uint64 is not recognized as int
Y uint64 is recognized as subtype of np.integer
N uint64 is not recognized as float
N uint64 is not recognized as subtype of np.floating
N uint64 is not recognized as object
N uint32 is not recognized as int
Y uint32 is recognized as subtype of np.integer
N uint32 is not recognized as float
N uint32 is not recognized as subtype of np.floating
N uint32 is not recognized as object
N uint16 is not recognized as int
Y uint16 is recognized as subtype of np.integer
N uint16 is not recognized as float
N uint16 is not recognized as subtype of np.floating
N uint16 is not recognized as object
N uint8 is not recognized as int
Y uint8 is recognized as subtype of np.integer
N uint8 is not recognized as float
N uint8 is not recognized as subtype of np.floating
N uint8 is not recognized as object
N float64 is not recognized as int
N float64 is not recognized as subtype of np.integer
Y float64 is recognized as float
Y float64 is recognized as subtype of np.floating
N float64 is not recognized as object
N float32 is not recognized as int
N float32 is not recognized as subtype of np.integer
N float32 is not recognized as float
Y float32 is recognized as subtype of np.floating
N float32 is not recognized as object
N float16 is not recognized as int
N float16 is not recognized as subtype of np.integer
N float16 is not recognized as float
Y float16 is recognized as subtype of np.floating
N float16 is not recognized as object
N object is not recognized as int
N object is not recognized as subtype of np.integer
N object is not recognized as float
N object is not recognized as subtype of np.floating
Y object is recognized as subtype of object
N object is not recognized as int
N object is not recognized as subtype of np.integer
N object is not recognized as float
N object is not recognized as subtype of np.floating
Y object is recognized as subtype of object

On Mac, it says int64 == int, and therefore the defect you mentioned would not have occurred. I don't have a Windows machine available at the moment to test on Windows. Could you attempt to run this code snippet and see the results? If np.integer and np.floating works on Windows, we will probably switch to np.integer and np.floating for our checks to be more robust.

[ZhangSanMo]

Thanks for your attention.

I run the following code that you list:

import numpy as np
import pandas as pd

dtypes_to_test = ['int64', 'int32', 'int16', 'int8', 'uint64', 'uint32', 'uint16', 'uint8', 'float64', 'float32', 'float16', 'object', 'str']

for dtype in dtypes_to_test:
    series = pd.Series(data=[1, 2, 3], dtype=dtype)
    if series.dtype == 'int':
        print('Y %s is recognized as int' % series.dtype)
    else:
        print('N %s is not recognized as int' % series.dtype)
    if np.issubdtype(series.dtype, np.integer):
        print('Y %s is recognized as subtype of np.integer' % series.dtype)
    else:
        print('N %s is not recognized as subtype of np.integer' % series.dtype)
    if series.dtype == 'float':
        print('Y %s is recognized as float' % series.dtype)
    else:
        print('N %s is not recognized as float' % series.dtype)
    if np.issubdtype(series.dtype, np.floating):
        print('Y %s is recognized as subtype of np.floating' % series.dtype)
    else:
        print('N %s is not recognized as subtype of np.floating' % series.dtype)
    if series.dtype == 'object':
        print('Y %s is recognized as subtype of object' % series.dtype)
    else:
        print('N %s is not recognized as object' % series.dtype)

The output is:

N int64 is not recognized as int
Y int64 is recognized as subtype of np.integer
N int64 is not recognized as float
N int64 is not recognized as subtype of np.floating
N int64 is not recognized as object
Y int32 is recognized as int
Y int32 is recognized as subtype of np.integer
N int32 is not recognized as float
N int32 is not recognized as subtype of np.floating
N int32 is not recognized as object
N int16 is not recognized as int
Y int16 is recognized as subtype of np.integer
N int16 is not recognized as float
N int16 is not recognized as subtype of np.floating
N int16 is not recognized as object
N int8 is not recognized as int
Y int8 is recognized as subtype of np.integer
N int8 is not recognized as float
N int8 is not recognized as subtype of np.floating
N int8 is not recognized as object
N uint64 is not recognized as int
Y uint64 is recognized as subtype of np.integer
N uint64 is not recognized as float
N uint64 is not recognized as subtype of np.floating
N uint64 is not recognized as object
N uint32 is not recognized as int
Y uint32 is recognized as subtype of np.integer
N uint32 is not recognized as float
N uint32 is not recognized as subtype of np.floating
N uint32 is not recognized as object
N uint16 is not recognized as int
Y uint16 is recognized as subtype of np.integer
N uint16 is not recognized as float
N uint16 is not recognized as subtype of np.floating
N uint16 is not recognized as object
N uint8 is not recognized as int
Y uint8 is recognized as subtype of np.integer
N uint8 is not recognized as float
N uint8 is not recognized as subtype of np.floating
N uint8 is not recognized as object
N float64 is not recognized as int
N float64 is not recognized as subtype of np.integer
Y float64 is recognized as float
Y float64 is recognized as subtype of np.floating
N float64 is not recognized as object
N float32 is not recognized as int
N float32 is not recognized as subtype of np.integer
N float32 is not recognized as float
Y float32 is recognized as subtype of np.floating
N float32 is not recognized as object
N float16 is not recognized as int
N float16 is not recognized as subtype of np.integer
N float16 is not recognized as float
Y float16 is recognized as subtype of np.floating
N float16 is not recognized as object
N object is not recognized as int
N object is not recognized as subtype of np.integer
N object is not recognized as float
N object is not recognized as subtype of np.floating
Y object is recognized as subtype of object
N object is not recognized as int
N object is not recognized as subtype of np.integer
N object is not recognized as float
N object is not recognized as subtype of np.floating
Y object is recognized as subtype of object

It seems that your guess is right ~

[Innixma]

Very interesting, for you, float64 == float, float32 != float, int64 != int, and int32 == int.

I will update the checks to use np.integer and np.floating shortly.

In the meantime, could you describe the process you used to get AutoGluon working on Windows? We are looking into adding official Windows support and I'd like to know the commands you ran to run the code successfully.

Best,
Nick

[ZhangSanMo]

1. Get AutoGluon by using commands pip install --upgrade mxnet and pip install autogluon.
2. Just run the tutorial example. It works~

[Innixma]

Changes to fix this issue have been merged and are available in version 0.0.4.