Autogluon: Google Colab - error in dist_manager.py _refresh_resource(cls)

[nathan-wood]

Google Colab notebook for reference:
https://drive.google.com/open?id=1DAVTtAD7zGbJzrkvRKRjWvIhKzCawNkH

Code:
detector = task.fit(dataset)

Error:

ValueError Traceback (most recent call last)
in ()
----> 1 detector = task.fit(dataset)

5 frames
/usr/local/lib/python3.6/dist-packages/autogluon/scheduler/resource/dist_manager.py in _refresh_resource(cls)
48 @classmethod
49 def _refresh_resource(cls):
---> 50 cls.MAX_CPU_COUNT = max([x.get_all_resources()[0] for x in cls.NODE_RESOURCE_MANAGER.values()])
51 cls.MAX_GPU_COUNT = max([x.get_all_resources()[1] for x in cls.NODE_RESOURCE_MANAGER.values()])
52

ValueError: max() arg is an empty sequence

[zhanghang1989]

Thanks for the feedback! Looks like the remote on Collab is not initialized correctly. I will take a look.

[zhanghang1989]

related issue: https://github.com/awslabs/autogluon/issues/162

[zhanghang1989]

Looks like dask distributed issue: https://github.com/dask/distributed/issues/3330
Link to related Colab notebook https://colab.research.google.com/drive/1kzQNNaJwTAAVWjLopuRAySX-QSps5tLZ

[fjibj]

thanks a lot.
i meet the same error and solve it by using:
pip uninstall -y distributed
pip install distributed
pip install -U ipykernel
and RESTART RUNTIME

[Innixma]

Using AutoGluon 0.0.6, I am able to get it working in Colab through:

pip uninstall -y mkl
pip install --upgrade mxnet
pip install autogluon

pip install -U ipykernel

RESTART RUNTIME

from autogluon import TabularPrediction as task
train_data = task.Dataset(file_path='https://autogluon.s3.amazonaws.com/datasets/Inc/train.csv')
test_data = task.Dataset(file_path='https://autogluon.s3.amazonaws.com/datasets/Inc/test.csv')
predictor = task.fit(train_data=train_data, label='class')
performance = predictor.evaluate(test_data)

One thing I noticed was that NN takes ~3x longer to train than on my mac laptop. Intel MKL is installed on Colab and should be uninstalled to speed-up AutoGluon. Uninstalling MKL gives at least 2x speedup.

Link to Colab Notebook: https://colab.research.google.com/drive/1ULOytrRHcw4GLl0qWgGnxUpr9zVQVYlr

I think this issue has to be fixed on Google Colab's end to avoid the pip install -U ipykernel requirement. They need to upgrade their default ipykernel version. Until then, this should be the simplest way to get things working on Colab.

Marking this issue as resolved.