Autogluon: How to load a pre trained classifier params file in auto gluon for inferencing ?

[Pentar0o]

Hi,

Once I've trained a classifier in Autogluon and I have the params file how can I load this params file in another script for inferencing (with my code below for training) ?
dataset = task.Dataset('DataSet')
classifier = task.fit(dataset)

network = classifier.model
network.save_parameters('network.params')

test_dataset = task.Dataset('DataSet/test', train=False)
indice, resultat, probs_all = classifier.predict(test_dataset)

In the classification examples you do both training and inferencing in the same script and I've found nothing in the documentation to load params in a autogluon classifier.
Is there like a hidden task.load function ?

Thx for your help. (hope this is understandable I'm new here)

[Innixma]

@zhreshold Could you look into this?

[jwmueller]

@Pentar0o Assuming your task of interest is ImageClassification, you can reference the save/load documentation here:
https://autogluon.mxnet.io/api/autogluon.task.html#classifier

You should be doing:

classifier.save(PATH)
...
classifier = autogluon.task.image_classification.Classifier.load(PATH)

[jwmueller]

@zhreshold @Jerryzcn Any idea what happened to the old example scripts (ie. simple.py/advanced.py) that used to live in examples/image_classification/? I vote we add those back into this directory (and move the blog-stuff to examples/image_classification/kaggle/ subdirectory). I think example scripts are mainly supposed to help newcomers see what functionality is possible & provide an easy test-script for contributors to trial their new code, but the Kaggle blog example-scripts seem overly complex and do not demonstrate basic functionality like save/load.

[Pentar0o]

Hi, so with the code :

import autogluon
from autogluon import ImageClassification as task

classifier = autogluon.task.image_classification.Classifier.load('network_98_4.params')
indice, resultat, probs_all = classifier.predict('Test')

Test is my folder of test pictures

I got the following error :

/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
Traceback (most recent call last):
File "ClassifierLoad.py", line 6, in
classifier = autogluon.task.image_classification.Classifier.load('network_98_4.params')
File "/home/penta/.local/lib/python3.8/site-packages/autogluon/task/image_classification/classifier.py", line 56, in load
state_dict = load(checkpoint)
File "/home/penta/.local/lib/python3.8/site-packages/autogluon/utils/serialization.py", line 79, in load
return _load(f, map_location, pickle_module, *pickle_load_args)
File "/home/penta/.local/lib/python3.8/site-packages/autogluon/utils/serialization.py", line 361, in _load
magic_number = pickle_module.load(f, *pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'x12'.

[zhreshold]

@Pentar0o You are mixing save/load that's designed for different purposes.

Refering your original code snipet

dataset = task.Dataset('DataSet')
classifier = task.fit(dataset)

You can save/load the complete state of the estimator by

classifier.save('cls.states')
new_classifier = autogluon.task.image_classification.Classifier.load('cls.states')

or simply save the network parameters if you don't necessarily need to resume the full state of fitting but want to revert or copy the weights from certain checkpoint:

classifier.model.save_parameters('network.params')
classifier.model.load_parameters('network.params')

For you usecase I'd suggest you stick with the first example so you don't have to instantiate a new classifier.

[Pentar0o]

Question : how to load the params without having to start the learning process first ? because the task.fit(dataset) will launch the training.

Is there something like :
new_classifier = autogluon.task.image_classification.Classifier.load_parameters('network.params')

[Innixma]

@zhreshold Could we add an example of saving, loading, and predicting on the loaded model to the image classification tutorial for quick reference?

[zhreshold]

The current api for serializing the estimator is incomplete so I think it only make sense when the save/load functionalities are complete, I will update the corresponding tutorial after merging the new estimator refactorization.