Ignite: How to resume the best model saved and evaluate on the test dataset

[Sudy]

Thanks for this excellent project. Since I'm new to Ignite, I have encountered a problem when I try to load the best saved model and run on the test dataset at the end of training. I don't know how can this be achieved under current framework. Can anyone help with this.

[alykhantejani]

Hi @Sudy,

We are in the works of adding easier model checkpointing (#75) but in the mean time, when you are done training or just periodically you can save models using the standard pytorch way of saving and loading models (see here).

For example, to save at the end of training you can attach a handler to the COMPLETED event i.e.:

def save_model(trainer, model):
     out_path = "model_epoch_{}.pth".format(trainer.current_epoch)
     torch.save(model.state_dict, out_path)

trainer.add_event_handler(Events.COMPLETED, save_model, model)

Then you can load it with:

the_model = TheModelClass(*args, **kwargs)
the_model.load_state_dict(torch.load(PATH))

and evaluate as necessary (btw we are improving the way to evaluate models in #78)

[Sudy]

Hi @alykhantejani ,

Thanks for the reply. My problem is not loading or saving models, but how to apply the best model to the test dataset. I finally solve the problem by the following code, which is not elegant but is working.

testor.add_event_handler(Events.COMPLETED, get_performance(logger)) model.load_state_dict(torch.load('../model/' + model_name))
testor.run(test_loader)

[alykhantejani]

Oh I see, so the logic to run stuff through your tester should be in the function passed to its constructor.

[alykhantejani]

Closing the issue as it appears to be resolved. Please feel free to comment on it if you need further help :)