Ignite: ValueError: Object should have `state_dict` method.

[yuliuyu]

hi，when I train your reid_baseline model ,I met a error that I can't solve. the error is as follows:

Traceback (most recent call last):
File "/home/lc/Desktop/reid_baseline/tools/train.py", line 87, in
main()
File "/home/lc/Desktop/reid_baseline/tools/train.py", line 83, in main
train(cfg)
File "/home/lc/Desktop/reid_baseline/tools/train.py", line 46, in train
num_query
File "/home/lc/Desktop/reid_baseline/engine/trainer.py", line 150, in do_train
trainer.run(train_loader, max_epochs=epochs)
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/engine/engine.py", line 388, in run
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/engine/engine.py", line 352, in _handle_exception
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/engine/engine.py", line 379, in run
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/engine/engine.py", line 287, in _fire_event
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/handlers/checkpoint.py", line 172, in __call__
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/handlers/checkpoint.py", line 130, in _save
File "/usr/local/lib/python3.5/dist-packages/pytorch_ignite-0.2.0-py3.5.egg/ignite/handlers/checkpoint.py", line 144, in _internal_save
ValueError: Object should have state_dict method.

How should I fix it?

[vfdev-5]

@yuliuyu can you provide a snippet of which objects you are trying to save ?
Meanwhile, you can take a look at the doc of ModelCheckpoint, argument save_as_state_dict and the example of usage. Maybe this can help.

[yuliuyu]

[vfdev-5]

@yuliuyu unfortunately, this image wont help. Can you provide the code you are running ?

[vfdev-5]

@yuliuyu your problem is here:
https://github.com/L1aoXingyu/reid_baseline/blob/852bb8ae8bd6635527e5a39609f8beed81be26c0/engine/trainer.py#L108-L109

Change it to this and it should be fine:

trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model, 'optimizer': optimizer})

[yuliuyu]

thank you for your help , I will try

[vfdev-5]

@yuliuyu i'll close the issue as resolved, if you still experience an issue, please feel free to reopen it

[jhxing]

@yuliuyu i'll close the issue as resolved, if you still experience an issue, please feel free to reopen it

Hello， I got the same issue as mentioned, and I tried your solution above, still, the issue is there. What should I do next to solve this issue ?

[vfdev-5]

@jhxing could you please post your code that produce the issue, such that I could inspect the problem.

[jhxing]

@jhxing could you please post your code that produce the issue, such that I could inspect the problem.

I used the code "Experiment-all_tricks-tri_center-market.sh" in the program and I modified the code to:

CUDA_VISIBLE_DEVICES=1 python3 tools/train.py --config_file='configs/softmax_triplet_with_center.yml' MODEL.DEVICE_ID "('0')" DATASETS.NAMES "('market1501')" DATASETS.ROOT_DIR "('/home/smc/hongxing/data')" OUTPUT_DIR "('/home/smc/hongxing/re_ID/reid-strong-baseline/market1501/Experiment-all-tricks-tri_center-256x128-bs16x4-warmup10-erase0_5-labelsmooth_on-laststride1-bnneck_on-triplet_centerloss0_0005')"

In the begining I run this code, it seemed all worked , but when it trained in Epoch[40], the same err:

ValueError: Object should havestate_dictmethod
occurred.

Now I am trying another train code named "Experiment-all_tricks-without_center-market.sh" , it has run at Epoch[54], and nothing happend so far.

[vfdev-5]

@jhxing as far as I understand this is an issue of re_ID code. Please update your code according to above comment : https://github.com/pytorch/ignite/issues/529#issuecomment-494287537

[jhxing]

Thank you for your advice , I tried this solution , still the same issue happened .
Are there any other solutions ？

[vfdev-5]

@jhxing it is very difficult to inspect your problem without seeing the exact code. When I see the actual code of reid_baseline:
https://github.com/L1aoXingyu/reid_baseline/blob/852bb8ae8bd6635527e5a39609f8beed81be26c0/engine/trainer.py#L108-L109
it still uses wrong API, but should be

trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model, 'optimizer': optimizer})

[jhxing]

@jhxing it is very difficult to inspect your problem without seeing the exact code. When I see the actual code of reid_baseline:
https://github.com/L1aoXingyu/reid_baseline/blob/852bb8ae8bd6635527e5a39609f8beed81be26c0/engine/trainer.py#L108-L109
it still uses wrong API, but should be

trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model, 'optimizer': optimizer})

Thank you for your patience, I will show my code below (/engine/trainer.py):
#trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model.state_dict(), # 'optimizer': optimizer.state_dict()}) trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model, 'optimizer': optimizer})

And, I successfully run the training code "Experiment-all_tricks-without_center-market.sh",
the only question is, when I run another training code "Experiment-all_tricks-tri_center-market.sh", the errors came out.

[vfdev-5]

@jhxing what is exactly Experiment-all_tricks-tri_center-market.sh script and which code it executes ?

[jhxing]

@jhxing what is exactly Experiment-all_tricks-tri_center-market.sh script and which code it executes ?

It is a code about training Re ID models, I download this form the github https://github.com/michuanhaohao/reid-strong-baseline. And in the "/engine/trainer.py" , I have fix the code line you mentioned above. And the errors I can post below:
Traceback (most recent call last): File "tools/train.py", line 158, in <module> main() File "tools/train.py", line 154, in main train(cfg) File "tools/train.py", line 113, in train start_epoch # add for using self trained model File "./engine/trainer.py", line 291, in do_train_with_center trainer.run(train_loader, max_epochs=epochs) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/engine/engine.py", line 359, in run self._handle_exception(e) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/engine/engine.py", line 324, in _handle_exception raise e File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/engine/engine.py", line 350, in run self._fire_event(Events.EPOCH_COMPLETED) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/engine/engine.py", line 259, in _fire_event func(self, *(event_args + args), **kwargs) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/handlers/checkpoint.py", line 172, in __call__ self._save(obj=obj, path=path) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/handlers/checkpoint.py", line 130, in _save self._internal_save(obj, tmp.file) File "/home/smc/work/vcmt/lib/python3.6/site-packages/ignite/handlers/checkpoint.py", line 144, in _internal_save raise ValueError("Object should havestate_dictmethod.") ValueError: Object should havestate_dictmethod.

[vfdev-5]

@jhxing did you change the code here ?
https://github.com/michuanhaohao/reid-strong-baseline/blob/da3f2fea1a183d125f3009f9269759a8bd474830/engine/trainer.py#L238

    trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model.state_dict(),
                                                                     'optimizer': optimizer.state_dict(),
                                                                     'center_param': center_criterion.state_dict(),
'optimizer_center': optimizer_center.state_dict()})

[jhxing]

@jhxing did you change the code here ?
https://github.com/michuanhaohao/reid-strong-baseline/blob/da3f2fea1a183d125f3009f9269759a8bd474830/engine/trainer.py#L238

    trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model.state_dict(),
                                                                     'optimizer': optimizer.state_dict(),
                                                                     'center_param': center_criterion.state_dict(),
'optimizer_center': optimizer_center.state_dict()})

No, I did not changed code here, so , what should I do ? What should I changed this line to ?

[vfdev-5]

@jhxing you need to remove .state_dict()

trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model,
                                                                     'optimizer': optimizer,
                                                                     'center_param': center_criterion,
'optimizer_center': optimizer_center})

[jhxing]

@jhxing you need to remove .state_dict()

trainer.add_event_handler(Events.EPOCH_COMPLETED, checkpointer, {'model': model,
                                                                     'optimizer': optimizer,
                                                                     'center_param': center_criterion,
'optimizer_center': optimizer_center})

Thank you for your patient help, I will try it. Thank you angin !