Maskrcnn-benchmark: Cannot train on model_final.pth

[auniquesun]

❓ Questions and Help

When I finished the training process, I got model_final.pth, log.txt and last_checkpoint file in the dir of OUTPUT_DIR. However, I want to start a new training process on the basis of last generated model_final.pth, set the last model_final.pth as MODEL.WEIGHT, delete log.txt and last_checkpoint.txt, the program skip the training process and go straight forward to the evaluation, any operatons I lost? I need help.

[stevehjc]

Modifiy the MAX_ITER in*.yaml

[chengyangfu]

Hi @auniquesun,
You need to remove the old training parameters in the model.

    import torch
    model = torch.load("YOUR_MODEL_FINAL.PTH")
    print(model.keys()) 
    # Remove optimizer, iteration, and schedular
    del model['optimizer']
    del model['iteration']
    del model['scheduler']
    # Save the model
    torch.save(model, "MODIFIED_MODEL_FINAL.PTH")

[chengyangfu]

I am closing this one. Please let me know if you have further questions.

[auniquesun]

Hi @auniquesun,
You need to remove the old training parameters in the model.

    import torch
    model = torch.load("YOUR_MODEL_FINAL.PTH")
    print(model.keys()) 
    # Remove optimizer, iteration, and schedular
    del model['optimizer']
    del model['iteration']
    del model['scheduler']
    # Save the model
    torch.save(model, "MODIFIED_MODEL_FINAL.PTH")

solve my problem exactly, thank you very much @chengyangfu

[lvpinrui]

❓ Questions and Help

When I finished the training process, I got model_final.pth, log.txt and last_checkpoint file in the dir of OUTPUT_DIR. However, I want to start a new training process on the basis of last generated model_final.pth, set the last model_final.pth as MODEL.WEIGHT, delete log.txt and last_checkpoint.txt, the program skip the training process and go straight forward to the evaluation, any operatons I lost? I need help.

when i build train_net.py ,i just got log.txt,How did you run it?waiting for your advice!~

[tangze5258]

Hi @auniquesun,
You need to remove the old training parameters in the model.
python
import torch
model = torch.load("YOUR_MODEL_FINAL.PTH")
print(model.keys())
# Remove optimizer, iteration, and schedular
del model['optimizer']
del model['iteration']
del model['scheduler']
# Save the model
torch.save(model, "MODIFIED_MODEL_FINAL.PTH")

@chengyangfu hi, i have the same problem and i did delete 'optimizer','iteration','scheduler', but when re_run train_net.py，have no new model_final.pth generated. please help me,thx...

i just modified the trainer.py file (line 104 ) to:
if iteration % checkpoint_period == 0:
# checkpointer.save("model_{:07d}".format(iteration), *arguments)
continue
if iteration == max_iter:
checkpointer.save("model_final", *arguments)