Vision: Problem train/finetuning segmentation (fcn_resnet101) on voc data

[Denlar2]

Hi
Thanks for a great api.

I am trying to train/finetuning the trained fcn_resnet101 trained on coco dataset, but it seems like after 1. epoch it is way worse on voc data, than it is before.

If i test the already trained fcn_resnet101 on the voc data i get mean IoU: 73.3.

Then i train the fcn_resnet101 on the voc data and after 1. epoch i get mean IoU: 3.8.
Why is it so much worse after training 1. epoch?
Seems like i am not using the pretrained network for training.

The command line i am using for finetuning the network is:
python3 -m torch.distributed.launch --use_env train.py --lr 0.02 --dataset voc -b 2 --model fcn_resnet101 --aux-loss --pretrained

And for testing i use:
python3 -m torch.distributed.launch --use_env train.py --lr 0.02 --dataset voc -b 2 --model fcn_resnet101 --aux-loss --pretrained --test-only

Have somebody experinced the same?

Hope somebody can help?

Because after this i want to train on my own dataset.

[fmassa]

Hi,

The learning rate you are using is too high. We used a learning rate of 0.01 with 8 GPUs. So if you want to finetune from an already-pretrained model, you should start with a smaller learning rate, and also take into account the number of GPUs you are using.
If using 1 GPU instead of 8, you should at least divide the original learning rate by 8.

[Denlar2]

Thanks @fmassa for the fast help.
Yes i am only finetuning on one GPU
A lower learning-rate make sense, since it is just finetuning.

[Denlar2]

Thanks again @fmassa
After i have trained my model, and i stay around the same IoU almost 70 on the voc data.

I am trying to evaluate my model on one image, and trying to show this image, it looks like the model is way worse.
I am using this to load my model (it works fine with the pretrained model on voc, but not my own model after one epoch):

#Load model
model_fcn = torchvision.models.segmentation.fcn_resnet101(num_classes=21, aux_loss=True, pretrained=False)
model_name = 'fcn_resnet101_voc.pth'
model_fcn.load_state_dict(torch.load(model_name),  strict=False)

and is following this example to show the segmented image https://www.learnopencv.com/pytorch-for-beginners-semantic-segmentation-using-torchvision/

Which means for segment the image i am doing this:

def segment(model, path):
  img = Image.open(path)
  plt.imshow(img); plt.axis('off'); plt.show()
  # Comment the Resize and CenterCrop for better inference results
  trf = T.Compose([T.Resize(520), 
                   T.CenterCrop(480), 
                   T.ToTensor(), 
                   T.Normalize(mean = [0.485, 0.456, 0.406], 
                               std = [0.229, 0.224, 0.225])])
  inp = trf(img).unsqueeze(0)
  out = model(inp)['out']
  om = torch.argmax(out.squeeze(), dim=0).detach().cpu().numpy()
  rgb = decode_segmap(om)
  plt.imshow(rgb); plt.axis('off'); plt.show()

Can you explain why i cant see the evaluatio on my own model, even i havent trained it almost, and the IoU is almost the same. But i can on the pretrained model ?
Hope you will be able to help again thanks in advance.

[fmassa]

@Denlar2 are you putting your model in evaluation mode (via .eval()) before computing the predictions?

Also, the mAP not changing in your dataset might not be surprising, it all depends on how similar your data is to the COCO images we have used to train the models.

[Denlar2]

@fmassa thanks for the fast answer.
Yes, i put it in evaluation mode, this is my script:

#Load model
model_fcn = torchvision.models.segmentation.fcn_resnet101(num_classes=21, aux_loss=False, pretrained=False)
model_name = 'fcn_resnet101_voc.pth'
model_fcn.load_state_dict(torch.load(model_name), strict=False)

#Evaluate model
model_fcn.eval()
segment(model_fcn, 'airplane.jpg')

Then the segment function is the one in the previous comment.

Figured out my mistake was i needed to change the way i load the model till this instead:

checkpoint = torch.load(model_name, map_location='cpu')
model_fcn.load_state_dict(checkpoint['model'], strict=False)

Thanks for you help again