Maskrcnn-benchmark: Explanation on using custom datasets + pretraining

[mattans]

Hi,
I defined my own dataset. In my annotations JSON file, I have 6 classes.

1. When I perform training, how does the model knows how many classes exist? Is it taken from the annotation JSON file, counting the number of categories? Is it from the YAML file, in the OUT_CHANNELS field?

2. How does this affect loading a pretrained model? The number of weights on the last layer depend on the number of classes, which could be different between the pretrained model and the newly trained model.

[fmassa]

Hi,
About your questions:
1 - you need to specify NUM_CLASSES in https://github.com/facebookresearch/maskrcnn-benchmark/blob/ca9531b9f7439e48a94729d0fe2a3335954b454d/maskrcnn_benchmark/config/defaults.py#L182
2 - You need to do something about the last layer. See the discussion from https://github.com/facebookresearch/maskrcnn-benchmark/issues/15 for more details.

[mattans]

1. NUM_CLASSES should actually be the number of classes + 1, am I correct? As the coco dataset classes are 1-based, from 1 to 80.
2. How does this explain that it's possible to load a pretrained ImageNet weights (for 1000 classes) to train a model on coco (80 classes)? Unless, the provided pickle for ImageNet weights does not include the last layer?

[fmassa]

1 - yes, it's the number of classes + background
2 - the name of the imagenet classifier is different, it's fc1000, while the name of the last layer we have in maskrcnn-benchmark ends with cls_score.

[mattans]

I'm not sure I understood about 2. So the last classifier layer of the pretrained ImageNet model is not loaded to the maskrcnn-benchmark model?
If so, then can it be loaded when training on coco or other datasets as well, without handling the last layer specifically like in the link you gave?

[fmassa]

About point 2, the last classifier of an ImageNet model has 1000 classes, and it has a different name than the last classifier from maskrcnn-benchmark.
You can re-use the ImageNet weights to train on other datasets, no need to change anything.
But if you want to re-use the pre-trained weights from a detection model, you need to remove the last classifier weights so that there is no conflict.