Detectron2: Multi-scale testing mistakenly discards all instances of the last category.

[Tete-Xiao]

How To Reproduce the Issue

Running Faster RCNN with multi-scale testing.

The Problem

For Faster RCNN, MODEL.ROI_HEADS.NUM_CLASSES will be automatically += 1 for background class (I'm not sure for other models. I observed the Faster RCNN on COCO detection is 81 classes in the model.)

However, when running multi-scale testing, as _inference_one_image, num_classes is not added by 1.
Then, in fast_rcnn_inference_single_image, the last class is discarded, as it's supposed to be the background class. But because the num_classes is not added by 1, it results in the lost of all instances from the last category.

The problem is especially severe when training the detector on datasets with only two or three categories.

[Tete-Xiao]

Sorry for mistakenly closed the issue :(

[ppwwyyxx]

should be fixed in 3506d46bcf933432a745f96aaaecccd2c72514cb. Thanks for reporting this!