Vision: Simple way to extract features from faster r-cnn

[cdancette]

Currently it is complicated to extract the object features from the faster r-cnn model. It would be nice to have a simple function like model.features(batch), that would give the features in addition to the boxes and labels.

[fmassa]

The question is always: which features do you want to extract? Just after RoIAlign? The one before the last classifier? Something in between?

I don't think we should be extending the API of particular models for extracting intermediate activations. This can be very quickly complicated, and will not be generic enough.

One possible (and easy to do) solution is to just attach a forward hook in the modules where you want to get the output from.

But I do agree that having a way to extract intermediate activations is important. I've opened an issue in PyTorch in the past for that, see https://github.com/pytorch/pytorch/issues/21064

GIven the above, I'd say that we should not try to make the API more complicated for some particular models (and not others?), and thus I'm closing this issue.
But please feel free to reopen it if you think otherwise.

[shankar-agarwal]

Hi, faster-rcnn (and mask-rcnn) gives the bounding box + classes as output. Is there a way to extract the feature vectors of these objects? For example, how can I get the activations before the last classifier?

Regards -Shankar