Hub: Retrain image classification networks with images of different size

[zimmerrol]

I think it would be useful, to allow one to use a pre-trained image classification network as a feature generator for images with other shapes (greater width/height). At the moment this is forbidden as the images then do not have the same shape as get_expected_image_size() returns, which results in an exception.

[SeguinBe]

This is definitely necessary, images are not so often square, and for any form of semantic segmentation.

[jalabort]

First of all thank you very much for putting this together; imho this is a huge step towards a more standard way of sharing and reutilising tensorflow models.

I think it would be very useful to have access to the already provided image models up to their last convolutional layer so that something as what is suggested above is possible. This would allow the community to utilise these modules as backbone feature extractors in more general detection and segmentation tasks without being restricted to specific input sizes.

[SeguinBe]

@jalabort accessing convolutional layers is already possible. Unfortunately, only with the predefined size.

p = tf.placeholder(tf.float32, [None, height, width, 3])
module(p, signature='image_feature_vector', as_dict=True)

See https://drive.google.com/file/d/10daL16CkUfhDaAo7MUIq7jiNVle3HmxG/view?usp=sharing

[jalabort]

Yes, sure. I meant having access to convolutional feature extractor backbones that do not rely on predefined sizes (so truly convolutional). As I see it, the fact that these models are shipped with their fully connected layers is what is forcing them to have predefined sizes (otherwise dimensions would not play well together). It would be great if there was official support for these models as true fully convolutional models although it is also true that the tensorflow folks provide all the necessary utilities between this repo and the slim models in tensorflow/models to create these modules on our own...

[daegeumer]

Hi. Could you tell me if a plan is in place to resolve this issue? I'd like to the pre-trained model as a feature extractor for images with unconstrained shape (not square). I hope this option will be added in future versions (tf 2.0?).

[Harshini-Gadige]

@vbardiovskyg Any thoughts on this request ?

[arnoegw]

Hi @daegeumer, thanks for raising this. The tensorflow-hub 0.2.0 release has made some preparations for image modules with unconstrained input shapes. Due to limitations in some of the slim nets we are exporting from, this would not be possible for MobileNetV2 and (P)NASNet, but for our other networks.

With these constraints, is there still interest to get this fixed for TF1?

[daegeumer]

Hi @arnoegw . Absolutely! There are famous pretrained networks in the hub, as far as I know. For example, Resnet, InceptionNet. InceptionResNet. These feature extractors are can be used usefully for many researchers. But, in my opinion, it's better to add other famous pretrained networks like VGGNet, Alexnet, Xception, Darknet, etc. Thanks for accepting for this issue, and I hopefully to get this fixed soon in the future versions.

[arnoegw]

For Inception, ResNet, and MobileNet V1 (except quantops), module versions /3 (released today) now support variable-size inputs. For the full release announcement, see https://github.com/tensorflow/hub/issues/294.