Turicreate: Does TuriCreate resize or crop images?

[johnyquest7]

Does turicreate resize / crop images during image recognition based on the model used?

[srikris]

Yes. For image classification models, we automatically crop and resize images to the right one depending on the network.

[gustavla]

Hi @johnyquest7, I thought I would add some more detailed info. All vision toolkits (classifier/similarity/detector) perform implicit resizing to get the data into a fixed size expected by the underlying model. For instance, the ResNet-50 backend for the classifier takes a 224x224 image, so if you provide an image of size 500x400, it will be automatically resized to 224x224 before passed to the model. It may violate the aspect ratio doing this, but the classifier networks are pretty robust to this kind of deformation. There is no cropping involved.

This is what happens at test time for all three models and during training for classifier/similarity. The object detector during training on the other hand does extensive data augmentation, which means each time an image is presented to the algorithm (each image repeated many times) will be randomly resized, cropped, translated, color perturbed, and mirror flipped.

[johnyquest7]

@gustavla Thank you for the detailed explanation. I was planning to use this on medical images. Change in aspect ratio may affect the predictions from a human intelligence standpoint. Anyway, will try it out and see. I was also planning to do mirror flip and other augmentations by code and create a bigger library of pictures before passing it to model create function. Since this is in built, it makes my life much easier.

[amoderate]

what about during scoring - both scoring using the saved turicreate model and if i score using the exported model in my application? Should i resize the image before scoring or is that part of the model scoring pipeline?

[gustavla]

@amoderate There is no need to manually resize images for scoring, since Turi Create will do it for you. The only situation where you may need to manually resize images is when you deploy a Core ML and you make the predictions manually (as opposed to going through Core Vision, which can do the resizing for you).