Keras: predict() functions takes more time during the first time it is called

[akshay-vaidya]

I am using pretrained CNN model built using keras to get predictions on image data. The first time I hit the function, predict() to get the model prediction, it takes a log time to produce results. From the second time on, it produces the results pretty quick. Any idea what the issue is?

Models and weights are loaded already in the memory before I pass the image object for prediction.

Thanks!

[Dref360]

This is normal, the first pass is always long because the backends are allocating the memory and stuff like that.

[akshay-vaidya]

Thanks @Dref360. But as I had mentioned, I have already loaded the model in the memory. Is there anyway I can reduce the time during the first pass?

[Dref360]

Not really, you can disable optimization, but your network will then be slower.
The first pass is really important. Not much is done when you load the weight, everything is set up during the first pass like cache, memory on the GPU, graph optimization, etc.
More info : https://groups.google.com/a/tensorflow.org/forum/#!topic/discuss/PDIBnp1ftxk

[akshay-vaidya]

@Dref360 Thanks much for the info.

[JoshSag]

I see that the first "predict" is taking longer than others even after I do some "fit" operations before the predictions.
Allegedly, everything needed to perform "predict" is required for "fit" too.
So, can you please explain why is that? And maybe, in this case, I can reduce this time using some pre-allocated tensors?

[ws-lin]

I'm using CRNN for text recognition, and each time I input a picture with a different size, I find that the prediction time of each input model with a different size is very long.If the input is of the same size, the rest of the images will be predicted about 10 times faster, except for the first time