Keras: what's model._make_predict_function() used for?

[leckie-chn]

I have some problem using Keras model for prediction in parallel, details described here. Today I read some code from jaara/AI-blog/CartPole-A3C.py and found this:

model = Model(inputs=[l_input], outputs=[out_actions, out_value])
model._make_predict_function()  # have to initialize before threading

I think this is what makes difference. So can anyone tells me how this function works? I can't find any documentation about it.

edit:
I put the same issue here and got an reply. But I want more information from the Keras Community.

[bstriner]

Using theano or tensorflow is a two step process: build and compile the function on the GPU, then run it as necessary. make predict function performs that first step.

Keras builds the GPU function the first time you call predict(). That way, if you never call predict, you save some time and resources. However, the first time you call predict is slightly slower than every other time.

This isn't safe if you're calling predict from several threads, so you need to build the function ahead of time. That line gets everything ready to run on the GPU ahead of time.

Cheers

[jaromiru]

The problem is there is zero documentation on this issue.

[bstriner]

@jaara you're not wrong about that. Maybe some comments on that function would be nice. As a bigger problem, there are a lot of issues related to the whole compilation/execution division. Issues like "get_updates is only called once, what is going on?" or "why is _call_ only called once?" or "can't use numpy in custom layer".

The whole idea of creating the function and then calling the function throws a lot of people for a loop. Just check out the issue log. Not sure the best way to clear up the confusion but it is definitely a common problem.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[eugenemi]

Running on 1-gpu machine, adding tf.Print() in keras/optimizers.py:SGD.get_updates() allowed me to see lr being modified after every batch.

     lr = self.lr
     if self.initial_decay > 0:
        lr *= (1. / (1. + self.decay * self.iterations))
        self.updates .append(K.update_add(self.iterations, 3))
     lr = tf.Print(lr, [lr], message="This is SGD lr: ") # print lr on every call

[agilebean]

isn't _make_predict_function() deprecated? i couldn't find it in the official Keras documentation any more...

[iamtodor]

+1
no documentation about model._make_predict_function()

[ugrceyln]

+1
no documentation about model._make_predict_function()

[MachineJeff]

@bstriner Hi
read your explanation,

This isn't safe if you're calling predict from several threads, so you need to build the function ahead of time.

I wanna know why that's not save.

this answer in stack_overflow also mentioned:

In multi thread setting, you have to manually call this function to compile predict in advance, otherwise the predict function will not be compiled until you run it the first time, which will be problematic when many threading calling it at once.

However, I can not really understand it. Could you please help?