Keras: Why does LearningRateScheduler not write the learning rate to Tensorboard ?

[rohit-gupta]

The title pretty much says it all, when you use model.fit with ReduceLRonPlateau or without it along with the Tensorboard callback, Keras writes the learning rate by epoch to Tensorboard, but if you LearningRateScheduler, it doesn't ? Is there any particular reason for this ? Its technically not a bug, but its very unexpected behavior.

• [X] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• [X] Check that your version of TensorFlow is up-to-date. The installation instructions can be found here.

• [X] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[gabrieldemarmiesse]

I think it has to do with the order in which the callback are executed or something. I can't remember, but I believe there is a PR somewhere (in keras or tf.keras) about adding the learning rate by default in the logs (making tensorboard write this learning rate automatically). If you can find out more it'd be nice. I believe it would be a nice feature to add the learning rate to tensorboard automatically. But I don't know if @fchollet will agree.

[stante]

I looked a bit into the source code for curiosity. The TensorBoard callback write out basically what is in the logs at the end of a batch. The ReduceLRonPlateau adds the learning rate to the logs, that is why it is written by the TensorBoard callback and also returned via the history. The LearningRateScheduler however does not add anything to the logs. It would be an easy task to add that via implementing on_epoch_end function for consistency between ReduceLRonPlateau and LearningRateScheduler.

[stante]

I've just seen that this was done 5 month ago with https://github.com/keras-team/keras/commit/cf7d3d259f93b1ddbc601208280a460ee3379591. @rohit-gupta if you are using a Keras version which includes that patch, than its probably only a matter of the order of your callbacks. The TensorBoard callback should come after the LearningRateScheduler.

[rohit-gupta]

@stante Thanks for the digging, I will check this out and report if this works.

[stante]

The patch mentioned above was introduced with Keras 2.2.3.

[jvishnuvardhan]

I think it was resolved. I am closing the issue. Please open a new ticket if you see similar issue again. Thanks!