Keras: EarlyStopping Callbakc not working with Multiple Callbacks

[opensubb009]

Please make sure that this is a Bug or a Feature Request and provide all applicable information asked by the template.
If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.

System information

• Have I written custom code (as opposed to using example directory):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
• TensorFlow backend (yes / no): yes
• TensorFlow version: 1.14.0
• Keras version: 2.2.4
• Python version: 3.6
• CUDA/cuDNN version:
• GPU model and memory:

Issue
I am using custom auc metric by following code and using multiple callbacks early_stopping and auc as well.
But now my model traning does not stop even my loss is not improving much. Can you tell me what is wrong in code or It is kind of bug.
If I am not using Metric in callbacks then early_stopping working fine.

class Metric(Callback):
    def __init__(self, val, y_val):
        self.val = val
        self.y_val = y_val

    def on_epoch_end(self, epoch, logs):
        y_data = np.argmax(self.y, axis=1)
        y_pred = self.model.predict(self.train)
                roc = roc_auc_score(y_data, y_pred[:, 1])
        print('roc-score:', roc)


early_stop = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=2, min_delta=1e-7)
callbacks = [early_stop, Metric(X_CV, y_cv)]


model.fit(train_data, y_train, epochs=100, batch_size=32, validation_data=(X_CV, y_cv), callbacks=callbacks)

[gdyuldin]

EarlyStopping works well if it passed as the last callback in the list.
E.g.
callbacks = [Metric(X_CV, y_cv), early_stop]

Looks like during applying other (next) callback model.stop_trainig property resets to False.

[eduardofv]

I can confirm this. If EarlyStopping is not the last callback it will not stop training.

• Would it be reasonable to sort the callbacks forcing EarlyStopping be the last?
• Suppose we add an attribute to the callback "must_be_last", but some other callback also request to be the last (or the first for that matter.) How that could be resolved?
• Or simply stating in the documentantion that you must add EarlyStopping as the last callback in the list would be enough?

[eduardofv]

Just for the record, the issue is solved in tensorflow v2.2rc0. I suggest to close this issue.