Keras: alternative accuracy measures

[matudor]

It would be nice to have additional accuracy measures reported during training. I think callbacks could be used, but I'd prefer to have things like recall, precision, and F1 calculated as part of fit.
However from my perusal of the code, the accuracy measure is fairly entwined in the compile and other code in the model class and, while one could overload a single alternative accuracy calc, it's not straightforward to report more than one or all.
In addition, would be nice to report a few different losses (e.g. MSE and cross entropy) even if only one is used to train.
Cheers
M

[fchollet]

I guess we could introduce a generic Metrics callback, configurable with common metrics (such as F1, etc).

[matudor]

fchollet,
Thanks, that would be a good solution.
I explored this but couldn't make headway, I guess my issue was how to access batch data, both training and validation holdout from the callback class.
Cheers
M

[fchollet]

Thinking about it. There are two potential solutions, but we want to go
with the right one, not the easiest one. I'll come back to this a bit later.

The easiest solution would be to make the training and validation data (as
well as any weights provided) attributes of the model from inside the fit
methods of Graph and Sequential.

The (probably) better solution would to make the ins arrays and their
text labels accessible to the callbacks, and abstract this away for metric
monitoring by developing a new callback that exposes this system with a
friendly API.

On 2 September 2015 at 15:39, matudor [email protected] wrote:

fchollet,
Thanks, that would be a good solution.
I explored this but couldn't make headway, I guess my issue was how to
access batch data, both training and validation holdout from the callback
class.

Cheers
M

—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/issues/626#issuecomment-137264739.

[matudor]

great, thanks for looking into it!

[aribenjamin]

Re-upping this old issue. Is there now a way to access batch training data in a callback? (For metrics; simply trying to observe the loss on a per-batch basis rather than per-epoch).

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.