Ignite: Insert logged times into state

[vfdev-5]

🚀 Feature

By default, Engine measures times for single epoch and total run. These times are logged with engine.logger:
https://github.com/pytorch/ignite/blob/efc45a79ee0a5bf8105c3caff194758439db2426/ignite/engine/engine.py#L793-L796
and
https://github.com/pytorch/ignite/blob/efc45a79ee0a5bf8105c3caff194758439db2426/ignite/engine/engine.py#L497-L500

Idea is to put these times (at least time_taken for epoch and total time) into engine.state under a name (e.g. "times"). Such that we can also use this information without need to use Timer or setup logger.

[sdesrozis]

Very good idea!

[erip]

I'm interested in working on this, @vfdev-5. What's your envisioned API? Something like engine.state._complete_epoch() and engine.state._complete_train()?

[vfdev-5]

@erip thanks ! I was thinking about something like that:

import time
from ignite.engine import Engine, Events

trainer = Engine(lambda e, b: time.sleep(0.01))

# INSTEAD OF MANUAL INIT, SETUP times INTO ENGINE.STATE
@trainer.on(Events.STARTED)
def init():
    trainer.state.times = {
        Events.EPOCH_COMPLETED.name: 0.0,
        Events.COMPLETED.name: 0.0
    }

# API :
@trainer.on(Events.EPOCH_COMPLETED | Events.COMPLETED)
def log_times():
    print(
        trainer.last_event_name.name,
        trainer.state.times[trainer.last_event_name.name]
    )

What do you think ?

EDIT: corrected wrong events.

[erip]

That seems good - it would imply that a timer of some sort will need to be instantiated internal to Engine.state, right? I'm trying to think through how the Δt will be computed in the log_times call above.

[vfdev-5]

@erip Actually, we above snippet is wrong. I fixed it.

it would imply that a timer of some sort will need to be instantiated internal to Engine.state, right?

Yes, in run method or even in State by default...

I'm trying to think through how the Δt will be computed in the log_times call above.

Maybe we do not need any deltas to compute as

• time_taken in _run_once_on_dataset will give the time for 1 epoch.
• time_taken in _internal_run will give the time for complete run.

Does it sound good ?

[erip]

One last question: do we want to maintain these times for _each_ epoch or only for the last completed epoch? i.e., do we want trainer.state.times[Events.EPOCH_COMPLETED.name] to be a list of times or just a time? I could see both being useful. I think it makes most since to just maintain a single value for now and let the user determine if/how they want to maintain that list if necessary.

[vfdev-5]

@erip yes, both can be helpful and I agree with you about

I think it makes most since to just maintain a single value for now and let the user determine if/how they want to maintain that list if necessary.

We can leave to the user the way to store all times, as other values we store in the state, e.g. metrics, batch, output etc.