Pytorch-lightning: Accessing measured time per epoch shown in progress bar.

Created on 2 Apr 2020  路  2Comments  路  Source: PyTorchLightning/pytorch-lightning

What is your question?

How do I access the measured time per epoch shown in progress bar?

Epoch 8: : 150it [00:03, 46.84it/s, loss=1.625, v_num=0]

It clearly says 00:03 i.e. 3 seconds and I could parse the logs as a hack, but I was wondering if there was any clean way to access the measured time elapsed per training epoch, as I would like to write it to a file.

And if not, how would you recommend timing each epoch? I could put a time.time() in training_step but then would need to add it all up in training_epoch_end, would this work? Thank you!

What's your environment?

  • OS: Mac OSX
  • Packaging: Pip-installed packages in a conda environment
  • Version: 0.7.1
question

Most helpful comment

Just an idea, if you use tensorboard as the logger, it has timestamps for everything, you could open the log (in python) and find the stamps.

All 2 comments

Just an idea, if you use tensorboard as the logger, it has timestamps for everything, you could open the log (in python) and find the stamps.

I would use tensorboard logs.

Otherwise just build your own callback and overwrite epoch_start and epoch_end

https://pytorch-lightning.readthedocs.io/en/latest/callbacks.html

Was this page helpful?
0 / 5 - 0 ratings

Related issues

williamFalcon picture williamFalcon  路  3Comments

srush picture srush  路  3Comments

DavidRuhe picture DavidRuhe  路  3Comments

iakremnev picture iakremnev  路  3Comments

monney picture monney  路  3Comments