Pytorch-lightning: Add a basic logger which does not require ports

[CharlieCheckpt]

🚀 Feature

It would be great to have a basic logger that does not require opening ports.

Motivation

At this time, all loggers supported by Lightning requires to use ports. There are some environments where I don't have total control and I cannot open ports. It would be great to implement a logger that could dump the metrics during training, and easily visualize their evolution, without the need to forward a port.

Pitch

As proposed by @justusschock in the slack channel, one could add a basic logger FileLogger that saves metrics into a file (like json) and plots them on a regular basis.

Thank you for the great package !

[williamFalcon]

Maybe this should just be a csv logger? because most viz libraries use CSVs?

[minimaxir]

A CSV logger is actually a good idea; the Keras CSV logger got a lot of mileage in my work.

https://keras.io/api/callbacks/csv_logger/

[github-actions[bot]]

Hi! thanks for your contribution!, great first issue!

[williamFalcon]

ok cool. Yeah, test-tube already has these ideas (in fact, v1 was just a csv logger). Maybe we can repurpose that code. Although this should be pretty simple to do.

Caveat: we don't want to use pandas for this haha.

Anyone want to submit the PR?
@minimaxir @CharlieCheckpt

[CharlieCheckpt]

It will be difficult for me to do it in the following weeks, sorry @williamFalcon !

[xmotli02]

Hi, I would like to take this one if it is still available.

If I understand it correctly, it should be CSV logger, with simple output directory structure

• logs/
  
  
  
  • exp_name/
  
  
  • exp_version/
    
    
    
    
    
    • hparams.csv
    
    
    
    • metrics.csv
    
    
    
    • experiment.csv
    
    
    
  
  

Is that right? Should there be anything else?

[justusschock]

for now, we usually don't save hparams in csv any more but use yaml instead (at least in checkpointing). you could maybe have a look on whether you want to stay with csv or just use the code from checkpointing...

You might also think about saving images later on :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[justusschock]

@xmotli02 how is it going here?

[xmotli02]

Hi, sorry I was pretty busy these last couple of weeks and then I forget a little about this issue. However, I have almost finished logger, I just need to have a look at some edge cases and add tests. I think I'll be able to find some time this weekend to finish this. I'll keep you updated.