Tensorboard: download more than 1000 plots in csv or json

[smatsumori]

Hi. It seems that when we export scalars in csv with tensorboard, we can not get more than 1000 plots. Is there a way to save more plots?

[wchargin]

Nope. As described in the README, TensorBoard downsamples your data. We only store 1000 scalars per run/tag combination. (You can adjust this value by editing that source file.)

This isn't just a matter of us not sending all the data to the frontend; we actually discard it in memory, so implementing this feature would not be feasible. (We'd have to reload all the summary data and also process it via a separate pipeline.) However, all hope is not lost. First, we're working on implementing a SQL backend, for which we'll handle downsampling differently, and for which this request might be more tractable. You can track this progress in #92. Second, it should not be too hard to write a script that loads scalar data from your summaries and dumps selected data to a CSV file. If this is something that you're interested in, I can write some starter code to point you in the right direction—what do you think?

[demmerichs]

Hi!

Second, it should not be too hard to write a script that loads scalar data from your summaries and dumps selected data to a CSV file. If this is something that you're interested in, I can write some starter code to point you in the right direction—what do you think?

I would be very interested in this. I've looked a little bit around (https://stackoverflow.com/questions/41074688/how-do-you-read-tensorboard-files-programmatically) how to write my own script exporting tfevent files to csv (especially as I have multiple runs and tags and don't want to download them one by one in tensorboard, would take forever). The link suggested using EventAccumulator, however since tf 1.1 this class does not seem to exist anymore. #9532

If someone could point me/us in the right direction, that would be great!

[wchargin]

This class now lives in the event_multiplexer module of the tensorboard.backend.event_processing package; for new applications, we recommend that you use the plugin_event_accumulator module.

Here's a sample script that extract some data generated by the scalar demo script:
https://gist.github.com/wchargin/31eee50b9aaebf387b380f70054575c5

Note that multiplexer.Reload() will load _all_ data under the log directory. If you have lots of data, then this will take a while, even if most of the data is unrelated to what you're trying to export. (In such a case, consider using the more granular EventAccumulator API.)

[smatsumori]

Hi @wchargin @DavidS3141
Thank you for giving me the exact solution that I was looking for! It would be really helpful that this feature will be implemented in the near future.