Glow: [Feature request] Additional debug output

[ayermolo]

Hi

Is there interest for patch, below, to be propagated in to glow? It dumps graphs before/after each optimization, and before/after lower.
https://github.com/pytorch/glow/commit/9c31ac96ef6f097adcf206e7ac30137891a994dc

[bertmaher]

Hmm... I like having additional additional observability, but I also kind of feel like this is really heavyweight. If I turn on this option I'm going to get a metric ton of dot files, which gives me a lot to look at to find the one I'm interested in. I might prefer just editing the code to add a dumpDAG when I need it.

Do you envision end-users wanting this, or just Glow developers?

I'm curious what others think too. I'm open to persuasion.

[ayermolo]

This is primarily for developers to see how each optimization affects the graph. It's similar to the -print-before-all and -print-after-all in llvm, but combines them in to one.

One potential improvement can be ability to specify specific optimization, like DCE, or ALL to dump out all?

[nadavrot]

When I debug graph transformations I usually add calls to dump() by changing the source code in GraphOptimizer.cpp. It's really easy to insert a call to dumpDAG or even call dumpDAG in the debugger. The problem with dump-before and dump-after is that I am flooded with information that I can't process.

[jfix71]

I appreciate the idea. However one of my issues with it is that it will be hard to determine where each graph comes from during the optimization pipeline. Perhaps it could be nicer if a string name is given to each graph optimization that could be used in the name of the dot file. Or such a name could be used to specify what optimizations to write the dot files for before/after, instead of an all-or-nothing approach.

Another problem with the presented code is that we call optimize() many times during the pipeline. For example, optimize is called up to 4 times during optimizeFunction(), and at least twice. Ideally there would need to be some control there over when to call it exactly, as using the invocation counter as of right now makes it even harder to understand what's coming from where.

[ayermolo]

Hmm? It won't. There is monotonically increasing counter that is used and is appended to output name.
Convetion counter_optimizationName_{before,after}.dot

Not sure I follow the first part. Right now when outputDAG is involved a string is passed in. For my initial/internal implementation I just put the optimization name.

As I mentioned earlier, and you alluded to, this can be improved so that in command line we can specify specific optimization, or ALL. So if "DCE" is specified it will dump out dag each time DCE is executed.

[ayermolo]

So is it something that has potential, and just needs refinement. Like controlling which optimization is dumped, or completely useless?

[jfix71]

@ayermolo Sorry, I edited my comment right after I posted it 😄-- agreed it wouldn't be overwritten, but it would still be a little annoying to dump upwards of ~80 dot files and then determine where each comes from and where/when.

If you added a command line option to specify which optimizations to dump before/after it would be better. But it's also something relatively easy to just add into the code on demand. I don't have strong feelings here.

[opti-mix]

This issue seems to be related to #1659, that was never finished.

IMHO, finishing the re-factoring along the lines of #1659 could be a good next step and could be used to define optimization pipelines and to implement a fine-grained control over each optimization. This could include dumping graphs before/after all optimizations or only specific optimizations, skipping certain optimizations, stopping after certain optimizations, etc.

[qcolombet]

+1 on that @opti-mix said.
I thought we had an issue about that, but I can't find it.

[ayermolo]

Ah! yeah refactoring would be a way better way of doing this. Feel free to close this one.