Incubator-mxnet: Profiling compute time for individual operators in a graph

Created on 21 May 2016 · 7Comments · Source: apache/incubator-mxnet

Hello,

I'm trying to work out the compute times for every operator in my computation graph when I do a forward pass. Is it possible to obtain this information for GPU and CPU modes? If not, is it possible I can instrument the code in some way to retrieve this information?

Thanks,
Piyush3dB

Source

Piyush3dB

Most helpful comment

we currently don't have a profiler. You can try adding it in graph_executor.

piiswrong on 21 May 2016

👍2

All 7 comments

we currently don't have a profiler. You can try adding it in graph_executor.

piiswrong on 21 May 2016

👍2

Related to https://github.com/dmlc/mxnet/issues/406, tianqi has given some helpful instructions. I was not familiar with MXNet at that time so I had not continued.

sxjscience on 22 May 2016

👍1

ok, thanks for the pointer. @sxjscience, I'm not too familiar with that part of the MXNet code. I'll have a look and work out what to do. Did you have a better idea about it now? Thanks.

Piyush3dB on 22 May 2016

@Piyush3dB We can possibly insert timer in the execution logic. I need more tests to confirm the idea. Also, timing may not be trivial due to the asynchronous executions.

sxjscience on 22 May 2016

Also timing of operations executed on the gpu will need special consideration