Glow: Consider using google benchmark for GemmBench

Created on 11 Sep 2018  路  5Comments  路  Source: pytorch/glow

We currently have a handrolled framework for benchmarking matrix multiplication, and it might be valuable to use a more rigorous framework, especially as we add more benchmarks. (It might also be kind of annoying, hence the title of this issue starting with "Consider" ;-) ).

All 5 comments

Another consideration I had in mind is that since we'd need to run the benchmark on fbcode, I'd look into folly benchmark for easier integration down the road.

See here: https://our.intern.facebook.com/intern/wiki/Cpp/testing--benchmarking/#benchmarking
Only available with internal access.

@rdzhabarov Before we add a new dependency to the project let's talk about the requirements. What problem are we trying to solve here?

@nadavrot I'm not proposing to add a dependency, but if we do, it would be worthwhile considering commonly used internal alternatives to google benchmark.

The more I think about this task, the more I'm leaning against it, especially now that I realize our internal benchmarking setup uses folly. I really don't want to take a dependency on folly.

Before seeing these comments, I spent a little time looking into Google benchmark and wasn't convinced it was a good idea for Glow. After talking with Bert, we've decided against this approach.

Was this page helpful?
0 / 5 - 0 ratings