We currently have a handrolled framework for benchmarking matrix multiplication, and it might be valuable to use a more rigorous framework, especially as we add more benchmarks. (It might also be kind of annoying, hence the title of this issue starting with "Consider" ;-) ).
Another consideration I had in mind is that since we'd need to run the benchmark on fbcode, I'd look into folly benchmark for easier integration down the road.
See here: https://our.intern.facebook.com/intern/wiki/Cpp/testing--benchmarking/#benchmarking
Only available with internal access.
@rdzhabarov Before we add a new dependency to the project let's talk about the requirements. What problem are we trying to solve here?
@nadavrot I'm not proposing to add a dependency, but if we do, it would be worthwhile considering commonly used internal alternatives to google benchmark.
The more I think about this task, the more I'm leaning against it, especially now that I realize our internal benchmarking setup uses folly. I really don't want to take a dependency on folly.
Before seeing these comments, I spent a little time looking into Google benchmark and wasn't convinced it was a good idea for Glow. After talking with Bert, we've decided against this approach.