Benchmarkdotnet: How to read the results

[daybson]

My class MyBenchmarkContainer contains 10 [Benchmark] methods. When I execute only the first method and comment the others, it gives me a time 20ns. When I execute all 10 methods, the first one becomes 100x slower, like 20.000,00ns. Why is this happening? The methods do not call each other. I'm using all default configurations.

[adamsitnik]

Hello @daybson

Could you upload your results for both runs to https://gist.github.com/? They are in BenchmarkDotNet.Artifacts folder.

It seems like a formatting issue (it's always 20ns, but when you have more benchmarks they are scaled to common time unit)

[daybson]

You are right. It's the formating. Kind of anoying because it gives me 19,260.6 ns in the .csv, and the comma thousand is not well recognized on Excel when I change the cell from Generic to Number. The better approach should be 19.260,6 ns, or remove for once the thousand digit separator.

Other doubt: what does it means the column Gen0, Gen1? And where can I find more explanation about each kind of column? I'm not seeing in the documentation... I also would like to see more examples of custom jobs and configurations...

[adamsitnik]

The problem with csv is a never ending story. I believe that we use the right separators.

Other doubt: what does it means the column Gen0, Gen1?

I described it on my blog

And where can I find more explanation about each kind of column?

At the end of runs we display a legend:

// * Legends *
  Mean      : Arithmetic mean of all measurements
  Error     : Half of 99.9% confidence interval
  StdDev    : Standard deviation of all measurements
  Gen 0     : GC Generation 0 collects per 1k Operations
  Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  1 ns      : 1 Nanosecond (0.000000001 sec)

I also would like to see more examples of custom jobs and configurations...

I can recommend you this project, my blog and Andrey's blog or Kestrel benchmarks

[AndreyAkinshin]

I will check the csv exporter.

[Matthew-Bonner]

It would be good if we could also change ns to ms as I'm seeing huge numbers that are hard to read.

[adamsitnik]

Closing as it has been possible for a while to set the time unit.

var summaryStyle = new SummaryStyle
{
    PrintUnitsInHeader = true,
    PrintUnitsInContent = false,
    SizeUnit = SizeUnit.B,
    TimeUnit = TimeUnit.Millisecond
};

config = config.With(summaryStyle); // or ManualConfig.Set(summaryStyle)