Benchmarkdotnet: Question - This benchmark apparently allocates, but why?

[DanielLoth]

Hi everyone,

I wrote a benchmark to test iterating over integer arrays using various approaches (mostly a benchmark of the effects of cache locality).

Benchmark can be found here: https://github.com/Lothy/DotNetBenchmarks/blob/master/src/NET46/ArrayBenchmarks.cs

Here are the results (noting that, for this run of the benchmarks, I altered the fields to be static to see if it impacted the result):

BenchmarkDotNet=v0.10.5, OS=Windows 10.0.14393
Processor=Intel Core i7-2600K CPU 3.40GHz (Sandy Bridge), ProcessorCount=8
Frequency=3331181 Hz, Resolution=300.1938 ns, Timer=TSC
  [Host]     : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1637.0
  DefaultJob : Clr 4.0.30319.42000, 32bit LegacyJIT-v4.6.1637.0

| Method | Mean | Error | StdDev | StdErr | Min | Q1 | Median | Q3 | Max | Op/s | Scaled | ScaledSD | Allocated |
|---------------------- |-----------:|----------:|-----------:|----------:|-----------:|-----------:|-----------:|-----------:|-----------:|--------:|-------:|---------:|----------:|
| OneDimensionOneLoop | 532.2 us | 10.161 us | 11.701 us | 2.617 us | 524.8 us | 525.3 us | 526.4 us | 535.8 us | 566.5 us | 1,879.0 | 0.35 | 0.01 | 7 B |
| OneDimensionA | 568.2 us | 7.510 us | 7.025 us | 1.814 us | 559.7 us | 560.2 us | 569.3 us | 574.6 us | 580.0 us | 1,760.0 | 0.38 | 0.00 | 7 B |
| OneDimensionB | 708.1 us | 13.532 us | 13.896 us | 3.370 us | 694.8 us | 695.6 us | 705.0 us | 715.6 us | 740.7 us | 1,412.2 | 0.47 | 0.01 | 7 B |
| TwoDimensionsD1ThenD2 | 1,499.5 us | 8.701 us | 8.139 us | 2.101 us | 1,490.5 us | 1,492.6 us | 1,497.2 us | 1,504.1 us | 1,520.5 us | 666.9 | 1.00 | 0.00 | 15 B |
| ThreeDimensionsD1D2D3 | 2,324.6 us | 53.898 us | 97.190 us | 15.179 us | 2,257.3 us | 2,261.6 us | 2,285.3 us | 2,349.7 us | 2,619.5 us | 430.2 | 1.55 | 0.06 | 28 B |
| ThreeDimensionsD3D2D1 | 3,765.9 us | 76.435 us | 154.403 us | 21.836 us | 3,562.2 us | 3,659.4 us | 3,715.4 us | 3,809.0 us | 4,292.7 us | 265.5 | 2.51 | 0.10 | 58 B |
| TwoDimensionsD2ThenD1 | 6,030.0 us | 16.421 us | 13.712 us | 3.803 us | 6,002.6 us | 6,020.1 us | 6,031.8 us | 6,042.1 us | 6,050.3 us | 165.8 | 4.02 | 0.02 | 55 B |

Question: Why does MemoryDiagnoser detect an allocation occurring during this benchmark? It even detects the allocation when the array fields are static.

Is there something subtle I'm overlooking?

Thank you for reading my question.

[adamsitnik]

Hello, @Lothy !

Thanks for reporting the issue. Your benchmark should not allocate. It looks like a bug on our side. I will verify it tomorrow.

[adamsitnik]

@Lothy It was a bug. Once again thanks for providing bug report with good description!

I have fixed it, you can grab 0.10.5.169 NuGet package from or CI dev feed.

<packageSources>
  <add key="bdn-nightly" value="https://ci.appveyor.com/nuget/benchmarkdotnet" />
</packageSources>

To whom it may concern: the bug was visible for micro benchmarks, not nanobenchmarks. We had single allocation in our Engine. It has been removed and our integration tests has been changed to prevent from such situation in the future.

[DanielLoth]

No worries. Thanks for the quick fix. :)

Have a great weekend!