Benchmarkdotnet: Allocations for async methods measures BenchmarkDotNet

[benaadams]

e.g.

public class AsyncBenchmark
{
    [Benchmark]
    public Task NoAllocs()
    {
        return Task.CompletedTask;
    }
}

Will output

    Method |      Mean |    StdErr |    StdDev |         Op/s |  Gen 0 | Allocated |
---------- |---------- |---------- |---------- |------------- |------- |---------- |
  NoAllocs | 5.4809 ns | 0.0787 ns | 0.4311 ns | 182451067.11 | 0.0000 |      64 B |

Suggesting the method allocates 64B; when it allocates 0

[adamsitnik]

thanks! will fix within few days

[adamsitnik]

I am able to reproduce. Finally some interesting bug!

public class Runnable
{
    [Benchmark] public Task Completed() => Task.CompletedTask;
}

public static void ExecuteBlocking(Func<Task> future) 
    => future.Invoke().GetAwaiter().GetResult();

public static void Main(string[] args)
{
    var runnable = new Runnable();

    for (int i = 0; i < 100000; i++)
    {
        ExecuteBlocking(runnable.Completed);
    }
}

is translated to:

public static int Main(string[] args)
{
    Runnable runnable = new Runnable();
    for (int i = 0; i < 100000; i++)
    {
        Program.ExecuteBlocking(new Func<Task>(runnable.Completed));
    }
    return 0;
}

notice the new Func<>

[adamsitnik]

@benaadams thanks for reporting the bug. It was quite serious! I fixed it, 0.10.3.90+ is now bug-free.

[benaadams]

Thank you :)

[AndreyAkinshin]

@benaadams, thanks for the bug report!
@adamsitnik, thanks for the fix, it looks great! However, I still have some concerns about async benchmarks, see https://github.com/dotnet/BenchmarkDotNet/issues/418