Benchmarkdotnet: FileLoadException for v0.11.4 & v0.11.5 in LINQPad

[Turnerj]

In both versions 0.11.4 and 0.11.5 (works fine in 0.11.3), this benchmark fails in LINQPad (v5.36.03) with FileLoadException: Assembly is still being loaded. (Exception from HRESULT: 0x80131016) with the following stack trace:

   at BenchmarkDotNet.Portability.RuntimeInformation.get_IsNetCore()
   at BenchmarkDotNet.Toolchains.ToolchainExtensions.GetToolchain(Runtime runtime)
   at BenchmarkDotNet.Toolchains.ToolchainExtensions.GetToolchain(Job job)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.<>c__DisplayClass12_0.<GetSupportedBenchmarks>b__2(BenchmarkCase benchmark)
   at System.Linq.Enumerable.WhereArrayIterator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.<>c__DisplayClass12_0.<GetSupportedBenchmarks>b__0(BenchmarkRunInfo info)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.GetSupportedBenchmarks(BenchmarkRunInfo[] benchmarkRunInfos, ILogger logger, IResolver resolver)
   at BenchmarkDotNet.Running.BenchmarkRunnerClean.Run(BenchmarkRunInfo[] benchmarkRunInfos)
   at BenchmarkDotNet.Running.BenchmarkRunner.RunWithDirtyAssemblyResolveHelper(Type type, IConfig config)
   at BenchmarkDotNet.Running.BenchmarkRunner.<>c__DisplayClass0_0`1.<Run>b__0()
   at BenchmarkDotNet.Running.BenchmarkRunner.RunWithExceptionHandling(Func`1 run)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run[T](IConfig config)
   at UserQuery.Main()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

I had a look at the source code and history of that file and while there were changes to that file in 0.11.4, I can run the two lines of code for the IsNetCore directly in LINQPad and it works fine.

Benchmark code in LINQPad (Note: this is a ClrJob, not a CoreJob):

void Main()
{
    var summary = BenchmarkRunner.Run<Md5VsSha256>();
}

// Define other methods and classes here
[ClrJob]
public class Md5VsSha256
{
    private SHA256 sha256 = SHA256.Create();
    private MD5 md5 = MD5.Create();
    private byte[] data;

    [Params(1000, 10000)]
    public int N;

    [GlobalSetup]
    public void Setup()
    {
        data = new byte[N];
        new Random(42).NextBytes(data);
    }

    [Benchmark]
    public byte[] Sha256() => sha256.ComputeHash(data);

    [Benchmark]
    public byte[] Md5() => md5.ComputeHash(data);
}

[AndreyAkinshin]

@Turnerj I wasn't able to reproduce your problem, BenchmarkDotNet v0.11.5 works fine in LINQPad 5.36.03 on all of my Windows machines:

[AndreyAkinshin]

Aha, I found a discussion about the same problem on the LINQPad forum: https://forum.linqpad.net/discussion/1873/benchmarkdotnet-in-linqpad-anycpu-throws-an-exception It seems that this issue is relevant only for AnyCPU-builds of LINQPad

[Turnerj]

Ahhhh, OK. Should this be put down to a bug in LINQPad then or do you think there still might be an issue on the BDN end?

Happy if you want to close it if it is unrelated. A little strange it works fine with 0.11.3 though.

[adamsitnik]

@Turnerj I would prefer to close this issue, we have clear info about this in our FAQ