Benchmarkdotnet: Console Tool throws System.Reflection.ReflectionTypeLoadException

Created on 19 May 2020  路  5Comments  路  Source: dotnet/BenchmarkDotNet

So when I am trying to run the following command in the console:

dotnet benchmark Benchmarks.dll --filter *

it throws the following error at me:

Unhandled Exception: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

with the following StackTrace:

at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeModule.GetTypes()
   at System.Reflection.Assembly.GetTypes()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Extensions\ReflectionExtensions.cs:line 131
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 34
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 92
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 67
   at BenchmarkDotNet.Tool.Program.OnExecute() in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 44
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.Invoke(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass138_0.<OnExecute>b__0()
   at BenchmarkDotNet.Tool.Program.Main(String[] args) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 26

C# version: 8
.Net version: .Net Preview 5 preview 3
Console Tool version: 0.12.1

picture TwentyFourMinutes
👍3

Most helpful comment

For thoose that arrive here randomly, as documented here. You can achieve the same through cli doing:

``` c#
using BenchmarkDotNet.Running;

namespace Your.Namespace.Benchmarks
{
public class Program
{
static void Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
}
}
```

dotnet run -c Release --filter *YourBenchmarkClass*

Seems like the BenchmarkDotNet.Tool indeed are redundant.

picture Andrioden on 31 Jul 2020
👍6

All 5 comments

Same issue with dotnet 3.1.

Installed with dotnet tool install --framework netcoreapp3.1 -g BenchmarkDotNet.Tool

>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.100-preview.4.20258.7
 Commit:    65f0fc2cad

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.100-preview.4.20258.7\

Host (useful for support):
  Version: 5.0.0-preview.4.20251.6
  Commit:  47ec733ba7

.NET SDKs installed:
  3.1.201 [C:\Program Files\dotnet\sdk]
  3.1.202 [C:\Program Files\dotnet\sdk]
  3.1.300 [C:\Program Files\dotnet\sdk]
  3.1.301 [C:\Program Files\dotnet\sdk]
  5.0.100-preview.4.20258.7 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0-preview.4.20257.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0-preview.4.20251.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0-preview.4.20251.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

I'm using documented sample to create dot net project as described here
dotnet benchmark BenchmarkTest.dll --filter *

> dotnet benchmark BenchmarkTest.dll --filter *
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeModule.GetTypes()
   at System.Reflection.Assembly.GetTypes()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Extensions\ReflectionExtensions.cs:line 131
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 34
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 92
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 67
   at BenchmarkDotNet.Tool.Program.OnExecute() in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 44
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.Invoke(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass138_0.<OnExecute>b__0()
   at BenchmarkDotNet.Tool.Program.Main(String[] args) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 26
vsevolodp picture vsevolodp on 19 Jun 2020

Hi!

I was never a fan of adding the benchmark global tool that loads everything dynamically and I am not suprised that it fails...

My suggestion is to stop using it and just create a console app, install BDN and use BenchmarkRunner or BenchmakSwitcher and regular dotnet run to run it:

using System;
using System.Security.Cryptography;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

namespace MyBenchmarks
{
    public class Md5VsSha256
    {
        private const int N = 10000;
        private readonly byte[] data;

        private readonly SHA256 sha256 = SHA256.Create();
        private readonly MD5 md5 = MD5.Create();

        public Md5VsSha256()
        {
            data = new byte[N];
            new Random(42).NextBytes(data);
        }

        [Benchmark]
        public byte[] Sha256() => sha256.ComputeHash(data);

        [Benchmark]
        public byte[] Md5() => md5.ComputeHash(data);
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            var summary = BenchmarkRunner.Run(typeof(Program).Assembly);
        }
    }
}

dotnet run -c Release
adamsitnik picture adamsitnik on 19 Jun 2020
👍1

@CodeTherapist Maybe you have time to check it? I think you made the first version of the global tool?

@adamsitnik I agree with you that global tool has no added value and does not make work easier. But I believe this problem can be fixed. But to be honest, I'm not going to spend any private time on it, I can prepare PR to remove this tool 馃槃

WojciechNagorski picture WojciechNagorski on 17 Jul 2020

For thoose that arrive here randomly, as documented here. You can achieve the same through cli doing:

``` c#
using BenchmarkDotNet.Running;

namespace Your.Namespace.Benchmarks
{
public class Program
{
static void Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
}
}
```

dotnet run -c Release --filter *YourBenchmarkClass*

Seems like the BenchmarkDotNet.Tool indeed are redundant.

Andrioden picture Andrioden on 31 Jul 2020
👍6

Are there any plans to remove the page from the documentation?
Would such PR be welcome?

quixoticaxis picture quixoticaxis on 8 Oct 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

DanielLoth picture DanielLoth  路  4Comments
steveoh picture steveoh  路  4Comments
kirsan31 picture kirsan31  路  4Comments
DanielLoth picture DanielLoth  路  3Comments
ig-sinicyn picture ig-sinicyn  路  3Comments