Benchmarkdotnet: Documentation for .NET Core/Mono version?

[arthrp]

There are really nice docs for 'regular' .NET version, but I failed to find any for .NET Core version. Would be nice to have some simple usage samples, as API seems to be different.

[adamsitnik]

Hello @arthrp

The difference for apis is very small, for the .NET Core we don't support:

• BenchmarkRunner.RunUrl
• BenchmarkRunner.RunSource

For both .NET Core and Mono we don't support

• InliningDiangoser (based on Windows ETW)
• MemoryDiagnoser (based on Windows ETW, will change in upcoming 0.10.1)

As for samples you can take a look to our Samples projects

Please let me know if still there is something unclear for you.

[arthrp]

Ok, thanks, will take a look. My initial problem was that Getting started example doesn't work with .NET Core version. So one has to do more digging to discover the working sample.

[AndreyAkinshin]

@arthrp, it should work. Look at the BenchmarkDotNet.Samples application in the source code: it's a .NET Core project (based on project.json for now) which targets both net45 and netcoreapp1.1. And it includes the Md5VsSha256 example.
Probably, it's a good idea to have a set of predefined templates for different kinds of projects. @adamsitnik, what to you think?

[adamsitnik]

Probably, it's a good idea to have a set of predefined templates for different kinds of projects. @adamsitnik, what to you think?

Do you mean a template for .csproj for Visual Studio with some sample code?

[AndreyAkinshin]

Yep.

[arthrp]

@AndreyAkinshin Sorry for the late response :) Re-reading my comments I now have realized I didn't describe the issue properly.
I'm running Mono, so the only flavor I can run with it is .NET Core one (I can't even add the regular version).

Official guide states, to run the simple one you must do this:

var summary = BenchmarkRunner.Run<Md5VsSha256>();

No such class in .NET Core version :) There is a BenchmarkRunnerCore, however. Which doesn't have the same method signature.

Ok, looking through BenchmarkDotNet.Samples. It uses the BenchmarkSwitcher, which again is lacking in .NET Core version. Looking into project.json, and... it seems like it references the full version, not Core one.

  "dependencies": {
    "BenchmarkDotNet": {
      "target": "project"
}

I guess nobody's going to make samples tailored for Mono, because Mono is legacy now. But still would be nice to have some compatible samples, which would show some basic examples.

[adamsitnik]

@arthrp I believe that we have misunderstanding due to our packages naming convention and lack of comments at nuget.org

Our end users should install only BenchmarkDotNet (which exposes BenchmarkRunner and BenchmarkSwitcher classes) or BenchmarkDotNet.Diagnostics.Windows`.

Internally all packages use BenchmarkDotNet.Core which has nothing to do with .NET Core, it is just the core of benchmarkdonet which is not dependent to Roslyn compiler.

Could you please let us know what error you get when trying to install BenchmarkDotNet on Mono?

@AndreyAkinshin we need to set description "not for direct usage, please installl BenchmarDotNet package" to BenchmarkDotNet.Core and BenchmarkDotNet.Toolchains.Roslyn

[AndreyAkinshin]

@adamsitnik, yes, we should update the description. But I prefer something like "Core BenchmarkDotNet logic for all target frameworks". What do you think?

[adamsitnik]

@AndreyAkinshin ok, but let's add to it sth like "Do not install this package manually, it will be added as a prerequisite by other packages that require it." like Roslyn does so we can avoid this kind of problems in the future

• in the docs we should state it explicitly that there is single package for all frameworks, which takes care of all necessary things internally and that users don't need to worry about that

[AndreyAkinshin]

Ok, LGTM.

[arthrp]

@adamsitnik
This is the error when installing latest BenchmarkDotNet (0.10.1):

Adding BenchmarkDotNet...
Attempting to resolve dependency 'BenchmarkDotNet.Toolchains.Roslyn (≥ 0.10.1)'.
Downloading BenchmarkDotNet.Toolchains.Roslyn 0.10.1...
Attempting to resolve dependency 'BenchmarkDotNet.Core (≥ 0.10.1)'.
Attempting to resolve dependency 'System.Threading.Tasks.Extensions (≥ 4.0.0)'.
Attempting to resolve dependency 'Microsoft.CodeAnalysis.CSharp (≥ 1.3.2)'.
Attempting to resolve dependency 'Microsoft.CodeAnalysis.Common (= 1.3.2)'.
Attempting to resolve dependency 'System.Collections.Immutable (≥ 1.1.37)'.
Attempting to resolve dependency 'System.Reflection.Metadata (≥ 1.2.0)'.
Downloading System.Reflection.Metadata 1.2.0...
'System.Reflection.Metadata' already has a dependency defined for 'System.Collections.Immutable'.

[adamsitnik]

@arthrp I am 100% sure that it's a nuget issue. Could you update your nuget client version and try once again? Similar issue

[arthrp]

@adamsitnik sorry for the late response again. They talk about Nuget 2.8.3 in the issue (I have 2.12 which is latest in Mono repository). As far as I'm concerned there is no simple way to get latest Nuget mono binaries. Guess that means that it's incompatible with current Mono version :)

[adamsitnik]

@AndreyAkinshin what is your experience with our Mono support on Linux today? Can you install latest BDN?

[AndreyAkinshin]

In 2016, we had a lot of problems on Linux and macOS (especially on preview versions of .NET Core). Right now all basic use cases work fine, main questions about support of different features on different OS are covered in the documentation.

[spixy]

@arthrp I believe that we have misunderstanding due to our packages naming convention and lack of comments at nuget.org

Our end users should install only BenchmarkDotNet (which exposes BenchmarkRunner and BenchmarkSwitcher classes) or BenchmarkDotNet.Diagnostics.Windows`.

Internally all packages use BenchmarkDotNet.Core which has nothing to do with .NET Core, it is just the core of benchmarkdonet which is not dependent to Roslyn compiler.

Could you please let us know what error you get when trying to install BenchmarkDotNet on Mono?

@AndreyAkinshin we need to set description "not for direct usage, please installl BenchmarDotNet package" to BenchmarkDotNet.Core and BenchmarkDotNet.Toolchains.Roslyn

So either update nuget description or rename that package because I just lost half an hour how to run in with BenchmarkDotNet.Core. Thank you

[adamsitnik]

@spixy The package BenchmarkDotNet.Core no longer exists. It's available on NuGet to support old versions of BenchmarkDotNet.

As the end user, you should always just install BenchmarkDotNet

If you don't like the description, send a PR to make it better.

[idormenco]

Hi, @adamsitnik can we add in description of BenchmarkDotNet.Core that is deprecated and specify that user should use BenchmarkDotNet.

Thanks

[AndreyAkinshin]

@idormenco done:

By the way, NuGet team is working on a feature called "Deprecate packages" https://github.com/NuGet/Home/wiki/Deprecate-packages As soon as they finish it, I'm going to deprecate BenchmarkDotNet.Core via the new NuGet Gallary mechanism.

[idormenco]

Hi @AndreyAkinshin indeed is written in documentation on nuget page but many devs will search for this packet in nuget browser from visual studio and this is what they will see:

my idea was to write this in description.
I hope nuget will have that feature ASAP
Thanks for quick reply .

Best regards,
Ion

[AndreyAkinshin]

@idormenco
BenchmarkDotNet (without the .Core suffix) should be on the first place:

I can't change the description of the existed package. And I'm not going to upload "fake" version of BenchmarkDotNet.Core just to update description of the latest version.

[idormenco]

@AndreyAkinshin yeah indeed this is unimportant issue, sorry for bothering you.