BenchmarkDotNet doesn't understand netcoreapp2.0

[davidfowl]

I'm not sure how the generated csproj stuff works but it's broken when targeting .NET Core 2.0 (from myget).

// BinariesDirectoryPath = C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\bin\Release\netcoreapp2.0

// *** Build ***
// dotnet restore took 3.3s
// Result = Failure
// Exception:   Restoring packages for C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\BenchmarkDotNet.Autogenerated.csproj...
C:\Users\dfowler\AppData\Local\Microsoft\dotnet\sdk\1.0.0-rc4-004847\NuGet.targets(97,5): error : Unable to resolve 'Microsoft.NETCore.App (>= 2.0.0)' for '.NETCoreApp,Version=v2.0'. [C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\BenchmarkDotNet.Autogenerated.csproj]
  Generating MSBuild file C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\obj\BenchmarkDotNet.Autogenerated.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\obj\BenchmarkDotNet.Autogenerated.csproj.nuget.g.targets.
  Writing lock file to disk. Path: C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\obj\project.assets.json
  Restore failed in 2.08 sec for C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\BenchmarkDotNet.Autogenerated.csproj.

  Errors in C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\bin\Release\netcoreapp2.0\BDN.Generated\BenchmarkDotNet.Autogenerated.csproj
      Unable to resolve 'Microsoft.NETCore.App (>= 2.0.0)' for '.NETCoreApp,Version=v2.0'.

  NuGet Config files used:
      C:\Users\dfowler\Documents\GitHub\KestrelHttpServer\NuGet.Config
      C:\Users\dfowler\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://dotnet.myget.org/F/aspnetcore-ci-dev/api/v3/index.json
      https://dotnet.myget.org/F/aspnetcore-tools/api/v3/index.json
      https://dotnet.myget.org/F/dotnet-core/api/v3/index.json
      https://ci.appveyor.com/nuget/benchmarkdotnet
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

There might be some bad assumptions about mapping the TFM to the version of Microsoft.NETCore.App being targeted.

Here's the csproj file

<Project Sdk="Microsoft.NET.Sdk">

  <Import Project="..\..\build\common.props" />

  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <OutputType>Exe</OutputType>
    <ServerGarbageCollection>true</ServerGarbageCollection>
    <IsPackable>false</IsPackable>
    <NetCoreAppImplicitPackageVersion>2.0.0-beta-001607-00</NetCoreAppImplicitPackageVersion>
    <RuntimeFrameworkVersion>2.0.0-beta-001607-00</RuntimeFrameworkVersion>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="..\shared\TestApplicationErrorLogger.cs" />
    <Compile Include="..\shared\TestFrame.cs" />
    <Compile Include="..\shared\TestKestrelTrace.cs" />
    <Compile Include="..\shared\MockConnection.cs" />
    <Compile Include="..\shared\MockSocketOutput.cs" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\Microsoft.AspNetCore.Server.Kestrel\Microsoft.AspNetCore.Server.Kestrel.csproj" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.10.2.*" />
    <PackageReference Include="Moq" Version="4.6.36-*" />
  </ItemGroup>

</Project>

[davidfowl]

So the problem is the missing:

<NetCoreAppImplicitPackageVersion>2.0.0-beta-001607-00</NetCoreAppImplicitPackageVersion>
<RuntimeFrameworkVersion>2.0.0-beta-001607-00</RuntimeFrameworkVersion>

In the generated csproj file.

[eerhardt]

Technically you only need the "RuntimeFrameworkVersion" property with the latest tools.

[AndreyAkinshin]

Hey @davidfowl, thanks for the bug report. Could you also create a branch with the repro?
@adamsitnik, could you take a look?

P.S. @davidfowl, it's very cool that you decided to try BenchmarkDotNet in Kestrel. If you miss any features, feel free to create feature requests. =)

[adamsitnik]

@davidfowl thanks for the bug report, it's great that you did it before the new version release so I can fix it and it will be included.

I have cloned the Kestrel repo and built the master branch. Then I applied the changes to the csproj and it fails to build (error below). Do I need to install something extra to be able to build it? I would like to be able to repoduce it and test on my own.

C:\Projects\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance>dotnet build
Microsoft (R) Build Engine version 15.1.545.13942
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Program Files\dotnet\sdk\1.0.0-rc4-004771\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(154,5): error : Unexpected dependency 'Microsoft.NETCore.App' with no version number. [C:\Projects\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\Microsoft.AspNetCore.Server.Kestrel.Performance.csproj]
C:\Program Files\dotnet\sdk\1.0.0-rc4-004771\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(154,5): error : Unexpected dependency 'Microsoft.NETCore.App' with no version number. [C:\Projects\KestrelHttpServer\test\Microsoft.AspNetCore.Server.Kestrel.Performance\Microsoft.AspNetCore.Server.Kestrel.Performance.csproj]

Technically you only need the "RuntimeFrameworkVersion" property with the latest tools.

@eerhardt are the wildcards supported? I don't want to hardcode any versions in BenchmarkDotNet, especially for things like this one which change version every day. Can I just set

<RuntimeFrameworkVersion>2.0.0-*</RuntimeFrameworkVersion>

[davidfowl]

Here's a guide on how to get up and running on .NET Core 2.0:

https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/dogfooding.md#how-to-get-up-and-running-on-net-core-20

Here's an ASP.NET Sample:
https://github.com/davidfowl/AspNetCoreNextOnNetCore2

Here's a branch of Kestrel with most things setup:

https://github.com/aspnet/KestrelHttpServer/tree/davidfowl/dotnet-core-20

You should be able to use the RC4 build of the .NET Core SDK to reproduce the issue.

[eerhardt]

@eerhardt are the wildcards supported?

Wildcards are supported in builds after RC4. Here's the PR that enabled them: https://github.com/dotnet/sdk/pull/820, which didn't make it into RC4.

There are issues with using wildcards when your app uses a "Framework Dependent" deployment (i.e. shared framework). You need to have the same runtime version (or higher) installed on your machine, so say the latest version is 2.0.0-1234 and you install that version on your machine, everything works fine. Then a new build of .NET Core 2.0 comes out: 2.0.0-1235. Restoring the project will pick up the new 1235 build, but no one installed that build on the machine. So running the app then fails.

One thought: Maybe instead of generating the .csproj yourself, you could make your own dotnet new template to generate the project? Then you can use the same infrastructure we have for dotnet new console, which allows the tooling to generate the current shared framework version into the .csproj without having to hard-code it in our code.
The way that works is through these template.config files, which allows us to replace values in the .csproj file. See https://github.com/dotnet/templating/blob/master/template_feed/Microsoft.DotNet.Common.ProjectTemplates.2.0/content/ConsoleApplication-CSharp/.template.config/template.json#L20-L25

[eerhardt]

Here's some info on creating templates, in case you want to explore that route: https://github.com/dotnet/templating/wiki/%22Runnable-Project%22-Templates

[adamsitnik]

@davidfowl Thanks for the explanation. I have fixed the bug, you can now upgrade to 0.10.2.373 (Kestrel has our CI feed on it's nuget list)

Please let me know if you encounter any other problems.

@eerhardt thanks for the suggestion, I will think about it. We now simply copy the two missing settings if they are present in the csproj of the project that defines the benchmarks.

[davidfowl]

Works like a charm now. Thanks for the quick turnaround!

[AndreyAkinshin]

@davidfowl, great! New stable version is coming soon!