Benchmarkdotnet: Add new template for "dotnet new benchmark"

[adamsitnik]

I think that we should add a new template for dotnet cli that would create following app when used:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

namespace BenchmarkApp
{
    class Program
    {
        static void Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
    }

    public class TypeWithBenchmarks
    {
        [Benchmark]
        public int YourBenchmark()
        {
            // implement your benchmark here
            return 0;
        }
    }
}

@AndreyAkinshin what do you think?

[AndreyAkinshin]

@adamsitnik I like this idea!

[adamsitnik]

btw the .csproj should be:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>$whatUserHasAskedFor</TargetFramework>
    <OutputType>Exe</OutputType>
    <PlatformTarget>AnyCPU</PlatformTarget>
    <DebugType>pdbonly</DebugType>
    <DebugSymbols>true</DebugSymbols>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <Optimize>true</Optimize>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.1*" />
    <PackageReference Include="BenchmarkDotNet.Diagnostics.Windows" Version="0.1*" />
  </ItemGroup>

</Project>

[CodeTherapist]

I would like to do this :)

[adamsitnik]

@CodeTherapist awesome! please let me know if you need anything

[dzmitry-lahoda]

Would it be possible to have template for F# and with more code (i.e. template with examples) like next https://github.com/TheAngryByrd/BenchmarkDotNet.Template/issues/3 ?

[TheAngryByrd]

I'd be happy to just port the code I have in that repo to something under here as it will get more love and attention.

[JamieKeeling]

I've written a template or two before and would love to help.

Before I get started could you provide some basic information?

1. Where would you prefer the template content to be stored within the repository? Alternatively would a separate repository be better and ownership transferred.
2. Templating supports file system or Nuget installation. File system is a low bar of entry but Nuget is ultimately more flexible and easier for consumers albeit requires CI/CD to build the package. What's your preference?

[Kavignon]

How can I be of help here?

[AndreyAkinshin]

@Kavignon we already have a pull request by @CodeTherapist with basic template support: https://github.com/dotnet/BenchmarkDotNet/pull/1044 However, I raised some concerns about the current PR status: https://github.com/dotnet/BenchmarkDotNet/pull/1044#pullrequestreview-261539007 . I think it will be better to continue discussion there and finish the PR.
@CodeTherapist do you need any help with the PR?

[CodeTherapist]

@AndreyAkinshin No help needed, thanks - I will finish it this week(end).

[AndreyAkinshin]

@CodeTherapist awesome, thanks!

[am11]

@AndreyAkinshin, when would the official template package made available on nuget.org?

e.g.

• https://www.nuget.org/packages/NUnit3.DotNetNew.Template/
• https://www.nuget.org/packages/giraffe-template/

and unofficial packages of BenchmarkDotNet:

• https://www.nuget.org/packages/AtleX.BenchmarkDotNet.Template.CSharp/
• https://www.nuget.org/packages/TheAngryByrd.BenchmarkDotNet.Templates/

[AndreyAkinshin]

@am11 official template was published a few days ago: https://www.nuget.org/packages/BenchmarkDotNet.Templates/

[TheAngryByrd]

I just marked my github repo as archived and my nuget package as deprecated.

[AndreyAkinshin]

@TheAngryByrd great, thanks!

[am11]

Thank you @AndreyAkinshin! 🎉

[maybe not a huge deal but..]
IMHO, the template should be in a runnable state with these additions:

Click to expand

--- a/Benchmarks.cs
+++ b/Benchmarks.cs
using System;
using BenchmarkDotNet;
using BenchmarkDotNet.Attributes;
+using BenchmarkDotNet.Running;

 namespace Sample
 {
+    class Program
+    {
+        static void Main() => BenchmarkRunner.Run<Benchmarks>();
+    }
+
     public class Benchmarks
     {
         [Benchmark]

--- a/Sample.csproj
+++ b/Sample.csproj
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
-    <TargetFrameworks>netstandard2.0</TargetFrameworks>
+    <TargetFramework>netcoreapp3.0</TargetFramework>
+    <OutputType>Exe</OutputType>
   </PropertyGroup>
   <PropertyGroup>
     <PlatformTarget>AnyCPU</PlatformTarget>

So the getting started steps would look like:

dotnet new benchmark --name Sample
cd Sample
dotnet run

[TheAngryByrd]

there's a CLI flag for that with the template --console-app

dotnet new benchmark --console-app

I know I missed it the first time. I could argue for it being the default too :)

[CodeTherapist]

Thanks @am11 and @TheAngryByrd for your feedback and question. I'm happy to help.
The template has this "runnale state" as an option when you create the new project (docs):

dotnet new benchmark --name Sample --console-app
cd Sample
dotnet run

Alternatively, you could install and use the the global BenchmarkDotNet tool and invoke the benchmarks from the CLI (docs).

BenchmarkDotnet will strengthen it's integration with leading IDE's in the future to provide better user experience and workflow - similiar to unit tests.
Neverless, any feedback is appreciated and will be considered for further releases.

[am11]

Great! Thanks for your help and documentation. 👍