Benchmarkdotnet: System.NotSupportedException: Line must start with GC

[mmisztal1980]

This is my 1st day using BenchmarkDotNet 0.11.3and I must say I'm liking the overall experience, however I've run into an issue that puzzles me.

Program.cs

class Program
    {
        static void Main(string[] args) => BenchmarkSwitcher
            .FromAssembly(typeof(Program).Assembly)
            .Run(args, DefaultConfig.Instance.With(new MemoryDiagnoser()));
    }

My benchmarked methods contain both the [Benchmark] and [MemoryDiagnoser] attributes.

[BenchmarkCategory("Logging")]
    [CoreJob]
    [MarkdownExporter, HtmlExporter, CsvExporter()]
    [SimpleJob(RunStrategy.Throughput, warmupCount: 1, launchCount: 1, targetCount: 1)]
    public class MyLoggingAsyncBenchmark
    {
        private const string ConfigFile = "logging-async-benchmark.json";
        public const string LogMessage = "Benchmark Log Message";

        [GlobalSetup]
        public void Setup()
        {
            var config = new ConfigurationBuilder()
                .AddJsonFile(ConfigFile)
                .Build();
            var logger = new LoggerConfiguration()
                .ConfigureSerilog(config)
                .CreateLogger();            
            Log.SetLoggerFactory(new LoggerFactory(logger));            // this is our custom log wrapper around serilog
        }

        [GlobalCleanup]
        public void Cleanup()
        {
        }       

        [Benchmark]
        [MemoryDiagnoser]
        public void LogFasitCoreLoggingInformation()
        {
            Log.Information<ILogBenchmark>().LogStuff();
        }
    }

When the benchmark suite is about to be finished, the process is crashing with the following exception.

Unhandled Exception: System.NotSupportedException: Line must start with GC:
   at BenchmarkDotNet.Engines.GcStats.Parse(String line)
   at BenchmarkDotNet.Running.BenchmarkRunner.Execute(ILogger logger, BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver)
   at BenchmarkDotNet.Running.BenchmarkRunner.RunCore(BenchmarkCase benchmarkCase, BenchmarkId benchmarkId, ILogger logger, ReadOnlyConfig config, IResolver resolver, BuildResult buildResult)

   at BenchmarkDotNet.Running.BenchmarkRunner.Run(BenchmarkRunInfo benchmarkRunInfo, Dictionary`2 buildResults, IResolver resolver, ILogger logger, List`1 artifactsToCleanup, String rootArtifactsFolderPath, StartedClock& runChronometer)
   at BenchmarkDotNet.Running.BenchmarkRunner.Run(BenchmarkRunInfo[] benchmarkRunInfos, IConfig commonSettingsConfig)
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
   at My.Benchmarks.Program.Main(String[] args) in C:\Checkout\My.Project\src\My.Benchmarks\Program.cs:line 9

I haven't yet found any mention of the behavior in the docs, so I'd apreciate an explanation on what is happening here

[adamsitnik]

Hello @mmisztal1980

I am surprised that nobody else has hit this bug so far.

Could you try to use [MemoryDiagnoser] not for the method, but for the type? (it's just a workaround for now)

Also, if your benchmarks projects targets only .NET Core you don't need to use [CoreJob]. And [GlobalSetup] does not require a [GlobalCleanup] (it's empty in your example, I am not sure if you did not remove some code for the brevity

[mail4andrey]

Same error when I've written something in console.
And well if I do not write to console

[morgan-kn]

Hello @mmisztal1980
Could you please provide here your output (BenchmarkDotNet.Artifacts/result/*.log)?

[adamsitnik]

@morgan-kn I just hit this bug and solved it. The problem was when the user was writing something to output in [GlobalCleanup] and we were trying to parse this output instead of the output written by MemoryDiagnoser

[morgan-kn]

Yeah, I found out this, too. I reproduced this by adding into benchmark itself this line:
AppDomain.CurrentDomain.ProcessExit += (sender, e) => Console.WriteLine("Test");
Just wanted to make sure that user had a redundant line in his output.
Thanks, @adamsitnik =)