Benchmarkdotnet: Directory.Build.props hijacking generated csproj output path

[olanat]

In our build system we use a Directory.Build.props file to place the binaries in a target folder outside the source tree. The structure is something like:

Directory.Build.props
src/...
test/Directory.Build.props
test/benchmarks
target/src/...
target/test/benchmarks...

Benchmark.NET generates a csproj file under target/test and builds this using these commands:
call dotnet restore /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1
call dotnet build -c Release /p:UseSharedCompilation=false /p:BuildInParallel=false /m:1

It assume that the output of this goes to "bin\Release" in the same folder as the generated csproj file, but our Directory.Build.props overrides this and benchmark can't find the build result.

If Benchmark.NET generates either a csproj file won't get redirected by "Directory.Build.props" or its own "Directory.Build.props" override like this would not happen, regardless if the user has a special build setup (like we do).

[adamsitnik]

hello @olanat

We have faced the very same problem in the https://github.com/dotnet/performance/ repository

As a workaround, we specify the ArtifactsPath in an explicit way:

https://github.com/dotnet/performance/blob/6dc7cb17edeee4d8aef9b4514196a3f5504438ce/src/benchmarks/micro/Program.cs#L40

https://github.com/dotnet/performance/blob/6dc7cb17edeee4d8aef9b4514196a3f5504438ce/src/harness/BenchmarkDotNet.Extensions/RecommendedConfig.cs#L39

We are working on a proper fix of this problem: https://github.com/dotnet/BenchmarkDotNet/issues/1403 but I am not sure when the fix will be available, so please use the workaround for now.

Thanks,
Adam

[ekx77]

Hi @adamsitnik
I have same problem (Directory.build.props overrides BenchmarkDotNet.Autogenerated.csproj), so benchmark fails with message
Did not find dll, but folder contained dll.config
I tried workaround from above:
var config = DefaultConfig.Instance.WithArtifactsPath(@"c:\temp\benchmark"); BenchmarkRunner.Run<TBenchmark>(config);
but no luck - only logs go to ArtifactsPath, benchmark itself still looks for dll in project subfolder bin\Release\netcoreapp3.1.
Any idea how to solve that?
Thank you.