Sdk: Add parameter to ensure tests run in sequence regardless of when build completes

[flcdrg]

As initially reported in https://github.com/Microsoft/vstest/issues/1966, there isn't currently a way to prevent dotnet test from running separate test assemblies in parallel when this is not desirable behaviour.

@ShreyasRmsft suggested that adding a new parameter to the cli that would ensure tests run in sequence regardless of when build completes would enable this.

[OsirisTerje]

This is another level of parallelization, so suggest an appropriate parameter could be the same as used for the vstest , DisableParallelization

It is unfortunate however, that dotnet test doesn't use the information in the runsettings file. This file can be added as a parameter for dotnet test, and for the user it would feel natural that when one turns off parallel here, it also works for dotnet itself, and not just for the underlying vstest

[livarcocc]

dotnet test bugs should be tracked at microsoft/vstest. Did someone suggest you file an issue here? Otherwise, I will move this issue there.

[flcdrg]

Yes, @ShreyasRmsft in https://github.com/Microsoft/vstest/issues/1966#issuecomment-475500899 suggested raising a new issue here as it's the cli bit that's in play (before it hands off to vstest)

[mgkeen]

+1 for this. I'm currently having issues because I'm running dotnet test against my solution. The builds happen quickly, each one being passed off to vstest separately and effectively causing them all to run in parallel. On my not so great CI container it's causing the whole test run to grind to a halt and fail.

[mikeharder]

Setting the MSBuild property BuildInParallel=false seems to work for dirs.proj (using Microsoft.Build.Traversal), though it doesn't seem to work for *.sln files. Given the following dirs.proj:

<Project Sdk="Microsoft.Build.Traversal/2.0.24">
  <ItemGroup>
    <ProjectReference Include="**\*.*proj" />
  </ItemGroup>
</Project>

If you run dotnet test -p:BuildInParallel=false dirs.proj, each test project will be run in sequence.
However, dotnet test -p:BuildInParallel=false foo.sln still executes test projects in parallel.

[maciej-izak]

@mikeharder the way to go which works for me for *.sln files was additional usage of -m:1