Sdk: Running tests from published output

Created on 28 Apr 2016  路  2Comments  路  Source: dotnet/sdk

In many scenarios, it is useful to execute tests on published output rather than from source code. This is currently difficult to do.

Example scenario

Entity Framework runs the same test project on multiple operating systems, architectures, environments, and database versions. dotnet-restore and dotnet-build takes 10-15 min alone. To save time, we dotnet-publish test projects and re-distribute the artifact to all the environments.

The hacky part
Running dotnet-test-xunit.dll on the output of dotnet-publish requires mucking around with *.runtimeconfig.json and *.deps.json files. (cref https://github.com/aspnet/EntityFramework/issues/5077#issuecomment-215229437) It's works if files are renamed correctly, but the solution is brittle. We break every time CLI changes how shared framework works.

What CLI could change

  • Allow test projects to have the "static void Main" that launches tests. dotnet-test does dependency-command-invoking-magic that is hard to customize and harder to understand. Instead, dotnet-test could use an entry point from user code.

``` c#
// hypothetical code in the user project
public class Program
{
public static void Main(string[] args)
{
new CoreClrXunitTestRunner(args).RunTestsInThisAssembly();
}
}

- Allow specifying the assembly that gets the *.runtimeconfig.json file in published output.

dotnet publish --entry-point-assembly dotnet-test-xunit
```

This is still less than ideal, as it requires knowing the special syntax to start the tests: dotnet.exe dotnet-test-xunit.dll [AssemblyName.dll] [xunit options].

Area-DotNet Test question

Most helpful comment

Marking this as an rtm-timeframe question for now. The RTM runners are going to be getting fleshed out post RC2 and this is a scenario that should be accounted for then.

All 2 comments

Marking this as an rtm-timeframe question for now. The RTM runners are going to be getting fleshed out post RC2 and this is a scenario that should be accounted for then.

you can now use dotnet vstest to run tests on a dll directly.

@codito @Faizan2304 for more information.

Was this page helpful?
0 / 5 - 0 ratings