Vstest: Debug.Assert(false) causes poor experience in tests

[eerhardt]

Description

When a Debug.Assert fails during unit tests, there is no way to know what failed from looking at the output.

I've looked at https://github.com/Microsoft/vstest/issues/1022, but the behavior there seems to have regressed in the latest builds. I am not getting an output that says an assert failed, and in no place does it say a stack trace of the failing assert.

Steps to reproduce

Using this project

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.8.0-preview-20180610-02" />
    <PackageReference Include="xunit" Version="2.3.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
  </ItemGroup>
</Project>

and this code:
```c#
using System.Diagnostics;
using Xunit;

namespace XUnitTestProject1
{
public class UnitTest1
{
[Fact]
public void Test1()
{
Debug.Assert(false);
}
}
}

run the test in both `dotnet test` and in VS Test Explorer (15.8.0 Preview 4.0 [27907.0.d15.8])

## Expected behavior
I expect to at least get an error indicating a debug.assert failed and a stack trace of where in my code the assert failed.

## Actual behavior
Here is the full output from the command line.  I get similar output in the VS Output window.  Neither place says that it is `UnitTest1` that is failing causing an assert. My only recourse is to run the test again and debug.

C:\users\eerhardt\source\XUnitTestProject1> dotnet test
Build started, please wait...
Build completed.

Test run for C:\users\eerhardt\source\XUnitTestProject1\bin\Debug\netcoreapp2.1\XUnitTestProject1.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0-preview-20180605-02
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...
The active test run was aborted. Reason: (Func1 code) at Xunit.Sdk.TestRunner1.RunAsync()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestRunner1.RunAsync() at Xunit.Sdk.TestCaseRunner1.RunAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestCaseRunner.cs:line 82
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestCaseRunner1.RunAsync() at Xunit.Sdk.XunitTestMethodRunner.RunTestCaseAsync(IXunitTestCase testCase) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\XunitTestMethodRunner.cs:line 45 at Xunit.Sdk.TestMethodRunner1.RunTestCasesAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestMethodRunner.cs:line 136
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestMethodRunner1.RunTestCasesAsync() at Xunit.Sdk.TestMethodRunner1.RunAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestMethodRunner.cs:line 106
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestMethodRunner1.RunAsync() at Xunit.Sdk.TestClassRunner1.RunTestMethodsAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestClassRunner.cs:line 213
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestClassRunner1.RunTestMethodsAsync() at Xunit.Sdk.TestClassRunner1.RunAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestClassRunner.cs:line 171
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestClassRunner1.RunAsync() at Xunit.Sdk.TestCollectionRunner1.RunTestClassesAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestCollectionRunner.cs:line 130
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestCollectionRunner1.RunTestClassesAsync() at Xunit.Sdk.TestCollectionRunner1.RunAsync() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestCollectionRunner.cs:line 101
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.StartTStateMachine
at Xunit.Sdk.TestCollectionRunner1.RunAsync() at Xunit.Sdk.XunitTestAssemblyRunner.<>c__DisplayClass14_2.<RunTestCollectionsAsync>b__2() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\XunitTestAssemblyRunner.cs:line 184 at System.Threading.Tasks.Task1.InnerInvoke()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.Tasks.Task.ExecuteEntry()
at System.Threading.Tasks.SynchronizationContextTaskScheduler.<>c.<.cctor>b__8_0(Object s)
at Xunit.Sdk.MaxConcurrencySyncContext.RunOnSyncContext(SendOrPostCallback callback, Object state) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\MaxConcurrencySyncContext.cs:line 107
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at Xunit.Sdk.MaxConcurrencySyncContext.WorkerThreadProc() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\MaxConcurrencySyncContext.cs:line 89
at Xunit.Sdk.XunitWorkerThread.<>c.b__5_0(Object _) in C:\Dev\xunit\xunit\src\common\XunitWorkerThread.cs:line 37
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

Test Run Aborted.

## Diagnostic logs
[log.txt](https://github.com/Microsoft/vstest/files/2193389/log.txt)


## Environment

dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 2.1.400-preview-009063
Commit: dd0179a67c

Runtime Environment:
OS Name: Windows
OS Version: 10.0.17134
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\2.1.400-preview-009063\

Host (useful for support):
Version: 2.1.1
Commit: 6985b9f684

.NET Core SDKs installed:
1.1.9 [C:\Program Files\dotnet\sdk]
2.1.1 [C:\Program Files\dotnet\sdk]
2.1.2 [C:\Program Files\dotnet\sdk]
2.1.4 [C:\Program Files\dotnet\sdk]
2.1.100 [C:\Program Files\dotnet\sdk]
2.1.200 [C:\Program Files\dotnet\sdk]
2.1.201 [C:\Program Files\dotnet\sdk]
2.1.300 [C:\Program Files\dotnet\sdk]
2.1.400-preview-009063 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
```

/cc @singhsarab @smadala

[smadala]

@eerhardt This is not regression, complete stacktrace for xunit tests not appearing due to ErrorLength. I think we should increase ErrorLength to reasonable number to show complete stacktrace.

/cc @anurse

[StanislawSwierc]

Even if you cannot make it 100% reliable I would appreciate at least hinting that Debug.Assert might be a cause. Because the error message was uninformative I wasted several hours diagnosing my tests before I finally came across this thread.

[smadala]

Working on this. https://github.com/Microsoft/vstest/pull/1708

[eerhardt]

This is great to see, @smadala! Do you know, will this make it into a VS 2017 update? If so, which one?

[smadala]

VS 15.9

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Eric Erhardt [e-mail]
Sent: Wednesday, August 1, 2018 8:02:05 PM
To: Microsoft/vstest
Cc: Satya Madala; Mention
Subject: Re: [Microsoft/vstest] Debug.Assert(false) causes poor experience in tests (#1688)

This is great to hear, @smadalahttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsmadala&data=02%7C01%7Csamadala%40microsoft.com%7C85e274d549644be91fab08d5f7bb8ed4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636687307288205767&sdata=ShPyV56qn87RzXrWDATNxHadOXTfSYdh7GdgwlH1GjE%3D&reserved=0! Do you know, will this make it into a VS 2017 update? If so, which one?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fvstest%2Fissues%2F1688%23issuecomment-409595792&data=02%7C01%7Csamadala%40microsoft.com%7C85e274d549644be91fab08d5f7bb8ed4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636687307288205767&sdata=vPR%2B7RqxHYzX7Mquh1Yud6iNqbZLN3nmXICGro%2Ff%2FXk%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEIutGEK2VYrlULaG0HUEezSJuHv2FMyks5uMbvlgaJpZM4VPJSK&data=02%7C01%7Csamadala%40microsoft.com%7C85e274d549644be91fab08d5f7bb8ed4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636687307288215777&sdata=krgAvw02S%2FZJye1OVeTY80wYdJsMMFIEhQUExfmdaqg%3D&reserved=0.