Specflow: SpecRun 3.3.x fails to start with attached debugger.

[the-automation]

SpecFlow Version:

• [x] 3.1
• [ ] 3.0
• [ ] 2.4
• [ ] 2.3
• [ ] 2.2
• [ ] 2.1
• [ ] 2.0
• [ ] 1.9

Used Test Runner

• [x] SpecFlow+Runner
• [ ] MSTest
• [ ] NUnit
• [ ] Xunit

Version number: 3.3.17

Project Format of the SpecFlow project

• [ ] Classic project format using packages.config
• [ ] Classic project format using <PackageReference> tags
• [x] Sdk-style project format

.feature.cs files are generated using

• [x] SpecFlow.Tools.MsBuild.Generation NuGet package
• [ ] SpecFlowSingleFileGenerator custom tool

Visual Studio Version

• [x] VS 2019
• [ ] VS 2017
• [ ] VS 2015

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

• [ ] Enabled
• [x] Disabled

Are the latest Visual Studio updates installed?

• [x] Yes
• [ ] No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

• [ ] >= .NET 4.5
• [ ] before .NET 4.5
• [ ] .NET Core 2.0
• [ ] .NET Core 2.1
• [ ] .NET Core 2.2
• [ ] .NET Core 3.0
• [x] .NET Core 3.1

Test Execution Method:

• [x] Visual Studio Test Explorer
• [ ] TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• [ ] Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config or content of specflow.json

{
  "bindingCulture": {
    "language": "en-us"
  },
  "language": {
    "feature": "en-us"
  },
  "plugins": []
}

Issue Description

Test execution in Debug mode fails. Exception logged in test output:

Starting SpecFlow+ Runner executor process with attached debugger...
System.Runtime.InteropServices.COMException (0x89710016): Exception from HRESULT: 0x89710016
   at Microsoft.VisualStudio.Shell.Interop.IVsDebuggerLaunchAsync.LaunchDebugTargetsAsync(UInt32 debugTargetCount, VsDebugTargetInfo4[] debugTargetArray, IVsDebuggerLaunchCompletionCallback completionCallback)
   at Microsoft.VisualStudio.TestWindow.ShellServices.DebuggerService.<>c__DisplayClass20_0.<<LaunchDebuggerAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.TestWindow.Extensibility.ILoggerExtensions.<CallWithCatchAsync>d__10`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)

Steps to Reproduce

1. Create .NET Core 3.1 Test project
2. Install latest 3.3.x Specflow, Specflow.MsBuildGeneration and Specflow.Specrun versions
3. Add feature file, so that test is available in TestExplorer
4. Debug test from TestExplorer

Execution - aborted and exception logged in Test output.

[centu81]

I'm getting the same with VS19 16.6.2, with other PC (VS19 16.4.2) there exception is different:

Starting SpecFlow+ Runner executor process with attached debugger...
System.Runtime.InteropServices.COMException (0x89710016): Unable to start program 'dotnet'.

The system cannot find the file specified.

at Microsoft.VisualStudio.Shell.Interop.IVsDebugger4.LaunchDebugTargets4(UInt32 DebugTargetCount, VsDebugTargetInfo4[] pDebugTargets, VsDebugTargetProcessInfo[] pLaunchResults)
at Microsoft.VisualStudio.TestWindow.ShellServices.DebuggerService.<>c__DisplayClass20_0.<LaunchDebugger>b__0()
at Microsoft.VisualStudio.TestWindow.Extensibility.ILoggerExtensions.CallInternalWithLogging(ILogger2 log, Action action, Boolean shouldThrow)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)
The active test run was aborted. Reason: Exception of type 'Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException' was thrown.

[SabotageAndi]

I checked this last week for VS 16.6.3 and there it worked.
see https://github.com/SpecFlowOSS/SpecFlow/issues/1625#issuecomment-653415505

[the-automation]

I have VS 16.6.3 as well, but the issue still exist :/ tried different projects, and none allowed to debug. Only solution for now -> downgrading to specflow+ 3.2.x version.

[SabotageAndi]

@the-automation That's strange. Could you create a project to reproduce? I tried it with our examples and there the debugging worked.

[centu81]

After downgrading it works. Actually I'm using example to validate it https://github.com/SpecFlowOSS/SpecFlow.Plus.Examples/tree/master/SeleniumWebTest

    <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="3.1.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />
    <PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="80.0.3987.10600" />
    <PackageReference Include="Selenium.WebDriver.GeckoDriver" Version="0.26.0" />
    <PackageReference Include="Selenium.WebDriver.IEDriver" Version="3.150.1" />
    <PackageReference Include="SpecFlow" Version="3.1.97" />
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.1.97" />
    <PackageReference Include="FluentAssertions" Version="5.10.3" />
    <PackageReference Include="Appium.WebDriver" Version="4.1.1" />
    <PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
    <PackageReference Include="Selenium.Support" Version="3.141.0" />
    <PackageReference Include="SpecRun.SpecFlow" Version="3.2.31" />

[SabotageAndi]

@nemesv could now also reproduce the issue. Our assumption currently is, that it has to do something with the Process test thread isolation level. The example I tried is using the SharedAppDomain level and there it is working.

[eduardgrinberg]

I'm facing the same issue. When I downgrade SpecFlow and SperRun and everything works fine.

[SabotageAndi]

We are nearly finished with the fix for this and will release a new version in the next days.

[eduardgrinberg]

Looks like it was solved in 3.3.57

[SabotageAndi]

Yes, this is solved. We released the fix on Friday afternoon.

[SabotageAndi]

And it is 3.3.41 of the SpecRun.Runner packages. 3.3.57 is the version of the also released SpecFlow version.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.