Azure-pipelines-tasks: Specflow Test with Data Tables runs from Agent or Client Fails on Azure Pipeline VS Test Task

[RoakyWood]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug
See Issue on StackOverflow

Without Specflow Data Table tests run fine from Azure Pipeline
With Specflow Data Table tests fail from Azure Pipeline

Full Bug Description and Logs.zip

Enter Task Name: Visual Studio Test

list here (V# not needed):
See attached in YAML Section

Environment

• Server - Azure Pipelines

• If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

• Agent - Hosted or Private: Private

  • If using private agent, provide the OS of the machine running the agent and the agent version:
    Windows Server 2016

Issue Description

Specflow Test with Data Tables runs from Agent or Client Fails on Azure Pipeline VS Test Task
See attached

Task logs

[Enable debug logging and please provide the zip file containing all the logs for a speedy resolution]

Troubleshooting

Followed recommended guide and please See attached

Error logs

See attached

[temelj]

Guessing here: For SpecFlow to play nicely, I had to add MSBUILDSINGLELOADCONTEXT variable to my yaml

stages:
- stage: Build
  displayName: 'Build'
  variables:
    - name: MSBUILDSINGLELOADCONTEXT
      value: 1

As per:
https://github.com/SpecFlowOSS/SpecFlow/issues/1912
It does appear that they have resolved the issue in the Beta release.

I couldn't help but notice that you are using the latest for specflow but outdated nuget packages elsewhere - however I'd guess your issue is quite likely related to SpecFlow itself rather than anything else.

[RoakyWood]

@temelj I much appreciate your recommendation. I did try both solutions, but no luck. It may be a slightly different issue. I tried the Build Variable and I updated the Specflow Packages to a 3.3 Beta.

[temelj]

@temelj I much appreciate your recommendation. I did try both solutions, but no luck. It may be a slightly different issue. I tried the Build Variable and I updated the Specflow Packages to a 3.3 Beta.

May I ask, do you have an example project that generates this same error? I'd re-produce it locally to see what's happening. My gut feeling is that Specflow is the issue here.

[RoakyWood]

@temelj I am working on getting working but anonymized code out either in a zip on here or on Github, but I'll need a couple days to unravel it. I really do appreciate your interest in this.

[RoakyWood]

It was a red herring... it's a huge never mind. I ran it headless and got the same fail pattern. Then I found this recommendation from demoncodemonkey https://stackoverflow.com/questions/47061662/selenium-tests-fail-against-headless-chrome options.AddArgument("--disable-extensions");
options.AddArgument("--headless");
options.AddArgument("--disable-gpu");
options.AddArgument("--window-size=1280,800");
options.AddArgument("--allow-insecure-localhost");
options.AddAdditionalCapability("acceptInsecureCerts", true, true); Now it passes with the data tables. Ribbons of shame.