Azure-pipelines-tasks: setting failTaskOnFailedTests on PublishTestResults task does not fail task

[jp108]

Hello ,

I still find https://github.com/Microsoft/azure-pipelines-tasks/issues/9711 broken and we need this functionality working as a continuation of our workflow of next step to trigger bug creation for even single failed test inside.

Log file for TASK - Publish Test Results
7_Automated UI Test Results.txt

complete Pipeline log files
logs_1506.zip

Please , do needful earliest as it is bug and very important functionality for our workflow.

[sahitiyerramilli]

Hey,

Could you please cross verify if the task is failing or not. We tried reproducing this at our end and the logs seem to be matching but our task failed. If your task is not failing, then we will need more details about your scenario.

[jp108]

Hello, Task is not failing at our end. I have provided all logs in zip and .txt also. Let me know what exactly should I provide other that that? Thank you

[sahitiyerramilli]

Hey,

[https://dev.azure.com/sahy/myproject/_build?definitionId=4] this is the link to our project pipeline where the task failed. Please share your yaml file if possible.

[jp108]

Sorry for delay -> YAML

steps:

• task: PublishTestResults@2

  displayName: 'Automated UI Test Results'

  inputs:

  testResultsFiles: '*.xml'

  searchFolder: '$(System.DefaultWorkingDirectory)\screenshots'

  failTaskOnFailedTests: true

  continueOnError: true

[sahitiyerramilli]

Hey,

In the case where both continueOnError and failTaskOnFailedTests are true, continueOnError takes the precedence. So, it will fail the task, it proceeds to the next task. If you want to fail the task, then enable only failTaskOnFailedTests.

[navin22]

@jp108 : Please let me know if you need any help ? Based on the logs also we see that the task is failed. If you are still facing the problem please let us know. And if possible can you point to your pipeline for better understanding.

[Cellule]

I just encountered the same issue. It used to work as far as I can remember.
I checked that I don't have continueOnError set

Here's the log for the task

2020-01-16T17:08:27.5244428Z ##[section]Starting: Publish Test Results **/junit*.xml
2020-01-16T17:08:27.5248683Z ==============================================================================
2020-01-16T17:08:27.5248919Z Task         : Publish Test Results
2020-01-16T17:08:27.5249100Z Description  : Publish test results to Azure Pipelines
2020-01-16T17:08:27.5249265Z Version      : 2.162.2
2020-01-16T17:08:27.5249372Z Author       : Microsoft Corporation
2020-01-16T17:08:27.5249542Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results
2020-01-16T17:08:27.5249746Z ==============================================================================
2020-01-16T17:08:30.4903740Z [command]/usr/bin/dotnet --version
2020-01-16T17:08:30.8193352Z 3.0.101
2020-01-16T17:08:31.6148248Z Result Attachments will be stored in LogStore
2020-01-16T17:08:31.6152070Z Run Attachments will be stored in LogStore
2020-01-16T17:08:31.7537493Z ##[section]Async Command Start: Publish test results
2020-01-16T17:08:32.0095373Z Publishing test results to test run '4196'.
2020-01-16T17:08:32.0140532Z TestResults To Publish 630, Test run id:4196
2020-01-16T17:08:32.0239391Z Test results publishing 630, remaining: 0. Test run id: 4196
2020-01-16T17:08:32.9809858Z Published Test Run : https://dev.azure.com/MaintainX/MaintainX/_TestManagement/Runs?runId=4196&_a=runCharts.
2020-01-16T17:08:33.0146373Z ##[section]Async Command End: Publish test results
2020-01-16T17:08:33.0147663Z ##[section]Finishing: Publish Test Results **/junit*.xml

In the build's summary I can see the test failure, but the Publish task didn't fail and thus my build seems to pass when there are actually failures

[sahitiyerramilli]

@Cellule
Can you please share your yaml file if that's okay. Also, can you share the tasks's log file after System.debug is set to true.

[alexkitrum]

I have the same problem when the PublishTestResults step did not fail, however, the test run had 1 failed test.

my settings:

• task: PublishTestResults@2
  inputs:
  testResultsFormat: 'VSTest'
  testResultsFiles: '*/.trx'
  searchFolder: '$(System.DefaultWorkingDirectory)/testresults'
  publishRunAttachments: true
  mergeTestResults: true
  failTaskOnFailedTests: true

I found an interesting observation that could help you in fixing the problem. When the failed test was skipped, it looks like the results were merged into one. So I got the following log for the step:

As you can see, the build is green with 1 failed test.

On the other hand, I checked logs for failed builds and found that each result file was published independently. See the screenshot below:

We didn't change the task's settings, but for some reason, the behavior is different for the runs.
Do you know what affects it?

Please let me know if I can provide more details on this.

[Cellule]

Here's the yaml definition

steps:
- task: PublishTestResults@2
  displayName: 'Publish Test Results **/junit*.xml'
  inputs:
    testResultsFiles: '**/junit*.xml'
    failTaskOnFailedTests: true

And the detailed log

2020-01-17T17:07:41.0567633Z ##[debug]Evaluating condition for step: 'Publish Test Results **/junit*.xml'
2020-01-17T17:07:41.0568442Z ##[debug]Evaluating: succeeded()
2020-01-17T17:07:41.0569106Z ##[debug]Evaluating succeeded:
2020-01-17T17:07:41.0569897Z ##[debug]=> True
2020-01-17T17:07:41.0570286Z ##[debug]Result: True
2020-01-17T17:07:41.0571050Z ##[section]Starting: Publish Test Results **/junit*.xml
2020-01-17T17:07:41.0575098Z ==============================================================================
2020-01-17T17:07:41.0575608Z Task         : Publish Test Results
2020-01-17T17:07:41.0575719Z Description  : Publish test results to Azure Pipelines
2020-01-17T17:07:41.0575852Z Version      : 2.162.2
2020-01-17T17:07:41.0575974Z Author       : Microsoft Corporation
2020-01-17T17:07:41.0576298Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results
2020-01-17T17:07:41.0576473Z ==============================================================================
2020-01-17T17:07:41.1782740Z ##[debug]agent.TempDirectory=/home/vsts/work/_temp
2020-01-17T17:07:41.1806944Z ##[debug]loading inputs and endpoints
2020-01-17T17:07:41.1810145Z ##[debug]loading INPUT_TESTRUNNER
2020-01-17T17:07:41.1822233Z ##[debug]loading INPUT_TESTRESULTSFILES
2020-01-17T17:07:41.1825429Z ##[debug]loading INPUT_SEARCHFOLDER
2020-01-17T17:07:41.1827213Z ##[debug]loading INPUT_MERGETESTRESULTS
2020-01-17T17:07:41.1828977Z ##[debug]loading INPUT_FAILTASKONFAILEDTESTS
2020-01-17T17:07:41.1829941Z ##[debug]loading INPUT_PUBLISHRUNATTACHMENTS
2020-01-17T17:07:41.1830359Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2020-01-17T17:07:41.1830672Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2020-01-17T17:07:41.1830980Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2020-01-17T17:07:41.1835473Z ##[debug]loading SECRET_TWILIO_AUTHTOKEN
2020-01-17T17:07:41.1837330Z ##[debug]loading SECRET_TWILIO_ACCOUNTSID
2020-01-17T17:07:41.1838285Z ##[debug]loading SECRET_SSHPASSPHRASE
2020-01-17T17:07:41.1839377Z ##[debug]loading SECRET_APOLLOENGINEKEY
2020-01-17T17:07:41.1839701Z ##[debug]loaded 13
2020-01-17T17:07:41.1854686Z ##[debug]Agent.ProxyUrl=undefined
2020-01-17T17:07:41.1855226Z ##[debug]Agent.CAInfo=undefined
2020-01-17T17:07:41.1855522Z ##[debug]Agent.ClientCert=undefined
2020-01-17T17:07:41.1855798Z ##[debug]Agent.SkipCertValidation=undefined
2020-01-17T17:07:41.1890016Z ##[debug]check path : /home/vsts/work/_tasks/PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1/2.162.2/task.json
2020-01-17T17:07:41.1890713Z ##[debug]adding resource file: /home/vsts/work/_tasks/PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1/2.162.2/task.json
2020-01-17T17:07:41.1891278Z ##[debug]system.culture=en-US
2020-01-17T17:07:41.1906758Z ##[debug]testRunner=JUnit
2020-01-17T17:07:41.1917188Z ##[debug]testResultsFiles=**/junit*.xml
2020-01-17T17:07:41.1918871Z ##[debug]mergeTestResults=false
2020-01-17T17:07:41.1920676Z ##[debug]platform=null
2020-01-17T17:07:41.1921009Z ##[debug]configuration=null
2020-01-17T17:07:41.1921252Z ##[debug]testRunTitle=null
2020-01-17T17:07:41.1921543Z ##[debug]publishRunAttachments=true
2020-01-17T17:07:41.1922005Z ##[debug]failTaskOnFailedTests=true
2020-01-17T17:07:41.1922285Z ##[debug]searchFolder=/home/vsts/work/1/s
2020-01-17T17:07:41.1922929Z ##[debug]testRunner: JUnit
2020-01-17T17:07:41.1923225Z ##[debug]testResultsFiles: **/junit*.xml
2020-01-17T17:07:41.1924076Z ##[debug]mergeResults: false
2020-01-17T17:07:41.1924376Z ##[debug]platform: null
2020-01-17T17:07:41.1924616Z ##[debug]config: null
2020-01-17T17:07:41.1924893Z ##[debug]testRunTitle: null
2020-01-17T17:07:41.1927144Z ##[debug]publishRunAttachments: true
2020-01-17T17:07:41.1927454Z ##[debug]failTaskOnFailedTests: true
2020-01-17T17:07:41.1930009Z ##[debug]defaultRoot: '/home/vsts/work/1/s'
2020-01-17T17:07:41.1931299Z ##[debug]findOptions.allowBrokenSymbolicLinks: 'true'
2020-01-17T17:07:41.1932033Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
2020-01-17T17:07:41.1932474Z ##[debug]findOptions.followSymbolicLinks: 'true'
2020-01-17T17:07:41.1934742Z ##[debug]matchOptions.debug: 'false'
2020-01-17T17:07:41.1935435Z ##[debug]matchOptions.nobrace: 'true'
2020-01-17T17:07:41.1935922Z ##[debug]matchOptions.noglobstar: 'false'
2020-01-17T17:07:41.1936409Z ##[debug]matchOptions.dot: 'true'
2020-01-17T17:07:41.1936874Z ##[debug]matchOptions.noext: 'false'
2020-01-17T17:07:41.1937354Z ##[debug]matchOptions.nocase: 'false'
2020-01-17T17:07:41.1937830Z ##[debug]matchOptions.nonull: 'false'
2020-01-17T17:07:41.1938308Z ##[debug]matchOptions.matchBase: 'false'
2020-01-17T17:07:41.1938743Z ##[debug]matchOptions.nocomment: 'false'
2020-01-17T17:07:41.1939222Z ##[debug]matchOptions.nonegate: 'false'
2020-01-17T17:07:41.1939703Z ##[debug]matchOptions.flipNegate: 'false'
2020-01-17T17:07:41.1940179Z ##[debug]pattern: '**/junit*.xml'
2020-01-17T17:07:41.1964150Z ##[debug]findPath: '/home/vsts/work/1/s'
2020-01-17T17:07:41.1965688Z ##[debug]statOnly: 'false'
2020-01-17T17:07:41.1966318Z ##[debug]findPath: '/home/vsts/work/1/s'
2020-01-17T17:07:41.1966832Z ##[debug]findOptions.allowBrokenSymbolicLinks: 'true'
2020-01-17T17:07:41.1967281Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true'
2020-01-17T17:07:41.1968571Z ##[debug]findOptions.followSymbolicLinks: 'true'
2020-01-17T17:07:44.8636571Z ##[debug]52785 results
2020-01-17T17:07:44.8636779Z ##[debug]found 52785 paths
2020-01-17T17:07:44.8636959Z ##[debug]applying include pattern
2020-01-17T17:07:44.8637329Z ##[debug]adjustedPattern: '/home/vsts/work/1/s/**/junit*.xml'
2020-01-17T17:07:44.8637605Z ##[debug]1 matches
2020-01-17T17:07:44.8637822Z ##[debug]1 final results
2020-01-17T17:07:44.8638036Z ##[debug]Detected 1 test result files
2020-01-17T17:07:44.8638403Z ##[debug]which 'dotnet'
2020-01-17T17:07:44.8638807Z ##[debug]found: '/usr/bin/dotnet'
2020-01-17T17:07:44.8639221Z ##[debug]which '/usr/bin/dotnet'
2020-01-17T17:07:44.8639630Z ##[debug]found: '/usr/bin/dotnet'
2020-01-17T17:07:44.8640036Z ##[debug]which '/usr/bin/dotnet'
2020-01-17T17:07:44.8640448Z ##[debug]found: '/usr/bin/dotnet'
2020-01-17T17:07:44.8640870Z ##[debug]/usr/bin/dotnet arg: --version
2020-01-17T17:07:44.8641141Z ##[debug]exec tool: /usr/bin/dotnet
2020-01-17T17:07:44.8641367Z ##[debug]arguments:
2020-01-17T17:07:44.8641732Z ##[debug]   --version
2020-01-17T17:07:44.8642164Z [command]/usr/bin/dotnet --version
2020-01-17T17:07:44.8642365Z 3.0.101
2020-01-17T17:07:44.8642593Z ##[debug]PublishTestResults.OverrideExeFlow=undefined
2020-01-17T17:07:44.8642822Z ##[debug]OS type: Linux
2020-01-17T17:07:45.9183206Z ##[debug]Object of TestLogStore created.
2020-01-17T17:07:46.1393707Z Result Attachments will be stored in LogStore
2020-01-17T17:07:46.1395055Z ##[debug]Object of TestResultsPublisher created.
2020-01-17T17:07:46.1396817Z ##[debug]Object of TestLogStore created.
2020-01-17T17:07:46.1398088Z Run Attachments will be stored in LogStore
2020-01-17T17:07:46.1398952Z ##[debug]Object of TestRunPublisher created.
2020-01-17T17:07:46.1400692Z ##[debug]Object of TestLogStore created.
2020-01-17T17:07:46.1460839Z ##[debug]Object of JUnitResultParser created.
2020-01-17T17:07:46.1474189Z ##[debug]Entering ParseTestResultFiles
2020-01-17T17:07:46.1479325Z ##[debug]runContext.ReleaseURI is null
2020-01-17T17:07:46.1480203Z ##[debug]runContext.ReleaseEnvironmentUri is null
2020-01-17T17:07:46.1522052Z ##[debug]Reading test results from file '/home/vsts/work/1/s/backend/junit.xml'.
2020-01-17T17:07:46.1698352Z ##[debug]Total test results: 634.
2020-01-17T17:07:46.1756788Z ##[debug]Leaving ParseTestResultFiles
2020-01-17T17:07:46.1813208Z ##[debug]runContext.ReleaseURI is null
2020-01-17T17:07:46.1819600Z ##[debug]runContext.ReleaseEnvironmentUri is null
2020-01-17T17:07:46.1833477Z ##[debug]Uploading build level attachements individually
2020-01-17T17:07:46.1851489Z ##[debug]Processed: ##vso[results.publish type=JUnit;mergeResults=false;publishRunAttachments=true;resultFiles=/home/vsts/work/1/s/backend/junit.xml;failTaskOnFailedTests=true;testRunSystem=VSTS - PTR;]
2020-01-17T17:07:46.1852212Z ##[debug]task result: Succeeded
2020-01-17T17:07:46.1852920Z ##[debug]Processed: ##vso[task.complete result=Succeeded;]
2020-01-17T17:07:46.1853406Z ##[debug]Release.ReleaseUri=undefined
2020-01-17T17:07:46.1854054Z ##[debug]Release.ReleaseId=undefined
2020-01-17T17:07:46.1854311Z ##[debug]Build.BuildUri=vstfs:///Build/Build/4133
2020-01-17T17:07:46.1854554Z ##[debug]Build.Buildid=4133
2020-01-17T17:07:46.1944736Z ##[debug]Entering PublishToNewTestRunPerTestResultFileAsync
2020-01-17T17:07:46.2217108Z ##[debug]Entering CreateTestRunAsync
2020-01-17T17:07:46.2239489Z ##[debug]Leaving CreateTestRunAsync
2020-01-17T17:07:46.2396378Z ##[debug]Processed: ##vso[telemetry.publish area=TestExecution;feature=PublishTestResultsTask]{"builduri":"vstfs:///Build/Build/4133","buildid":"4133","osType":"Linux","testRunner":"JUnit","failTaskOnFailedTests":"true","mergeResultsUserPreference":"false","config":null,"platform":null,"testResultsFilesCount":1,"dotnetVersion":"3.0.101","subFeature":"publishTestResultsTaskConsolidatedCiEvent"}
2020-01-17T17:07:46.2441713Z ##[section]Async Command Start: Publish test results
2020-01-17T17:07:46.2842145Z ##[debug]Total build level attachments: 0.
2020-01-17T17:07:46.4272080Z ##[debug]Entering AddTestResultsAsync
2020-01-17T17:07:46.4835410Z Publishing test results to test run '4220'.
2020-01-17T17:07:46.4874187Z TestResults To Publish 634, Test run id:4220
2020-01-17T17:07:46.4951622Z Test results publishing 634, remaining: 0. Test run id: 4220
2020-01-17T17:07:46.8000873Z ##[debug]Leaving AddTestResultsAsync
2020-01-17T17:07:46.8050381Z ##[debug]Entering UpdateTestRunAsync
2020-01-17T17:07:46.8051096Z ##[debug]Leaving UpdateTestRunAsync
2020-01-17T17:07:46.8786921Z ##[debug]Entering CreateTestRunAttachmentsAsync
2020-01-17T17:07:46.9174164Z ##[debug]Leaving CreateTestRunAttachmentsAsync
2020-01-17T17:07:46.9192227Z ##[debug]Uploading test run attachements individually
2020-01-17T17:07:46.9268094Z ##[debug]Entering UploadTestRunLogAsync
2020-01-17T17:07:47.1004596Z ##[debug]Blob name: GeneralAttachment/junit.xml .
2020-01-17T17:07:47.7042057Z ##[debug]File uploaded successsfully on LogStore junit.xml
2020-01-17T17:07:47.7095970Z Published Test Run : https://dev.azure.com/MaintainX/MaintainX/_TestManagement/Runs?runId=4220&_a=runCharts.
2020-01-17T17:07:47.7107306Z ##[debug]Leaving PublishToNewTestRunPerTestResultFileAsync
2020-01-17T17:07:47.7827607Z ##[section]Async Command End: Publish test results
2020-01-17T17:07:47.7828743Z ##[section]Finishing: Publish Test Results **/junit*.xml

I have confirmed that a build from last Tuesday (Jan 14) had the correct behavior and the version of the task hasn't changed since then.

From what I can see, the processing is done through the command

2020-01-17T17:07:46.1851489Z ##[debug]Processed: ##vso[results.publish type=JUnit;mergeResults=false;publishRunAttachments=true;resultFiles=/home/vsts/work/1/s/backend/junit.xml;failTaskOnFailedTests=true;testRunSystem=VSTS - PTR;]

Which is a vso command. I don't know where/how this is run, but I suspect something in the implementation of results.publish have changed in the past few days

[navin22]

@alexkitrum and @Cellule I think the the task version 2.162.0 had an issue and task 2.162.2 has the issue fixed. And in latest task version also it's fixed.

One change we had made recently in the results.publish command is to use different library to parse and publish the results and attachments, which allows users to upload attachments greater than 75MB.

[alexkitrum]

I'm using the latest version of the task(see screenshots). Are there any ways to prevent publishing results in batch? I think the problem is caused by the merge

[navin22]

@alexkitrum
mergeTestResults: true setting is for merging multiple results file into one run in Azure Devops.
The batching you see in the log files actually happens when there are more than 1000 test case results in a results file. In one rest call there will be 1000 results to publish. I don't think this is causing any issue.

In local I was able to repro this issue with the task version 2.162.0 but not with 2.162.2. And also Skipped is not considered as Failed.

Is it possible to share the details of the pipeline? or share the pipeline logs with variable system.debug = true.? When failed is not failing the task?

[Cellule]

I just did another attempt. I confirmed I am using version 2.162.2 and still doesn't fail the build with failures in the tests.

Like I said in my previous comment.
The command

##vso[results.publish type=JUnit;mergeResults=false;publishRunAttachments=true;resultFiles=/home/vsts/work/1/s/backend/junit.xml;failTaskOnFailedTests=true;testRunSystem=VSTS - PTR;]

Determines that the build passes or fails. Right now that command says the build passes, but correctly parsed the error since the error is showing in the test summary.

[navin22]

@Cellule
What's the agent version used ? vso commands are shipped with the agent.
If possible, can you please share the entire pipeline logs with system.debug = true?

[alexkitrum]

2020-01-21T10:39:42.0221591Z ##[debug]Evaluating condition for step: 'Integration tests - publish test results' 2020-01-21T10:39:42.0222881Z ##[debug]Evaluating: SucceededNode() 2020-01-21T10:39:42.0222982Z ##[debug]Evaluating SucceededNode: 2020-01-21T10:39:42.0223203Z ##[debug]=> True 2020-01-21T10:39:42.0223397Z ##[debug]Result: True 2020-01-21T10:39:42.0223558Z ##[section]Starting: Integration tests - publish test results 2020-01-21T10:39:42.0229591Z ============================================================================== 2020-01-21T10:39:42.0229693Z Task : Publish Test Results 2020-01-21T10:39:42.0229737Z Description : Publish test results to Azure Pipelines 2020-01-21T10:39:42.0229782Z Version : 2.162.2 2020-01-21T10:39:42.0229849Z Author : Microsoft Corporation 2020-01-21T10:39:42.0230069Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results 2020-01-21T10:39:42.0230117Z ============================================================================== 2020-01-21T10:39:42.1489578Z ##[debug]agent.TempDirectory=/home/vsts/work/_temp 2020-01-21T10:39:42.1507030Z ##[debug]loading inputs and endpoints 2020-01-21T10:39:42.1509940Z ##[debug]loading INPUT_TESTRUNNER 2020-01-21T10:39:42.1517525Z ##[debug]loading INPUT_TESTRESULTSFILES 2020-01-21T10:39:42.1520464Z ##[debug]loading INPUT_SEARCHFOLDER 2020-01-21T10:39:42.1521133Z ##[debug]loading INPUT_MERGETESTRESULTS 2020-01-21T10:39:42.1521835Z ##[debug]loading INPUT_FAILTASKONFAILEDTESTS 2020-01-21T10:39:42.1522410Z ##[debug]loading INPUT_TESTRUNTITLE 2020-01-21T10:39:42.1523505Z ##[debug]loading INPUT_PUBLISHRUNATTACHMENTS 2020-01-21T10:39:42.1523826Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION 2020-01-21T10:39:42.1524613Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION 2020-01-21T10:39:42.1525243Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN 2020-01-21T10:39:42.1529654Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN 2020-01-21T10:39:42.1531602Z ##[debug]loading SECRET_DOCKER-PASSWORD 2020-01-21T10:39:42.1531942Z ##[debug]loaded 12 2020-01-21T10:39:42.1542773Z ##[debug]Agent.ProxyUrl=undefined 2020-01-21T10:39:42.1543555Z ##[debug]Agent.CAInfo=undefined 2020-01-21T10:39:42.1543828Z ##[debug]Agent.ClientCert=undefined 2020-01-21T10:39:42.1544051Z ##[debug]Agent.SkipCertValidation=undefined 2020-01-21T10:39:42.1584078Z ##[debug]check path : /home/vsts/work/_tasks/PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1/2.162.2/task.json 2020-01-21T10:39:42.1584846Z ##[debug]adding resource file: /home/vsts/work/_tasks/PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1/2.162.2/task.json 2020-01-21T10:39:42.1585311Z ##[debug]system.culture=en-US 2020-01-21T10:39:42.1601405Z ##[debug]testRunner=VSTest 2020-01-21T10:39:42.1606015Z ##[debug]testResultsFiles=**/*.trx 2020-01-21T10:39:42.1607993Z ##[debug]mergeTestResults=true 2020-01-21T10:39:42.1608249Z ##[debug]platform=null 2020-01-21T10:39:42.1608523Z ##[debug]configuration=null 2020-01-21T10:39:42.1608930Z ##[debug]testRunTitle=Integration tests 2020-01-21T10:39:42.1612587Z ##[debug]publishRunAttachments=true 2020-01-21T10:39:42.1612866Z ##[debug]failTaskOnFailedTests=true 2020-01-21T10:39:42.1613123Z ##[debug]searchFolder=/home/vsts/work/1/s/tests 2020-01-21T10:39:42.1613371Z ##[debug]testRunner: VSTest 2020-01-21T10:39:42.1613720Z ##[debug]testResultsFiles: **/*.trx 2020-01-21T10:39:42.1613982Z ##[debug]mergeResults: true 2020-01-21T10:39:42.1614207Z ##[debug]platform: null 2020-01-21T10:39:42.1614469Z ##[debug]config: null 2020-01-21T10:39:42.1614689Z ##[debug]testRunTitle: Integration tests 2020-01-21T10:39:42.1614935Z ##[debug]publishRunAttachments: true 2020-01-21T10:39:42.1615150Z ##[debug]failTaskOnFailedTests: true 2020-01-21T10:39:42.1619633Z ##[debug]defaultRoot: '/home/vsts/work/1/s/tests' 2020-01-21T10:39:42.1620148Z ##[debug]findOptions.allowBrokenSymbolicLinks: 'true' 2020-01-21T10:39:42.1620518Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true' 2020-01-21T10:39:42.1620801Z ##[debug]findOptions.followSymbolicLinks: 'true' 2020-01-21T10:39:42.1621357Z ##[debug]matchOptions.debug: 'false' 2020-01-21T10:39:42.1621632Z ##[debug]matchOptions.nobrace: 'true' 2020-01-21T10:39:42.1622249Z ##[debug]matchOptions.noglobstar: 'false' 2020-01-21T10:39:42.1622551Z ##[debug]matchOptions.dot: 'true' 2020-01-21T10:39:42.1622850Z ##[debug]matchOptions.noext: 'false' 2020-01-21T10:39:42.1623213Z ##[debug]matchOptions.nocase: 'false' 2020-01-21T10:39:42.1623483Z ##[debug]matchOptions.nonull: 'false' 2020-01-21T10:39:42.1623772Z ##[debug]matchOptions.matchBase: 'false' 2020-01-21T10:39:42.1624066Z ##[debug]matchOptions.nocomment: 'false' 2020-01-21T10:39:42.1624356Z ##[debug]matchOptions.nonegate: 'false' 2020-01-21T10:39:42.1624653Z ##[debug]matchOptions.flipNegate: 'false' 2020-01-21T10:39:42.1625377Z ##[debug]pattern: '**/*.trx' 2020-01-21T10:39:42.1650897Z ##[debug]findPath: '/home/vsts/work/1/s/tests' 2020-01-21T10:39:42.1651290Z ##[debug]statOnly: 'false' 2020-01-21T10:39:42.1652905Z ##[debug]findPath: '/home/vsts/work/1/s/tests' 2020-01-21T10:39:42.1653270Z ##[debug]findOptions.allowBrokenSymbolicLinks: 'true' 2020-01-21T10:39:42.1653572Z ##[debug]findOptions.followSpecifiedSymbolicLink: 'true' 2020-01-21T10:39:42.1653874Z ##[debug]findOptions.followSymbolicLinks: 'true' 2020-01-21T10:39:42.1655701Z ##[debug] /home/vsts/work/1/s/tests (directory) 2020-01-21T10:39:42.1660721Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_17_602.trx (file) 2020-01-21T10:39:42.1661069Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_25_232.trx (file) 2020-01-21T10:39:42.1661349Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_36_542.trx (file) 2020-01-21T10:39:42.1661652Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_55_938.trx (file) 2020-01-21T10:39:42.1662099Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_40_636.trx (file) 2020-01-21T10:39:42.1662420Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_47_194.trx (file) 2020-01-21T10:39:42.1662735Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_56_088.trx (file) 2020-01-21T10:39:42.1663034Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_39_230.trx (file) 2020-01-21T10:39:42.1663307Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_59_148.trx (file) 2020-01-21T10:39:42.1663630Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_07_474.trx (file) 2020-01-21T10:39:42.1664102Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_24_149.trx (file) 2020-01-21T10:39:42.1664452Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_01_700.trx (file) 2020-01-21T10:39:42.1664793Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_20_108.trx (file) 2020-01-21T10:39:42.1665115Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_25_875.trx (file) 2020-01-21T10:39:42.1665458Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_32_007.trx (file) 2020-01-21T10:39:42.1665785Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_37_827.trx (file) 2020-01-21T10:39:42.1666120Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_45_296.trx (file) 2020-01-21T10:39:42.1666458Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_51_132.trx (file) 2020-01-21T10:39:42.1666773Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_59_149.trx (file) 2020-01-21T10:39:42.1671502Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_07_104.trx (file) 2020-01-21T10:39:42.1671985Z ##[debug] /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_18_741.trx (file) 2020-01-21T10:39:42.1672324Z ##[debug] /home/vsts/work/1/s/tests/companies.csv (file) 2020-01-21T10:39:42.1672578Z ##[debug] /home/vsts/work/1/s/tests/companies_addresses.csv (file) 2020-01-21T10:39:42.1672785Z ##[debug] /home/vsts/work/1/s/tests/companies_connections.csv (file) 2020-01-21T10:39:42.1673008Z ##[debug] /home/vsts/work/1/s/tests/companies_invitations.csv (file) 2020-01-21T10:39:42.1673371Z ##[debug] /home/vsts/work/1/s/tests/contacts.csv (file) 2020-01-21T10:39:42.1749202Z ##[debug] /home/vsts/work/1/s/tests/logo (directory) 2020-01-21T10:39:42.1749618Z ##[debug] /home/vsts/work/1/s/tests/logo/bigbrokerage.png (file) 2020-01-21T10:39:42.1749817Z ##[debug] /home/vsts/work/1/s/tests/logo/canadian.png (file) 2020-01-21T10:39:42.1750064Z ##[debug] /home/vsts/work/1/s/tests/logo/costco.png (file) 2020-01-21T10:39:42.1750299Z ##[debug] /home/vsts/work/1/s/tests/logo/dollar.jpg (file) 2020-01-21T10:39:42.1750483Z ##[debug] /home/vsts/work/1/s/tests/logo/generalmills.png (file) 2020-01-21T10:39:42.1750720Z ##[debug] /home/vsts/work/1/s/tests/logo/georgiap.png (file) 2020-01-21T10:39:42.1751090Z ##[debug] /home/vsts/work/1/s/tests/logo/hangards.png (file) 2020-01-21T10:39:42.1751325Z ##[debug] /home/vsts/work/1/s/tests/logo/heinz.png (file) 2020-01-21T10:39:42.1751501Z ##[debug] /home/vsts/work/1/s/tests/logo/homelfoods.png (file) 2020-01-21T10:39:42.1751753Z ##[debug] /home/vsts/work/1/s/tests/logo/jenie.jpg (file) 2020-01-21T10:39:42.1752252Z ##[debug] /home/vsts/work/1/s/tests/logo/kelloggs.jpg (file) 2020-01-21T10:39:42.1752436Z ##[debug] /home/vsts/work/1/s/tests/logo/kroger.png (file) 2020-01-21T10:39:42.1752674Z ##[debug] /home/vsts/work/1/s/tests/logo/ledgerset.png (file) 2020-01-21T10:39:42.1752847Z ##[debug] /home/vsts/work/1/s/tests/logo/lowes.png (file) 2020-01-21T10:39:42.1753081Z ##[debug] /home/vsts/work/1/s/tests/logo/michael.jpg (file) 2020-01-21T10:39:42.1753255Z ##[debug] /home/vsts/work/1/s/tests/logo/nordstorm.jpg (file) 2020-01-21T10:39:42.1753492Z ##[debug] /home/vsts/work/1/s/tests/logo/norpac.jpg (file) 2020-01-21T10:39:42.1753734Z ##[debug] /home/vsts/work/1/s/tests/logo/rich.png (file) 2020-01-21T10:39:42.1753909Z ##[debug] /home/vsts/work/1/s/tests/logo/target.png (file) 2020-01-21T10:39:42.1754144Z ##[debug] /home/vsts/work/1/s/tests/logo/walmart.jpg (file) 2020-01-21T10:39:42.1754319Z ##[debug] /home/vsts/work/1/s/tests/reset_sequences.sql (file) 2020-01-21T10:39:42.1754618Z ##[debug] /home/vsts/work/1/s/tests/tasks.csv (file) 2020-01-21T10:39:42.1754818Z ##[debug] /home/vsts/work/1/s/tests/users.csv (file) 2020-01-21T10:39:42.1755231Z ##[debug] /home/vsts/work/1/s/tests/users_notifications.csv (file) 2020-01-21T10:39:42.1755499Z ##[debug] /home/vsts/work/1/s/tests/users_roles.csv (file) 2020-01-21T10:39:42.1755692Z ##[debug]53 results 2020-01-21T10:39:42.1755954Z ##[debug]found 53 paths 2020-01-21T10:39:42.1756145Z ##[debug]applying include pattern 2020-01-21T10:39:42.1756738Z ##[debug]adjustedPattern: '/home/vsts/work/1/s/tests/**/*.trx' 2020-01-21T10:39:42.1756956Z ##[debug]21 matches 2020-01-21T10:39:42.1757232Z ##[debug]21 final results 2020-01-21T10:39:42.1757432Z ##[debug]Detected 21 test result files 2020-01-21T10:39:42.1757835Z ##[debug]which 'dotnet' 2020-01-21T10:39:42.1758219Z ##[debug]found: '/usr/bin/dotnet' 2020-01-21T10:39:42.1758537Z ##[debug]which '/usr/bin/dotnet' 2020-01-21T10:39:42.1758924Z ##[debug]found: '/usr/bin/dotnet' 2020-01-21T10:39:42.1759305Z ##[debug]which '/usr/bin/dotnet' 2020-01-21T10:39:42.1759677Z ##[debug]found: '/usr/bin/dotnet' 2020-01-21T10:39:42.1759992Z ##[debug]/usr/bin/dotnet arg: --version 2020-01-21T10:39:42.1760267Z ##[debug]exec tool: /usr/bin/dotnet 2020-01-21T10:39:42.1760458Z ##[debug]arguments: 2020-01-21T10:39:42.1760822Z ##[debug] --version 2020-01-21T10:39:42.1761231Z [command]/usr/bin/dotnet --version 2020-01-21T10:39:42.3590713Z 3.0.101 2020-01-21T10:39:42.3637116Z ##[debug]PublishTestResults.OverrideExeFlow=undefined 2020-01-21T10:39:42.3682905Z ##[debug]OS type: Linux 2020-01-21T10:39:42.4961435Z ##[debug]Object of TestLogStore created. 2020-01-21T10:39:42.4963429Z Result Attachments will be stored in LogStore 2020-01-21T10:39:42.4963932Z ##[debug]Object of TestResultsPublisher created. 2020-01-21T10:39:42.4964806Z ##[debug]Object of TestLogStore created. 2020-01-21T10:39:42.4965379Z Run Attachments will be stored in LogStore 2020-01-21T10:39:42.4966132Z ##[debug]Object of TestRunPublisher created. 2020-01-21T10:39:42.4966908Z ##[debug]Object of TestLogStore created. 2020-01-21T10:39:42.4968810Z ##[debug]Object of TrxResultParser created. 2020-01-21T10:39:42.4969287Z ##[debug]Entering ParseTestResultFiles 2020-01-21T10:39:42.4969636Z ##[debug]runContext.ReleaseURI is null 2020-01-21T10:39:42.4969964Z ##[debug]runContext.ReleaseEnvironmentUri is null 2020-01-21T10:39:42.4970548Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_17_602.trx'. 2020-01-21T10:39:42.4983031Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.4983757Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_17_602/In. 2020-01-21T10:39:42.5054660Z ##[debug]Total test results: 45. 2020-01-21T10:39:42.5055084Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_25_232.trx'. 2020-01-21T10:39:42.5089142Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5093839Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_25_232/In. 2020-01-21T10:39:42.5102459Z ##[debug]Total test results: 24. 2020-01-21T10:39:42.5102925Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_36_542.trx'. 2020-01-21T10:39:42.5126237Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5126693Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_36_542/In. 2020-01-21T10:39:42.5137941Z ##[debug]Total test results: 81. 2020-01-21T10:39:42.5138314Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_55_938.trx'. 2020-01-21T10:39:42.5153528Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5153973Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_55_938/In. 2020-01-21T10:39:42.5161652Z ##[debug]Total test results: 56. 2020-01-21T10:39:42.5161991Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_40_636.trx'. 2020-01-21T10:39:42.5170184Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5170620Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_40_636/In. 2020-01-21T10:39:42.5178354Z ##[debug]Total test results: 34. 2020-01-21T10:39:42.5178706Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_47_194.trx'. 2020-01-21T10:39:42.5183393Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5183826Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_47_194/In. 2020-01-21T10:39:42.5188622Z ##[debug]Total test results: 16. 2020-01-21T10:39:42.5188947Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_56_088.trx'. 2020-01-21T10:39:42.5227376Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5228029Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_56_088/In. 2020-01-21T10:39:42.5283355Z ##[debug]Total test results: 141. 2020-01-21T10:39:42.5283805Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_39_230.trx'. 2020-01-21T10:39:42.5301378Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5302108Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_39_230/In. 2020-01-21T10:39:42.5313579Z ##[debug]Total test results: 72. 2020-01-21T10:39:42.5314018Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_59_148.trx'. 2020-01-21T10:39:42.5318773Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5319354Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_59_148/In. 2020-01-21T10:39:42.5322820Z ##[debug]Total test results: 11. 2020-01-21T10:39:42.5323264Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_07_474.trx'. 2020-01-21T10:39:42.5332910Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5333354Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_07_474/In. 2020-01-21T10:39:42.5339772Z ##[debug]Total test results: 34. 2020-01-21T10:39:42.5340101Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_24_149.trx'. 2020-01-21T10:39:42.5369326Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5369778Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_24_149/In. 2020-01-21T10:39:42.5395610Z ##[debug]Total test results: 119. 2020-01-21T10:39:42.5396172Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_01_700.trx'. 2020-01-21T10:39:42.5409938Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5410448Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_01_700/In. 2020-01-21T10:39:42.5426349Z ##[debug]Total test results: 44. 2020-01-21T10:39:42.5426816Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_20_108.trx'. 2020-01-21T10:39:42.5500719Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5501357Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_20_108/In. 2020-01-21T10:39:42.5505482Z ##[debug]Total test results: 12. 2020-01-21T10:39:42.5507219Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_25_875.trx'. 2020-01-21T10:39:42.5518942Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5519417Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_25_875/In. 2020-01-21T10:39:42.5526254Z ##[debug]Total test results: 32. 2020-01-21T10:39:42.5526770Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_32_007.trx'. 2020-01-21T10:39:42.5530869Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5531345Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_32_007/In. 2020-01-21T10:39:42.5534531Z ##[debug]Total test results: 13. 2020-01-21T10:39:42.5534882Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_37_827.trx'. 2020-01-21T10:39:42.5546875Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5547491Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_37_827/In. 2020-01-21T10:39:42.5554216Z ##[debug]Total test results: 30. 2020-01-21T10:39:42.5554589Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_45_296.trx'. 2020-01-21T10:39:42.5556970Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5557418Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_45_296/In. 2020-01-21T10:39:42.5559910Z ##[debug]Total test results: 4. 2020-01-21T10:39:42.5560258Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_51_132.trx'. 2020-01-21T10:39:42.5563790Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5564197Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_51_132/In. 2020-01-21T10:39:42.5569399Z ##[debug]Total test results: 11. 2020-01-21T10:39:42.5569828Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_59_149.trx'. 2020-01-21T10:39:42.5575018Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5575576Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_59_149/In. 2020-01-21T10:39:42.5579380Z ##[debug]Total test results: 14. 2020-01-21T10:39:42.5579730Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_07_104.trx'. 2020-01-21T10:39:42.5583926Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5584502Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_07_104/In. 2020-01-21T10:39:42.5587175Z ##[debug]Total test results: 8. 2020-01-21T10:39:42.5587592Z ##[debug]Reading test results from file '/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_18_741.trx'. 2020-01-21T10:39:42.5600601Z ##[debug]Setting run start and finish times. 2020-01-21T10:39:42.5601157Z ##[debug]Attachment location: /home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_18_741/In. 2020-01-21T10:39:42.5608099Z ##[debug]Total test results: 38. 2020-01-21T10:39:42.5608448Z ##[debug]Leaving ParseTestResultFiles 2020-01-21T10:39:42.5609120Z ##[debug]Processed: ##vso[results.publish type=VSTest;mergeResults=true;runTitle=Integration tests;publishRunAttachments=true;resultFiles=/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_17_602.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_25_232.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_36_542.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_34_55_938.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_40_636.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_47_194.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_35_56_088.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_39_230.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_36_59_148.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_07_474.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_37_24_149.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_01_700.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_20_108.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_25_875.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_32_007.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_37_827.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_45_296.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_51_132.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_38_59_149.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_07_104.trx,/home/vsts/work/1/s/tests/_c4b020725164_2020-01-21_10_39_18_741.trx;failTaskOnFailedTests=true;testRunSystem=VSTS - PTR;] 2020-01-21T10:39:42.5609760Z ##[debug]task result: Succeeded 2020-01-21T10:39:42.5610059Z ##[debug]Processed: ##vso[task.complete result=Succeeded;] 2020-01-21T10:39:42.5610131Z ##[debug]Release.ReleaseUri=undefined 2020-01-21T10:39:42.5610188Z ##[debug]Release.ReleaseId=undefined 2020-01-21T10:39:42.5610255Z ##[debug]Build.BuildUri=vstfs:///Build/Build/1377 2020-01-21T10:39:42.5610318Z ##[debug]Build.Buildid=1377 2020-01-21T10:39:42.5611496Z ##[debug]Uploading build level attachements individually 2020-01-21T10:39:42.5618278Z ##[debug]runContext.ReleaseURI is null 2020-01-21T10:39:42.5620850Z ##[debug]runContext.ReleaseEnvironmentUri is null 2020-01-21T10:39:42.5673193Z ##[debug]Entering PublishAllTestResultsToSingleTestRunAsync 2020-01-21T10:39:42.5807022Z ##[debug]Entering CreateTestRunAsync 2020-01-21T10:39:42.5809876Z ##[debug]Leaving CreateTestRunAsync 2020-01-21T10:39:42.5850449Z ##[debug]Total build level attachments: 0. 2020-01-21T10:39:42.5969641Z ##[debug]Processed: ##vso[telemetry.publish area=TestExecution;feature=PublishTestResultsTask]{"builduri":"vstfs:///Build/Build/1377","buildid":"1377","osType":"Linux","testRunner":"VSTest","failTaskOnFailedTests":"true","mergeResultsUserPreference":"true","config":null,"platform":null,"testResultsFilesCount":21,"dotnetVersion":"3.0.101","subFeature":"publishTestResultsTaskConsolidatedCiEvent"} 2020-01-21T10:39:42.5970632Z ##[section]Async Command Start: Publish test results 2020-01-21T10:39:42.8714078Z ##[debug]Entering AddTestResultsAsync 2020-01-21T10:39:42.8877371Z Publishing test results to test run '4548'. 2020-01-21T10:39:42.8877943Z TestResults To Publish 839, Test run id:4548 2020-01-21T10:39:42.8881542Z Test results publishing 839, remaining: 0. Test run id: 4548 2020-01-21T10:39:43.3171595Z ##[debug]Leaving AddTestResultsAsync 2020-01-21T10:39:43.3193453Z ##[debug]Entering UpdateTestRunAsync 2020-01-21T10:39:43.3193899Z ##[debug]Leaving UpdateTestRunAsync 2020-01-21T10:39:43.3549971Z ##[debug]Entering CreateTestRunAttachmentsAsync 2020-01-21T10:39:43.3846387Z ##[debug]Leaving CreateTestRunAttachmentsAsync 2020-01-21T10:39:43.4097059Z ##[debug]Entering UploadTestRunLogAsync 2020-01-21T10:39:43.7044263Z ##[debug]Blob name: GeneralAttachment/TestResults_4548.zip . 2020-01-21T10:39:43.8395101Z ##[debug]File uploaded successsfully on LogStore TestResults_4548.zip 2020-01-21T10:39:43.8407884Z Published Test Run : https://dev.azure.com/@@@@@@@@/Runs?runId=4548&_a=runCharts. 2020-01-21T10:39:43.8409037Z ##[debug]Leaving PublishAllTestResultsToSingleTestRunAsync 2020-01-21T10:39:43.8711183Z ##[section]Async Command End: Publish test results 2020-01-21T10:39:43.8712857Z ##[section]Finishing: Integration tests - publish test results

[alexkitrum]

Are there any ways to rollback to prev version?

[ghost]

I have pipelines that started seeing this issue on Jan 16 between 8 and 10 AM central.

[Cellule]

I'm using pool Hosted Ubuntu 1604 Image (Ubuntu 16.04.6 LTS)
unfortunately I can't share the whole log since it's a private repository, but here's the log for initialize job showing the agent version

##[section]Starting: Initialize job
Agent name: 'Hosted Agent'
Agent machine name: 'fv-az105'
Current agent version: '2.164.3'
Current image version: '20200102.1'
Agent running as: 'vsts'
##[debug]Primary repository: ******. repository type: GitHub
Prepare build directory.
##[debug]Creating build directory: '/home/vsts/work/1'
##[debug]Delete existing artifacts directory: '/home/vsts/work/1/a'
##[debug]Creating artifacts directory: '/home/vsts/work/1/a'
##[debug]Delete existing test results directory: '/home/vsts/work/1/TestResults'
##[debug]Creating test results directory: '/home/vsts/work/1/TestResults'
##[debug]Creating binaries directory: '/home/vsts/work/1/b'
##[debug]Creating source directory: '/home/vsts/work/1/s'
Set build variables.
Download all required tasks.
Downloading task: CmdLine (2.163.0)
##[debug]Task 'CmdLine' has been downloaded into '/home/vsts/work/_tasks/CmdLine_d9bafed4-0b18-4f58-968d-86655b4d2ce9/2.163.0'.
Downloading task: DockerInstaller (0.1.4)
##[debug]Task 'DockerInstaller' has been downloaded into '/home/vsts/work/_tasks/DockerInstaller_8e038650-f7bf-11e8-8a6c-8fff434b4eff/0.1.4'.
Downloading task: Bash (3.159.3)
##[debug]Task 'Bash' has been downloaded into '/home/vsts/work/_tasks/Bash_6c731c3c-3c68-459a-a5c9-bde6e6595b5b/3.159.3'.
Downloading task: DockerCompose (0.160.4)
##[debug]Task 'DockerCompose' has been downloaded into '/home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.160.4'.
Downloading task: NodeTool (0.159.0)
##[debug]Task 'NodeTool' has been downloaded into '/home/vsts/work/_tasks/NodeTool_31c75bbb-bcdf-4706-8d7c-4da6a1959bc2/0.159.0'.
Downloading task: Yarn (2.8.1001)
##[debug]Task 'Yarn' has been downloaded into '/home/vsts/work/_tasks/Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3/2.8.1001'.
Downloading task: PublishTestResults (2.162.2)
##[debug]Task 'PublishTestResults' has been downloaded into '/home/vsts/work/_tasks/PublishTestResults_0b0f01ed-7dde-43ff-9cbb-e48954daf9b1/2.162.2'.
##[debug]Task 'Cancel if no changes in paths' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Install Docker 17.09.0-ce' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Write Docker overwrites' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Docker build db cache' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Indicate docker was built' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Docker run db cache' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Write environment secrets' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Use Node 12.13' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Yarn install backend' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Yarn install api' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Compile Backend api - Prod' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Yarn install scheduler' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Compile Backend scheduler - Prod' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Yarn install rest' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Compile Backend rest - Prod' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Yarn add --dev jest-junit' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Run Backend Tests' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Check junit reporter file' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Publish Test Results **/junit*.xml' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Validate Migration' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Check Changelog was updated' has following condition: 'succeeded()'.
##[debug]Parsing expression: <succeeded()>
##[debug]succeeded
##[debug](
##[debug])
##[debug]Task 'Docker Cleanup' has following condition: 'and(always(), eq(variables['DockerBuilt'], 'true')) '.
##[debug]Parsing expression: <and(always(), eq(variables['DockerBuilt'], 'true')) >
##[debug]and
##[debug](
##[debug]..always
##[debug](
##[debug]..)
##[debug]..,
##[debug]..eq
##[debug](
##[debug]....variables
##[debug]....[
##[debug]......'DockerBuilt'
##[debug]....]
##[debug]....,
##[debug]....'true'
##[debug]..)
##[debug])
##[debug]Log plugin 'TestFilePublisherPlugin' is disabled.
Plugin: 'Test Result Parser plugin' is running in background.
Plugin: 'Test File Publisher plugin' is running in background.
Start tracking orphan processes.
##[section]Finishing: Initialize job

[alexkitrum]

Any ETA for fixing the problem?

It occurred with another project/pipeline where we made no changes to pipeline configs for several months?

Can I use the prev version of the plugin?

It affects our development a lot. We can't rely on the green builds anymore, and have to check test results manually.

[navin22]

@alexkitrum Is it possible to share the account information here ? otherwise you can share on the email to me [email protected].
we have rolled out new feature in agent recently, I will try to switch off that feature and see if it's because of that.

[ghost]

I'm seeing that the reported test pass rate is 100% but the number of tests is not accurate.

When there are clearly tests failing:

Could it be that the rule is still working but the pass rate is broken?

[TheRealWaldo]

Seeing the same issue here; does not appear to be any rhyme or reason, the same failing tests were failing the build last week, but are passing now.

Same as the others, can clearly see the parsed & published results indicate a failed test, but the step is not failing.

   - task: PublishTestResults@2
      displayName: Publish Test Results
      inputs:
        testResultsFormat: JUnit
        testRunTitle: Unit Tests
        testResultsFiles: '$(System.DefaultWorkingDirectory)/app/build/test-results/**/TEST-*.xml'
        failTaskOnFailedTests: true

We had originally used the Gradle task to run the unit tests with the publishJUnitResults options but it started exhibiting this same behaviour a while back, so we switched to PublishTestResults as a workaround.

At the moment, our Azure DevOps builds are letting failing code into production....

[TheRealWaldo]

Some more information; I looked back through history and found instances of 2.162.2 where the task was failing as expected, the only difference was the total number of test failures. With only 1 test failure, the task passed (not expected), and with multiple, it failed as expected.

[navin22]

@TheRealWaldo Is it possible to share the pipeline? And also check if the test result is marked as Flaky? if it's marked as flaky then it will not be considered as failed test and allow the task to pass as the test is flaky.

[rmarinho]

Hi, I m seeing the same issue in our release pipeline.

Is public so you can see the issue here :

https://dev.azure.com/xamarin/public/_apps/hub/ms.vss-releaseManagement-web.cd-release-progress?releaseId=1071&_a=release-environment-extension&environmentId=35502&extensionId=ms.vss-test-web.test-result-in-release-environment-editor-tab

You can see there's 1 failing test, but it doesn't fail the task.

2020-01-24T05:10:08.5038060Z ##[section]Starting: Publish Test Results
2020-01-24T05:10:08.5045310Z ==============================================================================
2020-01-24T05:10:08.5045410Z Task         : Publish Test Results
2020-01-24T05:10:08.5045470Z Description  : Publish test results to Azure Pipelines
2020-01-24T05:10:08.5045580Z Version      : 2.162.2
2020-01-24T05:10:08.5045640Z Author       : Microsoft Corporation
2020-01-24T05:10:08.5045700Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/publish-test-results
2020-01-24T05:10:08.5045800Z ==============================================================================
2020-01-24T05:10:08.7541000Z [command]/usr/local/bin/dotnet --version
2020-01-24T05:10:09.5482570Z 2.2.105
2020-01-24T05:10:10.9813420Z Result Attachments will be stored in FileService
2020-01-24T05:10:11.0663040Z Run Attachments will be stored in FileService
2020-01-24T05:10:11.3883360Z ##[section]Async Command Start: Publish test results
2020-01-24T05:10:12.1950370Z Publishing test results to test run '1058438'.
2020-01-24T05:10:12.2011220Z TestResults To Publish 550, Test run id:1058438
2020-01-24T05:10:12.2118820Z Test results publishing 550, remaining: 0. Test run id: 1058438
2020-01-24T05:10:12.8045660Z ##[warning]Failed to publish test result attachments
2020-01-24T05:10:17.3991800Z Published Test Run : https://tfsprodcus3.visualstudio.com/Ad0adf05a-e7d7-4b65-96fe-3f3884d42038/public/_TestManagement/Runs?runId=1058438&_a=runCharts.
2020-01-24T05:10:17.4738540Z ##[section]Async Command End: Publish test results
2020-01-24T05:10:17.4740640Z ##[section]Finishing: Publish Test Results

I m not using yaml as this is a release pipeline, but the yaml should be something like this

steps:
- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: NUnit
    testResultsFiles: '*.xml'
    searchFolder: '$(TestResultsOutputFolder)'
    failTaskOnFailedTests: true
  condition: succeededOrFailed()

[TheRealWaldo]

@navin22 sorry we're closed source so I cannot share the pipeline publicly. Multiple tests have failed through the history, none that I have reviewed were marked flaky.

I should also point out that we need the pipeline to fail REGARDLESS of any 'flaky' state that Microsoft might interpret. We don't accept flaky tests in production either, so please clarify that failTaskOnFailedTests should fail regardless, and if not, how to ensure that they fail the build no matter what the possible reason could be.

[navin22]

@TheRealWaldo Okay, the flaky doesn't interpreted automatically. The settings have to be enabled at project and pipeline level. If you haven't enabled it. Then you don't need to worry about it.

[navin22]

The recent code changes have caused this regression sorry for the inconvenience. I will try to fix the agent code ASAP. And update it here.

Since these changes were behind a flag, I have turned off the flag for few accounts. @rmarinho and @Cellule you guys should be unblocked and not see this issue. Can you please confirm? After that I will turn this off for every one. Until we fix the bug.

[Cellule]

I can confirm the task now correctly fails the build when there is a test failure in the MaintainX account.

[rmarinho]

Yes can confirm that seems to work now as expected!

[alexkitrum]

still have the same issue

[navin22]

@alexkitrum what's the agent version you are using ?

[alexkitrum]

Image: Ubuntu16
Agent: Hosted Agent

[gl1tch]

Can confirm i'm still having the issue. We have 2 failing test in the build
Image: Ubuntu-16.04
Hosted Agent: Yes

As mentioned by @TheRealWaldo this is potentially causing broken code to be released to our environments.

YAML below.

steps:
- task: PublishTestResults@2
  displayName: 'Publish Test Results'
  inputs:
    testResultsFormat: VSTest
    testResultsFiles: '**/test_results*.trx'
    failTaskOnFailedTests: true

[navin22]

The rollout is in the process, It might take couple of days to roll out to all the regions.
If it's okay to share the account name "https://dev.azure.com/" with us over email [email protected]. I can enable the flags to quickly unblock you guys.

[gl1tch]

Hi @navin22 we will wait a couple days for the roll out to the region, is there anything to do other than wait. We are in the South Africa region.

[navin22]

Since you are using Hosted agent, I don't have any option. Otherwise we can downgrade the agent version to 2.163 or 2.162.

[rngene]

Hi @navin22 we are in the US. Is there a way to know when the roll out will be finished for our Region? I've been watching this issue for a few days as it is affecting us. As of right now the failure is still happening.

[robend1990]

@navin22 Will this be fixed in v2.164.7 agent version?? If yes, do you know when it will be released?

[navin22]

It's hard to tell if the change is that region, until I get the account name. But by next Monday or Tuesday everybody should get the fix. I will try if it can be expedited.

@robend1990 It depends on the Azure Devops side fix as well. Hence just the new agent is not enough. But if you have private agents then old agents like 2.163 or 2.162 to can unblock you.

[navin22]

If you guys can update the agents, please update the agent to latest agent version to get unblocked.
For all others some might have received the latest fixes and not see this issue, for few the fix will rollout in few days.

[andrey-dubnik]

Is there any estimate for the West EU rollout (unless agents are rolled out per account)? All the hosted agents we use are on 2.164.6 and there are quite a lot of pipelines to do a manual checks now...

We have tested a private agent with the version 2.164.7 and it fixes the issue unfortunately we are using the hosted agents which are not yet updated...

[alexkitrum]

It was fixed, but now I'm having the issue again

[Faizan2304]

Are using using agent with version 2.164.7?

[navin22]

Can you guys please confirm, if the issue is resolved now?

[rngene]

@navin22 Issue is fixed for me now. Thanks!

[navin22]

If anyone is still facing the issue let me know, or reopen the issue.

[wenxiaoy]

I have the same issue as well.

[mmisztal1980]

This is an issue with 2.167.0. Please reopen.

[adrian-skybaker]

I'm still seeing this problem on 2.171.0.

[adrian-skybaker]

If anyone is still facing the issue let me know, or reopen the issue.

@navin22 I don't have permission to reopen.

[GaTechThomas]

We're suddenly seeing this issue. Agents are on 2.174.1.

[cawoodm]

We are seeing this issue on 2.165.0. failTaskOnFailedTests: true does nothing: failed tests do not fail the task.