Vscode-powershell: Pester test `Context` failures not showing up in Problems window

Created on 24 Jan 2020 · 4Comments · Source: PowerShell/vscode-powershell

Issue Description

If you have a Pester test where a BeforeAll or AfterAll block fails that failure doesn't show up in the Problems window. Test failures do properly display.

Here's a repro test you can use to try it out:

Describe "Reproduce Issue" {
    Context "Failing BeforeAll in Context block" {
        BeforeAll {
            # In a "real" scenario, this is
            # $pathToNuGet = Get-Command nuget.exe | Select-Object -ExpandProperty Source
            # &mono $pathToNuGet do-some-things-with-nuget.exe
            # to execute NuGet on a Linux/OS X box. The failure I'm seeing in the context
            # happens when mono isn't available, and possibly nuget.exe is also not there.
            $pathToCommand = Get-Command nosuchcommand.exe | Select-Object -ExpandProperty Source
            &othermissingcommand $pathToCommand
        }
    }
    It "has two failures - context and test" {
        $False | Should -Be $True
    }
}

The output of this in the console looks like:

> Executing task: Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true} <

    ____            __
   / __ \___  _____/ /____  _____
  / /_/ / _ \/ ___/ __/ _ \/ ___/
 / ____/  __(__  ) /_/  __/ /
/_/    \___/____/\__/\___/_/
Pester v4.9.0
Executing all tests in '.'

Executing script /Users/tillig/dev/repro/Repro.Tests.ps1

  Describing Reproduce Issue

    Context Failing BeforeAll in Context block
Get-Command: /Users/tillig/dev/repro/Repro.Tests.ps1:9:30
Line |
   9 |              $pathToCommand = Get-Command nosuchcommand.exe | Select-O …
     |                               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | The term 'nosuchcommand.exe' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the
     | spelling of the name, or if a path was included, verify that the path is correct and try again.

      [-] Error occurred in Context block 0ms
        At /Users/tillig/dev/repro/Repro.Tests.ps1:10 char:14
        The term 'othermissingcommand' is not recognized as the name of a cmdlet, function, script file, or operable program.
        Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
    [-] has two failures - context and test 77ms
      at <ScriptBlock>, /Users/tillig/dev/repro/Repro.Tests.ps1: line 14
      14:         $False | Should -Be $True
      Expected $true, but got $false.
Tests completed in 508ms
Tests Passed: 0, Failed: 2, Skipped: 0, Pending: 0, Inconclusive: 0 

Terminal will be reused by tasks, press any key to close it.

The Problems window does display the test failure - 'has two failures - context and test' but it does NOT display the failure in the test context itself.

If you switch the test above so it's $True | Should -Be $True (so there's an assertion and it passes), only the Context will fail. Re-run the test and you'll see there's nothing in the Problem window.

Attached Logs

I didn't notice anything in the logs that indicated a failure around this, it appears it's a problem matcher issue. I can gather logs if you think it'd help.

Environment Information

Visual Studio Code

| Name | Version |
| --- | --- |
| Operating System | Darwin x64 19.2.0 |
| VSCode | 1.41.1|
| PowerShell Extension Version | 2020.1.0 |

PowerShell Information

|Name|Value|
|---|---|
|PSVersion|7.0.0-rc.2|
|PSEdition|Core|
|GitCommitId|7.0.0-rc.2|
|OS|Darwin 19.2.0 Darwin Kernel Version 19.2.0: Sat Nov 9 03:47:04 PST 2019, root:xnu-6153.61.1~20/RELEASE_X86_64|
|Platform|Unix|
|PSCompatibleVersions|1.0 2.0 3.0 4.0 5.0 5.1.10032.0 6.0.0 6.1.0 6.2.0 7.0.0-rc.2|
|PSRemotingProtocolVersion|2.3|
|SerializationVersion|1.1.0.1|
|WSManStackVersion|3.0|

Visual Studio Code Extensions

Visual Studio Code Extensions(Click to Expand)

|Extension|Author|Version|
|---|---|---|
|auto-close-tag|formulahendry|0.5.6|
|auto-rename-tag|formulahendry|0.1.1|
|azure-account|ms-vscode|0.8.8|
|azurerm-vscode-tools|msazurermtools|0.8.3|
|beautify|HookyQR|1.5.0|
|code-runner|formulahendry|0.9.15|
|code-settings-sync|Shan|3.4.3|
|color-info|bierner|0.5.1|
|cpptools|ms-vscode|0.26.3|
|csharp|ms-vscode|1.21.9|
|debugger-for-chrome|msjsdiag|4.12.6|
|dotenv|mikestead|1.0.1|
|dotnet-test-explorer|formulahendry|0.7.1|
|ecdc|mitchdenny|1.3.0|
|EditorConfig|EditorConfig|0.14.4|
|gitlens|eamodio|10.2.0|
|Go|ms-vscode|0.12.0|
|guides|spywhere|0.9.3|
|html-css-class-completion|Zignd|1.19.0|
|java|redhat|0.55.1|
|markdown-preview-enhanced|shd101wyy|0.5.1|
|material-theme|zhuangtongfa|3.2.1|
|msbuild-project-tools|tintoy|0.3.3|
|mssql|ms-mssql|1.8.0|
|nc-gcode|ML|0.13.0|
|ng-template|Angular|0.900.6|
|npm-intellisense|christian-kohler|1.3.0|
|output-colorizer|IBM|0.1.2|
|path-intellisense|christian-kohler|1.4.2|
|polacode|pnp|0.3.4|
|powershell|ms-vscode|2020.1.0|
|python|ms-python|2020.1.58038|
|remote-wsl|ms-vscode-remote|0.41.7|
|restructuredtext|lextudio|117.0.0|
|rewrap|stkb|1.9.1|
|sort-lines|Tyriar|1.9.0|
|sublime-keybindings|ms-vscode|4.0.6|
|terraform|mauve|1.4.0|
|todo-tree|Gruntfuggly|0.0.166|
|vscode-aks-tools|ms-kubernetes-tools|0.0.4|
|vscode-arduino|vsciot-vscode|0.2.29|
|vscode-docker|ms-azuretools|0.9.0|
|vscode-eslint|dbaeumer|2.0.15|
|vscode-icons|vscode-icons-team|9.7.0|
|vscode-java-debug|vscjava|0.24.0|
|vscode-java-dependency|vscjava|0.8.0|
|vscode-java-pack|vscjava|0.8.1|
|vscode-java-test|vscjava|0.22.1|
|vscode-json-stable-stringify|TravisIllig|1.1.0|
|vscode-kubernetes-tools|ms-kubernetes-tools|1.0.9|
|vscode-markdownlint|DavidAnson|0.33.0|
|vscode-maven|vscjava|0.20.2|
|vscode-npm-script|eg2|0.3.10|
|vscode-typescript-tslint-plugin|ms-vscode|1.2.3|
|vscode-yaml|redhat|0.6.1|
|vscodeintellicode|VisualStudioExptTeam|1.2.4|
|xml|DotJoshJohnson|2.5.0|

Area-Pester Issue-Bug

Source

tillig

Most helpful comment

Done. The issue was in the location (line/file) matching part. Submitted a PR for that.

tillig on 29 Jan 2020

🚀2 🎉2

All 4 comments

@tillig thanks for reporting this issue, we are able to reproduce this. It is not an issue we will likely have the resources to solve in the short term (although we do welcome PRs and can point you in the right direction if you are interested)...it is an issue that may get addressed in the medium term as we will need to make changes to align with Pester 5...thanks!