Vscode-powershell: Can't run/debug Pester tests using code lens when test has TestCases

[majkinetor]

System Details

System Details Output

### VSCode version: 1.47.3 91899dcef7b8110878ea59626991a18c8a6a1b3e x64

### VSCode extensions:
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]


### PSES version: 2.2.0.0

### PowerShell version:

Name                           Value
----                           -----
PSVersion                      7.0.3
PSEdition                      Core
GitCommitId                    7.0.3
OS                             Microsoft Windows 10.0.19041
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Issue Description

When Pester test has TestCases, it can't be run with _Run test_ or _Debug test_. The test runs normally when Pester is invoked (or using F5).

Expected Behaviour

It should run.

Actual Behaviour

The vs code shows Tests Passed: 0, Failed: 0, Skipped: 0

In the bellow screenshot, the output of using _Run test_ is shown

[TylerLeonhardt]

@majkinetor just to confirm, are you using Pester 5? Can you provide the version of Pester please?

[TylerLeonhardt]

cc @nohwnd

[nohwnd]

@majkinetor could you set pester output to diagnostic in vscode options please? Then we might be able to see what is happening right from the output. I suspect line filter not matching correctly, maybe we are getting different line from the code lense than where the curly brace actually is.

[majkinetor]

I am using latest pester.

Line number seems to be correct.

Pester debug log

PS C:\Work\_\nil\jafin2k\top\top-rest\scripts\tests\pester> . 'c:\Users\mmilic\.vscode\extensions\ms-vscode.powershell-2020.6.0\InvokePesterStub.ps1' -ScriptPath 'c:\Work\_\nil\jafin2k\top\top-rest\scripts\tests\pester\security-roles.tests.ps1' -LineNumber 29 -MinimumVersion5 -Output 'Diagnostic'

Discovery: Starting test discovery in 1 test containers. 

Starting discovery in 1 files.
Discovery: Discovering tests in C:\Work\_\nil\jafin2k\top\top-rest\scripts\tests\pester\security-roles.tests.ps1 
Discovering in C:\Work\_\nil\jafin2k\top\top-rest\scripts\tests\pester\security-roles.tests.ps1.
Found 37 tests. 22ms
Discovery: Found 37 tests in 27 ms 
Discovery: Processing discovery result objects, to set root, parents, filters etc. 
Filter: (Roles management) Block did not match any of the include filters, but it will still be included in the run, it's children will determine if it will run. 
Filter: (Roles management.Creating role) Block did not match any of the include filters, but it will still be included in the run, it's children will determine if it will run.
Filter: (Roles management.Creating role.creates role) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role by user without permission) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with <Attribute> having invalid <Desc>) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with <Attribute> having invalid <Desc>) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with <Attribute> having invalid <Desc>) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with <Attribute> having invalid <Desc>) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with <Attribute> having invalid <Desc>) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.prevents creating role with already existing name) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role.allows creating role with already used name by other organization) Test did not match any of the include filters, it will not be included in the run.
Filter: (Roles management.Creating role) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Filter: (Roles management.Update role) Block did not match any of the include filters, but it will still be included in the run, it's children will determine if it will run.
Skip: (Roles management.Update role) Block is skipped. 
Filter: (Roles management.Update role.updates role) Test did not match any of the include filters, it will not be included in the run. 
Skip: (Roles management.Update role.updates role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Update role.does not update role by <Value> without permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update role by <Value> without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Update role.does not update role by <Value> without permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update role by <Value> without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Update role.does not update role with invalid data) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update role with invalid data) Test is skipped because a parent block was skipped.        
Filter: (Roles management.Update role.does not update role with already existing name for current organization) Test did not match any 
of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update role with already existing name for current organization) Test is skipped because a parent block was skipped.
Filter: (Roles management.Update role.updates role with already existing name for other organization) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.updates role with already existing name for other organization) Test is skipped because a parent block was skipped.
Filter: (Roles management.Update role.does not update not existing role) Test did not match any of the include filters, it will not be 
included in the run.
Skip: (Roles management.Update role.does not update not existing role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Update role.does not update role from other organization) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update role from other organization) Test is skipped because a parent block was skipped.  
Filter: (Roles management.Update role.does not update system role) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Update role.does not update system role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Update role) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Filter: (Roles management.Remove role) Block did not match any of the include filters, but it will still be included in the run, it's children will determine if it will run.
Skip: (Roles management.Remove role) Block is skipped. 
Filter: (Roles management.Remove role.removes role) Test did not match any of the include filters, it will not be included in the run. 
Skip: (Roles management.Remove role.removes role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Remove role.does not remove role by  <Value>  without permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove role by  <Value>  without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Remove role.does not remove role by  <Value>  without permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove role by  <Value>  without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Remove role.does not remove role with invalid data) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove role with invalid data) Test is skipped because a parent block was skipped.        
Filter: (Roles management.Remove role.does not remove not existing role) Test did not match any of the include filters, it will not be 
included in the run.
Skip: (Roles management.Remove role.does not remove not existing role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Remove role.does not remove role from other organization) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove role from other organization) Test is skipped because a parent block was skipped.  
Filter: (Roles management.Remove role.does not remove system role) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove system role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Remove role.does not remove role if role is assign to some users) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Remove role.does not remove role if role is assign to some users) Test is skipped because a parent block was skipped.
Filter: (Roles management.Remove role) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Filter: (Roles management.Retrieve role) Block did not match any of the include filters, but it will still be included in the run, it's children will determine if it will run.
Skip: (Roles management.Retrieve role) Block is skipped. 
Filter: (Roles management.Retrieve role.returns correct roles to <Username>) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.returns correct roles to <Username>) Test is skipped because a parent block was skipped.         
Filter: (Roles management.Retrieve role.returns correct roles to <Username>) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.returns correct roles to <Username>) Test is skipped because a parent block was skipped.         
Filter: (Roles management.Retrieve role.forbids vewing roles without Role_Review claim) Test did not match any of the include filters, 
it will not be included in the run.
Skip: (Roles management.Retrieve role.forbids vewing roles without Role_Review claim) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role.retrieves single role) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.retrieves single role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Retrieve role.does not retrieve single role by user   <Value>  without permission) Test did not match any of 
the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve single role by user   <Value>  without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role.does not retrieve single role by user   <Value>  without permission) Test did not match any of 
the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve single role by user   <Value>  without permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role.does not retrieve not existing role) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve not existing role) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Retrieve role.does not retrieve role from other organization) Test did not match any of the include filters, 
it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve role from other organization) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role.retrieves list of roles) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.retrieves list of roles) Test is skipped because a parent block was skipped. 
Filter: (Roles management.Retrieve role.does not retrieve list of roles by user <Value> wthout permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve list of roles by user <Value> wthout permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role.does not retrieve list of roles by user <Value> wthout permission) Test did not match any of the include filters, it will not be included in the run.
Skip: (Roles management.Retrieve role.does not retrieve list of roles by user <Value> wthout permission) Test is skipped because a parent block was skipped.
Filter: (Roles management.Retrieve role) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Filter: (Roles management) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Filter: (Path) Block was marked as Should run based on filters, but none of its tests or tests in children blocks were marked as should run. So the block won't run.
Discovery finished in 430ms.
Discovery: Test discovery finished.
Tests completed in 424ms

[majkinetor]

Here is the reproduction script: GH#2880.tests.ps1.txt

Context "https://github.com/PowerShell/vscode-powershell/issues/2880" {

    It 'applies XOR' {
        0 -xor 1 | Should -Be 1
    }

    It "applies XOR to inputs <a> and <b>" -TestCases @(
        @{a=0; b=1; c=1}
        @{a=1; b=0; c=1}
        @{a=1; b=1; c=0}
        @{a=0; b=0; c=0}
    ){
        $a -xor $b | Should -Be $c
    }
}

Here is a reproduction gif:

[nohwnd]

Yeah it says -LineNumber 29 but the curly brace starts at 35, which is what I use for matching. Maybe using myinvocation mycommand to get the location of It on the Pester side instead of the curly brace start position is the easiest way around this.

[nohwnd]

As for why it does not work when folder is not open I am not sure, I can't see that from the output.

[majkinetor]

Yeah it says -LineNumber 29 but the curly brace starts at 35, which is what I use for matching. Maybe using myinvocation mycommand to get the location of It on the Pester side instead of the curly brace start position is the easiest way around this.

Yeah, I can confirm - putting TestCases on the single line makes it work.

[TylerLeonhardt]

@majkinetor would you mind opening a separate issue about it not working when there is no folder issue? Just wanna make sure issues are talking about a specific behavior that isn't working. We've seen issues in the past where different behavior happens because of a folder is open or not.

Yeah it says -LineNumber 29 but the curly brace starts at 35, which is what I use for matching. Maybe using myinvocation mycommand to get the location of It on the Pester side instead of the curly brace start position is the easiest way around this.

@nohwnd does this mean the issue should be addressed in Pester?

[nohwnd]

@TylerLeonhardt yup. You could also fix it on your side, but fixing it in Pester will be more reliable, because your parsing logic won't get more complex, and Pester code does not care if we use the position of It or position of { for the line filter as long as they are set correctly.

[TylerLeonhardt]

Sounds good! Yes a change in Pester sounds like the right move. I'll close this as external then. Thanks folks!

[nohwnd]

Testcases fixed in Pester 5.0.4

[majkinetor]

Thanks, it works.