Pester 4.0 Release candidate

[nohwnd]

After few months of idling in our development branch the v4 Release Candidate is finally here, and you are all invited to get it and try it out. If you do please report back how it went, especially if you get errors :)

Current version is 4.0.2 and it is available directly on PowerShell gallery, and with prerelease switch on nuget and chocolatey.

All the latest stuff is merged forward from the v3 so all the features you know are there. There is a single thing missing and that is different color profiles for different editors, which you probably not even noticed was there.

So what is new?

Assertion syntax

As with every version, there is a new syntax for assertions, this time it looks like this:

1 | Should -Not -BeNullOrEmpty
{ throw "I know exactly what I am doing." } | Should -Throw 

Not much has changed, you only need to add dashes in front of Not, Throw, Be, BeLike (etc.), because they became parameters. This and internal changes open Should for extension so we can finaly give you more advanced assertions.

The old syntax still works so chances are your test base will _just work_ with v4. We will later deprecate the old syntax, but the will be announced later.

Assertion input (Update January 21)

Should Be now consumes whole collections from the pipeline and compares them as arrays, as highlighted in #696, so the following will not pass in v4:

Describe 'Should Be when piped multiple values' {
     It 'passes in v3 but not in v4' {
            @( $true, $true, $true ) | Should Be $true
     }
}

This is the first step towards assertions that are easy to use and easy to understand, because 90% of the time this should not pass.

New colors

We have new colors that look better than the previous ones, and the summary is also colored summary.

In Visual Studio Code the colors look totally awesome!

Nested Context and Describe

Context and Describe can be nested as deep as you see fit. You are no longer limited to just two levels.

I would still recommend to put the tested code in the most-child Describe/Context only, and use the parents exclusively for organizing the tests.

Gherkin support

An alternative syntax was added to support BDD language called Gherkin. Any examples @Jaykul ? :)

Deprecated and gone features

• -Quiet parameter is depracated, use -Show None
• New-TestDriveItem as most of the people do not even know it exists
• -OutputXml is gone, it was deprecated before, use -OutputFormat and -OutputFile

Huge thanks to @dlwyatt , @Jaykul , @it-praktyk and other contributors!

[Jaykul]

Probably the best current example of Pester Gherkin is my Configuration module, which has all of it's tests in this format.

Did we get the Invoke-Pester to automatically run Pester and Gherkin, or do we have to run Gherkin separately? (I'll just install it and try...)

[nohwnd]

@Jaykul Invoke-Gherkin imho, but I could not find any example of it being used, and Dave mentioned that he might have broken it. You are probably the only one in the world who can tell if it works.

[Jaykul]

LOL. Well, Dave and I had agreed, in principle, that we ought to rename Invoke-Pester to Invoke-PSpec (or something), and have a new Invoke-Pester that properly runs (all) your tests, regardless of which sort of BDD you prefer ;-)

I was hoping that had been done.

[nohwnd]

Understood. We can do that in v5 or v6 (which does not have to be in two years), I would like third syntax as well, one that looks like PowerShell. And few more changes.

[johlju]

You added commas to the summary output in last 3.x.x release. That is not in v4. Or is the new color palette a replacement?

[nohwnd]

@johlju saw that yesterday as well, after announcing the RC and did not want to do another package. If you look in the code it's there, but it does not show in the output, I need to have another look at it, or someone else may, should be easy to fix. #684

[it-praktyk]

In my opinion README.md need to be updated before releasing Pester v. 4.0.x RTM.

Specially a part related to the new Gherkin based tests need to be created. BTW, what is the best source about Gherkin? https://cucumber.io/docs ?

[nohwnd]

@it-praktyk you are right, thanks. It is on my todo list, created #687 for it.

[irwins]

Executing all tests in System.Collection.Hashtable

Got this when passing parameters...

$testADInfra = @(
   @{ 
      Path = $pesterFile
      Parameters = @{ 
         Source = $src
         Target = $tgt
      } 
   }
)

Invoke-Pester -Path $testADInfra -PassThru -Tag $tag

Rg./Irwin

[johlju]

Tried running the v4.0 on all the tests over at PowerShell/xSQLServer. I'm getting a strange error that I did not got in 3.4.3. But this was actually a bug in the test. This was not caught in 3.4.x. Improvement in 4.0 👍 😄

v3.4.x working: https://ci.appveyor.com/project/johlju/xsqlserver/build/4.0.305.0#L1609
v4 failed: https://ci.appveyor.com/project/johlju/xsqlserver/build/4.0.307.0#L1881

   Context When using SourceCredential parameter and SQL Server version is 13 and the system is not in the desired state for default instance
     [-] Should return the same values as passed as parameters 314ms
       You did not declare a mock of the Get-CimInstance Command in module MSFT_xSQLServerSetup.
       at line: 636 in C:\Program Files\WindowsPowerShell\Modules\Pester\4.0.2\Functions\Mock.ps1
     [+] Should not return any names of installed features 200ms
     [+] Should return the correct values in the hash table 233ms

Just want to report positive things too 😄

[nohwnd]

@johlju awesome!

@irwins I need to run it against v3 to see what it should do.

[dfinke]

How about a -Wait like npm has?

[it-praktyk]

The parameter Quiet should be described in help - like was previously - and information about deprecation should be added.

BTW, the parameter Show is not described in v. 3.4.6.

[it-praktyk]

If code cleaining (using the PSScriptAnalyzer module) should be a part of preparation to release Pester v.4 RTM ?

If PSScriptAnalyzer analyzer rules shuld be used as a part of 'Styles rules' ?
At least the rules

• PSAvoidUsingCmdletAliases
• PSUseDeclaredVarsMoreThanAssignments
• PSUseOutputTypeCorrectly
• PSAvoidDefaultValueForMandatoryParameter
• PSUseCmdletCorrectly
  should be - in my opinion - mandatory.

[nohwnd]

@dfinke This is for monitoring test files in the current folder and running them on save, right? I think we discussed this before, and concluded that it would best fit in external module, but if enough people want it, then it can be included in v4, but not necessarily the initial release.

@it-praktyk you are right about the -Quiet I don't know what I was thinking (in my defence it was 1 in the morning when I did those changes.)

The style rules I agree. And I can try running script analyser and see where Pester fails to meet the rules, but unless complying with the rules would mean breaking changes to the API then it is not that critical to do it in the initial release. Or if you want to do it and post report then please start a new issue for that.

[Jaykul]

In my opinion, PSAvoidUsingCmdletAliases is just noise, particularly where it concerns the *-Object commands, and just for the record, PSUseCmdletCorrectly complains about such things as shortened forms of parameter names, like Write-Information -Message "This is very informative"

If you're going to check for those two, we should run my Expand-Aliases on the whole module before hand...

[nohwnd]

@Jaykul I understand, There are already rules for whitespacing and I am sick of fixing whitespace manually, when it could have fix itself. Same goes for tagging, which fails for everyone all the time (except for us when we are ready to deploy). So I will be careful with adding more non-functional requirements (discussed here) that should fail the build. Ideally all the checks would run just before release, and for starters only against the public api.

[ronaldborman]

Is there a minimal version of PowerShell required?

We are still on PowerShell 2.0 and if possible I would like to avoid an update of PowerShell. No problems encountered when running Pester 3.x however. We would be using Pester 4.0 especially to take advantage of the (new) Gherkin syntax.

[Jaykul]

@nohwnd by the way, there's something broken in gherkin's counting in this release. I'm getting reports like:

Scenarios Passed: 3 Failed: 0

for a tests run that had over 60 scenarios, including more than a few failures -- so I clearly need to take a look at it tonight and add some more tests to Pester.

@ronaldborman I believe it continues to support PowerShell 2 -- I know that I had to do a bit of work to get Gherkin to work on the old .Net versions...

[majkinetor]

Still no Gherkin examples ?

[it-praktyk]

The basic example is in the folder https://github.com/pester/Pester/tree/master/Examples/Gherkin.

We are welcome any pull request, especially for a documentation.

[nohwnd]

Closing this as it will be covered in the release notes of the new release.