Pester: v5: Parametrized scripts are not implemented

[nohwnd]

The syntax for parametrizing scripts is not implemented.

_originally posted by @JustinGrote on twitter_

For Pester 5, how am I supposed to run pester tests with param blocks? I don't see any docs on it, in 4 there was the rather obtuse hashtable syntax. Is this not supported in 5 and should I switch to environment variables or something?

I found https://github.com/pester/Pester/blob/v5.0/README.md#advanced-interface

But no answers here. Seems to me that $config.run.Path needs to support a hashtable in addition to string:

path = @{
  path = 'path/to/my/test.ps1'
  parameters = @{
    mycustomparam = $true
    verbose = $true
  }
}

TODO:

• [x] Single path with single set of data

@{
    Path = "C:\temp\abc.Tests.ps1",
    Data = @{
        Name = "Jakub"
        Age = 31
    }
}

• [x] Multiple instances of the same path each with it's own data

@(
    @{
        Path = "C:\temp\abc.Tests.ps1",
        Data = @{
            Name = "Jakub"
            Age = 31
        }
    }
    @{
        Path = "C:\temp\abc.Tests.ps1",
        Data = @{
            Name = "Thomas"
            Age = 29
        }
    }
)

• [x] Single path with multiple sets of data (alternative to the above)

@{
    Path = "C:\temp\abc.Tests.ps1",
    Data = @(
        @{
            Name = "Jakub"
            Age = 31
        }
        @{ 
            Name = "Thomas"
            Age = 29
        }
    )
}

• [x] Multiple different paths each with it's own data

@(
    @{
        Path = "C:\temp\abc.Tests.ps1",
        Data = @{
            Name = "Jakub"
            Age = 31
        }
    }
    @{
        Path = "C:\temp\gef.Tests.ps1",
        Data = @{
            Color = "Blue"
        }
    }
)

• [x] Providing path with wildcard should expand the path to multiple containers

• [x] Providing Path that resolves to the same path as Container should skip the paths that already have data, because those files are parametrized and would likely fail

• [ ] API review

[JustinGrote]

A thought here would be to maybe do dynamic parameters like invokebuild but I don't think that gels with your new test discover process since that would kind of have to happen first.

[nohwnd]

Dunno, I am open to proposals how to do it better. :)

[markgar]

Does this really need to be implemented? The way that passing parameters automagically get assigned and passed in as the script parameter of the same name, while very nifty, is very confusing for new users.

I used environment variables to pass values from the Invoke-Pester to the test scripts.

Any reason why this can't be the way this is done going forward?

[nohwnd]

This feature is often used for running the same script with different values, and in that case env variables don't work that well, because you would need to run one script, and then set the vars and run second script, and so on.

Env variables are also not great because you have to serialize the values instead of getting real references.

To me the confusion is mostly because of the hashtable syntax magic. Do you have different experience?

[bormm]

I used environment variables to pass values from the Invoke-Pester to the test scripts.
Any reason why this can't be the way this is done going forward?

Passing parameters to scripts by setting environment variables is:

• cmd.exe style
• it does not allow passing non-string values
• its not object oriented as anything that is powershell should be
• it produces a mess because you have not to declare the "parameter" anymore but just use "some environment variable" at some point in the code and setting at some other point
• there is no standard way to add documentation for environment variables
• there is no way to set mandatory or non-mandatory environment variables
• its like using global static variables everywhere because its easier to use in first place

[donatwork]

This is a pretty huge breaking change. I used the hash table approach to pass in runtime variables and some of these variables are objects. We can debate whether passing objects created in one script should be used in another script but doing so allows saving time during test runs where I need to pass in objects created from another test script. I also needed to pass in a credentials object. Seems like now I have to load the credential in each test script. We need a way to get that hash table back in.

[nohwnd]

Thanks for the feedback, implementing this is scheduled for one of the 5.0.x releases (which means soon :) )

[Liquidmantis]

My group is affected by this too. We have test scripts we use for intrastructure testing and call them like a regular Powershell function, passing a target server and any parameters to that calling function, which builds the hashtable to pass to Invoke-Pester (which is gross, so I'm happy there's a change coming). Maybe this could be as simple as adding an optional Parameter (or equivalent intuitive name) parameter that the hashtable can be passed to, and have Path solely be the test script path?

[PanosGreg]

I'm on the same boat here. I'm about to write some integration tests and I'm excited to use v5, but then I can't pass params to the test.

Things like that have been done in the past, namely _Invoke-Command_ or _Start-Job_ are using ArgumentList parameter.
Why not use that, it's been done before, it's well-known and it's accepted.

There's a few ways an end-user can implement it.
In the test anyone can use straight up $args[0],$args[2].etc.. in the code , or do a more proper param ($ParamName1,$ParamName2,etc..), whatever suits.

[nohwnd]

Worked on this this Saturday, not done yet.

[PanosGreg]

Thanks for the update Jacub, your work is much appreciated, keep it up!

On a second thought about the parameters, even if there's a hashtable as an input parameter, which would work much like a splatted param, that would be nice I think.

For ex.

$params = @{
  ComputerName  = 'Myserver'
  Folder        = 'C:\temp'
  CloudProvider = 'Azure'
}
Invoke-Pester -Path MyTests.ps1 -Parameters $params

# and inside MyTests.ps1:

param (
  [string]$ComputerName,

  [ValidateScript({Test-Path $_})]
  [string]$Folder,

  [ValidateSet('Azure','AWS')]
  [string]$CloudProvider
)

Do-Stuff

[nohwnd]

You at least need to have the ability to provide a multiple sets of path + parameters which your syntax would not be able to do.

Ultimately we should be able to provide:
One object with multiple sets of data to run a given script. In this example you run the script twice, once for each hashtable:

@{
    Path = "C:\temp\abc.Tests.ps1",
    Data = @(
        @{
            Name = "Jakub"
            Age = 31
        }
        @{ 
            Name = "Thomas"
            Age = 29
        }
    )
}

Many objects with the same script path and one set of data. This would do the same as the above.

@(
    @{
        Path = "C:\temp\abc.Tests.ps1",
        Data = @{
            Name = "Jakub"
            Age = 31
        }
    }
    @{
        Path = "C:\temp\abc.Tests.ps1",
        Data = @{
            Name = "Thomas"
            Age = 29
        }
    }
)

exclude any paths that are parametrized this way from the set of scripts that are matched by Path.

Given the data above and $data | Invoke-Pester -Path "C:\temp\*", this should include all test files in C:\temp\, but run C:\temp\abc.Tests.ps1 with the data, but not without it.

[PanosGreg]

I see what you're saying, you want to accommodate for the fact that there are multiple .Test.ps1 files.

But if you may, there's also a lot of times where we only have a single test file and that's what we want to run.

In that case we don't need the path, since we already know it from -Path parameter. Hence you can simplify the Invoke-Pester end-user interface for that kind of use, by providing the ability to only give a single hashtable for that.
As opposed to having a hashtable nested inside another hashtable.

Much like in-line with the new option of the simple and advanced interface for Invoke-Pester.

[donatwork]

This is PowerShell after all so what I typically do is to use a script that calls Invoke-Pester in a loop that iterates over a list of scripts. This is simple and works for me. Something like,

foreach ($test in $Tests) { Invoke-Pester -Script @{Path = $test; Parameters = @{Config = $Config } } }
Where $Config is my dictionary of parameters, strings, runtime objects, etc. I am able to have two way data flow for those tests that need to work on objects created in a previous test.

[nohwnd]

@PanosGreg I don't think that providing data for just one script is easily feasible. It would mean syntax like this:

Invoke-Pester -Path <path> -Data @(
    @{
        Name = "Jakub"
        Age = 31
    },
    @{
        Name = "Thomas"
        Age = 29
    }
)

This is all nice as long as there is just a single set of data, and single path. But because -Path allows you to specify many paths, it is not very intuitive. If this should be possible I would do it after the more advanced option is in place.

[nohwnd]

There is PR #1671 please give it a review.

[nathan-given-health-catalyst]

@nohwnd ,

Thank you for working on this. Should the -Data parameter work now with Invoke-Pester in 5.1.0-beta2 and higher?

I just tried to download the 5.1.0-beta2 release and I'm getting this error:

Invoke-Pester : A parameter cannot be found that matches parameter name 'Data'.
At line:1 char:51
+ Invoke-Pester -Path .\PlatformTelemetry.tests.ps1 -Data @{configstore ...
+                                                   ~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Invoke-Pester], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Invoke-Pester

Thank you~!

[nohwnd]

https://github.com/pester/Pester/blob/5d7b27249e616c5cd1f35448b36621bca0abd2ee/tst/Pester.RSpec.ts.ps1#L571-L575

You would do it like this, using the New-TestContainer cmdlet. This allows you to provide multiple sets of data to different files.

[mgeorgebrown89]

So, I see this issue as closed, but none of this is working for me. I was using this to test our API in different environments, but some of those are sensitive, so the values were being pulled from Keyvaults. I need paramaterized scripts but it's not working in version 5. Is it only in the beta release? I have 5.0.4

[nohwnd]

Yes this is in 5.1.0-beta1

[guidooliveira]

Any progress on this for version 5?

This issue is pretty much what's keeping me from moving to v5

[JustinGrote]

@guidooliveira it's been implemented.
https://pester.dev/docs/usage/data-driven-tests#providing-external-data-to-tests

[si-kotic]

Really nice implementation of this. Thank you