Powershell: Passing $null to a [string] parameter unconditionally causes conversion to [string]::Empty

[alx9r]

Steps to reproduce

function f {
    param (
        [AllowNull()]
        [string]
        $x
    )
    return $x
}

$r = f -x $null
$null -eq $r

Expected behavior

I expected the output to be True because I expected $null to pass through f unaltered.

Actual behavior

The output is False because $null is converted to [string]::Empty when it is assigned to $x.

Environment data

> $PSVersionTable

Name                           Value                                           
----                           -----                                           
PSVersion                      6.0.0-beta                                      
PSEdition                      Core                                            
GitCommitId                    v6.0.0-beta.5                                   
OS                             Microsoft Windows 6.3.9600                      
Platform                       Win32NT                                         
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                         
PSRemotingProtocolVersion      2.3                                             
SerializationVersion           1.1.0.1                                         
WSManStackVersion              3.0                                             

Why do I expect it to be possible for $null to remain $null when passed to a string parameter?

For every other parameter type I have tested it is possible to pass $null without conversion. I have tested the following:

function f { param([System.Nullable[int]]$x) $x }
function f { param([System.Nullable[System.DayOfWeek]]$x) $x }
function f { param([hashtable]$x) $x }
function f { param([array]$x) $x }
function f { param([System.Collections.Generic.Dictionary[string,int]]$x) $x }
function f { param([System.Collections.ArrayList]$x) $x }
function f { param([System.Collections.BitArray]$x) $x }
function f { param([System.Collections.SortedList]$x) $x }
function f { param([System.Collections.Queue]$x) $x }
function f { param([System.Collections.Stack]$x) $x }

Passing $null to any of these functions outputs $null. The only parameter type I haven't found a way to which to pass $null without conversion is [string]; PowerShell always converts $null to [string]::Empty.

PowerShell's behavior in this regard is also inconsistent with C#. The corresponding function in C# is as follows:

C# public string f(string x) { return x; }
Calling f(null) returns null.

Why does it matter that $null is unconditionally converted to [string]::Empty?

All other collection-type parameters of a function can take three different empty-ish values:

• a collection object containing no items (eg. @(), @{}, [System.CollectionsArraylist], etc)
• $null
• not bound (ie. the parameter name is not in $MyInvocation.BoundParameters.Keys)

Each of these represents different arguments at the call site. Each can be distinguished from the other inside the function unless the parameter type is [string]. For [string] parameters $null always gets converted to [string]::empty. This causes information to be lost between the call site and the function body: It is not possible to distinguish between the $null and [string]:empty arguments as they both appear as [string]::empty inside the function.

This has significant implications for a function whose parameter has different meaning for $null versus [string]::empty. In DSC resources, for example, it is natural to pass groups of parameters from the Set and Test functions down through a call stack which eventually makes a system call to manipulate some configuration. In such scenarios preserving the distinction between $null and [string]::empty often has a critical distinction in meaning. Omitting a string parameter altogether customarily means "don't make a change to the configuration thing that this parameter would affect." Passing [string]::empty to a string parameter customarily means "clear the configuration thing that this parameter affects." But as soon as a string argument whose value is null encounters a function parameter that is statically typed as [string], PowerShell unexpectedly converts it to [string]::empty. That has the effect of implicitly converting all string parameters not mentioned in the configuration from $null "don't make a change" to [string]::empty "clear the thing".

It might be possible in certain cases to rely on the "not bound" state of function parameters, however, for class-based DSC resources the "parameters" are properties of the class and there's no analog to "not bound" for such properties. Incidentally, assignment of $null to class properties of different types seems to behave exactly the same as parameter binding of each type: They all allow assignment to $null except [string].

What am I hoping for?

I'm hoping for some kind of workaround that retains the static-typing of the parameter. I'm hoping there's some way I don't know about to tell PowerShell that a parameter can be either $null or a [string].

Long-term I'm hoping that PowerShell has comprehensive support for nullable strings as C# does and PowerShell already does for other collections.

See also

https://stackoverflow.com/q/45720150/1404637

[bergmeister]

I can understand your pain and I have been caught out by this behaviour before as well.

If you simply remove the [string] type then it will work as expected:

function f2 {
    param (
        $x
    )
    return $x
}
$null -eq (f2 -x $null)

There is no value in declaring a parameter as type string because all classes in .Net have to derive from the Object class. Therefore every class implements ToString() and thus PowerShell would convert you every object to a string anyway.

However, there are more complex scenarios such as e.g. passing $null into a method that got compiled in C# by PowerShell using Add-Type. In this scenario (see here for my SO question last year)
The workaround for this is to use the NullString class that got introduced in PowerShell v3 and pass in [NullString]::Value

Although I would also upvote if it is possible to improve it going forward, this will be up to the MSFT guys since this could be quite a big breaking change and given the fact that the NullString class got introduced in v3 suggests to me that they were aware of this but did not want to make the breaking change back then. But maybe now is the time where this breaking change would not hurt as much as it would've been in previous versions?

[lzybkr]

This is by design and as @bergmeister points out - changing the behavior would be a massive breaking change.

The thinking behind the design was that in most ways, $null and the empty string both represent the same error condition and that in the rare case where a distinction was important, PSBoundParameters would be sufficient to distinguish between knowing a value was provided or not.

Let's consider the alternatives:

if ($str -eq '') { ... }  # preferred
if ($str -eq $null) { ... }  # handles empty string in a similar manner
if ($str -eq $null -or $str -eq '') { ... } # pointless in PowerShell today, but necessary if we changed it
if ([string]::IsNullOrEmpty($str)) { ... }  # an alternative to 2 tests

The first is the shortest, and in most cases, needs to be checked anyway.
The second works in a similar manner, but does invoke a conversion from $null to the empty string.
The third and fourth options would typically be necessary if PowerShell did not do this conversion.

We do occasionally hear requests to not do this conversion, and the reason is usually to be more like C#, e.g. to test a C# api and verify that it validates the arguments.

To support this scenario, we added an api, you can use it like this:

PS> [string]$s = [NullString]::Value
PS> $null -eq $s
True

[alx9r]

@lzybkr

...changing the behavior would be a massive breaking change.

I appreciate and accept this.

We do occasionally hear requests to not do this conversion, and the reason is usually to be more like C#, e.g. to test a C# api and verify that it validates the arguments.
To support this scenario, we added an api, you can use it like this:
PS> [string]$s = [NullString]::Value
PS> $null -eq $s
True

Are you implying that [NullString]::Value is meant to support passing $null to a [string] parameter? That would afford some options that might alleviate some of the pain. But it doesn't seem to do that.
[NullString]::Value also gets converted to [string]::empty when passed to a function:

function f {
    param (
        [AllowNull()]
        [string]
        $x
    )
    return $x
}

$r = f -x ([NullString]::Value)
$r.GetType().Name

Executing that snippet outputs String. $r is [string]::Empty despite that [NullString]::Value was passed to $x.

Is this how [NullString]::Value is supposed to work when passed to a function?

[lzybkr]

I can't explain this behavior.

Parameters to C# methods was the target scenario for [NullString]::Value, and I will say that might be the only reasonable scenario. Typical usage would end up coercing null to the empty string sooner than later.

In this case though, it does seem sooner is too soon, but if this was changed, I do think it would be hard to take advantage of it in a useful way.

[alx9r]

...I do think it would be hard to take advantage of it in a useful way.

I disagree. Allowing [NullString]::Value to propagate without conversion to [string]::empty is useful when optional string parameters are handled in bulk using either ValueFromPipelineByPropertyName or via hashtable splatting. Here is an example using the former demonstrating parameter passing that arises naturally when working with class-based DscResources:

class SomeDscResource
{
    #[DscResource()]
    [string]$a = [NullString]::Value
    [string]$b = [NullString]::Value
    [string]$c = [NullString]::Value
    # ...

    [void] Set () {
        $this | Set-SomeConfiguration
    }
    # ...
}

function Set-SomeConfiguration
{
    param
    (
        [Parameter(ValueFromPipelineByPropertyName = $true)]
        [string]
        $a,

        [Parameter(ValueFromPipelineByPropertyName = $true)]
        [string]
        $b,

        [Parameter(ValueFromPipelineByPropertyName = $true)]
        [string]
        $c

        #...
    )
    process
    {
        foreach ( $paramName in $PSCmdlet.MyInvocation.BoundParameters.Keys )
        {
            Write-Host "Parameter $paramName : " -NoNewline
            switch ( Get-Variable $paramName -ValueOnly )
            {
                ($null)               { Write-Host "do nothing"; break }
                ([NullString]::Value) { Write-Host "do nothing"; break }
                ([string]::Empty)     { Write-Host "clear" } 
                Default               { Write-Host "set to $paramName"}
            }
        }
    }
}

When the resource needs to be set, .a, .b, and .c are optionally assigned and .Set() is called:

$rsrcObj = [SomeDscResource]::new()
$rsrcObj.a = 'desired value'
$rsrcObj.b = [string]::Empty
$rsrcObj.Set()

Currently that outputs

Parameter a : set to a
Parameter b : clear
Parameter c : clear

which is incorrect for c as it was not assigned and should be left untouched. Rather, the result should be as follows:

Parameter a : set to a
Parameter b : clear
Parameter c : do nothing

This can be achieved today by omitting the [string] static type in all functions that might have bulk properties passed in one of such ways. That, however, requires that all future maintainers and creators of all functions that accept such [string] parameters be aware that a statically-typed [string] parameter behaves fundamentally different from all other types with respect to $null. In real DSC resources with reasonably-factored code, it is usual that parameters pass through several functions before use. It seems unrealistic to expect that static-typing as [string] is omitted (but static-typing for any other type is permitted) at each of those sites.

It seems more realistic, however, to write a comment about the merits of [NullString]::value over $null at the few sites where the default values for the parameters object or hashtable are set.

In other words, if [NullString]::Value could be passed to [string] parameters unaltered, the pain of $null conversion to [string]::empty could be managed once where the parameter is created (or at the module boundary) rather than at every possible site where such a parameter might be passed to a parameter statically-typed as [string].

[lzybkr]

[NullString]::Value turns into $null immediately when you write:

    [string]$a = [NullString]::Value

string is sealed, NullString is a completely unrelated type, so PowerShell must convert the instance to string (which it does via a call to ToString that returns null) before the assignment.

It is this eager conversion that makes it difficult to effectively use [NullString]::Value in PowerShell.

We could introduce a new special empty string to represent the notion of an unspecified value, something like:

class SentinalEmptyString
{
    static [string] Value = [string]::new()
}

Doing so would be error prone though - you would need to use [object]::ReferenceEquals([SentinalEmptyString]::Value, $other) consistently to avoid accidentally testing against random empty strings. Alternatively, you could initialize this string with unlikely value.

At any rate, I think user defined properties might be the cleaner path to solving your problem with classes - that way you can detect when the property is set versus not.

[alx9r]

[NullString]::Value turns into $null immediately when you write:
[string]$a = [NullString]::Value

Good point.

We could introduce a new special empty string to represent the notion of an unspecified value...Doing so would be error prone though...

I tried that approach, but didn't find a net improvement.

It seems like having a type that has APIs just like System.String except is _not_ special-cased by PowerShell would alleviate this problem. Then it would just be a matter of using that string type where the typical behavior involving null is needed. I have been experimenting with introducing such a type. The preliminary results are promising.

If you're interested, an example is in this gist. I've implemented IEquatable and IComparable and it seems to work well in contexts that rely on those interfaces, and it doesn't have the painful properties related to $null. I'm interested in what you think.

[lzybkr]

That's an interesting approach. I would anticipate some confusing issues that typically come up when introducing a proxy like this, but nothing specific comes immediately to mind.

I do think user defined properties are the ideal solution for this problem.

[alx9r]

(It's not my intention to re-open this conversation. Rather, I'm leaving this information here for completeness.)

PowerShell cmdlet "Strongly Encouraged Design Guideline" SD03 "strongly encourages" the use of null to indicate "unspecified" similarly as was argued above:

If your parameter needs to differentiate between 3 values: $true, $false and “unspecified”, then define a parameter of type Nullable. The need for a 3rd, "unspecified" value typically occurs when the cmdlet can modify a Boolean property of an object. In this case "unspecified" means to not change the current value of the property.

[mklement0]

@lzybkr:

I can't explain this behavior.

(A [string] parameter with default value [NullString]::Value turning to '')

@PetSerAl thinks this behavior is related to the optimization bug reported in #4312.

And the workaround proposed there indeed fixes the problem:

PS> function foo { param([string] $bar = [nullstring]::value) $null -eq $bar }; foo
False # !! broken: $bar should be $null, but is ''

# Workaround
PS> function foo { param([string] $bar = [nullstring]::value) if ($false) { Remove-Variable }; $null -eq $bar }; foo
True # OK - the mere presence of Remove-Variable in the function body fixed the problem.

Thus, unless there are considerations I'm missing, fixing #4312:

• would allow [string] _parameters_ to _default to_ $null (via default value [NullString]::Value]), as above.

• would allow passing [string] _variables_ that contain $null (by having been assigned [NullString]::Value]- directly or via a parameter default value) on as arguments to [string]-typed parameters.