Powershell: Should all [psobject] cmdlet parameters be changed to [object]?

[mklement0]

From what I understand, [psobject] is an invisible helper type used behind the scenes.

However, some cmdlets do expose that type directly via their -InputObject parameters.

On a related note regarding _output_: When cmdlets write to the pipeline, the output objects are invariably [psobject]-wrapped:

# A number literal of type [int], for instance, is not [psobject]-wrapped
PS> 1 -is [psobject]
False

# Passing that number literal through Write-Output adds a [psobject] wrapper.
PS> (Write-Output 1) -is [psobject]
True

This extra [psobject] wrapper is _typically_ benign, but on occasion does result in different behavior - see #5579.

Is there a good reason to declare parameters as [psobject] or [psobject[]] - thereby arguably leaking an implementation detail - when [object] and [object[]] should do?

Here's the list of cmdlets that have [psobject] / [psobject[]] parameters as of PowerShell Core v6.0.0-rc:

> $type = [psobject]; Get-Command -pv c | % { if ($_.Parameters) { $_.Parameters.GetEnumerator() } } | % Value |  ? { $_.ParameterType -eq $type } | Select-Object @{ l='Command'; e={ $c }}, @{ l='Parameter'; e='Name'}, @{ l='Type'; e='ParameterType' }

Command              Parameter   Type                                 
-------              ---------   ----                                 
Format-Xml           InputObject System.Management.Automation.PSObject
Get-TypeInfo         InputObject System.Management.Automation.PSObject
Get-TypeName         InputObject System.Management.Automation.PSObject
Write-ErrorString    InputObject System.Management.Automation.PSObject
Write-StdErr         InputObject System.Management.Automation.PSObject
Add-Member           InputObject System.Management.Automation.PSObject
ConvertTo-Csv        InputObject System.Management.Automation.PSObject
ConvertTo-Html       InputObject System.Management.Automation.PSObject
ConvertTo-Xml        InputObject System.Management.Automation.PSObject
Export-Clixml        InputObject System.Management.Automation.PSObject
Export-Csv           InputObject System.Management.Automation.PSObject
ForEach-Object       InputObject System.Management.Automation.PSObject
Format-Custom        InputObject System.Management.Automation.PSObject
Format-Hex           InputObject System.Management.Automation.PSObject
Format-List          InputObject System.Management.Automation.PSObject
Format-Table         InputObject System.Management.Automation.PSObject
Format-Wide          InputObject System.Management.Automation.PSObject
Get-Member           InputObject System.Management.Automation.PSObject
Get-Unique           InputObject System.Management.Automation.PSObject
Group-Object         InputObject System.Management.Automation.PSObject
Invoke-Command       InputObject System.Management.Automation.PSObject
Measure-Command      InputObject System.Management.Automation.PSObject
Measure-Object       InputObject System.Management.Automation.PSObject
New-Event            Sender      System.Management.Automation.PSObject
New-Event            MessageData System.Management.Automation.PSObject
Out-Default          InputObject System.Management.Automation.PSObject
Out-File             InputObject System.Management.Automation.PSObject
Out-Host             InputObject System.Management.Automation.PSObject
Out-Null             InputObject System.Management.Automation.PSObject
Out-String           InputObject System.Management.Automation.PSObject
Register-EngineEvent MessageData System.Management.Automation.PSObject
Register-ObjectEvent InputObject System.Management.Automation.PSObject
Register-ObjectEvent MessageData System.Management.Automation.PSObject
Select-Object        InputObject System.Management.Automation.PSObject
Select-String        InputObject System.Management.Automation.PSObject
Set-ItemProperty     InputObject System.Management.Automation.PSObject
Sort-Object          InputObject System.Management.Automation.PSObject
Start-Job            InputObject System.Management.Automation.PSObject
Tee-Object           InputObject System.Management.Automation.PSObject
Trace-Command        InputObject System.Management.Automation.PSObject
Where-Object         InputObject System.Management.Automation.PSObject

> $type = [psobject[]]; Get-Command -pv c | % { if ($_.Parameters) { $_.Parameters.GetEnumerator() } } | % Value |  ? { $_.ParameterType -eq $type } | Select-Object @{ l='Command'; e={ $c }}, @{ l='Parameter'; e='Name'}, @{ l='Type'; e='ParameterType' }


Command         Parameter        Type                                   
-------         ---------        ----                                   
Add-History     InputObject      System.Management.Automation.PSObject[]
Compare-Object  ReferenceObject  System.Management.Automation.PSObject[]
Compare-Object  DifferenceObject System.Management.Automation.PSObject[]
ConvertFrom-Csv InputObject      System.Management.Automation.PSObject[]
New-Event       EventArguments   System.Management.Automation.PSObject[]
Write-Output    InputObject      System.Management.Automation.PSObject[]

[BrucePay]

While PSObject is largely invisible to the script author, it is explicitly not invisible to the cmdlet author. PSObject is the basis for the extended type system. Cmdlets with an -InputObject parameter should specify this parameter as being of type PSObject.

[lzybkr]

@BrucePay - IMO, this only makes sense if the cmdlet uses a PSObject api. If not, the PSObject wrapper is not necessary.

From a performance perspective, it makes sense to use PSObject as long as the PowerShell parameter binder relies heavily on the PSObject apis - a wrapper is created internally, so it's no extra work to pass that wrapper along.

But I can envision changes to the parameter binder where we could skip creating the PSObject wrapper, in which case there would be a benefit to specify object instead of PSObject, but that's a theoretical discussion right now - it's a ton of work to make said changes.

[BrucePay]

@lzybkr This was a matter of stated policy in V1 - everything was to be a PSObject. I would be happy to revisit this now :-)

[BrucePay]

OK - now I remember the reason for the policy: anything that takes [object] must also deal with [PSObject]. This means that every cmdlet that takes [object] has to do something along the lines of:

if (input is PSObject)
{
    // Do stuff
}
else
{
    // Do other stuff
}

and there have been lots of errors when people forgot to do this. Making the cmdlets take [PSObject] in the first place eliminates the need for extra code and reduces the risk of errors.

@mklement0 said

From what I understand, [psobject] is an invisible helper type used behind the scenes.

It's largely invisible at the _shell user's_ level. It is __very__ visible to _programmers_ as part of the PowerShell API.

@lzybkr Yeah - it might be nice to eliminate the [PSObject] wrappers but that would be a huge amount of work with a equally huge risk of regressions.

[mklement0]

@BrucePay:

Thanks for the clarification.

As stated before, [psobject] instances surfacing in unpredictable ways on _output_ (from an end user's perspective) is problematic too.

Combining the two aspects, to reiterate my question from https://github.com/PowerShell/PowerShell/issues/5643#issuecomment-378467986: Where is the place to discuss PowerShell "vNext", unfettered by backward-compatibility concerns - if any?

[iSazonov]

@mklement0 Is your last question still open? Maybe it should be RFC?

[mklement0]

Thanks, @iSazonov.

I guess we can start thinking about the specifics of a "vNext" RFC process once we have clarity on whether there is a fundamental willingness to pursue this at all - which I'm unclear on.

If the commitment is to everlasting backward compatibility only (with carefully managed, low-impact exceptions), we needn't think about this further.

True, I could make these very thoughts the subject of "meta RFC", but, in the absence of knowing whether it has _any_ chance of being considered, I'm hesitant to put in the work.

[iSazonov]

We need to have some very significant improvements in the engine and language to overlap the disadvantages of incompatible changes. We need to somehow organize this work.

[mklement0]

@iSazonov:

I fully agree.

My personal opinion is that there are several fundamental problems that can only be solved with massively breaking changes; to name a few:

• Error handling
• Preference-variable / common-parameter inheritance
• Quoting for external programs
• -Command CLI argument parsing
• Performance issues ([object[]] as the fundamental collection type)
  
  
  
  • While PowerShell understandably will never match the speed of Unix utilities, it is all the more important to make interop with them predictable and as painless as possible (that's where quoting and -Command parsing come in)
  
  

Additionally, countless "warts" have accumulated that could be removed in the process.

But, again: Are we willing to even _consider_ doing such a thing?

[iSazonov]

I would also like to know the plans and intentions of the team.

/cc @joeyaiello @SteveL-MSFT

[SteveL-MSFT]

@mklement0 since we've released 6.0 GA, we expectedly more adverse to big breaking changes. Moving to a 7.x release is not desirable as it bifurcates the PowerShell community further (compared to <=4, 5.1, and 6 users today).

If there's an opportunity to enable some of the features, but as opt-in, that might be acceptable but also creates its own problems of potentially splitting the community.

[iSazonov]

We have a local PHP-based service. At times we have to update the versions and it is always a huge headache due to backward compatibility problems of PHP. I suppose we need to have good reasons for such a change like implementing ultra modern concepts, increasing productivity by dozens of times, supporting critical scenarios.

In any case, the question is how we need to organize a working process to gather and discuss such ideas. For example, the current discussion is simply closed.

[SteveL-MSFT]

We can continue this discussion here and have separate issues to discuss other breaking changes

[mklement0]

@SteveL-MSFT:

Thanks, but just to clarify: you now want to discuss the _meta_ issue of how to handle fundamental, but breaking improvements _here_?

[SteveL-MSFT]

@mklement0 for the meta issue, let's have a new issue and see how that goes

[Jaykul]

You'd be better off using dynamic instead of object don't you think?

For what it's worth, even in _pure script_ there have been problems in the past if you have an [object] instead of a [psobject] and you try to extend said object with add-member but the extensions fall off along the way (add-member wraps the object in a psobject, but if you were holding the object at the time, messy things). Most of these problems have been fixed -- but I'd be worried that changing parameter types would introduce a bunch of tiny edge case regressions like that.

[mklement0]

@SteveL-MSFT: Sounds good; please see #6745.

[lzybkr]

@Jaykul - there are edge cases with add-member around values that are shared as an optimization - e.g. string interning and the moral equivalent for some specific value type values (like true, false, common int32 values which are shared to minimize boxing).

These instances do benefit from the PSObject wrapper, though one can argue that you shouldn't add a member to such values like true or false.

[alx9r]

Delayed scriptblock binding also hinges on the distinction between object and psobject (see #6419).