Powershell: Directly constructed .NET type instances lack properties that ones output by cmdlets have

[mklement0]

From what I can tell, this discrepancy has always existed, so there may be a good reason for it, but it deserves an explanation / documentation.

Specifically, where do the extra properties come from? They appear not to be part of the ETS type data as returned by Get-TypeData System.IO.DirectoryInfo (and there's also none for the parent type, Get-TypeData System.IO.FileSystemInfo).

The absence of the .PSPath property from the directly constructed [System.IO.DirectoryInfo] and [System.IO.FileInfo] instances is especially problematic, because pipeline-binding to -LiteralPath parameters is based on it.

The types used are just an example; the same discrepancy exists for other types, such as [datetime]::now vs. Get-Date.

Steps to reproduce

# Construct  seemingly identical [System.IO.DirectoryInfo] instances
([System.IO.DirectoryInfo] '/').psextended | Format-List
'---'
(Get-Item '/').psextended | Format-List

Expected behavior

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      : 

---

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      : 

Actual behavior

Mode     : d-r---
BaseName : /
Target   : 
LinkType : 

---

PSPath        : Microsoft.PowerShell.Core\FileSystem::/
PSParentPath  : 
PSChildName   : 
PSDrive       : /
PSProvider    : Microsoft.PowerShell.Core\FileSystem
PSIsContainer : True
Mode          : d-r---
BaseName      : /
Target        : 
LinkType      : 

As you can see, the PS-prefixed properties only exist on the instance output by Get-Item.

Environment data

PowerShell Core v6.0.0-beta.4 on macOS 10.12.5
PowerShell Core v6.0.0-beta.4 on Ubuntu 16.04.2 LTS
PowerShell Core v6.0.0-beta.4 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)
Windows PowerShell v5.1.15063.413 on Microsoft Windows 10 Pro (64-bit; v10.0.15063)

[lzybkr]

All provider cmdlets add these properties when writing objects to the pipeline, you can see the code here.

It would be better for performance if we used ETS, but there are likely cases where we would need to add these properties to override the ETS value, e.g. if a PSDrive was used in the path.

[mklement0]

Thanks, @lzybkr, good to know.

Hypothetically, is it technically feasible to combine the two approaches? (ETS properties that are selectively overridden by the provider cmdlets, if needed?)

Also, Get-Date is not a provider cmdlet.
Does that mean that _individual_ cmdlets also add additional properties (such as .DisplayHint in the case of Get-Date)?
Is that common, and is there an easy way to discover it across all cmdlets?

[iSazonov]

We add DisplayHint here (In Set-Date too.)
Although output type is defined as Datetime we modify it.

[mklement0]

Thanks, @iSazonov - good to know.

My overall concern is that bypassing the ETS can lead to both subtle and confusing bugs.

It is counter-intuitive for two [System.DateTime] instances to behave differently, solely based on how they were obtained.

If bypassing cannot be avoided for technical reasons (as it sounds is the case with provider cmdlets), the additional properties should at least be documented.

[iSazonov]

The trick is that it's not C# type - it is really a PSObject with same name.

[mklement0]

@iSazonov:

Understood, but that fact is normally _hidden_ from users.

Even the ETS properties are a departure from the underlying raw .NET type, but the point is that it can be confusing that something that presents as a given type _in PowerShell_ can have different properties in different contexts.

[lzybkr]

@mklement0 - I'm sure this is documented somewhere, but there are 3 kinds of members - instance (added via an api or Add-Member), type (added via types.ps1xml or Update-TypeData) - and native - .Net or AD or CIM or whatever the underlying object provides.

Instance members hide type members, and type members hide native members, so combining is definitely doable.

There isn't a super easy way to find all the places where members are added, but searching for new NoteProperty will find many of the locations - there are other types of members you can add of course, so you'd want to search for those too, but NoteProperty is the most common.

[zjalexander]

@lzybkr we are planning to review about_types soon, if this fits in there would you mind giving us some additional details during review?
thanks!

[lzybkr]

@zjalexander - sure, ask away when you have specific questions or something for me to review.

[BrucePay]

From what I can tell, this discrepancy has always existed, so there may be a good reason for it, but it deserves an explanation / documentation.

Instance members are added imperatively by cmdlets, etc, including directly with Add-Member or through the API:

PS[1] (103) >  $x = 2
PS[1] (104) >  $x.PSObject.Members.Add([System.Management.Automation.PSNoteProperty]::new("hi", "there"))
PS[1] (105) >  $x.Hi
there

Basically there is know way of knowing what properties are being added where so there's no way that

[io.file]::new("c:\temp\mypath.txt")

can add the properties that the provider would have added. Conversely, [io.file]::new() knows nothing about the current provider drive or path so you can't pass relative paths to these APIs.

More generally, we explicitly decided not to try wrapping .NET APIs because it was too fragile. There are just to many APIs to wrap. The result would be patchy and inconsistent. Currently it is at least self-consistent.

[joeyaiello]

No time no to go through this whole thread, so two questions:

• do folks feel like this has been adequately answered at this point
• if so, do folks have an idea of where it should be doc'd?

[mklement0]

do folks feel like this has been adequately answered at this point

Speaking as a [folks] instance: personally, yes.

if so, do folks have an idea of where it should be doc'd?

about_Types.ps1xml has been mentioned, but the problem discussed here is not ETS-related.
I therefore suggest about_Object_Creation.

Incidentally, the latter lacks documentation of the PSv5 static ::new() method for calling constructors - see https://github.com/PowerShell/PowerShell-Docs/issues/3209 - and deserves a bit of an overhaul in general - see https://github.com/PowerShell/PowerShell-Docs/issues/3210

[mklement0]

On second thought:

I think a note should be added to about_Types.ps1xm as well, based on @lzybkr's comment above.

The relationship between ETS and instance properties added via Add-Member is already mentioned there, but there's nothing about instances properties being _automatically_ added by _providers_ or (other) _cmdlets_.

A note in about_Object_Creation should then say that directly constructed instances (via ::new() / New-Object) may lack properties that instances of the same type obtained via _cmdlets_ may have.

[vexx32]

It strikes me as somewhat _odd_ that out of all the cmdlets, New-Object doesn't add the additional properties. 😕

[SteveL-MSFT]

@vexx32 think of it this way. You could have two cmdlets CmdletA and CmdletB that extend [System.Diagnostics.Process] in different ways. One adds a PropA and the other adds PropB (I would also expect both cmdlets to add to .pstypenames so one might be System.Diagnostics.Process#PropA and the other System.Diagnostics.Process#PropB (useful for formatting, for example). New-Object doesn't know anything about CmdletA nor CmdletB and thus doesn't know about the additional properties. Same reason ::new() wouldn't either.

[vexx32]

I guess I'm just not really... clear... on why some objects are done this way, whereas others simply have new types implemented that inherit from the original. 😄

[SteveL-MSFT]

@vexx32 I agree from a user perspective, it can be confusing why a [Process] is not the same as another [Process]. But I think the pros having this support in PowerShell outweighs the cons.

[vexx32]

I don't think there's anything inherently wrong with the approach... but I do think @mklement0 has a point in asking whether we ought to roll them into some kind of more standard member -- perhaps we should have a different binding point for these properties for consistency... Though I'm sure that's no small ask. 😄

[lzybkr]

@SteveL-MSFT describes a current pattern, but it doesn't make it a good pattern.

To use the Get-Process example, I think Get-Process -IncludeUserName is a bit gross. UserName seems important enough that it should always be available on a Process object, and ETS was designed for just that scenario.

If I want the user name in the output, then I'd rather see Get-Process | Format-Table -View ProcessWithUserName or something like that.

To expand a bit on how we abuse ETS unnecessarily in the name of formatting, I'll point out an idea first introduced by @KirkMunro in FormatPx.

The basic idea is that formatting directives are attached to an object without relying on the typename. This has the benefit of binding your formats in a script, e.g.:

function Get-Process
{
    # Use a very basic default format
    [System.Diagnostics.Process]::GetProcesses() | Format-Table Id,ProcessName
}

And use the function like this:

PS> $x = Get-Process
PS> $x[0].ProcessName
cmd
PS> $x

Id  ProcessName
==  ===========
11  cmd
22  powershell

PS> $x | Format-Table Id,SI,ProcessName

Id  SI  ProcessName
==  ==  ===========
11   1  cmd
22   2  powershell

So in this example, my function Get-Process fully specifies the default formatting while writing instances of System.Diagnostics.Process that can be used normally and later have a different format applied.

FormatPx works like this using the existing format system, and I've implemented this idea in PSMore which is just barely functional as a prototype.

In summary, I'd like to see PowerShell move away from adding members to each instance and use type members as much as possible. I think this would be a win from a performance and usability point of view.

[vexx32]

I would love to see the Format cmdlets work like that -- still return usable objects, just with the formatting directives altered -- wow!

That would save a significant amount of confusion for newbies, give us a way to simply and easily specify default formats programatically, the whole deal. Love it!

[iSazonov]

7857 for tracking PSMore.

[iSazonov]

@mklement0 I want close this issue. If you see something should be documented please open issue in Docs repo.

[msftbot[bot]]

This issue has been marked as answered and has not had any activity for 1 day. It has been closed for housekeeping purposes.

[KirkMunro]

Related to the PSMore/FormatPx discussion: https://github.com/PowerShell/PowerShell/issues/10463.