Powershell: Generic type definitions should be included in TypeNames for PSObject

[SeeminglyScience]

In the same vein as #7649, it would be great if you could assign type data to generic definitions of constructed generic types.

For example, List<int> would contain the type names:

System.Collections.Generic.List`1[[System.Int32, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]
System.Collections.Generic.List`1
System.Object

And would allow Update-TypeData to target any constructed version of List<T>

Environment data

PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.1.0-rc.1
PSEdition                      Core
GitCommitId                    6.1.0-rc.1
OS                             Microsoft Windows 10.0.15063
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[vexx32]

Tangentially related: why do generic type names in PS have those odd characters?

[BrucePay]

@SeeminglyScience The concrete generic types are already included in TypeNames, but use the raw reflection type name:

PS[1] (8) >   $x = [system.collections.generic.list[object]]::new()
PS[1] (9) > $x.psobject.typenames
System.Collections.Generic.List`1[[System.Object, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Object
PS[1] (10) > update-typedata -force -typename 'System.Collections.Generic.List`1[[System.Object, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]' -MemberType scriptmethod -membername doit -value {"doit"}
PS[1] (11) > $x.doit()
doit
PS[1] (12) >

Using the raw name in TypeNames is arguably a bug but correcting it would be a breaking change. As a non-breaking alternative, we could allow the the use of the sanitized name when specifying extensions converting it to the raw name in the type table.

[SeeminglyScience]

@BrucePay I'm asking for the generic definition to be added to the list. For example List<T> is the generic definition of List<object>. Not sure if my terminology here is technically correct, just going by the naming of Type.GetGenericTypeDefinition()

The example you gave works, but only if you want to target List<object> explicitly. Put more simply I'd like to be able to do something like this:

using namespace System.Collections.Generic

Update-TypeData -Force -TypeName 'System.Collections.Generic.List`1' -MemberType ScriptMethod -MemberName DoIt -Value { 'doit' }

[List[object]]::new().DoIt()
[List[int]]::new().DoIt()
[List[System.IO.FileSystemInfo]]::new().DoIt()

[mklement0]

@vexx32: The `<n> suffix is .NET's notation for indicating a generic type's number of type parameters.

It is required for an _open_ generic type (one whose type parameter(s) have not been bound yet; e.g., [System.Collections.Generic.List`1]), but optional for a _closed_ generic type (a generic type with all its type parameters instantiated with specific types; e.g., both [System.Collections.Generic.List`1[int]] and [System.Collections.Generic.List[int]] work).

[mklement0]

@SeeminglyScience: The term that applies to what you're looking for is _open_ generic type.

I haven't looked too hard, but here's a definition from https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/generics/generics-and-attributes (emphasis added):

[...] _open_ generic types, which are generic types for which no type arguments are supplied, and _closed_ constructed generic types, which supply arguments for all type parameters.

[BrucePay]

@mklement0 @vexx32 The raw form of the typename is what .NET returns when you ToString() a type. In PowerShell V1, the native .NET representation is the only form we used. In V2 we added type name sanitization. The <backtick><number> in the type name is the generic arity i.e. how many type parameters there are on a given generic type.

[BrucePay]

@SeeminglyScience Yikes - I hadn't really considered open generic types. I'll need to think about that. @daxian-dbw @lzybkr @powercode - you guys have any thoughts?

[mklement0]

Thanks, @BrucePay; it's what I meant to say with "indicating a generic type's number of type parameters", but it's good to know the official - and more concise - name for it, _generic arity_.

In V2 we added type name sanitization

Are you referring to the option of omitting, e.g.,

• `1 from [System.Collections.Generic.List`1[int]] (1..10)
• and not requiring assembly-qualified type names for the type _arguments_ (just [int] rather than [System.Int32, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e])?

Because Get-Member and .pstypenames still reflect the raw type name (only).

A tangent, for sure, but perhaps you happen to know:

In the raw .NET string representation, what is the benefit of mixing the _full-type-name-only_ generic-type name (e.g., System.Collections.Generic.List) with the _assembly-qualified_ names of the type _arguments_ (e.g., [System.Int32, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e])? Why not consistently use _either_ assembly-qualified type names _or_ full type names only?

[SeeminglyScience]

Just to fully flesh out that yikes, I'd really like the same to be possible with open generic interfaces (bringing your issue into the fold). So [System.Collections.Generic.List[string]]::new().PSTypeNames would be this monstrosity

System.Collections.Generic.List`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.List`1
System.Collections.Generic.IList`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.IList`1
System.Collections.Generic.ICollection`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.ICollection`1
System.Collections.Generic.IEnumerable`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.IEnumerable`1
System.Collections.IEnumerable
System.Collections.IList
System.Collections.ICollection
System.Collections.Generic.IReadOnlyList`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.IReadOnlyList`1
System.Collections.Generic.IReadOnlyCollection`1[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]
System.Collections.Generic.IReadOnlyCollection`1
System.Object

I don't know how much of an issue that would end up being, perhaps the solution lies outside of PSTypeNames. To again put it more simply, it would be great if this was possible:

using namespace System.Collections.Generic

Update-TypeData -Force -TypeName 'System.Collections.Generic.IEnumerable`1' -MemberType ScriptMethod -MemberName DoIt -Value { 'doit' }

[List[object]]::new().DoIt()
[List[int]]::new().DoIt()
[List[System.IO.FileSystemInfo]]::new().DoIt()
[string[]]::new(0).DoIt()
[System.Collections.ObjectModel.Collection[int]]::new().DoIt()

[mklement0]

@SeeminglyScience:

I think we should consider a separate .psinterfacenames property that parallels the .pstypenames property.

• It would provide a clean separation between non-interface types and interfaces.

• As a separate, new property, neither raw type names nor closed constructed types need be included (as there is no need for backward compatibility).

  • If type-argument-specific behavior is desired, though, the type-arguments could be included as well, such as System.Collections.Generic.IList[string] in addition to System.Collections.Generic.IList`1

• ETS definitions would then have to consult .psinterfacenames too.

For instance, [System.Collections.Generic.List[string]]::new().psinterfacenames would then be limited to this (still sizable) list (without type-argument-specific variants):

System.Collections.Generic.IList`1
System.Collections.Generic.ICollection`1
System.Collections.Generic.IEnumerable`1
System.Collections.IEnumerable
System.Collections.IList
System.Collections.ICollection
System.Collections.Generic.IReadOnlyList`1
System.Collections.Generic.IReadOnlyCollection`1

[lzybkr]

@mklement0 - specifying arity in the open generic type name is a convention introduced by C# to support types like Tuple, Action, and Func, but it is not a requirement in the CLR.

@BrucePay - the obvious scenario would be LINQ - basically providing extension methods in PowerShell without needing to implement extension methods or generic classes.

I think it wouldn't be hard to add support for both types and formats, but there might be some performance implications as the list of type names grows - you'll potentially notice that adding interfaces as well.

[mklement0]

@lzybkr: But If the .ToString() method returns this arity notation too (e.g., [System.Collections.Generic.List`1].ToString() yielding System.Collections.Generic.List`1[T]), doesn't that imply that it's not (just) a _C#_ convention?

[lzybkr]

@mklement0 - the value returned from Type.ToString() may or may not have any correlation with what a compiler generates. The Type.FullName property is the name the compiler generates.

The convention that C# introduced and every language I know of is to append the arity, but the CLR does not require this.

I'll leave it as an exercise for the reader, but you could use ilasm or the reflection apis to create a generic type with any name you like. Other language might or might not be capable of consuming this type, but the CLR has no problems with such types.

[mklement0]

Good to know, @lzybkr, thanks. So how, specifically, does a _PowerShell_ expression such as [System.Collections.Generic.List`1].FullName yield System.Collections.Generic.List`1? Is there a behind-the-scenes C# detour?

[SeeminglyScience]

@mklement0 I believe what he's saying is that the compiler adds the arity to the type name directly as a convention. And that if you manually create a generic type you can skip that if you'd like. e.g.

using namespace System.Reflection
using namespace System.Reflection.Emit

$assemblyBuilder = [AssemblyBuilder]::DefineDynamicAssembly(
    [AssemblyName]::new('TestAssembly'),
    [AssemblyBuilderAccess]::Run)

$moduleBuilder = $assemblyBuilder.DefineDynamicModule('TestModule')
$typeBuilder = $moduleBuilder.DefineType(
    'MyGeneric',
    [TypeAttributes]'Public, Class, Abstract',
    [object])

$null = $typeBuilder.DefineGenericParameters('T')
$null = $typeBuilder.CreateType()

[MyGeneric[int]].FullName
# MyGeneric[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]

[mklement0]

Thanks for the detailed and illuminating example, @SeeminglyScience. I realize that this tangent may be primarily for my own edification, but if you'll indulge me:

I gather that it's PowerShell - perhaps the sanitizing that @BrucePay referred to - that ignores a `<n> suffix if it aligns with the actual arity, so that even if you passed a name of 'MyGeneric`1' as the type name to $moduleBuilder.DefineType() you could still instantiate that type later as, for instance, [MyGeneric[int]] from PowerShell, correct?

Does PowerShell itself explicitly implement the arity-suffix convention when it evaluates expressions such as [System.Collections.Generic.List[int]].FullName?

[SeeminglyScience]

@mklement0 The parser creates a TypeExpressionAst to represent a type literal expression. That AST has a property TypeName that represents the type with an implementation of ITypeName. When it creates a TypeExpressionAst for a constructed generic type, the TypeName is of the type GenericTypeName.

When PowerShell goes to actually resolve the type, it calls ITypeName.GetReflectionType(). The implementation of that for GenericTypeName will first attempt to resolve the name as is, then it will append the arity if that was unsuccessful. Here's that method.

[mklement0]

I appreciate the explanation, @SeeminglyScience.
The way the method is written, you can accidentally thwart the implicit arity-suffix logic with a type name such as MyGeneric`Foo`1, but that's probably not a real-word concern.

[powercode]

@mklement0 There is a ToStringCodeMethods class with a string Type(Type type, bool dropNamespaces = false, string key = null) member that pretty-prints typenames.

[powercode]

The scenario seems valid. It would be interesting to see some motivating examples. How does the code look like when we extend an open generic type? Do we need to fix calling of generic methods to make this useful?

We should understand the whole scenario better before adding the PSTypenames.

[mklement0]

Thanks, @powercode, good to know how to get a friendly type representation even for generic types, but note the specific overload you mention is internal. There is a public overload - public static string Type(PSObject instance), expecting a type, which actually breaks with @SeeminglyScience's code ("'length' must be non-negative), but for CoreFx types it works well, e.g., [Microsoft.PowerShell.ToStringCodeMethods]::Type([System.Collections.Generic.List`1[[System.Object, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]]) returns System.Collections.Generic.List[System.Object]

But I'm happy to close this tangent now.

[SeeminglyScience]

@mklement0

The way the method is written, you can accidentally thwart the implicit arity-suffix logic with a type name such as MyGeneric`Foo`1, but that's probably not a real-word concern.

Shouldn't really be an issue unless there's a CLR language out there that parses it in a class name (neither PowerShell nor C# do). Other than that, if someone is using emit to generate a type with a class name that contains `, they probably don't really intend for the type to be resolvable.