Runtime: Support for enumerating non-default code pages added with RegisterProvider (System.Text.Encoding)

[mklement0]

The documentation for System.Text.Encoding.GetEncodings() tells us (emphasis added):

The list of supported encodings returned by the GetEncodings method does not include any
additional encodings made available by any EncodingProvider implementations that were registered by calls to the RegisterProvider method.

Is there no way to enumerate these additional encodings?

Unfortunately, System.Text.CodePagesEncodingProvider.Instance lacks a method for enumerating them.

[iSazonov]

I would say that it is very surprising that after registering a new provider we can not get the full list of supported code pages.

[danmosemsft]

@tarekgh

[iSazonov]

First we discussed this in PowerShell Core repo. Some cmdlets have -Encoding parameter. If we want to add IntelliSense support for full set of code pages then we have the problem.

[tarekgh]

Although I can see the issue, generally people interested in specific codepage encoding they already knew this codepage number and can just use it. another scenario would be getting the codepage name or number from some contents (like Xml, html, network stream,...etc.) and create encoding object. in such cases enumeration is not needed anyway.

Could you tell exactly your scenario you want this? I understand power shell case but again this is not the scenario people use it. looking at the usage of System.Text.Encoding.GetEncodings, I am seeing it has very low usage.

If you really in need for that, you can file an API proposal for adding GetEncodings method in the class EncodingProvider. and have the implementation support it.

[mklement0]

@tarekgh:

To elaborate on the scenario mentioned by @iSazonov:

PowerShell Core users may want to _discover_ available encodings and/or be assisted in selecting a specific encoding with _tab completion_:

Get-Content file.txt -Encoding <tab>  # should show all available encodings

The above is currently limited to the .NET Core default encodings, even though PowerShell registers additional encodings on startup.

[tarekgh]

@mklement0 I understand this and I am not objecting it. my point is this is not a real app scenario or real user scenario. in another word, my experience is not many people need to enumerate the code pages.

Feel free to submit an API proposal as I mentioned in my previous reply.

[mklement0]

@tarekgh:

Thanks; I am glad you understand. It sounded like you wanted more details, based on this question of yours, in response to which I posted my elaboration.

Could you tell exactly your scenario you want this?

As for deeming this not a real-world scenario:

but again this is not the scenario people use it.
looking at the usage of System.Text.Encoding.GetEncodings,

Of course, it's impossible to see usage from the PowerShell side of things yet, given that the feature currently _doesn't exist_ - for lack of API support (as well as the relative youth of PowerShell Core overall and, specifically, the -Encoding parameter's ability to accept Encoding instances directly).

I will submit an API proposal.

[iSazonov]

Also we have a request to implement new cmdlet for converting encoding like iconv. iconv has --list parameter so users can get a list of supported code pages. Today we can't do this effectively.
I suppose we could add a new overload (maybe with optional parameter) GetEncodings(bool IncludeExtendedCodePages = false).

[tarekgh]

given that the feature currently doesn't exist - for lack of API support

Does the desktop version of power shell support Get-Content file.txt -Encoding? I mean the version of power shell which used to depend on the full framework and not net core.

if so, this would be a good thing to look at the usage of this functionality in power shell there.

[mklement0]

@tarekgh:

Unfortunately, _Windows PowerShell_ (the Desktop edition of PowerShell built on the full .NET framework) does _not_ support Encoding instances directly (only a limited set of enumeration values that map onto a handful of common encodings).

In _PowerShell Core_'s cross-platform world, encoding conversions strike me as more common, and presumably that's why that ability was added.

@iSazonov's example of an iconv-like cmdlet - green-lighted, but not yet implemented - is indeed another good use case for the ability to enumerate all available encodings.

[mklement0]

@tarekgh:

Please see my API proposal at dotnet/corefx#28944.