Runtime: System.Globalization.CultureData.GetNFIValues throws System.IndexOutOfRangeException: Index was outside the bounds of the array

[iSazonov]

In PowerShell repo we get a report with stack trace:
https://github.com/PowerShell/PowerShell/issues/12476

Unhandled exception. System.TypeInitializationException: The type initializer for 'System.Management.Automation.Tracing.PSEtwLog' threw an exception.
---> System.TypeInitializationException: The type initializer for 'System.Management.Automation.Tracing.PSEtwLogProvider' threw an exception.
---> System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.Globalization.CultureData.GetNFIValues(NumberFormatInfo nfi)
at System.Globalization.NumberFormatInfo..ctor(CultureData cultureData)
at System.Globalization.CultureInfo.get_NumberFormat()
at System.Globalization.CultureInfo.GetFormat(Type formatType)
at System.Globalization.NumberFormatInfo.get_CurrentInfo()
at System.Guid.TryParseExactD(ReadOnlySpan`1 guidString, GuidResult& result)
at System.Guid.TryParseGuid(ReadOnlySpan`1 guidString, GuidResult& result)
at System.Guid..ctor(String g)
at System.Management.Automation.Tracing.PSEtwLogProvider..cctor()
--- End of inner exception stack trace ---
at System.Management.Automation.Tracing.PSEtwLog..cctor()
--- End of inner exception stack trace ---
at System.Management.Automation.Tracing.PSEtwLog.LogConsoleStartup()
at Microsoft.PowerShell.UnmanagedPSEntry.Start(String consoleFilePath, String[] args, Int32 argc)
at Microsoft.PowerShell.ManagedPSEntry.Main(String[] args)

I tried to reproduce without success. I don't know whether the issue in in .Net or OS (Windows Server 2016). Cuture 1053 sv-SE Swedish (Sweden).

I guess the Guid ctor is called in follow code:
c# internal static readonly Guid ProviderGuid = new Guid("F90714A8-5509-434A-BF6D-B1624C8A19A2");

[msftbot[bot]]

Tagging subscribers to this area: @tarekgh, @safern, @krwq
Notify danmosemsft if you want to be subscribed.

[Sascha-69]

the culture was set to Swedish and it happens on Windows Server 2016 (10.0.14393), patching is current (at least for the part which is provided in our company via WSUS after last patch Tuesday.)

[tarekgh]

@iSazonov which net core version you are using?

[iSazonov]

Original report says about PowerShell 7.0 and PowerShell 7.1 Preview2:

• PowerShell 7.0 - .Net Core 3.1
• PowerShell 7.1 Preview2 - .Net Core 5.0 Preview3

[iSazonov]

I was able to find only one place where the exception can come from
https://github.com/dotnet/runtime/blob/1db45ed953745c0e270d09df99828cb58772f4ad/src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.cs#L2202-L2208
Which makes me think that it comes from the OS system.

[tarekgh]

@iSazonov properly you are right. we had some issues before which we have fixed awhile ago and .Net Core 3.1 should have such fixes. For this one, I don't think this is actionable from .NET side. Please try to get a repro and then report the issue to the Windows team through Windows 10 Feedback Hub. I am closing this issue here, feel free to respond back if you think we can help in anything and we'll be happy to do so.

[tarekgh]

@Sascha-69 is it possible on the machine which repro the issue to run the following code and send back the results?

```C#
[DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
public static extern int GetLocaleInfoEx(string lpLocaleName, int LCType, char [] lpLCData, int cchData);

    // ....

        const int LOCALE_SNATIVEDIGITS = 0x00000013;   // native digits for 0-9
        char[] chars = new char[100];
        foreach (CultureInfo ci in CultureInfo.GetCultures(CultureTypes.AllCultures))
        {
            int result = GetLocaleInfoEx(ci.Name, LOCALE_SNATIVEDIGITS, chars, chars.Length);
            Console.WriteLine($"{ci.Name} .. {result}");
        }

```

[Sascha-69]

I'll need a bit more time, I have seen your comment at 3:00 am, seem I can't run(or I'm not able to do run) your code on the fly.
Sorry, I'm an engineer with solid PowerShell knowledge, but even if I have wrote something in c#, it takes me time, because it isn't my daily work. It is the fun and voluntary part of my work.
But it happens once every 1 or 2 Years only ;-). Means I don't have much time for such things.

I can get it to run on my normal computer, but at the Windows Server 2016 it is missing the hostfxr.dll.
That is an expected behaviour, because there isn't .Net Core installed by default. Powershell 7 comes with self contained .Net Core (if I get it right). So there wasn't need for it.

Regards from Germany, even if the issue happens with Swedish locale

[Sascha-69]

Couldn't find sleep again. It seems you hit it, every output is 11 besides the one for sv_SE here it is 10.

[iSazonov]

@Sascha-69 You can use follow on Windows PowerShell:

$Source = @"

using System.Runtime.InteropServices;
using System.Globalization;

public class BasicTest
{
        [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
        public static extern int GetLocaleInfoEx(string lpLocaleName, int LCType, char [] lpLCData, int cchData);

        public static void Test()
        {
            const int LOCALE_SNATIVEDIGITS = 0x00000013;   // native digits for 0-9
            char[] chars = new char[100];
            foreach (CultureInfo ci in CultureInfo.GetCultures(CultureTypes.AllCultures))
            {
                int result = GetLocaleInfoEx(ci.Name, LOCALE_SNATIVEDIGITS, chars, chars.Length);
                System.Console.WriteLine("{0} .. {1}", ci.Name, result);
            }
        }
}

"@

Add-Type -TypeDefinition $Source

[BasicTest]::Test()

[Sascha-69]

Thanks again, at the end the result was the same, but my way was a bit longer.

if I exclude every result with 11, then it looks more clear now:

sv-SE .. 10
zh-CHS .. 0
zh-CHT .. 0

[iSazonov]

I see 11 on my system (Windows 10/Server 2019) for all except
zh-CHS .. 0
zh-CHT .. 0

[Sascha-69]

But were you able to check this at a 2016 server? I can reproduce this at 2016 only for now. I‘m considering to check this at more servers, especially at 2016 and 2019. 2012 R2 is fine.

[iSazonov]

I updated my previous comment.

@Sascha-69 What is a server image you use? Original ISO or customized?

[Sascha-69]

The image isn‘t in my responsibility, so I need to investigate this next week.
Wasn’t there a flag when it has been changed?
I have tried to verify this behavior on a Azure created VM(Win2016) and a local VM on my laptop, and I couldn’t reproduce it either.
But there still many reasons, why it is different to others. There are at least 3 images released by Microsoft, it could be a specific hotfix/update and of course our data center team might have changed it.
Do you know by any chance where it is stored, other than accessing it via assembly?

[iSazonov]

Oh ,I don't know about low level but it presents in GUI (see Standard digits)

[tarekgh]

Usually, the globalization data on Windows are stored in the file C:\Windows\system32\locale.nls. The UI is just to customize this data for the current user.

I'll try to send this issue to the Windows team to know if they know about it.

CC @ShawnSteele

[Sascha-69]

Thanks to you both.
It seems, that the standard digits were removed,
Don't know why and if it was with purpose.
But I don't have a clue, why this is throwing the error.

Only for testing, I have build an app with the sample code from here:

NumberFormatInfo.CurrentInfo

It is throwing the exception build against dotnet core 3.1, but it isn't doing it with the full framework. It is above my horizont, to say it is caused by one or the other.
It was only a last try to enclose this behaviour.

[tarekgh]

It is throwing the exception build against dotnet core 3.1, but it isn't doing it with the full framework. It is above my horizont, to say it is caused by one or the other.

This is because it is a different implementation. on the full framework, there is no validation on the native digits reading from the OS. .NET Core validate that because it will be wrong to have native digits count other than 10. Even in the full framework, if you try to manually set the Native Digits through the property NumberFormatInfo.NativeDigits and passing array with length not equal to 10, it'll throw. You may consult the documentation https://docs.microsoft.com/en-us/dotnet/api/system.globalization.numberformatinfo.nativedigits?view=netcore-3.1 for more info.

It seems, that the standard digits were removed

Is that what you see in the UI that @iSazonov mentioned in the comment https://github.com/dotnet/runtime/issues/35407#issuecomment-619594732? that is will be very weird as the UI doesn't allow to edit the values and instead you choose one of the offered values. could you please try to run the following command and send the output?

reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CustomLocale /s

This just to tell if there is someone installed a customer locales on your systems which even can override the OS data.

Also, could you please share the file C:\Windows\system32\locale.nls from your broken system? And tell more how did you get the used image/VM of that Windows Server 2016?

[Sascha-69]

Just for curiosity, why do we get then a length with 11? (not important)

I can see this in the GUI only:

If I run the code above, than it looks like this:

D:admin\tools>NumberFormatInfoFullFramwork.exe
Objects equal: True
Equal references: True

Properties of NumberFormat.CurrentInfo object:
CurrencyDecimalDigits: 2
CurrencyDecimalSeparator: ,
CurrencyGroupSeparator: .
CurrencyGroupSizes: { 3 }
CurrencyNegativePattern: 8
CurrencyPositivePattern: 3
CurrencySymbol: kr
DigitSubstitution: None
IsReadOnly: True
NaNSymbol: 
NativeDigits: { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }
NegativeInfinitySymbol: -∞
NegativeSign: -
NumberDecimalDigits: 2
NumberDecimalSeparator: ,
NumberGroupSeparator:  
NumberGroupSizes: { 3 }
NumberNegativePattern: 1
PercentDecimalDigits: 2
PercentDecimalSeparator: ,
PercentGroupSeparator:  
PercentGroupSizes: { 3 }
PercentNegativePattern: 0
PercentPositivePattern: 0
PercentSymbol: %
PerMilleSymbol: %
PositiveInfinitySymbol: ∞
PositiveSign: +

D:admin\tools>cd netcoreapp3.1

D:admin\tools\netcoreapp3.1>NumberFormatInfo.exe
Unhandled exception. System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.Globalization.CultureData.GetNFIValues(NumberFormatInfo nfi)
at System.Globalization.NumberFormatInfo..ctor(CultureData cultureData)
at System.Globalization.CultureInfo.get_NumberFormat()
at System.Globalization.CultureInfo.GetFormat(Type formatType)
at System.Globalization.NumberFormatInfo.get_CurrentInfo()
at Example.Main() in C:\Users\myname\source\repos\NumberFormatInfo\NumberFormatInfo\Program.cs:line 10

The regkey you did ask for is empty.

We have already checked the image, it is clean, seems something after the installation is happening.
Even physical and virtual machines are the similar, but use a different deployment.
No GPO is involved either, but the investigation is still ongoing.
Hard to accept that we don't have a clue yet.

Here is the locale.nls:
locale.zip

[tarekgh]

@Sascha-69 thanks a lot for the information. one more ask, could you please send the reg data under the key: HKEY_CURRENT_USER\Control Panel\International?

Your locale.nls looks fine to me and shouldn't cause this issue. If the requested reg data didn't give a clue, I may ask to get Travel Time Tracing debugging dump. but let's look at what you have in the reg first.

[Sascha-69]

okay in the Registry sNativeDigits looks different to the default.

[HKEY_CURRENT_USER\Control Panel\International]
"Locale"="0000041D"
"LocaleName"="sv-SE"
"s1159"=""
"s2359"=""
"sCountry"="Sweden"
"sCurrency"="kr"
"sDate"="-"
"sDecimal"=","
"sGrouping"="3;0"
"sLanguage"="SVE"
"sList"=";"
"sLongDate"="'den 'd MMMM yyyy"
"sMonDecimalSep"=","
"sMonGrouping"="3;0"
"sMonThousandSep"="."
"sNativeDigits"="123456789"
"sNegativeSign"="-"
"sPositiveSign"=""
"sShortDate"="yyyy-MM-dd"
"sThousand"=" "
"sTime"=":"
"sTimeFormat"="HH:mm:ss"
"sShortTime"="HH:mm"
"sYearMonth"="MMMM yyyy"
"iCalendarType"="1"
"iCountry"="46"
"iCurrDigits"="2"
"iCurrency"="3"
"iDate"="2"
"iDigits"="2"
"NumShape"="1"
"iFirstDayOfWeek"="0"
"iFirstWeekOfYear"="2"
"iLZero"="1"
"iMeasure"="0"
"iNegCurr"="8"
"iNegNumber"="1"
"iPaperSize"="9"
"iTime"="1"
"iTimePrefix"="0"
"iTLZero"="1"

[HKEY_CURRENT_USER\Control Panel\International\Geo]
"Nation"="221"

[HKEY_CURRENT_USER\Control Panel\International\User Profile]
"Languages"=hex(7):65,00,6e,00,2d,00,55,00,53,00,00,00,00,00
"ShowAutoCorrection"=dword:00000001
"ShowTextPrediction"=dword:00000001
"ShowCasing"=dword:00000001
"ShowShiftLock"=dword:00000001
"UserLocaleFromLanguageProfileOptOut"=dword:00000001

[HKEY_CURRENT_USER\Control Panel\International\User Profile\en-US]
"0409:00000409"=dword:00000001

[Sascha-69]

This should be the language configuration, which is being used here.

if ($using:MachineOperatingSystem -notlike "*windows*2008*") { Set-Culture en-US Set-WinSystemLocale en-US Set-WinHomeLocation -GeoId 221 #Sweden Set-WinUserLanguageList en-US -Force }

[tarekgh]

@Sascha-69 first we know what is wrong now at least :-) now I am not sure how the language configuration work especially it is setting the culture to en-US.

I think you need to know who ended up customizing the data in the registry? Usually, this data gets either customized from the UI which I doubt is used in this case as the UI will not offer this wrong option. The other way customize this data is calling NLS APIs SetLocaleInfo(Ex)

[iSazonov]

I tried the cmdlets and can not repo.
If the install image was customized they could import corrupted registry file.

[Sascha-69]

Okay, thanks a lot. Then we have to do some homework. It may take a little longer for the next answer from me, because the next steps are not on my desk.

[Sascha-69]

@tarekgh Seems it wasn't the first issue related to this topic for you, found Today a similar here:
The interesting part is, that it was also about "sv-SE". But at the end, it wasn't stated what has caused it.

[tarekgh]

@Sascha-69 would be good if you can investigate from your side and figure out who is configuring the locale user overrides with wrong data. Yes, maybe it is the same cause of the other issue you pointed at.