Psscriptanalyzer: An exception is thrown when CurrentCulture is Turkish (tr-TR)

[alatas]

I got an exception when trying to use Invoke-Formatter command. And also, VSCode extension couldn't format the ps1 files, or couldn't analyze the files. When I called the formatter command from the console an exception is thrown that "The given key 'ıncluderules' was not present in the dictionary."

I searched the source code and found that the exception is related to this line:

https://github.com/PowerShell/PSScriptAnalyzer/blob/1f855ac693c6d8dcc9c87ecbf032178ca192cc53/Engine/Settings.cs#L398

There is a problem with the ToLower function in .net when the CurrentCulture is Turkish (tr-TR). Turkish charset is an alternate version of latin-1 charset. Turkish alphabet has ı and i characters which are different from each other and has different uppercase forms. I is the uppercase form of ı, and İ is the uppercase form of i. Therefore, ToLower function has an unexpected behaviour different than the latin-1 charset. When the CurrentCulture is (tr-TR), Include becomes ınclude (not include) which causes this exception.

Steps to reproduce

Invoke-Formatter -ScriptDefinition "`$test"

Expected behavior

$test

Actual behavior

Invoke-Formatter : The given key 'ıncluderules' was not present in the dictionary.
At line:1 char:1
+ Invoke-Formatter -ScriptDefinition "`$test"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidData: (CodeFormatting:String) [Invoke-Formatter], KeyNotFoundException
+ FullyQualifiedErrorId : SETTINGS_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeFormatterCommand

PS 6.1

PSMessageDetails      :
Exception             : System.Collections.Generic.KeyNotFoundException: The given key 'ıncluderules' was not present i
                        n the dictionary.
                           at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.parseSettingsHashtable(Hashtable set
                        tingsHashtable)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.parseSettingsFile(String settingsFil
                        ePath)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.Create(Object settingsObj, String cw
                        d, IOutputWriter outputWriter, GetResolvedProviderPathFromPSPath`3 getResolvedProviderPathFromP
                        SPathDelegate)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeFormatterCommand.BeginProcessi
                        ng()
TargetObject          : CodeFormatting
CategoryInfo          : InvalidData: (CodeFormatting:String) [Invoke-Formatter], KeyNotFoundException
FullyQualifiedErrorId : SETTINGS_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeFormatterCommand
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}

PS 5.1

PSMessageDetails      :
Exception             : System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionar
                        y.
                           at System.ThrowHelper.ThrowKeyNotFoundException()
                           at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.parseSettingsHashtable(Hashtable set
                        tingsHashtable)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.parseSettingsFile(String settingsFil
                        ePath)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Settings.Create(Object settingsObj, String cw
                        d, IOutputWriter outputWriter, GetResolvedProviderPathFromPSPath`3 getResolvedProviderPathFromP
                        SPathDelegate)
                           at Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeFormatterCommand.BeginProcessi
                        ng()
TargetObject          : CodeFormatting
CategoryInfo          : InvalidData: (CodeFormatting:String) [Invoke-Formatter], KeyNotFoundException
FullyQualifiedErrorId : SETTINGS_ERROR,Microsoft.Windows.PowerShell.ScriptAnalyzer.Commands.InvokeFormatterCommand
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {}

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0
PSEdition                      Core
GitCommitId                    6.1.0
OS                             Microsoft Windows 10.0.14393
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.17.1

[bergmeister]

Thanks for the report and detailed analysis. We'd be happy to accept a PR with a fix for this. I think using the overload with InvariantCulture might fix this.

[alatas]

I've offered InvariantCulture in a very similar problem in Roslyn. The team didn't accept InvariantCulture because it has bugs that will never be fixed. (the discussion is here)

In this case, It won't cause so much trouble. But, I think, "en-US" culture is more suitable. I'll open a PR and offer that solution.

[bergmeister]

@alatas
Thanks. I can reproduce now using [System.Threading.Thread]::CurrentThread.CurrentCulture = [cultureinfo]::CreateSpecificCulture("tr-TR"); invoke-formatter "foo"
Using ToLowerInvariant() seems to solve it

"IncludeRules".ToLower([cultureinfo]::CreateSpecificCulture("tr-TR")).equals("includerules") # returns false
"IncludeRules".ToLowerInvariant().equals("includerules") # returns true

[alatas]

@bergmeister I've just opened a PR including this fix and a test

[bergmeister]

@alatas Thanks. I saw it but your approach seems to break other cases. Don't worry I will take it from there. I submitted an alternative PR using ToLowerInvariant, let's see if that works better.

[bergmeister]

@alatas Thanks for your efforts. My PR above passes CI, we will take that one instead then. Thanks for the detailed report and initiative from your side!

[alatas]

@bergmeister thank you, I saw that the CI problems related with my test code. I used Set-Culture command which is not available other than windows. I think that causes the problem.

I appreciate that the problem is solved, thanks for your effort