Psscriptanalyzer: Auto Format corrupting paths in my scripts

[ili101]

Issue Type: Bug

1. Call a file in your script with "." or "&" with a relative path like & ..\Desktop\Temp\Test.ps1 (The file need to exist).
2. Trigger the Formatter by hitting Enter or Alt+Shift+F or any other way.
3. The path to the file is deleted.

This is a major issue as opening a big script, changing a line (That triggers the auto format) will result in all paths in the document to corrupt braking the script! And if you saved and don't have a backup you are in trouble.

Extension version: 1.12.0
VS Code version: Code 1.32.3 (a3db5be9b5c6ba46bb7555ec5d60178ecc2eaae4, 2019-03-14T23:43:35.476Z)
OS version: Windows_NT x64 10.0.17763

EditorServices.log
vscode-powershell.log

System Info

|Item|Value|
|---|---|
|CPUs|AMD Ryzen 7 1800X Eight-Core Processor (16 x 3593)|
|GPU Status|2d_canvas: enabled
checker_imaging: disabled_off
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
native_gpu_memory_buffers: disabled_software
rasterization: enabled
surface_synchronization: enabled_on
video_decode: enabled
webgl: enabled
webgl2: enabled|
|Memory (System)|15.95GB (9.09GB free)|
|Process Argv||
|Screen Reader|no|
|VM|50%|

[SydneyhSmith]

@ili101 thanks for reporting this, do you use any code formatting settings?

[ili101]

@SydneyhSmith No. Also tested now on a clean system, same problem.

[rkeithhill]

I can repro this with the default code formatting settings. I can't repro this with PSSA 1.18 from the command line when I supply no settings. I "suspect" it is one of the PSSA settings that is causing this issue since formatting is done by PSSA.

[rkeithhill]

I can repro this outside of PSES so I do think it is a PSSA bug:

Note that I'm doing this test of Invoke-Formatter in the normal VSCode terminal/pwsh.exe - not the PSIC. Maybe we should move this issue to the ScriptAnalyzer repo? cc @bergmeister

[bergmeister]

It is most likely a PSSA issue, I have seen something similar before but it was a very convoluted case that was hard to repro/debug. This looks much better, I'll see if I can repro and then we should move it to the PSSA repo.

[rkeithhill]

Note that while I was able to repro in pwsh running in VSCode, I wasn't able to repro in a stand-alone pwsh.exe. That is a bit on the "weird" side.

[rkeithhill]

@SydneyhSmith could you move this issue to the ScriptAnalyzer repo?

[ili101]

FYI Same problem reported with UNC paths:
https://github.com/PowerShell/vscode-powershell/issues/1830#issuecomment-479702167

ili101 unassigned rkeithhill

How did I do that? :laughing:

[Jeff-Lewis]

It's a bummer to have to turn off and not use the PS Formatter b/c of this. The truncation of paths occurs with any formatting operation including formatting selected text.

[ili101]

I just downgraded to VSCode PowerShell 1.11.0
The state of this software "Stable" releases is ridiculous.
Every developer group I know who would make a release and get a bug of this magnitude discovered would rollback the update or release a patch version immediately. but here they just let more users data get damaged every day without taking action.
If this was a single developer volunteering it's time I can understand that. but this is a team with the backing of Microsoft.

[msftrncs]

I don't see it mentioned anywhere, but this is caused by the setting for 'Use Correct Casing for Cmdlets'. While correcting the casing, it changes the extension, removes the path, etc...

[MarkKharitonov]

So, what is the status of this bug? Is there any expectation for the fix?

[ili101]

Is there someone from the development team monitoring the Issues here or is this module is kind of Abandonware?

[bergmeister]

I have not had the time to repro yet but I think PR #1216 and/or PR #1210 that are already merged into the development branch. It would be great if you could confirm that it has been fixed @ili101 You can take the binaries from this appveyor build here (just replace the module that is used with the binaries from the build, choose either the Windows PowerShell or PowerShell Core build depending on what you use)

[ili101]

@bergmeister I updated to the last VSCode PowerShell v2019.5.0
Tested with default setting - No formatting at all. (As it was turned off until this is fixed)
Added "powershell.codeFormatting.useCorrectCasing": true to settings and tested again - formatting worked and paths get corrupted.
Replaced the module in C:\Users\illym\.vscode\extensions\ms-vscode.powershell-2019.5.0\modules\PSScriptAnalyzer with AppVeyor version - problem not fixed paths still corrupted.

[bergmeister]

Sorry, I cannot repro this, can you please add more details about which formatting settings you use @ili101 or @rkeithhill ?Does the current working directory play a role, etc?

[rkeithhill]

I'm still able to repro:

C:\Users\Keith
06-03 19:54:43 21> $def = Get-Content .\PSSA-Formatting.ps1 -Raw
C:\Users\Keith
06-03 19:55:18 22> $def
. .\Desktop\Temp\Test.ps1
. '.\Desktop\Temp\Test.ps1'
. .\Desktop\Temp\Test.ps1

get-process -name notepad

C:\Users\Keith
06-03 19:55:23 23> Invoke-Formatter -ScriptDefinition $def -Settings .\PSSA-Formatting-Settings.psd1
. Test.ps1
. Test.ps1
. Test.ps1

Get-Process -name notepad

C:\Users\Keith
06-03 19:55:35 24> gmo

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.1.0      DirColors                           {ConvertFrom-LSColors, ConvertTo-LSColors, Format-Colorize… Manifest   6.1.0.0    Microsoft.PowerShell.Management     {Add-Content, Clear-Content, Clear-Item, Clear-ItemPropert… Manifest   6.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object…}
Script     1.0.0.0    posh-git                            {Add-PoshGitToProfile, Expand-GitCommand, Format-GitBranch… Script     2.0.0      PSReadLine                          {Get-PSReadLineKeyHandler, Get-PSReadLineOption, Remove-PS…
Script     1.18.0     PSScriptAnalyzer                    {Get-ScriptAnalyzerRule, Invoke-Formatter, Invoke-ScriptAn…

Here is my setting file:

PSSA-Formatting-Settings.zip

[msftrncs]

I cannot seem to reproduce this. but I do get the annoying change of cmd to cmd.exe.

I am pretty sure this is coming from the setting 'usecorrectcasing', but this is changing more than casing.

[ili101]

@bergmeister @msftrncs Regarding the reproduce problem, I did some more testing:

1. It looks like that there is some caching involved, so for example if the file is not in the path and you test it, then create the file then test again it will not change it. you will need to start a new PS session to "clear the cache".
2. In VSCode the relative path need to be valid in relative to the path the console was opened in, So if VSCode has a folder open it need to be relative to it and if not it need to be relative to C:\Users\MyUser. changing the folder after the console was opened has no effect.
   It is easier to reproduce this with UNC paths like \\server\Temp\T1.ps1 as it doesn't matter what folder you are in.
   I was able to reproduce it outside of VSCode with a UNC path but not with relative path, maybe VSCode send the base path to PSScriptAnalyzer Settings?

[bergmeister]

OK, I can repro now but only if I open PowerShell using the explorer context menu (which uses the -RemoveWorkingDirectoryTrailingCharacter -WorkingDirectory on pwsh.exe), I cannot repro in Windows PowerShell though. I can therefore confirm that this specific issue is still not fixed in the development branch. But that's enough as a start for me to look into it. I suspect it is a wrong CommandInfo cache entry, PSSA performance improvements in 1.18.0 were mainly due to the usage of caching therefore now we have since found some cases where entries are wrong, which we have to fix now. PSSA basically issues get-command calls to Powershell to get info about commands and sometimes it is even PowerShell's behaviour that gets into the way...
UPDATE: By looking at the debugger, it seems in the Get-Command call we should exclude ComandInfo types of type ExternalScript or apply the correction only if the correction is only differing in casing because PowerShell returns the fully qualified path. Expect a PR soon (I hope before the 1.18.1 release) .You can see this locally by doing Get-Command $PathToLocalFile

Just to repeat again: We have disabled the newly added vs code setting powershell.codeFormatting.useCorrectCasing to prevent this bug from affecting more users and if you find it annoying then until PSSA 1.18.1 is released (planned for next week), then I suggest to not use this new feature.
@msftrncs Your mentioned issue is already fixed in #1210