Vscode-powershell: Make newly created PowerShell files default to UTF-8 *with BOM* to avoid encoding misinterpretation

[mklement0]

Summary of the new feature

• VSCode creates UTF-8 files _without BOM_ by default.
• This causes _Windows PowerShell_ (but not PowerShell _Core_) to _misinterpret any non-ASCII-range characters_, because, in the absence of a BOM, it defaults to the system's legacy "ANSI" code page (e.g., Windows-1252).

It seems that this has been a perennial pain point (whose root cause isn't obvious), as evidenced by, for instance, by #629 or this StackOverflow question.

Making the extension default all new PowerShell files to UTF-8 _with BOM_ solves that problem.

Such files would be both cross-edition and cross-platform compatible (given that PowerShell _Core_ still correctly _interprets_ the BOM, even though it doesn't _require_ it).

Proposed technical implementation details

I don't know how it works in the context of _extension-specific_ settings, but in the general settings.json file you can simply add the following:

"[powershell]": {
  "files.encoding": "utf8bom"
}

[rjmholt]

The current big issue here is that the extension is third-party software in the eyes of both VSCode and PowerShell.

There's no exposed API for it to configure this VSCode setting on installation.

See https://github.com/Microsoft/vscode/issues/824.

Anyone seeing issues here, please lend your 👍 to https://github.com/Microsoft/vscode/issues/824 and take a look at https://github.com/MicrosoftDocs/PowerShell-Docs/pull/3743 (will update to the doc link when it's merged).

@mklement0 I assume that document prompted this issue?

[mklement0]

Thanks, @rjmholt - it's unfortunate that there's still no API for this (I've since given the linked issue a thumbs-up).

I assume that document prompted this issue?

No, I wasn't aware of that document (thanks for the link). It was my own experience and seeing people run into the problem on SO (Stack Overflow) that prompted me to create this issue.

I've now (hopefully) given it wider exposure with this SO answer.

[rjmholt]

I've now (hopefully) given it wider exposure with this SO answer.

Ah! I'll link to it in the new doc

[rkeithhill]

With the new doc on handling encoding WRT PowerShell and text editors, can we close this?

[mklement0]

I suggest keeping this open with an Resolution-External label (or equivalent) to await being able to implement a proper solution via the future API proposed in https://github.com/Microsoft/vscode/issues/824.

[rjmholt]

Doc: https://docs.microsoft.com/en-us/powershell/scripting/components/vscode/understanding-file-encoding?view=powershell-6