Psreadline: Windows PSReadline stores and restores multiple bits of console state before/after commands

[DHowett-MSFT]

Environment data

PSReadline version: 2.0.0-beta3
Windows

This applies to all extant versions of Powershell.

Steps to reproduce or exception report

There are a number of user preferences that are PSReadline stores and restores across command executions; it is arguable as to whether it is PSReadline's job to do so.

• [ ] the codepage
  
  
  
  • this is the cause of font flickering across commands in the windows console (still!)
  
  
  • mitigation: none; i'm curious as to why it changes the codepage to begin with?
  
  
• [ ] the index of the currently-set foreground and background colors
  
  
  
  • when using a modern terminal (or the terminal options in Windows 19H1), this causes colored lines to streak across the window because the "default colors" are not representable through color indices.
  
  
  • mitigation: CSI 0 m will reset the color to whatever the user requested as defaults
  
  
• ~[ ] the height of the legacy cursor (microsoft/terminal#1145)~
  
  
  
  • ~mitigation: use the standard VT sequences for cursor shaping~
  
  
• [ ] the cursor position, in some limited cases.
  
  
  
  • it would be better to (and this is probably a breaking change) conform to the *NIX model where you're not easily allowed to know where the cursor is
  
  

Most of these things happen here.

https://github.com/PowerShell/PSReadLine/blob/f2f553156063a8635b9d4882a878a828365cdd99/PSReadLine/ReadLine.cs#L436-L452

[lzybkr]

codepage

Setting the codepage was the easy way to fix rendering issues with CJK characters. It seemed to work well.

If there is a more appropriate fix, details would be appreciated.

background colors

Try beta4, I think this was fixed.

height of cursor

Should be an easy fix here.

cursor position

You'll need to provide a repro.

[daxian-dbw]

I investigated the font flickering issue on CJK locale machines. Here is what I found:

1. CJK locale machines doesn't use the Raster font by default. So the fix in #771 doesn't help.

   • For Chinese-simplified OS, the code page is 936 (Chinese Simplified (GB2312)). The corresponding default font is 新宋体 (Font Family: 54)
   • For Chinese-traditional OS, the code page is 950 (Chinese Traditional (Big5)). The corresponding default font is 細明體 (Font Family: 54)
   • For Japanese OS, the code page is 932 (Japanese (Shift-JIS)). The corresponding default font is ＭＳ ゴシック (Font Family: 54)
   • Note Font Family: 54 is not that special. Consolas and Lucida Sans Typewriter also have Font Family 54, and it could be the same for many other fonts too.

2. The font flickers because: when setting OutputEncoding to utf8, several more fonts will become available in the console host (Properties -> Font) comparing to when using the default OutputEncoding. For example, on the Chinese-simplified OS, Consolas, Courier New and Lucida Console become available after setting Console.OutputEncoding = Encoding.UTF8.

   • when the user chooses one of the newly available font, the font will be restored back to the default font when Console.OutputEncoding is restored to the initial default encoding, because the chosen font is unavailable when using the default encoding (default code page).
   • when the user chooses a font that's available for both the default and utf8 code pages, the font flickering won't happen because the font won't be restored.

I'm not sure how this should be fixed. For a workaround, we can check whether the initial encoding is one of the default CJK encoding's, and if so not change the output encoding.
However, it seems setting code page to utf8 was to fix rendering issues with CJK characters, and not changing the output encoding sounds contradict to that.
@lzybkr Could you please explain more about why PSReadLine changes the codepage to begin with?

[lzybkr]

@daxian-dbw - I'm not sure there's much more to say about why. I didn't do a lot of investigation - I was just guessing at a possible fix - but the change seemed to help according to some users.

[daxian-dbw]

@lzybkr Can you please point me to some of the issues that got fixed or mitigated by changing/restoring the code page? That would be helpful.

[lzybkr]

There may be other issues, but https://github.com/PowerShell/PSReadLine/issues/289 is one I think.

[daxian-dbw]

the height of the legacy cursor (microsoft/terminal#1145)

I want to update that microsoft/terminal#1145 is not a PSReadLine issue, but a Windows Terminal or .NET Core issue. The cursor shape is reverted when hiding the cursor and then showing it by setting Console.CursorVisible.
https://github.com/microsoft/terminal/issues/4124 is logged for that. Related issue in PSReadLine repo is https://github.com/PowerShell/PSReadLine/issues/903

[daxian-dbw]

For Windows users with CJK as default display language, there may be a workaround to the font flickering issue: unchecking the Unicode UTF-8 option in the language settings.