Psreadline: Escape character garbage on command line (Win81)

Created on 8 May 2020  ·  16Comments  ·  Source: PowerShell/PSReadLine

I've been using PWSH 6.1.1 since introduced and that shell has been working great, mostly.
Decided to give the 7.0.0-rc.2 a run...

  • Good News: start up is 25% faster
  • Bad news: Lots of escape character garbage on command line, appearing as soon as you type anything. (In particular the [104m.)

Steps to reproduce

# type a "$" (the first one is part of the prompt)
$ $⬜[104m

(I had to manually paste a box character above, since it didn't show in the github markup.)

----------------------------------------------------------
  PowerShell Version    : 7.0.0-rc.2
  OS Name               : Microsoft Windows 8.1 (64-bit)
  OS Version            : 6.3.9600  [2014-08-16 00:50:01]
  OS BuildLabEx         : 9600.19538
  OS HAL                : 6.3.9600.18969
  OS Kernel             : 6.3.9600.18217
  OS UBR                : 19599
  -------------------------------------------------------
  on Host               : xxxx
  as User               : xxxx
  with Privilege        : Administrator
  -------------------------------------------------------
  ExecutionPolicy :
        MachinePolicy   : Undefined
        UserPolicy      : Undefined
        Process         : Undefined
        CurrentUser     : Bypass
        LocalMachine    : RemoteSigned

  Console Settings:
      Type              : ConsoleHost
      OutputEncoding    : Unicode (UTF-8)
      Color Capability  : 151
      Registry VT Level : 1
      CodePage (input)  : 437
      CodePage (output) : 437
      Mode (input)      : 0x1f7
      Mode (output)     :

Name                           Value
----                           -----
PSVersion                      7.0.0-rc.2
PSEdition                      Core
GitCommitId                    7.0.0-rc.2
OS                             Microsoft Windows 6.3.9600
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Related issue with further descriptions:

Needs-Attention
Source
picture E3V3A

All 16 comments

@E3V3A Can you install the GA version of the PowerShell 7.0.0 and see if you get the same problem? If the issue repros,

  1. please run the following script to get the environment data and paste it back in here;
  2. please paste the result of Get-PSReadLineOption here.
& {
    $hostName = $Host.Name
    if ($hostName -eq "ConsoleHost" -and (Get-Command Get-CimInstance -ErrorAction SilentlyContinue)) {
        $id = $PID
        $inWindowsTerminal = $false
        while ($true) {
            $p = Get-CimInstance -ClassName Win32_Process -Filter "ProcessId Like $id"
            if (!$p -or !$p.Name) { break }
            if ($p.Name -eq "WindowsTerminal.exe") { $inWindowsTerminal = $true; break }
            $id = $p.ParentProcessId
        }
        if ($inWindowsTerminal) { $hostName += " (Windows Terminal)" }
    }

    "`nPS version: $($PSVersionTable.PSVersion)"
    $m = Get-Module PSReadline
    $v = $m.Version; $pre = $m.PrivateData.PSData.Prerelease
    if ($pre) { $v = "$v-$pre" }
    "PSReadline version: $v"
    if ($IsLinux -or $IsMacOS) {
        "os: $(uname -a)"
    } else {
        "os: $((dir $env:SystemRoot\System32\cmd.exe).VersionInfo.FileVersion)"
    }
    "PS file version: $($name = if ($PSVersionTable.PSEdition -eq "Core") { "pwsh.dll" } else { "powershell.exe" }; (dir $pshome\$name).VersionInfo.FileVersion)"
    "HostName: $hostName"
    "BufferWidth: $([console]::BufferWidth)"
    "BufferHeight: $([console]::BufferHeight)`n"
}
daxian-dbw picture daxian-dbw on 10 May 2020

Hi @daxian-dbw , I'm no longer using 7.0.0, but problem persists for 7.1.0...

Script:

PS version: 7.1.0-preview.2
PSReadline version: 2.0.1
os: 6.3.9600.16384 (winblue_rtm.130821-1623)
PS file version: 7.1.0.0
HostName: ConsoleHost
BufferWidth: 160
BufferHeight: 9999

and 

EditMode                               : Windows
AddToHistoryHandler                    : System.Func`2[System.String,System.Object]
HistoryNoDuplicates                    : True
HistorySavePath                        : C:\Users\xxxx\AppData\Roaming\Microsoft\Windows\PowerShell\PSReadLine\ConsoleHost_history.txt
HistorySaveStyle                       : SaveIncrementally
HistorySearchCaseSensitive             : False
HistorySearchCursorMovesToEnd          : False
MaximumHistoryCount                    : 4096
ContinuationPrompt                     : >>
ExtraPromptLineCount                   : 0
PromptText                             :
BellStyle                              : Audible
DingDuration                           : 50
DingTone                               : 1221
CommandsToValidateScriptBlockArguments : {ForEach-Object, %, Invoke-Command, icm…}
CommandValidationHandler               :
CompletionQueryItems                   : 100
MaximumKillRingCount                   : 10
ShowToolTips                           : True
ViModeIndicator                        : None
WordDelimiters                         : ;:,.[]{}()/\|^&*-=+'"–—―
AnsiEscapeTimeout                      : 100
CommandColor                           : ←[93m"`e[93m"←[0m
CommentColor                           : ←[32m"`e[32m"←[0m
ContinuationPromptColor                : ←[37m"`e[37m"←[0m
DefaultTokenColor                      : ←[37m"`e[37m"←[0m
EmphasisColor                          : ←[96m"`e[96m"←[0m
ErrorColor                             : ←[91m"`e[91m"←[0m
KeywordColor                           : ←[92m"`e[92m"←[0m
MemberColor                            : ←[97m"`e[97m"←[0m
NumberColor                            : ←[97m"`e[97m"←[0m
OperatorColor                          : ←[90m"`e[90m"←[0m
ParameterColor                         : ←[90m"`e[90m"←[0m
SelectionColor                         : ←[94;47m"`e[94;47m"←[0m
StringColor                            : ←[36m"`e[36m"←[0m
TypeColor                              : ←[37m"`e[37m"←[0m
VariableColor                          : ←[92m"`e[92m"←[0m
E3V3A picture E3V3A on 11 May 2020

@E3V3A Were you using the 2.1.0-beta1 version of PSReadLine even before switching to 7.1.0 preview of PowerShell? If so, it’s a known issue that the new predictive suggestion feature, which was introduced in 2.1.0-beta1, incorrectly spits out VT sequences on downlevel Windows OS. The fix is to disable the new feature when VT is not supported, which has been merged to master.

Can you please confirm if you see the same issue with the 2.0.1 version of PSReadLine?

daxian-dbw picture daxian-dbw on 11 May 2020

@daxian-dbw
I'm a bit confused to your comment. I have PSReadLine 2.0.1, so what do you want me to do?
How do I install a different or later version of PSReadLine in pwsh7?

E3V3A picture E3V3A on 15 May 2020

Oops, I misread your comment. I mistakenly believed PowerShell 7.1 preview would ship the 2.1 beta versions of PSReadLine, and thus the question.
Never mind my previous ask then. I will need to get a Win 8.1 to repro this.

daxian-dbw picture daxian-dbw on 15 May 2020
👍1

@daxian-dbw
Can you tell me how to install and run a later PSReadLine ?
Maybe I can help test something...

E3V3A picture E3V3A on 15 May 2020

Can you tell me how to install and run a later PSReadLine ?

Sure, you will find the "Upgrading" section helpful: https://github.com/PowerShell/PSReadLine#upgrading
You probably should avoid the 2.1.0-beta1 version of it, because it contains an experimental feature "Predictive Suggestion" that doesn't work well on downlevel Windows like Win 8.1. But knowing the upgrading instructions will help you get to latest PSReadLine in future.

daxian-dbw picture daxian-dbw on 15 May 2020
👍1

@daxian-dbw
Yes, now I tried latest PSReadLine, which seem faster and better (in general) but suffer same issue.
So, I think I've reached the limits what I can do to investigate.
Let me know if there's any other way I can help.

E3V3A picture E3V3A on 16 May 2020

:warning: WARNING! :exclamation:

After having installed PSReadLine 2.1.0-beta1, I was no longer able to remove it. I tried all possible Powershell commands but they all failed. The only solution was to force remove the entire installation directory found in:
C:\Users\xxxx\Documents\PowerShell\Modules\PSReadLine\2.1.0

The only way to find it was:

Get-Module -ListAvailable -Name PSReadLine

For some reason, the installation ended up in the Users directory instead of being under Program Files which probably was causing the errors. (Admin files written to local user directories.)

E3V3A picture E3V3A on 17 May 2020

I think you should be able to remove it by:

  1. Close all pwsh instances
  2. pwsh -nonprofile -noninteractive from CMD.exe
  3. Remove-Module -FullyQualifiedName @{ ModuleName = "PSReadLine"; ModuleVersion = "2.1" }

PSReadLine is loaded at powershell startup. Once the assembly is loaded, the assembly file on disk cannot be removed. Maybe that's why you couldn't uninstall it.
-noprofile -noninteractive basically tells powershell to start without loading PSReadLine.

daxian-dbw picture daxian-dbw on 18 May 2020
👍1

As I mentioned in the linked issue.

I think the issue is that that particular "blue" is a 24-bit color code, and on non Win10 shells, we don't have that. So that either PSReadLine should probably be fixed to accommodate by not attempting to output 24-bit colors, when not available or that powershell/pwsh is made to filter out those 24-bit escape codes, when found in stream.

E3V3A picture E3V3A on 24 May 2020

@E3V3A Were you using the 2.1.0-beta1 version of PSReadLine even before switching to 7.1.0 preview of PowerShell? If so, it’s a known issue that the new predictive suggestion feature, which was introduced in 2.1.0-beta1, incorrectly spits out VT sequences on downlevel Windows OS. The fix is to disable the new feature when VT is not supported, which has been merged to master.

Ha, I wondered why I had to downgrade from 2.1.0-beta2 -> 2.1.0-beta1 to get the prediction history to work (Win 8.1, Win10 machines). I have downgraded the VSCode PowerShell extension to 2020.04 too (as 2020.06 has a hard stop on prediction history)

PS version: 7.1.0-preview.5
PSReadline version: 2.1.0-beta1
os: 6.3.9600.16384 (winblue_rtm.130821-1623)
PS file version: 7.1.0.0
HostName: Visual Studio Code Host
BufferWidth: 114
BufferHeight: 3000

RooseBaker picture RooseBaker on 9 Aug 2020

Just tried to update to latest PSReadline but I still get garbage on command-line.

pwsh_2020-09-17_13-43-23

Any updates how to fix this?

PS. Again, I would like you to look into the issue I already mentioned above, about the 24-bit color code.

E3V3A picture E3V3A on 17 Sep 2020

As mentioned here:

Both \e[104m and \e[0;104m should select bright blue as the background color. The difference is that \e[0;104m also resets the foreground color, font weight, underlining, etc. to defaults. But the console in Windows 7 supports neither of these.

So for W7/8 that code should probably be left out altogether, or filtered out for non VT-compatible terminals.

From here:

firefox_2020-09-17_14-00-11

E3V3A picture E3V3A on 17 Sep 2020

Bump? :question:

This [104m looks like an escape sequence used for terminal instructions like changing colour of text, e.g. "dotnet" is yellow in your screenshot, so there will be a "change colour to yellow" sequence before it, and a "change colour back to white" after it that get handled by the terminal and not shown as text. This [104m is not being handled correctly and is breaking through where it shouldn't - maybe it's not written quite right.

\e[0;104m is close to it, and would be setting an intense blue background, which you have as the background colour in your screenshot. So maybe that's what it is supposed to say, and has a missing 0; in it, so it's not handled as a colour, and is being printed as text.
...
... the space charter is displayed as what I believe is the Unicode code. If it was a Unicode space, it would look the same as any other space; Unicode was built so that all the basic English letters, numbers and symbols from American (ASCII) computers would be the same.

and:

Both \e[104m and \e[0;104m should select bright blue as the background color. The difference is that \e[0;104m also resets the foreground color, font weight, underlining, etc. to defaults. But the console in Windows 7 supports neither of these.

Both from discussion in:

eabase picture eabase on 16 Oct 2020

I suspect the problem is here somewhere...

But may also be related to #1110

eabase picture eabase on 18 Oct 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

craiglandis picture craiglandis  ·  6Comments
DavidZidar picture DavidZidar  ·  6Comments
trha picture trha  ·  6Comments
bagajjal picture bagajjal  ·  3Comments
EArminjon picture EArminjon  ·  5Comments