Terminal: Automatically triggered Replace mode using vim in OpenSSH on cmd / powershell though Windows Terminal

[SuzunaeHoumi]

Replace input mode will triggered automatically when using vim though OpenSSH Client in cmd.

Environment

Windows build number: [Version 10.0.18362.175]
Windows Terminal version (if applicable): 0.2.1715.0

Remote Linux PC: Ubuntu LTS 1604

OpenSSH Client: Stock version in Windows 10 1903 Build 18362.175

Steps to reproduce

1. Open Windows Terminal
2. Open new tab running cmd / powershell / powershell 6
3. ssh to a Ubuntu 1604 LTS PC
4. Open any document using vim

Expected behavior

Not trigger any input mode, the status bar at the bottom displays the path of the file that vim opened.

Actual behavior

Triggered Replace input mode.

[DHowett-MSFT]

None of us can reproduce this. Is there anything else unusual about your machine, about your vim configuration, or about your keyboard layout?

[DHowett-MSFT]

Also, does it still happen if you use _wsl ssh?_

[SuzunaeHoumi]

Also, does it still happen if you use _wsl ssh?_

No, it would not happened when I use wsl ssh

[SuzunaeHoumi]

None of us can reproduce this. Is there anything else unusual about your machine, about your vim configuration, or about your keyboard layout?

Keyboard Layout is simple Standard US Layout

[jgeorgeson]

[Un]Fortunately I don't think this is a Windows Terminal issue. I see the same behavior using ConEmu on my work machine (not updated to W10 1903 so can't install Windows Terminal there), but not in either Windows Terminal or ConEmu on my home machine (W10 1903). Both machines have Standard US Layout and both English US and English US International input languages. I see no change in this behavior on either machine based on selected input langauge.

[DHowett-MSFT]

If it doesn't happen in wsl ssh, but it does in win32-openssh, I'd go to their repository and complain there. Thanks!

[linuxcyberlabs]

Seems to be an issue with utf-8 ambiguous characters and Windows cmd console. Flag t_u7 is set by default and so vim will request cursor position and get a bad reply from the ssh client.

Workaround: Adding set t_u7= or set ambw=double to your vimrc should fix the problem. set t_u7= will disable requesting cursor position and ambw=double will set the ambiguous characters mode to double.

For more info see vim reference manual: https://vimhelp.org/term.txt.html

Credits: https://superuser.com/a/1525060

set ambw=double appears to have conflict with airline/powerline characters. I see extra spaces after the <| arrows. set t_u7= works though! – David Woods Jul 9 at 17:12

[rsynnest]

Windows build number: 10.0.19041.0
Windows Terminal version: 1.2.2022.0 (Preview Build)
WSL2 distribution: Ubuntu 18.04.4 LTS and Debian 10 (both affected)

This bug happens in WSL for local sessions as well as remote SSH sessions when running the Terminal Preview build v1.2.2022.0. The latest non-preview release of Terminal v1.1.2021.0 does not inflict this bug on WSL sessions. A note on reproducibility - this happens at random, around 1 in 5 times I open vim it opens in Replace mode.

This comment fixed the issue for me, but is needed in my local .vimrc as well as any remote servers I'm working on. Another fix is to install and use Neovim which is unaffected by this, presumably because it uses utf-8 internally for everything.

[vladsol]

Windows 10.

Windows Terminal Preview
Version: 1.2.2022.0

Remote SSH (using windows ssh) -> vim -> starting in Replace mode :(

[DHowett]

Yes, this is a well-known issue with Win32-OpenSSH 7.7.

[rsynnest]

@DHowett I know this is closed as a Win32-OpenSSH bug but I think it was closed prematurely. I'm seeing this in all local non-SSH WSL sessions, as well as all WSL SSH sessions. See my previous comment for details.

[rar222]

Yes, I am also seeing it for local WSL sessions. This is not only a SSH bug and should not be closed.

[sec]

Also having this bug with Windows Terminal v1.2.2381.0 and Preview v1.3.2382.0
It's fine when using 1.1.2233.0

[haroldzhai]

Also having this bug with Windows Terminal since I updated to Version: 1.2.2381.0 on win10

[DHowett]

Thanks for the reports, all.

[PhyX-Meow]

https://github.com/vim/vim/issues/6365

[k-takata]

As reported at https://github.com/vim/vim/issues/6365#issuecomment-652583382, conhost seems to return a broken response to two running CPR (cursor position report) requests.

executed command

$ echo -ne "\e[2J\e[2;2H\e[6n\e[3;3H"; echo -ne "\e[6n"; hd

expected result

  ^[[2;2R^[[3;3R
00000000  1b 5b 32 3b 32 52 1b 5b  33 3b 33 52 0a           |.[2;2R.[3;3R.|
0000000d

actual result

  ^[^[[3;3R[2;2R
00000000  1b 1b 5b 33 3b 33 52 5b  32 3b 32 52 0a           |..[3;3R[2;2R.|
0000000d

[DHowett]

Thanks. We now have the grim task of determining exactly how this escaped the global console lock that gates every action in conhost :)

[volksvagen]

This does not happen with THE conventional Windows CMD terminal, nor the built-in Ubuntu CMD Terminal, nor Cmder, nor ConEmu, it only happens with Windows Terminal.

[Tekki]

This does not happen with THE conventional Windows CMD terminal, nor the built-in Ubuntu CMD Terminal, nor Cmder, nor ConEmu, it only happens with Windows Terminal.

I have this problem when I go to wsl from everywhere. Does the installation of Windows Terminal replace conhost.exe globally?

[DHowett]

replace conhost

No, but this bug has also existed in conhost for at least the past two Windows releases.

[DHowett]

@FremdSprach the reports in the linked thread on the Vim repository indicate that it _does_ happen in the traditional console host, so we have to treat this bug like it does happen in the traditional console host.

[volksvagen]

@DHowett So it is quite a strange one.

Here is my environment, and I tested it many times with different terminals, only the WT fails, weird thing:

C:>ver
Microsoft Windows [Version 10.0.19041.264]

C:>ubuntu
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"

[Tekki]

C:>ver
Microsoft Windows [Version 10.0.19041.264]

@FremdSprach Maybe not so weird. I'm on 10.0.20206.1000 (Insider Preview).

[volksvagen]

Today I came to notice that the issue was opened on Jun 26, 2019? Well, that's quite a long journey.

What is the biggest obstacle now? Trivial modification and low priority? Or even not debugged?

[DHowett]

@FremdSprach https://github.com/microsoft/terminal/issues/1637#issuecomment-687705482

This bug was only reopened a few days ago, after it was initially closed thanks to (what appeared to be) Win32-OpenSSH having a similar issue.

Since then, it’s been a weekend followed by a US holiday, which has occupied most of our US-based team’s time :smile:

(Edit: whoa, it has been a week since I reopened this.)

[DHowett]

Now, that might seem like a weird diagnosis. Until recently, win32-openssh implemented its own terminal emulator! It also implemented its own VT input generator.

[volksvagen]

Ah, Labor Day, that makes it. It must be a sound and good break without labor~~

Suppose God the savior himself take a same holiday, people across the world would get nobody to pray to. You guys are Savior to the WT-addicted die-hard fans. Hope God's got no holidays and wish you be blessed Thor's hammer :hammer: and hammer it hard :grin:

[SuzunaeHoumi]

Just surprised this issue has reopened after a year ago.
Since was told it was an issue in Win32-OpenSSH, then just ignored this issue when it happens.
Hum.

[volksvagen]

Historical bug awakening the first dragon-rider @SuzunaeHoumi 😀

[DHowett]

Okay, here's what's happening.

When we receive a DSR(CPR), the console host attempts to prepend the CPR to the input buffer here:

https://github.com/microsoft/terminal/blob/230b86c9905e0f91f07741664b0814cf43c3a483/src/terminal/adapter/adaptDispatch.cpp#L818-L819

https://github.com/microsoft/terminal/blob/230b86c9905e0f91f07741664b0814cf43c3a483/src/terminal/adapter/adaptDispatch.cpp#L857-L859

Because of the design of the input queue and how it has to remain compatible with ReadConsoleInput, the CPR is transformed into a series of KEY_EVENT_RECORDs -- one down and one up each for the entire sequence. Just like a user had pressed each of those keys.

Now, this poses a problem for two CPRs being received back-to-back: the _second CPR_ will cause the later position to be prepended to the input queue before the earlier position.

If an application has read input between the two CPRs -- even _a single keypress_, it will certainly encounter this issue.

In my testing, I saw that the input queue at the time of the second prepend contained...

ESC up
[ DOWN
[ UP
2 DOWN
2 UP
...

indicating that only the first keypress (esc down) had been read and popped out of the queue.

The resultant sequence was...

^[ ^[  [  [  3  3  ;  ;  3  3  R  R ^[  [  [  2  2  ;  ;  2  2  R  R
DN UP DN UP DN UP DN UP DN UP DN UP UP DN UP DN UP DN UP DN UP DN UP
                                    ^^ frame-shifted start of prior event

Because VT input events are _usually_ read on the keydown event, the first ESC has already been totally spooled into the application.

I'll need to wait for my team to be back from vacation/leave to figure out why we went with a _prepend_, as that decision predates me. :smile:

[DHowett]

Reader's notes:

Prepend seems like it was an intentional choice -- we do it in a number of places (every DSR, DA, DA2, DA3, VT52 Identify, DSR-OS, CPR) -- but I do not believe that it is ever correct.

It was introduced on 2017-10-17 in a pull request (!997738) that introduced support for _window manipulation sequences over the pty_, which we've never documented and quickly replaced with the signal channel. _(This will be of particular interest to the folks having the purity discussion about sending an ED down the input handle :grin:)._

commit [REDACTED]
Author: @zadjii-msft
Date:   Thu Oct 12 13:03:30 2017 -0700

    We need seperate Append and Perpend WriteInputs

      The InteractDispatch wants to append to the buffer,
      but the adapter needs to prepend to the buffer.

      Tests still don't build.

This comment suggests that we had to _add a separate prepend_ so that InteractDispatch could append. Because this pull request mainly dealt with _InteractDispatch_, the implication is that the "adapter" _was already prepending_ and we needed to add Prepend...Buffer to disentangle the prepending behavior that "adapter" was using.

However, the code change is fairly obvious in the "adapter". It explicitly goes from appending to prepending with this diff:

diff --git a/src/terminal/adapter/adaptDispatch.cpp b/src/terminal/adapter/adaptDispatch.cpp
index xxxxxxxxx..yyyyyyyyy 100644
--- a/src/terminal/adapter/adaptDispatch.cpp
+++ b/src/terminal/adapter/adaptDispatch.cpp
@@ -932,7 +932,7 @@ bool AdaptDispatch::_WriteResponse(_In_reads_(cReply) PCWSTR pwszReply, _In_ siz
     if (cInputBuffer > 0)
     {
         DWORD dwWritten = 0;
-        fSuccess = !!_pConApi->WriteConsoleInputW(rgInput, (DWORD)cInputBuffer, &dwWritten);
+        fSuccess = !!_pConApi->PrivatePrependConsoleInput(rgInput, (DWORD)cInputBuffer, &dwWritten);
     }

Mike's on paternity leave (yay!), so I can't exactly ask :smile:

[msftbot[bot]]

:tada:This issue was addressed in #7583, which has now been successfully released as Windows Terminal v1.3.2651.0.:tada:

Handy links:

• Release Notes
• Store Download

[msftbot[bot]]

:tada:This issue was addressed in #7583, which has now been successfully released as Windows Terminal Preview v1.4.2652.0.:tada:

Handy links:

• Release Notes
• Store Download

[DHowett]

This fix has also been ingested into Windows for a future release.

[electrofloat]

🎉This issue was addressed in #7583, which has now been successfully released as Windows Terminal v1.3.2651.0.🎉

Handy links:

* [Release Notes](https://github.com/microsoft/terminal/releases/tag/v1.3.2651.0)

* [Store Download](https://www.microsoft.com/store/apps/9n8g5rfz9xk3?cid=storebadge&ocid=badge)

The "Handy links" here takes you to the Preview app, not to the non-preview app.

[DHowett]

http://aka.ms/terminal enjoy

[DHowett]

(sorry, I flippantly gave you the preview link anyway ;P)

[volksvagen]

@DHowett Great work! Finally, and Wonderfully! Thanks so much!
Clouds fade away upon the sky and sunshine shines on earth again!

[plutonji]

Thanks!

[WSLUser]

This still sometimes happens where VIM reports being in replace mode over SSH. Othertimes it correctly goes into Insert Mode in v1.4

[DHowett]

@wsluser please file a separate bug following the bug template and note the versions of all software involved.