Terminal: Add support for or ignore legacy SI/SO encodings

[otmos]

Environment

Windows 10 1903 build 18362.418

Any other software?

Steps to reproduce

This happened suddenly after a recent windows 10 update , left my laptop installing some updates at night , to find this strange error in the morning when trying to ssh from a debian WSL instance

Expected behavior

Actual behavior

[otmos]

cascadia code

[zadjii-msft]

@otmos Could you copy and paste that character here, or tell us what character it is?

Does that repro if you ssh to the same server using the legacy console window (by launching Debian directly)?

Does htop display that character in the Terminal _without_ sshing?

Does the character appear in other commandline applications as well, or just htop?

[otmos]

On local , the character doesn't show at all , everything is normal both on terminal and legacy console:

but in SSH , This character appears everywhere , in nano and even in the shell itself .

tried to copy it from nano , pasting it on notepad gave this

but searching it on google chrome gave a question mark

[zadjii-msft]

@miniksa does this look like the replacement character thing you fixed?

Helpful to know that A: this isn't unique to the Windows Terminal and B: this only repros over ssh.

[miniksa]

No, not quite, @zadjii-msft.

@otmos... fill me in on the entire chain here. It could be any of the pieces involved.

I need you to fill in the blanks on this set of steps for me.

1. I'm starting from a Windows 10 1903 machine
2. I open a WSL ____(v1/v2)_____ instance in (___default conhost, I think___) that is using ____(Debian from the Windows Store, I think)____
3. Then I run ____(ssh.exe username@remotehost, with no other args, I think)____
4. After I log in to the ____(What version and flavor of Linux machine)____, I immediately run ____(htop, I think)_____.
5. Then I see the boxes.
6. I close the program. Then I run it again, redirecting it to a file by calling program > foo.txt.
7. It either runs to completion or I press Ctrl+C to end it after a moment or two.
8. I attached foo.txt here: ____(uploaded foo.txt)____

[otmos]

I'm starting from a Windows 10 1903 machine
I open a WSL V1 instance in default conhost or Windows Terminal that is using Debian from the Windows Store
Then I run ssh.exe username@remotehost
After I log in to the DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS" , I immediately run Htop or nano then i begin to see the boxes.

I close the program.
Then I run it again, redirecting it to a file by calling program > foo.txt.
It either runs to completion or I press Ctrl+C to end it after a moment or two.
I attached foo.txt here:
foo.txt
Same machine on ssh from a Cloud9 IDE :

[miniksa]

The boxes are literally 0x0f in the stream, per what's in foo.txt. @zadjii-msft, any ideas? Could it be some setting in the termcap?

EDIT. There's also 0x5e 0x4f pairs for a ^O in the stream a bunch too.

[zadjii-msft]

I haven't the darndest clue why we'd be getting "Shift In" (0xf, ^O) in the buffer ever. @j4james/ @egmontkob and ideas where those might come from?

[egmontkob]

I was staring at this the other day, and no, I have no idea. I don't recall ever seeing something like this.

foo.txt doesn't contain what you guys were asking for. It contains a 0x0f at the beginning and near its end, but the rest are literal caret ^ character followed by literal letter O or bracket [ or alike, no escape character in sight. It's sure not the raw output of htop. However, it might easily be the output of htop converted to these human-friendly ^-notations, e.g. by cat -v. And apparently whoever did this convertion also inserted actual ^O (0x0f aka. "shift in") characters??? Someone somehow broadcasting this character to all open file descriptors? Doesn't make too much sense.

Let's gzip it before uploading, that guarantees binary safe transfer (or, at least, it can be detected if the file gets corrupted).

Also, chances are that the app produces different output when redirected. Instead of redirection I recommend to launch script, then reproduce the bug, then quit script's shell (it tells you "Script done"), and then attach the (gzipped) typescript file.

---

The nano screenshot is IMO pretty misleading, as I have no idea what file is opened there, e.g. whether those rectangles are part of the file's contents or the actual bug we're hunting. Only the bottom rows prove that it's indeed buggy.

Could be a broken terminfo. Would be nice to see $TERM here vs. from that other terminal. Although the other terminal could just support ^O "shift in" and thus silently swallow these. I can't even exclude the possibility of a hardware issue, e.g. a transient memory failure resulting in broken terminfo residing in the memory. I'm not saying it's likely, but possible. Would be to see if the problem persists after the remote machine is rebooted, or is gone for good and we're left with an unresolved mystery.

[egmontkob]

In foo.txt, exactly the SGR sequences that reset all the attributes (and maybe enable bold), namely ^[[m, ^[[0m and ^[[0;1m are followed by ^O.

Sequences that tamper with the colors, such as ^[[30m, ^[[39;49m etc. are _not_ followed by ^O.

(There's no SGR sequence in the file that would contain both 0/1 and 3x/4x.)

My best bet at this moment is an ncurses/terminfo bug (or hardware bug triggering faulty ncurses behavior), combined with that other emulator supporting and thus no-op'ing on SI.

What's the output of tput sgr0 | cat -v?

Oh, wait a sec!!!

On my Ubuntu 19.10, the output of this is ^[(B^[[m (designates US charset to G0), which escape sequence is closely related to SI.

Could it be that emitting that SI was a _feature_ of some older ncurses/terminfo?

[egmontkob]

Excerpt from ncurses-6.1/misc/terminfo.src:

iTerm2.app [...] sgr0=\E[m\017

and similar for e.g. linux-vt, putty, rxvt and quite a few others.

Apparently emitting SI _is_ a feature of ncurses/terminfo.

At this point you can either blame the user for using an incorrect terminfo (which a perfectly valid blame as it might result in other problems too), or implement SI/SO :)

[zadjii-msft]

@egmontkob thanks for the great breakdown. I think there's probably two steps for us here:

1. Eat the SI/SO characters, to have users with this terminfo not totally broken.
2. Actually implement SI/SO.

I'd prefer break it down like this. Yea the terminfo is probably a bad terminfo and yea the user should change that, but also, we shouldn't just explode like this in that scenario. 1 is trivial enough to unblock this case, while (it sounds like) 2 is probably not that high of a priority feature that we can put off for a while.

[egmontkob]

I'd prefer break it down like this.

This would've been my conclusion too if it wasn't past midnight here :D

[otmos]

Here is a screenshot of nano trying to write into a new file if that will help

I tried to unset the $term variable too but nothing changed

[egmontkob]

And just to complete the picture:

Something that initially confused me was that the behavior seemed inconsistent. E.g. most of the time there's a rectangle when switching from grey to white, but no rectangle when switching from white to blue. However, in the very first screenshot, the row of PID 1334 is an exception to both of these "patterns".

The explanation to this is that probably we don't see the initial screen right after htop starts up, but after a couple of repaints, during which ncurses probably optimized by jumping over areas that it believed hadn't changed. And of course the reality was already off compared to ncurses's belief, so it could even matter if ncurses used relative or absolute cursor movement. Anyway, it's easy to imagine the aforementioned visual "pattern" breaking during such optimized partial repaints.

[egmontkob]

@otmos Thanks for the report and help, the mystery is solved now, no more screenshots are needed.

You might want to check your TERM (uppercase!) variable (echo $TERM) and share with us (I'm simply curious), and test if setting it to xterm-256color (export TERM=xterm-256color) fixes the issue for you (I think it should).

[otmos]

After setting TERM to xterm-256color , everything works as it should , and i understand how i came across this bug , it was a result of changing TERM to screen in order to fix a bug when scrolling in any app such as nano when in ssh mode from an AWS cloud9 instance

[egmontkob]

I think we'd love to hear _that_ bug as well :)

Generally, TERM=screen or friends should be used if and only if you're inside screen or tmux.

[otmos]

Here is a video showcasing the bug
data - Cloud9 - Google Chrome 2019-11-15 16-40-28.zip

[j4james]

Note that SI and SO are covered by issue #3377, and I have a work-in-progress PR for that which I'm hoping to get back to soon.

[andrp92]

Also if it helps, when $TERM = linux; and top is called, broken characters shows up too:

Thanks for your detailed and great analyst @egmontkob 👍 👌

[nicm]

The way it works is that enacs is \033(B\033)0 which configures G0 to be US-ASCII and G1 to be the ACS line drawing set. Then smacs and rmacs (and sgr0 as you have seen) use SI and SO to switch between G0 and G1 .

It is different in some entries (like xterm-256color) where enacs does nothing and smacs and rmacs just switch G0 between US-ASCII and ACS line drawing. This seems to be the new way of doing it, certainly for all the xterm entries that are still relevant.

[ismail]

The way it works is that enacs is \033(B\033)0 which configures G0 to be US-ASCII and G1 to be the ACS line drawing set. Then smacs and rmacs (and sgr0 as you have seen) use SI and SO to switch between G0 and G1 .

It is different in some entries (like xterm-256color) where enacs does nothing and smacs and rmacs just switch G0 between US-ASCII and ACS line drawing. This seems to be the new way of doing it, certainly for all the xterm entries that are still relevant.

Would it be possible to change tmux-256color to do something similar? I'm not privy to technical details, but looks like getting this bug fixed will probably take a long time.

[DHowett]

If I recall correctly, this is in pull request. We deferred looking at that PR (with the blessing of the author) until after we launched v1 of Terminal, but we’ll make sure it gets back on our list.

[nicm]

@ismail - unlikely. tmux-256color is intended for use inside tmux which does support this; you should make sure that your TERM matches the terminal you are running.

[ismail]

@ismail - unlikely. tmux-256color is intended for use inside tmux which does support this; you should make sure that your TERM matches the terminal you are running.

I use tmux-256color everywhere and it works fine, except one place ;) Thanks for your reply!

[msftbot[bot]]

:tada:This issue was addressed in #4496, which has now been successfully released as Windows Terminal Preview v1.1.1671.0.:tada:

Handy links:

• Release Notes
• Store Download