Xterm.js: Running clear in terminal removes viewport content from buffer instead of hiding it

[Tyriar]

Steps to reproduce:

1. Run ll twice in to fill the viewport
2. Run clear
3. Scroll up and notice that a number of rows equal to the height of the terminal were removed from the buffer.

[Tyriar]

gnome-terminal appears to be buggy in this regard, it removes that many rows from the very beginning of the buffer, not the old viewport.

Here is the spec for the signal that's received (2):

ED – Erase In Display

ESC [ Ps J  default value: 0
This sequence erases some or all of the characters in the display according to the parameter. Any complete line erased by this sequence will return that line to single width mode. Editor Function

Parameter   Parameter Meaning
0   Erase from the active position to the end of the screen, inclusive (default)
1   Erase from start of the screen to the active position, inclusive
2   Erase all of the display – all lines are erased, changed to single-width, and the cursor does not move.

http://www.vt100.net/docs/vt100-ug/chapter3.html

The right thing might be to clear absolutely everything like xterm does.

[parisk]

So should we move forward with closing this issue and leave the cleaning behavior as is?

[Tyriar]

@parisk well it looks like a bug to me, maybe just doing what xterm does would be better considering that is what this is trying to emulate?

[parisk]

Well, did some tests. I think that your proposal of "shifting rows up" instead of removing them from buffer provides the best UX, even if it is not 100% xterm compliant.

[CoenraadS]

I just ran into this with: https://github.com/Microsoft/vscode/issues/19934

I would like to paste my use-case here for consideration:

My use-case (also how powershell works):

I like to cls and then run npm test. Then the beginning of my terminal is where my test output starts.
Now I cls, but I have no idea where the beginning of my test output is.

Therefor I disagree with just shifting rows up.

[Tyriar]

Here's a roundup of behavior in various terminal emulators when they receive an Erase All ED signal (\e[2J):

• xterm.[email protected]: keep everything, just push it up
• gnome-terminal: clear everything on top of the current viewport, then push the current viewport up
• Fedora terminal: same as gnome-terminal
• xterm: Clear entire buffer
• Terminal.app: same as xterm.[email protected]
• iTerm2 v3: same as xterm.[email protected]

Based on some references I found, it seems ambiguous what exactly this escape sequence is meant to be since this one says "Erase All", and this one says "Erase Screen". The actual xterm behavior however would seem to indicate that it means clear everything.

To me xterm's behavior seems the most reasonable, particularly when we implement search (https://github.com/sourcelair/xterm.js/issues/553), but I can see why the current behavior may be good for some users. @parisk what do you think? Should this be a setting (clear everything vs push up) or should we move to xterm's behavior?

[parisk]

Based on the name of the signal, I think as well that xterm's implementation is the most reasonable.

But, I think also that xterm.js' implementation is the one that you will regret less, if it's done accidentally 😅 , so for me (personally) it's more desirable.

What is the use case that one person would prefer the entire buffer getting erased to hidden?

[Tyriar]

@CoenraadS mentions a big one:

1. clear
2. Run a command with long output
3. Go to start of buffer to see start of command

Another one is that when we do search, not being able to clear will cause issues for similar reasons as a bunch of old keywords that were explicitly cleared will get matched.

[dragonwolf83]

I think this should be a setting because Windows and Linux shells behave differently and I don't see that listed in the roundup.

Cmd and Powershell clear the buffer when cls is called so you can't scroll to see history.
CentOS using Bash, clear command wipes the screen but maintains buffer so you can scroll up to see history.

Here is what I think is needed to fully resolve this:

• Set a default based on the OS to maintain expectations based on platform.
• Allow user to override per shell. That way, Windows Bash users can change it to keep buffer.
• Allow creating shortcuts or UI buttons so that a user can do either one regardless of setting. Something like ClearBuffer vs ClearScreen would work.

[mofux]

This brings me to a really nice plugin idea: if you clear, the buffer gets cleared, but if you scroll up past the current fold, you will see a line "The previous output has been cleared, click here to view", which restores the output. It would behave similar to a code editor where you can expand / collapse sections, but our sections are grouped by clear commands. Would modifying the scrollback like that actually work as a plugin currently?

[Tyriar]

@mofux CircularList would probably need to be made aware of collapsible regions for something like that.

[chrisdothtml]

+1 for actually clearing the screen. I share @CoenraadS 's use case where I use it for unit tests

[freMea]

VS Code version | OS
------------------ | ----
1.10.2 | Win x64 pro (french)

I confirm that running cls from the integrated terminal doesn't really clear the terminal. You still can scroll up and see previous commands. This with powershell.exe or cmd.exe.

This Code command will produce same result

{ "key": "ctrl+k", "command": "workbench.action.terminal.clear", "when": "terminalFocus" }

Workaround, use the _clear terminal_ command from the palette or edit the keybinding removing the terminalFocus as

{ "key": "ctrl+k k", "command": "workbench.action.terminal.clear" }

Unfortunately you won't be able to run this shortkey successfully when terminal has focus.

This need to be fixed.

[01binary]

I would greatly appreciate a fix, I have to keep killing the terminal windows and re-opening to clear. It's very flow-breaking.

[KristofStroobants]

So is there any intention to fixing this any time soon? This is the 1 thing holding me back from dropping ISE and moving into VSCode. I've been waiting for a fix for about a year now :-/

[chrisdothtml]

@CookieCrumbles this has been fixed for me in VSCode Insider's build for months

[KristofStroobants]

@chrisdothtml

I had the issue in Insiders 1.8, that was the last one i had. I'll download the latest and give this a try.
Thanks for getting back to me. I'll post back as soon as i can.

[KristofStroobants]

@chrisdothtml
I'm sorry to say but if i do this with powershell:

gci c:\temp cls gci c:\temp cls gci c:\temp cls gci c:\temp cls gci c:\temp

I'm still able to scroll upwards and see previous GCI of c:temp.
So this isn't fixed.

[chrisdothtml]

Ah, I'm on a Mac. Probably wasn't fixed for Windows yet

[KristofStroobants]

I'm guessing this isn't high on their priority list. Kinda gave up on VSCode just because of this. Back to ISE it is, i'll check back in six months or so, hoping it got fixed by then. #FeelsBadMan

[Tyriar]

We support both clearing the viewport (2J) and the scrollback buffer (3J), not sure there is a fix for this for Linux/macOS other than including xterm.js in terminfo, see https://unix.stackexchange.com/questions/375743/why-clear-do-not-clear-whole-screen

As for Windows, I'm not sure if there's anything that can be done on our side either. @rprichard any ideas why winpty would send only \e[2J and not a \e[3J as well when a clear is done in PowerShell?

[jeremy-kothe]

@Tyriar fix please asap. This is expected behaviour 101 for over 20 years of windows terminals. Can it really be hard to have windows clear it's entire buffer from that signal?

[rprichard]

winpty definitely notices when the CLS command is run -- the console window moves back to the top of the console screen buffer. When that happens, winpty resets the terminal. Maybe it should send a different command than it currently sends.

My impression has been that clearing the screen doesn't reset the scrollback, but it looks like I was wrong, so maybe winpty needs to be changed here.

[Tyriar]

@rprichard I think on Windows in particularly cls consistently clears the scrollback, it's on the other platforms it differs and I think the reason for that is that terminfo doesn't declare support for 3J for some terminals.

[KristofStroobants]

Well, i'm glad to see it's being looked at. I hope you guys can fix it in the near future.

[KristofStroobants]

Any update @Tyriar ?

[reyou]

is there any way to make this with flag? because, for long outputs, it really kills UX, where I expect to clear the screen, not push it up. Or can you detect OS and behave that way?

cls --force
cls -f

[SHAGGAR]

Any updates? Still an issue in VS Code and they seem to be waiting on this rather than fixing it in VS Code.

[Legends]

Cls should definitely clear the buffer and screen in my opinion, as it is done with cmd.exe on windows.

[Tyriar]

Update: I expect this to be fixed when https://github.com/Microsoft/vscode/issues/45693 is addressed, which I will likely be working on in the coming months.

[Tyriar]

Closing this as it's not related to xterm.js but the pty emulation on Windows. Windows pty issues are being tracked in https://github.com/Microsoft/vscode/issues/45693