Terminal: Consider supporting [extension to VT,] DECSCUSR "0" to restore cursor to user default to help vim/others

[jtourlamain]

Environment

Windows build number: 1903
Windows Terminal version (if applicable): 0.2.1715

Any other software? neovim v0.3.4

Steps to reproduce

• Create a profile that uses cursorShape "vintage"
• Open nvim
• close nvim
  

Expected behavior

After closing nvim I would expect to have the "vintage" cursor again.

Actual behavior

Cursor is a filledBox that doesn't blink anymore

[egmontkob]

I'm not sure what escape sequences Windows Terminal supports for changing the cursor shape, nor what (Neo)Vim outputs, so sorry if I get it wrong.

I assume that the DECSCUSR escape sequence is being used (see e.g. at ctlseqs.html).

In gnome-terminal (vte) we intentionally deviate from the xterm's behavior (the behavior documented on the linked page). For parameters 1–6 we go with the given mode. However, if the parameter is 0 (or missing), we revert to whichever cursor shape the user specified as their favorite in the terminal's preferences.

This might be a behavior for WIndows Terminal to consider, too.

[xtremeperf]

This is actually intended behavior for Vim/Nvim and I can't recall any sort of "restore cursor leaving application mode" termcap that could be implemented in the terminal.

And yes, that's correct... DECSCUSR has been implemented in WindowsTerminal for changing cursor shape (1-6).

Vim explicitly changes the cursor shape for each of it's editing modes and it requires that you assign the ESC sequences in your vimrc file, as they are not using any sort of standard termcap variable for this.

Vim does not change the cursor shape on exit. The cursor shape from normal mode will remain, since you must exit from normal mode.

For more info in Vim:
:help termcap-cursor-shape

It's easy to change the cursor shape upon exit for yourself using an autocmd VimLeave
:help VimLeave

I hope that helps!

[jtourlamain]

@xtremeperf Thanks for your tip!
Found a possible solution on https://github.com/neovim/neovim/issues/4867.
Adding au VimLeave * set guicursor=a:hor10-blinkon1 gives a blinking underscore when you exit vim

[egmontkob]

The DECSCUSR sequence is a poorly designed one.

One problem is the lack of save/restore or revert, which we somewhat address in gnome-terminal by deviating from the standard. This way vim could restore the user's preferred version on exit.

The other problem is that two orthogonal properties are tied together. For an app like vim, it really makes sense to toggle between two different shapes, while leave blinking at the user's preferred setting. It's currently not possible.

The best solution could be to cooperate between terminal emulators and apps to come up with a better designed escape sequence. Until then, we live with the limitations, and optionally come up with hacks to mitigate the problems.

[DHowett-MSFT]

I like the idea of using a nonstandard DECSCUSR extension to reset the cursor to "user default."

[DHowett-MSFT]

I'll float that to @zadjii-msft -- if we don't want to do it, we'll close this issue as "unfortunately by design". :smile:

[xtremeperf]

I looked into this a little more. The main reason the cursor doesn't get reset when returning to the shell is because neovim doesn't enter alternate screen mode upon startup... therefore it doesn't return from any sort of alternate screen mode upon exit.

Here's how it's done in Vim using xterm sequences (as in DECRC)
t_ti=^[[?1049h
t_te=^[[?1049l

[egmontkob]

I don't get it, what's the connection between cursor shape and alternate screen? At least in xterm and gnome-terminal it's the same shape for both screens, not one per screen. Is it different in Windows Terminal?

[xtremeperf]

There are some XTERM-specific capabilities for an Alternate Screen Buffer and Save/Reset Cursor, among others. The intent is provide a picture of the full-screen application's display on the scrollback, but without wiping out the text that would be shown before the application was initialized. The display and cursor properties are saved and then the terminal switches to an alternate buffer with it's own properties. When done, the initial display, along with it's cursor, are restored.

DEC Private Mode _Set_ (DECSET) --- CSI ? _P m_ h
DEC Private Mode _Reset_ (DECRST) --- CSI ? _P m_ l

_P s_ = 1 0 4 7 --- Use Alternate Screen Buffer
_P s_ = 1 0 4 8 --- Save cursor as in DECSC
_P s_ = 1 0 4 9 --- Save cursor as in DECSC and use Alternate Screen Buffer, clearing the screen first. (This mode actually just combines the effects of the 1 0 4 7 and 1 0 4 8 modes.)

I haven't tested these sequences while using various different combinations of cursor shape between each screen buffer specifically, but after running vim and changing the normal cursor to a solid line, the original underscore cursor shape, blink, position, etc. was restore upon exiting the alt. screen buffer mode.

[egmontkob]

In Xterm and VTE, the same cursor shape+blinking is shared across the normal and alternate screen. Also, saving/restoring the cursor does not save/restore its shape+blinking. That is, it looks to me that the following three are independent:

1. normal vs. alternate screen
2. cursor, as in location, plus color and similar attributes used for printing subsequently arriving text
3. cursor shape and blinking

and there just happens to be an escape sequence which sets 1. and 2. at the same time.

the original underscore cursor shape, blink, position, etc. was restore upon exiting the alt. screen buffer mode.

It could be that Windows Terminal differs from Xterm and VTE here.

[egmontkob]

Also note that showing/hiding the cursor (DECTCEM) seems to be an exempt from saving/restoring or alternate screen switching.

So in bullet point 2 above, "cursor" stands for the logical concept, "active data/presentation position" as ECMA-48's calls it; whereas in bullet point 3 "cursor" refers to its graphical representation.

(ECMA-48: "In general, the active presentation position is indicated in a display by a cursor.")

ctlseqs.html doesn't have this clear distinction, it uses the word "cursor" for both concepts, just as probably most people do.

[zadjii-msft]

To add my 2c: Looking at the spec for DECSCUSR, it seems like 0 is strictly redundant.

CSI Ps SP q
          Set cursor style (DECSCUSR), VT520.
            Ps = 0  -> blinking block.
            Ps = 1  -> blinking block (default).
            Ps = 2  -> steady block.
            Ps = 3  -> blinking underline.
            Ps = 4  -> steady underline.
            Ps = 5  -> blinking bar (xterm).
            Ps = 6  -> steady bar (xterm).

I'm definitely okay with having 0 act as "restore to user default", _especially_ considering there's precedent in other terminal emulators for this behavior.

[jdebp]

The terminfo capabilities for set cursor style and reset cursor style are Ss and Se, respectively, and have been in terminfo since 2011. NeoVIM has a lot of code that tries to fixup these sequences for known terminal types if they are missing, as will be the case if the terminfo database is not up to date. When it does so, it (mostly) employs DECSCUSR with explicit parameters for setting the style, and nothing for resetting it. It never uses 0 as the parameter value.

0 is redundant because DEC VTs followed the original ECMA-48:1976, where 0 explicitly meant the "default value which depends on the control function". You'll notice this pattern in other control sequences. In later versions of ECMA-48 this changed, and 0 now explicitly means an actual zero value for the parameter. There's a lengthy explanation of this in annex E of ECMA-48:1986.

So in the case of DECSCUSR: since 1 is the default, 0 means 1. Arguably, since this oddity was fixed _back in 1986_, there's a case that a modern implementation can get away with adhering to the 34-year-old "new" semantics given by the standard. Certainly Windows Terminal would not be the first terminal emulator to do so. (There are already terminal emulators out there that do no-operations when cursor motions have counts of zero.)

However, an empty parameter string still means a default. I've used DECSCUSR with _an empty_ parameter to set the "default" in other softwares. There really is not a need to define _another_ way of saying "set the default cursor glyph". It's already there. There is a case for having a way to say "set the user's locally-configured cursor glyph", for which 0 seems a fair candidate, albeit one that _might_ fall foul of some unknown software out there somewhere that really does use DECSCUSR 0. But that's not the same as the _default_, which is defined by DEC not by the user. A _default_ in an ECMA-48 control sequence is not the same as _locally-configured_.

[DHowett-MSFT]

@jdebp sorry, I can't tell which of the three things (perhaps there're more than three? I counted _don't change the behavior here_, _use CSI SP q as set-locally-configured_ and _use CSI 0 SP q as set-locally-configured_) you're advocating for :smile:

[egmontkob]

However, an empty parameter string still means a default.

This was recently questioned here.

There really is not a need to define another way of saying "set the default cursor glyph". It's already there. There is a case for having a way to say "set the user's locally-configured cursor glyph" [...]

I assume VT520 didn't have a setup for the cursor preference -- let me know if I assume this incorrectly. I assume any time you switched it on, it started with a blinking block.

Same goes for xterm, it's not really configurable. Well, there's a preference to start with an underscore which I've just tried now (xterm -uc) (there's no corresponding option for I-beam, I wonder why), but then if you switch to block cursor, and then reset the terminal, it becomes horribly broken. It's unclear to me what it would do if it wasn't broken, i.e. which shape it would change to. So let's just say xterm doesn't support a different default either.

However, software that aims to be user friendly, such as Windows Terminal or GNOME Terminal, do offer a user setting to pick the preferred cursor shape.

And carrying two separate concepts: a "default cursor" (as specified in some 30+ year old documentation), and a separate "initial cursor" (which the user configured), is anything but user friendly. It also raises nontrivial questions like what should a hard reset do (should it switch to the user's preferred aka initial cursor [yeah perhaps - but then the block cursor being the "default" no longer means anything], or the VT520 block default cursor [presumably doesn't make any sense to users], or leave it untouched, and why), same question for soft reset, and also should there be a way to reset only the cursor and nothing else to the user's preferred version [preferably yes].

The only reasonable thing to do is to forget about the "default" state, as seen in VT520's and xterm's documentation, and optionally (preferably) introduce a way to revert to the user's "preferred" state (whatever is specified in the terminal emulator's configs). And I don't think that reassigning 0 and the missing parameter to switch to the "preferred" state is necessarily a bad thing. Supporthing this new feature, user customization is more important than the state-of-the-art 30+ years ago.

IMO standards, documentations of 30+ years have to be treated with respect, but it's okay to question and sometimes even override them on a case-by-case basis. Insisting to strictly implement them, no matter what it says, would block evolution of the ecosystem. I could point you to ambiguities and even errors in T.416 around true colors, or a quite bad BiDi in ECMA TR/53 and 48 (preceding any Unicode BiDi experience), fixing which required to clarify, extend, and in one case straight override those two ECMA standards. Deviating from an ancient, unmaintained standard is not necessarily bad. You should ask yourself: if responsible people updated the specs to reflect today's world and best practices, how would it presumably look like? Or if it had a public bugtracker, how would the relevant issue look like? In our case, how would a specification deal with the need for the user to configure their preferred cursor shape?

Mind you, I can think of ECMA-48 as a de jure standard, but it doesn't deal with DECSCUSR. VT520's documentation and xterm's documentation (ctlseqs.txt) are documentations of a particular implementation, and as such I believe they can be treated as de facto standards, but not de jure ones. Deviating from a de facto standard is even less of a theoretical problem than deviating from a de jure one.

VTE (GNOME Terminal and friends), which has in the ballpark of 50% market share among Linux terminal emulators, has supported this revised interpretation of DECSCUSR for 5 years now, and we haven't heard anyone complaining that 0 doesn't always switch to the solid rectangle. In practice, it turned out to be a fully safe choice.

[What is bad, though, is that DECSCUSR combines two orthogonal properties (shape and blinking) into one, you cannot restore one to the user's preference without restoring the other one as well. Ideally, DECSCUSR would be deprecated by two new sequences that modify one property each. And then the meaning of 0 could also be documented clearly.]

[jdebp]

ECMA-48 very much does deal with DECSCUSR and any other private control sequence, because of course it defines the form for private control sequences. The irony of your trying to denigrate the standards is that _as I just explained_ it is the standards that give you the behaviour that you want.

• Follow the standard from 1986, and you get 0 available as a settable value, which you can then take to mean _locally-configured by the user_.
• Follow the DEC VTs and do not adhere to ECMA-48:1986, as you are arguing in favour of, and you _do not_ get that, because per the behaviour embodied in DEC VTs a 0 parameter means to use _some other non-zero_ value which is specified as the fixed default for _both_ an empty parameter _and_ a zero parameter. This default is 1 in the case of DECSCUSR.

James Holderness wants to follow the DEC VTs and not the standards, too, doing which would mean that Windows Terminal cannot agree with VTE, because VTE is _not doing_ what the real terminals actually do.

Foolishly, you're approaching this completely backwards for the outcome that you want. Stick to ECMA-48. Encourage Microsoft to stick to ECMA-48, too. Ironically for all that blather about asking onesself about updating specs and attempts to justify ignoring what the DEC doco says, _it was the ECMA standard that was updated to fix this and give you the opening that you need_. 34 years ago.

[j4james]

Since I've been mentioned here, I feel I should clarify my position. My primary goal is to try and make conhost (and thus WT) a better VT terminal emulator. That means following the behaviour and specifications of the DEC terminals first and foremost. However, if there is a _consensus_ among modern terminal implementations that deviates from the DEC behaviour, and which has become a de facto standard, I wouldn't necessarily be opposed to such extensions.

That said, I don't make the decisions here. I'll offer my opinion (for what it's worth), and submit PRs for the features that I would like to see implemented, but I don't expect everyone to agree with me.

Regarding this particular issue, if there is a _consensus_ on treating DESCUSR 0 (or whatever) as a way to restore the default cursor, and that's something that apps like VIM are already using (I'm not sure if that is actually the case), then I'd agree it's worth implementing. However, if nobody is using it yet, I'd be inclined to leave it for now.

[j4james]

I should also mention that there is already a DEC standard for querying (and thus being able to restore) the user's cursor settings - the DECRQSS command. We don't yet support that, but it is on my long term TODO list.

[craigbarnes]

There are some XTERM-specific capabilities for an Alternate Screen Buffer and Save/Reset Cursor, among others. The intent is provide a picture of the full-screen application's display on the scrollback, but without wiping out the text that would be shown before the application was initialized. The display and cursor properties are saved and then the terminal switches to an alternate buffer with it's own properties. When done, the initial display, along with it's cursor, are restored.

DEC Private Mode Set (DECSET) --- CSI ? P m h
DEC Private Mode Reset (DECRST) --- CSI ? P m l

P s = 1 0 4 7 --- Use Alternate Screen Buffer
P s = 1 0 4 8 --- Save cursor as in DECSC
P s = 1 0 4 9 --- Save cursor as in DECSC and use Alternate Screen Buffer, clearing the screen first. (This mode actually just combines the effects of the 1 0 4 7 and 1 0 4 8 modes.)

I haven't tested these sequences...

@xtremeperf I have...

In xterm, DECSET 1049 seems to behave as documented, i.e. the [effects] on the cursor are the same as for DECSC, which doesn't save the cursor style.

Requesting the current state with DECRQSS might work, but it's subject to all the usual problems -- request/response latency, lack of widespread support, etc.

[j4james]

Requesting the current state with DECRQSS might work, but it's subject to all the usual problems -- request/response latency, lack of widespread support, etc.

I know of a number of terminal emulators that support DECRQSS. Those I've personally tested include xterm, kitty, mintty, and tera term, and I've seen many more references online claiming to have implemented it. I know the support isn't universal, but is this DECSCUSR 0 hack any better?

As for latency, that's surely not an issue for something like an editor (which was the original use case for this issue) . You fire off the request sequence at startup and then just carry as normal. If you receive a response at some point before shutting down, then you've got what you need to restore the cursor. If not, it's hardly the end of the world.

[craigbarnes]

As for latency, that's surely not an issue for something like an editor (which was the original use case for this issue) . You fire off the request sequence at startup and then just carry as normal. If you receive a response at some point before shutting down, then you've got what you need to restore the cursor. If not, it's hardly the end of the world.

Yeah, that's how I envisioned doing it for this particular issue too, although the latency can be annoying for other uses of DECRQSS. I've tried half a dozen terminals and the only problem I've run into is with tmux not supporting it. However, tmux does seem to support the DECSCUSR 0 hack, so the obvious thing to do would be use the DECRQSS response if there is one, otherwise fall back to DECSCUSR 0.

[gnachman]

I'm supportive of the proposal to repurpose 0 to restore the user's default in iTerm2, which came to my attention here: https://gitlab.com/gnachman/iterm2/-/issues/8769

I'd like to see one or two more terminals commit to this before making the change, though. I'm sure it will break some users' configurations, and it's only justifiable if it's the clear path forward.

[j4james]

@gnachman I've done a bit of testing on the terminals I have installed, and there are actually a few that support this extension. Other than VTE, there's also mintty and alacritty, which support both 0 and a missing parameter for selecting the user's preferred cursor. Then there is kitty (the Linux terminal, not the putty fork) which supports only 0 (a missing parameter selects the block cursor).

I still think DECRQSS makes more sense for apps that actually want to restore the cursor shape (rather than just resetting it to a default value), but DECSCUSR 0 is not a bad fall back option.

[gnachman]

Thanks for providing the additional context, @j4james . I'm not a fan of DECRQSS for two reasons:

1. Using it casually (like in a shell script) is too hard.
2. Reporting in general is full of race conditions, where the reported code sometimes gets printed to the screen if the user hits a key, a program crashes, or an ssh connection ends at just the wrong moment.

I think the strongest argument in favor of adopting this change is that 0 is the same as 1. Anyone who has problems can fix them without too much trouble. Considering VTTY, mintty, and alacritty already support it, that's strong evidence that there is a lot of hard-to-fix software in the wild.

[gnachman]

FYI I've implemented DECSCUSR 0 to reset cursor style & blink in https://github.com/gnachman/iTerm2/commit/5680f974ab28d436ef29bf562e124fbd3937b96d

[msftbot[bot]]

:tada:This issue was addressed in #7379, which has now been successfully released as Windows Terminal Preview v1.4.2652.0.:tada:

Handy links:

• Release Notes
• Store Download

[DHowett]

The fix for this issue was just released as part of Windows in Insider Build 20236!