Terminal: Incorrect keycode for "CTRL+H" in Windows Terminal

[skywind3000]

Environment

Windows: 
Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.18363.0 Microsoft Windows NT 10.0.18363.0

Vim.exe: 8.2.701

Steps to reproduce

1. start "cmd.exe" in windows terminal.
2. run vim.exe (https://github.com/vim/vim-win32-installer/releases)
3. input "iaaaaaaa" to enter insert mode and append some characters.
4. press "CTRL+h" several times

Expected behavior

The previous "aaaaa" should get deleted (CTRL+H in vim's insert mode will delete previous character), just like running vim.exe in the traditional cmd.exe window.

Actual behavior

"CTRL+h" inserts some strange characters: "ÎzÎzÎzÎzÎzÎzÎz"

after some inspection, I located the difference, if I use getch in msvcrt.dll to display keycode:

1. Run cmd.exe directly:

D:\>python -c "import msvcrt;print(msvcrt.getch())"
b'\x08'                   <--- press CTRL+H

D:\>python -c "import msvcrt;print(msvcrt.getch())"
b'\x08'                   <--- press BackSpace

Both "CTRL+H" and "BackSpace" are b'\x08', vim.exe works fine with it .

1. Run cmd.exe in Windows Terminal:

D:\>python -c "import msvcrt;print(msvcrt.getch())"
b'\x7f'                   <--- press CTRL+H

D:\>python -c "import msvcrt;print(msvcrt.getch())"
b'\x08'                   <--- press BackSpace

Now, "CTRL+H" becomes b'\x7f' in Windows Terminal, which is different from running cmd.exe directly without windows terminal, and that difference breaks some terminal softwares like vim.exe .

[skywind3000]

I checked with WSL in Windows Terminal:

It works as expected, "CTRL+H" sends 0x08 and "BackSpace" sends 0x1f. This is what most linux terminal do.

But why "CTRL+H" behaves differently with cmd.exe ??

[tasogare3710]

I'm not sure, but there may be this issue with app that uses PDCurses.

@skywind3000

But why "CTRL+H" behaves differently with cmd.exe ??

In the old days, no backspace key on keyboard, and ctrl + H (^H) was used to input the control character backspace(08).

The control key is a key for input control characters.

[DHowett]

So, yes. We placed the Windows encoding for Ctrl+H on the sacrificial altar when we fixed Ctrl+Backspace. We've got plans in the works that'll make this better.

Thanks.

[DHowett]

For reference, this will be part of #4999.

[skywind3000]

In the xterm standard:

• CTRL+H is 0x08.
• BackSpace is 0x1f.
• CTRL+BackSpace is 0x1f.

They won't conflict with each other.

[DHowett]

?

I just used xterm 351 and pressed Backspace, Ctrl+H and Ctrl+Backspace.

They have the same encoding.

[skywind3000]

@DHowett

Thanks for including this in milestone 1.1,

@tasogare3710 ,

Vim doesn't require PDCurses, it uses Win32 console APIs directly. You may say PDCurses and old console apps have bugs for this, but why getch in msvcrt.dll can't produce the same keycode for CTRL+H ?

Try this:

python -c "import msvcrt;print(msvcrt.getch())"

and press CTRL+H.

At least, Windows Terminal should not break msvcrt, isn't it ?

[skywind3000]

@DHowett , the xterm app you are using are too old, you can try one of:

• windows: Putty, MinTTY, XShell
• linux: gnome-terminal
• macos: iterm2, terminal.app

Most modern terminal apps send 0x08 for CTRL+H, 0x7f for BackSpace by default, and they provide options for customizing:

XShell's default:

PuTTY's default:

Terminal.app's default:

iTerm2's default:

I think I made my point.

[DHowett]

@skywind3000 None of those configures what _ctrl+backspace_ does, which is the issue we are discussing. I know that _backspace_ is configurable.

[GoodClover]

I tested this running cmd prompt in Windows Terminal, thought it might be useful here.

That keyboard section of the table is what I typed.

The common section of the table is what is "correct" for that keycode - not necacerily the most common usage.

| Keyboard | Decimal | Hex | Binary | Unicode | Common |
| ---------: | ------: | --: | --------: | --------- | ----------------------------------------|
| Backspace | 8 | 8 | 1000 | Backspace | Backspace, ^h |
| ^? | 8 | 8 | 1000 | Backspace | Backspace, ^h |
| ^h | 127 | 7F | 1111111 | Delete | Delete, ^? |
| Delete | 330 | 14A | 101001010 | Ŋ | LATIN CAPITAL LETTER ENG |
| ^Backspace | 127 | 7F | 1111111 | Delete | Delete, ^? |
| ^Delete | 527 | 20F | 100000111 | ȏ | LATIN SMALL LETTER O WITH INVERTED BREVE |

Following the pattern that ^a ^b ^c are 0x1 0x2 and 0x3, I would assume that ^h would be 0x8, it is not.

^h should be identical to Backspace.

These kinds of "errors" appear to happen with other keys as well. This isn't always a bad thing as keyboards keys meanings do evolve over time. And for correct usage the key may have to change what it represents.

However here is what I think the "correct" table would be:

| Keyboard | Decimal | Hex | Binary | Unicode | Common|
| --------: | ------: | --: | ------: | --------- | ------------- |
| Backspace | 8 | 8 | 1000 | Backspace | Backspace, ^h |
| ^? | 127 | 7F | 1111111 | Delete | Delete ^? |
| ^h | 8 | 8 | 1000 | Backspace | Backspace, ^h |
| Delete | 127 | 7F | 1111111 | Delete | Delete ^? |

^Backspace and ^Delete don't have a standard for what the should be as far as I know.

EDIT: My markdown was broken.
EDIT 2: Some other oddities:

| Keyboard | Output | Out Hex | Expected | Exp Hex |
| -----------| ---------|---------:|------------|---------:|
| ^H | ^? | 7F | ^H | 8 |
| ^J | ȑ | 211| ^J | A |
| ^M | ^J | A | ^M | D |
| ^Enter | ȑ | 211 | ^Enter | N/A |

(I can't test ^V for obvious reasons)

[skywind3000]

objection, I use CTRL+h and BackSpace in vim for different purpose,
CTRL+h/j/k/l is always for windows switching in my vimrc and BackSpace is for invalidate client area.
their can't share the same keycode. BackSpace uses 127 by default in many modern terminals.

[GoodClover]

This isn't always a bad thing as keyboards keys meanings do evolve over time. And for correct usage the key may have to change what it represents.

I didn't mean that it should be backspace I was just saying that due to the way the Control key works Ctrl+H ends up being hex 8 which is back space. The ctrl key wasn't intended to be used the way it is today and that causes issues. It was intended for entering control codes.

If you look at Unicode chart 0
Backspace has the code 0x8.

Now look at this:
| Keyboard | Hex | Hex Meaning |
| ---------- | ----: | ------------------ |
| ^A | 01 | Start of Heading |
| ^B | 02 | Start of Text |
| ^C | 03 | End of Text |
| ^D | 04 | End of Transmission |

See the pattern?
Following this ^H would be 08
But this would cause issues as that means Backspace and that character is still in common usage.
Therefore:
^H has been replaced with 7f (127) meaning delete.
^? has been replaced with 08 (8) meaning backspace.
Delete has been replaced with 14a (330) meaning "Ŋ".
Etcetera.

To make it clear:
^H literally means backspace.
^H and ^? have been swapped.

Another example if you need it:
^I (i not L) is 9
9 means Horizontal Tabulation (Known as tab).
Try it, ^I will usually give you a tab.

I'm not on either side of the argument, just pointing out what you seem to have missed and that is what the control key actually does.

[GoodClover]

Though of something. Here it is:
Imagine each letter being replaced by its place in the alphabet.
e.g A=1, B=2, C=3
Now to get that letter in ASCII/Unicode you add 64 (1000000) to it.
To get it lowercase add 96 (1100000) (another 32) to it.
Using that number by itself gives you a control code.
To get this control code your press the (guess what!) control key.

This causes issues using Ctrl for shortcuts as you don't want the control code as the key does, you want your shortcut.
A lot of control codes aren't used anymore so its usually ok though.
For the ones that do cause issues they get re-wired sometimes. (Like ^H and ^J) to other symbols that don't really get used (Like ȑ and ȏ).

I'm not saying this is the right thing to do, but it's what is done.