Terminal: Numeric input not accepted by choice.exe

[pasccom]

I discovered that choice.exe does not accept numeric input in Terminal whereas it does in the old Windows Console. I am on Windows 10 (update 19.03).
I wrote the following MWE:

@echo off

:_menu
cls
@echo Choose:
@echo   (1) Chocolate
@echo   (2) Vanilla
@echo   (S) Strawberry
@echo   (Q) Quit
@echo.

set _CHOICEOPTS_=q12s

rem -- Choice.com can't be launched in x64 environment ; use builtin "choice.exe" command
if exist %SystemRoot%\system32\choice.exe (
    choice.exe /n /c %_CHOICEOPTS_% /M "Your choice : "
)else (
    choice /n /c:%_CHOICEOPTS_% "Your choice : "
)

set /A _choice_=%ERRORLEVEL%-1
GOTO _menu%_choice_%


:_menu0
GOTO _exit
:_menu1
@echo You chose chocolate
pause
GOTO _menu
:_menu2
@echo You chose vanilla
pause
GOTO _menu
:_menu3
@echo You chose strawberry
pause
GOTO _menu
:_exit
exit /B 0

In Windows console, when I press 1 or 2, I get the expected output, but in Windows Terminal nothing happens (as if 1 and 2 are not accepted by choice.exe).
Could you investigate what is happening? I am ready to provide any relevant information.
Best regards.

[DHowett-MSFT]

This is definitely hitting the choice.exe codepath (not the choice.com one), right? If so: I can't reproduce it over here.

[pasccom]

Yes, I checked I have choice.exe in C:\Windows\System32 and invoked it directly using the fullpath. It still does not work. That is strange you cannot reproduce it.
By the way, I forgot to mention, that I am using 64 bits Windows.
Best regards.

[j4james]

For the record, I also couldn't reproduce this issue, and I'm using 64-bit Windows. I wonder whether it has something to do with your keyboard. Are you perhaps using a French keyboard?

[pasccom]

Yes, I have an AZERTY keyboard, with numpad.
I tried to enter numeric choices using the numpad and the usual keyboard (on French keyboards, you have to press Shift simultaneously).
Best regards.

[j4james]

I tried using the Windows language preferences to emulate a French keyboard, and I still couldn't reproduce the problem, but I don't think that's necessarily an accurate representation of how a real French keyboard would work. Based on past experience, I think we'd need an actual French keyboard to reproduce this issue. But the fact that you have to use the Shift key to enter numbers is a strong indication that the keyboard is a likely cause.

[pasccom]

I tried switching my keyboard to english or german language (I have FR, EN, DE laguages installed) before starting windows Terminal: Numeric input works fine :-)
Then, while the terminal is still open but the program is closed, I switched back to French: choice still works fine :-). If I open a new terminal, the program does not work as before.
But if I switch to En language, open the terminal, switch back to French language without running the program, and run the program for the first time in FR language it does not work!
There is definitely a problem with French language, but it is weird it works when I first run the program in English or German.
I hope this is clear enough.
Best regards,

[j4james]

OK, I can reproduce the problem now too. As you suggested, if I set my keyboard to French _before_ starting the terminal, it fails for me too. That's very useful to know. I thought you'd have to have an actual French keyboard, but that's not necessary. Hopefully that'll make it easier to fix.

[DHowett-MSFT]

Notes:

Possibly start an investigation in InputStateMachineEngine's ProcessString

[skyline75489]

An useful observation would be that this bug seems to be related to Shift modifier key.

With French keyboard you need to hold Shift and press 1 to type the number "1". A similar way to reproduce this without switching to French keyboard would be just using plain choice.exe and typing capitalized "N" using Shift.

@DHowett-MSFT It does not feel like a VT level thing. Take a look at the screenshot:

This is a result of me typing "N" repeatedly, but to no avail. You can see from the right panel that the letter "N" is correctly processed. But somehow choice.exe acts like nothing happens.

Surprisingly enough, using Caps Lock to type "N" also leads to this exact same bug.

[zadjii-msft]

That's super weird. Thanks for finding that, this is a super helpful observation

[pasccom]

NB: I have a French keyboard with a numpad. When typing digits on the numpad you do not need any additional modifier. But, digit input does not even work from the numpad.

[leonMSFT]

So, after digging for quite a while and losing 1/4 of my mind, it turns out that choice.exe has a giant conditional that checks if the first KeyEvent in the InputBuffer is one of _many_ modifiers and keys, which include shift and CapsLock. In response, it'll flush the buffer, and with it, the actual key you wanted to send.

CMD seems to work with choice because it'll attempt to send each KeyEvent the moment it receives it. So, choice will get at an InputBuffer of just SHIFT_DOWN, which choice will flush. The next peek will give choice a 1_DOWN that will be accepted. However, this relies on the fact that when a user is at their keyboard, they generate KeyEvents slowly enough. If CMD were to send both the SHIFT_DOWN and the 1_DOWN at the same time, the choice bug would repro. You can see this happening if you tried to paste "1" in French. Choice wouldn't take it because the pasted "1" gets converted into a single read packet.

Terminal sends all of the SHIFT_DOWN, 1_DOWN, 1_UP, and SHIFT_UP KeyEvents all at the same time, so that's why choice will see the SHIFT_DOWN and decide "welp screw it I don't want to see anymore" and flushes the buffer. To answer the numpad-1 bug, it's because Terminal converts "1" into VKs, so a "1" in a French keyboard will get converted into "shift-1 (in the top row)" because that's the most likely way to type 1 in a French keyboard.

Unfortunately, there's no easy way to fix this bug in Terminal, but honestly, choice shouldn't be flushing the entire input buffer the moment it decides it doesn't like the first KeyEvent. It also really should not be relying on KeyEvents being generated slowly enough so that it won't flush the KeyEvents following the modifier KeyEvents.

[skyline75489]

@leonMSFT Excellent finding! I've also noticed that there are 4 key events together, but I didn't realize it is the root cause of choice.exe's misbehaving.

I guess choice.exe is probably a thing from 1980s or something. My concern is that how many of this kind of old tools out there share this behaviour (aka bug) with choice.exe. And also are there some kind of modern substitubes for them? Since there's no easy fix for the bug, hopefully people can find a way to mitigate the damage.

[j4james]

If we decide to go with DECPCTERM as a solution for #879, I suspect that would solve this issue too, because the actual press and release events would be passed through conpty exactly as they happened.

[tavrez]

Did you guys know choice is also can't accept uppercase things in case-sensitive mode?

CHOICE /cs /c aA

You can never choose capital A, either by shift-a or turning on caps-lock and press a.
in cmd it is fine, bug is only in new terminal.
If this bug is new I'll open a new issue if needed.

[DHowett-MSFT]

This is actually caused by the same issue.

[msftbot[bot]]

:tada:This issue was addressed in #6309, which has now been successfully released as Windows Terminal Preview v1.1.1671.0.:tada:

Handy links:

• Release Notes
• Store Download