Winit: No VirtualKeyCode for question mark

[stevebob]

There is no question mark variant of the VirtualKeyCode type: https://docs.rs/winit/0.19.0/winit/enum.VirtualKeyCode.html

Typing a question mark produces the following (virtual_keycode is None):

KeyboardInput {
    device_id: DeviceId(
        X(
            DeviceId(
                3
            )
        )
    ),
    input: KeyboardInput {
        scancode: 26,
        state: Pressed,
        virtual_keycode: None,
        modifiers: ModifiersState {
            shift: true,
            ctrl: false,
            alt: false,
            logo: false
        }
    }
}

Originally opened in glutin: https://github.com/tomaka/glutin/issues/1104

[goddessfreya]

(Tested by bashing every key on my keyboard)
Missing keys in X11:

• [ ] !
• [ ] #
• [ ] {
• [ ] }
• [ ] [
• [ ] ]
• [ ] |
• [ ] "
• [ ] ?
• [ ] <
• [ ] >
• [ ] $
• [ ] %
• [ ] ^
• [ ] &
• [ ] *
• [ ] (
• [ ] )
• [ ] _
• [ ] ~
• [ ] + (numpad)
• [ ] - (numpad)
• [ ] * (numpad)
• [ ] / (numpad)
• [ ] Shift-Caps-Lock
• [ ] Print Screen
• [ ] Scroll lock
• [ ] Pause/Break
• [ ] Eject
• [ ] Num lock
• [ ] Meta Key (a.k.a. windows key)
• [ ] Rewind to start
• [ ] Play/Pause
• [ ] Rewind to end

[goddessfreya]

In wayland:

• [ ] ~
• [ ] `_
• [ ] |
• [ ] ]
• [ ] [
• [ ] {
• [ ] }
• [ ] >
• [ ] <
• [ ] ?
• [ ] "
• [ ] Caps Lock
• [ ] Eject
• [ ] Rewind to start
• [ ] Play/Pause
• [ ] Rewind to end
• [ ] Meta Key (a.k.a. windows key)
• [ ] Compose Key
• [ ] / (numpad)
• [ ] * (numpad)
• [ ] - (numpad)
• [ ] + (numpad)
• [ ] Up (numpad)
• [ ] down (numpad)
• [ ] left (numpad)
• [ ] right (numpad)
• [ ] home (numpad)
• [ ] page up (numpad)
• [ ] page down (numpad)
• [ ] end (numpad)
• [ ] insert (numpad)
• [ ] delete (numpad)
• [ ] . (numpad)

Possibly assigned wrong keys:

• [ ] !
• [ ] @
• [ ] #
• [ ] $
• [ ] %
• [ ] ^
• [ ] &
• [ ] *
• [ ] (
• [ ] )

[goddessfreya]

@vberger What would be the appropriate course of action for fixing this issue in Wayland and X11?

[stevebob]

Are you sure the problem is just with Weyland and X11? It looks like the VirtualKeyCode type is currently not capable of representing the question mark key. I don't understand how this could work on any platform.

[goddessfreya]

@stevebob This is going to be a problem on every platform, from skimming through the source. However, I've only tested it with X11 and Wayland. I'm asking @vberger for what is the appropriate solution for those two platforms.

More specifically, I want to know what is the expected behavior when using Shift. Consider, Key4. Should Shift-Key4 give KeyCode::Key4 as it currently does in Wayland, or should it give (the currently non-existent) KeyCode::Dollar.

wengwengweng said on gitter:

I personally think it should be KeyCode::Key4, e.g. if I want to match a keyboard short cut alt+shift+4, I think it's more intuitive to do

`if down(Alt) && down(Shift) && down(Key4) { /* stuff */}`

than

`if down(Alt) && down(Dollar) { /* stuff */}`

Of course, the obvious issue is, what if on some keyboards the shifts aren't where we expect them. In the Albanian QWERTZ keyboard for example, Shift-Key2 is ", not @, and what if the user wants to detect when someone presses Alt-@?

https://en.wikipedia.org/wiki/Keyboard_layout#/media/File:Albanian_keyboard_layout.jpg

[Osspial]

@stevebob What OS/keyboard layout are you using? Right now, the VirtualKeyCode enum is modeled after the [American QWERTY layout] with [Microsoft's virtual key-codes], under which this isn't really a problem: since ? is outputted with shift+/, the question mark key is simply represented with the Slash key-code. As a consequence of that windows-centric design, the virtual key-code mappings actually work pretty well on windows and I couldn't find a key with a missing code. If you're using QWERTY and not-Windows, this is likely a problem with VirtualKeyCode's intent not being clear to whoever initially implemented those backends and not understanding what the proper mappings should be.

Despite it working with Windows and QWERTY, I'd say that the keyboard input/key-code API needs to largely get reworked - given the existence of this issue, it's pretty clear that the intents of the non-alphanumeric variants in VirtualKeyCode are pretty badly explained, and said variants map pretty terribly to international layouts. There was a lot of discussion on what that should look like in #753 and I've got a pretty good idea of what I'd personally want it to look like (see my comment at the end of that thread), but I just haven't gotten around to implementing that design and seeing if it's feasible!

(also I feel very strongly that Shift shouldn't change the outputted VirtualKeyCode, since that makes almost everything about the API more confusing).

[goddessfreya]

@Osspial I've reproduced this on both X11 and Wayland with a standard QWERTY keyboard.

[stevebob]

@Osspial I'm using linux with a US dvorak keyboard layout. I definitely misunderstood the intent of VirtualKeyCode. What determines whether a key is represented in VirtualKeyCode? I notice that there is a variant for Colon, Add, Multiply, and many other keys which require shift to be held down in the American qwerty layout.

[Osspial]

@stevebob Right now, the biggest thing that determines if a key is in VirtualKeyCode is whether or not it's on the MSDN virtual key-code list (there have been a couple extensions from other OSes but otherwise that's mostly true). It's... not great for pretty much anyone that isn't using QWERTY, and even under ideal circumstances the API is something of a mess.

On Windows, Add, Multiply, and similar mathematical operation keys correspond to the numpad keys. It looks like Colon is one of the keys that got added in from other operating systems, but it's only supported on emscripten, wayland, and X11.

A lot of my other programming projects have hit various roadblocks, so I think I'm able to take a pass at cleaning this up.

[goddessfreya]

@Osspial I'm interested in seeing how this progresses, and would be willing to write a backend for Linux & Wayland. If you post the high-level changes you plan to do to events.rs and others, I can get cracking,

[Osspial]

@ZeGentzy I'll do that once I have it in place - I'm still a little bit torn about what I want the API to look like, so I need to get that resolved. One catch is that I'm doing this work against the eventloop-2.0 branch, since rebasing big reworks is an absolute PITA and I want to do as little of it as possible, but the X11 backend for that isn't working yet. Wayland should be good to start working on, though.

[vberger]

Okay, apparently we need to make a decision / clarification about the virtual keycodes, because we have some historical cruft.

The information we get from the OS in Linux (both Wayland and X11) allows us to find the keysym (or key symbol). This is a representation of the key, including context from modifiers. For example, on my azerty keyboard, if I press the first key in my numeric row, I'll get the key symbol corresponding to &, while I'll get 1 if I was holding <shift> at the same time.

In the early days, winit/glutin definitely had a more game-centric point of view, and in the context of games, you mostly want to consider physical keys, irrespective of the pressed modifiers (pressing W should make me go forward, irrespective of the status of <shift>). So it was decided to do some attempt at unification with the following rules:

• Uppercase & lowercase letter keys are unified into the same symbol (even though the OS provides us different symbols for them)
• The numeric row always uses the "number" symbol
• The reliable way to get the real, keymap-aware text is through ReceivedCharacter

These rules are currently hardcoded in the wayland backend. It was not really possible to unify further, because keymaps are too different from each other past this point.

From what you say in this issue, it appears these rules are not really uniformly followed with all backends of winit. So I guess it's the point where we need to clearly decide what to do.

[Osspial]

Alright, I've stubbed out the "dream API", which I'd like to have if it's possible to implement across all platforms, but I don't know how feasible it is. The API docs for that are here, and the source code is here.

Feeback and bikeshedding on the new key-code API is encouraged. It's heavily based around the W3C UI Events key-codes, with some modifications. I'm not entirely happy with some of the choices I've made so I'd be happy to see discussion or ideas on how to improve it.

EDIT: The Thumb-Shift keys in there are used for some Japanese Keyboards. Looks like I misread the articles on japanese keyboard input types. I've replaced Kana Lock with IMEMode, and thumb-shift with Convert/NonConvert. I don't know if any OSes actually expose those as key-codes though, and if they don't they should probably get removed.

[vberger]

I'm not sure I get your intent. Is it to just remap everything on QWERTY or ignore the system keymap?

On a more general tone, what about keyboard with keys that just don't exist on an US keyboard? For example the french bépo layout has keys dedicated to É, È, Ê or À.

[Osspial]

The intention is to provide APIs for both QWERTY and ignoring the system keymap, since there isn't really a one-size-fits-all approach: ignoring the keymap is ideal for games, and remapping QWERTY is ideal for desktop apps with keyboard shortcuts. PhysicalKey ignores the system keymap, and LogicalKey remaps the key-codes so they match up with the keymap.

Keys that don't exist on the US keyboard get remapped to the Intl* bindings. I tested it on windows, and bépo remaps the non-QWERTY keys to the OEM keycodes (e.g. é turned into 0xBA, which Microsoft defines as ;:, so é would get mapped to IntlSemicolon).

The documentation currently there is... sparse, to put it kindly, so that needs to get cleaned up to make the intent more clear.

EDIT: Intl is short for International there, a detail which should go in the docs.

[expenses]

I'm strugging with this problem at the moment. I'd really rather not use scancodes for this.