Winit: Keyboard input not captured on latest macOS 10.15.2 system

[luojia65]

Hello! I recently updated my macOS Catalina to 10.15.2. When I try to write winit code like this:

fn main() {
    let event_loop = winit::event_loop::EventLoop::new();
    winit::window::WindowBuilder::new()
        .with_title("Hello world!")
        .with_inner_size(winit::dpi::LogicalSize::new(1024.0, 768.0))
        .build(&event_loop).unwrap();

    event_loop.run(|ev, _target, flow| match ev {
        winit::event::Event::WindowEvent { event, .. } => match event {
            winit::event::WindowEvent::CloseRequested => 
                *flow = winit::event_loop::ControlFlow::Exit,
            winit::event::WindowEvent::KeyboardInput {
                device_id, input: kin, is_synthetic: _
            } => {
                println!("WindowEvent Key: {:?} DeviceId: {:?}", kin, device_id);
            },
            _ => {},
        },
        winit::event::Event::DeviceEvent { event, device_id } => match event {
            winit::event::DeviceEvent::Key(kin) => {
                println!("DeviceEvent Key: {:?} DeviceId: {:?}", kin, device_id);
            },
            _ => {},
        },
        _ => {},
    });
}

When I run and press any keys, I expect to have an both WindowEvent and DeviceEvent in the output. However the output is like this:

WindowEvent Key: KeyboardInput { scancode: 23, state: Pressed, virtual_keycode: Some(Key5), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 23, state: Released, virtual_keycode: Some(Key5), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 22, state: Pressed, virtual_keycode: Some(Key6), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 22, state: Released, virtual_keycode: Some(Key6), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 28, state: Pressed, virtual_keycode: Some(Key8), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 28, state: Released, virtual_keycode: Some(Key8), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 25, state: Pressed, virtual_keycode: Some(Key9), modifiers: (empty) } DeviceId: DeviceId(DeviceId)
WindowEvent Key: KeyboardInput { scancode: 25, state: Released, virtual_keycode: Some(Key9), modifiers: (empty) } DeviceId: DeviceId(DeviceId)

There is no DeviceEvent in the output. Is this expected behavior to have WindowEvent only? Or is there something wrong with my code or operating system (permission or something)? Thanks!

My winit version is winit = "0.21.0".

(I have raised issue at https://github.com/glium/glium/issues/1822 , but that could be an issue on winit)

[sanbox-irl]

If I'm not mistaken, macOS does not send global key press events to applications via AppKit. The only device events the winit macOS backend emits are mouse/trackpad events.

We maybe could reliably do this though using this https://developer.apple.com/documentation/appkit/nsevent/1535472-addglobalmonitorforeventsmatchin?language=objc. Not sure it's a great idea though.

[sanbox-irl]

For now, simply getting the keyboard input (and remember to also get the modifier states, those will also be reported globally correctly from Device Events) will cover you more than enough unless you specifically are trying to log inputs to other windows, in which case first, don't think winit can support it yet

[sanbox-irl]

Sorry -- I just looked through the Documents more carefully. macOS will only send "Key-related events..if accessibility is enabled or if your application is trusted for accessibility access".

That could be a long term thing to support in the future, along with various other macOS accessibility options, but for now, I think this issue should be closed. We could simulate and send DeviceEvents when we can, but that would just be a footgun to programmers who may rely on it for.

Is there a reason you wanted to have both events report?

[luojia65]

I may need to build cross-platform window applications, and do not expect underlying library behavior to change depending on platforms. Maybe it's possible to write to doc that due to accessbility settings DeviceEvent is in some conditions not fired on macOS; or winit could design new APIs to make macOS system calls to ask user for permission if possible (could save this API for other platforms, may always return Ok if permission grant is not needed)

[sanbox-irl]

Yes, it should be made clearer what's going on. Personally, the way I do it is that all platforms cfgh(not(target_os = "macos")) check DeviceEvents for keyinput, and then cfg(target_os = "macos") check for windowkey events.

@Osspial what do you think?

[leeola]

Is there a reason you wanted to have both events report?

I _may_ want this as well, though I'm new to the API so perhaps I'm misguided. My use case is a simple show/hide global key bind. Such that a user could be in one window, use a keyboard and have the Winit appear.

Sidenote, I've been trying to enable accessibility for a binary to get DeviceEvents with no success yet. With respect to:

macOS will only send "Key-related events..if accessibility is enabled or if your application is trusted for accessibility access".

I'm unsure how to enable accessibility within the binary, but I have at least added the Winit binary to the accessibility access under security. No luck getting background key events.

Thoughts?

edit: I fear the binary needs to be code signed.

[luojia65]

How does other macOS applications ask for enable accessibility settings in practice? Are some preconditions that must be satisfied?
(Users may use #cfg by now)

[leeola]

No idea. Hell, I've been toying with Electron and my Rust / WASM / Electron app can catch global key binds without any special accessibility requests or provisions. So accessibility might be a bit off the mark. But perhaps the core electron process is signed, allowing background events? All speculation.

[leeola]

Are there any hacks that might get this to work? So we can get any form of key events _(Device or Window)_ for a background window?

[leeola]

So I was poking around pretty blindly, and I found out that the https://github.com/Narsil/rdev crate successfully reports background key events. Seemingly it even works with winit, though with some logged errors _(I was running rdev in a background thread, of which OSX was not happy about)_.

While I have no clue _why_ rdev works, perhaps it may give insight that this is not solely a permission issue? The same binary allows rdev to receive background input events, but not winit.