Raylib: [core] Mouse movements are bound to the screen resolution

[GitSparTV]

Issue description

On the maze example the mouse movements are bound to the screen resolution I have.
Also the camera movements are shuttering when I hold move key.

Environment

Raspberry PI 4 no X11, 446f9ff commit. PLATFORM_DRM.

[GitSparTV]

Video:
https://imgur.com/R05SyMt

[GitSparTV]

Log:
```
INFO: Initializing raylib 3.1-dev
INFO: DISPLAY: no graphic card set, trying card1
INFO: DRM: choosen mode 848x480 (848x480p@60)
INFO: DISPLAY: Upscaling required: Screen size (800x450) smaller than display size (848x480)
INFO: DISPLAY: Device initialized successfully
INFO: > Display size: 848 x 480
INFO: > Render size: 800 x 453
INFO: > Screen size: 800 x 450
INFO: > Viewport offsets: 0, 3
INFO: GL: OpenGL device information:
INFO: > Vendor: Broadcom
INFO: > Renderer: V3D 4.2
INFO: > Version: OpenGL ES 3.1 Mesa 19.3.2
INFO: > GLSL: OpenGL ES GLSL ES 3.10
INFO: GL: Supported extensions count: 76
INFO: GL: VAO extension detected, VAO functions initialized successfully
INFO: GL: NPOT textures extension detected, full NPOT textures supported
INFO: GL: ETC1 compressed textures supported
INFO: TEXTURE: [ID 1] Texture created successfully (1x1 - 1 mipmaps)
INFO: TEXTURE: [ID 1] Default texture loaded successfully
INFO: SHADER: [ID 1] Compiled successfully
INFO: SHADER: [ID 2] Compiled successfully
INFO: SHADER: [ID 3] Program loaded successfully
INFO: SHADER: [ID 3] Default shader loaded successfully
INFO: RLGL: Internal vertex buffers initialized successfully in RAM (CPU)
INFO: RLGL: Render batch vertex buffers loaded successfully
INFO: RLGL: Default state initialized successfully
INFO: TEXTURE: [ID 2] Texture created successfully (128x128 - 1 mipmaps)
INFO: FONT: Default font loaded successfully
INFO: RPI: Opening input device: /dev/input/event1 (mouse )
INFO: RPI: Opening input device: /dev/input/event0 (keyboard )
WARNING: RPI: Failed to open Gamepad device, no gamepad available
INFO: FILEIO: [resources/cubicmap.png] File loaded successfully
INFO: IMAGE: [resources/cubicmap.png] Data loaded successfully (32x16)
INFO: TEXTURE: [ID 3] Texture created successfully (32x16 - 1 mipmaps)
INFO: VAO: [ID 2] Mesh uploaded successfully to VRAM (GPU)
INFO: FILEIO: (resources/cubicmap_atlas.png] File loaded successfully
INFO: IMAGE: Cresources/cubicmap_atlas.png] Data loaded successfully (256x256)
INFO: TEXTURE: [ID 4] Texture created successfully (256x256 - 1 mipmaps)
INFO: TIMER: Target time per frame: 16.667 milliseconds
INFO: TEXTURE: [ID 3] Unloaded texture data from VRAM (GPU)
INFO: TEXTURE: [ID 4] Unloaded texture data from VRAM (GPU)
INFO: VAO: [ID 2] Unloaded vertex data from VRAM (GPU)
INFO: MODEL: Unloaded model from RAM and YRAM
INFO: TEXTURE: [ID 2] Unloaded texture data from VRAM (GPU)
INFO: TEXTURE: [ID 1] Unloaded default texture data from VRAM (GPU)
INFO: Window closed successfully

The text was OCR'ed, might have typos.

[chriscamacho]

off the top of my head the mouse is being handled differently - on this platform you'll need to adjust the mouse position back to the centre of the screen after getting the mouse movement delta - this platform really should behave the same should behave the same.

[raysan5]

@GitSparTV I think @chriscamacho is right, managing the mouse while not X is up to the user, just reading mouse delta... problem is that letting user manage it is probably not possible at the moment...

Related code is in core.c module, in function EventThread(), that function creates inputs threads, it's used to read mouse...

There is a piece of code down there:

// Screen confinement
if (CORE.Input.Mouse.position.x < 0) CORE.Input.Mouse.position.x = 0;
if (CORE.Input.Mouse.position.x > CORE.Window.screen.width/CORE.Input.Mouse.scale.x) CORE.Input.Mouse.position.x = CORE.Window.screen.width/CORE.Input.Mouse.scale.x;

if (CORE.Input.Mouse.position.y < 0) CORE.Input.Mouse.position.y = 0;
if (CORE.Input.Mouse.position.y > CORE.Window.screen.height/CORE.Input.Mouse.scale.y) CORE.Input.Mouse.position.y = CORE.Window.screen.height/CORE.Input.Mouse.scale.y;

It seems the root of the problem, that function should be reviewed...

[kernelkinetic]

The EV_ABS event reports absolute values, I would ignore the relative values reported by EV_REL. But I know that nowadays many devices have more than one input device file to support different modes like normal mouse, touch, multi-touch etc. and you need carefully select the right input device file to get the right values of interest. On Raspbian you can install the evtest package and use it to see which device file reports which events and also which values. So maybe another input device file works better.

[kernelkinetic]

Okay. I'm completely wrong here with the input devices 😀 I think the __real__ source of the problem is, that X11 (like Windows) limits the mouse movements on the screen but reports the mouse movements to the programs, while on PLATFORM_DRM the limitation is already done (at https://github.com/raysan5/raylib/blob/fa357b8d5d934f54dd1b3cd343b05f06b732d28f/src/core.c#L5340-L5345 as you said, @raysan5 ) before it's reported from raylib to the user software. Imho this limitation should be applied on drawing the red mouse square but not on reporting the mouse movement.

[kernelkinetic]

@GitSparTV I see screen flickering only when on llvm pipe (which is software) rendering instead of V3D (which is hardware) rendering.

[raysan5]

@kernelkinetic I think expected behaviour should be the same as on PLATFORM_DESKTOP. When setting a CAMERA_FIRST_PERSON mode, DisableCursor() is called, that internally calls:

glfwSetInputMode(CORE.Window.handle, GLFW_CURSOR, GLFW_CURSOR_DISABLED);

The documentation for glfwSetInputMode() states that GLFW_CURSOR_DISABLED hides the cursor and provides virtual and unlimited cursor movement, probably reporting relative movement.

PLATFORM_RPI and PLATFORM_DRM do not implement that functionality at this moment...

[chriscamacho]

I don't have time to get to this at the moment, but i've done stuff with /dev/input/mouse0 years back and its all more than doable!

[kernelkinetic]

I'll take a look into that.

[kernelkinetic]

In GLFW the mouse cursor handling is different if the cursor is hidden. When it's not hidden, it's limited to the screen/window size, if it's hidden it is not limited. I've changed the current mouse cursor handling for PLATFORM_DRM and PLATFROM_RPI to work the same way.

[GitSparTV]

If the issue is fixed on your or raysan repo, tell me, I can test it

[kernelkinetic]

It's in https://github.com/kernelkinetic/raylib/tree/fix_1392 and I've already sent a PR to raysan5. Just two lines changed...

[GitSparTV]

Ok I built PLATFORM_DRM on RPI4. Mouse movements are fixed. However keyboard movements are still shuttering

[kernelkinetic]

@GitSparTV Absolutely, I only fixed the mouse for now. But during this I also saw the keyboard stuttering I didn't noticed before. So step by step, I'll look into that, too 😀

[GitSparTV]

Yeah it's a shuttering and also a different keyboard trapping.
On PC if you hold a key the player moves constantly and smoothly.
On RPI when I hold a key player moves once, then after a second it continue moving.

[GitSparTV]

Do I need to create a new issue about keyboard input and camera moving?

[kernelkinetic]

@GitSparTV Currently not, at least not for me.

[raysan5]

@GitSparTV as @kernelkinetic is already reviewing it (or intents to), we can wait before opening the issue.

[kernelkinetic]

@GitSparTV I've fixed the keyboard stuttering in https://github.com/kernelkinetic/raylib/tree/fix_1392. If you are so kind to test and report here then I'll prepare a push request if it works for you too.

[GitSparTV]

@kernelkinetic Didn't work

[kernelkinetic]

@GitSparTV It worked for me obviously. Okay, I'll look again but to be honest if I cannot reproduce it again it's hard to find it...

[GitSparTV]

I'm using apple keyboard, I will test with a "normal" one right now

[GitSparTV]

it helped, little bit, it just a different shuttering pattern

[GitSparTV]

I will record the video

[raysan5]

@kernelkinetic @GitSparTV actually, I remember working on this issue in the past with no success... but it was some years ago and I can't remember why I didn't fix it...

[kernelkinetic]

@raysan5 That's not a reason to not try it again 😀

[GitSparTV]

https://imgur.com/a/OtZG1sU
With different keyboard and the fix_1392 2 commits

[kernelkinetic]

@GitSparTV Thanks for the video. Your Raspberry Pi indicates low voltage with the ⚡ in the top right corner. It's also possible that this is the reason for the keyboard issue. Please check your power supply.

[GitSparTV]

Maybe, I can't find an adapter with sufficient power nearby.
Here is the latest log.

INFO: Initializing raylib 3.1-dev
INFO: DISPLAY: no graphic card set, trying card1
INFO: DRM: choosen mode 800x600 (800x600p@60)
INFO: DISPLAY: Upscaling required: Screen size (800x450) smaller than display size (800x600)
INFO: DISPLAY: Device initialized successfully
INFO: > Display size: 800 x 600
INFO: > Render size: 800 x 600
INFO: > Screen size: 800 x 450
INFO: > Viewport offsets: 0, 150
INFO: GL: OpenGL device information:
INFO: > Vendor: Broadcom
INFO: > Renderer: V3D 4.2
INFO: > Version: OpenGL ES 3.1 Mesa 19.3.2
INFO: > GLSL: OpenGL ES GLSL ES 3.10
INFO: GL: Supported extensions count: 76
INFO: GL: VAO extension detected, VAO functions initialized successfully
INFO: GL: NPOT textures extension detected, full NPOT textures supported
INFO: GL: ETC1 compressed textures supported
INFO: TEXTURE: [ID 1] Texture created successfully (1x1 - 1 mipmaps)
INFO: TEXTURE: [ID 1] Default texture loaded successfully
INFO: SHADER: [ID 1] Compiled successfully
INFO: SHADER: [ID 2] Compiled successfully
INFO: SHADER: [ID 3] Program loaded successfully
INFO: SHADER: [ID 3] Default shader loaded successfully
INFO: RLGL: Internal vertex buffers initialized successfully in RAM (CPU)
INFO: RLGL: Render batch vertex buffers loaded successfully
INFO: RLGL: Default state initialized successfully
INFO: TEXTURE: [ID 2] Texture created successfully (128x128 - 1 mipmaps)
INFO: FONT: Default font loaded successfully
INFO: RPI: Opening input device: /dev/input/event1 (mouse )
INFO: RPI: Opening input device: /dev/input/event0 (keyboard )
WARNING: RPI: Failed to open Gamepad device, no gamepad available
INFO: FILEIO: [resources/cubicmap.png] File loaded successfully
INFO: IMAGE: [resources/cubicmap.png] Data loaded successfully (32x16)
INFO: TEXTURE: [ID 3] Texture created successfully (32x16 - 1 mipmaps)
INFO: VAO: [ID 2] Mesh uploaded successfully to VRAM (GPU)
INFO: FILEIO: [resources/cubicmap_atlas.png] File loaded successfully
INFO: IMAGE: [resources/cubicmap_atlas.png] Data loaded successfully (256x256)
INFO: TEXTURE: [ID 4] Texture created successfully (256x256 - 1 mipmaps)
INFO: TIMER: Target time per frame: 16.667 milliseconds
INFO: TEXTURE: [ID 3] Unloaded texture data from VRAM (GPU)
INFO: TEXTURE: [ID 4] Unloaded texture data from VRAM (GPU)
INFO: VAO: [ID 2] Unloaded vertex data from VRAM (GPU)
INFO: MODEL: Unloaded model from RAM and VRAM
INFO: TEXTURE: [ID 2] Unloaded texture data from VRAM (GPU)
INFO: TEXTURE: [ID 1] Unloaded default texture data from VRAM (GPU)
INFO: Window closed successfully

[GitSparTV]

I think you can test it by plugging RPi into low-power usb.
Like PC one. Test low-voltage as well

[kernelkinetic]

@GitSparTV No, you should in general use a specified power source if you want a working setup 😀 But it is good idea to try to reproduce this error, I'll check that 👍

[chriscamacho]

@GitSparTV looks like its jittering up and down, can you mod the code to mode forward with the right mouse button held down, just maybe its not the keyboard....

[kernelkinetic]

@GitSparTV I've checked two different power plugs, one causes a reset when I start the maze example, the second leads to some sporadic stuttering like in your video. Using the official power supply works as expected.

[GitSparTV]

Alright, then we know for sure it's due to low voltage

[kernelkinetic]

@GitSparTV If you found a suitable power supply and it doesn't work please open an issue again 👍

[chriscamacho]

@GitSparTV top tip get an old laptop psu and search for an automotive DC-DC module (there are a variety of input and output voltages find one that matches your sacrificial laptop PSU and outputs 5V) they often deliver in the ball park of 10A if required....

I wouldn't recommend powering from gpio instead get a USB plug shell and solder that up, you can then use it with a USB-C adaptor...

that way hopefully you can just forget about further power issues!

[masterneme]

Hi, I want to say that I experience the same issues as @GitSparTV.

I tested the examples some months ago with a RPi 3A+ and had that flickering.
Last week I got a RPi 0W and it still happens.

I just tried again with just the Zero, a keyboard and the official power supply and it stutters, so I don't think is a voltage issue.

It also happens with other examples. I noticed the stuttering kinda mimmicks the input speed of a keyboard when you write on the console or with a word processor, do you know what I mean?

What if Raylib isn't taking full control of the input so it's just processing what comes given by Linux?

Thanks for fixing the mouse.

[kernelkinetic]

@masterneme Did you also try https://github.com/kernelkinetic/raylib/tree/fix_1392?

[raysan5]

@kernelkinetic feel free to send a PR with the fix if required.

[kernelkinetic]

@raysan5 Hm, I'm not sure because until now no one confirmed that it work's better than before 🤔

[GitSparTV]

@masterneme

[masterneme]

@masterneme Did you also try https://github.com/kernelkinetic/raylib/tree/fix_1392?

I just tested it with a Zero W using PLATFORM_RPI and unfortunately neither the mouse nor the keyboard work. It behaves like in @GitSparTV's video.

I will try now with a RPi 3A+...

[kernelkinetic]

@masterneme Thanks for testing, good to know! So I can try to reproduce and analyse it on a Raspberry Pi Zero W 👍

[masterneme]

I got the same result with the A+.

[GitSparTV]

I think the issue may be reopen

[kernelkinetic]

I also checked Raspberry Pi Zero W and moving around in the models_first_person_maze example is horrible and not useable in any way 😀 So I can analyze this now 👍

[kernelkinetic]

Is there a reason that there is absolutely no multithreading access control code for the data that EventThread accesses? I'm not finished with analyzing but that _could_ be a problem anyways...

[raysan5]

@kernelkinetic that code was user contributed, I reviewed it and it looked ok but it can probably be improved.

[chriscamacho]

@kernelkinetic I can see potential problems...!

for example if SetMousePosition is setting CORE.Input.Mouse.position and the event thread then stamps on it...

[kernelkinetic]

Thanks @raysan5.
Yes, me too on many other lines, @chriscamacho. I think we should try to stay away from threading because raylib is an immediate mode library and could handle these repeatable tasks well in it's main loop. I propose to add the necessary code to or call it from ProcessKeyboard(). Any doubts?

[kernelkinetic]

I found out that this _is_ the reason for the stuttering. The main thread clears and checks CORE.Input.Keyboard.currentKeyState while the thread must set it during this time. That works most of time on a PC that is really fast compared to a Raspberry Pi. There most of the keys set from the thread are lost in the time period between checking and clearing. So it heavily depends on the speed of the machine how much key events arrive in the main thread.
So we have to options here:

• Protecting all variables that the thread accesses at absolutely every location in the whole code.

• Staying in a single thread as I proposed in https://github.com/raysan5/raylib/issues/1392#issuecomment-716117643.

[raysan5]

@kernelkinetic Probably a separate issue should be open to track it.

About inputs reading on a separate thread, long time ago, raylib did all the inputs reading on main thread but there were problems with mouse resolution, so, it was moved to separate threads. We can keep keyboard reading on main thread but mouse should be done in a separate thread...

[kernelkinetic]

@raysan5 No problem, then I'll keep the separate thread and add the access protection code.

[kernelkinetic]

https://github.com/kernelkinetic/raylib/tree/fix_1392 contains a fix that removed the keyboard stuttering. Maybe @GitSparTV and @masterneme will be so kind to test it 😉

Multithreading data protection isn't implemented because it's a really big task and needs some discussions first. But anyways, this should at least fix the stuttering issue for now.

[GitSparTV]

Tested.
Mouse: fixed
Screen shuttering: fixed

New problem, if you hold a key and then press a second key, the first key is marked as released. Holding W, then pressing A. No forward moving
@kernelkinetic

[masterneme]

Tested with the 3A+, the mouse is fixed but the keyboard still stutters. And I confirm what @GitSparTV said about pressing multiple keys at the same time, diagonals are not posible.

I tried the Animation and YawPitchRoll examples too and the behavior is like I described in my first post, as if Raylib couldn't take over processing the keyboard inputs, mimicking the same rythm you see when you press a key on the console or a word processor.

I'm going to do a couple more tests.

[kernelkinetic]

Things getting strange on my Zero 😀 If I start model_first_person_maze example by via ssh connection and control it via a locally connected keyboard then it works as expected, two simultanoues key presses leads to a diagonal move and releasing one of the keys moves again in the right direction. But if I start locally using the connected keyboard, it shows the key errors you currently report. Stuttering isn't there in both cases. I'll also try on a Raspberry Pi 4 now.

[GitSparTV]

I can't get diagnoal movement. Only one direction

[kernelkinetic]

I've checked raysans raylib also and it seems that it has the same problems, so they are not introduced by my changes but are now visible because it's no more stuttering or at least less than before. So maybe I should PR this changes and we should open a new issue about this. I've also a an idea what's going wrong: As far is I've seen raylib grabs the key on two different ways to get local and ssh inputs. Maybe that doesn't work correctly and so raylib gets local inputs twice, which leads to the interrupting behaviour. It's definitely working if it's started via ssh.

[masterneme]

I tried the 3.0.0 release and the inability to do diagonals is there too.

Edit: I'm turning my Zero on to see if that works.

[GitSparTV]

I tried to play on raylib site and it was working fine

[masterneme]

My Zero acts exactly like the 3A+, mouse fixed but no diagonals and stuttering.

Edit: (Off Topic) My Escape key stopped working, is there a quick way to make Raylib close using F1 or F2?

[kernelkinetic]

I've discovered that diagonal walking is working when using the cmake build system and not working when using make. Can anyone confirm or dispute, please?

[masterneme]

I'd like to try but I have no idea how to use cmake 😳

[kernelkinetic]

@masterneme It's not that hard: cd raylib && mkdir build && cd build && cmake .. -D PLATFORM=DRM && cmake --build . should do it already. If not I can probably help you via discord.

[GitSparTV]

Tested with cmake. It's working there, lol

[kernelkinetic]

@GitSparTV Thanks for testing, I'm glad to see that I'm not totally dumb 😀 Hopefully I find the differences to get this working with makefiles too!

[masterneme]

OK, and what's the equivalent of PLATFORM_RPI, PLATFORM=RPI?

[kernelkinetic]

@masterneme Exactly.

[raysan5]

In the Makefile is PLATFORM=PLATFORM_RPI, not sure about CMake.

[kernelkinetic]

The cmake build includes a cmake/config.h with additional settings like

#define SUPPORT_SSH_KEYBOARD_RPI    1
#define SUPPORT_HALFBUSY_WAIT_LOOP

and others.
Sadly I don't know how to integrate these differences properly. Anyone can help?

[masterneme]

After some tinkering, CMake says that the correct values for PLATFORM= are Desktop;Web;Android;Raspberry Pi;DRM and if I just write cmake .. && cmake --build . I can see on the console that it uses PLATFORM=PLATFORM_DESKTOP with GRAPHICS=GRAPHICS_API_OPENGL_33

I'll do more experiments.

[masterneme]

OK I confirm that the flags for CMake are:

PLATFORM=Desktop
or
PLATFORM=Web
or
PLATFORM=Android
or
PLATFORM="Raspberry Pi"
or
PLATFORM=DRM

Edit: I can't manage to compile the examples, any help?

[GitSparTV]

If you have an error remove build folder entirely and make sure you called --build not build

[kernelkinetic]

@masterneme When you successfully ran cmake --build . you'll find the examples in your build directory in the subdirectory examples.

[masterneme]

Success!

I ran make PLATFORM=PLATFORM_RPI inside the examples folder to build the executables.

I confirm that everything works as expected, the mouse, diagonals and no stuttering.

I tried with core_3d_camera_first_person, models_animation, models_first_person_maze, models_yaw_pitch_roll and they move smooth as butter.

All this with the 3A+, I'll try with the Zero now.

[raysan5]

@kernelkinetic @masterneme great news! feel free to send a PR when ready! It will hopefully be included for raylib 3.5. :D

[kernelkinetic]

@masterneme Thanks for testing!
@raysan5 It's working for the CMake part only and we should fix the makefiles, too. But I can't do that, so I'll send a PR soon.

[masterneme]

It doesn't let me compile with the Zero, something about not finding X11, strange. I think I'll just let it go as everything works with the 3A+.