Imgui: Significant rendering lag when using Vulkan

[rytone]

Version/Branch of Dear ImGui:

Version: 1.73, 1.75 WIP
Branch: master
Back-end/Renderer/Compiler/OS

Back-ends: Yatekii/imgui-wgpu-rs, imgui_impl_opengl3.cpp, imgui_impl_vulkan.cpp
Operating System: Arch Linux, awesomewm

My Issue/Question:

In my own application using imgui-wgpu-rs, I've noticed that there is a significant input lag for me on Linux, and wgpu uses Vulkan, so I tested further and I've been able to reproduce the issue with the OpenGL and Vulkan examples that are here. Inputs, particularly things like moving windows, are noticeably delayed when using Vulkan compared to with OpenGL.

Screenshots/Video

(left is Vulkan, right is OpenGL)

Standalone, minimal, complete and verifiable example:

examples/example_sdl_opengl3

examples/example_sdl_vulkan

[ocornut]

What does looking at the example
vulkan renderer tells you? As you can imagine, if you were to draw a simple triangle following the mouse you would get the same effect, so this is not really an issue affecting core imgui.

However as many people are relying on the example app/backends we should aim at reducing those issues in them. Would need to investigate the Vulkan example, any help appreciated there!

[rytone]

I get that some lag is inevitable, and I can see that when using OpenGL, but its much more significant when using Vulkan which surprises me. Given that this issue exists in both the example backend and imgui-wgpu-rs, I agree that it's probably still an issue outside the scope of imgui itself, but I haven't been able to find anything regarding high input latency particularly with Vulkan. Maybe I haven't been searching the right thing, though.

[ocornut]

If you find a solution it would be great if you post about it, but I'm afraid we can't really help here.

Note that if you vsync you'll always get some form of lag between a hardware mouse cursor and normal GPU render path. Try to enable io.MouseDrawCursor = true your cursor will be drawing using regular GPU path and you might be able to compare GL and Vulkan better, but the mouse cursor will feel slower to the user. A possible solution to investigate would be to switch a normal GPU render path cursor when dragging and revert to a HW cursor when not.

[mocabe]

Not sure it's related to your problem, but I remember some of official Vulkan backends using FIFO presentation mode, which introduces much more input lag compared to Mailbox mode. I noticed FIFO introduces too much input lag so switched to Mailbox with FPS limit while implementing my own Vulkan backend.

[rytone]

Yeah, that's pretty much the solution I ended up on too. I'll leave this issue open for now if someone comes up with any other solution though.

[ocornut]

The comments above suggest changing the main.cpp of our example from:

    VkPresentModeKHR present_modes[] = { VK_PRESENT_MODE_FIFO_KHR };

Into

    VkPresentModeKHR present_modes[] = { VK_PRESENT_MODE_MAILBOX_KHR, VK_PRESENT_MODE_FIFO_KHR };

However I cannot get Mailbox to behave as expected here :(
Keeping up with Vulkan oddities has been quite tedious.

On my laptop with the Intel graphics card, only VK_PRESENT_MODE_FIFO_KHR is supported and even though the documentation says it should be waiting for vertical blank, I'm getting 250+ fps on a 60 Hz screen.

On same laptop with Nvidia with VK_PRESENT_MODE_FIFO_KHR I get a stable 60 FPS, and with VK_PRESENT_MODE_MAILBOX_KHR I get 2000+ FPS. VK_PRESENT_MODE_MAILBOX_KHR is ALSO described as waiting for vsync.

[mocabe]

@ocornut As vulkan spec says, VK_PRESENT_MODE_MAILBOX_KHR does wait vertical sync to present image on screen, but all images to present are internally queued, which means it does not block renderer to send image to the queue. In other words, it works similarly to Nvidia's FastSync. If you use VK_PRESENT_MODE_MAILBOX_KHR you need to implement frame rate limiter.

[ocornut]

Would you be able to sugget minimum code in the Vulkan examples to use VK_PRESENT_MODE_MAILBOX_KHR with a frame rate limiter?

[mocabe]

Maybe the simplest solution is just wait small amount of time after sending commands like this (from my own backend implementation):

    auto endTime         = std::chrono::high_resolution_clock::now();
    auto frameTime       = (endTime - m_pimpl->lastTime);
    auto frameTimeWindow = std::chrono::nanoseconds(1000000000) / m_pimpl->fps;
    auto sleepTime       = frameTimeWindow - frameTime;

    if (sleepTime.count() > 0) {
      m_pimpl->lastTime = endTime + sleepTime;
      std::this_thread::sleep_for(sleepTime);
    } else {
      m_pimpl->lastTime = endTime;
    }

I'm not Vulkan expert so someone can propose better solution though.

[danger89]

Maybe the simplest solution is just wait small amount of time after sending commands like this (from my own backend implementation):

No this is never the right solution. Never introduce sleeps/waits in your production code people. I think Imgui and GLFW/Vulkan really have some synchronisation issues here. I already tried to disable double buffering (glfwWindowHint(GLFW_DOUBLEBUFFER, GLFW_FALSE)) without any luck. And using glfwWaitEvents() instead of polling, without luck.

Ps. I'm using Imgui for a GUI application, not a game. This issue is definitely related to: https://github.com/ocornut/imgui/issues/1805

[rokups]

I thought this is an inherent flaw of Linux graphics stack. Windows and MacOS do not exhibit this behavior, and it is somewhat amortized on Linux if 144hz monitor is used. Experience is absolutely terrible with 60hz monitor however.

[danger89]

Well that could be the case, but imgui is the only interface under Linux that lagg so much. Applications like Firefox, who are also using hardware acceleration doesn't suffer from all this under Linux...

[rokups]

I em experiencing this in all applications. When refresh rate is set to 60hz, even dragging a desktop window produces a visible lag between position cursor grabbed on the window and actual mouse cursor position. Same is true for UI in other 3D applications. Likewise it gets much better with higher refresh rate.

[rytone]

Totally forgot about this issue... I'm glad to say that I have found a real solution! The problem comes from blocking on acquiring the next swapchain image, losing window events during that period. In my case using wgpu-rs, I moved swapchain image acquisition to another thread, notifying the main window event loop when the next image is ready such that window events are processed as soon as possible, and the input latency is identical to when using OpenGL. I'm not sure why this issue only occurs under Linux (and maybe even only X11), but I'm glad I was able to find a solution.

[danger89]

@rytone that is great to hear! Could you maybe add this fix to the Imgui glfw +opengl and/or glfw+vulkan example? That would be great for everybody!