Aurora: Mouse Input Lag/Visible Effects Lag after keyboard change

[InvertTails]

What are you experiencing an issue with?

• [x] Aurora
• [ ] Aurora supported game, (Fill in the game)
• [ ] Aurora Wrapper
• [ ] Aurora Profile
• [ ] Aurora supported device, (Fill in problematic devices)
• [ ] Aurora Scripts
• [ ] Aurora Updater
• [ ] Aurora Lighting issue
• [ ] Other, (Fill in another issue here)

What kind of an issue is it?

• [ ] Crash
• [x] Performance problems
• [ ] Visual bug
• [ ] Localization issue
• [ ] Enhancement suggestion
• [x] Device issue
• [ ] Other, (Fill in another type of issue here)

Expected behavior:

Lighting to work, no input lag.

Actual behavior:

What is happening is when I updated my keyboard setup (bought a Razer Ornata) and swapped keyboards, I am getting considerable input lag spikes with my G303 Mouse and the lightning effects also lag as they run on the keyboard. I noticed the lighting effect chop up greatly when achieving an ultimate on Overwatch. All Driver versions for my keyboard and mouse are to the latest spec but the spikes seem to still happen.

How often does this problem occur?

This problem occurs... Not just in games but in normal desktop use too. Aiming in FPS and first person perspective games gets chopped up every so often. Additionally, there is some CPU usage spikes from the program itself that happen rarely.

Please describe the steps to reproduce this issue:

The issue can be reproduced by doing following steps:

N/A

What is the version of Aurora you are experiencing this issue with?

My version is... v0.6.0b

Does the previous version of Aurora have this issue?

The previous version, both this and previous versions of Aurora did not do this before to where the input lag is noticeable.

Please include any logs, screenshots, or files that are related to this issue:

[simon-wh]

@InvertTails Could you try the latest development release to see if it makes a difference?

[VoronFX]

@simon-wh
Cause I am a gamer this problem is very important for me also. By this day I was using self-builded Aurora with completely disabled hooks to grant that it will not cause input lag.

We discussed it a bit with @antonpup. The root of the problem is global low level hooks. Microsoft designed them in such way that they are blocking input and causes so much problems in so many places (I think everybody knows computer responsiveness when League of Legends loads? it's not because you have weak pc, it's bad design. But it has workarounds, so riots are also guilty in not fixing this).

You've disabled mouse hooks and it seems that it solved the problem. But keyboard hooks are still remains. To minimize impact on input it will be best to write own hooks, as light as possible without using the whole 3d party lib MouseKeyHook. But the question is it worth?

To answer this I made some tests in Release mode, without debugger and under 100% cpu load.
The worst input lag I got was 48 ms and the average was 0 ms. Spikes over 5 ms are quiet rare and this is under 100% cpu, so probably I can consider this as good enough to leave it as it is.

I think the issue can be closed now.

[simon-wh]

@VoronFX The issue probably affects people differently depending on their CPU. I personally don't experience any input lag using an I7-4790K, but most people won't have something like that. I'm going to have a look at the MouseKeyHook to see if there is anything I can do with it to improve performance, but I think everything that could be done has been done in Aurora as a new thread is started to perform the actions based on the event so it shouldn't have that much of an affect anymore

[ghost]

w/o touching the MouseKeyHook itself or embed the hook inside Aurora, we can maybe not creating an Task inside the event. Instead it would be better if we have an queue that we fill up inside the event and have an Task/Thread that do the "real" action. Maybe this give some performance.
Would be good to check the performance for Task creation and the queue implemention.

If we implementing the hook by ourself, we wouldn't have the overhead of the mouse hook etc.

[VoronFX]

I think the overhead of MouseKeyHook abstractions and calls should be also avoided. The goal is to return from hook function as fast as possible. The way you proposed is exactly what I was thinking about.

Private queue for every hook to avoid using of syncronizations and thread/task/timer that reads data from those queues with some interval. So in hook there are will be only creating of new queue item and adding it to queue. Probably using simplest own written queue over standard one also will improve performance somewhat.

Other idea is to use circle buffer instead of queue, this can benefit in no need of new queue item allocation. As downside is a bit more memory usage, and possibilily of missing events if buffer will end faster then thread/task/timer will read from it. Also calculation of array index can neglect absence of allocation, so I think queue is still better. But this is subject to check.

All above is a sort of microptimizations, but this place is critical so I think it is relevant here.

I can do it on this week if you want.

[ghost]

Yep would be nice. At the end I can look over it, maybe I can get some µs out of it.
For a extra performance boost you also might consider unmanaged/unsafe C# code.

[VoronFX]

While searching for a good implementations of low level hooks I've found one article about messy windows API's that helped me discover another one API that should solve all our problems.

It's called RawInput and it's even more low level that low level hooks itself. It doesn't allow to intercept hooks and we don't need it. Instead it just posts message into message pump and goes on. That's mean no matter how long will we process those message it will not delay or impact the system or other applications.

So, no more need for micro optimizations and other non common solutions. Performance no longer will be the problem. Sounds like silver bullet, he, he =)

Although it's a bit more complex and less documented.
Now I will investigate how to handle it correctly and seamlessly replace low level hooks.

[brainbug89]

RawInput sounds like the right solution.

[ghost]

I also look into RawInput, trying to create an test application with it.
Edit: Also we can now setup the devices in Aurora by using the vendor id and product id. That's a nice side-effect. Would be nice for the first startup.

[InvertTails]

Just so I clear things up for my situation but I updated to the dev build and the performance issues have mostly fixed after updating.

[VoronFX]

@InvertTails You are right in "mostly fixed". Although current impact is not so significant the root of the problem is still there. Because we've found the solution that will fix this entirely and provide other benefits in addition, we have to take this to the end.

[VoronFX]

Anyone knows for what next fragment is intended for?

     //Handle special cases
     if ((e.KeyCode == Keys.VolumeUp || e.KeyCode == Keys.VolumeDown) && e.Modifiers == Keys.Alt)
           e.Handled = true;

What is that special case?
What we do here is blocking Alt + VolumeUp/VolumeDown for system and all apps.
Why do we need this and do we truly need this?
Anyway we can't suppress a key-press with RawInput but we can skip it for our app if that is needed.

This fragment was added in this commit by @antonpup in file Project-Aurora/InputEventsSubscriptions.cs.

[VoronFX]

Okay, I've found for what it was used for. It's for feature allowing to regulate keyboard brightness with Alt+Volume keys. Unfortunately for this feature to work we must use low level hooks. =(
I am thinking of using RawInput for general use and attaching low level hooks only if brightness feature is enabled. So high performance low level hooks implementation is still needed if want leave brightness feature.
One more headache is that we can't use RawInput and LLHooks in one process simultaneously.

[VoronFX]

Recent investigation update:
I've very well explored how all this stuff (communication with WinApi, interop callbacks, message pump... etc.) is working. I've really made personal level up in knowledge of windows architecture last several hours. Yay! =)

Now I see the problem entirely and how to solve it.

So how LLHooks works?
Windows is pushing message to thread's message pump (those one from which hook was installed). When thread get's CPU time it processes that message/executes callback.
So what is new here?
The time of processing callback is probably already so insignificant (less than 0.01 ms in worst cases) that there is no need for further micro-optimizations. Instead there is another factor that is causing issues: "How fast thread will get it's CPU time?".
As I can see there are only 2 possible origins of delay:

1. GC suspended the thread for garbage collection.
   Highly unlucky that this will make significant delay and almost nothing we can do here.
   Engaging LowLatency GC mode will increase memory usage and probably not worth it.
   Moving to unmanaged code also not worth it.

2. Concurrency with other threads.
   Here is the root and it can be fixed. We can run dedicated thread with message pump and high priority.
   As I said from the test I've made at the beginning there rare spikes of delay under high CPU load. It is proving that problem in thread just waiting for it's time to execute.

Now my plan is to leave using of MouseKeyHook with dedicated high-priority thread. Also I've made some work on RawInput and I want to make Aurora use LLHooks or RawInput depending on whether "brightness with volume key" feature is enabled.

PS: Sorry for making a blog from this thread, but I think I have to report my progress on this issue =)

[ghost]

Good to see, that somebody, other than me, understands the low-level architecture of Windows.
Do we really need the LLK-Hook atm? For me, Alt + VDown/VUp is doing nothing, so no reason for blocking it.