Oni: Benchmark typing latency

[bryphe]

This article: https://pavelfatin.com/typing-with-pleasure/ has a really great and in-depth investigation into the typing latency of various editors. This latency - the time from pressing the key to seeing the result on the screen - is incredibly important especially with developers. This latency is a huge part in whether an editor is a joy to use or if it feels slow.

For this work, we should look at the following:

• What are the current measurements of typing latency?
• What can we do to improve?
• Can we add metrics so that build-over-build we can see how the latency changes?

[Bretley]

As per the author's own software package (OS X)

[Bretley]

To Improve, I think a simple step would be to see if we can optimize the logic in the editor. The last keys to get pressed are the ones that aren't \

I know it's probably not a huge difference, but every tick counts

[bryphe]

Awesome, thanks for doing this profile, @bert88sta ! It unfortunately is a bit high, but its sort of what I expected today. I think some good goals for this would be to get the min < 10ms and the avg around 15 ms - those might be aggressive goals though.

A bunch of stuff happens when we press a key:
1) Keydown event gets triggered, and handled in Keyboard.ts
2) Keyboard converts to VIM-style key press and emits another keydown event
3) Key press is sent to the NeovimInstance via the msgpack-RPC API
4) Neovim processes the command, and then lets us know the updated state via an action
5) The screen listens to the action, and updates state
6) The renderer picks up the updated screen state, and draws on the screen

So there is actually a lot happening between pressing the key, and seeing it on the screen. When tackling performance improvements, it's important to know where the bottleneck is - so that we are focusing on the right place, and also so that we can measure the improvement.

One strategy we could do is add some console.logs in each of these places, like:
console.log(Keydown handler: ${performance.now()})
..
console.log(Renderer processing action: ${performance.now()}
etc

And then we can take a look at the console. The steps I'm suspecting are the round-trip between us to Neovim - steps 3 and 4, but I could be wrong, as the delay could be anywhere in that path above.

If it turns out that the round-trip between Neovim and our UI is the culprit, one thing we could do (at least in insert mode), is to apply optimistic updates - we start showing key presses on the screen before we get confirmation from Neovim, and then once Neovim sends us the updated state, we reconcile it. This actually is not too different from how multiplayer games work:
https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking

It's just that, in this case, our 'client' is our Electron UI, and the 'server' is NeoVim, and instead of latency being due to network, it's due to the latency in the msgpack-rpc API.

[jordwalke]

One idea I had (which is way beyond the scope of Oni/NeoVim) is a protocol for an editing engine like Vim, to be able to tell the client, in many situations, exactly what should happen in each of the possible key combinations that could be pressed - before the key is pressed. So at any given point in time, you know immediately how to update the screen depending on various key combinations. This wouldn't work everywhere obviously (like if you entered a command :doSomeSideEffect() - Vi couldn't tell you what to show on the screen if you were to hit <cr>, but I do believe it could work for the core editing experience. It is a fun challenge to try to relay this information efficiently, choosing to precompute and compress the subset of possibilities that are most likely to occur next for the given typing mode. There would also be some tolerance, where in some rare cases you let the optimistic update to be wrong (and ideally the user could enforce a high integrity mode which forbids this at the expense of latency).

[keforbes]

If I bring up the Developer Tools in Oni, I've noticed that with every keypress it dumps a bunch of neovim requests:

[PERFORMANCE] neovim.request.nvim_get_current_win.start: 7581.12
[PERFORMANCE] neovim.request.nvim_get_current_win.end: 7586.42
[PERFORMANCE] neovim.request.nvim_win_get_position.start: 7587.325000000001
[PERFORMANCE] neovim.request.nvim_win_get_width.start: 7587.905000000001
[PERFORMANCE] neovim.request.nvim_win_get_height.start: 7588.360000000001
[PERFORMANCE] neovim.request.nvim_win_get_position.end: 7590.535000000001
[PERFORMANCE] neovim.request.nvim_win_get_width.end: 7591.0250000000015
[PERFORMANCE] neovim.request.nvim_win_get_height.end: 7591.360000000001

You can see it took 10ms to complete all of these requests, which are repeated with every keypress.

If I comment out this one event handler (in index.tsx), all of those neovim requests go away with no adverse behavior that I can see.

    instance.on("window-display-update", (eventContext: Oni.EventContext, lineMapping: any) => {
        overlayManager.notifyWindowDimensionsChanged(eventContext, lineMapping)
    })

I tried navigating around a file and re-sizing the Oni window and everything still works (or at least, behaves the same). Now, there's probably some scenario where this window-display-update event is useful but I just wanted to flag it as being the cause of a flurry of activity that seems unnecessary to my ignorant mind.

[bryphe]

Cool idea, @jordwalke - having an "oracle" like you describe would make it really easy to apply optimistic updates. One crazy idea I had was to try and compile Neovim to WebAssembly and get it to run directly in electron (or have it built as a native node module) to reduce the interop time. But the engine you describe is interesting.

Thanks for checking those traces out, @keforbes ! I believe those are used for all the overlay UI. That's stuff like the error squiggles, live evaluation markers, and scrollbar. You can test if things are still okay after that by creating a split and getting some error messages, and switching between buffers. I don't think those calls are blocking (since all those calls are async) - but that 10ms appears to be the roundtrip latency between Oni <-> Neovim for most of the commands. It seems like for processes talking locally to each other we should be able to reduce that roundtrip time, which would have benefits across the board. The next step there would be to identify the bottleneck - is it something on our end when we are serializing the message to go to Neovim via msgpack-rpc? One reductionist approach would be to write a very minimal client - the minimal needed to talk to neovim - and benchmark how quickly that roundtrip is. Then we start adding pieces in until we find the bottleneck. But I don't think that particular call is impacting typing latency (although we'd have to test to be sure)