Godot: Websocket too much latency for outgoing packets.

Created on 22 Nov 2018  路  9Comments  路  Source: godotengine/godot

Godot version:
v3.1.dev.custom_build.c00edd9

OS/device including version:
Windows 7 Ultimate x64

Issue description:
Using WebSocketClient and WebSocketPeer.
If we send a packet A to the server (localhost) and the server immediately responds with another packet B. The time between the sending of packet A and the reception of packet B is of the order of tens of milliseconds. Around 30-50 ms in the tests that I have done. The expected time should be a few milliseconds, not tens.

I think this should be classified as a bug because such a large latency does not seem to be normal.

Steps to reproduce:
test_websocket.zip

discussion network

Most helpful comment

There are multiple reasons for this latency (on localhost too):

  1. The way Godot poll the network connection
  2. The way the WebSocket wrapper for the library and the library itself work.

Specifically:

  1. (and this is also relevant for ENet) when using the high level multiplayer API, the connection is polled (by default) at each frame. That means that, even with no network latency, you have an average 8 ms delay at 60 fps. I don't think there's much we can do here but I'm open to suggestions (this is basically why, in many games, only bots pings 0, a guy connected to the LAN still tend to ping at least 6/7, and while some games do report lower pings, I tend to believe that when the server is implemented using the same engine as the client in most cases they are "cheating", i.e. subtracting the network delta frame from the the reported latency).

  2. By the way the (WebSocket) module/library works it needs to buffer both incoming and outgoing packets and those packets will be transmitted during poll(), which will happen at the next frame (it's basically like a send_deferred :( ). I've been trying to figure out a solution for this, but it likely won't be in 3.1.
    We could hack-in an extra flush_packets to the NetworkedMultiplayerPeer, but that will not work in every configuration (notably, when disabling automatic polling).

We could also add extra servicing just for websocket after each send, but that would make for an awful design, and would make it impossible to have a fixed network step.

I've been trying to convince myself of the need of a NetworkingServer (like VisualServer) for handling network connections properly at fixed steps (at least for the multiplayer part). I've mentioned this more than once, but I still have to come up with a good design and a proper collection of use cases to propose to the other devs (again 3.2+).

With vsync enabled, average latency does not stop growing steadily when more than 60 packets are sent per second. Is it a bug? or has it been designed that way?

How much data are you sending in those 60 packets?
WebSocket uses TCP, so If the network connection can't send that much data at a given step, the leftover will be queued and sent at the next step (plus buffers will start to fill, till they get full, and then packets will be dropped, and you might get disconnected).

All 9 comments

The client's frame rate can have an impact on the measured/effective latency (this is not specific to Godot, I've seen this in many other games). Try disabling V-Sync to get as much FPS as possible, then measure again. (I'll try the example project when I get back home.)

CC @godotengine/network

The client's frame rate can have an impact on the measured/effective latency (this is not specific to Godot, I've seen this in many other games). Try disabling V-Sync to get as much FPS as possible, then measure again. (I'll try the example project when I get back home.)

You're right. By disabling V-Sync the latency drops to 2 or 3 ms.

But this solution creates another problem. I do not want to draw 1000 frames per second. And if I disable vsync this is just what happens.

I wonder if a solution to this would be to keep vsync enabled and call WebSocketClient.poll a few times on every call to _process.

I wonder if a solution to this would be to keep vsync enabled and call WebSocketClient.poll a few times on every call to _process.

Probably not since process will still be called just at the framerate of the application.

Since you're measuring the time from _process this delay is in an expected range in my opinion.
To get better latencies you'd have to run the packet handling logic at higher rates than the framerate (which is probably what other engines do, aswell), at least for the server.

Probably not since process will still be called just at the framerate of the application.

Since you're measuring the time from _process this delay is in an expected range in my opinion.
To get better latencies you'd have to run the packet handling logic at higher rates than the framerate (which is probably what other engines do, aswell), at least for the sever.

I've tried it and it reduces the average latency a lot. There are some peaks but in general it works. It helps a lot if _process takes a significant amount of the frame time and calls to poll can be distributed in that time.

Update: I have been doing more tests and it seems that the latency does not improve that much. It seems to me that the only option to obtain a better latency is deactivating vsync.

With vsync enabled, average latency does not stop growing steadily when more than 60 packets are sent per second. Is it a bug? or has it been designed that way?

There are multiple reasons for this latency (on localhost too):

  1. The way Godot poll the network connection
  2. The way the WebSocket wrapper for the library and the library itself work.

Specifically:

  1. (and this is also relevant for ENet) when using the high level multiplayer API, the connection is polled (by default) at each frame. That means that, even with no network latency, you have an average 8 ms delay at 60 fps. I don't think there's much we can do here but I'm open to suggestions (this is basically why, in many games, only bots pings 0, a guy connected to the LAN still tend to ping at least 6/7, and while some games do report lower pings, I tend to believe that when the server is implemented using the same engine as the client in most cases they are "cheating", i.e. subtracting the network delta frame from the the reported latency).

  2. By the way the (WebSocket) module/library works it needs to buffer both incoming and outgoing packets and those packets will be transmitted during poll(), which will happen at the next frame (it's basically like a send_deferred :( ). I've been trying to figure out a solution for this, but it likely won't be in 3.1.
    We could hack-in an extra flush_packets to the NetworkedMultiplayerPeer, but that will not work in every configuration (notably, when disabling automatic polling).

We could also add extra servicing just for websocket after each send, but that would make for an awful design, and would make it impossible to have a fixed network step.

I've been trying to convince myself of the need of a NetworkingServer (like VisualServer) for handling network connections properly at fixed steps (at least for the multiplayer part). I've mentioned this more than once, but I still have to come up with a good design and a proper collection of use cases to propose to the other devs (again 3.2+).

With vsync enabled, average latency does not stop growing steadily when more than 60 packets are sent per second. Is it a bug? or has it been designed that way?

How much data are you sending in those 60 packets?
WebSocket uses TCP, so If the network connection can't send that much data at a given step, the leftover will be queued and sent at the next step (plus buffers will start to fill, till they get full, and then packets will be dropped, and you might get disconnected).

How much data are you sending in those 60 packets?
WebSocket uses TCP, so If the network connection can't send that much data at a given step, the leftover will be queued and sent at the next step (plus buffers will start to fill, till they get full, and then packets will be dropped, and you might get disconnected).

I am sending a few bytes (not more than 8) per packet.

I am sending a few bytes (not more than 8) per packet.

Then it seems to be a bug, I'll investigate further.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

EdwardAngeles picture EdwardAngeles  路  3Comments

rgrams picture rgrams  路  3Comments

SleepProgger picture SleepProgger  路  3Comments

Spooner picture Spooner  路  3Comments

testman42 picture testman42  路  3Comments