Godot: Server binary max fps steady at 123

[vicguedez]

Godot version:
3.1.1 April 27, 2019

OS/device including version:
Linux vps227684 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 GNU/Linux
Its a OVH VPS Cloud 1 with a 3GHz vCore

Issue description:
I got a project that is a Node with a Node2D as a child, both empty. A singleton that pools a NetworkedMultiplayerEnet on _process().

I uploaded the project to the VPS and ran it. Out of curiosity went to check the FPS using print(Engine.get_frames_per_second()) every 1 sec using a Timer and just got a steady 123 FPS.

Now, if I run the project on my very old and crappy laptop (Celeron 847 @ 1.10GHz) the project runs at ~200 and even spiking to 300 FPS. Tried setting the project to run at fullscreen and also disabling vsync. Also wrote this as to try to force all of it but still doesnt work:

OS.low_processor_usage_mode = false
OS.set_use_vsync(false)
ProjectSettings.set_setting("display/window/vsync/use_vsync", false)

So I'm thinking it has something to do with the server binary. I'm sorry I'm not of more use.

[girng]

Well, a vCore at 3GHz in the context of VPSs _could be_ a thread of a thread, not an actual dedicated core, so your dedicated Celeron 847 is much faster.

However, the real question is, a server's tick rate should not fluxuate like that, it should be fixed if I am not mistaken.

I just saw your using get_frames_per_second, this is not the tick rate I believe. I don't know as I never used the server binary, someone else might chime in

[LikeLakers2]

Hi there! First off, video settings won't do anything on the server platform. The server platform uses a dummy video driver, as it doesn't need to draw anything to the screen, therefore it won't be affected by settings like vsync.

That being said, for a server to run at a lower frame-rate (usually called "tick-rate" when it comes to servers) than the game clients it will be serving is normal. For example, Minecraft servers normally run at 20 FPS, whereas their client counterparts try to run at 60 FPS or higher. However, these are normally intentional -- Minecraft servers are coded to run at 20 FPS.

So, for the server to be reporting as running at such a lower frame-rate than your 9-year-old laptop is a bit curious. Barring any sort of project setting that you set for just the server platform, I could not find any reason why it would be running so much lower, as I don't believe the server binary caps the frame-rate normally, and the defaults for the project settings (including the target frame-rate). Not to mention that it's pretty unlikely that a VPS provider would be using super-old hardware (let alone hardware worse than a Celeron).

If you could, please zip up the project files and send them here. I would like to test if I'm capable of replicating this issue on my end.

[vicguedez]

@girng I think a thread of VPS should be faster than this laptop core.

@LikeLakers2 I know servers run at a lower tickrate. But for now I'm just pooling on every frame, I do plan on changing that of course.

And yeah I dont think that a vCore could be slower than my laptop. I will upload the source in the morning thanks.

[girng]

@girng I think a thread of VPS should be faster than this laptop core.

ok. I don't think you realize how VPSs are so badly oversold. The "vCore" isn't even a real core, it could be a thread of a thread (as I stated above).

Your thread on a VPS is most likely a lot slower than your real dedicated CPU.

[vicguedez]

@girng I know they are not real cores. But I have trained simple AIs on that vCore and its 10x faster than my laptop's core.

@LikeLakers2 Here is the project source: map_area_shard.zip. I hope it might lead to something useful :slightly_smiling_face:

[LikeLakers2]

So this is odd. Using the official 3.1.1 binaries (the server export and the windows editor), I get the same results as you do -- 122 FPS on the server, and around 3000 FPS on the client (I have a dedicated GPU). Using the nightly builds of the server binary and the editor, I get the same results, except now the server FPS is wavering between 140 and 141. More than likely, this difference in FPS between the two versions is probably down to optimizations that have been made in the 4000 or so commits since 3.1.1's release.

So this really confuses me. When I initially checked out the issue, I had noted OVH's somewhat poor review scores. However, my server is from DigitalOcean, which seems rather reputable from what I can tell... yet I was getting basically the same results. So I think this somewhat rules out OVH just being a bad host?

My only guess right now is that it has to do with the dummy video driver that the server platform uses. Unfortunately, as I don't have a Linux computer (I run Windows), I can't test how the dummy video driver would perform on my computer, to see if it is the dummy driver, or if it's something with the Linux builds entirely. Perhaps if someone else has a Linux computer, they could check that?

Also @vicguedez I noticed that you included left some info about your actual server in the res://server.gd file, as well as something about a token in res://singleton/utils.gd. I'm unsure if it'd be okay to leave those be, so you might want to get a new server IP and maybe revoke that token too, just-in-case.

[Calinou]

@LikeLakers2 The editor FPS going from ~125 to ~144 after upgrading from 3.1.1 is likely due to #28792.

[LikeLakers2]

@Calinou I think you misread. The server export FPS goes from 122 to 140, not the client export/editor FPS.

However, that does bring up a question. Does the server export default to using low-processor-usage mode in some way, even when low-processor-usage mode is forced off? Or perhaps something similar?

[vicguedez]

@LikeLakers2 Uhm the server token was a test token so no worries. The IP kinda worries me a bit, took down the link just in case xd. If any core dev/contributor needs it I can upload it again.

I also tried to disable low processor usage mode in case that was it, but it does nothing.

[LikeLakers2]

@vicguedez If the IP worries you, you might want to get the actual server IP changed. There's some people on this repo (like myself) who receive emails about issue comments, and those emails don't get edited when you edit your comment.

[Calinou]

However, that does bring up a question. Does the server export default to using low-processor-usage mode in some way, even when low-processor-usage mode is forced off? Or perhaps something similar?

Yes, that's what I'd suspect too :slightly_smiling_face:

Try lowering Application > Run > Low Processor Mode Sleep Usec to 1 in the Project Settings. The default value (6900) is tuned to limit the framerate to ~144 FPS when low processor usage mode is enabled (so that GUIs feel smooth on a 144 Hz display).

[LikeLakers2]

@Calinou Yup, that's exactly it. The server binary forces low-processor-usage mode. I changed it to 1 and it immediately jumped to >10000 FPS on the server export.

[vicguedez]

@LikeLakers2 Yep, mine is at 12/13k. Should I mark this as solved? Perhaps this should be on the docs.

[LikeLakers2]

@vicguedez I personally think it's a little unintuitive that it forces low-processor-usage mode, but I don't know if it'd be better to mark it on the docs, or if it should be changed so it doesn't force that mode, or if it even matters. I'll let Calinou decide.

(P.S. if it's decided to change it so it doesn't force low-processor-usage mode, I don't know what files to make a PR on, so this might sit for a while)

[vicguedez]

I'll let Calinou decide.

Same here, leaving it open. Thanks for the help @Calinou @LikeLakers2

[LikeLakers2]

No problem.

Oh, and just to make sure you see my reply to your concern over the server IP @vicguedez (since you never responded to this in some way):

If the IP worries you, you might want to get the actual server IP changed. There's some people on this repo (like myself) who receive emails about issue comments, and those emails don't get edited when you edit your comment.

I meant to add to that, the files you uploaded with the comment don't seem to be deleted from Github servers, when the link is removed from the comment. Hence why I brought this up. So maybe contact OVH support about that. Or, assuming there's nothing else that server is being used for, you could probably cancel the server and get a new one (perhaps in a different region), and that should get you a different IP.

[vicguedez]

@LikeLakers2 I know it doesn't get deleted from Github servers when removing it from the comment :slightly_smiling_face:. I already contacted OVH support about it. I had other things running on it sadly. Oh well, this are mistakes that you learn from :+1:

[LikeLakers2]

So I figured out how this is being caused, and it appears it isn't so much a bug as it is an unintuitive feature:

https://github.com/godotengine/godot/blob/1f40117caa6ad25340261969d377e1b398cd5349/main/main.cpp#L2041-L2042

This is the code that handles low-processor-usage-mode. Note the !OS::get_singleton()->can_draw().

I also noted that the code calling the draw functions (about 50 lines above the low-processor-usage-mode stuff) is wrapped in its own, separate can_draw() conditional:

https://github.com/godotengine/godot/blob/1f40117caa6ad25340261969d377e1b398cd5349/main/main.cpp#L1982-L1994

However, I am concerned that removing the can_draw() conditional from the low-processor-usage-mode conditional may cause CPU usage to rise somewhat even in simple scenes, as we could be running the main loop (including game logic code) at upwards of thousands of times a second or more; compared to now where we are only running it so many times per second, with a flat delay in-between each iteration due to low-processor-usage-mode.

To add to that: In some servers, allowing one program to pin the CPU in sufficiently complex scenes may be acceptable, so long as the server maintains a decent tickrate. However, in others (such as where a server is being used for multiple services), allowing Godot to pin the CPU may not be a good idea.

What do you think, @Calinou (or whoever else wants to give their opinion)? Do we remove the conditional? Do we leave it? Maybe we could do something else, such as adding a "target tickrate" setting, or limiting the server to the "target FPS" setting. Or perhaps I'm overthinking this a little, and it doesn't actually matter?

[KoBeWi]

Can anyone still reproduce this bug in Godot 3.2.3 or any later release?