Godot: Multi-threaded resource loading

[endragor]

Currently load() blocks the main thread, which doesn't allow to implement animated loading screens or streaming scenes. Chunked loading is not a solution, since you still freeze the main thread for the duration of the load of a chunk, which becomes noticeable if the chunk isn't tiny (e.g. a texture). So we need a way to dispatch loading to a thread in a separate thread pool and a way to check whether loading has been finished and block until it's finished.

OpenGL supports sharing resources between contexts from multiple threads for this and it works in GLES2. Vulkan doesn't really impose any restrictions on CPU multithreading, as I know, as long as you synchronize correctly.

[avencherus]

I tried multi-threading some loads back in Godot 2, and didn't notice anything different than normal loading, so I wasn't sure about it. Maybe it has been an issue for a long while and no one has tried using it for a large scale game situation yet?

It is something I want to test out again, because I would really need to do some background loading of resources strategically to make my games have less loading in-between areas and stages. I hope it's possible, or at least it may be something that can be fixed relatively soon.

[bojidar-bg]

There is a multithreaded loading example in the demos, using a thread. Also, in the Background Loading tutorial in the docs, there is a Using Multiple Threads section.

If load() blocks the main thread when invoked from a separate thread, then this is a bug.
Otherwise, if all what is wanted is a simple start_load_in_background function which would notify a loader thread that work is to be done, it is a feature proposal, though it might be handled by the asset library.

[endragor]

Ah, I wasn't aware load() works in a separate thread. That's nice. However, given that OpenGL context is not created in that thread, it seems transferring textures and other objects to GPU still happens on render thread and could be off-loaded from it, so there is still a place for improvement.

So let's consider this request to be about support for fully asynchronous load, without adding burden on the rendering thread, perhaps except synchronization (render thread still has to be notified that a texture has been loaded).

[bojidar-bg]

Just to make things clear, load does not work in a separate thread by default, but can be started from a separate thread, in which case it shouldn't block main.

[endragor]

I understand that. It can still block the render thread (which can be the same as main) due to render thread still having to perform calls to load the object into GPU, and that can be avoided. So currently separate thread helps to offload reading/parsing the data from disk into memory, but if it needs to be sent to GPU, render thread still has to do that.

[Zylann]

It can still block the render thread (which can be the same as main) due to render thread still having to perform calls to load the object into GPU

I would love to be able to not stall the render thread. I have a project where meshes are being generated frequently and I had no choice but to move the Godot Mesh creation part on the main thread with some timeout. @endragor did you try with multithreaded VisualServer mode? (although even if that doesnt block the main thread it might still block the render thread so no much gain)

[endragor]

Yes, it always blocks the render thread in any case, which can be noticeable to the player, since frame rendering gets delayed. So there shouldn't be much of a difference between single-threaded and multi-threaded VisualServer modes for this.

[endragor]

So I got things a bit wrong before. load() does perform OpenGL calls, but, as far as I know, it's not safe to perform parallel calls on the same OpenGL context, as it shouldn't be assumed to be thread-safe. It works on Windows with a test my teammate made, but doesn't work on Android. The fact that it works on Windows can be luck, but I'm not sure. Quoting Microsoft doc on rendering contexts (https://docs.microsoft.com/en-us/windows/desktop/opengl/rendering-contexts):

A thread can have only one current rendering context. A rendering context can be current to only one thread.

So despite tutorials and docs showing examples of loading in a separate thread, it seems it cannot be used safely, and it doesn't work on Android at least.

Would be nice to get some input from @reduz on how this is supposed to work.

[KoBeWi]

I'm using asynchronous loading in my project and I run into lots of issues (trying to make a "seamless" 2D world). Using threads helps, but still causes slowdowns. I even had to add a short timeout before using LOADED resource, because it was crashing randomly.
Having a perfectly safe object that will load your resource in its own thread and emit signal when finished would be nice.

EDIT: Well, timeout doesn't help actually. Maybe I'll just create an issue about this.

[avencherus]

@KoBeWi That's a shame if it can't be made to work. :(

[avencherus]

@KoBeWi Did you ever create an issue regarding your random crashes? I did find some of this on my own when trying to do more finely controlled things with threads. Sometimes it worked, and sometimes it crashes with a net posix error.

The things I were trying also appeared to work at first, because it may be a timing issue.

[jitspoe]

I agree that this should be a core feature of the engine. There's the snippit of GDScript that lets you load things in a separate thread, but due to the random crashes in multithreaded gdscript, it's not usable. Plus, if it's that simple to implement, why not just implement that code in engine and have a multithreaded resource loaded that handles these things behind the scene to make it easier for developers? Also, the gdscript solution doesn't handle compiling a shader in another thread. That would be REALLY useful, as currently the shader compiling is far longer than the asset loading for me.

[Zylann]

There's the snippit of GDScript that lets you load things in a separate thread, but due to the random crashes in multithreaded gdscript, it's not usable

This needs to be fixed, cuz that's basically what needs to work in order to fix this issue anyways. Note that it's very likely not about GDScript, it's about loading from a thread, at all, whatever the language.

[Calinou]

@jitspoe Note that in 4.0, shaders will be compiled and cached when the project loads, so the shader compiling part should no longer be an issue during gameplay.

[jamie-pate]

There's actually two issues here,
1: loading in a separate thread seems to work ok for me.
2: PackedScene.instance() in a separate thread is the issue. It seems to work in tests, but causes the timing of the _ready() function to behave differently. In my case I was able to raise the the instance() performance to acceptable levels by removing ViewPort instances from the PackedScenes.

If scenes being instanced contained ViewPort nodes throughout the tree it caused massive slowdowns in some GPU drivers (intel) (on pc). Allocating (hundreds of) Viewport nodes all at once in a loop is very fast however.

[clayjohn]

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.

The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.

If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!

Note: regarding the GPU loading aspect of this proposal, a lot of that will be implemented in 4.0 with Vulkan.

[KoBeWi]

Wasn't this implemented already in #36640?

[clayjohn]

I forgot about that!

Thanks for pointing it out!