Godot: Change MessageQueue to a page allocator to prevent overflow

Created on 28 Jan 2020  路  3Comments  路  Source: godotengine/godot

Godot version:
Master (3cfb67e0f)

OS/device including version:
Any

Issue description:
As discussed a few times on IRC, and exemplified by many bug reports, our current fixed size for the Message Queue is problematic.

It currently defaults to 1024 (see memory/limits/message_queue/max_size_kb), and many users need to increase it to prevent overflow when handling a large amount of messages (which can happen easily in the editor when using e.g. large TileSets #29238 or Skeletons #35652).

As discussed with @reduz, Message Queue should be changed to use a page allocator so that its size can grow as needed, preventing overflows.

Situations which can cause a Message Queue overflow like #29238 or #35652 should however be fixed in the Inspector itself, as even increasing the MQ will just make things functional again, but it will still freeze a long time while populating all those controls.


Example output to help uses find this issue:

Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
Object was deleted while awaiting a callback
TOTAL BYTES: 1048560
NULL count: 8
CALL : 3
CALL _update_callback: 37181
CALL _update_minimum_size: 5
CALL _sort_children: 6479
CALL _emit_change: 1
CALL _update_script_names: 1
CALL _update_script_connections: 1
CALL _test_update_tree: 9
CALL _node_selected: 1
CALL _update_bone_list: 1
ERROR: push_call: Message queue out of memory. Try increasing 'message_queue_size_kb' in project settings.
   At: core/message_queue.cpp:55.
Failed method: VBoxContainer:_update_callback target ID: 74982
bug high priority core

Most helpful comment

How about increasing the default value to something like 2048-4096 (while we wait for page allocator to get merged/implemented)? I've set this value to 2048 in my own project and I've never got to experience those errors again. I can only guess though. Just see the number of "duplicate" bug reports with the same underlying issue (unless you're interested in collecting the use cases this limitation has on people's projects).

Some other projects which do happen to set the size to 4096: https://github.com/yumedev40/Yume-Visual-Novel-Editor/issues/1

Is the value of 1024 caused by mobile limitations?

All 3 comments

For the reference, I fixed the outdated setting name in the error message with cab555bb, so from now on errors will be:

Message queue out of memory. Try increasing 'memory/limits/message_queue/max_size_kb' in project settings.

My commit doesn't fix #35652 though.
_call_function(target, message->target, args, message->args, message->type & FLAG_SHOW_ERROR);
in MessageQueue::flush() corrupts memory; think it's not my fault, but not 100% sure.

How about increasing the default value to something like 2048-4096 (while we wait for page allocator to get merged/implemented)? I've set this value to 2048 in my own project and I've never got to experience those errors again. I can only guess though. Just see the number of "duplicate" bug reports with the same underlying issue (unless you're interested in collecting the use cases this limitation has on people's projects).

Some other projects which do happen to set the size to 4096: https://github.com/yumedev40/Yume-Visual-Novel-Editor/issues/1

Is the value of 1024 caused by mobile limitations?

Was this page helpful?
0 / 5 - 0 ratings