Godot: Object metadata. What to do with it?

[reduz]

As you guys probably know, all objects in Godot can have metadata. This is accessed from code as:

obj.set_meta("key",value)
obj.has_meta("key")
obj.get_meta("key")

This metadata is even serialized, but currently it's usage by the community is rather unclear to us developers. In the editor itself, It's mostly unused. There remain a few cases where the 2D editor puts metadata into nodes (for locking and marking as bones) but this could be moved away.

Metadata can't be edited by users from the UI. This would be doable, but then editor should not store data in there anymore. This is also easy to do.

I'm also not sure if anyone uses these functions from code in games.

So, we were discussing with other devs, what should we do with it? I see three paths for this:

1) Move the remaining usages of it away from it and deprecate the feature
2) Move the remaining usages of it away from it, but keep the feature AND make metadata available for editing in inspector
3) Move the remaining usages away from it and do nothing.

Feedback very welcome on this!

[ElfEars]

2 definately. This is a really useful feature for distinguishing stuff and 2 makes it so much more viable.

[ignaloidas]

1. It's a feature I would definitely use if it would be made more available, and I think it would be a waste to deprecate this feature

[Fasox]

3 For coherence
2 If you want to take an extra step.

The feature is extremely useful for plugins which import assets into the game.

Also, since we are talking about metadata , I would like to bring into the table the chance to define metadata in tiles.

[ghost]

I use it all the time when I want to quickly store data in nodes that don't have script attached (so I have no control over their member vars). Anything will do, just don't remove it. :P

[adeluiz]

If 1 makes Godot faster, then 1. Else 2

[pchasco]

I think in most cases where someone would use metadata, an export var would be more appropriate. That does require the additional step of creating a script and entering a line of code, so that is somewhat inconvenient.

One usage of meta that I can see is if another node wants to modify the metadata of a node. For example, to mark the node as having been seen in a stealth-type game. This obviously breaks all kinds of best practices for software design, but we’re just making games here. A finished game is the goal; the most elegant scripts are worth nothing if the game is never finished.

I say leave it in, but don’t use it for any internal Godot functions.

[BeayemX]

In the high level multiplayer tutorial it is used to store the network_peer

http://docs.godotengine.org/en/3.0/tutorials/networking/high_level_multiplayer.html

[waldson]

I use it to store Tiled's custom properties.

[willnationsdev]

In the custom type system, it is used to store the overriding _editor_icon that shows the custom type's icon in the Scene Dock / CreateDialog. In the stuff I'm working on in the editor, I'm currently using it to store the custom type's fully qualified typename (including namespace). This allows me to track editor-specific data without needing to modify core to add unnecessary, use-case-specific properties to the Object class.

If you DO go with option 2, you might(?) wanna make it so that meta properties can be hidden from the Editor view that edits the values (perhaps only displaying values that don't have a leading underscore?).

I would recommend 2 or 3. I generally think that using metadata for general properties on a node is a bad idea though. My use case just works well because the alternative would be adding code-bloat to the Object class in core.

[FabienDeshayes]

I see the main usage being importers adding their own custom properties / data structure. For example the tiled-importer.

I'm in favor of 3, just because I don't see much value in making this editable: if the data is imported via an addon, I treat is as read-only and will change it via the source only. It would be nice to see it but I don't think of it as important.

[Bauxitedev]

I vote for 2 simply because the Tiled importer I used half a year ago for a game of mine put custom properties into the metadata. Without metadata that would've been a big headache to deal with.

Additionally, being able to edit metadata from the Inspector would be very useful for storing "material properties" inside of SpatialMaterials, comparable to $surfaceprop in the Source engine. Basically that means some materials have different friction/decals/sounds when you step on them or particles when you shoot them. Metadata is an ideal way to store this.

[Norrox]

2!

[neikeq]

One thing to keep in mind is that the metadata field makes the Object class a bit fatter, so there is a price for keeping it.
An alternative to removing it could be improving it. I find the implementation being a Variant Dictionary to be unnecessarily slow. Why not something like an array of components? That would fit the needs of those using it the way the tiled-importer does, and improve performance.

[reduz]

I find the implementation being a Variant Dictionary to be unnecessarily slow. Why not something like an array of components?

Probably would be better as an actual hash map from StringName to Variant, keeping insertion order. This way it's only a pointer when unused.

[CodeChomper]

2 please that way I don't have to make a script to attach data to a scene.

[willnationsdev]

Is there any way to attach metadata to a scene itself? I wouldn't think so since it'd have to be saved into the scene file.

[neikeq]

@willnationsdev Either to the PackedScene, the SceneState or the root Node. There's nothing else closer to be considered a scene.

[willnationsdev]

@neikeq Right, but if you unloaded and then reloaded the PackedScene (and if you didn't want to instance the scene to store additional data about it), then that data would be lost, wouldn't it? Or would the PackedScene's metadata be stored inside the .tscn file too?

[neikeq]

@willnationsdev Not sure if I understand the cases you mention. But if the PackedScene resource has the metadata asigned at the time it's saved to disk, then it should be stored in the .tscn.

[vinterskog]

Didn't even know about this! Could this be used to store additional per-tile data in TileMap and GridMap? I guess probably not since all they store are simple IDs, right?
Anyway, I think this is a useful feature and shouldn't be removed. One question though: It seems as if for every object in Godot, an additonal dictionary is stored for this. Does this mean that there is some memory overhead to objects because of this? Even if the dictionary isn't actually used?

[willnationsdev]

@vinterskog I suspect so based on reduz's reply earlier:

Probably would be better as an actual hash map from StringName to Variant, keeping insertion order. This way it's only a pointer when unused.

And the only way you'd be able to use the TileMap's metadata to store tile data would be to use the tile ID as a key to a sub-HashMap or something like that...if I'm not mistaken. I know there is another Issue about that topic already.

[LikeLakers2]

2, most definitely. We already have many many ways to configure, sort, and interact with our nodes -- what's one more, especially since it's arbitrary data that the dev can parse to their needs?

[Zylann]

I never needed this, even when it comes to adding properties to objects without a script. It's more a last-resort quick-feature for cases where time lacks or every other design patterns fail to address, because it is more convenient in the long term to attach a script in order to declare them (be it a Node or a PackedScene). So even if they were editable in the inspector it would be potentially error-prone, especially when there are many properties. I'm fine with 1 or 2 if most people like the feature. Definitely the editor shouldn't pollute this unless absolutely necessary.

The only time I started needing it was in GDNative to store type tags when they weren't implemented, but that was an old workaround we got rid of.

[eon-s]

Is unused due to the lack of information about it, it can be useful if gets better access (like inspector) and can reduce some complexity in certain nodes, also used on scenes that do not need a script and for "data nodes".

so, 2 too.

[ol-smaug]

It's definitely a convenient feature and is nice to have available in certain scenarios. Either 2 or 3 would be fine so it doesn't go away. :)

[CowThing]

For a long time I didn't even know the metadata existed, and I'm still not completely sure what its use is. If it is kept there should probably be a page in the docs describing it and giving examples.

[Ranoller]

I don´t know if this concerns metadata of TreeItems and similar, but if it is, all my plugins use that (and there are some of they), and i don´t find other way to do that.
In game i remember used .set_meta .get_meta in any project... but currently, only in plugins.

So 2. If it include controls
And 1. If exposed metadata to gdscript- TreeItem and similar is respected.

[willnationsdev]

@Ranoller the metadata stored in TreeItems is something different I think. That is a Variant specific to TreeItem while we are talking about a Dictionary that every Object has. And the method in the TreeItem is set_metadata(index, value), not set_meta(name, value).

[akanewf]

In my personal opinion it doesn't matter whether it's useful or not, It shouldn't be there.

Be used like a hacker(put some tile data in metadata, used on scenes that do not need a script, etc.) is not a good design pattern.

Those usage should not be the original purpose of design.

Personally, I think it might be better to remove or clarify it.

[pchasco]

Generally I agree with akanewf in that it is not a good pattern, but again, we’re making games. I wouldn’t allow this in any business app of consequence. But sometimes you just have to finish the damn game!

[nobuyukinyuu]

having metadata is nice as a "catch-all" but maybe not necessary when it's so easy to extend a class with an export. It seems to be one of those things to save a little bit of boilerplate in-code without having to box and unbox vars. Although I haven't used the metadata property in Godot yet, it seems similar to how Control.Tag is used in winforms, and I used that a number of times to save some headache. It's "dirty" but it works.

I'd vote option 2 or 3. If the inspector can handle whatever gets thrown into metadata then it could be useful for tool scripts.

[volzhs]

i vote for 2. it's a nice feature to save something without creating and attaching a script on the node.

[sprite-1]

I have to agree with the majority here. 2 would be the the best option and would probably get more usage out of people who didn't even realize it was a thing beforehand if they can see it in the editor UI as an editable property.

[gorgo]

2, and could be a good idea to support importing obj metadata from Blender etc. (guess FBX supports metadata)

[beniwtv]

I can see this being useful in a number of scenarios, so I agree with the majority here, 2 gets my vote.

[Plopsiskopsis]

2

[StraToN]

I wasn't aware of this feature, so I have absolutely no idea what one can do with it.

[M4gma]

Does this means a major version bump?

[willnationsdev]

@M4gma the master branch has already been bumped? And I don't see why that would be related to this Issue anyway. If they keep metadata and add an editor for it, that wouldn't warrant a major version bump anyway.

[vnen]

Metadata is a bit like groups, in which you can tag objects with some data. But groups are only for nodes, while metadata is for every object. It's a very useful thing for developing interfaces (no wonder Godot editor itself uses it). In particular for plugins, but can be useful in-game too for GUI heavy games.

As some people have mentioned, my plugin to import maps from Tiled use metadata to bring stuff that is available on Tiled but there's no respective property in Godot nodes.

About plugins as well, if the metadata is removed from the editor, the API to access them should be exposed to EditorPlugin in some way.

However, I don't see much use for editing them on the Inspector. Metadata is only useful from scripting. For me would be a 3 if I have to pick, otherwise just leave as it is. I'm not against moving to a StringName -> Variant hash map though.

[mhilbrunner]

I second vnen, its useful for some workarounds; string -> variant hashmap should be fine.

[willnationsdev]

In fact, wouldn't creating a generic StringName -> Variant HashMap editor for the Editor domain be generally applicable? Sounds like you could use that for a variety of things down the road.

[M4gma]

Hmm, I understood move away the usage from the object class, not from the editor. I vote for 2 or 3 then

[toger5]

I don't like that ppl might be confused what it is good for and are unsure about where to store values (metadata or obj var...) Maybe also might get misused. And if there are elegant work arounds without that feature I think dropping it could be good.

[eon-s]

Is there any example of misuse of metadata in Godot or in general in game development?

[vnen]

I don't like that ppl might be confused what it is good for and are unsure about where to store values (metadata or obj var...)

I believe this can be solved with documentation. Many people don't read it, but still is a place to be pointing to if anyone ask or search for it. Everything can be misused, and it seems more likely that people wouldn't even know it exists (like many people replying to this).

The only workaround I see is applying scripts with exported variables. Not only this is cumbersome, especially when dealing with dynamic objects, but it probably affects performance as well.

[legione]

I would vote for 2 or 3 as many others already said. The main reason is that this feature is incredibly useful for UI elements, where you can store data on the items of the menus or in the itemlists.
A compromise would be to remove the feature from all the objects and keep it only for the UI, but I guess that this is more work than just keeping the feature as it is.

[Zylann]

@legione ItemList, menus and other "item-based" controls have a way to associate userdata already. Object metadata wouldn't be useful for them because these items are not nodes. Or maybe you were talking about something else?

[legione]

@Zylann as far as I understood metadata is a property of all objects, not just nodes, and I thought the items' metadata were just inherited from the object class. If it's not the case, option 1 would be fine for me, but the majority seems in favor of keeping them anyway.

[mwerezak]

Has anyone here mentioned any usage of metadata that can't be accomplished with script vars?

It seems like most things that have been mentioned here use them as an awkward work-around for monkey patching.

Personally I'm for 1 if it streamlines the engine. I don't see a use for them (outside of serialization) that isn't something you probably shouldn't be doing anyways.

Otherwise, I think it should be kept only if we decide on a clear reason for it existing.

If we do keep it around for the purpose of monkey patching, it would be nice if there was some support in the gdscript syntax for it, as long as it's small.

(It might be worth noting that using a dictionary in gdscript can emulate monkey patching just fine in many cases)

[vnen]

Has anyone here mentioned any usage of metadata that can't be accomplished with script vars?

Of course, script variables are powerful enough to do anything metadata can do, but I don't think that's the point. Adding scripts to anything you want metadata can be quite cumbersome.

You have essentially two options:

1. Adding script files to your project and load them when you need metadata and loading them into the objects.
2. Make scripts in runtime to attach them to the objects.

Both options are worse than metadata in terms of usability and performance. Technically you can make a single script that mimics the metadata functionality, but how is that any better? If it's "monkey patching", it's not different doing it with scripts or in the engine.

So the question is not whether it can replaced by scripting, but whether the use of metadata in itself is a bad practice. Which I don't think it is (this thread has some legitimate cases where it couldn't be easily replaced). Sometimes you only need extra data, not different behavior, and for that a script might be too much.

[mwerezak]

Built in scripts make (1) pretty easy, so I don't see a usability advantage. Probably whatever is gained by not having to create a built in script is outweighed by having to type get_meta/set_meta and enclosing the field name with quotes instead of accessing a var.

I don't know enough to comment on performance, so fair enough on that point.

[supagu]

I use this API in my game to store some additional info, particularly useful in UI stuff where I want to associate some game data to a UI element without having to write my own UI element.
It's cleaner than mapping from a UI element to a dictionary of game data.

[aaronfranke]

I'd like to add that I currently use metadata to transfer information between C# and GDScript scripts. Unfortunately get_meta crashes the game if there is no metadata defined yet, so I would find it very useful if I could set some default values in the inspector.

[Zylann]

@aaronfranke use has_meta()?

[aaronfranke]

I would prefer default values over checks, but yes that is a solution for now.

[Zylann]

@aaronfranke oh right you just meant set meta from inspector... I was confused by "default"

[sxkod]

I am hoping for 2.
Use case- an exploration game where I can drop a whole bunch of artifacts and attach meta to them from inspector - such as firstaid_box value, bomb strength, scroll_secret etc and the player can query the artifact and get info. With scripts I will need a script for each or some other way of keeping tab rather than dropping the artifacts and editing their values in the UI.

Thanks for all you devs do :-)

[ArdaE]

I'd prefer it to be kept, i.e. 3 or 2.

Other frameworks have similar features (e.g. the attached properties, and to a lesser degree, the closely related dependency properties of XAML/WPF/UWP) to solve various issues including storage of sparsely populated data efficiently and preventing unwanted design coupling and code bloat (https://docs.microsoft.com/en-us/windows/uwp/xaml-platform/attached-properties-overview).

For me, the metadata feature of Godot has solved a different problem in one project that would have otherwise required (a) a separately managed dictionary (not ideal as it requires central knowledge outside of object instances that would need to be kept in sync with the objects as objects are created and destroyed), (b) a very convoluted design where I would have to create an extra layer of node in the scene tree for each built-in type, as a parent to it, just to insert custom information into built-in node types, (c) creating a new custom type for each built-in type that I want to add my custom data to, causing my data definition to be repeated in multiple places, or (d) introducing a custom non-visible entry into the scene tree, as a child to each built-in type instance, to store my custom data. These are all undesirable solutions, and Godot's existing metadata feature enables a very clean, straightforward design that I was able to implement in two lines of code. Having said that my case is a very advanced one (related to live manipulation and reloading of scripts within a framework, not a game, where I can't rely on custom data defined within the nodes' scripts, and hence metadata stored by a common parent node comes in very handy). What I am doing is not something that the average godot user would or should do, so if the extra pointer needed for the metadata feature is not desired, I can live without it by creating a centrally managed dictionary, i.e. option (a) above. But I can think of various cases in my career when the need for custom metadata came up that would have been extremely convoluted to solve without a (pointer typed) custom tag or dictionary. I find it's a feature that's rarely ever needed, but when it is, having support for it is a lifesaver.

[ArdaE]

P.S. I'm assuming this should already be the case, but if not, I would make sure Godot does not create a metadata dictionary for an object unless there's at least one entry in it, so there isn't any overhead other than a single pointer/reference that's set to null when the feature is not used.

[SuperDIMMaX]

Access meta in editor?
Maybe it will be usefull... anyone :/

export var meta : Dictionary setget meta_set, meta_get

func meta_set(v):
    self.__meta__ = v

func meta_get():
    return self.__meta__

small tip
get_meta() and set set_meta() == self.__meta__

[nobuyukinyuu]

@SuperDIMMaX

https://docs.godotengine.org/en/3.0/classes/class_object.html#class-object-get-meta

[SuperDIMMaX]

@SuperDIMMaX

https://docs.godotengine.org/en/3.0/classes/class_object.html#class-object-get-meta

I know. but self.__meta__ - faster for me )
I.e. a = 1 == set("a", 1)

[SuperDIMMaX]

metadata is simple dictionary in object named __meta__ .... I guess ....

[willnationsdev]

Don't we just need to make a new EditorPropertyMetadata that's identical to the Dictionary one but hardcoded to have Strings for keys rather than any Variant? And then you just have to add __meta__ to the Object properties in the inspector with some sort of property hint. No?

[SuperDIMMaX]

Don't we just need to make a new EditorPropertyMetadata that's identical to the Dictionary one but hardcoded to have Strings for keys rather than any Variant? And then you just have to add __meta__ to the Object properties in the inspector with some sort of property hint. No?

I use this variant - temporary ... i see, someone need access metadata in editor - this temporary variant.
Or you mean write editor plugin?

[SuperDIMMaX]

And i see strange with get access to meta - self.__meta__ working __meta__ not work.

[willnationsdev]

I am referring to just exposing it as a property visible to the Inspector (since I've never seen it there, and I'm therefore guessing it doesn't have PROPERTY_USAGE_EDITOR), defining an EditorProperty object to render it similarly to a Dictionary, but only with String keys allowed, and then handling all the calls on the backend via _get, _set, and _get_property_list.

I'm not sure what you mean by "temporary variant".

[SuperDIMMaX]

now, in inspector you not see metadata of node, in my case, you see and can modify metadata in editor...
export

export var meta : Dictionary setget meta_set, meta_get

func meta_set(v):
    self.__meta__ = v

func meta_get():
    return self.__meta__

with remote debug :)

[SuperDIMMaX]

2nd variant:

extends Node2D

func _get_property_list():
    return  [{"hint": "Metadata", "usage": PROPERTY_USAGE_DEFAULT, "name": "Metadata", "type": TYPE_DICTIONARY}]

func _get(p):
    if p == "Metadata":
        return self.__meta__
func _set(p, v):
    if p == "Metadata":
        self.__meta__ = v
        return true

Write plugin or not....

[willnationsdev]

Yes, I was suggesting that something similar to this be done, but on the editor C++ side, so that it's built into the engine.

[SuperDIMMaX]

1 question: Why meta - self.__meta__ working __meta__ not work (node.__meta__ - working)?

[ArdaE]

@SuperDIMMax, what you’re doing is a nice attempt but it’ll only work in cases where metadata is not really needed in the first place. (If you were able to and have already created a script for the node, why not add your own dictionary to it?).

In more detail: Metadata solves a problem that occasionally arises when one uses someone else’s framework, where extra data needs to be attached to built-in types. Many frameworks solve this by providing a generic Tag you can use however you want, while others — including Godot — go a step further and provide a pre-made dictionary for it (for ease of use at the expense of some minor performance/memory penalty for some use cases). The problems that metadata solves only exist when there’s a distinction between framework (engine) code and user code. If you’re able to attach a script to a node type like you do in your case, that means you could have created your own dictionary to start with, and that you really didn’t need Godot’s metadata feature.

If metadata is to be displayed in Godot’s editor, support for it should be implemented within the editor. (Unless Godot supports extension of existing classes with new functions, in which case your code could be used by attaching it to the Object class. I don’t know if Godot has support for such a concept or not, though technically it should be able to support it, considering its general design).

[SuperDIMMaX]

In my case, need dynamically create nodes with unknown number of variables (and functions)...
I can write:

class foo extends Node:
    var a = {}

var t = foo.new()
t.a = {"j":100, "k":200}
add_child(t)

but, its slowly and need more ram...
P.s. set_meta("a",100) faster, than self.__meta__["a"] = 100

[mysticfall]

Currently, I'm using it to store additional data (i.e. keys) for TreeItem. But it can be replaced with something else, so I don't have a strong opinion about the matter.

On a side note, I wish it didn't throw an error when an unserializable value is given, so that it can treat such values as transient.

The reason is, I want to use it as a way to overcome such a case where you need to add a feature to a built-in node type but cannot extend it.

In C#, there's a concept called 'extension method' with which you can 'attach' a new method to a given type without actually modifying its source. But you can't add any property that way (I heard it's planned for a future release though) so object metadata could be used as a backing store for such ad-hoc properties, if it supported unserializable values.

It could be quite a corner case, but still I think it can be also useful in other scenarios as well, like for storing an actual game object in a TreeItem rather than just its key, for example.

[jonbonazza]

Could this be used (maybe with some QoL improvements from the editor) to easily support Saved Node State? (For example, for saving games).

[nobuyukinyuu]

Probably, but node state can already be saved with PackedScene into a serialized format. It might be more useful as a feature request to be able to have that output to a string if you want to save such things into metadata.

[slapin]

nice feature to add simple marks and data to objects when groups are not enough.
that would be great if I knew about this feature before... So many useless scripts would be eliminated.

[slapin]

I'd be fine with 2 or 3, also would be nice to have instance-specific metadata too.

[mhilbrunner]

Closing this, as the consensus seems to be to definitely keep object metadata. I opened a new issue for exposing metadata to the editor: https://github.com/godotengine/godot/issues/29081