Godot-proposals: Implement a trait/mixin system and combine it with groups

[MathiasBaumgartinger]

Describe the project you are working on:

A landscape-visualization software: https://github.com/boku-ilen/landscapelab. It uses geo-data to dynamically create a world. Having the right geo-data every place on the world can be mimiced. There is also VR-features which in the near future will be extended.

Describe the problem or limitation you are having in your project:

As our project get's huger every week, sometimes it is really hard to keep an overview. Inheritance can cause head-aches with projects of this size, also (thankfully) inheritance is relatively limited in godot. A trait-system like the one in rust could be the perfect workaround.
Currently we have the problem that the whole visualization that is happening should get capsulated and work as a child-scene of something else to possible have more than one visualization at the same time, or for example render the world on a table. Sadly we used a lot of global_transform which limits or progress.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

As mentioned before, a trait system could be very helpful reducing inheritance head-aches and at the same time could help the programming paradigm of DRY (don't repeat yourself). Rust offers a way to implement traits for a class, i think one could greatly combine a similar system with groups.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

Being able to add nodes to a group in the editor helps reducing programming work and scripts. My idea would be to be able to create a new script and add it to a godot group. Every node that is added in this group will automatically implement this trait as well. As gdscript works duck typed and is not really compareable to rust, i would not recommend using it the way rust does but much rather simply attaching the script to this node.
Alternatively if the idea with groups is not liked, i would recommend a system like the one in python: https://py3traits.readthedocs.io/en/latest/readme.html. Traits could then easily be added in the ready function in godot.

For my stated problem i would imagine something like this:

extends Spatial


var relative_transform: Transform setget set_relative_transform, get_relative_transform
var relation_node: Spatial = null


func set_relative_transform(t):
    assert(relation_node != null)

    relative_transform = t
    global_transform = relation_node.transform * t


func get_relative_transform():
    assert(relation_node != null)

    var node = self
    var t = transform

    while node != relation_node:
        node = node.get_parent()
        t *= node.transform

    return t

It should be automatically added and i can simply call for node.relative_transform = x just the way i would do it with global_transform

The scripts would be attached somehow like this:

If this enhancement will not be used often, can it be worked around with a few lines of script?:

Well kind of yes to be honest, probably the closest to this is inheritance. However, especially as multiple inheritance is not allowed, one is kind of limited in godot. Having a superclass you want many nodes to inherit, it could happen, that it already is an inheriting node. Also, when combining it with groups it could be really helpful with reducing scripts and other overhead.

Is there a reason why this should be core and not an add-on in the asset library?:

I honestly think it would be really challenging to implement something like this. Having the insights from the core developers would probably be helpful when doing this.

[Calinou]

Related to https://github.com/godotengine/godot/issues/23101.

[willnationsdev]

Also related to godotengine/godot#24262.

[willnationsdev]

I'm not really sure if it even is possible to make scripted traits dependent on groups without breaking OOP best practices pretty horribly. To have a script support traits, you either need the scripting language to implement support for recognizing them, or you need the Object/Script system in core to handle the entire delegation.

However, groups are a thing that only exist in Nodes whereas Scripts are things that all Objects, even non-Node ones, have. And it would be very strange to have the logic of a Node's script suddenly change from the way any other Object would respond to such a method call just because the Node is a member of a group. This would mean either overriding methods from Object inside Node to have custom behavior accommodating group-traits (plausible, but heavy-handed given the use case it would be meant to solve) or modifying Object's internals to provide a hook for an inherited class to affect its existing method's behavior. Neither of these solutions sounds appealing to me.

If we avoided these problems by just defining new details in the Script interface and having each scripting language handle traits themselves, it would lead to a bunch of custom core code being written simply to accommodate traits possibly being added to some languages in the future. This is also something that doesn't appeal to Godot's development standards.

As such, the only way I can see to implement traits at all is to do it on a per-language basis, with no engine or editor awareness of traits existing.

@MathiasBaumgartinger What do you think? Would GDScript Trait support be enough to satisfy your needs?

[vnen]

It's not feasible to use groups for this, simply because a script cannot know (at compile time) that it belongs to a particular group, especially because a group is a property of an instance while the script is a class. Also, the same script can be attached to multiple nodes which in turn can have different groups, making all possibles paths very complex and prone to fail.

I do have an idea for a trait system, but it would be implemented only on the script side, not relying on the editor or groups in any way.

[betalars]

I'd like to express my support for this proposal, mention a new use-case and also propose a possible implementation, where we might get the benefits of a trait-system without loosing the general nature of dynamic script languages:

The Use case
I'm building a scientific Research application in Godot, that is supposed to be modular: I want researchers to easily implement their own variants of scripts I've implemented so they may tailor the behavior of my application to their individual needs.

But it's not trivial, because I need to do tons of calls to different elements. So I'm constantly running into runtime-errors, because certain scripts don't have the functions I expect and this is becoming more and more of a hassle. Also it makes my documentation somewhat messy.

How to implement it
I actually would not implement traits in a manner, that'd prevent your code from being compiled (similar to, what rust is doing), because this kind of goes against the spirit of scripting languages. So this is my proposed implementation:

1. Define Traits in Project Settings.
   I'd personally just add a new Tab for the Project settings window, where you have a list of all the traits, you've defined in your project as well as all the functions and variables, they "force"
2. Templates
   Godot Script can generate from a template, when you make a new Script. So I'd like to see an option to choose inheriting traits so your new script file already comes with each needed empty function body or variable declaration.
   The function body should contain a keyword similar to pass. However: this keyword will raise a special missing-trait error when reached during runtime (see point 8)
3. Compliance overview
   (I'd probably put it with the project settings) it's an overview of all the scripts, that use a trait and, how many of them seem to fully implement it.
4. Editor Warnings
   Put some kind of Editor warning, that informs a developer, if a script is missing features of a certain trait...
5. Export Hints
   When exporting a variable, developers should be able to force one or traits similar to, how you can force a file format using export(String, FILE, "*.json") var config
6. Implement traits for function heads
   ... so you can define a function is expecting/returning an object of a certain trait ...
7. Implement trait-checks
   ... so you can check, if an object is of a specific trait.
8. Implement a runtime-error for not properly implemented traits
   Okay this is kind of the core idea of my "dynamic" trait system:
   If some scripts in your projects don't fully implement all the traits, they declared to have, you can still run your project.
   But as soon as one of these not properly implemented scripts get's called with a function they should have, but don't ... you get a special error informing you about, how you didn't properly implement that trait.
   Same goes for invalid return types or missing attributes.

I think this way, you kind of get all the benefits of having a trait-system but without breaking the dynamic nature of godot-script:
It still runs, if you have not properly implemented your traits, but it will raise dedicated errors and it will also give you proper feedback on, how well you've implemented a trait in editor.

[patrick-motard]

It's not feasible to use groups for this, simply because a script cannot know (at compile time) that it belongs to a particular group, especially because a group is a property of an _instance_ while the script is a _class_. Also, the same script can be attached to multiple nodes which in turn can have different groups, making all possibles paths very complex and prone to fail.

I do have an idea for a trait system, but it would be implemented only on the script side, not relying on the editor or groups in any way.

@vnen Is there a proposal for this that we can begin to chip away at? Possibly part of the 4.0 re-write? https://github.com/godotengine/godot/pull/39093

[vnen]

@vnen Is there a proposal for this that we can begin to chip away at? Possibly part of the 4.0 re-write?

No, there's not a proposal from my side (though I do have some ideas), much less atual implementation. I don't think it will be done for 4.0, only for 4.1 or later.

I do intend to open proposals for the planned GDScript features soon.

[Gnumaru]

I think some kind of trait/mixin system would be great. Please let me try to contribute something to the discussion.

The core problem solved with php traits/python mixins, with inheritance (and multiple inheritance), with 'include' preprocessor directives and so on is, in the more general sense possible, 'text reuse'. One just wants some way, and preferably a pretty way, of acting 'as if I have wrote that text in this place'. One could solve virtually any reuse problem by adopting a preprocessor workflow. Instead of writing gdscript directly, you write gnu m4 macros, or c preprocessor macros, or microsoft's t4 templates, jinja templates, anything, wich you routinely run and convert to actual gdscript files. It is a bit cumbersome, you lose the ability of writing code and seeing the changes at runtime, but it works around and solve virtually any code reuse and composition problem you might have.

Being the preprocess phase something separate from the 'compile phase', you do not have the problem you usually have in C of having to debug some code generated by a series o convoluted preprocessor macros, because godot will be debugging the preprocessor output, not the preprocessor input. And it would not need work hours from the godot team or the community at all, just your solo work.

That said, it is easier on the programmer to have a trait system and not have to come up with this kind of solutions yourself. I use myself a preprocess step to include templates inside gdscript and emulate traits, but I'm just getting tired of it by now =S.

To have code reuse alternatives to inheritance, traits is one answer (as well as preprocessor include steps). But there are other alternatives.

You could use for example C# style extension methods (as well as golang methods with receiver argument). Extension methods make very pretty code because you call them as if they where instance methods, just as you would with a trait. They are not able to 'inject fields/properties' in the class as a trait would but this can be partially (injecting fields only, not properties) worked around by implementing _get and _set and delegating accesses to undeclared fields to the object's meta dictionary. If we wanted to make things easier to extension methods, we could with two changes: the first would be defaulting to get/set in the metadict any attempt to access an undeclared field. The other would be to not raise any error when trying to get un-existing properties of a dictionary, but return null instead like in javascript (maybe print a warning in the console). With this the extension methods could try to get and set anything in any object an so the 'field injection' feature of traits would be unnecessary.

Extension methods could be implemented with a singleton (only one) defined in the project configuration, so this would be project dependent, would not work with standalone scripts. Since it would be only one singleton, there would be name conflicts only with the class instance and the singleton, but the class method would take priority.

Or it could be implemented with a new keyword like 'use' followed by a comma (or space) separated list of singleton names, class names or load('res://') calls (so this would be resolved at init time). This way you could write several utility libraries and use only the ones you want, but function name conflicts could occur. This could be resolved using the 'use' order, the engine would search the undefined method name in the objects mentioned in the use statement on the order they are written and would call the first it could find.

All that said, I still prefer a proper traits system over extension methods since it would be nicer for people wanting to make more strongly typed code and easier to associate a trait method with the data it operates on, since this info would be hidden in the extension methods, one would have to actually read the code to know what fields/properties are being accessed

[Gnumaru]

I got another alternative idea for a trait system. It would be entirely resolved at runtime.

The Object class (the c++ one, not the exposed gdscript API) would need to have a new field to store a list of objects. There would be three new methods exposed to gdscript named register_trait, unregister_trait and pop_trait. register_trait would take a single object as a parameter and would just append the object to the beggining (not the end) of the list if it is not already there. unregister_trait would take a single object as a parameter and would search for that object instance and remove it from the list. pop_trait would remove the most recently added object from the list, which is the first element of the list.

Upon registering one or more objects, you now have a resolution chain for resolving member accesses (property, method signal, const, anything). When trying to access a member using the self keyword (without self you would get parser errors for accessing undefined members) the engine would search for the member in the current object. If it cannot find it, it would search the first object in the resolution chain (which was explicitly added via register_trait). If not found, try the second and so on until you reach the end of the list. If the member cannot be found neither in the current object nor in any of the resolution chain objects, only then a runtime error for accesing an undefined member would be raised.

If More than one object in the chain have the desired property, than the objects in the beginning of the list will shadow/override the others. This is why register_trait would add to the beggining of the list, not the end, because anytime you can override the resolution behavior by adding a new object to the chain

Optionally (this would be a bigger change) 'self' itself would be at the end of the resolution chain, not the start. This ways you can always override the current objects behavior by registering new traits. To avoid confusion we could create a new keyword called 'chain' or 'us' ('us' would be funny) that would first try to resolve through the chain and, if it cant, try the current object at the end.

Another alternative would be to get rid of the chain array in the object class and use the direct children of the node for access resolution. I think this would be a worse solution. This would be simpler but would force you to use traits only with nodes, classes not inheriting from node would not have access to the feature.

Personally, I would prefer not having to use self, but make the trait chain be used for every access. This would be easier to use but would be the biggest change with most impact in current users codebases. Using self would not change current codebases, and using 'us' would need changes in the parser.

Please note that the behavior mentioned above can already be implemented by the user in gdscript alone, you can have a 'TraitsEnabled' super class, make all your classes inherit from it, you store your objects list in the super class, create the register and unregister methods there, create the 'us' property and resolve the chain access in the getter and setter methods for the us property. But having this in the core engine would allow users to have access to the feature without changing the inheritance hierarchy in their codebases, and making the access resolution in C++ land would probably be faster than in gdscript land. Also, being part of the core engine gives access to all users despite the scripting language they are using, be it c++, gdscript, clr languages(C# etc), or the python and javascript pluggins/modules I have seen out there.

EDIT
In the pure gdscript implementation I mentioned, I confused myself. I meant to resolve member accesses in the _get and _set functions implemented in the superclass, not in the 'us/we' property's getter and setter. Also we would not be able to handle nothing beyond property accesses because, different from python or php, you cannot override the behavior of calling a method, only the behavior of getting and setting properties. So this pure gdscript implementation would either need a call method of itself, or would be impossible unless we define a virtual '_call(name, args)' method in the object class and use it in case the script does not define the method and does implement '_call'.

Also I noticed that 'self' accesses are also type checked by the parser, so we also couldn't use it to access undeclared members. Either e use only 'us/we' (seems better) or we have to not type check the accesses to self, which does not seem a good thing

[Gnumaru]

'we' would be an even funnier keyword.

var v = we.get_this()
we.set_that(null)
we.make_anything("!")

var dude = we.get_node('acquaintances/dude')

# registering new traits
we.welcome_you(dude)

# unregistering specific trait
we.despise_you(dude)

# pop trait
var other_dude = we.should_be_fewer()

🤣

[Gnumaru]

Oh... right... About that "can be implemented in gdscript alone". I forgot that if you inherit from a base script you cannot anymore inherit from any other script or builtin type, like RigidBody2D. This should not be a problem for member access since 'self' is an actual RigidBody2D and member accesses attempts from self (or 'we') would succeed in finding the desired members at runtime (but access without self would result in parse errors just as usual). You can even override virtual methods present in the built-in node type, even if it is not in the inheritance chain, like _integrate_forces. I have just tested it out, a script inheriting from Node that implements a 'func _integrate_forces(state):' do have it called at runtime when attached to a RigidBody2D.