Godot: gdnative - help porting creature2d runtime to Godot 3

[blurymind]

The developer of creature2d ( @kestrelm )has recently announced that he is looking for help to port his runtime to godot3, using gdnative:
https://www.reddit.com/r/godot/comments/6rn7nt/help_port_creature_2d_skeletal_mesh_animation/

I contacted him and he discussed with me the possibility of doing it himself. He also shared limitations in gdnative that are preventing him to move forward:

Having said that, porting the Creature runtimes over to GDNative isn't a
trivial task, I was planning to actually take on the task myself next year
when I get the cycles.
How much experience do you have in C++11/14? Are you familiar with writing
well structured clean code that involves smart pointers ( unique_ptr vs
shared_ptr ), threading ( using std::thread ) and general data structures
( unordered_map, transforms etc. )? Also I need somebody who can handle
coordinate space transforms confidently ( transforming between local
character, bone and world spaces by constructing a coordinate
frame/orthonormal basis ) Are you also familiar with writing data
structures that are cache friendly/efficient for memory layout?

No worries if you are not confident in any of those; but that's basically
the sort of requirement I have for a C++ runtime. Having said that, if you
can find out for me how to construct a dynamic mesh with points, indices,
ups etc. in GDNative that will be great. So far I haven't being able to
find any of the docs or code documenting such a process.

The more I think of it, I am leaning towards handling the runtime myself
when the time comes because I am not sure if the average dev can handle
the complexities of the initial Creature runtime. After the first stable
version is pushed out, I will be happy to invite other people to join in
and contribute to the featureset.

He further clarified the issue:

Take a look at:
https://github.com/kestrelm/Creature_Godot/blob/master/creaturegodot/creaturegodot.cpp

The issue is currently the runtime inherits from Node2D. During the update notification callback, we update the points/indices/uvs with:

> VS::get_singleton()->canvas_item_add_triangle_array(get_canvas_item(),indices,points,colors,uvs,texture.is_valid()?texture->get_rid():RID());

However, this is done with inheriting directly from a Node2D object. The issue with GDNative is this:https://godotengine.org/article/dlscript -here It says: “A Native script is not a module. Modules have access to all the C++ classes available in Godot and can extend engine functionality. You want a different renderer? Module. You want to add networking capabilities to all nodes (ahem)? Modules.” So basically, I am not sure if I have the same access to the functionality ( specifically the ability to invoke a call to draw a triangle array ) from GDNative. I cannot find any docs/help on this matter. Maybe there is a way but right now I have no clue.
One more thing, I will refrain from using immediate geometry, that looks like a very inefficient way to send data into the system resulting in subpar performance. So they need to provide another way out.

In another issue @reduz said that it would be easy to do via gdnative.
Here: https://github.com/godotengine/godot/issues/2914#issuecomment-320366014

For 3.0 it will be easy to integrate the runtime via GDNative

My question is - is it though? @kestrelm doesn't seem to think so. Is he missing an important piece of information or does gdnative truly have a limitation that would prevent bringing in a creature2d object in godot without need to fork and recompile the entire engine?

Btw a good example of a c++ binding used in an engine seems to be the one dragonbones has here:
https://github.com/DragonBones/DragonBonesCPP
The devs seem to have neatly separated the actual runtime's c++ source code from the cocos bindings in separate folders. I dont know if it is even possible to do that with gdnative and creature, but I am perhaps not the one asking the right questions here. All I really dream of having is the runtime available as an addon using gdnative. Ability to add a creature2d node in godot's editor and use all of the features that are currently available for its Unity2d runtime atm.

Perhaps @karroffel would know a good approach for this? Can you help us with @kestrelm 's questions? I don't understand gdnative very well, so I decided to create this issue to discuss them with the community, rather than just between ourselves. He gave me his permission to share our discussion

[groud]

From what I know, the GDnative API is not final yet, so the documentation is not up to date.
That being said you can find some infos on how Gdnative works here.

Creating a new node type of node and exposing its function to the other nodes should be very easy to do with with gdnative (also in C++, see here). But I guess the developer of creature2d should wait for official 3.0 release to use it.

[kestrelm]

I guess my question is how do we render a triangle mesh with indices, points and uvs efficiently in the new framework? I haven't been able to find a sample/docs showing how to do it in GDNative. Any pointers will be great. Thanks.

[karroffel]

Try ArrayMesh

[kestrelm]

@karroffel Looks great! Any samples on how to use this in C++? Is there any code available for reference.
In particular, how do I create and update the vertices from GDNative using ArrayMesh.Thanks!

[kestrelm]

One more quick question: So regarding ArrayMesh, are you thinking that I should have a script that attaches to that, then for updates:

• Call surface_get_arrays() to get the array to the surface, then update that array?

Is there any equivalent of memcpy to quickly move lots of points into that array? Or do I have to perform a for loop element by element which is not efficient. In any case, is this the most efficient way to update the arrayMesh? Points, UVs and Indices will need to be updated per frame and there are lots of them.

Thanks

[blurymind]

Would this be better implemented if we wait for mesh2d to be developed for 3.1?
It's on the roadmap

[kestrelm]

Actually I took a look at both code in Godot on github a bit more closely and I think I already have an idea how to tackle this problem. Godot 3.0 Creature runtimes will most probably officially start after the next major Creature feature is rolled out. I will post more questions on this thread if I encounter any more issues when the development work starts.

[kestrelm]

@karroffel Hello, quick question: are we allowed to do something like call VS::get_singleton()->canvas_item_add_triangle_array() from within GDNative?

So the idea is we have a GDScript derived node that has a GDNative C++ script attached to it, I will ideally like to trigger a redraw by calling update() on the GDScript and then in the GDScript draw() callback, I will want to call into native code to do some rendering. So is this possible?

Thanks!

[karroffel]

Calling VS::get_singleton()->canvas_item_add_triangle_array() is not possible, those are the VisualServer methods you can use http://docs.godotengine.org/en/latest/classes/class_visualserver.html.

We could add it as a special method, but that doesn't feel right to me. Maybe in future we get more powerful custom rendering tools.

[kestrelm]

@karroffel Thanks for the reply. So in that case, is one other way to do this like this:

1) Make GDScript that inherits and customizes an ArrayMesh Node
2) Creata a GDNative plugin that attaches to 1)

@karroffel Question for you: Is ArrayMesh fast enough to handle changes in points, indices, uvs per frame? How should I be streaming in the data at every update? Do I call surface_get_arrays () on it and modify it? Sorry, but the examples on ArrayMesh are rather sparse.

If instead you can provide me with the ability to call canvas_item_add_triangle_array() or some variation of it, that will save a lot of trouble. Being able to render a mesh with points, indices, uvs is a rather basic operation imho. But if that is not possible, can you let me know if my approach using ArrayMesh will be fast enough for real-time performance? I was told I should be using GDNative to accomplish the plugin so please @karroffel let me know if this is the right thing to do.

Thanks!

[kestrelm]

Hello all,
After some research of my own ( I am not a Godot expert, but spent the entire day today digging through the code as much as I can), I have come to the preliminary conclusion that GDNative is not ready for high performance 2D mesh deformation code that the Creature runtime requires. This is because it does not allow for calling into functions like canvas_item_add_triangle_array() which limits its usefulness serverely with respect to the runtime I am developing.

I looked at a couple of other suggested options ( ArrayMesh ) but they are ( according to the official docs ) not suitable for dynamic geometry either. ImmediateGeometry is also inefficient and not suitable for the large complex meshes Creature actually requires for high performance animation.

However, all is not lost. I will simply revert back to the old way of making the Creature runtime a C++ module. This allows me full access to what is required. However, this also unfortunately means users need to know how to compile the engine ( probably not too big a deal ). I was rather excited about GDNative when I first heard about it, but as of right now ( since things are quite fluid and hopefully move fast ), it is simply not ready for high quality/high performance animation requirements of Creature. I will stick to using modules for now. Development on Creature for Godot 3 will hopefully start in a bit.

[leonkrause]

@kestrelm If you're still interested in evaluating GDNative for this, I've added bindings for 2D VisualServer functions in #12241, including canvas_item_add_triangle_array

[kestrelm]

@eska014 Fantastic! It is definitely something I was looking for :) I will take a look at it soon and see if it makes the GDNative method a possibility now. Thanks again!

[kestrelm]

@eska014 Sorry really basic question: How do I get a handle/singleton to the Visual Server in GDScript?
Is it just:
VS.canvas_item_add_triangle_array(...) ?

And in GDNative, do you just do something similar in C++? Do I grab the singleton in C++ or something else?

Thanks

[leonkrause]

For GDScript, that's correct, except the VS alias was removed in Godot 3, so it has to be VisualServer.canvas_item_add_triangle_array(...).

I've no experience with GDNative, @karroffel can you help?

[karroffel]

There is godot_get_singleton("VisualServer") and then use a regular method bind to call the method

[kestrelm]

@karroffel Thanks! When I searched for "godot_get_singleton" in the godot github codebase, nothing showed up. What am I missing here?

Update: I think you had a typo. You meant: godot_global_get_singleton() instead correct?

Also when you mean regular method bind, sorry but I am not familiar with godot's method/function binding system. Is this a bit like C++11/14 std::bind? Can you point me to any code in the repository that does something like this?

Thanks!

[kestrelm]

@karroffel @eska014 Hello, just wanted to say thanks a lot for all the help, I really appreciate it. However after trying out the entire GDNative build process, I think the more prudent thing to do right now is to just go the C++ module route. The GDNative experience for regular users ( to build out to a different platform that was not pre-compiled ) is still not polished enough for production use. Having said that, I look forward to further improvements in GDNative and will definitely consider using the framework when it becomes more mature. For now, I have managed to get the first initial port of the Creature runtimes onto the latest Godot 3.0 Alpha, more work will continue in the next few weeks. I firmly believe in the progress/future of Godot Engine and am excited to support the release of Godot 3.0!

[blurymind]

@kestrelm what is the problem with the build process? Perhaps your feedback could help improve it.
Gdnative is constantly updated and will most likely have a much smoother and polished user experience by the time it reaches the 3.0 stable release

It's too bad you gave up on it :) it could remove the need to compile godot engine to get the runtime

Is it right to assume that you made some progress in creating the runtime for gdnative, but found the build process to be too user unfriendly?
What specifically needs to be improved in the build process as it is at the moment?
Can you suggest things that need polishing? What would convince you to use the gdnative framework - the things that are missing in it at the moment?

Btw you might get quicker feedback if you try to reach the developers via discord

[27thLiz]

Btw you might get quicker feedback if you try to reach the developers via discord

Or good ol' irc, most of the devs hang out in #godotengine-devel on freenode.

[karroffel]

What @Hinsbart said, I don't use Discord anymore, but I'm always on IRC (or Matrix). :)

[kestrelm]

@blurymind So if you were to actually understand what the Creature runtime needs to do, you will realise that making it a regular Godot C++ Module makes a lot more sense than GDNative. This has nothing to do with whether GDNative is easy to use or not, this is just based off the official docs for what GDNative is designed for/not designed for.

Creature runtime needs:

• Custom rendering ( yes I know the new VisualServer bindings were added but nothing is more direct than being able to use whatever API at your disposal in an actual module ) In the future, there might be more custom operations that the runtime requires which GDNative has not exposed. It is much easier to support a module with direct engine access vs one that does not.

• High performance. GDNative is compiled C++ but it's still an indirect way of getting the final results. It is very powerful but is more like writing custom native libraries that are callable from GDScript ( or C# moving forwards ), so there is still one more layer of indirection. Look at C++ modules, the code runs directly in there with no indirection, so you are as close to the engine as possible which is great for Creature. GDNative still needs to be attached to an actual node to function, module allows you to directly create a node ( much more convenient imho )

• Code maintainence. With GDNative, I will need to maintain 2 code bases: the GDScript/C# Node and the native C++ layer, plus make sure the API/data transfer works across them seamlessly. With a C++ module, I can focus my efforts on adding runtime features and maintain it much more easily. This means less bugs and more stability moving forwards.

• Compilation/Docs. I work primarily on the Windows platform, that is where the majority of the Creature userbase is. The Windows compilation docs for GDNative aren't too great right now. Even in the current linux docs using clang, the docs tell you to build with raw clang commands which really isn't ideal. In addition to that, when I tried building a windows test GDNative library ( i finally got it all working ), there were linking issues with my main Godot Engine build because of different Microsoft Runtime Library versions. This again got resolved but remember, I am a dev and my users might not be as used to sorting out these sorts of issues. At the end of the day, if I do not have time or access to the platform/OS the target user is on, the person will still need to learn how to build the GDNative code which at this point in time, is more complicated than building the engine + module ( actually that is quite straighforward on all platforms )

The ideal situation is to have a system like in UE4 where you can build custom C++ plugins that are dynamically loaded into the engine. Those C++ plugins still have full access to the engine ( they should ) instead of sandboxed in like what GDNative currently is. GDNative does not allow actual node creation, it allows you to write native routines in say C++ to speed up any node that calls into your GDNative code but the code you write does not have full access to the underlying engine.

Having said that, is this really a criticism of GDNative or not? @karroffel has put a ton of time in it I am sure and I fully appreciate all the work. My suspicion is that I was just approaching it from the wrong angle. In other words, the issue lies with me, not the hardworking folks at Godot like @karroffel @eska014 . The Creature runtime is more of a C++ module thing, not a GDNative thing given its requirements. For the majority of users who do not have the requirements of the Creature runtime, I am pretty sure GDNative fits the bill.

@blurymind There is nothing difficult about compiling the module with the engine, you really need to learn how to do it if you want to develop games anyway. Plus, the process is very straightforward and very well documented on both the Godot and Creature runtimes sites. There is already a new alpha version of the Creature runtime that builds with the latest Godot 3 from Github, I am in the process of making the latest features like SkinSwapping etc. work. Will make an official announcement + write up the new docs in a bit.

Thanks

[blurymind]

@kestrelm thank you for the feedback. I think it was important for godot to have someone come and push gdnative to its limits, for the developers to understand what it needs to be able to do in order to allow true binding of c++ libraries.
In another thread Godot's main developer @reduz made the claim that "a creature2d runtime would be easy to do with gdnative".
That currently doesn't seem to be the case, however thanks to your feedback, we know what the reasons are. Creature2d is not the only runtime that is in the process of being ported to godot 3 and the other devs also want to use gdnative in the future to make their runtimes more accessible - Spine2d for example. I heard there is also work on dragonbones runtime.

On the point that you make that there is no way for actual custom node creation via gdnative - that is possibly not true. Gdscript has an plugin api, that makes it incredibly easy to create custom nodes in godot. Many of the current existing addons for godot are actually custom nodes that extend a class. Even I can create a custom Node in the form of an addon:
https://godotengine.org/asset-library/asset/91

If it can be done in gdscript, surely gdnative should be able to also do it? Perhaps @karroffel could confirm that?

Here is the doc on plugins:
http://docs.godotengine.org/en/stable/development/plugins/making_plugins.html

That said, If you choose to keep it a module, I totally understand. It is your runtime, so you understand its design better than anyone. The only problem in that is of course the inability to have it included with regular godot releases. I don't mind compiling, it is more hassle to do in windows than linux-yes. If you want your game to support more platforms (like android and osx) - you have to also compile the exporters with the module included. It is just not as convenient as just grabbing the official release from godot's website and getting creature2d runtime straight from godot's asset store(I linked to earlier)- without even leaving godot :)

In any case, I am very much excited to get to play with the full featured creature runtime in godot3 - be it in a module or a gdnative form

[karroffel]

@kestrelm

GDNative still needs to be attached to an actual node to function, module allows you to directly create a node

This is false and true 😄 There are so called "GDNative singletons" which are libraries that expose a special function (godot_gdnative_singleton()) which gets called at engine start up. Currently it's used for ARVR driver implementations and PluginScript. So if there is a C function that allows you to register things at the start of the engine then those things can potentially be used through the whole runtime of the game/engine. Also you can use a GDNative library as just a library with the GDNative class. You can only attach scripts to objects, for that you'd use NativeScript.

In future we can maybe add a way to add "real" classes to Godot using a GDNative library, we would have to add an interface to ClassDB for that, but I think it should be doable. But yes, that point is totally valid :)

---

I will need to maintain 2 code bases: the GDScript/C# Node and the native C++ layer, plus make sure the API/data transfer works across them seamlessly

The bindings and the Variant class in Godot are making sure that "data transfer" happens without much pain. It's that class that allows you to call GDNative from C# and GDScript from GDNative, pretty much everything is wrapped in Variants as soon as it only remotely touches scripts or user code (for example the call functionality in AnimationPlayers uses it too).

---

The Windows compilation docs for GDNative aren't too great right now. Even in the current linux docs using clang, the docs tell you to build with raw clang commands which really isn't ideal.

Yes, docs are lacking, I know that, but Windows is not any different from all the other platforms anymore in terms of linking and the overall build process.

There are no commands for Windows given there because, to be honest, I have no clue about MSVC compiler flags and nobody figured them out and made a PR. I insist on having those "manual build commands" in there (for clang and gcc it's the same, you just swap the name of the program, the args are the same) because it's much clearer what actually needs to be done. I had Scons scripts for all of that, but when it comes to understanding what the build process looks like, a high level build script is worse than seeing what really does the work in the background. That was my rationale. You can always wrap those commands into a build script for a build system of your choice, but for the learning effect providing a scons script is a lot worse than seeing the actual compiler calls.

---

The ideal situation is to have a system like in UE4 where you can build custom C++ plugins that are dynamically loaded into the engine. Those C++ plugins still have full access to the engine ( they should ) instead of sandboxed in like what GDNative currently is.

Yes, that would be great, but there is one little problem, and that problem is C++.

I would've done that from the start if it was a good idea, but it's not, because C++ is horrible when it comes to ABI/name-mangling/calling-conventions/portability between compilers, so the only real option is C. This is why GDNative is limited to what the C interface lets it to. But using MethodBinds already gives you the same power as GDScript and all other scripting languages in Godot have. When you need more power you need a C API for that so it can interface with the libraries in a stable way. That's what those "wrapper modules" like the ARVRInterfaceGDNative and PluginScript do. They are "simple" wrappers in the C++ side of things that provide a C API and then wrap the C code so it can be used like regular C++ from inside the engine.

---

@blurymind

If it can be done in gdscript, surely gdnative should be able to also do it?

Yes, it's just a Node with a script already attached to it that shows up in the list of Nodes and has an optionally different icon, so it's not really a "new node", it's just a regular node with a script already attached, you can't attach another one after that.

---

Generally sorry that I've not been as active in this thread here, but GDNative itself is pretty small, so to make it useful there are many moving parts around it. The proper C API, NativeScript implementation, language bindings, ARVR. If the C API changes (which did happen recently) then all the other parts need to be updated as well, and I didn't do that for the C++ bindings, so that's why there were build problems and all that good stuff.

If you think a module is better suite - awesome, then you know what you won't use. That's also useful information. But maybe there's just some API missing to make stuff useful to you.

I think that docs are really badly necessary to clear up some stuff, I'll try to work on that before a first beta build, so it's all clear when "the masses" hit 😄

[akien-mga]

I think it's a good idea to stick to a C++ module for now and let GDNative mature, it's changing a lot, documentation is lacking and real life use cases need to be developed to continue improve the interfaces (such as PluginScript merged a week ago which opens many new possibilities for binding languages).

The main reason why you might want to reconsider GDNative in the future (and it shouldn't be hard to port a C++ module to a GDNative library) is actually that then you remove the need for your users to build anything. You would be the provider of precompiled libraries for all platforms that your runtime supports, and Godot users would just plug them in directly and benefit from added features. So instead of having all users recompile the engine + the export templates for the platforms they want to support, they'd just download what you compiled yourself. So even if the GDNative compilation is a bit complex, you'd be the only one facing it, not your users.

On the other hand, you can also do that with a C++ module and distribute your custom version of Godot with the Creature runtime.

[kestrelm]

@akien-mga @karroffel Thanks for the info! I think the correct strategy for now is this: I will continue down on the module path so that I can put in the latest features of Creature ( SkinSwap, Layer Order animation etc. ) into the Creature Godot runtimes. This will at least bring the runtimes onto par with what is in Unity and/or UE4.

When GDNative is ready and more mature ( including better docs ), I will make another effort to port it onto the GDNative framework. This makes a lot more sense when the project is stable and no longer in a state of flux. GDNative is very powerful but I think right now it needs the following:

• Much better documentation to help devs like me wanting to support Godot Engine. We want to be able to get our features into your userbase ( which ultimately helps your effort ) and not get bogged down too much with trying to figure out the internals of your framework. For the C++ bindings of Godot, having an easy SCons template that we can use to add our native plugins will be fantastic

• The ability to directly create a true Godot Node in C++ using GDNative easily will be huge. I am a C++ person ( don't care too much about C#, python or some other scripting language ) and believe to make code performant, there is really only one option ( ok 2, C but I prefer C++ :) ) which is C++. I fully understand the ABI issues of C++ when it comes to plugin modules. Having said that, if there is a way around that problem it will help a lot to truly unleash the power of GDNative without being reliant on some GDScript or C# script to "bootstrap" it ( the current situation right now )

So in summary: I will get the module version of the Creature runtime for Godot 3 up and running first, then port over to GDNative when the framework is ready. My latest checkin already has a running test fox character running in Godot 3 successfully and there will be more to follow!

@blurymind Thanks for your enthusiasm, I really appreciate it! 😄 However, I do want to emphasize to you that from a dev's standpoint you don't just attempt to put code you want for production environments onto a new alpha framework that is still undergoing large amounts of changes. I hope you understand the implications of this. The module method might not be as ideal as you envisioned for the initial release but at least we know it works and is a solid foundation to get features up and running. You need to understand to get actual production quality code into the hands of users, the entire environment has to be stable enough for deployment. @karroffel @akien-mga and the rest of the Godot team are working super hard and doing an amazing job getting Godot 3 up to speed. The entire Godot engine is a great piece of software which is very complex, add on top of that another complex piece, the Creature runtimes, and you get the idea. If you want to fit a complex piece of code onto some framework, you should ensure the underlying structure is stable and ready for production use, otherwise it is a rather risky task. Out of respect for the Godot Devs, I will want to give them time to finish up Godot 3 and solidify GDNative before working on a GDNative port.

Cheers

[BastiaanOlij]

I haven't followed this thread in any detail, when I have some more time on my hands I wouldn't mind having a closer look at the module to see what it does, and doesn't do.

One thing I suspect needs to be improved is what can be done on the rendering side. It'll be interesting to see how you're dealing with this in the module.

Anyway, real reason I'm adding my voice here is just to point this little nitbit out:
https://github.com/BastiaanOlij/GDNative-demos/tree/space_colonization/cpp/space_colonization

It's not finished but it does show how you can create a node with GDNative. In this case its a node that "subclasses" (its not real inheritance) ArrayMesh so we can generate a mesh on the GDNative side and use it.

My main issue atm is that you do need to create a GDNS file that you need to instantiate with code in order to use the resource but once done seeing GDNative currently doesn't register objects like these to the ClassDB. It would indeed be nice if we could actually register GDNative objects in ClassDB and use them as any other class, some day perhaps :)

[kestrelm]

@BastiaanOlij Great info + links! You can follow the progress of the Creature Godot 3 development here:
https://github.com/kestrelm/Creature_Godot/tree/master/creaturegodot3/creaturegodot

It's still WIP but the core animation engine/playback for Creature authored characters ( Mesh Skinning + Deformation ) already works.

Quick question for Godot Devs: Is there any equivalent to do a faster copy of many elements using your built in Vector type?

With C++11/14, I can use std::copy() which the underlying compiler ( like VS2015 ) will optimize to a very fast memcpy ( or related operation ). Or I can use std::memcpy as well to accomplish that. In Godot, because I am not familiar with the API, I have resorted to copying elements using a for loop which is not ideal. Let me know if I am missing anything.

Cheers

[kestrelm]

@akien-mga @karroffel @BastiaanOlij @blurymind

Here is the latest video of the upcoming release of Creature runtimes for Godot 3:
https://twitter.com/KestrelmMoon/status/923778463424581632

More to follow.

Cheers

[BastiaanOlij]

Man that looks cool!

[mhilbrunner]

Cool stuff and good work, kestrelm :)

As this seems now mostly done and available at https://github.com/kestrelm/Creature_Godot, this issue should probably be closed.