Godot: Implement a way to use deep neural networks in Godot

Created on 16 Jul 2019 · 28Comments · Source: godotengine/godot

PyTorch, SciKit-Learn, TensorFlow, etc. can be used to create responsive and evolving AI actors
into computer games to provide greater interactivity and deeper gameplay for players.

Godot has the opportunity to become on-par with Unity Engine, where they are already being widely used.

Therefore I propose to integrate a new feature into Godot: the ability to use deep neural network frameworks like TensorFlow to build, train and test DNN models and use them for making in-game predictions.

I've proposed this as an official addon, because most machine learning frameworks use Python, C++, JavaScript, or Java to operate and GDNative is ridiculously difficult for your average developer to use due to lack of documentation. Also, unofficial Python bindings for Godot require the user to compile the Godot Engine, which is just not something anybody can do to create a reliable and working binary.

feature proposal core plugin

Source

aspenforest

👍25

Most helpful comment

I think it's important to clarify things a bit, because for instance "Integrating Tensorflow in Godot _for training_" makes absolutely no sense to me.

1) First, supposing we are not taking about Reinforcement Learning but classic ML
(so just your Neural Net train & inference)

Then it is separated into two very distinct and independent phases : train and inference.

For training :

It make no sense to bind Tensorflow, Caffe, PyTorch,
to Godot for the following reasons :
- Training requires you to write code that defines the model (the compute/autograd graph). This is done generally in Python (or Julia, Matlab, R, etc.), this has no dependency or anything to do with Godot, so it makes no sense to write that from Godot.
- Training requires data, where does the data comes from ? :
- From outside your game ? (Denoiser, mesh generation, texture generation, audio generation, voice recognition, etc. ) then you don't care about Godot.
- From inside your game ? Then it's 2 lines of code to just write your data to a .csv, once again we don't care about Godot.

For Inference :

A surprisingly not so well known fact : PyTorch, Tensorflow, etc. are not interesting for inference in production, they are used for training. In the industry people do not use those much for inference in production, they rather use :
- inference engines (like here or there). Those are dedicated to support lots of platform (not CUDA only like tensorflow), have special optimization to use the best of the hardware and what's available (SIMD, Vulkan, GLSL, CUDA, special instructions, NEON, OpenCL, etc.).
- Or, in the future, we can expect the OS / Platform to offer Native ML APIs i.e an integrated inference engines in the OS (Android & IOS are already offering / working on that if I'm not mistaken, and there is a proposal of an API for the Web too)

Conclusion :

_All we want is just bind an inference engine to Godot_ (ideally abstract the API enough so we can have different inference engine / Native ML APIs for different platforms), but we don't care about Tensorflow, Caffe, PyTorch, PlaidML, SkLearn, etc. Moreover, different people work with different framework anyways, so not a good idea to impose any framework especially when we don't need to.

To give an analogy : Godot does not support "Gimp" or "Photoshop", it supports ".png". Same for ML : Godot should support "ONNX" and/or "NNEF" not "Tensorflow" or "PyTorch".

2) Supposing we are taking about Reinforcement Learning

Now here the training and inference phase are less independent.

Using on-policy & episodic (for eg REINFORCE, Monte-Carlo, episodic Actor-Critic, etc.) you typically run one episode and collect data, then update your policy/value network and repeat. Other algo may update the policy at each step instead.

So now we need to alternate between Godot (perform one episode and collect data) and PyTorch, Tensorflow, etc. To do so, a very badly documented standard exists : Gym by OpenAI.

What we can do is wrap Godot as an Gym environment (over IPC). I have been prototyping such a feature and I have things starting to work. However I still need an inference engine for using it in production.

Sry for the long post but in TL:DR :

Tensorflow, PyTorch, Sklearn, PlaidML, etc. are good for training. They are less interesting for in-production inference, and that's okay because they are made for research, testing and developing models.
Trained models can be exported to ONNX or NNEF which can then be executed efficiently, in real-time and across all platform either using dedicated inference engines (onnxruntime, MACE, etc.) or the OS Native ML APIs.
Training will not happen in-game (you are not going to do a full backprop of millions of neurons in real-time in parallel of your game running) so it doesn't make sense to try to integrate training frameworks to Godot.
Data is either external to the game or can be exported from the game. The only special case is Reinforcement Learning where train and inference are interlaced. But then we should look at Gym + IPC (or maybe something to do with shared memory + GDNative to avoid communication overhead). I've been playing around with this idea and got encouraging results but I cannot use it in production because I still need an inference engine to execute the learned model in game without IPC & Python overhead.

So basically, in my opinion all we need is just an abstract enough API over a GDNative bind of an inference engine. The API would be basically load and then execute over input tensor :

model = loadTrainedModel(myONNXModel)
ouput = model.predict(input)

hackini on 17 Jul 2019

👍13 ❤5

All 28 comments

An official module/addon/plugin would be good.

Note:

"I mentioned that GDNative is not a language. It loads shared libraries. These shared libraries can be written in any language you want. Godot just needs to know how to use them. Because of that, there's no code editor for GDNative in the Godot editor. Because there is no source code. There only is a shared library, nothing more, nothing less. So you have to write your code outside of the editor."

Source:https://godotengine.org/article/dlscript-here (Really really old article)

Documentation might be lacking(not sure) but it just needs time.
Some info for now: https://github.com/GodotNativeTools/godot_headers/blob/master/README.md, https://stackedboxes.org/2017/08/20/trying-out-gdnative/

Also, similar attempts on this have been done before:
https://github.com/Dariasteam/Geode
https://goblinrefuge.com/mediagoblin/u/darias/m/evolving-neural-networks-for-racing-agents/
smart-design.kiev.ua/7lus/python-ascii-game-engine.html
(dead link but google search had this snippet "An engine for small games for reinforcement learning agents. .... This essay introduces a deep learning framework for creating autonomous agents .... a Godot contributor and Python lover who develops a GDNative interface to use Python 3 as ...")

Anutrix on 16 Jul 2019

https://github.com/godot-extended-libraries/godot-tensorflow-workspace For reference.

fire on 16 Jul 2019

Great, I wasn't aware of that project.
Still, though, I noticed that it requires compiling Godot and it uses Python 2, which is becoming unsupported.

aspenforest on 16 Jul 2019

The resulting project doesn't use the python runtime.

fire on 16 Jul 2019

I think it's important to clarify things a bit, because for instance "Integrating Tensorflow in Godot _for training_" makes absolutely no sense to me.

1) First, supposing we are not taking about Reinforcement Learning but classic ML
(so just your Neural Net train & inference)

Then it is separated into two very distinct and independent phases : train and inference.

For training :

It make no sense to bind Tensorflow, Caffe, PyTorch,
to Godot for the following reasons :
- Training requires you to write code that defines the model (the compute/autograd graph). This is done generally in Python (or Julia, Matlab, R, etc.), this has no dependency or anything to do with Godot, so it makes no sense to write that from Godot.
- Training requires data, where does the data comes from ? :
- From outside your game ? (Denoiser, mesh generation, texture generation, audio generation, voice recognition, etc. ) then you don't care about Godot.
- From inside your game ? Then it's 2 lines of code to just write your data to a .csv, once again we don't care about Godot.

For Inference :

A surprisingly not so well known fact : PyTorch, Tensorflow, etc. are not interesting for inference in production, they are used for training. In the industry people do not use those much for inference in production, they rather use :
- inference engines (like here or there). Those are dedicated to support lots of platform (not CUDA only like tensorflow), have special optimization to use the best of the hardware and what's available (SIMD, Vulkan, GLSL, CUDA, special instructions, NEON, OpenCL, etc.).
- Or, in the future, we can expect the OS / Platform to offer Native ML APIs i.e an integrated inference engines in the OS (Android & IOS are already offering / working on that if I'm not mistaken, and there is a proposal of an API for the Web too)

Conclusion :

To give an analogy : Godot does not support "Gimp" or "Photoshop", it supports ".png". Same for ML : Godot should support "ONNX" and/or "NNEF" not "Tensorflow" or "PyTorch".

2) Supposing we are taking about Reinforcement Learning

Now here the training and inference phase are less independent.

So now we need to alternate between Godot (perform one episode and collect data) and PyTorch, Tensorflow, etc. To do so, a very badly documented standard exists : Gym by OpenAI.

Sry for the long post but in TL:DR :

Tensorflow, PyTorch, Sklearn, PlaidML, etc. are good for training. They are less interesting for in-production inference, and that's okay because they are made for research, testing and developing models.
Trained models can be exported to ONNX or NNEF which can then be executed efficiently, in real-time and across all platform either using dedicated inference engines (onnxruntime, MACE, etc.) or the OS Native ML APIs.
Training will not happen in-game (you are not going to do a full backprop of millions of neurons in real-time in parallel of your game running) so it doesn't make sense to try to integrate training frameworks to Godot.
Data is either external to the game or can be exported from the game. The only special case is Reinforcement Learning where train and inference are interlaced. But then we should look at Gym + IPC (or maybe something to do with shared memory + GDNative to avoid communication overhead). I've been playing around with this idea and got encouraging results but I cannot use it in production because I still need an inference engine to execute the learned model in game without IPC & Python overhead.

So basically, in my opinion all we need is just an abstract enough API over a GDNative bind of an inference engine. The API would be basically load and then execute over input tensor :

model = loadTrainedModel(myONNXModel)
ouput = model.predict(input)

hackini on 17 Jul 2019

👍13 ❤5

You seem to be requesting that ONNX be the model and that https://github.com/microsoft/onnxruntime or https://github.com/XiaoMi/mace be the exact engine.

The tensorflow-lite seems to slot in the same place compared to tensorflow (full).

fire on 17 Jul 2019

👎1 👍1

The ideal engine is one that is small enough to be embedded into the engine or a gdnative plugin and can function with opencl, cuda, cpu, accelerated apis or compute shaders.

It has to be popular enough that any random model can be converted to it.

It should be able to be compiled for all the platforms godot supports. At least desktop and mobile.

fire on 17 Jul 2019

You seem to be requesting that ONNX be the model and that https://github.com/microsoft/onnxruntime or https://github.com/XiaoMi/mace be the exact engine.

The tensorflow-lite seems to slot in the same place compared to tensorflow (full).

More like : ONNX the model format and then a Godot API that abstracts over any inference engine / Native ML API that we choose to use underneath.

MACE / onnxruntime are just examples, it could be also Tensorflow-lite or ncnn or any other that fit the best the requirements you've mentioned.

ncnn seems to fit those requirements rather well actually (small, embeddable, support ONNX and thus any popular framework can export to it, pure c++, not CUDA-only but uses also Vulkan or CPU special instructions when available, etc...).

I think the Godot API should be simple and define only one supported format rather than arbitrarily implement dozens of formats to support each framework. This way we can even use different backend engines if needed without breaking the user's code and support most ML training frameworks.

ONNX seems clearly the winner format here as all main frameworks can export to it (I would personally prefer NNEF as it's backed by Khronos rather than Facebook, Tencent, MS, etc. but nobody seem to support it, so ONNX seems the best choice)

hackini on 17 Jul 2019

👍1

Basically we should define :

which format the Godot API accepts (so here ONNX seems the most widely supported by all framework but maybe I missed some other widely supported formats ?)
which engine to use : one small engine for all platform ? different ones for each platform ? which ones ? Native OS APIs ? I don't know what's the best choice but as long as the API abstracts over those, we can always experiment with one engine and later change or use other engine for other platform without breaking the user code.

hackini on 17 Jul 2019

If someone wants it right now they can convert nccn to a Godot module, if not maybe over the week I could look into it.

fire on 17 Jul 2019

Nccn's issue tracker seems to be full of unresolved issues, is there a next best alternative?

fire on 17 Jul 2019

I'm especially interested in using reinforcement learning to make the game adapt to players choices in-game.
Example: to make NPC enemies more dangerous or create a storyteller like in RimWorld, but much more powerful.
For RL, you need to be able to train the model while the game is running, therefore I propse that no limitations be placed on how it's possible to use a NN framework in Godot, but make it the obligation of developer to use it in a wise manner which does not break their game.

aspenforest on 17 Jul 2019

👍2

When it comes to the data formats and API-s, I think the most compatible solution would be to use HDF5 file format for storing models and/or weights and the Keras API for talking with backend NN frameworks, which support the Keras specification.

aspenforest on 17 Jul 2019

👍1

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.

Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

groud on 18 Jul 2019

👍3

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.

Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

I agree, Game AI is all about fun for players, not for super realistic or high precision predictions, therefore Game "AI" is not the "AI" from people talks today. if your AI make player like a fool and beat the shit out of player, your game will fail!

I uses many "AI" technique from the book "Programming Game AI by Example", combine some simple AI we can get a very complex game AI.

I think the best way to implement these AIs is to implement them in high level language like py, gdscript, focus on algorithm/game logic not the specific, especially don't integrated them in engine core

If Godot must provider game AI framework, I hope it is collection of gdscripts

kruglinski on 22 Jul 2019

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.
Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

I agree, Game AI is all about fun for players, not for super realistic or high precision predictions, therefore Game "AI" is not the "AI" from people talks today. if your AI make player like a fool and beat the shit out of player, your game will fail!

I uses many "AI" technique from the book "Programming Game AI by Example", combine some simple AI we can get a very complex game AI.

I think the best way to implement these AIs is to implement them in high level language like py, gdscript, focus on algorithm/game logic not the specific, especially don't integrated them in engine core

If Godot must provider game AI framework, I hope it is collection of gdscripts

I don't think that the point of Ai is programming intelligent game enemies. But for example predicting a realistic blend between 3D animations or adapting music to what is happening in the screen.

ElGatoNinja on 9 Aug 2019

I don't have much bandwidth for this, but it should be possible to do a non-core module for Godot that uses the NCNN ai library and ONNX models via waifu2x to scale images.

https://github.com/nihui/waifu2x-ncnn-vulkan

Here is an example scaling the input and output twice as big:

Input

Output

The plan is to integrate NCNN and think of an interface for the onnx models. It doesn't need to be much, just the ability to load the ONNXmodels via a resource. I've already done this for Tensorflow Lite. https://github.com/godot-extended-libraries/godot-tensorflow/blob/master/loader_tflite.h#L39

As stated above the NCNN is useful because it has GPU acceleration WITHOUT installing NVIDIA CUDA and supports multiple platforms.

fire on 13 Aug 2019

👍6

@hackini @aspenforest SciSharp has many options to apply deep learning in ways similar to python.
One particular use case is reinforced learning of 3D skeletal/vertex agents (from Godot) using OpenAI/GYM but C# .NET

GeorgeS2019 on 7 Nov 2019

Instead of using ONNX with ML.NET, it seems straight forwards just to use SciSharp TensorFlow.NET or Touch.NET

GeorgeS2019 on 7 Nov 2019

Please see the option for .NET Deep Learning for Godot use cases

GeorgeS2019 on 15 Jan 2020

👍2

There's also GodotGymAI that uses PyTorch, which allows for both TRAINING and INFERENCE within godot.

As mentioned above, training inside Godot would be very useful, as it would allow enemy AI to train/learn within the confines of a game environment. This is similar to what Unity is doing with it's ML Agents project.

gururise on 16 Jan 2020

👍2

Please forgive my ignorance if what I'm saying does not really apply to the topic.

I guess it's safe to say that we won't see any kind of neural network implementation in Godot Engine being integrated in core any time soon. If it does, I think this should take around 10-20 years for the use cases to become common enough among developers, but at that time we'll likely be using some other neural-network-based game engine. 🙂

I do believe that a general-purpose game engine should make the development of not-so-common features easier on the module/plugin avenue at the very least.

I'm only scratching the surface around the topic myself, I'm especially interested in reinforcement learning field though. I might be wrong but it seems to me that RL by definition does not/should not rely on anything which hardcodes AI behavior. The ideal RL system should only care about the environment, agents, and the policy. So, the behavior is largely determined by environment and some kind of an input. As humans we have to use a medium such as a joypad. Likewise, the RL agents should also interact via a very similar medium (represented virtually as input states).

I've been thinking that it might make sense for the engine to provide such medium seamlessly to RL agents. This lengthy introduction leads to something which I've been working on, the use cases not necessarily motivated by RL but still I think this might be useful for RL and/or input-driven AI, if interested read my proposal at godotengine/godot-proposals#104.

Talking specifically, I think exposing the input state as some kind of a decoupled data structure should help the learning process of RL agents, especially if we talk about multi-agent RL because we need to have independent, local input state instances for each agent which wouldn't interfere with the global state, especially important if we want to match the exact input medium which humans use to control the agents themselves.

If the proposed feature is at least marginally useful for this, that's a win in my book. :eyes:

Xrayez on 23 Feb 2020

👍3 ❤1

For the reinforcement learning use case, the GodotGymAI looks like a great start, but it might also be worthwhile looking into the communication setup that Unity's ml-agents package uses. If we could copy that (i.e. implement the RPC communicator / Agent classes / ... for godot) and implement a GodotEnvironment version of their UnityEnvironment python class, the whole training setup of that package could be used.