Godot: The need for an official static performant language

[mateusak]

GDScript is enough for most simple games. Now, however, Godot is about to enter a new era of prosperity, were it matches professional engines in terms of features. But would you expect games like The Witcher 3, GTA V and many others to be feasible using only GDScript? Of course not. There is the need to a custom language like GDScript, but focused on performance instead of easy prototyping.

Why not C#?
For three main reasons; C# doesn't communicate well with the engine, there's a lack of documentation on it and the dynamics between C# and GDScript are terrible: and of course it is, C# have a ton of very advanced features that couldn't be translated to such a simple language as GDScript, and it never will be. Not only that, using both camelCase and snake_case in a project becomes confusing very quickly.

Use only C# then
You could. But there's a reason GDScript is the official language. Fast prototyping is the most important thing in game development. You could never match the ease of writing that GDScript has using C#. And then again it takes painfully long to communicate to the engine.

Okay, but the lack of documentation isn't a valid argument
It is though. C# being a language that is maintained separately to Godot official build, you can't expect it to ever have documentation on level with the current state of Godot, even if it's pretty good. If the actual documentation already falls behind the current state of the engine, you can't expect a side kick language to catch up.

Alright, and C++/GDNative?
It's painful. It literally hurts. It's fairly difficulty to setup, which can drive away the less experienced game developer, and once setup it messes your file structure really bad. Plus you have to delve into knowledge that you shouldn't have to delve into.

If you can't bother to learn to setup C++/GDNative maybe you shouldn't be developing a game
Remember that many people have amazing ideas and they're not programmers. Many great games have been made by all kinds of people and it's our job to allow them to do so as easily as possible.

Jesus, okay, but making another language? GDScript isn't even fully featured yet
I know it's hard. But so was implementing state-of-the-art 3D in the small time frame Godot just did. The fact is that this community is growing and I hope people see the need for this.

Also, this new language doesn't need hundreds of features that most people don't ever use like C#. It just needs to have a close relationship to the engine, GDScript and be performant enough for those critical pieces of code like complex AI, procedural generation and so on.

But people already hate GDScript for being a custom language, having to learn two new languages to be able to use Godot is going to drive people away
Yes, people have been complaining about GDScript ever since it came out. But I have never seen someone complain after using GDScript for a month. The fact is that it's just easy to learn, because there's not much to it, and you learn it naturally while using Godot. The people who refused to use Godot because of GDScript never actually tried to use it, that's the thing. Now that C# is here, these people will come for C# and stay because of GDScript and it's new partner GDStatic (hopefully).

Concluding
With some of the features of what I'm proposing:

• 1:1 relationship with GDScript (such as being able to seamlessly call functions between them and no conversion hacks to types and classes like C#)
• close relationship to the engine (which means calls to A*, for example, will cost much less than it does for C#)
• no need for another layer of GC like C#
• being able to write it directly on the editor, unlike C++

Or maybe everything I said up to now is wrong and I should hang myself. That's a possibility.

[mysticfall]

I'm not an expert in this matter, but are we certain that the performance of GDScript would be inadquate for creating games with decent visual quality?

I know that GDScript is slower compared to other popular languages, but if I'm not mistaken, raw speed of a binding language doesn't affect the overall performance that much, so even when GDScript is 10 times slower than some other language, it'd probably translate to less than 1 fps boost in normal cases, unless the game is unusually calculation heavy, like a simulation.

Again, my actual experience on this matter is quite minimal so I'm mostly saying this from my gut feelings. But I'm more familiar with optimizing server applications and I've seen many similar arguments in that field before.

There are many people who think writing a server application in something like C would boost the performance significantly, but normally it's not the case because the biggest bottleneck in such a type of system is found in I/O, and the time spent in actual business code is quite fractional compared to it.

So it's my understanding that a similar rule would apply to video game performance as well, meaning that the largest bottleneck would be found else where than the game logic, possibly in things like rendering pipeline or physics simulation. And in that case, replacing GDScript with something that can run 10 times faster would only result in a marginal performance gain.

I think, in that case, it'd be much better to try to optimize GDScript where it's unreasonably slow than to replace it with something else, since the benefit would hardly justify all the resources the task would take.

And if what I supposed above is true, than probably we should add a section in the documentation, explaining our users how the raw speed of a binding language relates to the overall game performance, since I believe this to be a recurring topic in such communities like Reddit (r/godot) or our Discord server.

[mateusak]

@mysticfall Actually GDScript performs really bad making even simple calculations. For example, applying cellular automata to a 20x20 array on GDScript takes 0.02s. In C#, it takes 0s (so little that it doesn't register). Combining cellular automata, prim's algorithm, A*, delaunay and permutations to create a basic procedurally generated 20x20 map takes 23s in GDScript. It takes on average 0.04s in C#.

Now, imagine making a game like Terraria in GDScript. Minecraft maybe. You simply can't do it and expect it to perform reasonably.

You say of replacing GDScript; that's not what I want. I love GDScript and it's definitely more than useful for most things. It's definitely a wonderful tool to have. The thing is sometimes you need that boost, and when that times comes currently you have two options: C# or C++. Neither of which interact with GDScript without hassle.

I find difficult to believe a dynamic language could improve that much more that it would compare to C#. If it's possible, I'll happily close this thread and run yelling through the street about the wonders of today's technologies.

[NathanWarden]

I disagree pretty strongly on this.

C# doesn't communicate well with the engine

It's pretty much 1:1 with GDScript last I checked, but if not hooks can be added. Or did you mean something else?

There's a lack of documentation on it

It hasn't even been officially released yet and things are still in flux with it so the motivation for documenting it is pretty low. If it had been documented when it first came out as an alpha a large portion of that documentation would have been changed already.

But, there's a bigger issue at hand with this statement. The entire .Net API is available and is very well documented! This is one of the primary reasons why people use C#. Sure, the language is great, but the APIs are better! Would these be available and documented in said static language?

The dynamics between C# and GDScript are terrible

It's actually pretty good for being a compiled language and an interpreted language. It's definitely head and shoulders above what Unity had between C# and Unity Script.

---

Creating a separate static language would be far worse. Just the language itself would need to developed from the ground up and would be missing features that have already been available in C# for a very long time. That's the easy part! :) Now, all of the libraries for X random thing that clients want has to also be written from scratch in the new custom static language. It would make Godot a very poor choice to do any sort of work beyond indie games. It's hard enough to consider it for paid client work right now. C# is a huge step in the right direction for this type of work.

I can integrate all sorts of various libraries my clients ask for without needing to compile Godot from scratch or write more than a few lines of code. IE. Steamworks.NET. I have things like Xml and Json all available to me and they work great and are very robust (Godot's implementation is pretty hard to use and very limited). There are a few things that companies/clients ask for that integrate proprietary SDKs. Due to wanting compatibility with game engines that already use C#, most of these SDKs are available in... you guessed it... C#! :) If one of these services was a requirement by the client then Godot would immediately be off the table as a usable option :(

[mateusak]

@NathanWarden I'm not sure if you're talking about 1:1 C# to GDScript or C# to engine. I'll talk about both just in case, hahaha.

Your 1st point
Multidimensional arrays, structs, separated random number generator, all the classes that need to be used to achieve what you could with GDScript's basic types (like StringBuilder, OrderedDictionary), functions using advanced features like predicates or delegates. How do you translate that to GDScript? How do you call a function from C# that uses predicate from GDScript? You could hack your way around some of it, but there's things you just simply can't. That's what I meant.

Relationship to the engine is the time it takes for C# to call engine functions, like using SetCell on a TileMap or calling GetPointPath from AStar. It takes a really long time, if you consider the general speed of C#. Plus the double GC that can freeze certain games (Minecraft freezes with only Java's GC).

Your 2nd point
I was talking more about documentation of how C# works with the engine. Did you try to use a Rect2 in C#? I have. I have no idea how to update position and size after it's created. It says it can't be altered. Plus I had to guess that the right way to access them were .Position and .Size because it was written nowhere. You could argue just create your own rect struct, but then we get back at the first point of how to pass that information to GDScript.

Your 3rd point
Okay, libraries. Good point. But I'm not asking to remove C#. Plus you can use the decades of C++ libraries if you need. There'll always be the need for non custom languages to make use of libraries. But there's still the need for custom languages to talk more closely to the engine. They aren't nemesis, surely. The thing is; can you just integrate a library of C# seamlessly with GDScript? Generally not. You have to convert many things in types GDScript understands. That's okay to do for a few libraries, but doing that during the entire development is quite time consuming. Because of that I think there's the need to a language closer to GDScript.

Another thing that could be done is make GDScript closer to C#, but I think it would be a lot more work.

[NathanLovato]

@mateusak You pinged the wrong Nathan, you wanted to ping @NathanWarden
Edit: I edited your post so it pings the right person

[mateusak]

@NathanLovato the hell are you talking about?

Oh okay you edited it. For a moment there I was so confused, hahahaha.

[NathanLovato]

There's work planned on the GDScript compiler to bring JIT compilation and performance improvements with the optional static typing. With GDNative available (not only C++ but also C, Nim, Rust, or PyPi and Cython) I don't see the devs creating a new language from scratch while there's so much work to do besides that. I don't see the need for it either with all the available options and the front-end language delegating so much work to the back-end: yes gdscript might be the same speed as interpreted Python right now, but if you rely on the editor, built-in nodes, physics, etc., you're delegating a lot of work to the engine.

[mateusak]

@NathanLovato so GDScript will become sort of an hybrid? That's very interesting. Hope to see it soon.

So, about GDNative. It's troublesome to say the least to setup. There's not much documentation on it either. Do you think there'll ever be a stage where using GDNative will be as easy as using GDScript? Even if it is, I'm afraid it'll run in the same problems as C#; you'll need to do some hard work to use it with GDScript.

A lot of the work is done with nodes, and that's why GDScript is sufficient most of the times. But when nodes aren't enough, then what? I often use procedural generation as an example because Godot lacks so much in this area and it involves heavy number crunching code.

Anyways I know it probably wont happen. But a man can dream.

[novhack]

GDScript already has an optional static typing. It doesn't improve performance for now but if I remember correctly @vnen was writing about multiple ideas to improve it in the future. JIT was also mentioned in the older roadmap but that will come most likely much later in the Godot's timeline.
There is also this person who posts in the Facebook group who works on the GDScript to GDNative compiler: http://blog.moblcade.com. So maybe something cool will come out of it.
Considering all of this I don't think that another official custom language is viable because the engine itself is already really big and adding more modules would make it even harder to manage.

[aaronfranke]

@mateusak It's not that you're wrong that these various languages have problems, but I don't think that your proposed solution (a different language entirely) is the way to go. Support for the existing languages can be improved (and constantly is being improved, especially GDScript and C# in 3.1).

Any custom language will have a number of inherent disadvantages, including the lack of existing libraries for that language, and the fact that the language wouldn't ever be used outside of Godot. Professional developers would prefer to learn languages that are useful in many places.

[mateusak]

@aaronfranke My train of thought is that no established language will ever have a close relationship with GDScript. But maybe GDScript will be improved to perform just as fast as C# in the number crunching sort of way. Who knows.

Some guy said something about optional static typing in GDScript possibly increasing performance in the future. I'm really interested. I didn't think it was possible to have a hybrid language.

[NathanWarden]

@NathanWarden I'm not sure if you're talking about 1:1 C# to GDScript or C# to engine. I'll talk about both just in case, hahaha.

Your 1st point
Multidimensional arrays, structs, separated random number generator, all the classes that need to be used to achieve what you could with GDScript's basic types (like StringBuilder, OrderedDictionary), functions using advanced features like predicates or delegates.

That can be confusing, haha, sorry about that. I meant 1:1 in the sense of API, definitely not the language and the .Net APIs.

Otherwise, I agree, you would probably need to translate quite a few things into a form GDScript can use if you needed to :)

Relationship to the engine is the time it takes for C# to call engine functions, like using SetCell on a TileMap or calling GetPointPath from AStar. It takes a really long time, if you consider the general speed of C#. Plus the double GC that can freeze certain games (Minecraft freezes with only Java's GC).

Yep, you definitely have to watch out for that every once in a while. Some of that I would probably try and either do upfront if at all possible, pool it, or have those parts handled by GDScript, especially if the code wasn't too complicated. GC though isn't really a problem if you are writing code with it in mind. I've been using C# to write games professionally for nearly a decade now and can probably count on one hand the times I've really had issues with it.

Your 2nd point
I was talking more about documentation of how C# works with the engine. Did you try to use a Rect2 in C#? I have. I have no idea how to update position and size after it's created. It says it can't be altered.

This is something I ran into with C# (and was quite annoying) when I first got started, but it becomes second nature pretty quickly. It's because you can't modify a field of a struct property (get, set) in C# directly since they are value types. You have to get a copy of the struct, modify it, then pass it back in.

C# Vector2 size = rect.Size; size.x = 10; size.y = 10; rect.Size = size;

Your 3rd point
Okay, libraries. Good point. But I'm not asking to remove C#.

Ah, okay. I misunderstood that.

Plus you can use the decades of C++ libraries if you need. There'll always be the need for non custom languages to make use of libraries. But there's still the need for custom languages to talk more closely to the engine. They aren't nemesis, surely.

The thing is; can you just integrate a library of C# seamlessly with GDScript?

I personally wouldn't do that. I'm pretty sure most people who would use C# will use it for majority of their code base. Not because GDScript is bad, I happen to like it a lot, but because it becomes unmaintainable as the code base grows.

But, even if I did use GDScript as the primary language I'd let the C# code handle all of those parts and only call into it from GDScript when I need to. Most of the libraries usually have one or two hooks where they really do something. For instance, I'm developing a game right now where it submits a score to a server to keep track of the state of a tournament. So, if I were to use GDScript for the game logic I would need to make a single call to the library at the end of the play to $wrapperNode.SubmitScore(score). Other libraries are advertisement platforms where you would need to call to $wrapperNode.PlayAd(callbackNode, callbackFunc). Even for Steamworks you could limit your communication to it with just a few scripts.

I can see where you're coming from on this though as there could be some cases where you might need to use a library in many places especially if it's for data manipulation.

Another thing that could be done is make GDScript closer to C#, but I think it would be a lot more work.

I agree that this would be a lot more work as C# has some really advanced stuff that no other language I'm aware of has. Async / Await translates your code at compile time into a state machine so you can run it asyncronously. Pretty neat stuff :)

http://dominickm.com/pallet-town-async-await/

[NathanWarden]

@aaronfranke My train of thought is that no established language will ever have a close relationship with GDScript. But maybe GDScript will be improved to perform just as fast as C# in the number crunching sort of way. Who knows.

Some guy said something about optional static typing in GDScript possibly increasing performance in the future. I'm really interested. I didn't think it was possible to have a hybrid language.

I think Ivan Safrin had some form of this and with some success in PolyCode by translating Lua into C++ a few years back. I think it's a really neat idea!

[aaronfranke]

You have to get a copy of the struct, modify it, then pass it back in.

Or, for example, with a new Vector2:

rect.Size = new Vector2(rect.Size.x, 10);

[QbieShay]

Hello @mateusak, allow me to disagree with your very first sentence:

GDScript is enough for most simple games. Now, however, Godot is about to enter a new era of prosperity, were it matches professional engines in terms of features. But would you expect games like The Witcher 3, GTA V and many others to be feasible using only GDScript? Of course not. There is the need to a custom language like GDScript, but focused on performance instead of easy prototyping.

AAA Games often struggle with problems that are not related to the game logic: their problem is, in general, how do I render this huge scene at 60Fps? This kind of computation is already done in a language with better perfomance (C++) inside of Godot.

What if I have something that does a lot of computation? How do I handle that?

Well, in that case GDNative is your friend. I do not have experience with that myself but I do know someone that ported a part of his game to GDNative to gain performance.

I am very much against yet another language. There are already enough supported.

Although I do understand where this suggestions are coming from, I think that Godot has more pressing issues to take care of, and making another language is not worth the developer time investment.

[mateusak]

@QbieShay @aaronfranke I knew this community wouldn't accept a new language the moment I began writing. I just used it to raise awareness to an actual problem; when you need a high performant language that needs no conversion to GDScript and can easily and costlessly cowork with it, what do you do?

Maybe the answer is improving GDScript, making the performant language itself. Maybe it's making a hybrid out of GDScript. Maybe making writing GDNative C++ as easy as writing GDScript. Maybe removing GDScript entirely and adding a new static and performant yet easy to prototype language. Maybe there is already a feature like that and I'm missing it. I don't know the ins and outs of this engine like many of you do.

But there is, however, a problem, ladies and gentleman. What to do about it?

[QbieShay]

Did you experience bad performance on something you are developing?

[akien-mga]

I just used it to raise awareness to an actual problem; when you need a high performant language that needs no conversion to GDScript and can easily and costlessly cowork with it, what do you do?

Use C#.

[mateusak]

@akien-mga As I said previously;

multidimensional arrays, structs, separated random number generator, all the classes that need to be used to achieve what you could with GDScript's basic types (like StringBuilder, OrderedDictionary), functions using advanced features like predicates or delegates. How do you translate that to GDScript? How do you call a function from C# that uses predicate from GDScript? You could hack your way around some of it, but there's things you just simply can't. That's what I meant.

Relationship to the engine is the time it takes for C# to call engine functions, like using SetCell on a TileMap or calling GetPointPath from AStar. It takes a really long time, if you consider the general speed of C#. Plus the double GC that can freeze certain games (Minecraft freezes with only Java's GC).

[mateusak]

@QbieShay Yes. I develop mainly procedural generated games. Those can be done on GDScript on a small scale, but sometimes you need more. But if you use any of the available options you can't really use GDScript together with them. There's a lot of things you need to translate between the two, calling functions from each other costs a lot and is generally hard to setup and maintain (in the case of C++/GDNative).

I just wanna use GDScript goddammit 😢

[QbieShay]

I suggest you rephrase your issue mentioning first the problems that you are having rather than arguing over triple A games, it is much more useful to understand where to improve

[mateusak]

@QbieShay I guess so. When I was writing the issue I thought a new language was the only solution, although I didn't believe it would be accepted. But now it seems there are many solutions to this issue. Rewrite or close and open a new one?

[QbieShay]

I'd close this one and open a new one, where we can discuss how to improve the workflow also for you :)

[mateusak]

25775

[QbieShay]

Thanks!