Godot-proposals: Add a built-in terrain system

[MostafaMTH]

Describe the project you are working on: I am working on mini open World TPP Game. I want it to be built with a built in terrain system so i don't run into compatibility issues and i want features to be added to. It will be very useful for a built in or official terrain system made by the Godot Core Team.

Describe the problem or limitation you are having in your project: I don't have a built in terrain system so i can build my open world game.

Describe how this feature / enhancement will help you overcome this problem or limitation: This feature will help overcome this problem/limitation because it will help game Dev using Godot build open world or mini worlds easier and will enhance the workflow.

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work: How should i do that?

Describe implementation detail for your proposal (in code), if possible: I dont know how to code in C++

If this enhancement will not be used often, can it be worked around with a few lines of script?: No.

Is there a reason why this should be core and not an add-on in the asset library?: Yes, because it is needed for a lot of game developers.

[HectorCastelli]

@Zylann is already working on a terrain plugin that would add such functionalities to the editor.

This Reddit thread discusses a similar request to yours on a little more detail.

I understand that other engines like Unity and Unreal have tools for terrain building. Those are not exclusive features in the industry. You can always use other tools, like Blender, to make your terrain and import it into Godot.

[MostafaMTH]

@Zylann is already working on a terrain plugin that would add such functionalities to the editor.

This Reddit thread discusses a similar request to yours on a little more detail.

I understand that other engines like Unity and Unreal have tools for terrain building. Those are not exclusive features in the industry. You can always use other tools, like Blender, to make your terrain and import it into Godot.

I know but when its official and builtin you run into less limitations and it enhances the workflow

[Xrayez]

Another question is whether someone could give @Zylann authority over the features that he's been developing almost solo to be maintained directly in Godot. He's been pushing quite a bunch of changes to help him make the terrain module more useful over the years anyway (at least that's what I've noticed). I'm almost sure that it would be the "dream come true" moment for him, but I might be exaggerating a bit. 🙂

Given how certain ideology is being kept locked for Godot development, there's another possibility that @Zylann decides that he'd rather keep everything as-is, making his development decisions more independent.

I understand that other engines like Unity and Unreal have tools for terrain building. Those are not exclusive features in the industry. You can always use other tools, like Blender, to make your terrain and import it into Godot.

AFAIK most of the terrain stuff are actually modules which can be toggled off if someone doesn't need it. I suppose it also would create problems with bloated binary sizes for mobile platforms if the module is included by default (which is the point) but I believe the bloat is inevitable anyway even if Godot manages to merge only the smallest enhancements over the long time unless they completely stop accepting PRs.

We basically need to come up with an easy-to-use mechanism to dynamically include needed modules which would work on all platforms. I know we have GDNative for that, but recompiling the engine is still go-to option for me personally.

Imagine downloading a two separate Godot builds:

• Download "Godot: bare"
• Download "Godot: extended"

But if we had an easy ability to simply "copy-n-paste/drag-n-drop" needed modules, this proposal wouldn't be created in the first place. Imagine C++-like modules easily shared via AssetLib. I would also be fine downloading the source code via AssetLib, but then Godot would have to provide an easy way to compile the binaries for all supported platforms given the source code.

Say "No" to naysayers! 🙂

[pycbouh]

@Xrayez
Maybe it should be one download, but optimizations are performed per project or per install, based on the type of project you are planning on creating? It may require some compile time, maybe, but ideally at that point it should just download necessary modules and plug them in. Would definitely require changes to module system and distribution, but can streamline the experience for a lot of newcomers. Think how IntelliJ IDEA or Visual Studio are set up during the first launch.

This is probably not feasible in the near future, though.

[MostafaMTH]

or maybe like unity offers multiple types of project like 3D and 3D with extras there can be an options like Project Minimal and Project

[HectorCastelli]

Not wanting to be a naysayer, but, instead of recompiling the engine, doesn't this fit much better as an editor plugin?

After all, it uses the existing engine, but changes the editor behavior. Plugins have the advantage of being enabled on a per-project basis.

I would imagine that it is more realistical to request the Godot community adopts @Zylann's plugin and maintains it with him than to incorporate this feature into the core editor.

[MostafaMTH]

Not wanting to be a naysayer, but, instead of recompiling the engine, doesn't this fit much better as an editor plugin?

After all, it uses the existing engine, but changes the editor behavior. Plugins have the advantage of being enabled on a per-project basis.

I would imagine that it is more realistical to request the Godot community adopts @Zylann's plugin and maintains it with him than to incorporate this feature into the core editor.

what do you mean by "incorporate this feature in the Core editor" but saying "doesn't this fit much better as an editor plugin" it does not make sense.

[Xrayez]

Performance is paramount so a simple GDScript editor plugin is not enough I suppose. Even if the editor is implemented in GDScript, the core functionality must be implemented via C++. One is mostly useless without the other If this becomes built-in, this leads to "C++ for everything" then.

[MostafaMTH]

Why dont they just add it built in with the editor, it will only cost like 5 - 10 Megbytes

[MostafaMTH]

Either way godot will eventually grow in size(in like 5 years it will be aroud 800MB

[clayjohn]

The editor vs built-in is obviously the major issue here. Everyone agrees that a terrain system is useful for many 3D games. So I would like to lay out some of the considerations the devs consider when deciding whether to make something "core"/built-in, and whether to leave it as an addon.

1. Does this feature work as an addon?
Certain things (like batching) have to be implemented in core and cannot possibly be an addon. However, a terrain system can work as an addon (see Zylann's incredible terrain addon).

2. Is this feature needed in all projects, or just some?
Similarly, certain features are needed in all 3D projects (e.g. camera). Others are only needed for certain game types. Terrain is only needed for certain 3D games.

3. Can we easily implement the feature in a way that benefits all use cases?
If we implement a terrain node, can we make it general purpose enough that it would work for every user who needs a terrain? The answer here is likely no. Even if we have a great general-purpose implementation, many users will likely need something more catered to their use case and will be unable to use the feature as is.

4. How much demand is there for this feature?
A lot :)

5. What is the benefit to being in core rather than as an addon?
This is a rephrasal of the last question in the template. It is not enough to say that it is easier for your game to have the feature. Similarly, discoverability is not an adequate response. If something can be an addon (as terrain already is), the only reasons to bring it into core is that the cost of being an addon makes the feature prohibitively slow, or that the feature cannot be fully implemented and is limited as an addon. Zylann has said before, he could make his terrain _faster_ by moving it into core, however, at this time, it looks like his addon is fast enough. And would likely see the same speed benefits by moving to GDNative.

In consideration of all the above, it doesn't make sense to make terrain a core node. If you need a prebuilt terrain node, use the one that Zylann makes. It is extremely well made and should have all the features you need.

Further, Juan has explained in the past that he has no intentions of adding a terrain node for 4.0. So don't get your hopes up for that. However, in my opinion, once 4.0 is out, if we can show that a general purpose terrain node could be made sufficiently fast and user friendly, I think we can make a convincing argument to make one for 4.1 or 4.2.

[MostafaMTH]

That seems like a good plan, lets hope that Juan is convinced and releases it with 4.xx

[Calinou]

One reason to let the community handle modules like terrain is that it lets the Godot team better focus on core tasks. If we add too many modules into core, they can end up being poorly maintained as none of the core developers would have the time or skills to dedicate to it. The CSG module is one such example: Qodot has been largely exceeding its feature set and usability by now.

I think this is especially valid as there's not "one true way" to make terrain: heightmaps, marching cubes, chunking/LOD mechanisms, … This only adds up to the implementation complexity. This is why one of Godot 4.0's goals is to be more extensible (e.g. overrideable rendering pipeline, new plugin system).

@Xrayez I can see value in providing an official minimal build variant, but I get a feeling some people will be confused as to why some features aren't working when they're running a minimal build. The minimal feature set would also be a subject of hearthy debate :slightly_smiling_face:

[Zylann]

As far as I understand, this proposal is actually asking for consideration of a terrain system, and does not actually propose any. There are indeed several "main" ways to create 3D terrains, and there is a most common one, so I suppose we are talking about heightmaps here.

a terrain system can work as an addon (see Zylann's incredible terrain addon).

After a long series of PR-ing features between 2.1.5 and 3.1, yes it does, by using a 10-years old technique in a pretty bold way (and more features are still needed :p), but...

he could make his terrain faster by moving it into core

This is where the problem is at the moment^^ It's an old technique, and the editor brushes don't scale well. Lots of operations to do, and not just painting images. Such system doesn't only render a terrain, but also supports undo/redo, calculates bounding boxes for Godot's culling to not go crazy, updates LOD at runtime in 3D using those boxes by showing/hiding meshes (so only one player), bakes normals in a custom way to avoid bias, copies heights around to build colliders, updates grass chunks around the camera etc... and I didn't get to layers, trees, rivers and roads yet. All this multiplies quickly because terrains are often really big continuous objects.

And would likely see the same speed benefits by moving to GDNative.

I would like to, but right now GDNative isn't very attractive. It is technically a very good solution, but currently it's lacking (stability, availability, maintainers, docs).
C# is a compromise, but is everyone happy to depend on Mono for this? Careful, it's contagious :p

If we add too many modules into core, they can end up being poorly maintained as none of the core developers would have enough time to dedicate to it.

True. I fix bugs in mine when they are reported, but lately I worked on other things and I'm not even fulltime, so I couldn't improve it or add new features. There will be improvements, but at the current pace it's just going to take a long time (as in months, years).

there's not "one true way" to make terrain: heightmaps, marching cubes, chunking/LOD mechanisms

Actually you described two common groups of terrain: voxel, and heightmap. First one is marginal and so complex that it makes sense to not be in core. Blocky version of voxels is actually in the engine already: it's called GridMap and is built-in, although it doesn't scale well when going big.
Heightmaps however are very common, and in both cases they are chunked/LODed in some way or another (otherwise we'd continue using mega meshes :p).
I'd say the main bugger for implementation choices is GLES2 vs Vulkan. Completely different constraints, enough to justify entirely different systems (or not even bother doing GLES2).

Overall... I just feel a lot of time will pass before we get more terrain stuff, but Godot 4.x series have lots of features incoming which may allow a few steps forward.
There can be a lot to dig into when making terrains, and I may have repeated a few things I expressed already in my repo's issue tracker (which you can also look into if you want to see techniques/problems/good/bad ideas).
Community is what makes things advance in this case. People who want, can, or even need this, AND want to do it in Godot. Apart from me and Bastiaan, I wonder if someone else made one, when I type "terrain" on the assetlib I feel lonely^^"

[Xrayez]

One reason to let the community handle modules like terrain is that it lets the Godot team better focus on core tasks. If we add too many modules into core, they can end up being poorly maintained as none of the core developers would have the time or skills to dedicate to it.

If a feature is at least 30% useful and Godot has enough userbase, I feel like there would be a high chance for someone to chime in and contribute to the development, not mentioning that including something in core already makes the features popular without investing much into advertisement. One example is how visual shader got a complete overhaul by @Chaosus as if out of nowhere. I'm not sure whether visual shader was developed by someone outside of the core team, but the shader code just got commented out when we moved to 3.0 despite of it being developed by core devs if that's the case.

Sure, you wouldn't see such people as one of the "core developers" at first, but it's a matter of delegating just enough of decision making without having to worry what other core contributors would say as their expertise might not even cover the feature set, but otherwise useful for many other people who don't see their development needs as specific. Because when you use something for a long time, it's difficult to call something "specific" in terms of a project, and not in terms of everyone's needs. Most people don't really care whether their desired feature would be used by many people or not, they just express the need which is quite real to them.

[MostafaMTH]

I think that if the terrain system was merged to core but as a plugin then it will be faster and not everyone will have to have it installed or maybe just have it merged to core and there by default in the node tree, it will only cost a couple of MB.It will be more user friendly if its there by default because they wont have to go look for it in the plugins

[Calinou]

@MostafaMTH Please use the Edit button instead of replying several times in the same issue:

[akien-mga]

@jasperbrooks79 This repository is for technical discussion between developers about specific feature proposals, how and why they should be implemented.

Your enthusiasm is good to see but it's not constructive and it's all over the place, so by commenting here you actually divert contributors away from furthering a fruitful discussion on the technical details of implementing specific features. Please only comment if you have actual technical feedback to provide about use cases that you have experimented with and are familiar with.

I removed your above three comments are they are not constructive for this proposal's discussion and are generally off-topic.

[asheraryam]

It's worth also considering implementing vector fields as an addition to the basic height fields.

https://youtu.be/In1wzUDopLM?t=470 (relevant content from 7:50 to 14:00)

This is backwards compatible with height fields, so users can still effectively import height fields and basically use them as vector fields that have 2 out of their 3 channels zeroed out.

So users can still make use of whatever height fields they have from external software, but they have added editing flexibility when treating them as vector fields inside the engine.

[Zylann]

users can still effectively import height fields and basically use them as vector fields that have 2 out of their 3 channels zeroed out

Users have to be well aware of using the feature or not, because 2 unused 16-bit channels (or 32-bit) is a big waste to avoid.

[asheraryam]

There can definitely be an explicit toggle to enabling vector fields, with a warning so that users know it will add an extra overhead.

As for the second point of using a separate texture for the other channels, I think we can delay this optimization for now until we have some test-case numbers to study.

[jasperbrooks79]

Thanks, I am starting to realize how much cool one needs to be, to even contribute to the engine, like have knowledge . . . One needs lot more skill, than I have, with code, 3D, engine . .

Will try to take it elsewhere, I'm too noob, to help much, sry . .

<3 . . Again, thanks for this amazing engine, so much fun, will listen, not talk . .

[ghost]

Thanks, I am starting to realize how much cool one needs to be, to even contribute to the engine, like have knowledge . . . One needs lot more skill, than I have, with code, 3D, engine . .

Will try to take it elsewhere, I'm too noob, to help much, sry . .

<3 . . Again, thanks for this amazing engine, so much fun, will listen, not talk . .

ah that's too bad, i really wanted to see your feedback... it sucks that your comments were removed