Vega-strike-engine-source: Investigate alternative game engine technologies

[nabaco]

As discussed in the last community meeting, we want to drop OGRE code, and consider future integration with a different game engine.
This issue is to discuss with alternative game engine we should choose.

-- UPDATE --
Currently, the options are:

• Game Engine:
  
  
  
  • [ ] Replace the VS game engine with Godot engine.
  
  
• Replace the Render Engine:
  
  
  
  • [ ] Direct Vulkan integration instead of OpenGL.
  
  
  • [ ] Replace just the render engine with BGFX.
  
  
  • [ ] Replace just the render engine with OpenSceneGraph.
    
    
    
    
    
    • [ ] Should look at VulkanSceneGraph too as it is replacing OpenSceneGraph.
    
    
    
  
  
  • [ ] Replace just the render engine with DiligentEngine
  
  
• Other:
  
  
  
  • [ ] Replace SDL and FreeGLUT with GLFW/GLFWM.
  
  
  • [ ] Add ASSIMP for better external tooling support.
  
  

[BenjamenMeyer]

Sorry I missed the meeting; Q: Isn't VS its own game engine?

[pheonixstorm]

Yes

[nabaco]

Yes VS has a custom game engine. But that brings several issues.

1. Currently, it's based on old libraries (e.g. opengl1) and implements some low-level code, which turns this into a real maintenance challenge. Add to that poor documentation and lack of developers that know what's actually happening there.
2. Since it's a custom engine, it requires custom tools for content creation (see the mesher tool as an example), which need to be maintained too.
3. The previous two points, also add a higher entry barrier for new content creators - which is the number one resource that we lack currently.

Porting the code onto an existing game engine means that we will need to deal much less with rendering issues, low level libraries etc.
An existing engine brings with it a community that can support us, documentation, possibly developers that will join us that already know it, etc.
Additionally, most modern game engines are able to use content created by tools that are widely accepted in the industry today (blender, maya, etc).

I guess that also defines in a way the acceptance criteria.

All of this is based on my personal understanding, correct me if I'm wrong.

[BenjamenMeyer]

So for 1 - that's the nature of a game engine.
For 2 and 3 - perhaps we could integrate with Gimp and Blender to do some of that work?

Not sure there are many open source game engines that do what VegaStrike does, or could easily be extended to do so. There's already projects out there that use VS as a backend too, so:

• We need to ensure we maintain some support for older tooling, even if that's a means of importing it into a newer tool chain so that we can help downstream projects update to our latest stuff.
• We should do what we can to move to more common tooling (Gimp, Blender, etc) so that folks can easily use us without large learning curves
• We should still continue to be a game engine in our own right, albeit one focused on the Space warfare style that VegaStrike has long run.

Thoughts?

[BottledByte]

If I may join to the discussion, I have some thoughts on this.

First about the Godot engine:
I have been investigating open source game engines for a long time now and have some knowledge about (and personal experience with) Godot:
1) Godot is a general-purpose game engine (like Unity is, for example), therefore it lacks some optimizations that might be desired for a space game like VS is. (but it is an open source, so it can be hacked... :) )
2) It has big support, evolves fast and supports more modern rendering backends (OpenGL ES 2, 3) than current VS's game engine does (OpenGL 1.x). There is some plan that it will support Vulkan as well in the future.
3) It is designed to support a lot of platforms. (Even it is sometimes a bit difficult to export to some platform properly)
4) Even though that Godot has C++ bindings, it is really designed to use it's own GDScript (Python derivative) and C# (through Mono) for programming. As far as I know, there does not exist a pure C++ game written with Godot. Unless this project wants to become a pioneer in this field.... :)

As of Vulkan?
It is a game graphics API of the future, but Vulkan is entirely different beast than OpenGL. Furthermore, it is supported on modern hardware only (some 2015 or so, I think), so if used exclusively, you loose a lot of backward compatibility. I am no computer graphics programmer, but I estimate that cost of creating and maintaining Vulkan backend for VS's rendering engine is somewhat high, especially if OpenGL (hopefuly more modern version of it) would be maintained at the same time.

About external tooling:
How sad this may be, most projects based on the VS I have seen... appear to be dead or in "super low maintenance mode" (just as VS itself was :) ). It is very important to have support for modern tools such as Blender and backwards compatibility is not that important. If different game engine or rendering engine is used, a one-time mesh/texture converter to new tools is a must-have, as the meshes existing already are a precious resource. Or, as an alternative, if VS game engine is to be kept, create a Blender plugin for conversion between Blender and VS's custom mesh tools (but that means that the plugin and tools have to be maintained).

Summary of opinions on different "engine" selection:
1) Ogre3D is an old 3D rendering engine. While it is maintained, it supports only somewhat older graphics APIs (no good Vulkan support in near future, they are in super-early alpha stage).
2) Godot, on the other side, is a modern game engine. But because of this, it does far more than just graphics and integration would be an extremely costly process (as VS has a game engine). Using it would probably mean to replace the current game engine completely.
3) Vulkan is not an engine, just an graphics API. While improving VS's rendering engine with a new backend might be a great performance gain, it also requires some maintenance.
4) Also, if the VS game engine is to be kept, but only it's rendering engine is to be replaced, there is something called bgfx that might be a good solution to "poor rendering backend" problem. (I am not able to estimate it's cost, someone else should check it)

Question I have is: "Is this about dropping/changing a game engine (rendering, physics, data loading, neworking....) or a rendering engine (rendering and meshes/textures)?"

If this is about changing a game engine (to Godot engine or something) as this issue name suggests, I think that there is __no need to refactor__ current code at all, as __starting from scratch is probably a lot easier__.

PS: I was writing this for some two hours, just to summarize things a bit :)
These are just my two cents and I if you recognize some of my points as too radical, please do not take it an offensive way. I just wanted to present some "economically based" ideas / view and fix terminological confusion.

Edit: BGFX is the name of the library, not BFGX. Sorry for the typo in such an important place :). Fixed already.

[Loki1950]

My main concern with Godot is can we have our huge star systems to proper scale using it.We currently use Python 2.7 for a script language and for the GUI it's EOL so GDScript could be a replacement if we go with Godot BTW bfgx looks nice too now we really need a handle on asset creation pipeline I'm a basic level blender user and also with https://github.com/makehumancommunity/makehuman as a forum moderator the assets code is currently being looked at refactoring the unit class.

[BenjamenMeyer]

@BottledByte thanks for the info. I'll have to digest it some.

I will say that from a variety of perspectives I will be opposed to adopting .Net/Mono, not the least of which is licensing and parents. So from that perspectives I would be opposed to using GoDot as a backend. It would almost certainly be easier to start from scratch for it too given the myriad of differences between C, C++, and C# - including many subtle keyword differences.

I think we can port from Py2 to Py3 without much issue. I can walk folks through the process there, it's not terribly hard just a bit of work.

I would keep this to the rendering side of things. Otherwise what value is Vega Strike bringing?

More later when I'm at a computer instead of my phone.

[nabaco]

@BottledByte, thank you for such a detailed digest. It does make a lot of things clearer.
In light of what you have written, I am more inclined towards the BGFX option + Plugins for toolings.

I have updated the initial issue description with the new info/options.

[Loki1950]

I also am inclined to BGFX it may help us future proof the engine

[bmorel]

For the 3D graphical tooling problem, libassimp could be used. It's portable, does nothing with rendering, supports STL, OBJ and other formats. Bones stuff, too, IIRC.

My opinion is that generally, projects should rely on libraries, and not on frameworks or full built engines, if the long term existence is a goal. It's easy to replace a library, hard to replace a framework. A 3D rendering engine is almost a framework, and does not do all the jobs: there is still 2D (HUD), physics, sounds, etc.
Game engines does everything. From what I've read, they often are based on a specific languages, sometimes homemade (neverwinter nights, unreal engine), and when not, adds it a framework.
Using a common language (C, C++, java, C#, whatever) without framework implies that any dev that know that language can jump in without spending weeks or days learning a framework or specific architecture.

The annoying thing about OpenGL is actually how to build HUD. There are libraries around, but not much. They are often based on XML/html stuff, which might be a blessing or a curse, depending on your opinion.
I may still have some links around, have to dive into personal history to find those...

In short, I'd say, better to 1st refactor VS engine, when the code will be easier to maintain then move to more modern rendering libraries (openGL3+, vulkan, whatever), libraries specialized into mesh loading, etc. Or maybe frameworks. Your choice.
Also, you might be interested by the fact OpenMW project have moved out of Ogre3D toward OpenSceneGraph. I'm not exactly certain of the reason, but I think it was because there was problems swtiching parts of a model, like, heads, swords ~, lasers~...

[edit]
Also, I forgot about that. Current VS seems to use both SDL and FreeGLUT. FreeGLUT being based on that nasty callback hell, I don't really see many SDL calls there.
Since this thread is about changing underlying stuff, may I suggest taking a look at glfw3 to manage the whole windowing and input stuff?
Of course, i'ts only suggestion. This is a library that does only windowing and input processing, it does not even try to do rendering, but since we are using openGL for that anyway...

[nabaco]

Thank you @bmorel for the precious input. I have updated the issue description to unclude all of the discussed options.

[BenjamenMeyer]

@nabaco I added VulkanSceneGraph to the list since you have OpenSceneGraph in there and it's suppose to replace OSG.

[BottledByte]

I have further findings and notes about this issue. I have seen your recent conversation at Gitter (but do not expect me to join directly, I am not an IRC/Gitter fan), so I will clarify some things.

@BenjamenMeyer Godot has really nothing to do with Mono/.Net/C#. It uses it as a module and it is entirely optional.

I have seen that there was a discussion about windowing library. BGFX does not do windowing and it recommends to use external libraries for that (such as GLFW).

Another thing. My development machine got disabled yesterday :(, so at least I did some research. Apart from BGFX (which is a rendering API wrapper) there is another similar thing called DiligentEngine. It claims to be more modern than BGFX and designed with DX12, Vulkan and Metal in mind, while not supporting some older APIs. It is under Apache 2.0 License. From my research on BGFX, it turned out that it really has problems with modern APIs. So, if some API agnostic rendering backend library is desired, and it should be a more modern one, it might worth a try. Last note is that BGFX has a long history (so meaning more support and bugfixes), while DiligentEngine is obviously a recent and new thing, so this thing has to be considered.

As of OpenSceneGraph and VulkanSceneGraph, I will take closer look on those. There has to be a reason why OpenMW did the switch to OSG and until my development machine gets back into functional state, I will do some research (at least).

And last thing. Could anyone please change this issue title? It seems that this is not really about alternative game engines, but more about alternative game engine technologies. It is just my obsession with terminology :)

[BenjamenMeyer]

@BottledByte I'll stand corrected on that - my initial look earlier showed a link, but I can't find that right now. They do support C#/Mono. That said, it's still more of a competitor to what Vega Strike does since it's a framework. Likewise Vega Strike is more of a framework too, just more specialized around a certain style of game play. I'd favor using a library for the graphic integration/support to help with that, but we should still remain our own thing - a game engine.

[DMJC]

For the 3D graphical tooling problem, libassimp could be used. It's portable, does nothing with rendering, supports STL, OBJ and other formats. Bones stuff, too, IIRC.

My opinion is that generally, projects should rely on libraries, and not on frameworks or full built engines, if the long term existence is a goal. It's easy to replace a library, hard to replace a framework. A 3D _rendering_ engine is almost a framework, and does not do all the jobs: there is still 2D (HUD), physics, sounds, etc.
Game engines does everything. From what I've read, they often are based on a specific languages, sometimes homemade (neverwinter nights, unreal engine), and when not, adds it a framework.
Using a common language (C, C++, java, C#, whatever) without framework implies that _any_ dev that know that language can jump in without spending weeks or days learning a framework or specific architecture.

The annoying thing about OpenGL is actually how to build HUD. There are libraries around, but not much. They are often based on XML/html stuff, which might be a blessing or a curse, depending on your opinion.
I may still have some links around, have to dive into personal history to find those...

In short, I'd say, better to 1st refactor VS engine, when the code will be easier to maintain then move to more modern rendering libraries (openGL3+, vulkan, whatever), libraries specialized into mesh loading, etc. Or maybe frameworks. Your choice.
Also, you might be interested by the fact OpenMW project have moved out of Ogre3D toward OpenSceneGraph. I'm not exactly certain of the reason, but I think it was because there was problems swtiching parts of a model, like, heads, swords ~, lasers~...

[edit]
Also, I forgot about that. Current VS seems to use _both_ SDL and FreeGLUT. FreeGLUT being based on that nasty callback hell, I don't really see many SDL calls there.
Since this thread is about changing underlying stuff, may I suggest taking a look at glfw3 to manage the whole windowing and input stuff?
Of course, i'ts only suggestion. This is a library that does only windowing and input processing, it does not even try to do rendering, but since we are using openGL for that anyway...

Vegastrike doesn't use FreeGLUT normally, vegastrike can use FreeGLUT when SDL isn't available. The engine was designed around portability to as many platforms as possible as a priority. There's a planetary terrain engine based on heightmaps that most people aren't aware of in there as well. On Ogre: The API keeps changing, and I don't consider Ogre engine to have much benefit. I've got an old build of vegaogre running in a virtual machine but I've been unable to forward port it to modern Ogre. No attempt at modernising the Ogre rendering has succeeded. One approach I was considering to the graphics engine was to get rid of the reliance on quads/polys and get the engine updated to using triangle rendering. Once that's done I was going to look at moving to arrays/array objects, and then modernise to GL4.

[BenjamenMeyer]

Just something to think about...

It'd be neat if we could publish UtCS on Steam (there are free games) as it could drive a signficant user base; but our dependencies would have to fit within what the Steam Client provides (see https://partner.steamgames.com/doc/store/application/platforms/linux). I think we're actually pretty good there.

Mostly mentioning this as we think about what dependencies to use or what libraries to adopt in this discussion as if we do want to do that it'd be something to consider. If we don't, then it doesn't matter.

[bmorel]

Another one maybe for the list. Few days ago I found a thing named "raylib" in my contrib folder. Got a new quick look on it, and the code seemed relatively clean. Also ran a valgrind on some examples for fun, unfortunately if found some problems, some memory leaks IIRC.
I didn't invest time in actually finding those or even reporting, I am still busy on other stuff, but I still think it might be an interesting tool, especially if some parts can be used without others. The main drawback I'd see in it is: it is based on glfw3, where VS is based on SDL1.2. Just thought it could be mentionned.

[HeadClot]

On the topic of Godot - Wait for 4.0 which should be out hopefully in late 2020 or early 2021.

Currently it does not have occlusion culling, double precision support, or working LOD support in the current stable version 3.2. However 4.0 should bring those things.

It does have allot of great tooling and those things I listed can be added fairly quickly as they are either already written or are going to be added by the Godot team.

I know some one was working on a Quad-Tree based planet system in Godot. If we choose Godot engine I will ask them if they would be willing to release it as open source. :)

One last thing - We will need to have some form of path finding system that works in 3D space if we choose Godot. I assume we have one already but I am just raising that concern as Godot engine only does navigation for flat surfaces using Recast and Detour. :)

[HeadClot]

One last thing.
We should also investigate Torque3D 4.0 you can find it here.

Torque3D 4.0 has PhysX support so we can shift the world origin as needed to avoid precision issues. It has very good networking support. It powered Tribes, Earth Siege, and Star Siege back in the day. It went up to 128 players or more. Torque 3D has occlusion culling support, A modern rendering pipeline with PBR support, LOD Support, and a modern level editor.

[BenjamenMeyer]

While I know we have a long discussion here, this issue has also become a little bit confusing as it has discussed the possibility of entirely replacing the game engine which we will not do.

So to reduce confusion I'm closing this out in favor of #170 which will be focused on the outcome of this discussion - updating the render engine.