Godot-proposals: Add or document a way to write unit tests for Godot C# projects (e.g. with nunit3)

[dsge]

Describe the project you are working on:

A 3D spaceship/spacebase building game (think Space Engineers)

Describe the problem or limitation you are having in your project:

The more code you write to do custom calculations the more cumbersome it becomes to keep looking back to see if previously written code still functions as expected. Adding more code or refactoring the current code is also much easier if you can build on properly (unit)tested foundations.

As an actual example from my current project, I have to calculate how fast my spaceship can accelerate/decelerate given:

• the current power level of the spaceship
• thrusters power consumption
• number of thrusters
• thrusters maximum performance in a given direction
• ...and possibly more factors

Manually testing all of these again and again (and again) is possible, however it's time consuming and inefficient.

Describe how this feature / enhancement will help you overcome this problem or limitation:

Proper unittests can save a lot of time by quickly telling me if my (new or old) code still functions as expected, thus saving me time that can be instead spent on new features.

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:

While having a GUI is nice and all, I think CLI support is the more critical for this particular feature. Running the tests locally and in CI can eliminate further bugs while also allowing for better cooperation on the project with other contibutors.

Workflow in case it is possible to run tests directly using nunit3-console.exe:

• (build the game dll using Godot editor)
• ./path/to/nunit3-console.exe /path/to/game.dll --where "<filter for testcases>" --pause
  
  
  
  • the where option allows to filter for testcases (for when you only want to run specific ones instead of all of them)
  
  
  • the pause option allows to attach external debuggers to debug test cases
  
  
• nunit3-console.exe outputs the tests results and returns 0 if all tests passed (for CI support)

Workflow in case the tests need to be ran through the Godot editor's CLI:

• (build the game using Godot editor)
• ./path/to/Godot_v3.2-stable_yourplatform --path "/path/to/game" --run-nunit3-console --where "<filter for testcases>" --pause
  
  
  
  • the where option allows to filter for testcases (for when you only want to run specific ones instead of all of them)
  
  
  • the pause option allows to attach external debuggers to debug test cases
  
  
• Godot_v3.2-stable_yourplatform outputs the tests results provided by nunit3-console.exe and returns 0 if all tests passed (for CI support)

Describe implementation detail for your proposal (in code), if possible:

Unfortunately I only have limited understanding on Godot's source code or how c# (mono) support is implemented, so I do not know what needs to be implemented or changed to allow this (if anything). If using nunit3 is already possible, then this feature request is actually a request for documentation/examples.

What I do know however is that right now if you try to run your game dll with nunit3-console.exe your testcases will actually work until your testcase reaches the first call to Godot (more specifically to anything inside GodotSharp.dll). You will be bombarded with error messages like this:

cant resolve internal call to "Godot.NativeCalls::godot_icall_Camera_Ctor" (tested without signature also)

Your mono runtime and class libraries are out of sync.
The out of sync library is: /app/.mono/temp/bin/Tools/GodotSharp.dll

When you update one from git you need to update, compile and install
the other too.
Do not report this as a bug unless you're sure you have updated correctly:
you probably have a broken mono install.
If you see other errors or faults after this message they are probably related
and you need to fix your mono install first.

What this error message actually means (despite what it says) is that the c# function called Godot.NativeCalls::godot_icall_Camera_Ctor is defined as an extern function in GodotSharp.dll but no actual implementation is found anywhere. The same code (same project) does work as expected when exporting/running it through Godot Editor.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

Not that I am aware of.

Is there a reason why this should be core and not an add-on in the asset library?:

I think at least documentation on this should be included within Godot's official documentation, even if any implementation is outside of the main source code. The reason for this is to avoid confusion by providing a singular official way for running c# unittests.

[van800]

Normally tests are executed directly in TestRunner process. This causes any unmanaged calls to Godot api-s to fail. So if you want to test a piece of logic without calling unmanaged Godot api-s it would work. Basic example https://github.com/van800/SkyOfSteel/commit/6515c3fdd508083950d57300f1f6bf25e08a0025

I have seen 2 different implementations, which overcome this limitation:

1. TestRunner has communication channel with Application via network and tells it to load some assembly and execute some specific piece of code. http://dev-in-test.blogspot.com/2014/04/resharper-integration-tests.html
2. TestRunner is embedded in the Application and tests are executed in the application process. For example Unity does that. It provides API for a plugin to subscribe to future test-results and run tests. Like this one https://github.com/van800/com.unity.ide.rider/blob/master/Packages/com.unity.ide.rider/Rider/Editor/UnitTesting/RiderTestRunner.cs#L67

Both approaches are not easy/fast to be implemented.

[shianiawhite]

This seems like it should be a pretty important issue. Most serious projects will probably need to have unit tests, and having the standard unit testing tools available seems essential. Even for my little hobby project in Godot, I feel quite hindered by not being able to easily use standard style C# tests. It's been making me reconsider if Godot is really the engine I should be using, or if I should be switching to something that better enables regular coding practices. I can only imagine how much this deters more serious game developers from using Godot. This is probably too extreme of a blanket statement, but typically more experienced developers are more likely to rely on unit tests. And having experienced developers choose Godot as their standard tool will certainly help Godot move forward.

[martinruefenacht]

I can only agree with @shianiawhite, this issue is single-handedly the reason why I switched from Godot to Unity. Even a temporary, before full integration, documented path would be useful to have.

[Calinou]

@martinruefenacht Please don't bump issues without contributing significant new information. Use the :+1: reaction button on the first post instead.

[evan-boissonnot]

So, I'm sorry, I don't understand the release plan about this feature ?

• How could I unit test now my godot C# project ?

Is there a way I can do it until you implement in Godot ?

• What do you plan to do ?
  > Create option in IDE (in the menu) will be great
  > For example : stop launch and run the UI without green unit tests ?

[CodeDarigan]

So this is self-promotion but it is also very relevant to the issue at hand.

I ran into this issue recently myself when I wanted to change from GDScript to C#. I decided to wrap WAT (The Godot GDScript Plugin) in C#. You can find the reddit thread with more detailed information and links here.

This has a GUI, CLI, Parameterized Testing & a number of other features available. Since it is built mainly using GDScript (with C# on user-interfacing scripts) it doesn't run into the same errors as the more established C# Frameworks meaning you can test any Godot Class without worry.

With that said I do think it is probably a good idea to implement some form of communication to the godot api if 1) it is relatively painless to implement 2) is generic so this fix can allow Godot to work with any standard c# test framework.

[shianiawhite]

There were a couple comments that involved something to do with the UI. I just wanted to clarify that (I think) the main part of this issue requires no sort of UI changes. It's just that a test runner currently can't call any Godot related code, and for the tests to be run, the runner needs to be able to make those calls. I suspect a majority of the people who would be writing tests would be using an external IDE anyway (though I certainly may be wrong).

[van800]

I managed to make a proof-of-concept integration Rider UnitTestRunner with Godot.

With the following changes:

1. On Rider plugin side https://github.com/JetBrains/godot-support/pull/58
2. On the game side
   https://github.com/van800/godot-demo-projects/pull/4
   Both plugins are open-source on github, anyone can try.

Still some work ahead.

1. Make debugging work
2. Make runner.tscn an invisible commandline, if possible.
3. Integrate EntryPoint.cs and runner.tscn into main Godot, so users wouldn't need to copy them manually inside their project.

If I make a PR which solves (3), would it be welcomed? @neikeq
Should I create a separate proposal or we can discuss here?

[neikeq]

1. Integrate EntryPoint.cs and runner.tscn into main Godot, so users wouldn't need to copy them manually inside their project.

If I make a PR which solves (3), would it be welcomed? @neikeq

Yes, that would be very welcome!

[van800]

_Good News and Bad News:_

1. Plugin is working, it allows running and debugging NUnit tests.
2. I would not be able to release it with Rider 2020.3, because it conflicts with bundled Unity support plugin.

Would be cool though, if someone would start using it and let me know how it goes. I have prepared full instructions on how to do it.