Godot: Performance increase on Windows

[Zylann]

I just ran quick tests in GDScript to see how fast it performs, things like that:

    _time_before = OS.get_ticks_msec()

    start()

    for i in range(0,ITERATIONS):
        # Simple code

    _for_time = stop()
    print("For time: " + str(_for_time))

And I quickly realized that the version of Godot I built is 5 times faster than the official 2.0.2.
I didn't modified anything in SCons or Godot, just ran a build after following this tutorial from GamesFromScratch: http://www.gamefromscratch.com/post/2015/06/18/Godot-and-Visual-Studio.aspx

As discussed on IRC, this could be caused by the use of another compiler.
So I tried with prior versions, and 2.0.0 is also 5 times faster than 2.0.1 and 2.0.2.
I personally used msvc12, but msvc14 could be even better.

If we can get such performance gains, would it be possible to provide builds made with these compilers on the appropriate platforms?

[akien-mga]

Are you build with tools or only the template? And release_debug or debug? Are you running your script in the editor or in an exported game?

There might be compiler-specific differences, though 5 times faster seems quite surprising. We'd need someone to do a real comparison of binaries made with MSVC, MinGW on Windows and MinGW on Linux, all built on the same machine with the same arguments, so that we can properly compare their performance.

The 2.0 binaries were also built on Linux with MinGW AFAIK, but on a different server, so potentially different version of the compiler.

[slapin]

print is a bad benchmark, Windows console is slow as hell.

On Thu, Apr 14, 2016 at 8:38 AM, Rémi Verschelde [email protected]
wrote:

Are you build with tools or only the template? And release_debug or debug?
Are you running your script in the editor or in an exported game?

There might be compiler-specific differences, though 5 times faster seems
quite surprising. We'd need someone to do a real comparison of binaries
made with MSVC, MinGW on Windows and MinGW on Linux, all built on the same
machine with the same arguments, so that we can properly compare their
performance.

The 2.0 binaries were also built on Linux with MinGW AFAIK, but on a
different server, so potentially different version of the compiler.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-209771902

[Zylann]

I compiled with Debug configuration, from Visual Studio (which runs SCons in the background anyways).
I am running the script in the editor. I also tested in an exported game (official build) and performance is roughly the same.

build.bat command line:

set vc_path=%1
call %vc_path% & scons platform=windows

@slapin printing is not included in the benchmark, I run my tests inside a for loop with a lot of iterations, so Windows console has nothing to do with the speed. Despite a quite naive method, it gives a quick preview, and 5 times is a lot for a preview.

Here is the project I run: http://zylannprods.fr/dl/godot/GDScriptPerformance.zip

[volzhs]

I have this experience too.
I tested similar way as this post.

I got slow performance with windows tool built on Linux, MinGW.
I got better performace (about 5~7 times faster) with windows tool built on Windows, MSVC 2015 community edition.
And tool binary size is much smaller, it's about 17MB.

[Glissando]

Same on my end, using MSVC14

[fracteed]

Damn, that is impressive! Sounds like the official builds for Windows should definitely be built on Windows :)

[slapin]

or optimizations should be enabled.

On Sat, Apr 16, 2016 at 3:47 AM, James Redmond [email protected]
wrote:

Damn, that is impressive! Sounds like the official builds for Windows
should definitely be built on Windows :)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-210699151

[Zylann]

As akien-mga said, it could be more than just Windows.

@slapin I checked the compilation log, here is a sample of what it says:

cl /Focore\array.windows.tools.obj /c core\array.cpp /TP /TP /nologo /DFREETYPE_ENABLED /Z7 /DDEBUG_ENABLED /DDEBUG_MEMORY_ENABLED /DD3D_DEBUG_INFO /Od /MT /Gd /GR /nologo [...]

There is the /Od flag on, which means no optimization. https://msdn.microsoft.com/en-us/library/k1ack8f1%28v=vs.120%29.aspx
Performance could then be even better in release mode :]

[techtonik]

That benchmark would be nice to have from cmdline.

Stable release binaries are compiled with gdb symbols. I don't know how it affect performance, but it makes debugging godot crashes much easier. As easy as r and bt.

[DriNeo]

This is surprising. It could be interesting to run other benchmarks.

[vnen]

Stable release binaries are compiled with gdb symbols.

Only the editor IINM, to allow in-editor debugging. The export templates have no debug symbols.

Anyway, it is interesting to have more benchmarks.

[slapin]

About debugging symbols - they don't affect performance at all. But Godot
debug biaries are built without optimizations, too (that makes debugging
easier
because it is easier to factor-out problem code in this case).

On Mon, Apr 18, 2016 at 2:38 AM, George Marques [email protected]
wrote:

Stable release binaries are compiled with gdb symbols.

Only the editor IINM, to allow in-editor debugging. The export templates
have no debug symbols.

Anyway, it is interesting to have more benchmarks.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-211128823

[alexisgea]

I don't know how to compile Godot but I would like to confirm I noticed the same performance drop from 2.0 and 2.0.2 (I was sent here from reddit) : https://www.reddit.com/r/godot/comments/4fjlua/performance_drop_with_latest_update/

[Zylann]

I created a repository with my tests: https://github.com/Zylann/gdscript_performance

[mattiascibien]

Would it be a good idea to add appveyor as a build server to prevent cross compile?

[akien-mga]

Would it be a good idea to add appveyor as a build server to prevent cross compile?

Yes that's something that we should look into. I'd prefer having everything done in Travis CI, but if there's a real performance loss due to cross-compiling, doing native builds with AppVeyor is the way to go.

[mattiascibien]

@akien-mga I am actually investigating on an appveyor.yml for building godot with appveyor. Should build both x86 and x64 for both release and debug right? PlutoBoy seems to have a good appveyor.yml for building something with scons. I will try something in my fork

[akien-mga]

@mattiascibien It should be the tools (tools=yes) with target=release_debug, and the export templates (tools=no) with target=release and target=release_debug. And that for both bits=32 and bits=64 yeah (though the bits argument just got removed in #4274 for the master branch, but it's still in use in the 2.0 branch for now).

IIRC @Marqin also started looking into building Godot with AppVeyor, he might have a WIP config in his fork too.

[mattiascibien]

Ok... It seems that @Marqin's work is definetely almost complete i guess. I'll leave it to him then, sorry for the duplicate.

[Marqin]

Yep, but my work is not compatibile with #4274 ( I'm using my own changed detect.py there )

[techtonik]

I'd like official builds to come with gdb support regardless of performance.

[Zylann]

@techtonik on Windows? Anyways, as akien-mga said, 2.0 was built with MinGW so using the right version rather than switching to msvc could also bring better results. I'm curious :)
I would prefer using Visual Studio to debug on Windows (the debugger is great and command line is crappy on Windows), even though I mostly debug with a custom built Godot.

Also, I should note that my benchmarks only concern GDScript. I'm not sure if the ratio is the same for native code, I don't have the same environment setup to test cross-compile with MinGW.
Benchmarking C++ could give a clue about what is actually slower, maybe the way memory is allocated, or how well GDScript is optimized by the compiler? (because obviously, GDScript does different things than the graphics engine).

[techtonik]

@Zylann yes, on Windows. Getting into gdb and getting the right stacktrace in copy/pasteable format is so much easier than trying to find your way through all the details. I am not sure if same Linux tools can be used to profile godot binaries there.

BTW, can we collect benchmark results on https://github.com/Zylann/gdscript_performance page? Using downloadable version and custom one with and without debug. Godot still fails to run on my system without ANGLE, so I can't test it myself for the moment.

[Marqin]

@mattiascibien I've just PRed working appveyor.yml to godot-builds repo - https://github.com/GodotBuilder/godot-builds/pull/1

[akien-mga]

Could you guys try the Windows binaries from https://github.com/GodotBuilder/godot-builds/releases/tag/master_20160512 ? There's only the 64-bit Windows editor right now, but other binaries should arrive as AppVeyor builds them with MSVC and uploads them (the other binaries are from Travis).

You could then compare with the Windows binaries from https://github.com/GodotBuilder/godot-builds/releases/tag/2.0_20160512, which were cross-compiled on Travis CI with MinGW. Those are from the 2.0 branch while the former are from the master branch, but performance wise it shouldn't make a big difference.

[Zylann]

Just tried, here are my result times for GDScript, with the tools, 64 bits:

Binaries from https://github.com/GodotBuilder/godot-builds/releases/tag/master_20160512 (MSVC):

Empty func (void function call cost): 82
Increment: 34
Increment x5: 173
Increment with member var: 34
Increment with local (outside loop): 32
Increment with local (inside loop): 61
Increment Vector2: 36
Increment Vector3: 36
Increment Vector3 with constant: 34
Increment Vector3 coordinate by coordinate: 933
Unused local (declaration cost): 8
Divide: 58
Increment with dictionary member: 149
Increment with array member: 60
While time (for equivalent with manual increment): 84
if(true) time: 17
if(true)else time: 19
VariantArray resize: 959
VariantArray set element: 19
IntArray resize: 342

Binaries from https://github.com/GodotBuilder/godot-builds/releases/tag/2.0_20160512 (MinGW cross-compiled):

Empty func (void function call cost): 517
Increment: 548
Increment x5: 2765
Increment with member var: 549
Increment with local (outside loop): 546
Increment with local (inside loop): 744
Increment Vector2: 551
Increment Vector3: 539
Increment Vector3 with constant: 550
Increment Vector3 coordinate by coordinate: 6101
Unused local (declaration cost): 170
Divide: 564
Increment with dictionary member: 1484
Increment with array member: 957
While time (for equivalent with manual increment): 1032
if(true) time: 118
if(true)else time: 125
VariantArray resize: 1644
VariantArray set element: 342
IntArray resize: 2417
IntArray set element: 1509

My own custom build (quite old now, I should git pull):
_note: this is debug mode too, and optimizations are off_

Increment: 54
Increment x5: 330
Increment with member var: 65
Increment with local (outside loop): 55
Increment with local (inside loop): 98
Increment Vector2: 88
Increment Vector3: 75
Increment Vector3 with constant: 74
Increment Vector3 coordinate by coordinate: 2983
Unused local (declaration cost): 6
Divide: 82
Increment with dictionary member: 435
Increment with array member: 105
While time (for equivalent with manual increment): 171
if(true) time: 31
if(true)else time: 16
VariantArray resize: 9808
VariantArray set element: 36
IntArray resize: 5665
IntArray set element: 615

[slapin]

Well, the results look like MinGW builds do have optimizations switched off
and MSVC builds do optimize.
the increment can't be 10 times slower with gcc :)

What test do you run, I'd like to test locally?

On Thu, May 12, 2016 at 7:37 PM, Marc [email protected] wrote:

Just tried, here are my result times for GDScript:

Binaries from
https://github.com/GodotBuilder/godot-builds/releases/tag/master_20160512
(MSVC):

Empty func (void function call cost): 82
Increment: 34
Increment x5: 173
Increment with member var: 34
Increment with local (outside loop): 32
Increment with local (inside loop): 61
Increment Vector2: 36
Increment Vector3: 36
Increment Vector3 with constant: 34
Increment Vector3 coordinate by coordinate: 933
Unused local (declaration cost): 8
Divide: 58
Increment with dictionary member: 149
Increment with array member: 60
While time (for equivalent with manual increment): 84
if(true) time: 17
if(true)else time: 19
VariantArray resize: 959
VariantArray set element: 19
IntArray resize: 342

Binaries from
https://github.com/GodotBuilder/godot-builds/releases/tag/2.0_20160512
(MinGW cross-compiled):

Empty func (void function call cost): 517
Increment: 548
Increment x5: 2765
Increment with member var: 549
Increment with local (outside loop): 546
Increment with local (inside loop): 744
Increment Vector2: 551
Increment Vector3: 539
Increment Vector3 with constant: 550
Increment Vector3 coordinate by coordinate: 6101
Unused local (declaration cost): 170
Divide: 564
Increment with dictionary member: 1484
Increment with array member: 957
While time (for equivalent with manual increment): 1032
if(true) time: 118
if(true)else time: 125
VariantArray resize: 1644
VariantArray set element: 342
IntArray resize: 2417
IntArray set element: 1509

My own custom build (quite old now, I should git pull):

Increment: 54
Increment x5: 330
Increment with member var: 65
Increment with local (outside loop): 55
Increment with local (inside loop): 98
Increment Vector2: 88
Increment Vector3: 75
Increment Vector3 with constant: 74
Increment Vector3 coordinate by coordinate: 2983
Unused local (declaration cost): 6
Divide: 82
Increment with dictionary member: 435
Increment with array member: 105
While time (for equivalent with manual increment): 171
if(true) time: 31
if(true)else time: 16
VariantArray resize: 9808
VariantArray set element: 36
IntArray resize: 5665
IntArray set element: 615

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218813784

[reduz]

compiling godot with debug or tools also adds a large amount of runtime
checks, are you testing both mingw and msvc on release?

On Thu, May 12, 2016 at 2:22 PM, Sergey Lapin [email protected]
wrote:

Well, the results look like MinGW builds do have optimizations switched off
and MSVC builds do optimize.
the increment can't be 10 times slower with gcc :)

What test do you run, I'd like to test locally?

On Thu, May 12, 2016 at 7:37 PM, Marc [email protected] wrote:

Just tried, here are my result times for GDScript:

Binaries from

https://github.com/GodotBuilder/godot-builds/releases/tag/master_20160512
(MSVC):

Empty func (void function call cost): 82
Increment: 34
Increment x5: 173
Increment with member var: 34
Increment with local (outside loop): 32
Increment with local (inside loop): 61
Increment Vector2: 36
Increment Vector3: 36
Increment Vector3 with constant: 34
Increment Vector3 coordinate by coordinate: 933
Unused local (declaration cost): 8
Divide: 58
Increment with dictionary member: 149
Increment with array member: 60
While time (for equivalent with manual increment): 84
if(true) time: 17
if(true)else time: 19
VariantArray resize: 959
VariantArray set element: 19
IntArray resize: 342

Binaries from
https://github.com/GodotBuilder/godot-builds/releases/tag/2.0_20160512
(MinGW cross-compiled):

Empty func (void function call cost): 517
Increment: 548
Increment x5: 2765
Increment with member var: 549
Increment with local (outside loop): 546
Increment with local (inside loop): 744
Increment Vector2: 551
Increment Vector3: 539
Increment Vector3 with constant: 550
Increment Vector3 coordinate by coordinate: 6101
Unused local (declaration cost): 170
Divide: 564
Increment with dictionary member: 1484
Increment with array member: 957
While time (for equivalent with manual increment): 1032
if(true) time: 118
if(true)else time: 125
VariantArray resize: 1644
VariantArray set element: 342
IntArray resize: 2417
IntArray set element: 1509

My own custom build (quite old now, I should git pull):

Increment: 54
Increment x5: 330
Increment with member var: 65
Increment with local (outside loop): 55
Increment with local (inside loop): 98
Increment Vector2: 88
Increment Vector3: 75
Increment Vector3 with constant: 74
Increment Vector3 coordinate by coordinate: 2983
Unused local (declaration cost): 6
Divide: 82
Increment with dictionary member: 435
Increment with array member: 105
While time (for equivalent with manual increment): 171
if(true) time: 31
if(true)else time: 16
VariantArray resize: 9808
VariantArray set element: 36
IntArray resize: 5665
IntArray set element: 615

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
<https://github.com/godotengine/godot/issues/4313#issuecomment-218813784

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218826088

[Zylann]

I don't know about the exact build options for the two versions from Travis and AppVeyor, but I can say the custom version I built is in debug mode, uses SCons (I didn't touched anything) and according to the logs, optimizations are off, as shown in my previous messages.

I run these simple GDScript tests: https://github.com/Zylann/gdscript_performance

[Marqin]

Try also Bunnnymark.

W dniu czwartek, 12 maja 2016 Marc [email protected] napisał(a):

I don't know about the exact build options for the two versions from
Travis and AppVeyor, but I can say the custom version I built is in debug
mode, uses SCons (I didn't touched anything) and according to the logs,
optimizations are off, as shown in my previous messages.

I run these simple GDScript tests:
https://github.com/Zylann/gdscript_performance

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218831777

[vnen]

It seems print does not work on release. But a project running only the script runs noticeably faster with the MSVC-compiled version (I built both my own at 48dc4d9). I'll try later with release_debug but without tools to check the prints.

The Bunnymark cited by @Marqin crashes when I use the Mingw 64-bit version. It can go around ~6000 bunnies in without losing FPS with MSVC-compiled, both 32- and 64-bits. With Mingw-compiled 32-bit I get slowdowns before reaching 2000 bunnies.

---

@slapin please enlighten us on how to add optimization to Mingw build, because AFAIK it's already enabled.

I wouldn't be surprised if Microsoft's compiler to its own platform is more optimized than a port of GCC for Windows. But if there's a way to optimize Mingw builds even further without breaking Godot I'm all open to it.

[slapin]

MSVC can be faster but not 10 times faster. The result is not realistic.
Something is just plain broken.

On Thu, May 12, 2016 at 10:43 PM, George Marques [email protected]
wrote:

It seems print does not work on release. But a project running only the
script runs noticeably faster with the MSVC-compiled version (I built both
my own at 48dc4d9
https://github.com/godotengine/godot/commit/48dc4d92a8f3c9eb42456eda2433c6fcaff0c723).
I'll try later with release_debug but without tools to check the prints.

The Bunnymark https://github.com/jotson/godot-bunnymark cited by @Marqin
https://github.com/Marqin crashes when I use the Mingw 64-bit version.
It can go around ~6000 bunnies in without losing FPS with MSVC-compiled,
both 32- and 64-bits. With Mingw-compiled 32-bit I get slowdowns before

reaching 2000 bunnies.

@slapin https://github.com/slapin please enlighten us on how to add
optimization to Mingw build, because AFAIK it's already enabled
https://github.com/godotengine/godot/blob/e4765c4831e8249696ac3c94ca28da2218cb236c/platform/windows/detect.py#L338-L347
.

I wouldn't be surprised if Microsoft's compiler to its own platform is
more optimized than a port of GCC for Windows. But if there's a way to
optimize Mingw builds even further without breaking Godot I'm all open to
it.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218864578

[slapin]

On my PC bunnymark shows 2K bunnies before slowdown, i7 2600K GTX660, Linux
and mingw32 build. So it is definitely broken, as on the same
PC AAA games work quite well on high settings. So to fix all Godot problems
we just need to drop Linux support and move to MSVC.
Any Mac testers here?

On Thu, May 12, 2016 at 10:58 PM, Sergey Lapin [email protected] wrote:

MSVC can be faster but not 10 times faster. The result is not realistic.
Something is just plain broken.

On Thu, May 12, 2016 at 10:43 PM, George Marques <[email protected]

wrote:

It seems print does not work on release. But a project running only the
script runs noticeably faster with the MSVC-compiled version (I built both
my own at 48dc4d9
https://github.com/godotengine/godot/commit/48dc4d92a8f3c9eb42456eda2433c6fcaff0c723).
I'll try later with release_debug but without tools to check the prints.

The Bunnymark https://github.com/jotson/godot-bunnymark cited by
@Marqin https://github.com/Marqin crashes when I use the Mingw 64-bit
version. It can go around ~6000 bunnies in without losing FPS with
MSVC-compiled, both 32- and 64-bits. With Mingw-compiled 32-bit I get

slowdowns before reaching 2000 bunnies.

@slapin https://github.com/slapin please enlighten us on how to add
optimization to Mingw build, because AFAIK it's already enabled
https://github.com/godotengine/godot/blob/e4765c4831e8249696ac3c94ca28da2218cb236c/platform/windows/detect.py#L338-L347
.

I wouldn't be surprised if Microsoft's compiler to its own platform is
more optimized than a port of GCC for Windows. But if there's a way to
optimize Mingw builds even further without breaking Godot I'm all open to
it.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218864578

[Marqin]

Well, also keep in mind that travis uses mingw 4.2 from 2007 :P And MSVC used here is from 2015.

we just need to drop Linux support

WTF. What does Linux support have to do with Windows builds?

[mefihl]

Bunnymark on Godot 2.0.2 official on my PC (i7 2600K GTX670, Windows 10 64bit) frame drop to 56 at 1640 object after export to exe in release mode.
https://github.com/jotson/godot-bunnymark

[Marqin]

@mefihl and for 2.0.3 https://github.com/GodotBuilder/godot-builds/releases/tag/2.0.3-stable ?

[akien-mga]

@mefihl and for 2.0.3 https://github.com/GodotBuilder/godot-builds/releases/tag/2.0.3-stable ?

Try http://downloads.tuxfamily.org/godotengine/2.0.3/ instead, I just deleted this tag and recreated it to push a hotfix to 2.0.3 ;)

[slapin]

Deleting tags in public repos is very, very bad idea. When you do that a
new family of kitties is put in another sack for drowning.
You better create branches for all releases for hotfixing, and tag them
with additional tags if necessary. This is real practice.

On Fri, May 13, 2016 at 11:05 AM, Rémi Verschelde [email protected]
wrote:

@mefihl https://github.com/mefihl and for 2.0.3
https://github.com/GodotBuilder/godot-builds/releases/tag/2.0.3-stable ?

Try http://downloads.tuxfamily.org/godotengine/2.0.3/ instead, I just
deleted this tag and recreated it to push a hotfix to 2.0.3 ;)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218977922

[akien-mga]

Yes and no. Deleting a tag for a release that has never been announced is not problematic, and way less clutter than having to issue a 2.0.3-1 or 2.0.4 to add a commit. If 2.0.3 had been published already, I would have done differently, but here, trust me, I know what I'm doing.

[slapin]

Well, only Linux and mingw depend on gcc, so dropping it will have problem
solved.
And no, that is not gcc 4.2 vs gcc-4.x or 6.x, the current Linux results
show the same problems as Windows-mingw results.

On Fri, May 13, 2016 at 2:06 AM, Hubert Jarosz [email protected]
wrote:

Well, also keep in mind that travis uses mingw 4.2 from 2007 :P And MSCV
used here is from 2015.

we just need to drop Linux support

WTF. What does Linux support have to do with Windows builds?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-218910731

[slapin]

You better create the branches. You will thank me later for advising you
this.
Look at existing workflows from people who know shit, like Kernel people -
use stable branches for hotfixing.
This makes hotfixing a normal practice, not hacky workouts.
And you can use different tagging system - like tag 10.0 for release, and
for fixes use tag 10.0.1, 10.0.2, but tag only hotfixes in stable branches,
not master
Branches are cheap and powerful, use them. It quite looks like you don't
understand what tag is for and what branch is for,

On Fri, May 13, 2016 at 4:41 PM, Rémi Verschelde [email protected]
wrote:

Yes and no. Deleting a tag for a release that has never been announced is
not problematic, and way less clutter than having to issue a 2.0.3-1 or
2.0.4 to add a commit. If 2.0.3 had been published already, I would have
done differently, but here, trust me, I know what I'm doing.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219046026

[akien-mga]

We're not dropping anything. End of the discussion.

Now if we want to investigate properly why we see such performance differences, that could be helpful. And if it's established that all GCC builds are less performant, and that it's not an optimization flag issue on our side, we could ask upstream directly.

[slapin]

This is not gcc problem either, it is either optimization problem or bad
code problem.
MSVC does a lot of tricks, so comparison of unoptimized builds of gcc vs
MSVC is useless. You can't tell MSVC to not optimize, while gcc
if asked to not optimize will prevent optimizations.
Otherwise comparison of tools-optimized (release_debug) adds only basic gcc
optimizations.
But relying only on compiler here is not really a good thing, as bad code
will give random results with different compilers. There looks to be
something
very badly written, which probably needed to pe optimized by brain, not
compiler.

On Fri, May 13, 2016 at 4:48 PM, Rémi Verschelde [email protected]
wrote:

We're not dropping anything. End of the discussion.

Now if we want to investigate properly why we see such performance
differences, that could be helpful. And if it's established that all GCC
builds are less performant, and that it's not an optimization flag issue on
our side, we could ask upstream directly.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219047810

[akien-mga]

PRs welcome.

[slapin]

On Fri, May 13, 2016 at 4:48 PM, Rémi Verschelde [email protected]
wrote:

We're not dropping anything. End of the discussion.

Also I think Godot needs to focus on engine features and publicity,
so dropping a few broken platforms will mean less bad reviews.
As I see no power to fix this all issues in any foreseeable future, I think
dropping is not
so bad option.

Now if we want to investigate properly why we see such performance
differences, that could be helpful. And if it's established that all GCC
builds are less performant, and that it's not an optimization flag issue on
our side, we could ask upstream directly.

Well, gcc is well established power, much more powerful than tiny godot.
Before blaming compiler, it is much better to run profilers on windows and
Linux and think.
But thinking is not an option.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219047810

[slapin]

As it seems we're really going to be constructive here, I think we can
safely close this as "GCC vs MSVC issue. Nobody knows and nobody cares".
will look good, and laso can be used
as "black hole bug" for other bugs like this in future.

On Fri, May 13, 2016 at 4:58 PM, Rémi Verschelde [email protected]
wrote:

PRs welcome.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219050471

[akien-mga]

@slapin Thanks for your valuable input, we will pass than onto our Customer Support Team and they will come back to you once your request has been assessed.

[volzhs]

I'm not sure since when.
As I remember, performance was fine with gcc compile until 2.0 first release.
But performance drop happened some days later.

[slapin]

Lets not turn this into flame war just out of everybody's lack of
commitment.
If you want to discuss this farther, feel free to ping me on IRC.

On Fri, May 13, 2016 at 5:03 PM, Rémi Verschelde [email protected]
wrote:

@slapin https://github.com/slapin Thanks for your valuable input, we
will pass than onto our Customer Support Team and they will come back to
you once your request has been assessed.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219051813

[Zylann]

@slapin: we can tell MSVC to not optimize: /Od https://msdn.microsoft.com/en-us/library/k1ack8f1%28v=vs.120%29.aspx (and optimizations are off).
Comparing with optimizations on both sides sounds relevant.

@volzhs true. 2.0 and 2.1 have this difference, and both used MinGW according to @akien-mga (way up in the thread).

[slapin]

If anybody here shares the automatic script which runs the GDScript test,
and tells me the commit ID when all worked fine. I can do bisect and find
offending commit
(or something relevant). Onless this happens I can't help with anything but
sarcasm.

On Fri, May 13, 2016 at 5:06 PM, Marc [email protected] wrote:

@slapin https://github.com/slapin: we can tell MSVC to not optimize:
/Od https://msdn.microsoft.com/en-us/library/k1ack8f1%28v=vs.120%29.aspx
(and optimizations are off).
Comparing with optimizations on both sides sounds relevant.

@volzhs https://github.com/volzhs true. 2.0 and 2.1 have this
difference, and both used GCC according to @akien-mga
https://github.com/akien-mga (way up in the thread).

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219052870

[reduz]

This is the same argument over and over, and you'll keep getting the same answers over and over.
I'm not against optimizing Godot. Myself and a lot of other contributors would gladly work on that.

But we need with benchmarks for this, and based on real use cases. We make Godot, we don't use it as much as you.

If the community can provide us with benchmarks, then we will do optimization work. Otherwise, be happy with what you have.

@slapin Would you kindly offer yourself to help create and admin a set of benchmarks so we can test optimizations and regressions?

[reduz]

We could create a repository in godotengine github for benchmarks, then you guys can contribute what you have or future benchmark tools there.

[slapin]

In this bug I see a lot of mentions of existing script. I want this.

On Fri, May 13, 2016 at 5:12 PM, Juan Linietsky [email protected]
wrote:

We could create a repository in godotengine github for benchmarks, then
you guys can contribute what you have or future benchmark tools there.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219054319

[Zylann]

Here they are again:
Bunnymark: https://github.com/jotson/godot-bunnymark
Micro-tests: https://github.com/Zylann/gdscript_performance
Both GDScript.

I don't know how to link to a post in an issue, so:

Also, I should note that my benchmarks only concern GDScript. I'm not sure if the ratio is the same for native code, I don't have the same environment setup to test cross-compile with MinGW.
Benchmarking C++ could give a clue about what is actually slower, maybe the way memory is allocated, or how well GDScript is optimized by the compiler? (because obviously, GDScript does different things than the graphics engine).

[akien-mga]

In this bug I see a lot of mentions of existing script. I want this.

Seriously? All mentioned scripts have been linked already:

• Bunnymark: https://github.com/godotengine/godot/issues/4313#issuecomment-218944570 and https://github.com/godotengine/godot/issues/4313#issuecomment-218864578
• @Zylann's tests: https://github.com/godotengine/godot/issues/4313#issuecomment-218831777

(_Self-edited to remove rude language_)

[akien-mga]

Regarding creating an official repo for benchmarks, that's a good idea. But honestly for this very issue (#4313) it's not a matter of benchmarking Godot or GDScript to improve bottlenecks (which is still a valid issue, but IMO out of the scope of #4313 and should thus not be discussed here).

There is obviously something fishy when one compiler provides 10 times more performance than an other. And most likely, this issue is located in the SConscript file, where some optimisation flags are passed to one compiler and not to the other.

So to properly debug this specific issue, we should just build Windows binaries of a given Godot commit with the exact same parameters (tools and target), using MSVC, MinGW and Clang, and try e.g. @Zylann's tests and Bunnymark with all three versions. Maybe do that also for release_debug vs debug for the tools, or release vs release_debug for the templates, and see if there's a difference at this level.

[slapin]

Thanks a lot! Will setup something shortly. I need terminal-level script to
automate bisecting.
bunnymark is graphical, so not so good for automatic testing, will try
gdscript_performance.

On Fri, May 13, 2016 at 5:18 PM, Marc [email protected] wrote:

Here there are again:
Bunnymark: https://github.com/jotson/godot-bunnymark
Micro-tests: https://github.com/Zylann/gdscript_performance
Both GDScript.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219056276

[slapin]

I'd agree to maintain Godot tests repo, hoping it would not contain only
benchmarks, but also API tests.

On Fri, May 13, 2016 at 5:32 PM, Rémi Verschelde [email protected]
wrote:

Regarding create an official repo for benchmarks, that's a good idea. But
honestly for this very issue (#4313
https://github.com/godotengine/godot/issues/4313) it's not a matter of
benchmark Godot or GDScript to improve bottlenecks.

There is obviously something fishy when one compiler provides 10 times
more performance than an other. And most likely, this issue is located in
the SConscript file, where some optimisation flags are passed to one
compiler and not to the other.

So to properly debug this specific issue, we should just build Windows
binaries of a given Godot commit with the exact same parameters (tools
and target), using MSVC, MinGW and Clang, and try e.g. @Zylann
https://github.com/Zylann's tests and Bunnymark with all three
versions. Maybe do that also for release_debug vs debug for the tools, or
release vs release_debug for the templates, and see if there's a
difference at this level.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219060296

[slapin]

Well, it looks like you found an easy ass way - to call me a troll.

On Fri, May 13, 2016 at 5:27 PM, Rémi Verschelde [email protected]
wrote:

In this bug I see a lot of mentions of existing script. I want this.

Seriously? All mentioned scripts have been linked already:

• Bunnymark: #4313 (comment)
  https://github.com/godotengine/godot/issues/4313#issuecomment-218944570
  and #4313 (comment)
  https://github.com/godotengine/godot/issues/4313#issuecomment-218864578
• @Zylann https://github.com/Zylann's tests: #4313 (comment)
  https://github.com/godotengine/godot/issues/4313#issuecomment-218831777

Stop looking for a simple way to be an ass, and do read the thread you're
trolling beforehand.

Don't be like this. Supplying direct links would do much better for your
karma.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219058985

[vnen]

Well, it looks like you found an easy ass way - to call me a troll.

Sorry, but your attitude shows only lazyness, since you had not given the effort to read the thread (or so it seems, otherwise you'd have seen the links already). You want _us_ to do it the way _you_ want. It doesn't work like this. We're accepting your help if you're actually willing to help.

[Marqin]

@reduz, please block this conversation to prevent more offtopic spam. We just need some proof to see that using MSVC for 2.0.3 helped for optimization ( there is already one). And then consider this bug closed.

[slapin]

Well, it looks that we have +1 lazy people in this thread. Welcome to the
club.

Sometimes it is easier to see stuff in single list than searching scattered
emails for
something. I just hate to search things. Also, I try to be productive here.
And what is so bad about being lazy? It is not that I ask just out of being
an ass,
I really going to work on it.

On Fri, May 13, 2016 at 5:53 PM, George Marques [email protected]
wrote:

Well, it looks like you found an easy ass way - to call me a troll.

Sorry, but your attitude shows only lazyness, since you had not given the
effort to read the thread (or so it seems, otherwise you'd have seen the
links already). You want _us_ to do it the way _you_ want. It doesn't
work like this. We're accepting your help if you're actually willing to
help.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219066549

[mefihl]

@Marqin Bunnymark on Godot 2.0.3 official on my PC (i7 2600K GTX670, Windows 10 64bit) frame drop to 57 at 6540 object after export to exe in release mode. Nice boost from 2.0.2 ;)

[mattiascibien]

@mefihl msvc or gcc?

[Marqin]

So we have 4x better performance on your PC with bunnymark with 2.0.3, wow.

[mefihl]

@mattiascibien 2.0.3 download from official website.

[mattiascibien]

@mefihl MSVC so... Thanks

[Zylann]

For the record, I ran my test on Linux (same machine with dual boot, Godot with tools). In short, results are similar to what I get with msvc on Windows. So there is definitely something weird with MinGW, Linux is all fine :)

Official 2.0.3:

Empty func (void function call cost): 56
Increment: 38
Increment x5: 196
Increment with member var: 39
Increment with local (outside loop): 35
Increment with local (inside loop): 54
Increment Vector2: 26
Increment Vector3: 47
Increment Vector3 with constant: 47
Increment Vector3 coordinate by coordinate: 848
Unused local (declaration cost): 7
Divide: 62
Increment with dictionary member: 133
Increment with array member: 59
While time (for equivalent with manual increment): 77
if(true) time: 15
if(true)else time: 6
VariantArray resize: 745
VariantArray set element: 14
IntArray resize: 370
IntArray set element: 154

Custom build (gcc):

Empty func (void function call cost): 247
Increment: 59
Increment x5: 296
Increment with member var: 68
Increment with local (outside loop): 54
Increment with local (inside loop): 92
Increment Vector2: 77
Increment Vector3: 65
Increment Vector3 with constant: 69
Increment Vector3 coordinate by coordinate: 2600
Unused local (declaration cost): 11
Divide: 82
Increment with dictionary member: 408
Increment with array member: 108
While time (for equivalent with manual increment): 170
if(true) time: 27
if(true)else time: 17
VariantArray resize: 9101
VariantArray set element: 41
IntArray resize: 3524
IntArray set element: 379

[akien-mga]

Nice :) Was that debug or release_debug?

[techtonik]

Thinking about tests page template.

=== Godot performance (smaller is better)


tools           MSVC  MinGW  Clang

@Zylann's
  debug
  release_debug
  release
Bunnymark
  debug
  release_debug
  release


target          MSVC  MinGW  Clang

@Zylann's
  debug
  release_debug
  release
Bunnymark
  debug
  release_debug
  release


1. https://github.com/jotson/godot-bunnymark
2. https://github.com/Zylann/gdscript_performance

Tools and debug, release_debug explained.

Why tools (editor) affects performance?

[Zylann]

The custom Linux build is the result of scons platform=11 out of the box. Is it debug or release_debug?

Other news: the PC on which I did my tests until now has a core i5. I'm relocating, so I built the same way on another Windows 10 PC, a recent laptop with a core i7. But... I get bad results again, with any Godot version, official or self-built Oo"

Empty func (void function call cost): 330
Increment: 351
Increment x5: 1976
Increment with member var: 355
Increment with local (outside loop): 350
Increment with local (inside loop): 485
Increment Vector2: 355
Increment Vector3: 355
Increment Vector3 with constant: 356
Increment Vector3 coordinate by coordinate: 4534
Unused local (declaration cost): 88
Divide: 374
Increment with dictionary member: 1047
Increment with array member: 681
While time (for equivalent with manual increment): 780
if(true) time: 37
if(true)else time: 45
VariantArray resize: 1231
VariantArray set element: 219
IntArray resize: 1756
IntArray set element: 1056

Maybe that's just because the laptop is slower. But it has core i7... it's better than i5, right? :s

[reduz]

I made a tests repository in godot organization, please let's put all
benchmark code there :)

On Sun, May 15, 2016 at 12:15 PM, Marc [email protected] wrote:

The custom build is the result of scons platform=11 out of the box. Is it
debug or release_debug?

Other news: the PC on which I did my tests until now has a core i5. I'm
relocating, so I built the same way on another Windows 10 PC, a recent
laptop with a core i7. But... I get bad results again, with any Godot
version, official or self-built Oo"

Empty func (void function call cost): 330
Increment: 351
Increment x5: 1976
Increment with member var: 355
Increment with local (outside loop): 350
Increment with local (inside loop): 485
Increment Vector2: 355
Increment Vector3: 355
Increment Vector3 with constant: 356
Increment Vector3 coordinate by coordinate: 4534
Unused local (declaration cost): 88
Divide: 374
Increment with dictionary member: 1047
Increment with array member: 681
While time (for equivalent with manual increment): 780
if(true) time: 37
if(true)else time: 45
VariantArray resize: 1231
VariantArray set element: 219
IntArray resize: 1756
IntArray set element: 1056

Maybe that's just because the laptop is slower. But it has core i7... it's
better than i5, right? :s

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/godotengine/godot/issues/4313#issuecomment-219291504

[vnen]

The custom Linux build is the result of scons platform=11 out of the box. Is it debug or release_debug?

The defaults are tools=yes target=debug

Why tools (editor) affects performance?

Not sure on that, but I guess it's because there's some extra steps to work with the debugger.

[Zylann]

@reduz I made a pull request to the godot-tests repo to add my GDScript benchmark.

[techtonik]

https://github.com/godotengine/godot-tests

[vnen]

Bumping a year-old thread about a "non-issue" is certainly the most productive way to spend the time.

[neikeq]

i really love how @Marqin and @vnen contributed nothing to this issue except asking reduz to ban trolls.

There's been a heated discussion here but, as far as I can read, no one asked banning anyone.

[ni-code]

I reported the troll account. He's probably will get banned.

[Marqin]

@MakeGodotGreatAgain so this is nothing -https://github.com/GodotBuilder/godot-builds/pull/1 ?

Please, read all of my comments, not only last one...

[akien-mga]

Remember that free software development is meritocratic. Check the contributions @vnen has made to this project, check the ones you did, and then check who has "room to talk here".

Anyway, closing this issue and locking the discussion as it should have been done a year ago.

[akien-mga]

And regarding the initial topic, the issue is clear. Binaries compiled with a recent MSVC (2015), like our current 2.1.x official Windows binaries, are way more efficient than binaries cross-compiled with a 8 years old GCC version on Ubuntu 12.04 (like our early 2.0.x binaries).