Godot: Mono/Linux 64-bit project is 101.3MB with kbs of assets

[nightblade9]

This is related to #29241, which addresses the problem by compiling Godot from source (not something I can easily handle).

As of now (Godot 3.1.1), I have a Godot/Mono project (.NET 4.6.2) with a single image (54kb) and a single font (100kb). When I export it for Linux, the resulting files are:

• .pck file: 29.9mb
• .x86_64 executable: 65.8mb
• data_<project_name> folder: 5.9mb

The total rolls in at 101.3MB, which is quite large. I tried changing several project properties (Zlib compression, 32- vs. 64-bit, debug vs. release, etc.) to no avail. (32-bit exports are a few MB smaller, but not much.)

I suspect my third-party NuGet packages is partly to blame; the packages directory is 75.6mb, although each package includes the original zip (.nupkg file) and one set of DLLs per .NET target supported (eg. some have net45, netstandard1.1, and netstandard20 -- even though I'm only linking/using the .NET 4.5 ones).

For reference, an empty .NET project clocks in at 71.7mb, with a similar breakdown:

• .pck file: 8.9kb
• .x86_64 executable: 65.8mb
• data_EmptyDotNet folder: 5.9mb

I'm guessing the difference of 30MB in the .pck file is from my NuGet packages.

Is there any way to significantly shrink the project down?

[neikeq]

Can you upload the exported files? Specially the pck so I can have a look inside.

[akien-mga]

Your executable is likely a debug build. A stripped release build should be smaller than that (~35 MB I guess).

[nightblade9]

Your executable is likely a debug build. A stripped release build should be smaller than that (~35 MB I guess).

@akien-mga I tried exporting in release mode, and the final executable/PCK were the same size. How do I generate a "stripped release build"?

@neikeq It's a closed-source project (for now) so I can't upload it. If you send me an email, I will upload it and send you a link for the .pck file.

[Calinou]

How do I generate a "stripped release build"?

You can use the strip command-line utility to remove debugging symbols from any binary compiled with GCC or Clang. It's usually a good idea to always run it on binaries you plan to distribute, because debugging symbols tend to sneak in even if you use target=release (in my experience at least). That is, unless people need to run it in a debugger somehow :slightly_smiling_face:

[nightblade9]

You can use the strip command-line utility to remove debugging symbols from any binary compiled with GCC or Clang

Wow. This is amazing - it dropped the exe from 68MB to 40MB.

This brings the total zipped archive to 26.8MB, which is roughly what my GDscript game compressed to (21MB). I'm happy with that.

@Calinou is there any way to integrate this into the editor in a future release - maybe automatically as part of exporting in Release mode?

[Calinou]

@nightblade9 This should be integrated at the SCons level when using debug_symbols=no, I guess.

[akien-mga]

Well debug_symbols=no already fixes this issue, but was likely not used by OP.

[nightblade9]

Well debug_symbols=no already fixes this issue, but was likely not used by OP.

That's correct. I mentioned that I can't recompile the code from scratch (I tried), and I imagine I'm not the only one who will like the usage of strip to be automatic when you export in Release mode.

[Calinou]

I believe official release export templates should be built with debug_symbols=no then :slightly_smiling_face:

I don't think this should be done while exporting a project, as it'd require to have strip on the PATH. This is very unlikely for Windows users, as it's not installed by default.

cc @hpvb

[akien-mga]

I believe official release export templates should be built with debug_symbols=no then

They are :)

I don't know where @nightblade9's templates come from but they're custom built. Official 3.1.1 templates:

-rw-r--r-- 1 akien akien  37M May 15 10:06 linux_x11_32_debug
-rw-r--r-- 1 akien akien  33M May 15 10:06 linux_x11_32_release
-rw-r--r-- 1 akien akien  38M May 15 10:06 linux_x11_64_debug
-rw-r--r-- 1 akien akien  35M May 15 10:06 linux_x11_64_release

[nightblade9]

I'm 100% sure that they're not custom-built and that I downloaded them from within Godot itself.

@akien-mga let me know where I can find them on a Linux system. I'll delete them, re-download them again, and re-run the export again.

[akien-mga]

They're in ~/.local/share/godot/templates/<version>/.

[akien-mga]

Ah though the numbers I gave are for the classical build, maybe the mini version is bigger. 65 MB sounds a bit much, but that may be possible.

[nightblade9]

Before doing anything else:

-rw-r--r--  1 nightblade nightblade 66900550 Aug  8 18:27 linux_x11_32_debug
-rw-r--r--  1 nightblade nightblade 63011082 Aug  8 18:27 linux_x11_32_release
-rw-r--r--  1 nightblade nightblade 68702911 Aug  8 18:27 linux_x11_64_debug
-rw-r--r--  1 nightblade nightblade 65796671 Aug  8 18:27 linux_x11_64_release
-rw-r--r--  1 nightblade nightblade 33262185 Aug  8 18:27 osx.zip
-rw-r--r--  1 nightblade nightblade       18 Aug  8 18:27 version.txt
-rw-r--r--  1 nightblade nightblade 38990136 Aug  8 18:27 windows_32_debug.exe
-rw-r--r--  1 nightblade nightblade 34151224 Aug  8 18:27 windows_32_release.exe
-rw-r--r--  1 nightblade nightblade 35339064 Aug  8 18:27 windows_64_debug.exe
-rw-r--r--  1 nightblade nightblade 32822584 Aug  8 18:27 windows_64_release.exe

Reinstallation method:

• rm -rf'd the entire 3.1.1.stable.mono
• Launch Godot and try to export as Linux
• Click the link that shows up about missing templates
• Click the download link
• Pick the official (one and only) mirror
• Wait. The download was around 181MB.
• ls -al again gives me this:

-rw-r--r--  1 nightblade nightblade 66900550 Aug 27 17:48 linux_x11_32_debug
-rw-r--r--  1 nightblade nightblade 63011082 Aug 27 17:48 linux_x11_32_release
-rw-r--r--  1 nightblade nightblade 68702911 Aug 27 17:48 linux_x11_64_debug
-rw-r--r--  1 nightblade nightblade 65796671 Aug 27 17:48 linux_x11_64_release
-rw-r--r--  1 nightblade nightblade 33262185 Aug 27 17:48 osx.zip
-rw-r--r--  1 nightblade nightblade       18 Aug 27 17:48 version.txt
-rw-r--r--  1 nightblade nightblade 38990136 Aug 27 17:48 windows_32_debug.exe
-rw-r--r--  1 nightblade nightblade 34151224 Aug 27 17:48 windows_32_release.exe
-rw-r--r--  1 nightblade nightblade 35339064 Aug 27 17:48 windows_64_debug.exe
-rw-r--r--  1 nightblade nightblade 32822584 Aug 27 17:48 windows_64_release.exe

The sizes are identical to what they were before.

What, if anything, am I doing wrong?

[akien-mga]

Nothing wrong a priori, the Linux Mono templates seem to be like that. I'll see if I can find out why.

[scottkunkel]

using UPX will yield 11Mb for linux and windows.
Note that this needs to be applied after export, not to the build template

[nightblade9]

It would be nice to have a list of things to run against an export (eg. strip and UPX). It would be idela if the export process does that automatically (assuming licensing works out), because it's easy to forget or not know about it.

[Calinou]

@nightblade9 There are some downsides associated with the use of UPX, such as antivirus programs throwing false positives. I would recommend against using it; just make sure to compress your game's distribution in a ZIP archive or similar. If you're uploading your game to Steam, it will also compress the data when transferring it to the client (if I'm not mistaken).

[nightblade9]

@Calinou I primarily upload my games to Itch, which uses Butler, which gives very efficient file diffs for anyone using their Itch store app. For example, my .NET Core game (40MB uncompressed), publishing a new release clocks in at kilobytes.

I don't recall exactly how well it works with unstripped Godot releases, but if I recall correctly, it's sending maybe 1-2MB tops - it recognizes well enough that almost all the contents aren't changed.

Still, smaller zips are a good thing, especially for players with poor internet.

Thanks for letting me know about UPX, I wasn't aware of it (or its downsides).

[akien-mga]

This is not a bug, the Mono runtime is just that big. That's one of the downsides of using a external language such as Mono vs a built-in, lightweight solution like GDScript.

Compressing the binary for distribution, or even storage with upx, might be a way to work this around, but as it stand it seems to be working as intended. The ~65 MB binary is ~40 MB of Godot + ~25 MB of Mono. The 30 MB PCK includes the Mono runtime needed to run your game code.

Note that it's the base size, if you add 400 MB of assets, it will be a 500 MB game.

[nightblade9]

@akien-mga so there are no plans to run strip automatically as part of the export process? What about documenting this on the wiki somewhere? (I forgot about this until now and would like some way to look it up for my future projects.)

[akien-mga]

@nightblade9 Thanks for the reminder, I had missed that. I opened an issue to consider this further: godotengine/godot-build-scripts#11.

Still, even with a stripped Linux binary, a base Mono project will always be significantly bigger than a base non-Mono project, which is already at least ~30 MB due to the size of the Godot runtime itself (unless you disable modules, remove fonts, etc.).

[neikeq]

IIRC The size difference in the Godot binary with Mono once stripped should be around 5 MB, not much more, maybe even less.
Most of the size overhead comes from the assemblies, as you can see in the size of the .pck which contains them. I plan to add a step to run the IL Linker during export at some point in the future. That should help reduce that bloat.