Godot: Refactor build object suffix to standardized tuples/target triples identifying build options

[akien-mga]

Godot version:
Master branch (3a93499f899)

OS/device including version:
All

Issue description:
We currently set a suffix string in SConstruct that is appended to the name of all build objects, and most importantly the final Godot binary. It contains some information about the build options (platform, target, tools, ~arch/bitness, ~compiler), but not in a very consistent way for all platforms: most don't include compiler info, some have arch info, others have bitness info, the use of tools impact what kind of target info is included, etc.

The current form is more or less:

godot.<platform>.<target and/or tools>.<bits or arch>[.<module suffix>][.<compiler>][.<user-defined extra suffix>][<extension>]

(order might differ a bit towards the tail, didn't double check)
So we typically get binary names like these:

godot.windows.opt.tools.64.mono.exe
godot.x11.opt.debug.32.llvm

(Some other platforms don't follow this scheme and have instead:

platform/android/java/libs/debug/armeabi-v7a/libgodot_android.so

and a resulting bin/android_debug.apk bundling the libs for all built arches.
Their intermediate build options however follow the scheme:

core_bind.android.debug.armv7.neon.o

albeit with some variation (see e.g. .neon here))

---

I propose to refactor this system to use well-documented and standardized tuples, so that you can see at a glance how a binary was configured. Such information could also be included in godot --version and printed to stdout on start, so that we get reliable info on users' build in bug reports.

It would then also be easier for thirdparty buildsystems (e.g. GDNative plugins) to use compatible tuples, especially if we factor out this SCons logic in a Python module that could be reused outside Godot.

This should also be the opportunity to sanitize the mess we currently have with arch, android_arch and bits options. We should only use arch and drop the rest IMO. (If we want to keep bits for convenience, its mapping to "common-sense" arches -- e.g. arch=x86_64 for bits=64 on PC) for the given platform should likely be done in a module in a big switch, to avoid polluting each platform's detect.py)

I don't have a proposal yet for how it should look like exactly, so feel free to do some research on what other projects do and make suggestions. Some data:

• Rust target triples: https://forge.rust-lang.org/platform-support.html
• Debian multiarch tuples: https://wiki.debian.org/Multiarch/Tuples

Information that should be included by our target tuple (won't be a triplet as we need more than 3 fields):

• platform (currently x11, windows, android, etc.)
• target (currently opt, opt.debug, debug)
• architecture (currently a mess mixing bits, arch and android_arch based on the target platform)
• tools (currently tools or nothing)
• compiler (currently only llvm exists, but we should have gcc, llvm, msvc, etc.)
• noteworthy build options (currently only mono, defined with env.add_module_version_string). We can't document every single build option this way though, so it should be limited to those which really matter for everyday use, like whether the build is linked againt the Mono SDK or not

The extra_suffix may not need to be included in the tuple and could just be appended at the end.

---

Related issues:

• #19316: Disable jit for regex in a platform independent way
• #19319: platform independent way to choose libvpx optimization code
• #21418: Build products compiled with an automatically-detected MSVC do not contain an architecture in the name
• #23506: libtheora compiling errors for ARM (Hard Float)
• #23575: Use env["arch"] instead of platform.machine() in scons build scripts
• #24008: Cross compilation ignores arch and bits options

[silverkorn]

Here's a quick suggestion borrowing from both Rust and Debian naming convention.

This suggestion also target an easy parsing method which would avoid problem since all information are required.

Feel free to give feedback and adjust anything that might need additional attention.

---

Grouping characters

All portions could be parsed easily by splitting them with a specific character:

• Main groups with underscores (_)
• Subgroups with hyphens (-).
• _[Suggested]_ Lower end groups with dots (.), as for the versions.

Application

| Name | Type | Custom Options | Environment |
|------- |--------- |--------------------------- |------------- |
| godot | tools | default | debug |
| | runtime | headless | release |
| | server | mono | |
| | | {personal custom} | |

Target

| Architecture (arch) | Operating System (os) | Compiler/SDK/Executable form & additional options (opts) |
|------------------- |--------------------- |-------------------------------------------------------- |
| aarch64 | freebsd | android |
| amd64 | haiku | androideabi |
| arm | ios | gnu |
| armhf | linux | gnueabi |
| i386 | macosx | gnueabihf |
| i586 | netbsd | msvc |
| i686 | openbsd | {console} |
| powerpc | unknown | |
| powerpc64 | windows | |
| ~x86_64~ ¹ | | |

¹ __[Edit]__: Underscore (_) would conflict with parsing convention and, as mentioned by @bruvzg , amd64 could be used instead since it's the same as x86_64 and it's used by the well-known Go language.

Version

| Version | Stage |
|------------------------------------- |---------- |
| v.{Major}.{Minor}.{Patch} | dev |
| {Commit Shorthash} | alpha{#} |
| {YYYY}.{MM}.{DD}.{Commit Shorthash} | beta{#} |
| | rc{#} |
| | stable |

° _{YYYY}.{MM}.{DD}.{Commit Shorthash}_ is suggested for cleaner sorting, daily builds and CI/CD

---

[Example no.1] Official stable release (Windows x64)

godot-runtime-default-release_amd64-windows-msvc_v.3.1.2-stable.exe

[Example no.2] Custom editor build based on stable codebase for tests (FreeBSD ARM)

godot-tools-default-debug_arm-freebsd-gnu_v.3.1.2-stable

[Example no.3] Custom template build with mono support based on a test codebase for tests (Android smartphones)

godot-runtime-mono-debug_arm-unknown-android_2020.01.06.e2a6cae-dev
_{(Not sure if it's usually linux, unknown or any others)}

---

This said, I've got some question around this:

• I'm not sure about the x11 and darwin target, is there any possibility or interest to support more than one GUI/Windowing system per operating system at the same time?
• Should we include/support all ARM versions' architectures or stick to one as arm?
• Should we include/support all Intel's x86 architectures or stick to one? (i586 vs i686?)
• Should we include/support both amd64 and x86_64 or only one of them should be used?

By hoping that we will find the best way to suffix all the information in one! :smile:

[bruvzg]

I'm not sure about the x11 and darwin target

• There's WIP PR (https://github.com/godotengine/godot/pull/23426) for wayland support
• darwin is open-source part of macOS/iSO - XNU kernel and core services, some compilers/build systems (including rust) use this name for macOS builds.

[Suggested] Lower end groups with dots (.), as for the versions.

Dots may cause some problems with some not very smart file managers (like macOS Finder).

[bruvzg]

Should we include/support all ARM versions' architectures or stick to one as arm?

Currently multiple arm versions (v6, v7 + neon) are supported for Android builds

Should we include/support all Intel's x86 architectures or stick to one? (i586 vs i686?)

i585 is first Pentium (released in 1993), i686 is Pentium Pro (released in 1995), more info: https://gcc.gnu.org/onlinedocs/gcc-4.8.2/gcc/i386-and-x86-64-Options.html

Should we include/support both amd64 and x86_64 or only one of them should be used?

That's the same thing, Windows uses AMD64 name, everything else uses x86_64

[silverkorn]

Thanks for the info @bruvzg

According to your information and as trying to find a good cross-platform/language in-between suffix based on the current form mentioned above and the convention used by others for a maximum flexibility and details, let me know if the following compromises would be fair as the new naming convention:

So there would be no issues of using only x86_64 for the 64-bit AMD/Intel architecture?
Parsing could detect Windows + x86_64 + msvc to change to amd64, if needed.

I understand that dots should not be used in the tuples/target triples and to some other places. Anyhow, dots are already used for the current form mentioned above in the filenames, should we stop using this?

I see the point with x86 and x86_64 with the palette of options... It's sure not including all of the variations and instruction sets in the name but as for the official support, maybe we could assume the current standard is set and additional options from the compiling end user will be at their discretion.
Otherwise, would it be a good option to use a configuration file like a JSON or something with Scons for the official build names and their compiling options? Or better, a cached file with all the build information/parameters linked maybe with an hash of the object? (I'm clearly getting too far there :sweat_smile: )

[silverkorn]

There's WIP PR (#23426) for wayland support

It seems that x11 is being replaced with "linux" there, so it would be trivial to use linux instead, or maybe, if the dots are no problems, linux.x11 and linux.wayland?
This would also be useful for bsd flavors and the target GUI/Windowing system.

[Xrayez]

I think it would be nice to get the final name on demand based on passed command-line arguments. See how it's currently hard-coded for running simple tests on server platform:

https://github.com/godotengine/godot/blob/5efd43609b84c2fec2ea452a1c27b028388e8389/.travis.yml#L140-L145

[Anutrix]

I still prefer dots over hyphen/underscore. Might be my personal preference(or might be used to seeing it) or there may be others who like it that way.

[clayjohn]

Too old and low-quality. Closing... :P

[akien-mga]

Duh :P

I'll have to remember to open a GIP about this as it would be good to change for 4.0.

[Xrayez]

Suggestion: have no_suffix option. This would be mainly useful for CI where we don't necessarily care about the resulting program name (like running tests with just built Godot binary).