Zig: official support for portable linux binaries that support being executed with and without a dynamic linker

[andrewrk]

Here's my idea. Change std.builtin.link_libc to be an enum:

--- a/lib/std/builtin.zig
+++ b/lib/std/builtin.zig
@@ -26,6 +26,14 @@ pub const SubSystem = std.Target.SubSystem;
 /// Deprecated: use `std.Target.Cpu`.
 pub const Cpu = std.Target.Cpu;

+/// This data structure is used by the Zig language code generation and
+/// therefore must be kept in sync with the compiler implementation.
+pub const LinkLibC = enum {
+    never,
+    always,
+    runtime,
+};
+
 /// `explicit_subsystem` is missing when the subsystem is automatically detected,
 /// so Zig standard library has the subsystem detection logic here. This should generally be
 /// used rather than `explicit_subsystem`.

Most applications would be never or always. To get runtime you would need to ask for it with a special compiler flag, such as: -fruntime-libc. Then we introduce a new decl, std.process.link_libc:

--- a/lib/std/process.zig
+++ b/lib/std/process.zig
@@ -14,6 +14,20 @@ const Allocator = mem.Allocator;
 const assert = std.debug.assert;
 const testing = std.testing;

+pub usingnamespace switch (builtin.link_libc) {
+    .always => struct {
+        pub const link_libc = true;
+    },
+    .never => struct {
+        pub const link_libc = false;
+    },
+    .runtime => struct {
+        /// This value will be set to `true` in start code when it is detected
+        /// the application is running in a dynamic linker.
+        pub var link_libc: bool = false;
+    },
+};
+
 pub const abort = os.abort;
 pub const exit = os.exit;
 pub const changeCurDir = os.chdir;

This is interesting because it is always a bool, but it could be comptime or runtime known. Most that currently uses std.builtin.link_libc would be updated to use this value instead.

The start code:

--- a/lib/std/start.zig
+++ b/lib/std/start.zig
@@ -210,9 +210,15 @@ fn posixCallMainAndExit() noreturn {
         // Initialize the TLS area. We do a runtime check here to make sure
         // this code is truly being statically executed and not inside a dynamic
         // loader, otherwise this would clobber the thread ID register.
-        const is_dynamic = @import("dynamic_library.zig").get_DYNAMIC() != null;
-        if (!is_dynamic) {
-            std.os.linux.tls.initStaticTLS();
+        switch (builtin.link_libc) {
+            .runtime => {
+                const is_dynamic = @import("dynamic_library.zig").get_DYNAMIC() != null;
+                std.process.link_libc = is_dynamic;
+                if (!is_dynamic) {
+                    std.os.linux.tls.initStaticTLS();
+                }
+            },
+            else => std.os.linux.tls.initStaticTLS(),
         }

There would be some other code that would need to be changed in response to this, but most code would already work just fine - what would happen is when the value is runtime known, the other fallback code for when the code is not linked with libc would get included in the binary in addition to the libc path, and it would turn into a runtime check.

The main use case for this would be to create portable Linux binaries that work on any distribution, that are capable of doing dlopen and therefore pop up a window and do graphics drivers stuff, as well as other stuff that only works when you interact with the system libc. I have a proof-of-concept of this over in https://github.com/andrewrk/zig-window.

@LemonBoy pointed out in https://github.com/ziglang/zig/commit/479f259ea4799583f4062b2dffb7d9a289f7e18b#commitcomment-44519027 that the proof-of-concept was flawed because if it ever tried to do threads or thread-local-storage, it would cause UB. This proposal is what it would look like to solve the problem in a robust way. The only downside of this I can see is additional complexity, for what could be argued is an obscure use case. And now, maybe I'm getting too big for my britches here, but it seems to me that gaming on Linux being an obscure use case might have something to do with how non-portable binaries are. I personally think it's a very exciting use case, something I haven't seen ever done before, that could bring a lot of value to game developers and players alike. There are also use cases beyond gaming; essentially what this opens up is the ability to create portable binary distributions of complex applications.

The breaking change that would affect everyone who doesn't care about this use case would be to simply update all your std.builtin.link_libc to std.process.link_libc.

[LemonBoy]

The only downside of this I can see is additional complexity, for what could be argued is an obscure use case.

only?

• Remember the hell we have to go trough in order to make glibc pick the correctly-versioned symbols?
• The moment a libc is involved you can't this trick anymore as the crt0 is out of your control,
• Making the link_libc dynamic means means a lot of code bloat, you now have to ship code that calls the libc _and_ the self-hosted implementation,
• If link_libc may be true the whole stdlib will require the libc to be linked in, how do you explain you may not call any part of it to the linker? Opening all the symbols with dlopen/dlsym is definitely bonkers,
• You can't mix & match libc versions/kinds,
• posixCallMainAndExit is tailored for Zig's startup, if the libc expects something more/less/different you're screwed (at runtime)

My two cents: the number of people working on Zig is already small and the project scope is already _enormous_ (a language, a compiler, a hopefully better compiler written in Zig, an intermediate representation, a bunch of code-generator backends, three or four linkers, a standard library), it would be better to focus on something more practical than this.