Zig: Proposal: Zig ABI for language specific features

[suirad]

During my experiment of implementing an error abi, I came to the conclusion that it should be do-able to add an ABI to zig to ease integration between zig-only projects. This also would expose libraries/objects of other languages to interact directly with zig specific features for better integration. While it could be a library, it would be most beneficial as an official ABI.

My proposal is to define an ABI that could be used by zig/other languages to better integrate between the module boundary. It certainly could be supported by usermode code in the std.

Below are language features that could be nice to be exposed by the abi:

• errors (/sets/unions)
• tagged enums(if the tag itself changes per compilation)
• slices

[tecanec]

I agree heavily with this, and would even consider it a necessity. A fully Zig-compatible ABI is something we should have. As an example use case (if that's even needed), this allows the creation of dynamic libraries to split up an existing application and allow lighter patches without having to refactor the whole program to allow this.

A possible route for this would be to build on top of the C ABI by defining special cases without ruining compatibility. For example, it could define slices and tagged unions as structs or as multiple values. This has some advantages, such as making things simpler by not having multiple ABIs, maintaining C-compatibility even when using Zig-features (albeit not entirely seamless) and likely being easier to define and implement. One downside is that it won't allow the same level of optimization, but performance likely isn't prioritized if you're using dynamic libraries, anyway.

[pixelherodev]

I think that if we do define an ABI, it should be only for exports / externs. Otherwise, we lose a lot of the advantage of release modes.

[MasterQ32]

I don't think it's possible to have an ABI for Zig without wasting a lot of potential optimizations. Imho we don't gain a lot by having an ABI.

So, before i start: First of all, everything that touches comptime is not ABI-possible. You take a anytype param? Not gonna have an ABI for that. You take a tuple as a argument? Not gonna have an ABI for that. Same for everything else that uses comptime parameters. The same is probably true for functions that are async.

"Pro":

• A defined ABI would allow closed-source implementations of libraries with a small, prechosen set of supported platforms, destroying portability of those libs to other platforms (source based libraries, even if you may not change the source can be compiled to many more platforms than your vendor of binaries thought of, think arm-linux vs riscv-linx).
• A defined ABI would allow to load shared libraries with a subset of Zig features, stripping out most of the useful stuff

Con:

• You don't have transparent async. A function will either be async or not. Very likely to be not async in libraries as the functions cannot access your event loop without having a shared runtime library.
• You don't have comptime, making a lot of code useless when trying to have an ABI
• Error types will most likely be not possible without having severe performance hits. Error values must be negotiated between dynamic libraries, otherwise error.Foo and error.Bar may collide with the same error mapping. So loading a shared library means you have to relocate not only addresses, but also patch error values -> Zig would require to be minded in all shared library loaders on every platform. Not gonna happen. @suirad you're strategy of error hashing is likely to collide and there's no way to statically resolve this. It has to be a dynamic process.
• Having an ABI enforces stuff like struct/union layouts. This prevents Zig for a whole class of optimizations currently possible due to having undefined struct layout. Simple example: Compiling for release-fast could align struct members such that they are aligned for the fastest memory access, release-safe can reorder the fields to be somewhat compact and fast, and release-small could enforce all struct to be packed struct, but with reordering, making the memory footprint of the program way smaller than ever possible in C/C++

I am strictly against defining an ABI for Zig programs. The language is meant to be compiled from source (which has a lot of important properties like maintainability, improved whole-program optimization, ...) and not having precompiled libraries, but let's consider we actually built an abi. What would we have to do?

First: An ABI is platform/CPU specific. Having a cross-cpu ABI makes no sense. So we would have to define a additional ABI in a document for Zig+CPU. Shared libraries using that ABI will only be compatible to other Zig programs and probably have no chance to ever interact with other languages but Zig (otherwise we could just use the C(ommon) ABI for that platform which is already possible).

Apart from that, what has to be defined?

• struct layouts
• layout for tagged enum
• memory layout for slice types
• alignment and size for all non-pow2-integers
• layout for optionals
  
  
  
  • All non-nonnull pointers are special cases here!
  
  
• Sizes for untagged enum types
• error value negotation
• error union type (could just be a tagged union)
• result location semantics
• async semantics and frame layouts

So as you can see, i'm strongly against having an ABI for Zig. It would hurt the project both in performance, but even more in maintainability. Keep your interactions with the outer world to the C(ommon) ABI, so non-Zig projects will also profit from your libraries and pure-zig libraries can still be super-optimized compared to other native languages.

[vi]

Can Zig ever be a "main system programming language for a shared libraries-based platform" without such ABI?

Swift have done its ABI. Why not Zig?

Obviously, it is not possible to have a proper ABI without a compromise in maintainability and performance. But that performance hit is only expected to be around the ABI surface (which includes the types referenced by those functions), not around private functions.

---

Imagine a Zig operating system. How would it share code? How would it update WhateverSSL? How audio effects plugins would work?

---

The ABI doesn't needs to be called "a Zig ABI". I think there can be multiple ABIs with different compromises. This can be a .C ABI plus some additional rules for additional features. It can be a good idea to just support Swift ABI directly (as far as read somewhere, it is a nice work of engineering art).

[suirad]

@MasterQ32 While I agree with pretty much all of your points, I don't believe this proposal is advocating for as drastic as a change as you are addressing.

I may have been too vague in my description but my vision of the scope of this proposal is effectively the following attributes:

• Only applicable during extern/export; this shouldn't change how anything else works internally to zig
• Being during extern/export; it will also keep all existing limitations during that time
• Its use in zig would effectively be a calling convention that is the C calling convention + extensions
• Being based on the C ABI, other languages that want to have closer interop with a zig library could easily use it by just adding code for the extentions and in all other cases treat it just like the C ABI.
• The extentions would facilitate interacting with zig features(i.e. returning errors)

---

The result of that would allow more Zig features to be available during export/extern; an example is the following code snippet would work:

export fn thing() callconv(.Zig) !void {
    //....
}

---

It was mentioned that this would conflict with how error unions work, however this could be handled differently/seperately(extern errorset?) from normal errors and I am not suggesting any specific implementation.

[MasterQ32]

Okay, this sounds way better. So you actually want to define some type layouts for callconv(.C) functions, and don't build a fully featured Zig ABI. I would still stick to extern types then for struct / union / enum, define a memory layout for slices and tagged unions (with only external fields). I still don't think that errors will work though.
Not even in a technical, but in a theoretical sense. As ABIs require hard values and errors are a set of values defined by integers, but errors are equal-by-name which makes it hard to define an ABI for that. It isn't done by assigning each error a unique number as these numbers may differ between libraries!