Zig: Comptime code generation

[drec357]

I have just come across Zig so I apologize if I don't have the syntax etc. quite right, but something like the following code-generation meta-programming functionality would be great, and would be simple and general enough to fit in with Zig's "zen" I think:

const Header = struct {
    magic: u32,
    name: []const u8,
};
fn addGetFuncs(comptime T: type) type {
    return struct {
        comptime {
            @qparse("data: Header,"); //PROPOSED
            inline for (@typeInfo(T).Struct.fields) |field| {
                @qparse("fn get_", field.name, "() ", field.field_type.name, " { return data.", field.name, "; }" ) //PROPOSED
            }
        } // "queued" string literals are parsed into enclosing context when non-dependent
    };
}

I added a similar experimental feature to Clang for C++ awhile back, but I think Zig would benefit even more because it seems to have nailed just about every design decision to enable really powerful metaprogramming -- using the same language for compile time and run time code (no constexpr headaches in C++), using the same language for the build system as well (no CMake headaches), and generally keeping everything as simple as possible so there aren't a million different AST nodes to reflect.

If you can eventually add reflection support for all the information in the AST, statements/expressions included, and combine that with a fully-general reification mechanism such as this, Zig could be at once the lowest-level language possible AND the highest-level language possible, I think.

[andrewrk]

If you want this, I recommend the ~D programming language~ Rust programming language. Zig intentionally does not have this kind of meta programming. However, this use case can be addressed in the build system by using ordinary string processing techniques. The benefits of this are clear: you end up with an actual source file that goes into the build system, gets properly integrated into the caching system, debug info, text editors, language server tooling, and IDEs.

[marler8997]

It looks like @drec357 was actually proposing compile-time language access to the AST, something the D Programming Language does not support. Walter Bright has expressed many times his opposition to that feature.

D does support code generation through string mixins, something which Zig does not support. But Andrew does bring up some good points against a language supporting this, and an alternative solution that avoids the issues that come with language support for string mixins. The issues Andrew mentions with string mixins are real and discussed in the D community. However, I haven't tried alot of code generation with Zig to know how it would compare with D mixins, so I can't offer much in practical comparisons between the two.

[drec357]

Thanks, I'll take a look a closer look at what you suggest.

The one nice thing about doing this sort of thing in a single source file, instead of generating new sources via the build system, is that you could still use fn syntax to represent such transformations, and access the results via function call syntax alongside where you defined arguments, a la const SomePrototypeStruct = struct { ...};, Interface(SomePrototypeStruct), AutoDiff(SomePrototypeStruct) etc.

That would require new non-trivial IDE support though, and further consideration of caching, and of source locations for debugging etc., so definitely understand why you would want to avoid that.

FWIW, my ultimate desire is for a single language with three features: 1) simple, complete, and fast low level interface, 2) full compile-time reflection support, and 3) fully general reification support a la "string mixins". 1 lets you get as operate with code in as low a level as you want, providing a firm foundation, 2 and 3 lets you build as high level as you want, without any mystery as to how it translates to the low level.

It's not too much of a compromise just to do 2 and 3 via AST tools and build system code generation for now though; 1 is the hard part and you seem to have done a great job with that.

Thanks again and good luck!

[smolck]

However, this use case can be addressed in the build system by using ordinary string processing techniques.

Do you (or anybody else) have a quick example of how to do this?

[Snektron]

I use it for generation of Vulkan bindings from the XML registery:
https://github.com/Snektron/vulkan-zig/blob/ab7b1835a6fa419127b77840f171c51cad806515/build.zig#L68
see also:
https://github.com/Snektron/vulkan-zig/blob/master/generator/vulkan/build_integration.zig

Its important to note that currently that is generated every time you call zig build. I think code generation via the build system is the way to go, but there is still some stuff missing like integration with the cache system to prevent regeneraiton.