Zig: OpenGL not found on newer macOS

[hryx]

OpenGL on macOS seems to have moved recently, so projects that use it may fail to build:

/path/to/tetris/src/c.zig:1:9: error: C import failed
pub use @cImport({
        ^
/usr/local/include/GLFW/glfw3.h:140:12: note: 'OpenGL/gl.h' file not found
  #include <OpenGL/gl.h>
           ^

The new path to the framework is:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks

...and seems to require XCode being fully installed.

See @mikdusan's workaround (I think using OpenGL from source rather than Homebrew): https://gist.github.com/mikdusan/0645a06f14c2167130ac4b00feed82ed#gistcomment-2883034

[mikdusan]

Just to add a few tidbits that might help in issues such as these:

-H option

use the -H option for gcc/clang and it will show you pathnames of resolved header files

-v option

use the -v option for gcc/clang to show sub-process launch args and include/framework search paths

-F option

this is like -L option for libraries but for apple frameworks only

-f option

this is like the -l option for libraries but for apple frameworks only

#include

using frameworks, the include directive always is prefixed with framework's name, in this case OpenGL. The filesystem layout does not include OpenGL as a parent to file.h just in case you go around snooping.

What The Fun is going on with Apple toolchains?

Apple tools coming from Apple's developer site come in 2 flavours; Xcode and Command Line Tools (CLT). CLT is the "unix" way to build stuff - that is tools, SDK, etc, is all placed in locations found in /usr or /System/Library . The actual "application" home of CLT is actually tucked away in /Library/Developer. This is analogous to Xcode's typical /Applications/Xcode.app location.

CLT is only required for unix-style development. If you only intend to use Xcode IDE-driven development, then there is no need for CLT. But things like building your own free software, homebrew, macports, all just work easier with CLT. Without it is possible but you need to be a build guru or a masochist.

Be aware CLT has some subtle artifacts. macOS comes with Framework runtimes installed but not headers (analogous to -dev packages in linux land). Installing CLT will also populate things like gl.h from OpenGL framework. So if you don't have gl.h in /System/Library/Frameworks it's because you never installed CLT.

Uninstall can usually be done by removing /Library/Developer but sadly this still leaves header files and such that were added to /System/Library/Frameworks.

Xcode has everything CLT has and more. IDE, and other GUI tools. Technically Xcode doesn't depend on CLT (or at least it didn't last time I verified).

xcode-select

This is the part that muddles things for devs mainly because it's a misnomer. It in facts lets you select CLT or Xcode and set that as your "unix" environment default. That is, what /usr/bin/clang or /usr/bin/gcc will actually use.

Here's a small session that:

1. starts off with CLT and an arbitrarily named Xcode-beta.app folder installed on macOS 10.13
2. CLT is default (tools available to unix environment)
3. make Xcode-beta default with --switch
4. reset back to CLT with --reset

NOTE: it this case CLT and Xcode both have identical versions of clang. But they can easily be different.

HOW TO BUILD CLEAN BINARIES FOR DISTRIBUTION

When building on macOS you can just build against whatever host and frameworks are on your platform. Various command-line options to the compiler can be used to cap (limit) API usage to a specific version. But even before you get there, your toolchain is capable of building against a target SDK. This target SDK could be the SDK from an older version of macOS. Let's assume your build host is 10.14, you could build against a 10.12 SDK to produce 10.12 compatible binaries. This is also how to build for iOS, tvOS, watchOS and their respective sumulator SDKs.

xcrun can help with that. You can prefix your command lines with that tool as follows; but beware it has to be used consistently. If you do it for a autotools configure pass, it should also be used for make and make install launches; note: you must xcode-select an Xcode for this to work. ie: CLT does not support -sdk builds.

To give you an example of the serious effect xcrun -sdk ... has; let's select an xcode, and compare clang -v output without and with xcrun:

clang -v -c foo.c

Apple LLVM version 9.1.0 (clang-902.0.39.1)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.13.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name foo.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -target-linker-version 351.8 -v -dwarf-column-info -debugger-tuning=lldb -coverage-notes-file /Users/mike/foo.gcno -resource-dir /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.1.0 -fdebug-compilation-dir /Users/mike -ferror-limit 19 -fmessage-length 0 -stack-protector 1 -fblocks -fobjc-runtime=macosx-10.13.0 -fencode-extended-block-signature -fmax-type-align=16 -fdiagnostics-show-option -o foo.o -x c foo.c
clang -cc1 version 9.1.0 (clang-902.0.39.1) default target x86_64-apple-darwin17.5.0
ignoring nonexistent directory "/usr/local/include"
#include "..." search starts here:
#include <...> search starts here:
 /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.1.0/include
 /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.

xcrun -sdk macosx10.13 clang -v -c foo.c

Apple LLVM version 9.1.0 (clang-902.0.39.1)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.13.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name foo.c -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -target-linker-version 351.8 -v -dwarf-column-info -debugger-tuning=lldb -coverage-notes-file /Users/mike/foo.gcno -resource-dir /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.1.0 -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -fdebug-compilation-dir /Users/mike -ferror-limit 19 -fmessage-length 0 -stack-protector 1 -fblocks -fobjc-runtime=macosx-10.13.0 -fencode-extended-block-signature -fmax-type-align=16 -fdiagnostics-show-option -o foo.o -x c foo.c
clang -cc1 version 9.1.0 (clang-902.0.39.1) default target x86_64-apple-darwin17.5.0
ignoring nonexistent directory "/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/local/include"
ignoring nonexistent directory "/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.1.0/include
 /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include
 /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks (framework directory)
End of search list.

[whatupdave]

Here's my massively hacky solution:

$ ln -s "$(xcrun --sdk macosx --show-sdk-path)/System/Library/Frameworks/OpenGL.framework/Headers" \
  /usr/local/include/OpenGL

That got the tetris example to run on macos.

[andrewrk]

Related: #2041

[SeekingMeaning]

Extension of Dave's workaround (so that you don't have to change /usr/local/include):

1. Create directory named include
2. Look for OpenGL.framework/Headers in known locations
3. Create symlink to <path>/OpenGL.framework/Headers as include/OpenGL
4. Add include as include directory

build.zig:

const std = @import("std");
const Builder = std.build.Builder;
const fs = std.fs;

pub fn build(b: *Builder) !void {
    const exe = b.addExecutable("main", "main.zig");
    exe.setBuildMode(b.standardReleaseOptions());
    b.default_step.dependOn(&exe.step);

    exe.linkSystemLibrary("glfw3");
    if (comptime std.Target.current.isDarwin()) {
        if (!try fileExists("include")) {
            try fs.makeDir("include");
        }
        if (!try fileExists("include/OpenGL")) {
            const locations = [_][]const u8{
                "/System/Library/Frameworks/OpenGL.framework/Headers",
                "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/OpenGL.framework/Headers",
                "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/OpenGL.framework/Headers",
            };
            blk: {
                for (locations) |dir| {
                    if (try fileExists(dir)) {
                        try fs.symLink(dir, "include/OpenGL");
                        break :blk;
                    }
                }
                @panic("Could not find OpenGL headers");
            }
        }
        exe.addIncludeDir("include");
    }

    const run_cmd = exe.run();

    const run_step = b.step("run", "Run the program");
    run_step.dependOn(&run_cmd.step);
}

fn fileExists(filename: []const u8) !bool {
    fs.File.access(filename) catch |err| switch (err) {
        error.FileNotFound => return false,
        else => return err,
    };
    return true;
}

And then run zig build run

[mikdusan]

Some example source to help build an OpenGL app on macOS:

• build-time probe of sdk path
• append to get frameworks path
• add frameworks option to executable

const std = @import("std");
const Builder = @import("std").build.Builder;

fn probeSdkPath(b: *Builder, buf: *std.Buffer) void {
    const args: [][:0]const u8 = &[_][:0]const u8{ "xcrun", "-show-sdk-path" };

    const child = std.ChildProcess.init(args, b.allocator) catch |err| {
        std.debug.panic("Unable to initialize child process {}: {}\n", .{args[0], @errorName(err)});
    };
    defer child.deinit();

    child.stdin_behavior = .Ignore;
    child.stdout_behavior = .Pipe;
    child.stderr_behavior = .Ignore;
    child.env_map = b.env_map;

    child.spawn() catch |err| std.debug.panic("Unable to spawn {}: {}\n", .{args[0], @errorName(err)});

    var stdout_file_in_stream = child.stdout.?.inStream();
    stdout_file_in_stream.stream.readUntilDelimiterBuffer(buf, '\n', 256) catch {
        std.debug.panic("Failed to read output from {}\n", .{ args[0] });
    };

    const term = child.wait() catch |err| {
        std.debug.panic("Unable to spawn {}: {}\n", .{args[0], @errorName(err)});
    };

    switch (term) {
        .Exited => |code| {
            const expect_code: u32 = 0;
            if (code != expect_code) {
                std.debug.panic("Process {} exited with error code {} but expected code {}\n", .{
                    args[0],
                    code,
                    expect_code,
                });
            }
        },
        .Signal => |signum| {
            std.debug.panic("Process {} terminated on signal {}\n", .{ args[0], signum });
        },
        .Stopped => |signum| {
            std.debug.panic("Process {} stopped on signal {}\n", .{ args[0], signum });
        },
        .Unknown => |code| {
            std.debug.panic("Process {} terminated unexpectedly with error code {}\n", .{ args[0], code });
        },
    }
}

pub fn build(b: *Builder) void {
    var buf = std.Buffer.initSize(b.allocator, 0) catch unreachable;
    probeSdkPath(b, &buf);
    buf.append("/System/Library/Frameworks") catch unreachable;
    const frameworks_path = buf.toSliceConst();

    std.debug.warn("--> FRAMEWORKS: {}\n", .{frameworks_path});

    const mode = b.standardReleaseOptions();
    const exe = b.addExecutable("opengl", "src/main.zig");
    exe.setBuildMode(mode);
    exe.install();
    exe.addFrameworkDir(frameworks_path);

    const run_cmd = exe.run();
    run_cmd.step.dependOn(b.getInstallStep());

    const run_step = b.step("run", "Run the app");
    run_step.dependOn(&run_cmd.step);
}