Ripgrep: Make macOS prebuilt binary universal

[nico]

It'd be nice if there was a prebuilt binary of rg that ran without Rosetta 2 on ARM Macs. There could either be a separate macOS-arm binary, or there could be a single universal binary that runs on both ISAs.

https://github.com/rust-lang/rust/issues/73908 covers support for this in upstream rust. This feature request is probably blocked on that completing.

https://github.com/shepmaster/rust/blob/silicon/silicon/README.md has some notes on cross-compiling.

[shepmaster]

FWIW, ripgrep compiles fine on these machines, so that shouldn't be a problem.

% file $(which rg)
~/.cargo/bin/rg: Mach-O 64-bit executable arm64

[BurntSushi]

Thanks for the feature request. But there are some things I don't understand. Please consider that when filing feature requests, that not every uses macOS. So it would help to provide more details. Some questions I have for example:

• I have no idea what it means to "run without Rosetta 2." Probably because I don't know what Rosetta 2 is. But even if I did, I don't know what it means to produce a binary without it.
• I don't know what you mean by a "universal binary" in the feature title. Universal in what sense?

Either way, I don't see myself spending time on this any time soon. If someone else wanted to work on this, then my requirements are probably the following:

1. It should work with Rust stable. I'm open to something that only works with Rust nightly, but it should be using something that is on track to be stabilized and is unlikely to see major changes.
2. It should work with cross. @shepmaster's instructions look great, but I'd really prefer that it work like other binaries that are produced via cross compiling.
3. And finally, that it can be done automatically via GitHub Actions.

If these requirements aren't met, then I'd probably say that Rust and macOS arm aren't ready yet. The main thing I want to avoid is adding CI for a binary that breaks frequently.

[atouchet]

@BurntSushi

Rosetta 2 is a x86-64 to ARM64 translator. See: https://developer.apple.com/documentation/apple_silicon/about_the_rosetta_translation_environment

Running without Rosetta 2 means running as a native ARM64 application.

Universal binary is Apple's name for fat binaries which include binaries for multiple instruction sets. See: https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary

Universal binary in this sense means that it's packaged with both x86-64 and ARM64 binaries.

[BurntSushi]

OK, thanks, that helps clarify things. Unless producing a universal binary with the Rust toolchain is easy, I'd expect to host two binaries: one for x86_64 and one for arm. Just like we do for other platforms.

[shepmaster]

• It should work with Rust stable.

Not yet. In 6 weeks, perhaps, but right now you need beta.

• It should work with cross. @shepmaster's instructions look great, but I'd really prefer that it work like other binaries that are produced via cross compiling.

Perhaps this comment should start some effort in the cross repo. This works today outside of cross, however:

% rustup component add rust-std --toolchain beta --target=aarch64-apple-darwin

% SDKROOT=$(xcrun -sdk macosx11.0 --show-sdk-path) \
MACOSX_DEPLOYMENT_TARGET=$(xcrun -sdk macosx11.0 --show-sdk-platform-version) \
cargo +beta build --target=aarch64-apple-darwin

    Finished dev [unoptimized + debuginfo] target(s) in 29.32s

% file target/aarch64-apple-darwin/debug/rg
target/aarch64-apple-darwin/debug/rg: Mach-O 64-bit executable arm64

• And finally, that it can be done automatically via GitHub Actions.

This should be achievable.

Unless producing a universal binary with the Rust toolchain is easy

Depends what "with the Rust toolchain" entails. After doing the above build...

% cargo +beta build --target=x86_64-apple-darwin

% lipo -create -output rg target/{aarch64,x86_64}-apple-darwin/debug/rg

% file rg
rg: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
rg (for architecture x86_64):   Mach-O 64-bit executable x86_64
rg (for architecture arm64):    Mach-O 64-bit executable arm64

[BurntSushi]

That looks great! I'd be happy adopting all of that at any time. Looks simple enough, and it being in beta would be fine with me.

[BurntSushi]

@shepmaster Thank you for answering my questions with great detail. :-)