Crystal: Default target triple baked into compiler fails on musl

[straight-shoota]

Linux builds of the Crystal compiler produced by crystal-lang/distribution-scripts (such as the ones provided for releases and nightlies) have a default target triple x86_64-unknown-linux-gnu. This seems to be hard coded into the binary.

Binaries are statically linked and run as well on linux-gnu as on linux-musl. When running on Alpine however, this default target triple is incorrect and results in linker errors (e.g. undefined reference to `errno'). This makes it pretty much unusable. The correct target triple would be x86_64-alpine-linux-musl. This can be mitigated by explicitly passing --cross-compile --target x86_64-alpine-linux-musl, but that's not very user-friendly and doesn't work when running a crystal file as macro because the default target triple is used for compiling it.

TL;DR The target triple should default to the system the compiler is running on. Because the same binary runs on linux-gnu and linux-musl it needs to detect which one to use.

[asterite]

TL;DR The target triple should default to the system the compiler is running on

Yes, it is. We just ask this from LLVM via the C API, which should be equivalent of running llvm-config --host-target on the command line.

If you run that on that machine, what's the output?

[RX14]

It doesn't. The LLVM default target triple is baked in too, and doesn't change at all when the binary is moved from machine to machine. LLVM is statically linked for the compiler binary, which is what we want. We need to do what C compilers have always done and have a build, host, and target triple. https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/Specifying-Target-Triplets.html

[straight-shoota]

@asterite I literally replaced the one occurence of x86_64-unknown-linux-gnu by x86_64-alpine-linux-musl in the compiler binary and it works as expected. So it's definitely baked in.

[RX14]

I suggest defining a host triple in Crystal::Config, which is always the --target the compiler is built with. This is different from the default_target_triple, which should be {{env("CRYSTAL_CONFIG_TARGET")}} || host_triple. I don't think we will ever need to define the --host at runtime, we can leave it out until we get a usecase.

[straight-shoota]

The issue is that "--target the compiler is built with" can differ from the host it is running on (e.g. gnu vs. musl).

I think it shouldn't matter what target triple the compiler was built with, only what it's running on. This needs to read from the system environment like llvm-config --host-target.

[RX14]

You're right, sorry. Both the host and default target need to be configurable.

There's no way to read the host triple from the environment. It must be compiled-in.

[straight-shoota]

Having the host baked into the binary is usually not an issue, because most targets won't run on any other system anyway.
The only problem (at least what I can think of) arises with my initial use case, having one linux binary which works on systems with either glibc or musl. Making the host target configurable is obviously an improvement, but still it won't work out of the box, which can be annoying.

Can't we just parse the output of ldd --version whether it's musl or glibc and use that as a default? It's certainly not a completely failsafe solution, but should provide the most likely choice in most situations.

[RX14]

I guess we can copy config.guess's logic for the platforms we support, and support --host as a build option like --target. I'd love to be able to embed config.guess, however it's GPL3 and I can't see a good way to distribute it without licence incompatibilities.

[straight-shoota]

It would certainly be nice, but I don't think we currently need to identify all platforms. The relevant part for now is really just telling appart glibc and musl and that comes from evaluating ldd --version.

[RX14]

I actually want to fix Crystal's target triple handing, so I'll assign myself.

[straight-shoota]

7282 has been a great improvement but that alone doesn't bring us closer to solving this issue.

What do you think about adding something along this to Crystal::Config.default_target_triple?

target = Crystal::Codegen::Target.new {{env("CRYSTAL_CONFIG_TARGET")}} || LLVM.default_target_triple       

if target.linux?
  musl_environment = `ldd --version`.starts_with?("musl")
  if musl_environment && target.gnu?
    return target.to_s.sub("-gnu", "-musl")
  elsif !musl_environment && target.musl?
    return target.to_s.sub("-musl", "-gnu")
  end
end

[ysbaddaden]

That it doesn't take armeabihf into account.

[straight-shoota]

In which sense? This snippet should work with arm-linux-gnueabihf and theoretically also with arm-linux-musleabihf once that is supported.

[ysbaddaden]

Oh sorry, I misread.