Msys2-packages: Get a functional ARM64 compiler

[anaisbetts]

Hi, I'm looking into what it would entail to get MSys2 running under ARM64 on Windows, and I wanted to kick off a separate thread with a few questions / making sure I understand the challenges. The biggest one seems to be how to get a functioning compiler working.

From what I understand, at the end of the day, msys2 uses the gcc compiler (via mingw-w64), which doesn't support ARM64 on Windows, so we'd have to build MSys2 with clang. But! The tricky part is, MSys clang package seems to require gcc!

So, it seems that we somehow have to build enough of mingw-w64 using clang cross-compilation to be able to build things with aarch64-w64-windows-gnu, then cross-compile clang targeting the ARM64 mingw-w64. This will eventually be way easier once we've solved the bootstrapping problem and we can just use prebuilt binaries to download ARM64 clang and go.

Once we're able to build ARM64 MSys2 packages, it seems like most of the work will be to make sure they all build under clang instead of gcc, and we'll probably need to also be able to build arm64 binaries from 64-bit Intel since setting up a native ARM64 build machine is probably a non-starter.

Am I missing anything? I'm basically coming into this blind so I may be extremely mistaken / confused! Any guidance / pointers to the right direction would definitely be appreciated 😅

Refs #1276 #1787

[Biswa96]

As discussed in this thread Will There be an ARM/ARM64 Windows Support Soon?, the llvm-mingw can be used in MSYS2 environment to create ARM & ARM64 PE binaries. But it can not be fully replaced with GCC. Most of the packages (e.g. gmp, binutils, coreutils etc.) need some tuning for ARM. But it is possible to invoke llvm-mingw and compile higher level programs.

btw, are you interested to compile any package for ARM?

[anaisbetts]

@Biswa96 My end-goal is to get Git for Windows built for native ARM64, but I'd like to generalize this work / make something maintainable instead of making something really one-off and hacky. I'll have a look at llvm-mingw.

Is ARM64 support something that msys2 maintainers would be willing to host if someone did the initial Work to get things working? I don't mind trying my hand at cleaning up all of the build breaks for clang + llvm, but I can't really set up my own msys2 mirror / infrastructure, etc etc

[lazka]

My end-goal is to get Git for Windows built for native ARM64

@dscho ^ have you looked into ARM64 for git?

[dscho]

There is an active PR to compile Git for Windows for ARM64 using MSVC, and the plan was to drop the executables into an i686 version of Git for Windows (which can apparently be run on ARM64 Windows). That's all we got for now.

[Biswa96]

If git-for-windows and all the dependencies can be compiled using clang then llvm-mingw can also be used. cc. @mstorsjo

[mstorsjo]

There is an active PR to compile Git for Windows for ARM64 using MSVC, and the plan was to drop the executables into an i686 version of Git for Windows (which can apparently be run on ARM64 Windows). That's all we got for now.

That sounds like it should work, yes. Compiling the core git parts with llvm-mingw could also work, as they're already portable if they can be built with MSVC, but whichever toolchain you are most comfortable with for those parts work. The bash/msys underpinnings is the big question, and just shipping i686 binaries for that should be a decent stopgap solution. I've tested 32 bit msys2 on arm64, and iirc it worked just fine, so it didn't do anything that would trip up the emulation.

If git-for-windows and all the dependencies can be compiled using clang then llvm-mingw can also be used.

In general, it's not only about being able to compile things with clang, but also working with lld instead of binutils ld (most high level things work just fine, but there are features of binutils ld that aren't supported in lld when targeting COFF), and a number of other things.

As git-for-windows essentially also contains a copy of msys/msys2 in some form, you'd need support for compiling msys binaries with the llvm-mingw toolchain, and that's not supported yet.

Compiling things for MSYS essentially requires compiling things for the cygwin target, if I've understood things correctly. And clang/llvm does contain some amount of support for targeting cygwin, but it is essentially unmaintained as far as I know - I don't know of anybody using it, and I haven't seen a single patch towards fixing any aspect of it (during the last few years when I've done hundreds of patches to fix MinGW related aspects in LLVM, Clang and LLD).

So far I've had my hands full with just MinGW related bits, so I haven't even started peeking into these things. (Also fwiw, my policy regarding that work so far, has been to work closely with upstream so that llvm-mingw uses a fully unpatched llvm/clang/lld build, and the same goes for mingw-w64. The little bit of customization that there is, is in the scripts that builds and assembles it all, and in tool wrapper scripts.)

Some of the fixes for MinGW directly also benefit Cygwin, so things might have improved implicitly, but other fixes are behind cases like if (mingw) in the codebase. It might be that those also just work if the cases are changed into if (mingwOrCygwin), but there could also be lots of other cases where further changes are needed - and as the general state of cygwin support in llvm is unknown, I haven't spent time along the way to see how my fixes behave in a cygwin context.

As far as I see it, the steps towards MSYS2 (or the msys underpinnings of git-for-windows) on ARM64 would be this:

1. Make sure that clang and lld work for the cygwin target in general (on x86), within a normal cygwin install. For trivial C apps, this might kind of work as is. For large C++ apps, it might require enabling a number of the mingw specific tweaks I've done over the years.
2. Once things works for cygwin (for which there's at least some form of support for in LLVM/Clang), make it work for MSYS2 as well. Not sure if this is a no-op or if it requires some amount of differences for e.g. linking default libraries (with different names).
3. Figure out how to bootstrap the MSYS2 runtime via cross compilation, using the regular GCC used in MSYS2 today. Not sure if this is common knowledge and a commonly done operation, or if MSYS2 only is developed on top of itself. Th
4. Once cross-bootstrapping with GCC works, replace GCC with Clang, LLD and the llvm based runtimes (compiler-rt, libunwind, libcxxabi, libcxx - replacing libgcc and libstdc++). This can probably benefit from cross-wrappers like the ones in llvm-mingw, but with a different target triple than the mingw ones used there. For the runtimes, it can probably also reuse a lot of the build scripts from llvm-mingw. All of this could potentially also be merged into llvm-mingw in the end.
5. In the cross-setup, target aarch64-pc-msys instead of x86_64-pc-msys. This can potentially be the step with the least amount of work - unless there are cases in the msys runtime with assembly or other architecture specific bits. I'd imagine there can be very architecture specific bits in e.g. the fork emulation in the base of the runtime - in that case this step can potentially be hard.

So as you can see, for the general "MSYS2 on ARM64", there's a lot to do and figure out, before questions like hosting and infrastructure are relevant.

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

[harrisonmetz]

This question is probably naive (and an educationally focused question), but why can't llvm nor gcc be built with msvc? Is it the fact that the compiler code itself can't be compiled, or that the issue is just lack of target support for windows arm64? Thanks.

[mstorsjo]

This question is probably naive (and an educationally focused question), but why can't llvm nor gcc be built with msvc?

Llvm can be built with msvc. Gcc probably can't (I don't think it's supported or commonly done), although it might be possible with some effort.

Is it the fact that the compiler code itself can't be compiled, or that the issue is just lack of target support for windows arm64? Thanks.

Gcc doesn't support generating code for windows/arm64, so even if you'd use a different compiler for having a gcc running _on_ windows/arm64, you could only have a gcc producing code for a different OS or architecture.

Llvm supports generating code for windows/arm64, and can itself also run on windows/arm64, if compiled with either msvc or clang/llvm. https://github.com/mstorsjo/llvm-mingw provides this; both cross compilers for running on other OSes producing code for windows/arm64, and for running directly on windows/arm64 as well.

The discussion above is about how to get the msys2 runtime built with clang/llvm.

[harrisonmetz]

Ok, thanks for the explanation.

[Biswa96]

One question, if someone has knowledge can Windows 10 ARM64 support be added in whole GCC toolchain _without relying on clang_? Or is it impossible to do so?

[mstorsjo]

One question, if someone has knowledge can Windows 10 ARM64 support be added in whole GCC toolchain _without relying on clang_? Or is it impossible to do so?

Of course it's possible, but you should not think that it's trivial - it's not; I added support for it in Clang/LLVM.

Before Windows on ARM64, there were also efforts for Windows on ARM32 (which Clang also supported before), and a GCC maintainer was involved and wrote some initial proof of concept patches for GCC to target Windows on ARM32, but those patches were never completed or upstreamed. So for someone with less experience of GCC it's certainly an even bigger task.

[Myriachan]

Even having an GNU assembler for ARM64 that supports Windows ARM64 .pdata is hard enough...

[stackTom]

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

So would it be feasible to compile GMP, MPFR, and MPC for arm64? I got static and dynamic x86 and x64 bit builds of these libraries with msys2, and was wondering how feasible ARM would be.

[stackTom]

For just compiling normal mingw based projects for ARM64, llvm-mingw works fine as is right now, both for compiling on MSYS2 on x86, or for cross compiling from a linux environment.

So would it be feasible to compile GMP, MPFR, and MPC for arm64? I got static and dynamic x86 and x64 bit builds of these libraries with msys2, and was wondering how feasible ARM would be.

It is possible, see: https://github.com/mstorsjo/llvm-mingw/issues/131

I don't have a windows for arm machine to test if the built libraries run, but they do successfully compile and can be linked in visual studio.

[Biswa96]

If Git in Windows can be compiled without msys2/cygwin then I can try to build it for Windows ARM64. In other words, if mingw-w64-git package can be created then it is possible to port it to ARM with no or little effort using llvm-mingw toolchain.

[dscho]

Git for Windows can indeed be compiled without MSYS2/Cygwin, at least with the upcoming CMake support. Without CMake support, it would require a GNU make, I think.

[driver1998]

Git for Windows can indeed be compiled without MSYS2/Cygwin, at least with the upcoming CMake support. Without CMake support, it would require a GNU make, I think.

GNU make is fine, we can emulate make, or build the thing in WSL (which is native)

[dscho]

@driver1998 feel free to test this hypothesis... :grinning:

[dennisameling]

Just leaving this here for reference: we're currently working on a native Git for Windows executable (git.exe etc.) while relying on mingw32 for the packages (bash/curl/etc.). This could be a good first step prior to having an _actual_ compiler for arm64, and already leads to a ~2x speed improvement for basic Git commands 🚀

Updates here: https://github.com/git-for-windows/git/issues/2346#issuecomment-751687623

[jeremyd2019]

1. Make sure that clang and lld work for the cygwin target in general (on x86), within a normal cygwin install. For trivial C apps, this might kind of work as is. For large C++ apps, it might require enabling a number of the mingw specific tweaks I've done over the years.
2. Once things works for cygwin (for which there's at least some form of support for in LLVM/Clang), make it work for MSYS2 as well. Not sure if this is a no-op or if it requires some amount of differences for e.g. linking default libraries (with different names).

There is now a clang/lld packaged on MSYS2, thanks to @mati865 🚀