Rust: No rustc or cargo for aarch64-pc-windows-msvc

[tleasor]

Currently there are no native rustc or cargo binaries for Windows on ARM64. The 32-bit x86 binaries (i686-pc-windows-msvc) work correctly, but suffer a noticeable performance and battery life penalty.

I've tried to build rustc myself, but have had quite a few issues trying to cross-compile it since there are no pre-built bootstrap binaries for aarch64-pc-windows-msvc. Cargo builds without issue, but doesn't make much sense without a native rustc.

What would it take to get this working and available via rustup? I'm happy to keep working on this myself but am slightly lost as to how to proceed.

[tleasor]

I've managed to cross-compile rustc, cargo and most of the default components now after fixing a build issue in a dependency (psm), but unfortunately the built compiler doesn't work. Trying to compile the default "hello world" project fails silently when using rustc directly, and cargo fails with a "STATUS_ACCESS_VIOLATION" error, which seems to be a memory-related issue.

[Stammark]

I have also been experimenting with this on our side! I had successfully built a rustc using older source code (April 30th nightly comes to mind), but the STATUS_ACCESS_VIOLATION errors started happening after the stacker/psm dependency got added, so I investigated that for a bit and just raised this:

https://developercommunity.visualstudio.com/content/problem/1088946/armasm64-not-erroring-when-given-no-area-name-give.html

I _think_ that just adding a dummy area name might solve that. I haven't tested this properly for rustc, yet, though. The latest I got to last was getting problems with my stage0 being too old to compile current head-of-trunk-nightly.

I also had issues with compiling RLS due to the use of the old winapi 0.2.8, through mio 0.6.X and tokio, but this should fix itself when tokio fully updates to mio 0.7 instead of mio 0.6.X (see https://github.com/tokio-rs/tokio/issues/1190 )

[tleasor]

@Stammark Thanks for the tip! Adding a dummy area name in psm has resulted in a working rustc binary.

Compilation times are greatly improved in my limited testing (I've seen 40-50% reductions so far, I'll compile a few popular projects to get some accurate numbers which I'll post here).

One issue that I've noticed is that it fails to find link.exe, which the x86 rustc finds without issue (I'm working around this using the VS Developer Command Prompt at the moment). I've skimmed the relevant code and it looks like I just need to fix the cc dependency to use x86 tools on an ARM64 host. I'll hopefully have that done within the next few days.

I also had issues with compiling RLS due to the use of the old winapi 0.2.8, through mio 0.6.X and tokio, but this should fix itself when tokio fully updates to mio 0.7 instead of mio 0.6.X (see tokio-rs/tokio#1190 )

I noticed the same issue myself when doing an extended build, fortunately rust-analyzer works well on ARM64 when built from source and the rest of the tools (cargo, clippy, rustfmt, rust-src and rustdoc) compile fine, although I haven't tested them yet.

[Stammark]

Regarding psm: https://github.com/rust-lang/stacker/issues/37 (and our PR), so that should be good when the next deps bump happens!

Agreed on link.exe, I'm basically still doing the same thing as you by using the vcvarsx86_arm64 script to set up environment variables.
I have then also been experimenting with hacks on transferring those up into Powershell or MSYS2 for the run-make-fulldeps testsuite, but need to check these with people so that I'm more confident that I'm not breaking anything else in the process

For changes in cc @danielframpton might be interested in hearing what you have in mind, too ;)

Also, massive thank you for the contributions!

[tleasor]

Yep, I'd just seen those fixes on stacker and cc and was about to edit my comment that I'd spoken too soon!

I have then also been experimenting with hacks on transferring those up into Powershell or MSYS2 for the run-make-fulldeps testsuite, but need to check these with people so that I'm more confident that I'm not breaking anything else in the process

That sounds great, I'll have a look into adding support for this target to the release builders.

Also, massive thank you for the contributions!

Thanks for all your help! I'd have been lost without you pointing out that silent assembly error!

[arlosi]

I've been working on getting the cross-build of aarch64 working so we can add it to the CI builders.

alexcrichton/cc-rs#524 fixes the failure to run the aarch64 rc.exe on x86_64.

https://reviews.llvm.org/D83022 resolves an issue where the LLVM cross build recurses and tries to do a native build of llvm-nm by allowing the Rust build to specify the path to the native LLVM_NM. This is not in yet. Once it is in, we can backport it to the Rust fork of LLVM.

arlosi/rust@7b73704148b7e4daa33dba3a9063a1b976a25160 has the updates to the Rust build to use the new LLVM_NM option and set the --target flag when using clang-cl (as the Rust CI builders do). I will create the PR for this once the LLVM change is in.

With all that in place, I can cross-build for aarch64-pc-windows-msvc from x86_64-pc-windows-msvc from a normal command prompt with x.py build.

[Alovchin91]

Hi, what's the status of this so far?

I've recently bought a Surface Pro X so I would be happy to help out if I can.

There are two things that I've found recently:

1. Apparently LLVM 10.0.1 can be compiled to run natively on Windows on Arm64: https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/native-clang-for-windows-on-arm
   LLVM 10.0.1 seems to be ready although there are no pre-built binaries available yet: https://github.com/llvm/llvm-project/releases/tag/llvmorg-10.0.1

2. Microsoft has instructions on how to build Python for Windows on Arm64: https://docs.microsoft.com/en-us/windows/iot-core/developer-tools/python#using-python-on-windows-iot-core-arm64
   These are for Windows IoT Core, but I guess they can be used to compile Python for the usual Windows as well.

Would it make sense to try to build the compiler on an Arm64 machine?

Also, I see there is a tracking issue to Use lld by default on x64 msvc windows. Would it make sense to create a similar issue for aarch64 msvc windows?

Additionally, would it probably make sense to try and use clang-cl instead of MSVC (since the latter doesn't run natively on Windows on Arm64 yet)?

Given this, plus the fact that the latest Visual Studio Code runs natively on Arm64, the full development toolchain could be successfully used on Windows on Arm64.

[Stammark]

Hi Alexander! Things are much better now that Arlo's changes have all been merged. We can easily cross-compile a native compiler and then use that in a fully-aarch64-native-build, but the MSVC toolchain (which is not aarch64-native) is still needed for the linker and the libraries.

Would it make sense to try to build the compiler on an Arm64 machine?

Yes it would! Especially if you plan on using the Surface as a host machine (compile on it), not just a target (compile on some x86 machine to --target=aarch64-pc-windows-msvc and transfer the binary over)

Also, I see there is a tracking issue to Use lld by default on x64 msvc windows. Would it make sense to create a similar issue for aarch64 msvc windows?

I'm not very familiar enough with how this works to comment, but I'd guess that there's nothing wrong with creating a similar issue for aarch64 or commenting there for added aarch64 support. Maybe one of the MS folks or @JamieCunliffe have more insight :D

Additionally, would it probably make sense to try and use clang-cl instead of MSVC (since the latter doesn't run natively on Windows on Arm64 yet)?

Yes it does! I had only tried an x86_32 clang-cl at the time but native should be just as possible. This also gets around some intermittent failures when compiling LLVM with MSVC's cl.exe (but this was a while back and may have been fixed now).

ARM64 Python

No idea! But I want to try this now. I always relied on the x86_32 version.

I'll paste here the ./configure options I used last time I looked at this (these generate a config.toml). Note that an MSVC installation with all ARM64 workloads is needed for this (the installation gets auto-detected) and I was running in MSYS2-32bit (Mingw-w64 32 bit command line) with the following packages:

pacman -S --noconfirm --needed base-devel ca-certificates make diffutils tar git mingw-w64-i686-cmake mingw-w64-i686-ninja mingw-w64-i686-python3

Step 1: Cross-compile (this will take FOREVER):

./configure --build=i686-pc-windows-msvc                 \
   --host=i686-pc-windows-msvc,aarch64-pc-windows-msvc   \
   --target=i686-pc-windows-msvc,aarch64-pc-windows-msvc \
   --enable-full-tools --enable-profiler                 \
   --enable-missing-tools                                \
   --set llvm.clang-cl=C:/LLVM/bin/clang-cl.exe

(that was a x86_32 clang-cl from the LLVM releases but a native one should work fine, too)

and then x.py build

Step 2: Fully native build:

./configure \
    --build=aarch64-pc-windows-msvc               \
    --host=aarch64-pc-windows-msvc                \
    --target=aarch64-pc-windows-msvc              \
    --enable-full-tools --enable-profiler         \
    --enable-missing-tools --enable-ninja         \
    --set build.rustc=C:/some/path/to/native/rustc.exe      \
    --set build.cargo=C:/some/path/to/native/cargo.exe      \
    --set build.rustfmt=C:/some/path/to/native/rustfmt.exe  \
    --set llvm.clang-cl=C:/some/path/clang-cl.exe (native or x86_32, probably both work?)\
    --set rust.incremental                        \
    --set rust.backtrace                            \
    --enable-verbose-tests                        \
    --enable-codegen-tests                        \
    --enable-lld

and then x.py build

EDIT: To update on the second one, I realised that this one won't work out-of-the-box with the MSVC auto-detection in cc-rs. Instead, it can be run in cmd in a vcvarsall environment (e.g. by running CALL "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvarsx86_arm64.bat" in that command line) -- but you can still generate the config.toml in MSYS2 as I wrote above.

(in both cases RLS will fail btw)

[Alovchin91]

@arlosi Nice to see that it has been promoted to Tier 2 Development Platform!

However, how can it be used? 🙃 When I try to install it using rustup-init.exe, I get an error that there is no such platform:

info: profile set to 'minimal'
info: setting default host triple to aarch64-pc-windows-msvc
info: syncing channel updates for 'nightly-aarch64-pc-windows-msvc'
info: latest update on 2020-10-13, rust version 1.49.0-nightly (8dae8cdcc 2020-10-12)
error: target 'aarch64-pc-windows-msvc' not found in channel.  Perhaps check https://forge.rust-lang.org/release/platform-support.html for available targets

[ehuss]

I think similar to #77895, the HOSTS list needs to be updated.

[arlosi]

I can submit that now 👍

[Alovchin91]

Oh, logged in to do just that 😅 Please go ahead @arlosi 🙂