Rustup: Provide full Apple Silicon (arm64/AArch64-Apple-Darwin) support in rustup

[monkeydom]

Describe the problem you are trying to solve

Provide full compatibility for Apple Silicon (AArch64/Arm64) based macs. DTKs are out there now, user facing machines will be soonish.

Describe the solution you'd like

Let AArch64-Apple-Darwin just be a fully supported platform for rustup

Notes

Note that apple currently reports arm64 as architecture.

current output:

rust % curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup: unknown CPU type: arm64 

[monkeydom]

when adding a arm64 to the aarch64 switch case, the next error one gets is:

rustup: command failed: downloader https://static.rust-lang.org/rustup/dist/aarch64-apple-darwin/rustup-init /var/folders/sq/89_rr1cd3h7g3643mmd1lg280000gn/T/tmp.fP8JB2EA/rustup-init aarch64-apple-darwin

[monkeydom]

Main Compiler support seems to be tracked at
https://github.com/rust-lang/rust/issues/73908

[shepmaster]

There's no _real_ work to be done here — Rustup compiles fine on the DTK. Once there are official Rust builds, rustup tarballs will be able to be created and uploaded. The most I see so far is updating the shell script as appropriate.

% file $(which rustup)
/Users/shep/.cargo/bin/rustup: Mach-O 64-bit executable arm64

% rustup --version
rustup 1.21.1+259 (138d33e50 2020-07-01)

Unfortunately, the tests don't yet run:

---- update_no_manifest stdout ----
thread 'update_no_manifest' panicked at 'not implemented', tests/mock/clitools.rs:1131:13

test result: FAILED. 0 passed; 23 failed; 0 ignored; 0 measured; 0 filtered out

[kinnison]

The shell script is still _real_ work. However, if there's no Rust coding to do then the _real_ work boils down to the shell script changes needed and the CI work needed.

[shepmaster]

Yes, I neglected to mention the CI work, mostly because that's something that is broadly unknown — we don't even know how we are going to do it for Rust itself yet. There's likely to be some way to cross-compile from x86_64 to aarch64 (which is what I've done by hand so far), but not actually execute the tests.

My hope is that the shell script changes will be a one or two line change (maybe just rewriting arm64 to aarch64, for example).

[kinnison]

I am hoping we will implement something to ensure that the target is properly supported as a triple, as part of build.rs at some point, so yes, hopefully there'll just be some shell tweaks and then the CI. If the Github Actions provided MacOS runners can cross to the target then we'll be sorted once the toolchain dockers are ready (assuming those run on macos too?)

[shepmaster]

Status:

• https://github.com/rust-lang/rustup/pull/2521
• https://github.com/rust-lang/rustup/pull/2525

The current assumption is that things will "just work" in the next rustup release.

[AhmedSoliman]

When is the next release date to get support for Apple M1?

[kinnison]

There's a few more issues we need to resolve before the next release, but I'm hoping within a couple of weeks for the rustup release.

[shepmaster]

For people who don't mind getting their hands dirty, you can download the x86_64 version of rustup and then build the current development branch natively.

• Install x86_64 rustup and aarch64 rustc / cargo:

  arch --x86_64 \
  sh <(curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs) \
  --default-host aarch64-apple-darwin \
  --default-toolchain nightly
  

• Clone this repo

• Build it and install as usual

Of course, this will be whatever arbitrary commit you check out, but I think it's fine 😇

[MaxDesiatov]

* Install x86_64 `rustup` and aarch64 `rustc` / `cargo`:
  ```
  arch --x86_64 \
  sh <(curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs) \
  --default-host aarch64-apple-darwin \
  --default-toolchain nightly
  ```

* Clone this repo

* [Build it and install as usual](https://github.com/rust-lang/rustup/blob/master/CONTRIBUTING.md)

I found that with these instructions most crates fail to build. I saw that building with RUSTC=$(rustup which rustc) $(rustup which cargo) install <your_crate> should fix that until native rustup is released, but I haven't tried that yet.

[shepmaster]

Ah, yeah, that might be running into some (as far as I can tell) Apple bugs invoking cc and friends. The environment trick you mention causes the build to jump back into a native arm64 process.

# install current stable x86_64 rustup...

% file $(which rustup)
/Users/shepmaster/.cargo/bin/rustup: Mach-O 64-bit executable x86_64

% git show --pretty=oneline
5674652a45ecf9066128df32fbc51e965586e4b2 (HEAD -> master, origin/master, origin/HEAD) Merge pull request #2534 from frbimo/warn_download

# Make sure you do a `cargo clean` if you got any errors from trying without this

% RUSTC=$(rustup which rustc) $(rustup which cargo) build
    Finished dev [unoptimized + debuginfo] target(s) in 56.34s

% RUSTC=$(rustup which rustc) $(rustup which cargo) dev-install

% file $(which rustup)
/Users/shepmaster/.cargo/bin/rustup: Mach-O 64-bit executable arm64

And, after doing that:

% cargo clean

% cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 1m 46s

Thanks for that specific environment variable trick; I've added it to the intermediate README.

See also:

• Why does my native application compiled on Apple Silicon sometimes build as arm64 and sometimes build as x86_64?
• Why does my native arm64 application built using an x86_64 build system fail to be code signed unless I remove the previous executable?

[kornelski]

The workaround with arch --x86_64 sh did't work for me on M1. I got:

arch: posix_spawn: sh: Bad CPU type in executable

edit: Got it! It won't work until some GUI app on macOS causes it to install Rosetta first.

[TrashPandacoot]

The workaround with arch --x86_64 sh did't work for me on M1. I got:

arch: posix_spawn: sh: Bad CPU type in executable

edit: Got it! It won't work until some GUI app on macOS causes it to install Rosetta first.

For others that run into this, iirc you can also install Rosetta 2 via terminal by running
softwareupdate --install-rosetta

[anlumo]

In case anybody else besides me runs into the issue that compiling with cargo doesn't work properly via Rosetta 2 (SIGKILL at random times, binary doesn't execute), take a look at this solution: https://github.com/mozilla/sccache/issues/880#issuecomment-728323586

[richiksc]

Looks like rustup is about to release v1.23.0 with native aarch64-apple-darwin support really soon, as they've updated the changelog and bumped the version in preparation for release today and merged master into stable. We're just waiting for the final steps, whatever those are, and deployment. The workaround shouldn't be neccessary after the update, right?

[anlumo]

Yes, because this issue is caused by running cargo in x86 emulation, which happens because rustup is running in x86 emulation. For now, this workaround also has the advantage of having faster compile times (although I'm not really feeling a difference here TBH, Rosetta 2 is pretty good).

[shepmaster]

@anlumo is that any different from the solution posted seven days ago in this thread?

[richiksc]

How long does it take between the merge to stable and the release commit incrementing the version number to when new rustup binaries and rustup-init are actually released on https://sh.rustup.rs?

[hayesgm]

When it is published, it should be available here: https://static.rust-lang.org/rustup/dist/aarch64-apple-darwin/rustup-init

[kinnison]

Since there's so much interest here, I thought I'd give an update.

We are probably a day or two away from releasing publically assuming nothing odd goes wrong with our test release.

To test the current test release you could try:

curl --proto '=https' --tlsv1.2 -sSf https://dev-static.rust-lang.org/rustup/rustup-init.sh | env RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup sh -

If any of that doesn't work then please let me know.

[shepmaster]

Looks good to me!

% rustup self uninstall

% curl --proto '=https' --tlsv1.2 -sSf https://dev-static.rust-lang.org/rustup/rustup-init.sh | env RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup sh -

Current installation options:

   default host triple: aarch64-apple-darwin
     default toolchain: beta
               profile: minimal
  modify PATH variable: yes

% file $(which rustup)
~/.cargo/bin/rustup: Mach-O 64-bit executable arm64

% rustup install nightly

  nightly-aarch64-apple-darwin installed - rustc 1.50.0-nightly (1c389ffef 2020-11-24)

% cd /tmp

% cargo new hola

% cd hola

% cargo run
   Compiling hola v0.1.0 (/private/tmp/hola)
    Finished dev [unoptimized + debuginfo] target(s) in 0.39s
     Running `target/debug/hola`
Hello, world!

% file target/debug/hola
target/debug/hola: Mach-O 64-bit executable arm64

[gabereiser]

$ curl --proto '=https' --tlsv1.2 -sSf https://dev-static.rust-lang.org/rustup/rustup-init.sh | env RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup sh -
info: downloading installer

Welcome to Rust!

This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.

Rustup metadata and toolchains will be installed into the Rustup
home directory, located at:

  /Users/me/.rustup

This can be modified with the RUSTUP_HOME environment variable.

The Cargo home directory located at:

  /Users/me/.cargo

This can be modified with the CARGO_HOME environment variable.

The cargo, rustc, rustup and other commands will be added to
Cargo's bin directory, located at:

  /Users/me/.cargo/bin

This path will then be added to your PATH environment variable by
modifying the profile files located at:

  /Users/me/.profile
  /Users/me/.zshenv

You can uninstall at any time with rustup self uninstall and
these changes will be reverted.

Current installation options:


   default host triple: aarch64-apple-darwin
     default toolchain: stable (default)
               profile: default
  modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>1

info: profile set to 'default'
info: default host triple is aarch64-apple-darwin
info: syncing channel updates for 'stable-aarch64-apple-darwin'
info: latest update on 2020-11-19, rust version 1.48.0 (7eac88abb 2020-11-16)
error: target 'aarch64-apple-darwin' not found in channel.  Perhaps check https://forge.rust-lang.org/release/platform-support.html for available targets

@kinnison running your supplied curl command string results in this error when attempting an install on a vanilla MBA 13 M1.

[glandium]

You want to change the default toolchain to either beta or nightly.
A CLI way to do that is to use:

RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup sh <(curl --proto '=https' --tlsv1.2 -sSf https://dev-static.rust-lang.org/rustup/rustup-init.sh) --default-toolchain beta

[kinnison]

This is handy to know @glandium thanks. When I write the blog post I'll try and remember to mention that.

[kinnison]

Rustup 1.23.0 is now out.

[shepmaster]

Thank you!

[kinnison]

@shepmaster you did most of the work :D

[gabereiser]

@kinnison @shepmaster you guys are awesome!

[jeroen]

I'm still seeing this:

info: downloading installer
info: profile set to 'default'
info: default host triple is aarch64-apple-darwin
info: syncing channel updates for 'stable-aarch64-apple-darwin'
info: latest update on 2020-11-19, rust version 1.48.0 (7eac88abb 2020-11-16)
error: target 'aarch64-apple-darwin' not found in channel. 

Do I still need to enable the beta channel?

[glandium]

Do I still need to enable the beta channel?

Yes, until 1.49.0 is released, which, if I'm not mistaken, should happen in 2 weeks (although considering that would be during the holidays, I don't know if the usual 6 weeks schedule is going to stand).

[jeroen]

OK so I need to pass the --default-toolchain beta flag. Do I also still need the aforementioned RUSTUP_UPDATE_ROOT or is this no longer needed with the latest rustup?

[glandium]

Latest rustup is native. You can use sh <(curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs) --default-toolchain beta, now.