Rust: Use lld by default on x64 msvc windows

[Gankra]

This is a metabug, constraining the unbound scope of #39915.

What is lld

A linker that's part of the llvm project, which is desirable for two reasons:

• it's very friendly to cross compilation (hence its emphasis for embedded targets)
• it's very fast (often runs in half the time as Gold -- linking can take several minutes for big projects (rustc, servo, etc.) and linking can be a huge % of the compile with incremental builds, so halving this runtime is a Big Deal.)

Rust currently ships its own copy of lld which it calls rust-lld. This is used by default to link bare-metal targets and wasm.

Goal

The goal of this metabug is to use rust-lld by default when targeting x64 msvc windows (as opposed to e.g. mingw windows). It's possible this will incidentally get other windows targets working, but I'm constraining scope for the sake of focusing efforts.

This configuration is already in a decent state, with backend support already implemented. It's just a matter of making that configuration stable enough to use by default.

To test using rust-lld for msvc targets, use -C linker=lld

Blocking Issues

lld on windows status page

• [x] #68647 - linking libtest results in undefined symbols
• [ ] #71504 - rust-analyser segfault with lto=thin, linker=lld-link
• [ ] #72145 - LLD/COFF does not correctly align TLS section

[petrochenkov]

(TODO: how is it accessed? -C link-flavor?)

-C linker=lld (but -C linker-flavor=lld should also work).

[luser]

This ought to be a bit more straightforward than using lld with gcc since with MSVC you do conventionally invoke the linker directly, and lld has an lld-link.exe binary that tries to be a drop-in replacement for link.exe. You can also run lld -flavor link and pass MSVC-style commandline arguments after that. I would expect that if you set up the environment correctly you ought to be able to simply invoke lld-link.exe instead of link.exe and have it Just Work.

Note that you still need a Visual C++ install (they have a smaller "Build Tools for Visual Studio" install that doesn't include the IDE) or maybe just a Windows SDK in order to have the system .lib files it wants to link to.

[Gankra]

Yes, as noted in the main comment (which I update to track current status) the infra is already hooked up here for invoking it. It's just a bit buggy, and requires a more thorough investigation of failure-modes and how we can smooth them over, e.g.:

• if you don't have Visual C++ installed what happens?
• do we provide lld-link.exe, or do we look for a system one? both? is
• does it work just as well on win 8 and 7? should we restrict efforts to win 10?
• can we directly invoke the lld-link binary on macos/linux when cross-compiling to windows?
• how important are the missing features on the lld-link status page? can we recover and fallback to link when encountering them?

Once someone's done a quick audit of basic failure modes, we should do some community outreach asking people to test -C linker=lld and report how it affects their build times (clean build, big change incremental, tiny change incremental, etc.).

Definitely looking for someone to champion this effort, since my primary goal was just creating more focused initiatives / clarifying the status confusion in the main lld bug.

[luser]

Ah, apologies, I did read that comment but must have missed that.

if you don't have Visual C++ installed what happens?

This is a good question but I think the better question is "if you try to use lld-link but don't have Visual C++ installed is it any worse than what happens in the default case without Visual C++ installed?" Currently rustc does try to locate MSVC but will apparently just try its best if it can't find it:
https://github.com/rust-lang/rust/blob/f8d394e5184fe3af761ea1e5ba73f993cfb36dfe/src/librustc_codegen_ssa/back/link.rs#L164

[mati865]

if you don't have Visual C++ installed what happens?

IIRC lld-link errors listing all Windows SDK libraries it cannot find.

can we directly invoke the lld-link binary on macos/linux when cross-compiling to windows?

You cannot cross-compile to MSVC from other OSes unless you find a way to extract Windows SDK and feed it to the LLD. AFAIK MinGW with ld.lld is the only somewhat supported way to cross compile for Windows.

[luser]

Cross-compiling to Windows is not much different than any other OS these days now that LLVM has good support for generating Windows binaries. You do need all the import libs from MSVC at the very least, and probably a Windows SDK for anything non-trivial, but you can absolutely do it:
https://github.com/ProdriveTechnologies/clang-msvc-sdk

Given that requirement it's probably not worth really worrying about as long as it doesn't error in a way that's any more confusing than trying to cross-compile to Windows without specifying an alternate linker. That fails like this for me on Linux with Rust 1.42.0:

luser@eye7:/build/snippet$ cargo build --target=x86_64-pc-windows-msvc
   Compiling itoa v0.3.4
   Compiling snippet v0.1.4-alpha.0 (/build/snippet)
error: linker `link.exe` not found
  |
  = note: No such file or directory (os error 2)

note: the msvc targets depend on the msvc linker but `link.exe` was not found

note: please ensure that VS 2013, VS 2015, VS 2017 or VS 2019 was installed with the Visual C++ option

error: aborting due to previous error

error: could not compile `snippet`.

To learn more, run the command again with --verbose.

Once #58713 is fixed it is probably worth revisiting that topic since it'll become feasible to write pure-Rust descriptions of Windows import libraries and generate working binaries without having MSVC or a Windows SDK.

[luser]

do we provide lld-link.exe, or do we look for a system one? both?

If Rust is already shipping an lld binary it makes sense to allow using it, but given the myriad ways people build software there will assuredly be a desire to use specific lld/lld-link binaries. Given that -C linker already exists and works and there's a corresponding cargo config setting to set it, and that the -C linker-flavor docs say that passing a path to -C linker will automatically detect the correct flavor it doesn't sound like there's anything else that would need to be done to make it work for those scenarios.

does it work just as well on win 8 and 7? should we restrict efforts to win 10?

This definitely bears some investigation but I suspect it's not a terribly big deal in practice. The LLVM docs about building clang for use in Visual Studio say "Any system that can adequately run Visual Studio 2017 is fine". The Visual Studio 2017 System Requirements documentation says it supports Windows 7 SP1 or newer.

how important are the missing features on the lld-link status page? can we recover and fallback to link when encountering them?

Given that both Firefox and Chrome use lld-link for linking their Windows builds nowadays I'd hazard a guess that any missing features are not incredibly important. If someone were to propose making lld-link the default linker I would worry more about that, but anyone opting-in to using it and also relying on niche linker features can presumably just...not use it?

[matklad]

The issue says "To test using rust-lld for msvc targets, use -C linker=lld". Shouldn't it be -C linker=rust-lld? At least, that's what i need to put in .cargo/config to make this work (my experiments):

[target.x86_64-pc-windows-msvc]
linker = "rust-lld"

[petrochenkov]

~Potential blocker - https://github.com/rust-lang/rust/issues/76398 "Rust + MSVC + LLD = Segfault on attempt to access TLS".
This issue currently prevents bootstrap of rustc on x86_64-pc-windows-msvc using LLD as a linker.~
Already mentioned under the "LLD/COFF does not correctly align TLS section" item.