Rust: Tools are not available on cross compiled platforms since #61861

Created on 10 Jul 2019  路  52Comments  路  Source: rust-lang/rust

Moved from https://github.com/rust-lang/rust-clippy/issues/4267

61861 was merged in #61962, since then Rustfmt fails to build when cross compiling which is the case for almost all non Tier 1 platforms.

Clippy and RLS are failing with similar error.

Error on aarch64-unknown-linux-gnu:

[01:14:03] error[E0464]: multiple matching crates for `rustc_macros`
[01:14:03]   --> /cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_target-491.0.0/abi/mod.rs:10:5
[01:14:03]    |
[01:14:03]    | use syntax_pos::symbol::{sym, Symbol};
[01:14:03]    |     ^^^^^^^^^^
[01:14:03]    |
[01:14:03]    = note: candidates:
[01:14:03]            crate `rustc_macros`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_macros-97bf1e9c420f9886.so
[01:14:03]            crate `rustc_macros`: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/librustc_macros-2f4546e1163449fb.so
[01:14:03] 
[01:14:03] error[E0461]: couldn't find crate `rustc_macros` with expected target triple x86_64-unknown-linux-gnu which `syntax_pos` depends on
[01:14:03]   --> /cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_target-491.0.0/abi/mod.rs:10:5
[01:14:03]    |
[01:14:03] 10 | use syntax_pos::symbol::{sym, Symbol};
[01:14:03]    |     ^^^^^^^^^^
[01:14:03]    |
[01:14:03]    = note: the following crate versions were found:
[01:14:03]            crate `rustc_macros`, target triple aarch64-unknown-linux-gnu: /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/aarch64-unknown-linux-gnu/release/deps/librustc_macros-55389246a2ce204b.so
[01:14:03] 
[01:14:03] error: aborting due to 2 previous errors

https://api.travis-ci.com/v3/job/209363492/log.txt

@rustbot modify labels: T-dev-tools A-rustbuild

A-cross A-rustbuild C-bug O-ARM O-freebsd T-dev-tools T-infra regression-from-stable-to-stable

Most helpful comment

All 52 comments

@rustbot modify labels: T-dev-tools A-rustbuild

Why does Clippy depend on rustfmt? We only have it in our Travis tests, and even then a test-fail shouldn't cause an unshipped tool

Modified issue description, it depends on fmt_macros but the error is basically the same.

only way i got it to build cross tools was from example in original post copying librustc_macros-97bf1e9c420f9886.so over librustc_macros-2f4546e1163449fb.so
this and suddenly rust nightly std library doubling in size... fun times.

ping. Any progress here? This issue blocks tier2 users from upgrading past rustc nightly 1.36.0.

I can't tell where the logs for Clippy on cross platforms are so I don't know how to fix this. Rust has a ton of builders :/

Hmm that seems like the rust build system being brittle. I've seen that kind of error before, it's usually solved by deleting something in the build dir.

cc @alexcrichton @pietroalbini

That isn't an issue about stale build directories or anything like that. Something is misconfigured to cause the rustc-macros crate to get built twice. The rustc_macros crate should not be getting compiled into the stage1-tools directory, but something is pulling it in. I haven't investigated to see what, though.

(it'd be good as well btw to not assume default-flakiness and instead assume that because it is a consistent regression that there's something which needs to be investigated)

I think rustc_macros are expected to be available in stage1: https://github.com/rust-lang/rust/pull/58013/files#diff-4bc384d35e57dc577b9378d4d733d7c3R818
The issue started when rustfmt was updated and pulled new dependecy: rustc-ap-rustc_macros in https://github.com/rust-lang/rust/pull/61861

Please restore cargo install clippy, which seems to have gone away, until you have the rustup issue fixed, and similarly for the other affected tools.

Is it possible to get this issue resolved please quickly ? Some of us are using "tier 2" platforms and we would like to have our development tools back. As I said earlier, please at least restore the earlier installation path using cargo until the current issue is resolved.

The cargo install path is not something that can be re-added, we're long past that point. Clippy no longer works that way.

Any idea of how long it will take to fix this problem?

@martinellison why don't you go ahead and fix it yourself?
There is no ETA and spamming this thread is not helpful.

@mati865 sorry, I am not familiar with the code base for this. I was hoping that someone more up to speed than me could help. Sorry if I have offended anyone.

Where should someone start looking to try to fix this, x.py?

Doubled.

After disabling dual-proc-macros (building proc-macro crates for both host and target) I was able to get Rustfmt and RLS to cross compile for Aarch64 but Clippy broke even more (rustfmt_macros is not available when cross compiling without dual-proc-macros).
But it's just poor man workaround, not the solution because it worked fine that way before rustc-ap-rustc_macros crate was pulled into dependency chain.

Maybe rustc-ap-rustc_macros doesn't play nice with dual-proc-macros?

cc @Zoxc

See also https://github.com/rust-lang/rust-clippy/issues/4015#issuecomment-485396021.

This is the same breakage - when cross-compiling, we have a proc macro crate mismatch when it's used both by the compiler and by the included Rust distribution tool itself.

The fix used there was to explicitly disambiguate the multiple matching proc macros by passing --extern <crate>=<path_we_want_to_use> but I think it was rather a band-aid fix and we should find something more principled here.

If we could solve this at a build level, I believe this would unlock using crates.io proc macros in rustc as well as in the tools (one example might be using serde_derive in crates used by RLS or rustdoc itself)

Or we can properly rename the crates in the publish script for rustc-ap_* crates.

@msizanoen I've tried it and it won't work (or at leas not easily).
Those crates specify exact library name which can be easily changed in automated way but automatically finding and changing code which loads them is a bigger task.

I'd like to add that it's a hack but it won't solve the case of proc macros shared by the compiler and the tools (e.g. serde_derive) and we'd like to see this use case supported as well.

Also clippy is failing with another error:

Building stage1 tool clippy-driver (armv7-unknown-linux-gnueabihf)
   Compiling cfg-if v0.1.8
   Compiling matches v0.1.8
   Compiling rustc-demangle v0.1.16
   Compiling unicode-normalization v0.1.7
   Compiling unicode-width v0.1.6
   Compiling pulldown-cmark v0.5.3
   Compiling itoa v0.4.4
   Compiling percent-encoding v2.0.0
   Compiling lazy_static v1.3.0
   Compiling ucd-util v0.1.3
   Compiling rustc_tools_util v0.2.0 (/builds/nsidious/rust-ci/rust/src/tools/clippy/rustc_tools_util)
   Compiling quine-mc_cluskey v0.2.4
   Compiling smallvec v0.6.10
   Compiling utf8-ranges v1.0.2
   Compiling if_chain v1.0.0
   Compiling libc v0.2.62
   Compiling ryu v1.0.0
   Compiling unicode-bidi v0.3.4
   Compiling memchr v2.2.0
   Compiling bitflags v1.1.0
   Compiling getopts v0.2.21
   Compiling thread_local v0.3.6
   Compiling regex-syntax v0.6.6
   Compiling regex v1.1.6
   Compiling clippy v0.0.212 (/builds/nsidious/rust-ci/rust/src/tools/clippy)
   Compiling backtrace-sys v0.1.30
   Compiling unicase v2.4.0
   Compiling idna v0.2.0
   Compiling aho-corasick v0.7.3
   Compiling backtrace v0.3.37
   Compiling serde_json v1.0.40
   Compiling toml v0.5.3
   Compiling url v2.1.0
   Compiling failure v0.1.5
   Compiling cargo_metadata v0.8.0
   Compiling clippy_lints v0.0.212 (/builds/nsidious/rust-ci/rust/src/tools/clippy/clippy_lints)
error[E0463]: can't find crate for `fmt_macros`
  --> src/tools/clippy/clippy_lints/src/lib.rs:20:1
   |
20 | extern crate fmt_macros;
   | ^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

For more information about this error, try `rustc --explain E0463`.
error: Could not compile `clippy_lints`.

To learn more, run the command again with --verbose.

@mati865 We can use Cargo external crate renaming

This is the Cargo.toml file for rustc-ap_syntax_pos: https://docs.rs/crate/rustc-ap-syntax_pos/585.0.0/source/Cargo.toml

Instead of renaming, we will use cargo's external crate renaming like this: https://docs.rs/crate/msiz_rustc-ap-syntax_pos/1.0.0/source/Cargo.toml so that it automatically passes the right --extern <name>=... flags.
Compare it with the original above (which is missing the package=... property)

This is the changes done to rustc-auto-publish:

diff --git a/src/main.rs b/src/main.rs
index 40425ba..0ceabdc 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -303,6 +303,13 @@ fn publish(pkg: &Package, commit: &str, vers: &semver::Version) {
                         }
                     }
                     new_table.insert("version".to_string(), toml::Value::String(vers.to_string()));
+                    if !has_package {
+                        new_table.insert(
+                            "package".to_string(),
+                            toml::Value::String(format!("{}-{}", PREFIX, name)),
+                        );
+                        has_package = true;
+                    }
                     let key_name = if has_package {
                         name.clone()
                     } else {

That PR did not fix the issue. Also clippy is failing with another error (see https://github.com/rust-lang/rust/issues/62558#issuecomment-529884322)

The issue won't be fixed till the next nightly.

Keeping it open until then.

There is a hack awaiting to be merged in alexcrichton/rustc-auto-publish#12. This make sure rustc-ap-rustc_macros is compiled as librustc_ap_rustc_macros.so and not librustc_macros.so to prevent confusing the compiler.

@Manishearth @pietroalbini This issue is still not fixed yet. Even after alexcrichton/rustc-auto-publish#12 is merged then the rustc-ap crates still need to be re-published and somebody will need to send a few PRs to tool repositories to update the autopublish crate versions.

Even after that somebody will need to update the tool submodules in the main repo.

I guess it will take a while.

@msizanoen I've merged your PR and published a new suite of crates on crates.io as v587

Looks like we have another error:

Dist Rustfmt stage1 (armv7-unknown-linux-gnueabihf)
Building stage1 tool rustfmt (armv7-unknown-linux-gnueabihf)
   Compiling syn v0.15.35
   Compiling stable_deref_trait v1.1.0
   Compiling proc-macro2 v1.0.3
   Compiling unicode-xid v0.2.0
   Compiling indexmap v1.0.2
   Compiling syn v1.0.5
   Compiling rustc-ap-graphviz v583.0.0
   Compiling either v1.5.0
   Compiling scoped-tls v1.0.0
   Compiling unicode-segmentation v1.2.1
   Compiling rustc-ap-rustc_target v583.0.0
   Compiling rustc-ap-syntax v583.0.0
   Compiling rustfmt-nightly v1.4.8 (/builds/nsidious/rust-ci/rust/src/tools/rustfmt)
   Compiling unicode_categories v0.1.1
   Compiling diff v0.1.11
   Compiling bytecount v0.6.0
   Compiling owning_ref v0.3.3
   Compiling rustc-ap-rustc_lexer v583.0.0
   Compiling rustc-ap-serialize v583.0.0
   Compiling itertools v0.8.0
   Compiling rustc-hash v1.0.1
   Compiling heck v0.3.0
   Compiling getopts v0.2.21
   Compiling annotate-snippets v0.6.1
   Compiling lock_api v0.1.3
   Compiling dirs-sys v0.3.3
   Compiling term_size v0.3.1
   Compiling ena v0.13.0
   Compiling quote v1.0.2
   Compiling dirs v2.0.1
   Compiling synstructure v0.10.2
   Compiling serde_derive v1.0.81
   Compiling derive-new v0.5.6
   Compiling term v0.6.0
   Compiling proc-macro-error v0.2.6
   Compiling rustfmt-config_proc_macro v0.2.0 (/builds/nsidious/rust-ci/rust/src/tools/rustfmt/config_proc_macro)
   Compiling rustc-ap-rustc_macros v583.0.0
   Compiling failure_derive v0.1.5
   Compiling serde v1.0.99
   Compiling structopt-derive v0.3.1
   Compiling semver v0.9.0
   Compiling failure v0.1.5
   Compiling serde_json v1.0.40
error[E0464]: multiple matching crates for `serde_derive`
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.40/src/lib.rs:324:1
    |
324 | extern crate serde;
    | ^^^^^^^^^^^^^^^^^^^
    |
    = note: candidates:
            crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-c0d30d94e16a3450.so
            crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-189c9323f91276cb.so

error[E0461]: couldn't find crate `serde_derive` with expected target triple x86_64-unknown-linux-gnu which `serde` depends on
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.40/src/lib.rs:324:1
    |
324 | extern crate serde;
    | ^^^^^^^^^^^^^^^^^^^
    |
    = note: the following crate versions were found:
            crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-df48c4ec576951c7.so
            crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-d6fc64ba761a662b.so

error: aborting due to 2 previous errors

error: Could not compile `serde_json`.

To learn more, run the command again with --verbose.
command did not execute successfully: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "armv7-unknown-linux-gnueabihf" "-Zdual-proc-macros" "-Zbinary-dep-depinfo" "-j" "1" "--release" "--manifest-path" "/builds/nsidious/rust-ci/rust/src/tools/rustfmt/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
Unable to build Rustfmt, skipping dist
Dist RLS stage1 (armv7-unknown-linux-gnueabihf)
Building stage1 tool clippy-driver (armv7-unknown-linux-gnueabihf)
   Compiling cfg-if v0.1.8
   Compiling matches v0.1.8
   Compiling rustc-demangle v0.1.16
   Compiling unicode-normalization v0.1.7
   Compiling unicode-width v0.1.6
   Compiling itoa v0.4.4
   Compiling pulldown-cmark v0.5.3
   Compiling ucd-util v0.1.3
   Compiling lazy_static v1.3.0
   Compiling percent-encoding v2.0.0
   Compiling rustc_tools_util v0.2.0 (/builds/nsidious/rust-ci/rust/src/tools/clippy/rustc_tools_util)
   Compiling if_chain v1.0.0
   Compiling quine-mc_cluskey v0.2.4
   Compiling smallvec v0.6.10
   Compiling utf8-ranges v1.0.2
   Compiling libc v0.2.62
   Compiling ryu v1.0.0
   Compiling unicode-bidi v0.3.4
   Compiling bitflags v1.1.0
   Compiling memchr v2.2.0
   Compiling getopts v0.2.21
   Compiling regex-syntax v0.6.6
   Compiling thread_local v0.3.6
   Compiling regex v1.1.6
   Compiling clippy v0.0.212 (/builds/nsidious/rust-ci/rust/src/tools/clippy)
   Compiling backtrace-sys v0.1.30
   Compiling unicase v2.4.0
   Compiling idna v0.2.0
   Compiling aho-corasick v0.7.3
   Compiling backtrace v0.3.37
   Compiling serde_json v1.0.40
   Compiling url v2.1.0
   Compiling toml v0.5.3
   Compiling failure v0.1.5
   Compiling cargo_metadata v0.8.0
error[E0464]: multiple matching crates for `failure_derive`
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/cargo_metadata-0.8.0/src/lib.rs:160:1
    |
160 | extern crate failure;
    | ^^^^^^^^^^^^^^^^^^^^^
    |
    = note: candidates:
            crate `failure_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libfailure_derive-833d57e0e5a26a75.so
            crate `failure_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libfailure_derive-12fc2b124f30c9c1.so

error[E0461]: couldn't find crate `failure_derive` with expected target triple x86_64-unknown-linux-gnu which `failure` depends on
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/cargo_metadata-0.8.0/src/lib.rs:160:1
    |
160 | extern crate failure;
    | ^^^^^^^^^^^^^^^^^^^^^
    |
    = note: the following crate versions were found:
            crate `failure_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libfailure_derive-98b23980bbca09c2.so
            crate `failure_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libfailure_derive-ce2627bb158b3922.so

error: aborting due to 2 previous errors

error: Could not compile `cargo_metadata`.

To learn more, run the command again with --verbose.
command did not execute successfully: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "armv7-unknown-linux-gnueabihf" "-Zdual-proc-macros" "-Zbinary-dep-depinfo" "-j" "1" "--release" "--manifest-path" "/builds/nsidious/rust-ci/rust/src/tools/clippy/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
Building stage1 tool rls (armv7-unknown-linux-gnueabihf)
   Compiling scopeguard v1.0.0
   Compiling rayon-core v1.5.0
   Compiling rustc_tools_util v0.2.0
   Compiling json v0.11.13
   Compiling futures v0.1.28
   Compiling ordslice v0.3.0
   Compiling rustc-serialize v0.3.24
   Compiling itertools v0.7.8
   Compiling fst v0.3.0
   Compiling rls v1.39.0 (/builds/nsidious/rust-ci/rust/src/tools/rls)
   Compiling crossbeam-queue v0.1.2
   Compiling derive_more v0.13.0
   Compiling serde_repr v0.1.5
   Compiling url v2.1.0
error[E0464]: multiple matching crates for `serde_derive`
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/url-2.1.0/src/lib.rs:115:1
    |
115 | extern crate serde;
    | ^^^^^^^^^^^^^^^^^^^
    |
    = note: candidates:
            crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-c0d30d94e16a3450.so
            crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-189c9323f91276cb.so

error[E0461]: couldn't find crate `serde_derive` with expected target triple x86_64-unknown-linux-gnu which `serde` depends on
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/url-2.1.0/src/lib.rs:115:1
    |
115 | extern crate serde;
    | ^^^^^^^^^^^^^^^^^^^
    |
    = note: the following crate versions were found:
            crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-df48c4ec576951c7.so
            crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-d6fc64ba761a662b.so

error: aborting due to 2 previous errors

error: Could not compile `url`.

To learn more, run the command again with --verbose.
command did not execute successfully: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "armv7-unknown-linux-gnueabihf" "-Zdual-proc-macros" "-Zbinary-dep-depinfo" "-j" "1" "--release" "--manifest-path" "/builds/nsidious/rust-ci/rust/src/tools/rls/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
Unable to build RLS, skipping dist
Skipping LlvmTools (armv7-unknown-linux-gnueabihf): external LLVM
Dist clippy stage1 (armv7-unknown-linux-gnueabihf)
Unable to build clippy, skipping dist
Dist miri stage1 (armv7-unknown-linux-gnueabihf)
Building stage1 tool miri (armv7-unknown-linux-gnueabihf)
   Compiling num-traits v0.2.6
   Compiling num-integer v0.1.39
   Compiling getrandom v0.1.12
   Compiling time v0.1.40
   Compiling chrono v0.4.6
   Compiling directories v2.0.1
   Compiling vergen v3.0.4
   Compiling rustc_version v0.2.3
error[E0464]: multiple matching crates for `serde_derive`
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc_version-0.2.3/src/lib.rs:55:1
   |
55 | extern crate semver;
   | ^^^^^^^^^^^^^^^^^^^^
   |
   = note: candidates:
           crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-c0d30d94e16a3450.so
           crate `serde_derive`: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde_derive-189c9323f91276cb.so

error[E0461]: couldn't find crate `serde_derive` with expected target triple x86_64-unknown-linux-gnu which `semver` depends on
  --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc_version-0.2.3/src/lib.rs:55:1
   |
55 | extern crate semver;
   | ^^^^^^^^^^^^^^^^^^^^
   |
   = note: the following crate versions were found:
           crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-d6fc64ba761a662b.so
           crate `serde_derive`, target triple armv7-unknown-linux-gnueabihf: /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/armv7-unknown-linux-gnueabihf/release/deps/libserde_derive-df48c4ec576951c7.so

error: aborting due to 2 previous errors

error: Could not compile `rustc_version`.

To learn more, run the command again with --verbose.
command did not execute successfully: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "armv7-unknown-linux-gnueabihf" "-Zdual-proc-macros" "-Zbinary-dep-depinfo" "-j" "1" "--release" "--manifest-path" "/builds/nsidious/rust-ci/rust/src/tools/miri/Cargo.toml" "--message-format" "json-render-diagnostics"
expected success, got: exit code: 101
Unable to build miri, skipping dist
Dist std stage2 (armv7-unknown-linux-gnueabihf -> armv7-unknown-linux-gnueabihf)
    skipping, not a build host
Build completed successfully in 2:45:45

Setting verbose=2 exposes the cause that causes clippy to fail with fmt_macros not found:

Compiling clippy_lints v0.0.212 (/builds/nsidious/rust-ci/rust/src/tools/clippy/clippy_lints)
     Running `/builds/nsidious/rust-ci/rust/build/bootstrap/debug/rustc --edition=2018 --crate-name clippy_lints src/tools/clippy/clippy_lints/src/lib.rs --error-format json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C opt-level=2 -C metadata=05f6b439c75db891 -C extra-filename=-05f6b439c75db891 --out-dir /builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps -L dependency=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps --extern cargo_metadata=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libcargo_metadata-b6383692b0bb6679.rmeta --extern if_chain=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libif_chain-9c23c72dca32979f.rmeta --extern itertools=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libitertools-1832f23158689783.rmeta --extern lazy_static=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/liblazy_static-05c684a841ed472a.rmeta --extern matches=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libmatches-61cf540a1ac2deb4.rmeta --extern pulldown_cmark=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libpulldown_cmark-f827a890558b8504.rmeta --extern quine_mc_cluskey=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libquine_mc_cluskey-221cb3213d8097ad.rmeta --extern regex_syntax=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libregex_syntax-cad403b97af4f3fb.rmeta --extern semver=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libsemver-d455adfc963974f9.rmeta --extern serde=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde-cc4d472167daa67f.rmeta --extern smallvec=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libsmallvec-f7334b787bd8ab0e.rmeta --extern toml=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libtoml-a92ea37771dd40ac.rmeta --extern unicode_normalization=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libunicode_normalization-663c92f314907150.rmeta --extern url=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/liburl-94a5a1f1fa5ceb72.rmeta -Zbinary-dep-depinfo -L native=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/build/backtrace-sys-b9bdfe1a29c5a201/out`
rustc command: "LD_LIBRARY_PATH"="/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1/lib:/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps:/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1/lib" "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "--edition=2018" "--crate-name" "clippy_lints" "src/tools/clippy/clippy_lints/src/lib.rs" "--error-format" "json" "--json=diagnostic-rendered-ansi" "--crate-type" "lib" "--emit=dep-info,metadata,link" "-C" "opt-level=2" "-C" "metadata=05f6b439c75db891" "-C" "extra-filename=-05f6b439c75db891" "--out-dir" "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps" "-L" "dependency=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps" "--extern" "cargo_metadata=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libcargo_metadata-b6383692b0bb6679.rmeta" "--extern" "if_chain=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libif_chain-9c23c72dca32979f.rmeta" "--extern" "itertools=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libitertools-1832f23158689783.rmeta" "--extern" "lazy_static=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/liblazy_static-05c684a841ed472a.rmeta" "--extern" "matches=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libmatches-61cf540a1ac2deb4.rmeta" "--extern" "pulldown_cmark=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libpulldown_cmark-f827a890558b8504.rmeta" "--extern" "quine_mc_cluskey=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libquine_mc_cluskey-221cb3213d8097ad.rmeta" "--extern" "regex_syntax=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libregex_syntax-cad403b97af4f3fb.rmeta" "--extern" "semver=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libsemver-d455adfc963974f9.rmeta" "--extern" "serde=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libserde-cc4d472167daa67f.rmeta" "--extern" "smallvec=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libsmallvec-f7334b787bd8ab0e.rmeta" "--extern" "toml=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libtoml-a92ea37771dd40ac.rmeta" "--extern" "unicode_normalization=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/libunicode_normalization-663c92f314907150.rmeta" "--extern" "url=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/deps/liburl-94a5a1f1fa5ceb72.rmeta" "-Zbinary-dep-depinfo" "-L" "native=/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1-tools/release/build/backtrace-sys-b9bdfe1a29c5a201/out" "-Cdebuginfo=0" "--remap-path-prefix" "/builds/nsidious/rust-ci/rust=/rustc/a44881d892fb4f4a8ed93f8f392bab942fac7a41"
sysroot: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1"
libdir: "/builds/nsidious/rust-ci/rust/build/x86_64-unknown-linux-gnu/stage1/lib"
error[E0463]: can't find crate for `fmt_macros`
  --> src/tools/clippy/clippy_lints/src/lib.rs:20:1

The rustc command above does not have --target flag so it seems like Cargo built it for the host. This expect host stage2 rustc (instead of target stage2) to be available which is not the case in cross compilation.
Removing plugin = true line in clippy's Cargo.toml (which causes it to be compiled for host) is an effective workaround.

(Confirming this is still broken on at least FreeBSD.)

Confirming this is still broken on at least NetBSD with rust 1.38.0, even after applying the patch given in #64862 (build errors listed as a comment there).

This bug will be affecting many more users now that Rustup 1.20.0 installs rustfmt and clippy by default.
https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

Effectively all users were already affected as of latest stable.

-Z dual-proc-macros has the limitation that only one proc macro with the same name can be used. Making sure there's only a single version of each proc macro crate in use should fix this error.

To lift this restriction we must record the crate disambiguator of both the target and host proc macro crates when linking to it. We currently only record it for the target, so rustc can't find the right one for the host if there's multiple crates with the same name.

This bug will be affecting many more users now that Rustup 1.20.0 installs rustfmt and clippy by default.
https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html

For those looking for a workaround, @mati865 fixed it for libc CI which started failing due to this today by using --profile=minimal when installing rustup in https://github.com/rust-lang/libc/pull/1550 .

65698 has been merged, now there is only rust-lang/rust-clippy#4714 and a follow-up submodule update PR (https://github.com/rust-lang/rust/pull/66207) and we can close this.

@rustbot modify labels: +O-ARM

https://github.com/rust-lang/rust-clippy/pull/4714 is not very active so I am going with this instead: #66052

@msizanoen1 please don't remove the clippy plugin yet. We're waiting to publish https://github.com/rust-lang/blog.rust-lang.org/pull/435 before we remove it.

Please check the PR description carefully - it does not remove the clippy plugin but avoid building it in rustbuild.

Oh, for some reason I thought we shipped the clippy plugin package as well, but plugin users don't use rustup.

Should be fixed by #66207

Was this page helpful?
0 / 5 - 0 ratings