Rust: tarball for rustc 1.27.0 is not able to rebuildable itself

[semarie]

while preparing packaging of upcoming rustc 1.27.0 (stable) for OpenBSD, I found it is not able to fully rebuild itself.

exactly, rustc-1.27.0 fails to build rust-installer tool, due to error-chain crate, due to unused_doc_comment has been renamed to unused_doc_comments lint error.

strictly speaking, rustc 1.27.0 is able to bootstrap itself. but the distributed tarball doesn't permit such bootstrapping. it is a niche problem that could affect downstream while packaging.

   Compiling installer v0.0.0 (file:///home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/src/tools/rust-installer)
     Running `/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/bootstrap/debug/rustc --crate-name installer tools/rust-installer/src/lib.rs --error-format json --crate-type lib --emit=dep-info,link -C opt-level=3 -C metadata=cb0edbb133b08d30 -C extra-filename=-cb0edbb133b08d30 --out-dir /data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps --target x86_64-unknown-openbsd -L dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps -L dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/release/deps --extern tar=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libtar-a79879d0aaa5d01f.rlib --extern clap=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libclap-1f5fff6cd390ab26.rlib --extern xz2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libxz2-f0ab174e0a4d5fa8.rlib --extern error_chain=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/liberror_chain-e8011be67578c54d.rlib --extern rayon=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/librayon-025f5b0b00b7001b.rlib --extern walkdir=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libwalkdir-296d738a531e22af.rlib --extern flate2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libflate2-366741584a48b864.rlib -L native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/backtrace-sys-0392c0625ee93f39/out/.libs -L native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/miniz-sys-17c754f01d2fd7d7/out -L /data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/lzma-sys-ad13c785dc5afda7/out/lib`
rustc command: "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-bootstrap-amd64-1.27.0-20180619/bin/rustc" "--crate-name" "installer" "tools/rust-installer/src/lib.rs" "--crate-type" "lib" "--emit=dep-info,link" "-C" "opt-level=3" "-C" "metadata=cb0edbb133b08d30" "-C" "extra-filename=-cb0edbb133b08d30" "--out-dir" "/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps" "--target" "x86_64-unknown-openbsd" "-L" "dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps" "-L" "dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/release/deps" "--extern" "tar=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libtar-a79879d0aaa5d01f.rlib" "--extern" "clap=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libclap-1f5fff6cd390ab26.rlib" "--extern" "xz2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libxz2-f0ab174e0a4d5fa8.rlib" "--extern" "error_chain=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/liberror_chain-e8011be67578c54d.rlib" "--extern" "rayon=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/librayon-025f5b0b00b7001b.rlib" "--extern" "walkdir=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libwalkdir-296d738a531e22af.rlib" "--extern" "flate2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libflate2-366741584a48b864.rlib" "-L" "native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/backtrace-sys-0392c0625ee93f39/out/.libs" "-L" "native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/miniz-sys-17c754f01d2fd7d7/out" "-L" "/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/lzma-sys-ad13c785dc5afda7/out/lib" "--cfg" "stage1" "--sysroot" "/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-sysroot" "-C" "debug-assertions=n" "--color=always" "-Dwarnings"
sysroot: "/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-sysroot"
libdir: "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-bootstrap-amd64-1.27.0-20180619/lib"
error: lint unused_doc_comment has been renamed to unused_doc_comments
  --> tools/rust-installer/src/lib.rs:26:5
   |
26 | /     error_chain!{
27 | |         foreign_links {
28 | |             Io(::std::io::Error);
29 | |             StripPrefix(::std::path::StripPrefixError);
30 | |             WalkDir(::walkdir::Error);
31 | |         }
32 | |     }
   | |_____^
   |
   = note: `-D renamed-and-removed-lints` implied by `-D warnings`
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: lint unused_doc_comment has been renamed to unused_doc_comments
  --> tools/rust-installer/src/lib.rs:26:5
   |
26 | /     error_chain!{
27 | |         foreign_links {
28 | |             Io(::std::io::Error);
29 | |             StripPrefix(::std::path::StripPrefixError);
30 | |             WalkDir(::walkdir::Error);
31 | |         }
32 | |     }
   | |_____^
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: lint unused_doc_comment has been renamed to unused_doc_comments
  --> tools/rust-installer/src/lib.rs:26:5
   |
26 | /     error_chain!{
27 | |         foreign_links {
28 | |             Io(::std::io::Error);
29 | |             StripPrefix(::std::path::StripPrefixError);
30 | |             WalkDir(::walkdir::Error);
31 | |         }
32 | |     }
   | |_____^
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: lint unused_doc_comment has been renamed to unused_doc_comments
  --> tools/rust-installer/src/lib.rs:26:5
   |
26 | /     error_chain!{
27 | |         foreign_links {
28 | |             Io(::std::io::Error);
29 | |             StripPrefix(::std::path::StripPrefixError);
30 | |             WalkDir(::walkdir::Error);
31 | |         }
32 | |     }
   | |_____^
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to 4 previous errors

error: Could not compile `installer`.

Caused by:
  process didn't exit successfully: `/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/bootstrap/debug/rustc --crate-name installer tools/rust-installer/src/lib.rs --error-format json --crate-type lib --emit=dep-info,link -C opt-level=3 -C metadata=cb0edbb133b08d30 -C extra-filename=-cb0edbb133b08d30 --out-dir /data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps --target x86_64-unknown-openbsd -L dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps -L dependency=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/release/deps --extern tar=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libtar-a79879d0aaa5d01f.rlib --extern clap=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libclap-1f5fff6cd390ab26.rlib --extern xz2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libxz2-f0ab174e0a4d5fa8.rlib --extern error_chain=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/liberror_chain-e8011be67578c54d.rlib --extern rayon=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/librayon-025f5b0b00b7001b.rlib --extern walkdir=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libwalkdir-296d738a531e22af.rlib --extern flate2=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/deps/libflate2-366741584a48b864.rlib -L native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/backtrace-sys-0392c0625ee93f39/out/.libs -L native=/data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/miniz-sys-17c754f01d2fd7d7/out -L /data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/x86_64-unknown-openbsd/stage0-tools/x86_64-unknown-openbsd/release/build/lzma-sys-ad13c785dc5afda7/out/lib` (exit code: 101)
command did not execute successfully: "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-bootstrap-amd64-1.27.0-20180619/bin/cargo" "build" "--target" "x86_64-unknown-openbsd" "-j" "4" "-v" "--release" "--frozen" "--manifest-path" "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/src/tools/rust-installer/Cargo.toml" "--features" "" "--message-format" "json"
expected success, got: exit code: 101
Traceback (most recent call last):
  File "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/x.py", line 20, in <module>
    bootstrap.main()
  File "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/src/bootstrap/bootstrap.py", line 800, in main
    bootstrap(help_triggered)
  File "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/src/bootstrap/bootstrap.py", line 791, in bootstrap
    run(args, env=env, verbose=build.verbose)
  File "/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/rustc-1.27.0-src/src/bootstrap/bootstrap.py", line 148, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /data/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/build/bootstrap/debug/bootstrap dist --jobs=4 src/libstd src/librustc src/doc cargo
*** Error 1 in . (Makefile:175 'do-build')
*** Error 1 in . (/home/semarie/repos/openbsd/ports/infrastructure/mk/bsd.port.mk:2700 '/home/semarie/repos/openbsd/ports/pobj/rust-1.27.0/build-amd64/.build_done')
*** Error 1 in /home/semarie/ports/mystuff/lang/rust (/home/semarie/repos/openbsd/ports/infrastructure/mk/bsd.port.mk:2383 'all')

note:

• the used tarball has git-commit-hash pointing to e0d93fc43c68ed370315c88a4dab67f0268fcd75
• the error-chain crate packaged has information package (in .cargo-checksum.json) ff511d5dc435d703f4971bc399647c9bc38e20cb41452e3b9feb4765419ed3f3

[est31]

There isn't much we can do without a new error-chain release. https://github.com/rust-lang/rust/issues/51501

Beyond switching rust-installer to failure of course :p.

[pietroalbini]

cc @Mark-Simulacrum

[est31]

Why hasn't this been caught while 1.27.0 was in beta?

[semarie]

@est31 it is failing in a specific configuration: rebuilding. bots are testing rustc-1.26.2 is able to build rustc-1.27.0, but not rustc-1.27.0 to be able to build rustc-1.27.0

[est31]

@semarie yeah but it should show up when 1.27.0 tries to build whatever is on nightly.

[semarie]

@est31 here my analysis.

currently nightly is build using 03fb2f447a . I downloaded a beta tarball pointed by https://github.com/rust-lang/rust/blob/master/src/stage0.txt#L15 and checked the version file.

The rename unused_doc_comment to unused_doc_comments was landed later to beta branch in #51037.

so if I didn't mess myself, it seems to me that the latest commit in the beta branch isn't used to build nightly. so not all changes are tested in beta.

[est31]

Oh that's unfortunate. Thanks @semarie for digging this up.

Right so I see the following options:

1. remove (parts of) #50879 from the beta (and stable) branch... one could argue that parts of it are a breaking change and thus should ride the normal trains
2. don't pass -D warnings for the rust-installer crate
3. patch the rust-installer crate to add #[allow(warnings)] or #[allow(renamed_and_removed_lints)] or something like that
4. wait for a new release of error-chain and update to it
5. update to a git version of error-chain... cargo-vendor should support git dependencies now IIRC

[Mark-Simulacrum]

This is indeed unfortunate (sorry about the problem here!) -- I think you should be able to fix this by changing config.toml to have deny-warnings set to false in the [rust] section.

[semarie]

Passing deny-warnings=false in config.toml workaround efficiently the problem. And patching error-chain to add https://github.com/rust-lang-nursery/error-chain/pull/247 too.

[zserik]

https://github.com/rust-lang-nursery/error-chain/issues/251
https://github.com/rust-lang-nursery/error-chain/pull/252

[rossmacarthur]

What needs to be done for this issue to be resolved?

[pietroalbini]

I think the recommended fix for this issue is https://github.com/rust-lang/rust/issues/51650#issuecomment-398838294: it's not hard for downstream users to follow it, and it's too late for us to rebuild the tarballs of 1.27.0.