rustup nightly install failing

[davidroeca]

Just getting back into rust and installed rustup. Tried to install the nightly build and encountered the following error:

$ rustup install nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: installing component 'rustc'
info: rolling back changes
error: failed to extract package
info: caused by: failed to unpack `rustc-nightly-x86_64-unknown-linux-gnu/rustc/share/man/man1/rustdoc.1` into `/home/david/.rustup/tmp/wnj79prt3m8vg6f2_dir/rustc/share/man/man1/rustdoc.1`
info: caused by: No such file or directory (os error 2)

Not sure if this is a nightly issue, a rustup issue, or my own lack of experience with rustup. Currently running Linux Mint 18

[messense]

Same issue on macOS 11.12

[est31]

I can confirm on Ubuntu 17.04. rustup update is broken as well. @davidroeca this is a very recent issue.

[gyscos]

Here is what I get after installing the current rustup from git (to get debug info and backtraces):

% ~/.cargo/bin/rustup update nightly
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: downloading component 'rust-analysis'
info: downloading component 'rls'
info: downloading component 'rust-src'
info: downloading component 'rust-std' for 'asmjs-unknown-emscripten'
info: downloading component 'rust-std' for 'wasm32-unknown-emscripten'
info: installing component 'rustc'
info: rolling back changes
error: failed to extract package
info: caused by: failed to unpack `rustc-nightly-x86_64-unknown-linux-gnu/rustc/share/man/man1/rustdoc.1` into `/home/gyscos/.rustup/tmp/_mffp8148_7wf8w9_dir/rustc/share/man/man1/rustdoc.1`
info: caused by: No such file or directory (os error 2)
info: backtrace:

stack backtrace:
   0:     0x55f2010a2894 - backtrace::backtrace::libunwind::trace
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.0/src/backtrace/libunwind.rs:53
                         - backtrace::backtrace::trace<closure>
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.0/src/backtrace/mod.rs:42
   1:     0x55f2010a372f - backtrace::capture::{{impl}}::new
                        at /home/gyscos/fetched/rustup.rs/target/debug/build/backtrace-6ddf4a4263d5dc15/out/capture.rs:79
   2:     0x55f20109ae3c - error_chain::make_backtrace
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/error-chain-0.7.1/src/lib.rs:383
   3:     0x55f200b8032c - core::ops::FnOnce::call_once<fn() -> core::option::Option<alloc::arc::Arc<backtrace::capture::Backtrace>>,()>
                        at /checkout/src/libcore/ops.rs:2626
   4:     0x55f200b5926c - core::option::{{impl}}::or_else<alloc::arc::Arc<backtrace::capture::Backtrace>,fn() -> core::option::Option<alloc::arc::Arc<backtrace::capture::Backtrace>>>
                        at /checkout/src/libcore/option.rs:640
   5:     0x55f200b44c6a - error_chain::{{impl}}::new<rustup_dist::errors::Error>
                        at /home/gyscos/.cargo/registry/src/github.com-1ecc6299db9ec823/error-chain-0.7.1/src/lib.rs:437
   6:     0x55f200bf8a5b - rustup_dist::errors::{{impl}}::chain_err::{{closure}}<(),std::io::error::Error,closure,rustup_dist::errors::ErrorKind>
                        at /home/gyscos/fetched/rustup.rs/<error_chain_processed macros>:117
   7:     0x55f200b74c46 - core::result::{{impl}}::map_err<(),std::io::error::Error,rustup_dist::errors::Error,closure>
                        at /checkout/src/libcore/result.rs:486
   8:     0x55f200bf79d1 - rustup_dist::errors::{{impl}}::chain_err<(),std::io::error::Error,closure,rustup_dist::errors::ErrorKind>
                        at /home/gyscos/fetched/rustup.rs/<error_chain_processed macros>:115
   9:     0x55f200bdacde - rustup_dist::component::package::unpack_without_first_dir<flate2::gz::DecoderReader<std::fs::File>>
                        at src/rustup-dist/src/component/package.rs:202
  10:     0x55f200bd9f7b - rustup_dist::component::package::{{impl}}::new<flate2::gz::DecoderReader<std::fs::File>>
                        at src/rustup-dist/src/component/package.rs:183
  11:     0x55f200bdb4f9 - rustup_dist::component::package::{{impl}}::new<std::fs::File>
                        at src/rustup-dist/src/component/package.rs:232
  12:     0x55f200bdba4c - rustup_dist::component::package::{{impl}}::new_file
                        at src/rustup-dist/src/component/package.rs:236
  13:     0x55f200be427b - rustup_dist::manifestation::{{impl}}::update
                        at src/rustup-dist/src/manifestation.rs:170
  14:     0x55f200bcc817 - rustup_dist::dist::update_from_dist_
                        at src/rustup-dist/src/dist.rs:656
  15:     0x55f200bcbcce - rustup_dist::dist::update_from_dist
                        at src/rustup-dist/src/dist.rs:619
  16:     0x55f200b1e26e - rustup::install::{{impl}}::run
                        at src/rustup/install.rs:50
  17:     0x55f200b09851 - rustup::toolchain::{{impl}}::install
                        at src/rustup/toolchain.rs:113
  18:     0x55f200b0aa6f - rustup::toolchain::{{impl}}::install_from_dist_inner
                        at src/rustup/toolchain.rs:175
  19:     0x55f200b0a5b5 - rustup::toolchain::{{impl}}::install_from_dist
                        at src/rustup/toolchain.rs:170
  20:     0x55f2009f7a2d - rustup_init::rustup_mode::update
                        at src/rustup-cli/rustup_mode.rs:488
  21:     0x55f2009ec639 - rustup_init::rustup_mode::main
                        at src/rustup-cli/rustup_mode.rs:33
  22:     0x55f200a15dd8 - rustup_init::run_multirust
                        at src/rustup-cli/main.rs:82
  23:     0x55f200a15b86 - rustup_init::main
                        at src/rustup-cli/main.rs:57
  24:     0x55f2010fc15a - panic_unwind::__rust_maybe_catch_panic
                        at /checkout/src/libpanic_unwind/lib.rs:98
  25:     0x55f2010f59b1 - std::panicking::try<(),fn()>
                        at /checkout/src/libstd/panicking.rs:433
                         - std::panic::catch_unwind<fn(),()>
                        at /checkout/src/libstd/panic.rs:361
                         - std::rt::lang_start
                        at /checkout/src/libstd/rt.rs:57
  26:     0x55f200a1a8a2 - main
  27:     0x7f5759081510 - __libc_start_main
  28:     0x55f200971539 - _start
  29:                0x0 - <unknown>

[TimNN]

strace output:

info: installing component 'rustc'
open("/home/logic/.rustup/tmp/hmzwj4cotkanpzvj_file", O_RDONLY|O_CLOEXEC) = 3
mkdir("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir", 0777) = 0
open("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir/rustc/share/man/man1/rustdoc.1", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 ENOENT (No such file or directory)
open("/home/logic/.rustup/tmp/3v47lhv9_ptrqrvj_dir", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3

[TimNN]

Ok, the problem seems to be that the tar.gz generation changed and no longer includes directory entries, so the tar.gz extractor used by rustup no longer creates any directories.

[jonhoo]

Can confirm on Arch Linux as well.

[alexcrichton]

@ranma42 this is likely related to https://github.com/rust-lang/rust/pull/41600, notably this range of changes, mind taking a look?

[brson]

Thanks for investigating @TimNN.

I might guess it was this PR updating rust-installer that caused the change on Rust's side.

[brson]

Damn you @alexcrichton!

[TimNN]

This causes the problem, I think https://github.com/rust-lang/rust-installer/compare/4f994850808a572e2cc8d43f968893c8e942e9bf...4cf7397fb0566e745f0bce4c5b009cfeb5d12c53#diff-534b12d64247754d4938410a9ba07bd3R274

find "$CFG_INPUT" \( -type d -empty \) -or \( -not -type d \) \
    | rev | sort | rev | tar -cf "$CFG_OUTPUT.tar" -T -
need_ok "failed to tar"

Not a find expert here, but I think that means that only empty directories are included in the output .tar (and files, of course).

[TimNN]

Rustup does the unpacking of tar files here: https://github.com/rust-lang-nursery/rustup.rs/blob/master/src/rustup-dist/src/component/package.rs#L202, however unpack requires all intermediary directories to have been created: https://docs.rs/tar/0.4.11/tar/struct.Entry.html#method.unpack

[brson]

Is it obvious to anyone how to change the tar command to include the directories again? I think we could just remove all of "( -type d -empty ) -or ( -not -type d ) " but I'm not sure.

Safe thing would be to just revert rust-installer.

@alexcrichton What time does the next nightly start building?

[alexcrichton]

@brson needs to be past @bors by 0:00 UTC which gives us ~19 hours from this comment

[jonhoo]

@brson I believe the find options can be removed, and in fact that point we could probably just use tar directly instead of using find at all..

[jonhoo]

Haha, yes, exactly like rust-lang/rust-installer#61.

[ranma42]

@brson just removing the \( -type d -empty \) -or \( -not -type d \) condition would cause files to be included multiple times, because by default tar recurses into subdirectories.
This would list all folders (both empty and non-empty) in the archive, but I expect it to still cause issues because the folder entry might appear after the files it contains.

find "$CFG_INPUT" | rev | sort | rev | tar --no-recursion -cf "$CFG_OUTPUT.tar" -T -
need_ok "failed to tar"

@jonhoo yes, that restores the order in which tar lists the archive entries, but it also worsen the compression.

A fix on the rustup side would be to always create the full path (just like tar and the other tools that manage tar files do).
EDIT: a fix on the rust-installer side that does not regress the compression might be something along the lines of

(find "$CFG_INPUT" -type d ; \ # depth-first listing of directory tree
 find "$CFG_INPUT" -not -type d | rev | sort | rev) \ # regrouped listing
  | tar --no-recursion -cf "$CFG_OUTPUT.tar" -T -
need_ok "failed to tar"

What is the best way to test if it works?

[brson]

I submitted a PR to update rust-installer with @TimNN's patch to use the simple tar invocation. It can be reoptimized later.

A fix on the rustup side would be to always create the full path (just like tar and the other tools that manage tar files do).

That does seem reasonable. After it was in place for some length of time we could modify the tarballs again to remove these directories.

What is the best way to test if it works?

Hm, it's actually not simple to feed rustup an arbitrary tarball that I can think of. The easiest way to test might be to make the update and send it back through nightly and see if it breaks again.

[leonardo-m]

I hate tar. I prefer to use a single program like 7-zip that fetches by itself the files, compresses them, and later rebuilds the original directory tree and decompresses the files.

[fschutt]

Is there a way to fix this yet for travis, like a configuration setting to fetch the latest rustup? Tried to setup Travis CI, but it currently fails to install rustc: https://travis-ci.org/sharazam/printpdf/jobs/229028283#L131

[frewsxcv]

@sharazam If I were you (and to others who get this email), I'd enable the allow_failures setting for nightly Rust on Travis CI. Even their docs recommend this:

https://docs.travis-ci.com/user/languages/rust/#Choosing-the-Rust-version

language: rust
rust:
  - stable
  - beta
  - nightly
matrix:
  allow_failures:
    - rust: nightly

EDIT: forgot to mention 'nightly Rust'

[cuviper]

@ranma42 FWIW in my Rust rewrite (rust-lang/rust#41569) I can easily tar just directories and then still have your rev-sorted file list. I've noted this issue, so I'll add something to that effect today.

[ranma42]

@cuviper You're right that is probably easier and less error prone in your rewrite (I see that you also skip the temporary tar file... nice! :D ).
I still think rustup should be able to untar archives like those generated last night, but leaving rust-installer as-is until the Rust rewrite is merged should be fine (well, I hope so; the next nightly build will tell us).

[brson]

@sharazam One way is to temporarily 'pin' your CI to a specific nightly from the archives with e.g. --toolchain=nightly-2017-05-03.

[brson]

Not actually fixed yet @bors!

[brson]

The fix is upstream and tomorrow's nightly should work.

[fschutt]

Nightly works again (Travis). Thanks.