rustup doc does not work on Windows/WSL

[kobenauf]

Problem
On Windows, using WSL 2, rustup doc --std (or any other rustup doc command) opens firefox correctly (because I have BROWSER=firefox.exe), but the browser does not load a file; the browser is empty/blank.

I found the rustup docs here:

~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/share/doc/rust/html/index.html

I could open this manually like this:

firefox.exe file://///wsl$/Ubuntu/home/kobenauf/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/share/doc/rust/html/index.html

Or like this:

firefox.exe `wslpath -w ~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/share/doc/rust/html/index.html`

Possible Solution(s)
Consider making rustup doc aware of how to open files on WSL so they open the docs as expected.

Notes

Output of rustup --version:
rustup 1.21.1 (7832b2ebe 2019-12-20)

Output of rustup show:
Default host: x86_64-unknown-linux-gnu
rustup home: /home/kobenauf/.rustup

installed toolchains

stable-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu (default)

installed targets for active toolchain

wasm32-unknown-unknown
x86_64-pc-windows-gnu
x86_64-unknown-linux-gnu

active toolchain

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.42.0-nightly (689fca01c 2020-01-17)

[kinnison]

WSL is, as far as Rustup is concerned, Linux. As such, it's using the xdg-open embedded in the opener crate. @Seeker14491 do you have any ideas how this might be accomodated?

[rbtcollins]

We could sniff the kernel to detect WSLv1, but that won't help on WSLv2. I am not sure how to detect WSLv2 at this point. Perhaps there will be an lsb-release metadata tag of some sort.

I think the BROWSER command here is the problem: firefox.exe isn't a wsl-aware command. I suggest making a firefox wrapper script within WSL itself that:

• checks $1 for filesystem parameters
• converts them as appropriate with wslpath etc
• then calls firefox.exe

[Seeker14491]

As kinnison said, the opener crate is currently oblivious to WSL and so just uses xdg-open like any other Linux. While the open node package (which opener took inspiration from) has checks for WSL, I'm not sure that's the right approach. Perhaps it is xdg-open which should support WSL, like what's proposed in this patch.

[kobenauf]

For what it's worth, the patch @Seeker14491 linked for xdg-open was originally reported Dec 2018, so chances it is going to get fixed there seem low.

[kinnison]

@kobenauf It seems to me that it'd be up to the WSL distributions to realise they're on WSL/WSL2 and locally patch their xdg-open if upstream won't. @Seeker14491 is there any way (for now?) that the opener crate might detect WSL/WSL2 and use an internal patched version of xfg-open short-term?

[therealkenc]

I am not sure how to detect WSLv2 at this point

This, but RIIR.

Perhaps it is xdg-open which should support WSL

Or WSL users can (and do) swap out their xdg-open for a wrapper that takes into account the path conversion.

This is unrelated to MIcrosoft/WSL#4649. There is some navel gazing in #3907, others. Another option is to have rustup serve the page on http://localhost:someport/somepath instead of file://somewhere if you want to be schmancy. Bonne chance.

[Seeker14491]

Well currently users can't just swap out their xdg-open, because we're shipping our own xdg-open, though we could change the opener crate to try using the system xdg-open first before falling back to our own xdg-open.
Of course, it would be nice if this worked out-of-the-box, so @kinnison's suggestion seems like a viable option. Basically, we would use the wslpath command in our modified xdg-open to convert from a WSL path to a Windows path, whenever the script tries to launch an .exe.

[therealkenc]

Well currently users can't just swap out their xdg-open

Okay got it. Users not only _can_ swap their xdg-open(1), they _do_; but that won't work because xdg-open(1) it isn't actually spawned.

@kinnison's suggestion is workable.

we could change the opener crate to try using the system xdg-open first before falling back to our own xdg-open.

Yes, there's a large(ish) community using native chrome/firefox on WSL with remote X11 (VcXsrv and such). For them native system xdg-open(1) (and opener::open()) works. One way to spot those folks is check existence $DISPLAY.

whenever the script tries to launch an .exe

Counting on the .exe extension is one heuristic you can use. That approach leans hard on $BROWSER being set per the OP. Another way to go is to spawn powershell.exe start <wsl path>, which is the approach taken in the patch you cite. That's probably better (for some value of better).

Great to see out-of-the box would-be-niceness for WSL being entertained. Big 👍

[rbtcollins]

How does invoking the non-WSL binary work? Does it show up in a recognisable path? We could do path normalisation things if (pseudo code):

which(browser)
if wsl-windows-image
   wsl-ify-path
fork-and-exec

[therealkenc]

How does invoking the non-WSL binary work?

It's a binfmt_misc under the hood. At the execve() syscall level it looks just like anything else that has a custom interpreter. The exeve() syscall follows the $PATH resolution rules same as usual. You can mostly ignore the details.

Does it show up in a recognisable path?

That's a good question, and the answer is "not necessarily". There are options controlled by wsl.conf. (1) WSL users might not have drvfs (f.e. /mnt/c) automounted. That much can be a hard fail. (2) Even if they mount drvfs, users can opt out of importing the Windows %PATH% into WSL $PATH.

And... _even if_ users haven't futzed the wsl.conf defaults, there isn't any guarantee firefox.exe is in their Window %PATH%. In particular chrome.exe usually _isn't_. MS Edge poses it's own challenges (and that's the default for anyone who hasn't gotten around to installing Firefox or Chrome).

This is one reason spawning powershell.exe start is probably a (scary quote) "better" approach than hoping that path walking firefox.exe works out. Or that $BROWSER is set in the first place. You know where powershell.exe lives, %PATH% import or not.

We could do path normalisation things if (pseudo code):

I assume implied here is trying to do a wslpath(1) equivalent yourself rather than spawning the utility. Anything _can_ be done, but it isn't as straightforward as it sounds. In particular there is no straightforward way to get the name of the distro once you're inside WSL (and hence glean the correct \\wsl$\distroname\).

All that said there isn't anything (strictly-speaking) _incorrect_ with the psudocode. Left undrescribed is how if wsl-windows-image should behave. You _could_ try a file(1) equivalent, but that might not give you the sort of result you expect (because... complicated). Alternately you can glean whether the browser executable is on a "windows path", but that isn't as straightforward as it sounds either because there is no guarantee the "Windows path" lives on /mnt/c. Docker-types in particular like mounting on /c via the wsl.conf [automount] root option. You have to parse /proc/self/mounts to get it right.

All of which is _doable_.

Alternately something like:

if not xdg-open-equivalent-succeeded($native_url_path)
  if on-wsl()   # WSL 1 or 2 doesn't actually matter
     $win_url_path = fork_and_exec("/bin/wslpath", "-w", $native_url_path)
     fork_and_exec("powershell.exe", "start", $win_url_path)

Noting deliberate absence of $browser variable above, because not this pseudocode's responsibility.

[kinnison]

If the system xdg-open is correctly updated on WSL, then that seems like the safest option frankly.

[therealkenc]

If the system xdg-open is correctly updated on WSL, then that seems like the safest option frankly.

That's the standard operating procedure (if there is such a thing). WSL users wrap their xdg-open. It's (probably) a stretch for xdg-open(1) proper to change their ways; it isn't xdg-util's problem, at least arguably. If opener::open() wants to act as the xplat intermediary, that's fine. WSL is "another platform". That is probably where the "if (wsl)" would go, if the crate is insistent on rolling it's own xdg-open equivalent for the Real Linux™ (meaning, freedesktop.org) case rather than spawning the real thing.

[paulirotta]

WSL Ubuntu Rustup is still not working correctly:

paul@MSI:~$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
info: downloading installer

Welcome to Rust!

This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.

Rustup metadata and toolchains will be installed into the Rustup
home directory, located at:

  /home/paul/.rustup

This can be modified with the RUSTUP_HOME environment variable.

The Cargo home directory located at:

  /home/paul/.cargo

This can be modified with the CARGO_HOME environment variable.

The cargo, rustc, rustup and other commands will be added to
Cargo's bin directory, located at:

  /home/paul/.cargo/bin

This path will then be added to your PATH environment variable by
modifying the profile file located at:

  /home/paul/.profile

You can uninstall at any time with rustup self uninstall and
these changes will be reverted.

Current installation options:


   default host triple: x86_64-unknown-linux-gnu
     default toolchain: stable (default)
               profile: default
  modify PATH variable: yes

1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>

info: profile set to 'default'
info: default host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2020-08-03, rust version 1.45.2 (d3fb005a3 2020-07-31)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: Defaulting to 500.0 MiB unpack ram
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `22`,
 right: `4`', src/libstd/sys/unix/thread.rs:179:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `22`,
 right: `4`', src/libstd/sys/unix/thread.rs:179:21
stack backtrace:
   0:     0x7ffc00cdaffd - backtrace::backtrace::libunwind::trace::h812748238d609e46
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1:     0x7ffc00cdaffd - backtrace::backtrace::trace_unsynchronized::h7c97e818aebf09c8
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2:     0x7ffc00cdaffd - std::sys_common::backtrace::_print_fmt::h60d914263b0ccd71
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x7ffc00cdaffd - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf78227137afc7565
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x7ffc0096524c - core::fmt::write::h543cdf60775f89bf
                               at src/libcore/fmt/mod.rs:1069
   5:     0x7ffc00cda904 - std::io::Write::write_fmt::h0c7f3ce24c679426
                               at src/libstd/io/mod.rs:1504
   6:     0x7ffc00cda105 - std::sys_common::backtrace::_print::h80e55e24be231368
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x7ffc00cda105 - std::sys_common::backtrace::print::h3b197b9c1261c865
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x7ffc00cda105 - std::panicking::default_hook::{{closure}}::ha6c807149ce20f8f
                               at src/libstd/panicking.rs:198
   9:     0x7ffc00cd98f4 - std::panicking::default_hook::he49a9c12e358cc45
                               at src/libstd/panicking.rs:218
  10:     0x7ffc00cd93b6 - std::panicking::rust_panic_with_hook::h93f74f5ef2f71f31
                               at src/libstd/panicking.rs:515
  11:     0x7ffc00cd9198 - rust_begin_unwind
                               at src/libstd/panicking.rs:419
  12:     0x7ffc00cd9140 - std::panicking::begin_panic_fmt::hfa6ef29ba81f400e
                               at src/libstd/panicking.rs:373
  13:     0x7ffc00c11671 - <rustup::diskio::threaded::Threaded as rustup::diskio::Executor>::join::habf6ee901a47a8a4
  14:     0x7ffc00c10c78 - core::ptr::drop_in_place::h0b1b0dcb4b07c267
  15:     0x7ffc00b76c40 - core::ptr::drop_in_place::h09572fbec04e583f
  16:     0x7ffc00c2842e - rustup::dist::component::package::unpack_without_first_dir::h99f4b0c2cae349ff
  17:     0x7ffc00be559f - rustup::dist::manifestation::Manifestation::update::hacf67fad3b44f03a
  18:     0x7ffc00bd563c - rustup::dist::dist::update_from_dist_::h1462627f9c58c494
  19:     0x7ffc00bd227a - rustup::install::InstallMethod::install::hbb6430b2dfaf5792
  20:     0x7ffc00bd0d21 - rustup::toolchain::DistributableToolchain::install_from_dist::h7b9bf9317c9aa2c5
  21:     0x7ffc00cc5aa2 - rustup::cli::self_update::install::h8ca0f61a773526a9
  22:     0x7ffc00ccbd3d - rustup::cli::setup_mode::main::h4d335e1871435d70
  23:     0x7ffc008df42a - rustup_init::main::h22da971c634937a2
  24:     0x7ffc00cf3e83 - std::rt::lang_start_internal::{{closure}}::{{closure}}::h4ed4ab1fb893cc93
                               at src/libstd/rt.rs:52
  25:     0x7ffc00cf3e83 - std::sys_common::backtrace::__rust_begin_short_backtrace::h1f01c818c00c4f70
                               at src/libstd/sys_common/backtrace.rs:130
  26:     0x7ffc008e053f - main
  27:     0x7ffc004170b3 - __libc_start_main
  28:     0x7ffc008db029 - <unknown>
thread panicked while panicking. aborting.
Illegal instruction (core dumped)
paul@MSI:~$

[rbtcollins]

@paulirotta check your wsl version. If it is 1, then this is the known issue with newer glibc, and you either need to downgrade glibc or upgrade wsl. However, you've commented on an unrelated bug sso I'm going to delete your comment, and mine both in a few minutes.

[vkottler]

If the system xdg-open is correctly updated on WSL, then that seems like the safest option frankly.

That's the standard operating procedure (if there is such a thing). WSL users wrap their xdg-open. It's (probably) a stretch for xdg-open(1) proper to change their ways; it isn't xdg-util's problem, at least arguably. If opener::open() wants to act as the xplat intermediary, that's fine. WSL is "another platform". That is probably where the "if (wsl)" would go, if the crate is insistent on rolling it's own xdg-open equivalent for the Real Linux™ (meaning, freedesktop.org) case rather than spawning the real thing.

Example solution I came up with:

#!/bin/bash

if [ "$#"  -lt 1 ]; then
        echo "$0: no args"
        exit 1
fi

OPEN_PATH=`wslpath -w $1 2>/dev/null`

if [ $? -eq 0 ]; then
        wslview $OPEN_PATH
else
        wslview $1
fi

Write this bash to a script (e.g. $HOME/bin/wsl_browser.sh), then export BROWSER="$HOME/bin/wsl_browser.sh" and sudo ln -s $HOME/bin/wsl_browser.sh /usr/local/bin/xdg-open and seems to get rid of most problems I have at CLI. Haven't found a use-case to keep any form of a real xdg-open around on wsl.

[kinnison]

That's an interesting approach, and much closer to logic that opener could plausibly think about using. It could detect the presence of wslpath and wslview and do the above, falling back to the system xdg-open and/or its internal implementation as needed. @Seeker14491 would it make sense for someone to open an issue to that effect over on your repo?

[therealkenc]

The wslpath application is is part of WSL and always available (it is actually a symlink to WSL's /init). The wslview app is part of wslu is here. It is a short shell script which is effectively powershell.exe start <url> with some sophistication. Same as "alternately something like" from January.

[Seeker14491]

@kinnison definitely. I think this approach could work, but I'm thinking instead we first check if there is a system xdg-open, falling back to the wslpath and wslview approach. This is to allow the use of a customized xdg-open if the user has one, and also to allow opening with native Linux applications. At least on the Ubuntu WSL, xdg-open is not installed by default, so the wslpath and wslview fallback would get used by default, which I think is the right default.

I'm additionally considering introducing a second API function to the opener crate; something like open_browser() that would explicitly use the $BROWSER environment variable before falling back to the generic opening strategy. On WSL, we could check for $BROWSER being set to an .exe, and if so, convert the path with wslpath before opening it.

[workingjubilee]

@rustbot label: +O-windows +O-linux

[DesmondWillowbrook]

Any progress on this PR?

[kinnison]

@DesmondWillowbrook I'm afraid we're beholden to someone providing @Seeker14491 with the help needed to get the opener crate to the point that it can better support this use-case.