Powershell: Unhandled exception. System.ComponentModel.Win32Exception (2): No such file or directory on trying to set pwsh as default shell

[JacksonKearl]

Steps to reproduce

MacOS 10.14.6

dotnet tool install --global PowerShell
ln -s /Users/jkearl/.dotnet/tools/pwsh /usr/local/bin/pwsh
vim /etc/shells  # add /usr/local/bin/pwsh
pwsh # works
exit
chsh -s /usr/local/bin/pwsh

open new terminal

Last login: Sat Mar 14 00:05:26 on ttys005
Unhandled exception. System.ComponentModel.Win32Exception (2): No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start(String fileName, String arguments)
   at Microsoft.PowerShell.GlobalTool.Shim.EntryPoint.Main(String[] args)

[Process completed]

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0
PSEdition                      Core
GitCommitId                    7.0.0
OS                             Darwin 18.7.0 Darwin Kernel Version 18.7.0: Sat Oct 12 00:02:19 PDT 2019; root:xnu-4903.278.12~1/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[iSazonov]

/cc @adityapatwardhan

I am not sure that pwsh from global tools is designed to being used as standalone shell and moreover as default shell.
Perhaps @rjmholt could add more info.

[mklement0]

Currently, the dotnet-tool PowerShell version can indeed not function as the default shell on macOS, because it is invoked with -l there, which fails:

PS> ~/.dotnet/tools/pwsh -l
Unknown option: -l
.NET Core SDK (3.1.102)
Usage: dotnet [runtime-options] [path-to-application] [arguments]
...

The problem happens on reinvocation via /bin/sh.

The divergent behavior of the dotnet-tool PowerShell version is a recurring problem, which notably includes severely broken parsing of command-line arguments in general - see #11747

I think it is certainly desirable to allow a dotnet-tool installed version to act as the default shell.

[JacksonKearl]

I have explicitly disabled passing -l and still lget the error

[mklement0]

@JacksonKearl, out of curiosity: how did you disable -l?

[JacksonKearl]

I'm using hyper, and it has a setting for shell launch parameters.

[mklement0]

In short:

• The problem is related to the dotnet-tool-installed version being a _framework-dependent_ application.

• As a workaround, tell your terminal to start PowerShell as follows:

/bin/sh -lc 'PATH=~/".dotnet:$PATH" DOTNET_ROOT=~/.dotnet ~/.dotnet/tools/pwsh'

---

(I realized that creating a default login shell via Terminal.app, for instance, doesn't actually use option -l to request a login shell, but sets $0 (the first argument passed to the binary) to -<shellBinaryName>, with the - prefix by convention requesting a _login_ shell - and a regularly installed PowerShell version does seem to honor that.)

Given how you invoke PowerShell, the problem is not directly related to having configured it as your _default shell_ - though that doesn't work either.

When I try what you tried with Hyper analogously with iTerm2 - that is, launch the shell executable explicitly, without options - I get a different error message:

A fatal error occurred. The required library libhostfxr.dll could not be found.
If this is a self-contained application, that library should exist in [~/.dotnet/tools/.store/powershell/7.0.0/powershell/7.0.0/tools/netcoreapp3.1/any/]
If this is a framework-dependent application, install the runtime in the global location [/usr/local/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location or register the runtime location in [/etc/dotnet/install_location]

This suggests that the dotnet-tool-installed version is a _framework-dependent_ application, and that it therefore needs to be told where to find the .NET Core _runtime_ (if it isn't in /usr/local/share/dotnet or listed in file /etc/dotnet/install_location).

If I do tell it where to find the runtime - e.g. by launching via /bin/sh -c 'DOTNET_ROOT=~/.dotnet ~/.dotnet/tools/pwsh' - I do _on occasion_ get the error message you saw, but more typically I just see Unhandled exception.

However, the error you saw can reliably be reproduced as follows:

• Enter a pristine shell that doesn't have the usual /etc/profile environment variables as follows: env -i /bin/sh

• From that shell, run DOTNET_ROOT=~/.dotnet ~/.dotnet/tools/pwsh

As it turns out, it is not only env. var DOTNET_ROOT that must point to ~/.dotnet, but this directory must also be in the PATH variable.

Therefore, the following startup command works:

/bin/sh -lc 'PATH+=~/.dotnet DOTNET_ROOT=~/.dotnet ~/.dotnet/tools/pwsh'

However, trying to use a dotnet tool-installed PowerShell version as the _default shell_ presents a bootstrapping problem, even if you were to add these entries to /etc/profile, given that pwsh when invoked as a login shell must be able to start _first_, before re-invoking itself via /bin/sh -l in order to ensure processing of /etc/profile.

[mklement0]

@JacksonKearl, I've added a workaround and more findings to my previous comment.

[iSazonov]

We have an report that dash is broken in the scenario. See #12058

[mklement0]

@iSazonov, I had to amend my previous comment again; what it comes down to: not only must DOTNET_ROOT be set to ~/.dotnet, but the latter must also be listed in PATH.

The linked issue doesn't apply, because there is no dash on macOS; /bin/sh is, in essence, bash on macOS (it's a separate binary that is a custom build of bash with a few changes hard-coded; verify with sh --version).

Also, I just saw that we're actually using /bin/zsh -l on macOS, though I'm unclear on why.

[iSazonov]

@mklement0 Thanks for in-depth investigating the issue!

The linked issue doesn't apply, because there is no dash on macOS;

It was more for @rjmholt experience who works on improvements of the login shell feature code.

presents a bootstrapping problem

Should we delegate this problem to MSFT team?

As it turns out, it is not only env. var DOTNET_ROOT that must point to ~/.dotnet, but this directory must also be in the PATH variable.

The same question. Also should we fix PowerShell global tool installer to add the path in PATH variable?

[mklement0]

I think we must first decide whether we want to officially support the dotnet tool-installed version as a user's default shell.

I think it won't be possible if we keep it as a framework-dependent application.
But changing that may not be an option for this installation method (I know little about .NET tools).

For running it as a terminal emulator's manually configured startup shell we at least have a workaround now.

Also should we fix PowerShell global tool installer to add the path in PATH variable?

I don't think that would help: macOS has no single location (anymore) for defining environment variables that GUI applications (which includes terminal emulators) all see.

[iSazonov]

Is global tools installed in user profile?

[mklement0]

From what I can tell, global tools by default always install in $HOME/.dotnet/tools, across platforms, i.e., the installation is by default always in a user-specific location.

[iSazonov]

Thanks for confirmation my guess! In the case global tools installer does not change system global configs and we can not expect that the system can start the shell well out of the user session. In particular, as result pwsh cannot find a custom .Net installation.
In other words, the installer only configures _the user profile_ so that pwsh works well. And any attempt to use it from another profile (or system init) is doomed.

[JacksonKearl]

@mklement0

As a workaround, tell your terminal to start PowerShell as follows:
/bin/sh -lc 'PATH=~/".dotnet:$PATH" DOTNET_ROOT=~/.dotnet ~/.dotnet/tools/pwsh'

This gives me error:

A fatal error occurred. The required library libhostfxr.dylib could not be found.
If this is a self-contained application, that library should exist in [/Users/jkearl/.dotnet/tools/.store/powershell/7.0.0/powershell/7.0.0/tools/netcoreapp3.1/any/].
If this is a framework-dependent application, install the runtime in the global location [/usr/local/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location or register the runtime location in [/etc/dotnet/install_location].

The .NET Core runtime can be found at:
  - https://aka.ms/dotnet-core-applaunch?missing_runtime=true&arch=x64&rid=osx.10.14-x64

Is there a different way to install powershell that would make using it as a default shell work better?

[mklement0]

Is there a different way to install powershell that would make using it as a default shell work better?

Yes, if you use the official installer package (*.pkg) available for download from this repo (https://github.com/PowerShell/PowerShell#get-powershell) or, if you have Homebrew installed, with brew cask install powershell - see the instructions for macOS.

That way, PowerShell will be installed as a self-contained (not framework-dependent) application, and the problem shouldn't occur.

---

That said, you _can_ make this work with the .NET global tool version, even as the default shell - I'll post a summary later.

As for the workaround for use as a terminal startup command:

Sorry I didn't make clearer that ~/.dotnet in the workaround was a placeholder for whatever the .NET Core SDK / runtime installation directory is on your system - by default, it is /usr/local/share/dotnet.

Therefore the abstract form of the workaround is:

/bin/sh -lc 'PATH="<dotnet-install-dir>:$PATH" DOTNET_ROOT="<dotnet-install-dir>" ~/.dotnet/tools/pwsh'

With the standard location:

/bin/sh -lc 'PATH="/usr/local/share/dotnet:$PATH" DOTNET_ROOT="/usr/local/share/dotnet" ~/.dotnet/tools/pwsh'

(Strictly speaking, you don't need the DOTNET_ROOT=... part with the _standard_ location, but you need it with any nonstandard one.)

GitHub

PowerShell/PowerShell

PowerShell for every system! Contribute to PowerShell/PowerShell development by creating an account on GitHub.

[mklement0]

The short of it is:

If PowerShell is installed as a .NET global tool, and you want to configure it as your _default shell_.

• With a _standard, machine-level_ .NET Core SDK / runtime installation in _standard locations_:

  • on Linux (at least on Ubuntu), should work as-is (CLI dotnet symlinked to /usr/bin, install directory in standard location /usr/share/dotnet/).

  • on macOS (CLI dotnet symlinked to /usr/local/bin, install directory in standard location /usr/local/share/dotnet), requires extra, one-time configuration that involves sudo, and therefore requires administrative privileges:

  • sudo launchctl config system path /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet

• With a .NET Core installation in a _custom location_:

  • Careful extra configuration is required on both platforms, requiring sudo - see bottom section.

---

As a less invasive alternative that doesn't require sudo, instead of defining pwsh as your default shell, configure your terminal emulator to launch PowerShell as follows:

• Linux:

  • With a _standard_ installation:

    • bin/sh -c 'exec ~/.dotnet/tools/pwsh'

  • With a _custom_ installation (substitute the full path of your custom .NET install directory for <dotnet-root-dir>):

    • /bin/sh -c 'PATH="<dotnet-root-dir>:$PATH" DOTNET_ROOT="<dotnet-root-dir>" exec ~/.dotnet/tools/pwsh'

• macOS:

  • With a _standard_ installation:

    • /bin/sh -lc 'PATH="/usr/local/share/dotnet:$PATH" exec ~/.dotnet/tools/pwsh'

  • With a _custom_ installation (substitute the full path of your custom .NET install directory for <dotnet-root-dir>):

    • /bin/sh -lc 'PATH="<dotnet-root-dir>:$PATH" DOTNET_ROOT="<dotnet-root-dir>" exec ~/.dotnet/tools/pwsh'

---

@iSazonov, the section below addresses this comment of yours:

the system can start the shell well out of the user session. In particular, as a result pwsh cannot find a custom .Net installation.
In other words, the installer only configures the user profile so that pwsh works well.

Background information, default shell use with _custom_ .NET installations:

The .NET Core SDK installer adds the .NET global tools directory to $env:PATH _system-wide_, even though it is a _user_ location. That is, each user sees _their_ ~/.dotnet/tools directory in $env:PATH:

• On macOS, this directory added via plaint-text file /etc/paths.d/dotnet-cli-tools, which /etc/profile processes by invoking /usr/libexec/path_helper.

• On Linux, it is shell script /etc/profile.d/dotnet-cli-tools-bin-path.sh, also processed by /etc/profile.

However, it isn't the _tools_ directory that matters here, it is the location of the dotnet CLI and the the .NET _install_ directory (inside of which the runtimes are located) that matters - both of them must be discoverable when a _framework-dependent_ .NET application - such as pwsh if installed as a .NET global tool - starts.

(Whether the _tools_ directory is discoverable is irrelevant in this scenario, because in order to define a shell as the default shell, its _full path_ is passed to cshsh -s anyway).

The dotnet CLI and the install directory _are_ discoverable with standard installations of the .NET Core SDK / runtime.

However, the challenge is that this discoverability _always_ (macOS) / _situationally_ (Linux) depends on having processed /etc/profile and ~/.profile _first_.

That is a catch-22 (bootstrapping problem) for a framework-dependent PowerShell: Because It cannot process these files itself (but needs to, if invoked as a _login_ shell), it can only process them by reinvoking itself via /bin/sh -l (or equivalent). But in order to do so it needs to be able to start first - which, by default, it _categorically cannot_ (macOS) / _situationally cannot_ (Linux).

---

macOS

With a standard installation (via the official .pkg installer, as used by brew cask install dotnet-sdk), a framework-dependent application:

• _can_ discover the install directory due its presence in its _standard location_, /usr/local/share/dotnet/

• _cannot_ discover the dotnet CLI, because it is symlinked to from /usr/local/bin/dotnet, and /usr/local/bin/ is only added to $env:PATH _via /etc/profile_; on macOS _non-shell_(-launched) processes - such as GUI applications - see _only_ the following directories in $env:PATH: /usr/bin:/bin:/usr/sbin:/sbin

Trying to work around this catch-22 by creating a symlink to the CLI in /usr/bin, for instance, is tempting, but the macOS SIP (System Integrity Protection) feature prevents that.

The fix is to add the install directory (where the CLI resides as well) to the _system-level default $env:PATH value_, after its default entries, using launchctl, which requires sudo and therefore _administrative privileges_ (and a reboot for the change to take effect).
(Note that attempts to modify the user-level definition are seemingly quietly ignored).

sudo launchctl config system path /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet

Note that this has two implications, though I generally wouldn't expect them to be problematic:

• Even non-shell(-launched) processes (GUI apps) then see this new $env:PATH entry.

• Shell(-launched) processes see the new $env:PATH entry _before_ any additional entries are made via /etc/profile / ~/.profile / their own initialization files.

With a custom installation of .NET Core, _additional work_ is required:

• First, adapt and submit sudo launch command above with the full path of your custom install directory substituted for /usr/local/share/dotnet.

• Then, create plain-text file /etc/dotnet/install_location (requires sudo; create directory /etc/dotnet on demand) and place the full path of your custom install directory in there.

Note that the alternative method for making a custom install directory location discoverable - environment variable DOTNET_ROOT is _not_ an option, because macOS - by design, for security reasons - doesn't offer a way to set environment variables other than PATH at the system level.

---

Linux (as tested on Ubuntu 18.04)

With a standard installation (sudo apt install powershell-sdk-<major>-<minor>; e.g., sudo apt install powershell-sdk-3-1 as of PowerShell 7.0), everything should work as-is:

Neither the discoverability of the CLI nor the install directory depend on /etc/profile processing in that case (/usr/bin, where the dotnet CLI symlink is placed, is in $env:PATH by virtue of being defined in /etc/environment, and the CLI can discover the install directory in its standard location, /usr/share/dotnet/).

With a custom installation of .NET Core, _additional work_ is required, which requires sudo and therefore administrative privileges:

• First, if not already present, create a symlink to the dotnet CLI in one of the $env:PATH directories _listed directly in /etc/environment,_ such as /usr/bin or /usr/local/bin (requires sudo).

  • Note that, for instance, a snap-installed PowerShell version creates a /snap/bin/dotnet symlink, but /snap/bin is only added to $env:PATH if /etc/profile is processed.

• Then, create plain-text file /etc/dotnet/install_location (requires sudo; create directory /etc/dotnet on demand) and place the full path of your custom install directory in there.

Note: It may be tempting to try a user-level configuration via ~/.profile, but that won't work in all situations:

• It _does_ work when you log on _locally_, via the desktop environment, because a hidden /bin/bash instance then processes /etc/profile and ~/.profile _at logon time_ (at least on Ubuntu, where the GNOME-based desktop is initialized that way), and all processes in that session inherit the resulting environment modifications. It is for this reason that - unlike on macOS - Linux terminal emulators do _not_ launch shells as _login_ shells, because it isn't necessary.

• However, it does _not_ work in ssh sessions, where the user's default shell _is_ invoked as a _login_ shell, and is itself expected to process /etc/profile and ~/.profile

[JacksonKearl]

thanks for your help, I'm using the

/bin/sh -lc 'PATH="/usr/local/share/dotnet:$PATH" DOTNET_ROOT="/usr/local/share/dotnet" ~/.dotnet/tools/pwsh'

approach as I want Hyper to use Powershell but Terminal.app to use zsh. (for easy context switching)

It all seems to work well with this setup.

Maybe add this info to the README?

[mklement0]

Glad to hear it, @JacksonKearl.

Note the list of current bugs when you've use PowerShell as a .NET global tool (all CLI-related):

• CLI argument handling is broken with respect to arguments with embedded spaces and quotes: #11747

• On macOS, you can't use pwsh to launch another instance (use ~/.dotnet/tools/pwsh as a workaround): #12205

• -l / -login doesn't work: #12176

[SteveL-MSFT]

cc @adityapatwardhan