PowerShell should respect /etc/paths.d/* and /etc/paths on macOS

[abock]

Steps to reproduce

Write-Host $env:PATH

Expected behavior

$env:PATH should be built from paths defined in files under /etc/paths.d (where third parties may define custom paths) and the /etc/paths file (where macOS itself defines paths).

Example Paths

• /etc/paths (which is "owned" by macOS itself, third parties should not
  write to it):
  /usr/local/bin /usr/bin /bin /usr/sbin /sbin
• /etc/paths.d/dotnet:
  /usr/local/share/dotnet
• /etc/paths.d/mono-commands:
  /Library/Frameworks/Mono.framework/Versions/Current/Commands

Actual behavior

$env:PATH contains seemingly hard-coded paths:

/usr/local/microsoft/powershell/6.0.0-beta.9:/usr/bin:/bin:/usr/sbin:/sbin

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta.9
PSEdition                      Core
GitCommitId                    v6.0.0-beta.9
OS                             Darwin 16.7.0 Darwin Kernel Version 16.7.0: T...
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Profile-based implementation

I have implemented the following in my PowerShell profile which redefines $env:PATH as I would expect (don't judge - I've been using PowerShell for about a week 😬).

if ($IsMacOS) {
  function Append-Path {
    $allPaths = @()
    for ($i = 0; $i -lt $args.length; $i++) {
      $path = $args[$i];
      if (Test-Path -PathType Leaf $path) {
        [IO.File]::ReadAllLines($path) | Foreach-Object {
          $allPaths += $_
        }
      } elseif (Test-Path -PathType Container $path) {
        Get-ChildItem $path | Foreach-Object {
          $allPaths += Append-Path($_.FullName)
        }
      }
    }
    return $allPaths
  }

  $path = @(
    Get-Command pwsh | `
      Select-Object -ExpandProperty Definition | `
      Split-Path -parent
  )

  $path += Append-Path "/etc/paths" "/etc/paths.d"

  $env:PATH = ($path -join ":")

  Remove-Variable -Name path
  Remove-Item -Path Function:\Append-Path
}

$env:PATH is now fully populated as I'd expect when I start a new PowerShell session:

> Write-Host $env:PATH
/usr/local/microsoft/powershell/6.0.0-beta.9:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/share/dotnet:/Library/Frameworks/Mono.framework/Versions/Current/Commands

Implementation using path_helper -c

Thanks to @markekraus for pointing out path_helper - we can make this a lot simpler:

if ($IsMacOS -And (Test-Path -PathType Leaf /usr/libexec/path_helper)) {
  function setenv ($variable, $value) {
    [Environment]::SetEnvironmentVariable($variable, $value)
  }

  Invoke-Expression $(/usr/libexec/path_helper -c)

  Remove-Item -Path Function:\setenv
}

[iSazonov]

Related acticle.

[markekraus]

Hmm, is it too much to ask Apple to add a -p option to path_helper and have PowerShell call the binary?

I'm all for having pwsh support this, but, I'm somewhat against reinventing it within PowerShell. If macOS updates their implementation to include something like ~./profile/paths.d/, then we'd have to stay on top of that change and add it to our own implementation. It makes better sense to me to call out to path_helper and either parse or evaluate the result.

[abock]

@markekraus nice, I was completely unaware of path_helper. If invoked with -c we can use it directly from PowerShell already (we just need to define setenv first):

if ($IsMacOS -And (Test-Path -PathType Leaf /usr/libexec/path_helper)) {
  function setenv ($variable, $value) {
    [Environment]::SetEnvironmentVariable($variable, $value)
  }

  Invoke-Expression $(/usr/libexec/path_helper -c)

  Remove-Item -Path Function:\setenv
}

[thezim]

@markekraus as PowerShell in Windows does pull in PATH the same equivalency should be afforded to macOS or any other OS. If you ask Core for PATH does it respect /etc/paths values?

[markekraus]

@thezim the way /etc/paths and /etc/paths.d/ is implemented is through a call to path_helper in /etc/profile. so this isn't the OS giving PATH to the login shell, its the login shell generating PATH on initialization through its profile. There are 2 possible switched to path_helper: -c and -s, -c generated a csh string that can be evaluated by csh to setup the path and likewise -s does the same for sh and friends.

What I'm suggesting is the best ultimate path is to have Apple add a -p which emits a string that can evaluated by Invoke-Expression and then we can implement it similar to the other shells in our own pwsh global profile.

Until then, I'm suggesting we parse the output of path_helper -s or path_helper -c rather than in our own code try to duplicate the logic of path_helper. That way we can avoid having to race to maintain our implementation when Apple possibly changes to either include new PATH definition locations or remove existing ones. Also, then we wont have to worry about differences that may exist between, say, macOS 10.12 and macOS 10.15, just as bash/sh/csh don't need to worry about such things.

For clarity, if you have bash/sh as your login shell and then launch pwsh, the PATH as defined in the parent shell IS respected in PowerShell. /etc/paths and /etc/paths.d/ only auto loads for the login shell. if you were to log in with bash, add a new item to /etc/paths.d/, and then launch python, it would not see the new path. Same with pwsh. The reason you see new paths created since login in child sh/bash/csh shells is that they evaluate /etc/profile on launch too.

The reason pwsh as a login shell doesn't process /etc/paths and /etc/paths.d/ is because pwsh does not run /etc/profile and really couldn't because it is written for other shells that use a completely different language.

[abock]

@markekraus @thezim as I commented/updated the issue for: by invoking path_helper -c, nothing needs to be parsed. The C-Shell command is syntactically valid PowerShell - we just need to temporarily define setenv:

setenv PATH "... paths ...";

If anyone wants to lobby Apple for a -p option and if they ever implement it, that'd be great, but I certainly don't think PowerShell should block on that.

[markekraus]

@abock I feel like creating a temporary function and then deleting it is a bit hacky.

In my experience, Environment variables are used way more heavily in *nix than Windows. With that in mind, I think it probably makes sense to expose a Set-EnvironmentVariable with the setenv alias along with a Get-EnvironmentVariable with the getenv alias.

Set-EnvironmentVariable would take a -Name and -Value with -Name as position 0 and -Value position 1. This would also help with some confusion about ENV vars only accepting strings as the -Value parameter would accept a String.

With that in place, the profile can work very similar to the csh: Test-Path for path_helper, execute with -c, and then evaluate with Invoke-Expression.

This, of course, means exposing 2 new cmdlets that will probably not see much heavy usage outside of macOS and possibly other *nix. They are kind of redundant given powershell exposes the $env: variable prefix. But I still think this is the cleaner solution.

Another possibility is to have setenv as an alias of Set-Variable and some kind of logic to set an environment variable when the setenv alias is called. I'm not real thrilled with this solution. I dislike command behavior changing based on InvocationName. This is just as "hacky" to me as creating a temporary function and removing it.

[iSazonov]

If we run PowerShell from Bash why we don't get PATH extended by path_helper ?

[markekraus]

@iSazonov We do:

If you have something already in /etc/paths or /etc/paths.d/ when you started your bash session, then PATH will be set with those settings and pwsh will receive them.

If you log in with bash and then add a file under /etc/paths.d/ with a new path and then start pwsh (without having first exited and relaunched bash), then you will not see the new path in pwsh.

If you log in with bash, add a file under /etc/paths.d/ with a new path, exit bash, log in with bash again, start pwsh you will see the new path.

If you log in with bash, add a file under /etc/paths.d/ with a new path, launch a child bash shell, and then start pwsh in the child bash shell you will see the new path.

if you set pwsh as your login shell and start a console or ssh to the node, none of the paths in /etc/paths or /etc/paths.d/ are loaded (because pwsh is not running path_helper)

So long as PATH is set correctly before pwsh is launched it will work. The problem is that pwsh will not respect changes made to /etc/paths or /etc/paths.d/ since the parent shell was launched and will also not respect them when pwsh is the login shell.

[iSazonov]

@markekraus Thanks for clarify!

It is standard bevavior for environment variables on all platforms. All processes inherit them this way.

Also I believe we still never consider PowerShell as login shell on Unix because of incompatibility. /cc @mklement0

[markekraus]

@iSazonov

Also I believe we still never consider PowerShell as login shell on Unix because of incompatibility

Well, if you are like me and don't run into native globing issues, it makes an excellent login shell. :)

[mklement0]

@markekraus: Good analysis; clarification:

The reason you see new paths created since login in child sh/bash/csh shells is that they evaluate /etc/profile on launch too.

POSIX-compatible shells such as bash (including bash in disguise on macOS, sh) and ksh (but _not_ zsh) source /etc/profile _only_ when (effectively) launched as a _login_ shell, with option -l or via login (see below).
This happens _implicitly_ when Terminal.app, the macOS terminal program, creates a new tab / window, but it doesn't happen when you, say, invoke bash (without -l) from an existing shell.
This behavior is specific to macOS: terminal programs on Linux create _non-login_ shells by default - see https://stackoverflow.com/a/23233967/45375

pwsh -l would actually break (PowerShell neither knows -l, nor does it know the distinction between a login shell and a(n interactive) non-login shell), but login, which is what Terminal.app uses, employs a different, convention-based technique to signal to the shell invoked that it should consider itself a _login_ shell: it places the name of the shell preceded by - in the first argv argument ($0, in POSIX terms); e.g., -bash.
This is clearly something that PowerShell can - and does - ignore.

Arguably, though, PowerShell _should_ pay attention to that, and evaluate /usr/libexec/path_helper output only then.

Not sure if this is _technically_ feasible: both [environment]::CommandLine and [environment]::GetCommandLineArgs() seem to only reflect the _PowerShell DLL name_ from within PowerShell; e.g., /usr/local/microsoft/powershell/6.0.0-beta.9/pwsh.dll.

---

As for a simpler way to apply /usr/libexec/path_helper output, using PowerShell code, without requiring ephemeral functions:

if ($IsMacOs) { $env:PATH=((/usr/libexec/path_helper) -split '"')[1] }

Note that this assumes that there are no _escaped, embedded_ " chars., but that's generally a reasonable assumption (which @abock's function-based, /usr/libexec/path_helper -c relies on, too).

Arguably, this code belongs in $profile.AllUsersAllHosts - ideally, however, with a conditional that only applies it if $0 starts with -.

---

More generally, https://github.com/PowerShell/PowerShell/issues/975#issuecomment-331049792 discusses how PowerShell might fit into the world of POSIX-compatible shells.

[markekraus]

Hmm what is the mechanism that is making child bash sells pickup new additions to /etc/paths.d/ since login if it is not coming from /etc/profile?

[mklement0]

_Child_ bash shells do _not_ pick up new additions (unless you invoke them with -l or via login).

_New terminal tabs / windows_, however, do - because they're invoked via login, as described.

[markekraus]

erm.. then something is off in my environment?? Child bash shells are picking up the changes. This is why I assumed the /etc/profile is being run (none of the other profile files have anything dealing with paths). I haven't done anything fancy with this mac since it was re-imaged and upgraded to sierra.

repro:

1. ssh to macOS
2. add new path to /etc/paths.d/newpath
3. echo $PATH to verify path is not shown
4. run bash with no arguments
5. echo $PATH new path shown.

*shrugs. I'll admit I have almost no clue what I'm doing in macOS 😄 It is quite possible I have done something wrong along the way.

[mklement0]

ssh introduces additional behaviors, so where you ssh _from_ may matter.

I just tried from a Ubuntu 16.04 system, and I do _not_ see the behavior you're describing - so: where _are_ you SSH-ing from?

And, _without_ ssh in the picture: do you see behavior that differs from what I've described?

[iSazonov]

I think we can close the Isuue as #975 dup.

[markekraus]

@iSazonov agreed. this looks like a dup.

@mklement0 Not sure what the cause was but after re-imaging and upgrading again I can't repro it so there must have been something special about that environment. *shrugs