PowerShell on macOS to work with MacPorts OpenSSL Libraries

[kai-h]

Further to issue https://github.com/PowerShell/PowerShell/issues/5561

Can we please have PowerShell work out of the box with MacPorts' installed OpenSSL libraries.

PowerShell is hard coded to specifically look in /usr/local/opt/openssl/lib for the OpenSSL Libraries. This is where Homebrew installs them.

MacPorts installs the libraries in /opt/local/lib instead.

If I symlink the Homebrew location to the MacPorts location, then PowerShell works but it would be very nice to not have this extra step when installing PowerShell.
sudo ln -s /opt/local/lib /usr/local/opt/openssl/lib

_As an aside, many systems administrators consider Homebrew to be a significant security risk as it changes the ownership of key system folders from root to the currently logged in user. It does not work in multi-user environments and it goes against the Unix philosophy of making 3rd party software have a minimal impact on a running system and be self-contained._

[iSazonov]

Please look the comment https://github.com/PowerShell/PowerShell/issues/5561#issuecomment-429142776

[kai-h]

Based on my testing, PowerShell is still hard-coded to look in /usr/local/opt/openssl/lib for OpenSSL libraries. It is not also looking in /opt/local/lib as well.
For PowerShell to work with OpenSSL from MacPorts, you must symlink the libraries like so:
sudo ln -s /opt/local/lib /usr/local/opt/openssl/lib
It would be very good if PowerShell could look in /opt/local/lib for OpenSSL libraries, particularly as per
https://github.com/PowerShell/PowerShell/issues/5561#issuecomment-429142776 the latest 6.1.0 version was not working with Homebrew on macOS Mojave, whereas it is working with MacPorts, iff you symlink the libraries to where Homebrew puts them.

[iSazonov]

/cc @SteveL-MSFT

[SteveL-MSFT]

@TravisEz13 can you look into this?

[TravisEz13]

@kai-h Can we get repro steps of the problem you are having?

I only see a similar issue to what your are talking about in the code in libmi but I hit a different issue before I ever get to that code: https://github.com/PowerShell/MMI/issues/33

[kai-h]

@TravisEz13 PowerShell remoting does not work out of the box with a clean install of PowerShell. It relies on OpenSSL libraries to be installed as well.

Without any 3rd party OpenSSL libraries installed, this is what you get:
```` PowerShell
[kai@blackmac ~]$ pwsh
PowerShell 6.1.0
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/pscore6-docs
Type 'help' to get help.

PS /Users/kai> $UserCredential = Get-Credential

PowerShell credential request
Enter your credentials.
User: [email protected]
Password for user [email protected]: **

PS /Users/kai> $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
New-PSSession : This parameter set requires WSMan, and no supported WSMan client library was found. WSMan is either not installed or unavailable for this system.
At line:1 char:12

• $Session = New-PSSession -ConfigurationName Microsoft.Exchange -Conne ...
• ~~~~~~~~~~~~~~
• CategoryInfo : ResourceUnavailable: (:) [New-PSSession], PSRemotingTransportException
• FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException,Microsoft.PowerShell.Commands.NewPSSessionCommand

PS /Users/kai>
````
The fix for this is to install Homebrew which is a package management system and software ports library for macOS. Homebrew installs the OpenSSL libraries in /usr/local/opt/openssl/lib and if PowerShell can find these libraries installed in this location, PowerShell remoting should work. In particular, I believe it's looking for libssl.dylib

Many systems administrators, myself included, believe that Homebrew represents an unacceptable security risk. It changes ownership of key unix directories to that of the currently logged-in user. It explicitly does not support keeping these directories owned by root:wheel and running as root via sudo. Homebrew will also not work with multiple users on the one machine.

MacPorts is an alternative to Homebrew. It follows more closely to traditional Unix and Linux package management systems like apt and the FreeBSD ports system. It does not change permissions on any system folders and works in a multi-user environment.

MacPorts installs the OpenSSL libraries in /opt/local/lib
After installing the MacPorts installer .pkg, you can install OpenSSL via
sudo port install openssl

If I symlink this location to the Homebrew location, then PowerShell can locate the OpenSSL libraries and remote sessions can be established.

e.g.:
```` PowerShell

[kai@blackmac ~]$ sudo ln -s /opt/local/lib /usr/local/opt/openssl/lib
Password:
[kai@blackmac ~]$ pwsh
PowerShell 6.1.0
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/pscore6-docs
Type 'help' to get help.

PS /Users/kai> $UserCredential = Get-Credential

PowerShell credential request
Enter your credentials.
User: [email protected]
Password for user [email protected]: ***

PS /Users/kai> $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
PS /Users/kai> Import-PSSession $Session
WARNING: The names of some imported commands from the module 'tmp_hsj4sv05.rbt' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.

ModuleType Version Name ExportedCommands
---------- ------- ---- ----------------
Script 1.0 tmp_hsj4sv05.rbt {Add-AvailabilityAddressSpace, Add-DistributionGroupMember, Add-Mai...

PS /Users/kai>
````

[TravisEz13]

Notes to self:
The error about requiring WSMan is coming from here: https://github.com/PowerShell/PowerShell/blob/64c1ca8926b15c476f6bb14814c55b25b844fc6e/src/System.Management.Automation/engine/remoting/fanin/WSManTransportManager.cs#L2622

Which loads

/usr/local/microsoft/powershell/6/libpsrpclient.dylib:
    @rpath/libpsrpclient.dylib (compatibility version 0.0.0, current version 0.0.0)
    @executable_path/libmi.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)

@kai-h Do you have any problems with Invoke-WebRequest to HTTPS sites on machines configured how you want them to be configured?

libmi looks like this:

    /usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)

vs System.Security.Cryptography.Native.OpenSsl.dylib (used by the web cmdlets):

    @rpath/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)

[kai-h]

@TravisEz13 - the main reason I use PowerShell is to admin Office 365 from a Mac.
As it stands, PowerShell, when executing remote sessions, relies on the OpenSSL libraries, and requires this dependancy to be installed separately.

In my experience, you can not get remoting to work without OpenSSL installed, and out of the box, PowerShell expects it to be in /usr/local/opt/openssl/lib

On my system, without Homebrew installed, /usr/local/opt doesn't exist. This directory, and everything below it, is installed by Homebrew - including the OpenSSL libraries.

If I instead use MacPorts to install OpenSSL, the libraries are installed in /opt/local/lib

I then need to symlink /opt/local/lib to /usr/local/opt/openssl/lib - then PowerShell can find the libraries where it expects to see them, and a remote session to Office 365 works.

macOS also ships with a couple of different versions of libssl.dylib (and libcrypto) in /usr/lib
ls -l /usr/lib/libssl* /usr/lib/libssl.0.9.7.dylib /usr/lib/libssl.0.9.8.dylib /usr/lib/libssl.35.dylib /usr/lib/libssl.43.dylib /usr/lib/libssl.44.dylib /usr/lib/libssl.dylib -> libssl.35.dylib

otool shows me that libpsrpclient.dylib uses the following libraries:
otool -L /usr/local/microsoft/powershell/6/libpsrpclient.dylib /usr/local/microsoft/powershell/6/libpsrpclient.dylib: @rpath/libpsrpclient.dylib (compatibility version 0.0.0, current version 0.0.0) @executable_path/libmi.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
then, looking at libmi.dylib
otool -L /usr/local/microsoft/powershell/6/libmi.dylib /usr/local/microsoft/powershell/6/libmi.dylib: @rpath/libmi.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2) /usr/lib/libpam.2.dylib (compatibility version 3.0.0, current version 3.0.0) /usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)

macOS ships with OpenSSL libraries, and in my extremely limited testing, they seem to work with PowerShell. These libraries are installed in /usr/ib
The difference is the name of the libraries - PowerShell seems like it's specifically looking for libssl.1.0.0.dylib and libcrypto.1.0.0.dylib
These exact filenames do not match anything in /usr/lib - instead there libraries with different version numbers and then a symlink of a lib with no version number - e.g.
ls -al /usr/lib/libcrypto.dylib /usr/lib/libssl.dylib /usr/lib/libcrypto.dylib -> libcrypto.35.dylib /usr/lib/libssl.dylib -> libssl.35.dylib
If I instead symlink these libraries into where PowerShell expects to see them, then remoting also seems to work - again, this is limited testing, I established a session to Office 365 and ran Get-Mailbox [email protected] just to verify I could send and receive some useful information.
[kai@blackmac /usr/local/opt/openssl/lib]$ pwd /usr/local/opt/openssl/lib [kai@blackmac /usr/local/opt/openssl/lib]$ ls -l libcrypto.1.0.0.dylib -> /usr/lib/libcrypto.dylib libssl.1.0.0.dylib -> /usr/lib/libssl.dylib

[TravisEz13]

@kai-h Thanks, but to make sure we can create the fix correctly, I REALLY need the following information.
Do you have any problems with Invoke-WebRequest to HTTPS sites on machines configured how you want them to be configured?

[kai-h]

I'm sorry, I don't know what Invoke-WebRequest is, or what it does.
If I check the docs for it, there's a lot of information - I'm not sure what's applicable to this particular situation.
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-6
If I were to run it as per one of the examples, would that give you the information we need?
e.g.
Example 2: Get links from a web page
PowerShell (Invoke-WebRequest -Uri "https://aka.ms/pscore6-docs").Links.Href

[TravisEz13]

If you run that and it works, it would be a good indication that the library it uses has no problem loading OpenSSL. Make sure that the machine does NOT have OpenSSL installed via HomeBrew, any extra symbolic links or anything you don't want to have on the machine.

I get these results when I run that command

#                                                                               
#
/profile
#
https://github.com/sdwheeler
https://github.com/kizerkizer
https://github.com/tylerl0706
https://github.com/matt9ucci
https://github.com/thedanfernandez
https://github.com/PowerShell/PowerShell-Docs/blob/staging/reference/docs-conceptual/PowerShell-Scripting.md
https://github.com/powershell/powershell
https://github.com/PowerShell/PowerShell#get-powershell
https://github.com/PowerShell/PowerShell/blob/master/docs/learning-powershell
#
https://docs.microsoft.com/previous-versions/
https://docs.microsoft.com/teamblog
https://docs.microsoft.com/contribute
https://go.microsoft.com/fwlink/?LinkId=521839
/en-us/legal/termsofuse
https://aka.ms/sitefeedback
https://www.microsoft.com/en-us/legal/intellectualproperty/Trademarks/EN-US.aspx
#
https://docs.microsoft.com/previous-versions/
https://docs.microsoft.com/teamblog
https://docs.microsoft.com/contribute
https://go.microsoft.com/fwlink/?LinkId=521839
/en-us/legal/termsofuse
https://aka.ms/sitefeedback
https://www.microsoft.com/en-us/legal/intellectualproperty/Trademarks/EN-US.aspx

[kai-h]

[TravisEz13]

[kai-h]

[TravisEz13]