Powershell: New-PSSession now fails with a NullReferenceException and core dump

[JamesWTruher]

Note from @andschwa (hijacking for visibility):

As this is the oldest issue, I'm going to keep it and close all others. New-PSSession (whether its loopback on the local machine or not) is PSRP (as is Enter-PSSession and Invoke-Command etc.), which is in progress but not yet supported. Please follow the progress on the PSRP on Linux repo. We (mostly @paulcallen and @palladia) are working hard to get this ready, and clearly hear how much you all want it! Please be patient while we finish the implementation.

Original issue:

Before submitting this issue, please first:

• [ ] Search the existing issues.
• [ ] Refer to the FAQ.
• [ ] Refer to the known issues.
• [x] If it is a bug, fill out this form:

Steps to reproduce

new-pssession

Expected behavior

graceful failure

Actual behavior

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Management.Automation.Remoting.PrioritySendDataCollection.Clear()
   at System.Management.Automation.Remoting.Client.BaseClientTransportManager.CloseAsync()
   at System.Management.Automation.Remoting.Client.WSManClientSessionTransportManager.CloseAsync()
   at System.Management.Automation.Remoting.Client.BaseClientTransportManager.Finalize()
Aborted (core dumped)

Environment data

Linux jimtru-ops2 4.2.0-35-generic #40~14.04.1-Ubuntu SMP Fri Mar 18 16:37:35 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Please fill in the results of the following commands:

$ dotnet --info
.NET Command Line Tools (1.0.0-rc2-002543)

Product Information:
 Version:     1.0.0-rc2-002543
 Commit Sha:  38d0c28a1e

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  14.04
 OS Platform: Linux
 RID:         ubuntu.14.04-x64


$ git rev-parse HEAD && git status
PS> git rev-parse HEAD;git status
aa9547930792e5d536f17002a69346cc6c9a0612
On branch master
Your branch is up-to-date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        dotnet-host-ubuntu-x64.latest.deb
        dotnet-sdk-ubuntu-x64.latest.deb
        dotnet-sharedframework-ubuntu-x64.latest.deb
        newitem.tests.ps1

nothing added to commit but untracked files present (use "git add" to track)

[andschwa]

I have good news and bad news. In #939, the ConsoleHost at least doesn't fail immediately. It first bails with:

New-PSSession : Unable to load DLL 'libpsrpomiprov': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
At line:1 char:1
+ New-PSSession
+ ~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-PSSession], DllNotFoundException
    + FullyQualifiedErrorId : System.DllNotFoundException,Microsoft.PowerShell.Commands.NewPSSessionCommand

For the Linux configuration, it's looking for the remoting library, which on Linux, is @paulcallen's PSRP provider. Once I provide it:

New-PSSession : Unable to find an entry point named 'WSManInitialize' in DLL 'libpsrpomiprov'.
At line:1 char:1
+ New-PSSession
+ ~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-PSSession], EntryPointNotFoundException
    + FullyQualifiedErrorId : System.EntryPointNotFoundException,Microsoft.PowerShell.Commands.NewPSSessionCommand

Since WSManInitialize is unimplemented. @ealexjordan and @paulcallen, is this something you plan to implement?

[andschwa]

And on Windows:

New-PSSession : [localhost] Connecting to remote server localhost failed with the following error message : Access is
denied. For more information, see the about_Remote_Troubleshooting Help topic.
At line:1 char:1
+ New-PSSession
+ ~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotin
   gTransportException
    + FullyQualifiedErrorId : AccessDenied,PSSessionOpenFailed

I suspect this is just a configuration issue for me.

[andschwa]

We also don't quite support this:

> Enable-PSRemoting -Force
Enable-PSRemoting : The term 'Enable-PSRemoting' is not recognized as the name of a cmdlet, function, script file, or
operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At line:1 char:1
+ Enable-PSRemoting -Force
+ ~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Enable-PSRemoting:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

And even though I successfully ran it in an administrative session and restarted my ConsoleHost, I still couldn't run it.

[JamesWTruher]

The last one makes sense - it's essentially a registry editor and sets a bunch of WSMan state (although it should work just fine on windows)

[JamesWTruher]

as for the error New-PSSession : [localhost] Connecting to remote server localhost failed take a look at $error[0].Exception (and the rest) - the exception from the remote context should also be available to you, as well as a call stack

[itadder]

Just typed New-PSSession
Then I got this.
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at System.Management.Automation.Remoting.PrioritySendDataCollection.Clear()
at System.Management.Automation.Remoting.Client.BaseClientTransportManager.CloseAsync()
at System.Management.Automation.Remoting.Client.WSManClientSessionTransportManager.CloseAsync()
at System.Management.Automation.Remoting.Client.BaseClientTransportManager.Finalize()
Abort trap: 6

Environment
Darwin MacBook-Pro.posh.net 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 23 18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64

[vijayjt]

I get a similar error as itadder except it core dumps with the Enter-PSSession command on Ubuntu 14.04 as well as when using the CentOS docker image here https://hub.docker.com/r/centos/powershell/.

`PS /home/vijay> Enter-PSSession -ComputerName x.x.x.x -Credential myuser

Windows PowerShell credential request
Enter your credentials.
Password for user myuser: ***

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at System.Management.Automation.Remoting.PrioritySendDataCollection.Clear()
at System.Management.Automation.Remoting.Client.BaseClientTransportManager.CloseAsync()
at System.Management.Automation.Remoting.Client.WSManClientSessionTransportManager.CloseAsync()
at System.Management.Automation.Remoting.Client.BaseClientTransportManager.Finalize()
Aborted (core dumped)`

[andschwa]

As this is the oldest issue, I'm going to keep it and close all others. New-PSSession (whether its loopback on the local machine or not) is PSRP (as is Enter-PSSession and Invoke-Command etc.), which is in progress but not yet supported. Please follow the progress on the PSRP on Linux repo.

[HemantMahawar]

@paulcallen Can you add a label to indicate progress being made?

[gabriel-samfira]

Awesome seeing powershell remoting from Linux to Windows become a reality. Can hardly wait to test this!

[sathya-bhat]

+1, I am eager to test remoting from Linux to Windows too. Thanks in advance!

[arnaud-landry]

"Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object."
Can't wait to manage my servers from OS X ^^

[paulcallen]

latest update: A change was checked in today that will enable powershell remoting from osx/Linux to windows and it should get into the next release of powershell unless something bad happens between now and then.
Things to note:

• basic auth is recommended for now
• because basic auth is recommended you will need a local windows machine account and use that. That account will need to be an admin.
• setting up certs on windows is a pain, which means you probably will be doing basic auth over http. Bit of a security problem.
• there are still a bunch of bugs I am working through in the psrp wsman client.
• Cannot disconnect/reconnect yet -- working on it
• ntlm over spnego authentication is present and working for Linux to Windows, and we can do encryption over http which is the windows default. We are working on documentation to enable this though as it is not straight forward. It also has problems on some older machines. OSX is even more of a pain and not enabled yet.

By all means play with it, but note we are still finishing off the feature and still fixing bugs in stability.

[gabriel-samfira]

This is amazing. Just tested it. Works like a charm with basic auth. Any chance we can get x509 authentication as well? :D

[dvillega]

This makes me so happy. Thank you so much.

[ElFerminador]

Thank you all VERY MUCH for all the effort you're putting into this!! I've recently switched to Mac OS, but I'd really like to retain the ability to remote PowerShell into my Windows servers and also some stuff in the cloud (Office 365, Azure).

I don't code well enough to be much help, but I speak five languages. Let me know if you need any translation done for German, French, Italian, Spanish (or English, obviously ;-) ).

[vinodc]

@paulcallen Great to hear that! Looking forward to this feature.

I tried connecting to Office 365 PowerShell from Ubuntu 14.04 and OS X and received the error message below although it worked as expected from Windows. Is further development required prior to supporting this capability?

PS > $secpasswd = ConvertTo-SecureString $password -AsPlainText -Force                                        
PS > $UserCredential = New-Object System.Management.Automation.PSCredential($email, $secpasswd)
PS > $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection

New-PSSession : [ps.compliance.protection.outlook.com] Connecting to remote server ps.compliance.protection.outlook.com failed with the following error
message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic.
At line:1 char:12
+ $Session = New-PSSession -ConfigurationName Microsoft.Exchange -Conne ...
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemotingTransportException
    + FullyQualifiedErrorId : 1,PSSessionOpenFailed

[paulcallen]

@vinodc the underlying OMI client transport does not support http redirection, so if this results is a http redirect then that is the problem.

[paulcallen]

The .14 alpha release was released before the holidays and it has the client-side remoting I mentioned. On CentOS 7 you will need to create a couple of symbolic links to make certain things work though due to the way we build the underlying OMI binaries and we are still ironing out a few packaging kinks.

cd /opt/microsoft/powershell/6.0.0-alpha.14
sudo ln -s /lib64/libcrypto.so.10 libcrypto.so.1.0.0
sudo ln -s /lib64/libssl.so.10 libssl.so.1.0.0

Other than that basic auth seems to work for psrp remoting over wsman without any further problems.

As a side note we have added SPNEGO authentication for NTLM, but we are hitting a number of issues across different platforms

• OSX does not support NTLM. I wish we realized that before we started trying to get that working :)
• Ubuntu 14.4 does not have the necessary packages available to install
• Ubuntu 16.4 does not have the latest packages available yet, although they are in the process of pushing up some new versions of packages we depend on that fix some interop issues we are having with Windows
• If you have the correct stuff installed and configured on CentOS 7 then ntlm via spnego does work, and we can also do encryption over HTTP eliminating the need for SSL certificates.
  I will make sure we have some documentation on getting NTLM for CentOS 7 installed, but I couldn't find them while writing this message.

We are also working on other cool stuff too to try and make interop from Linux/OSX to Windows easier.

[vinodc]

@paulcallen Thanks for the quick response. Running curl -v -u user:pass https://ps.compliance.protection.outlook.com/powershell-liveid/ returns a 200 status code with no redirects. The New-PSSession command also continues to succeed on Windows even after removing the -AllowRedirection option. Would there be any other steps I could take to determine the cause of the error?

[paulcallen]

@vinodc unfortunately at this time we don't support the ConnectionUri way of specifying a destination for Linux remoting, and it does not seem like there are sufficient other options to allow you to specify the http end-point in the pssessionoptions. You can use -computername and -usessl, but the powershell-liveid has no way of been specified any other way.
Issue: PowerShell/psl-omi-provider#67

[paulcallen]

@gabriel-samfira For x509 support, are you referring to the use of client SSL certificates as authentication, and WinRM using certificate mapping feature on Windows?

[gabriel-samfira]

@paulcallen yes :-)

[paulcallen]

@gabriel-samfira adding issue to track this: PowerShell/psl-omi-provider#68

[gorankor]

Regarding your comment for the alpha .14 release - can we get some details for getting NTLM and SPNEGO working from Linux to Windows for Centos 7? I have PowerShell from git built and I am using it with https://github.com/PowerShell/psl-omi-provider. It works with Basic authentication but I am interested in getting it going with HTTP encryption. In the end I'd like to use this with my own tool, similar to "omicli" tool from https://github.com/Microsoft/omi, and using the libraries coming form there and from the https://github.com/PowerShell/psl-omi-provider repository.

[paulcallen]

PowerShell/psl-omi-provider#75 is tracking the documentation and is assigned to @yakman2020

[yakman2020]

I am working on doc as we speak. To see in progress, take a look at the OMI branch, more specifically setup doc for NTLM

[JamesWTruher]

it would be great to get some definitive documentation on how to set this up. The new-pssession test currently causes PowerShell to exit. I know this works in CI, what can I do on my local system to get this working?

[paulcallen]

We will be dropping new nuget packages to powershell in a week or two that will make the client more stable. It will also enable initial work for getting Linux/OSX powershell connection to exchange/office365. At the same time new documents for NTLM will be posted. You can look at the documents referred to by @yakman2020. If there is something specific we can certainly help you.

[ElFerminador]

Excellent news about being able to access Office 365 from Mac OS! On another subject, how will updating PowerShell on the Mac work? I've recently switched from Windows, still a bit new to git. Thanks for any advice!

[JamesWTruher]

I have a little more data to add - I have updated my base PowerShell to version .15 and I'm now not seeing the issue with that version. However, I just built and ran, with the following strangeness:

PS /home/james/src/github/Forks/JamesWTruher/PowerShell-1> new-pssession -ComputerName foobarbazbip -Authentication basic
new-pssession : Unable to load DLL 'libpsrpclient': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
At line:1 char:1
+ new-pssession -ComputerName foobarbazbip -Authentication basic
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-PSSession], DllNotFoundException
    + FullyQualifiedErrorId : System.DllNotFoundException,Microsoft.PowerShell.Commands.NewPSSessionCommand

PS /home/james/src/github/Forks/JamesWTruher/PowerShell-1> try { new-pssession -ComputerName foobarbazbip -Authentication basic } catch { "hi" }
hi
PS /home/james/src/github/Forks/JamesWTruher/PowerShell-1> cd test/powershell

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Management.Automation.Remoting.PrioritySendDataCollection.Clear() in /home/james/src/github/Forks/JamesWTruher/PowerShell-1/src/System.Management.Automation/engine/remoting/fanin/PriorityCollection.cs:line 158
   at System.Management.Automation.Remoting.Client.BaseClientTransportManager.CloseAsync() in /home/james/src/github/Forks/JamesWTruher/PowerShell-1/src/System.Management.Automation/engine/remoting/fanin/BaseTransportManager.cs:line 949
   at System.Management.Automation.Remoting.Client.WSManClientSessionTransportManager.CloseAsync() in /home/james/src/github/Forks/JamesWTruher/PowerShell-1/src/System.Management.Automation/engine/remoting/fanin/WSManTransportManager.cs:line 1219
   at System.Management.Automation.Remoting.Client.BaseClientTransportManager.Finalize() in /home/james/src/github/Forks/JamesWTruher/PowerShell-1/src/System.Management.Automation/engine/remoting/fanin/BaseTransportManager.cs:line 998

I don't know whether this helps you at all, but I thought I would let you know.

[paulcallen]

@JamesWTruher I just installed .15 on my Ubuntu 16.04 machine and things worked just fine. What platform are you on?

[paulcallen]

@ElFerminador mac osx installation details are here. I believe it deletes the old directory and installs a new one. That is what happens on Ubuntu 16.04 at least.

[paulcallen]

I am closing this thread as the primary issue should now be fixed, along with some of the sub issues that have shown up on this thread. The office 365 fixes should be in the alpha.17 build when it comes out.

[vinodc]

@paulcallen I've confirmed connectivity to Office 365 via PowerShell from Ubuntu 14.04 works great!! Thank you and @yakman2020 very much for your hard work the past two months to make this happen. Really appreciate it.

[andschwa]

This is awesome!

[paulcallen]

@vinodc Glad it is working for you.
Also wanted to mention @palladia as well. Both @yakman2020 and @palladia have been instrumental to our OMI/PSRP work on Linux.

[yakman2020]

Thanks. It really means a lot to hear.

Now on to the next thing, already in progress.

[rodel0723]

@paulcallen as you have mentioned to @vinodc here that proxy setting/authentication is not yet supported, is this still true on below version?

PSVersion 6.0.0-beta
PSEdition Core
GitCommitId v6.0.0-beta.5
OS Linux 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017
Platform Unix
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0

if yes, are there any workaround to use powershell behind proxy on linux for New-PSSession cmdlet?

thank you in advance.

[paulcallen]

OMI client does not support proxy settings, and does not respond to proxy responses. Sorry.
If there are enough asks for it we could certainly consider it. If it were done, then the psrp client layer for Linux would need to be updated once completed.
Don't know how Microsoft does it inside their firewall, but our proxy servers have a component installed on the client machine that does the proxying automatically so we don't need support at the application level. I think it handles it at the socket layer.

[rodel0723]

@paulcallen Thank you for your reply. How about setting the http_proxy before running powershell, will it take too long to change the code? It will be really helpful for us working behind proxies if it will be implemented.

[paulcallen]

I created an OMI issue to track adding proxy support for the omi client: omi issue 403