PowerShell hangs when exiting having used a PowerShell runspace pool/runspace

[TylerLeonhardt]

PowerShell Core hangs when exiting and using a PowerShell runspace pool. On Windows PowerShell it exits successfully.

Steps to reproduce

here's the gist for my code: https://gist.github.com/tylerl0706/2d87a13c6147278bb4e8565585aa9722

Steps:

PS > Add-Type -Path "path/to/TestHttpListener/bin/Debug/netstandard2.0/TestHttpListener.dll"
PS > $app = [TestHttpListener.TestHttpListener]::new()
PS > $app.Start(8081)
PS > Invoke-WebRequest -Uri http://localhost:8081 # VERY IMPORTANT
PS > exit

Expected behavior

PowerShell successfully exists

Actual behavior

PowerShell hangs and doesn't give the prompt back - this is causing my AppVeyor and Travis builds to fail due to timeout :(

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta.8
PSEdition                      Core
GitCommitId                    v6.0.0-beta.8
OS                             Darwin 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[TylerLeonhardt]

@vors and I were both able to repro this locally.

[SteveL-MSFT]

I think I have a simpler repro without using httplistener

PS /Users/steve> $rs = [runspacefactory]::CreateRunspacePool(1,5)                                                                                               
PS /Users/steve> $rs.Open()                                                                                                                                     
PS /Users/steve> $ps = [powershell]::Create()                                                                                                                   
PS /Users/steve> $ps.RunspacePool = $rs                                                                                                                         
PS /Users/steve> $null = $ps.AddScript({1+1})                                                                                                                           
PS /Users/steve> $ps.Invoke()
PS /Users/steve> exit #hangs            

[PaulHigin]

This is hanging on the PipelineThread._workItemReady event wait. On exit this event should get set and release the thread wait via the finalizer:
https://github.com/PowerShell/PowerShell/blob/master/src/System.Management.Automation/engine/hostifaces/LocalPipeline.cs#L1340

But it looks like the finalizer is not being called.
@daxian-dbw This appears to be a bug in the .Net Core CLR since it works in Windows PowerShell.

[TylerLeonhardt]

Is this something we should open on .NET Core CLR?

[SteveL-MSFT]

@tylerl0706 need more data to provide to dotnetcore team before we hand it off to them

[powercode]

I don't think we can depend on a finalizer being run. We should invoke dispose explicitly if we depend on it to run. There are quite a few cases where finalizers are not run - see https://blogs.msdn.microsoft.com/oldnewthing/20100809-00/?p=13203/ and
https://stackoverflow.com/questions/3458177/are-net-finalizers-always-executed.

[PaulHigin]

Sure, finalizers sometimes don't run due to various reasons, but in this case the finalizer never runs. Also this cleanup has relied on finalizers for years without any problems. So this appears to be a CLR regression.

We can (and do) do some cleanup during exit, such as closing remote runspaces and disconnect running remote jobs. But this cleanup is not guaranteed to run in all cases same as the finalizer thread.

[PaulHigin]

Unless you are suggesting that there is a bug in PowerShell that is causing the finalizer thread not to run (such as throwing an unhandled exception on the thread). That would be a regression on our part. But with my experimentation I don't see any unhandled exceptions. It looks like the finalizer is just not getting called.

[powercode]

I'm saying that it is brittle. Any type a user loads with a misbehaving finalizer may cause this to happen.
If we have control of these objects, (and the PipelineThread seems like it should fall in this category) we should also be able to write infrastructure to dispose of them.

[PaulHigin]

I understand. And we have that infrastructure on exit. My point is that anything we write is just as susceptible to catastrophic failures.

[anmenaga]

@PaulHigin I don't know... Yes, finalizer never getting called might be a CLR regression, but it feels like we should be able to set _workItemReady (e.g. by calling LocalPipeline:Dispose()) in our code without relying on the mercy of the garbage collector.

[PaulHigin]

@anmenaga, We really need to be able to depend on the CLR to work as advertised. We have other places in the code that relies on the finalizer to work. I'll investigate more and if this looks like a CLR regression I'll notify the team.

But for the short term I'll create a workaround to fix this particular hang problem for RC using our existing cleanup on exit code.

[daxian-dbw]

Close via #5356