Powershell: PSInvalidOperationException: There is no Runspace available to run scripts in this thread

[ALIENQuake]

Hi,

I have very simple Windows Forms Powershell code, it works perfectly for PS 5.1

PS 7 is using NET Core 3.x so I can also use the Forms framework. But the code below gives me an error:

Steps to reproduce

Paste below code into PS 7.x, click 'Run' button.

[reflection.assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
[reflection.assembly]::LoadWithPartialName("System.Drawing") | Out-Null

[System.Windows.Forms.Application]::EnableVisualStyles()
$form = New-Object 'System.Windows.Forms.Form'
$buttonRunProcess = New-Object 'System.Windows.Forms.Button'
$richtextboxOutput = New-Object 'System.Windows.Forms.RichTextBox'
$buttonExit = New-Object 'System.Windows.Forms.Button'
$InitialFormWindowState = New-Object 'System.Windows.Forms.FormWindowState'

$buttonExit_Click={
    $form.Close()
}

$buttonRunProcess_Click = {
    $buttonRunProcess.Enabled = $false

    $richtextboxOutput.Clear()

    $process = New-Object System.Diagnostics.Process
    $process.StartInfo.FileName = 'ping.exe'
    $process.StartInfo.Arguments = 'google.com'
    $process.StartInfo.UseShellExecute = $false
    $process.StartInfo.CreateNoWindow = $true
    $process.StartInfo.RedirectStandardInput = $false
    $process.StartInfo.RedirectStandardOutput = $true
    $process.EnableRaisingEvents = $true
    $process.SynchronizingObject = $buttonRunProcess
    $process.add_OutputDataReceived( {
        # Use $_.Data to access the output text
        $richtextboxOutput.AppendText($_.Data)
        $richtextboxOutput.AppendText("`r`n") }
)

    $process.Start() | Out-Null
    $process.BeginOutputReadLine()
}


$Form_StateCorrection_Load=
{
    #Correct the initial state of the form to prevent the .Net maximized form issue
    $form.WindowState = $InitialFormWindowState
}

$Form_Cleanup_FormClosed=
{
    try
    {
        $buttonRunProcess.remove_Click($buttonRunProcess_Click)
        $buttonExit.remove_Click($buttonExit_Click)
        $form.remove_FormClosed($processTracker_FormClosed)
        $form.remove_Load($Form_StateCorrection_Load)
        $form.remove_FormClosed($Form_Cleanup_FormClosed)
    }
    catch { Out-Null <# Prevent PSScriptAnalyzer warning #> }
}

$form.SuspendLayout()

$form.Controls.Add($buttonRunProcess)
$form.Controls.Add($richtextboxOutput)
$form.Controls.Add($buttonExit)
$form.ClientSize = [System.Drawing.Size]::new(584, 362)
$form.Margin = '4, 4, 4, 4'
$form.MinimumSize = [System.Drawing.Size]::new(304, 315)
$form.Name = 'Redirect Process Output'
$form.StartPosition = 'CenterScreen'
$form.Text = 'Redirect Process Output'

$buttonRunProcess.Anchor = 'Bottom, Left'
$buttonRunProcess.Location = [System.Drawing.Point]::new(12, 327)
$buttonRunProcess.Name = 'buttonRunProcess'
$buttonRunProcess.Size = [System.Drawing.Size]::new(75, 23)
$buttonRunProcess.TabIndex = 0
$buttonRunProcess.Text = 'Run'
$buttonRunProcess.UseCompatibleTextRendering = $True
$buttonRunProcess.UseVisualStyleBackColor = $True
$buttonRunProcess.add_Click($buttonRunProcess_Click)

$richtextboxOutput.Anchor = 'Top, Bottom, Left, Right'
$richtextboxOutput.HideSelection = $False
$richtextboxOutput.Location = [System.Drawing.Point]::new(12, 12)
$richtextboxOutput.Name = 'richtextboxOutput'
$richtextboxOutput.ReadOnly = $True
$richtextboxOutput.Size = [System.Drawing.Size]::new(559, 305)
$richtextboxOutput.TabIndex = 6
$richtextboxOutput.Text = ''
$richtextboxOutput.WordWrap = $False

$buttonExit.Anchor = 'Bottom, Right'
$buttonExit.Location = [System.Drawing.Point]::new(497, 327)
$buttonExit.Name = 'buttonExit'
$buttonExit.Size =  [System.Drawing.Size]::new(75, 23)
$buttonExit.TabIndex = 2
$buttonExit.Text = 'E&xit'
$buttonExit.UseCompatibleTextRendering = $True
$buttonExit.UseVisualStyleBackColor = $True
$buttonExit.add_Click($buttonExit_Click)

$form.ResumeLayout()

$InitialFormWindowState = $form.WindowState
$form.add_Load($Form_StateCorrection_Load)
$form.add_FormClosed($Form_Cleanup_FormClosed)
$form.ShowDialog()

Expected behavior

After clicking the "Run" button the output from ping google.com is displayed inside the text box.

Actual behavior

No output is displayed and Powershell crash with

Unhandled exception. System.Management.Automation.PSInvalidOperationException: There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was:
        # Use $_.AppendText("`r`n")
   at System.Management.Automation.ScriptBlock.GetContextFromTLS()
   at System.Management.Automation.ScriptBlock.InvokeAsDelegateHelper(Object dollarUnder, Object dollarThis, Object[] args)
   at lambda_method(Closure , Object , DataReceivedEventArgs )
   at System.Diagnostics.Process.OutputReadNotifyUser(String data)
   at System.Diagnostics.AsyncStreamReader.FlushMessageQueue(Boolean rethrowInNewThread)
--- End of stack trace from previous location where exception was thrown ---
   at System.Diagnostics.AsyncStreamReader.<>c.<FlushMessageQueue>b__18_0(Object edi)
   at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi)
   at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[TState](ExecutionContext executionContext, Action`1 callback, TState& state)
   at System.Threading.QueueUserWorkItemCallback.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0-rc.2
PSEdition                      Core
GitCommitId                    7.0.0-rc.2
OS                             Microsoft Windows 10.0.18362
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[iSazonov]

From debug build we can see a line of the throw:

An error has occurred that was not properly handled. Additional information is shown below. The PowerShell process will exit.
Unhandled exception. System.Management.Automation.PSInvalidOperationException: There is no Runspace available to run scripts in this thread. You can provide one in the DefaultRunspace property of the System.Management.Automation.Runspaces.Runspace type. The script block you attempted to invoke was:
        # Use $_.D.AppendText("`r`n")
   at System.Management.Automation.ScriptBlock.GetContextFromTLS() in C:\Users\1\Documents\GitHub\iSazonov\PowerShell\src\System.Management.Automation\engine\lang\scriptblock.cs:line 840
   at System.Management.Automation.ScriptBlock.InvokeAsDelegateHelper(Object dollarUnder, Object dollarThis, Object[] args) in C:\Users\1\Documents\GitHub\iSazonov\PowerShell\src\System.Management.Automation\engine\lang\scriptblock.cs:line 796
   at lambda_method(Closure , Object , DataReceivedEventArgs )
   at System.Diagnostics.Process.OutputReadNotifyUser(String data)
   at System.Diagnostics.AsyncStreamReader.FlushMessageQueue(Boolean rethrowInNewThread)
--- End of stack trace from previous location where exception was thrown ---
   at System.Diagnostics.AsyncStreamReader.<>c.<FlushMessageQueue>b__18_0(Object edi)
   at System.Threading.QueueUserWorkItemCallback.<>c.<.cctor>b__6_0(QueueUserWorkItemCallback quwi)
   at System.Threading.ExecutionContext.RunForThreadPoolUnsafe[TState](ExecutionContext executionContext, Action`1 callback, TState& state)
   at System.Threading.QueueUserWorkItemCallback.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()

/cc @daxian-dbw @PaulHigin Could you please look the issue?

[PaulHigin]

PowerShell scriptblocks have runspace affinity, and runspaces are associated with threads. This error is occurring because the callback scriptblock is created on one runspace, but is being invoked on a random pool thread that has no associated runspace ('There is no runspace available...').

In general, scriptblocks should be run in the same thread/runspace in which they were created.
This kind of script can be problematic because Windows Forms can make call backs (to script blocks) on random pool threads, causing threading issues.

[iSazonov]

@PaulHigin Thanks! It is still not clear why the example works in Windows PowerShell but not in PowerShell Core.

Update: Windows PowerShell crash too.

[SeeminglyScience]

This kind of script can be problematic because Windows Forms can make call backs (to script blocks) on random pool threads, causing threading issues.

~It's worth noting that the use of Forms is actually incidental here. The change appears to be due to Process using an async stream for it's output reading. This is enough to repro:~ Edit: Nevermind, the SynchronizingObject was what made it work.

$process = New-Object System.Diagnostics.Process
$process.StartInfo.FileName = 'ping.exe'
$process.StartInfo.Arguments = 'google.com'
$process.StartInfo.UseShellExecute = $false
$process.StartInfo.CreateNoWindow = $true
$process.StartInfo.RedirectStandardInput = $false
$process.StartInfo.RedirectStandardOutput = $true
$process.EnableRaisingEvents = $true
$process.SynchronizingObject = $buttonRunProcess
$process.add_OutputDataReceived{
    # Use $_.Data to access the output text
    $richtextboxOutput.AppendText($_.Data)
    $richtextboxOutput.AppendText("`r`n")
}

$process.Start() | Out-Null
$process.BeginOutputReadLine()

[iSazonov]

Windows PowerShell crash too.

[daxian-dbw]

The script runs fine in Windows PowerShell on my machine.

In Windows PowerShell, the OutputDataReceived event handler is invoked on the pipeline thread because the synchronization context created by the form is used when calling the handler in Process.OutputReadNotifyUser.

However, in PowerShell 7, Process.OutputReadNotifyUser doesn't respect the SynchronizingObject anymore. The callback will eventually be invoked in AsyncStreamReader.ReadBufferAsync, after await _stream.ReadAsync(...).ConfigureAwait(false). Be noted about the ConfigureAwait(false) part, it means everything after this will be continued on a thread pool thread. Given that Process.OutputReadNotifyUser doesn't respect the sync context anymore, the delegate created from the script block will just run on a thread pool thread without a default Runspace, and hence the exception.

So, conclusion: I think this is a .NET Core issue, and will report it there.

[ALIENQuake]

@daxian-dbw That was a fantastic debugging! This issue actually affects all admin-gui-like tools which would like to use PS7. Thank you for the quick reaction. I will also comment on the net issue.

[daxian-dbw]

The .NET Core issue was opened: https://github.com/dotnet/runtime/issues/2024.
I hope they can port the fix to the subsequent servicing release of 3.1, so our servicing release can pick up the fix.

[SeeminglyScience]

@ALIENQuake In the mean time, here's a workaround:

using namespace System.Linq.Expressions

[reflection.assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
[reflection.assembly]::LoadWithPartialName("System.Drawing") | Out-Null

[System.Windows.Forms.Application]::EnableVisualStyles()
$form = New-Object 'System.Windows.Forms.Form'
$buttonRunProcess = New-Object 'System.Windows.Forms.Button'
$richtextboxOutput = New-Object 'System.Windows.Forms.RichTextBox'
$buttonExit = New-Object 'System.Windows.Forms.Button'
$InitialFormWindowState = New-Object 'System.Windows.Forms.FormWindowState'

$buttonExit_Click={
    $form.Close()
}

$buttonRunProcess_Click = {
    $buttonRunProcess.Enabled = $false

    $richtextboxOutput.Clear()

    $process = New-Object System.Diagnostics.Process
    $process.StartInfo.FileName = 'ping.exe'
    $process.StartInfo.Arguments = 'google.com'
    $process.StartInfo.UseShellExecute = $false
    $process.StartInfo.CreateNoWindow = $true
    $process.StartInfo.RedirectStandardInput = $false
    $process.StartInfo.RedirectStandardOutput = $true
    $process.EnableRaisingEvents = $true
    $process.SynchronizingObject = $buttonRunProcess

    $sender = [Expression]::Parameter([object])
    $e = [Expression]::Parameter([System.Diagnostics.DataReceivedEventArgs])
    $syncObject = [Expression]::Constant($buttonRunProcess)
    $beginInvoke = [System.ComponentModel.ISynchronizeInvoke].GetMethod(
        'BeginInvoke',
        [System.Reflection.BindingFlags]'Public, Instance',
        $null,
        [type[]]([Delegate], [object[]]),
        $null)

    $sbDelegate = [System.Diagnostics.DataReceivedEventHandler]{
        # Use $_.Data to access the output text
        $richtextboxOutput.AppendText($_.Data)
        $richtextboxOutput.AppendText("`r`n")
    }

    $delegate = [Expression]::Lambda(
        [System.Diagnostics.DataReceivedEventHandler],
        [Expression]::Call(
            $syncObject,
            $beginInvoke,
            [Expression]::Constant($sbDelegate),
            [Expression]::NewArrayInit(
                [object],
                $sender,
                $e)),
        [ParameterExpression[]]($sender, $e)).
        Compile()

    $process.add_OutputDataReceived($delegate)
    $process.Start() | Out-Null
    $process.BeginOutputReadLine()
}


$Form_StateCorrection_Load=
{
    #Correct the initial state of the form to prevent the .Net maximized form issue
    $form.WindowState = $InitialFormWindowState
}

$Form_Cleanup_FormClosed=
{
    try
    {
        $buttonRunProcess.remove_Click($buttonRunProcess_Click)
        $buttonExit.remove_Click($buttonExit_Click)
        $form.remove_FormClosed($processTracker_FormClosed)
        $form.remove_Load($Form_StateCorrection_Load)
        $form.remove_FormClosed($Form_Cleanup_FormClosed)
    }
    catch { Out-Null <# Prevent PSScriptAnalyzer warning #> }
}

$form.SuspendLayout()

$form.Controls.Add($buttonRunProcess)
$form.Controls.Add($richtextboxOutput)
$form.Controls.Add($buttonExit)
$form.ClientSize = [System.Drawing.Size]::new(584, 362)
$form.Margin = '4, 4, 4, 4'
$form.MinimumSize = [System.Drawing.Size]::new(304, 315)
$form.Name = 'Redirect Process Output'
$form.StartPosition = 'CenterScreen'
$form.Text = 'Redirect Process Output'

$buttonRunProcess.Anchor = 'Bottom, Left'
$buttonRunProcess.Location = [System.Drawing.Point]::new(12, 327)
$buttonRunProcess.Name = 'buttonRunProcess'
$buttonRunProcess.Size = [System.Drawing.Size]::new(75, 23)
$buttonRunProcess.TabIndex = 0
$buttonRunProcess.Text = 'Run'
$buttonRunProcess.UseCompatibleTextRendering = $True
$buttonRunProcess.UseVisualStyleBackColor = $True
$buttonRunProcess.add_Click($buttonRunProcess_Click)

$richtextboxOutput.Anchor = 'Top, Bottom, Left, Right'
$richtextboxOutput.HideSelection = $False
$richtextboxOutput.Location = [System.Drawing.Point]::new(12, 12)
$richtextboxOutput.Name = 'richtextboxOutput'
$richtextboxOutput.ReadOnly = $True
$richtextboxOutput.Size = [System.Drawing.Size]::new(559, 305)
$richtextboxOutput.TabIndex = 6
$richtextboxOutput.Text = ''
$richtextboxOutput.WordWrap = $False

$buttonExit.Anchor = 'Bottom, Right'
$buttonExit.Location = [System.Drawing.Point]::new(497, 327)
$buttonExit.Name = 'buttonExit'
$buttonExit.Size =  [System.Drawing.Size]::new(75, 23)
$buttonExit.TabIndex = 2
$buttonExit.Text = 'E&xit'
$buttonExit.UseCompatibleTextRendering = $True
$buttonExit.UseVisualStyleBackColor = $True
$buttonExit.add_Click($buttonExit_Click)

$form.ResumeLayout()

$InitialFormWindowState = $form.WindowState
$form.add_Load($Form_StateCorrection_Load)
$form.add_FormClosed($Form_Cleanup_FormClosed)
$form.ShowDialog()

[ALIENQuake]

@SeeminglyScience Och my och my! That's some hardcore stuff! Thank you!

[MaximoTrinidad]

Anyone knows if this issue is considered resolved!
:)

[ALIENQuake]

@MaximoTrinidad It won't be resolved until the github.com/dotnet/runtime/issues/2024 will be fixed.

[MaximoTrinidad]

Just an update on this matter!

I was able to fix this issue without the need of building a delegate function in PowerShell to handle to object.

Basically, during debugging I trap the following error:

Displaying general exception message:  + Exception calling "BeginOutputReadLine" with "0" argument(s): "Cannot mix synchronous and asynchronous operation on process stream."

The following link helps me identify what was wrong with the code:
https://social.technet.microsoft.com/Forums/fr-FR/597ee275-52eb-4aa6-81d9-7e62c3e55eff/powershell-systemdiagnosticsprocess-and-asynchronous-reads?forum=ITCG

It also clears the issue with the exception

Here's how to fix it:

1. In the button Click, don't include the control objects in the $process.add_OutputDataReceived().

## - Use below:
$process.add_OutputDataReceived($_.Data)

1. Don't use:

$process.BeginOutputReadLine()

This line cause the synch/asynch exception, and I think helps cause the "PSInvalidOperationException."

Instead, replace with the following:

$process.Start() | Out-Null
#  $process.BeginOutputReadLine()

## - Handles output results.
$processOutput = $process.StandardOutput.ReadToEnd()
if (![String]::IsNullOrEmpty($processOutput))
{
    $richtextboxOutput.AppendText($processOutput)
}

Make these changes and the code work without any exceptions.

Bottom line:
The PSInvalidOperationException issue can be cleared by fixing the code with no workarounds.

Powershell, System.Diagnostics.Process and asynchronous reads

[ALIENQuake]

@MaximoTrinidad Thanks for the investigation. Unfortunately, your method cannot be considered as a valid workaround because there is no 'real-time' output from the process, which is the essence of what the code is trying to do in the first place. It's a pity, though.

[ALIENQuake]

@derek-xia I'm monitoring this issue in order to confirm fix:
- according to this PR: https://github.com/dotnet/runtime/pull/37308, dotnet bug was fixed and included in NET 5 Preview 5
- I'm testing this issue using this daily build: https://dev.azure.com/powershell/PowerShell/_build/results?buildId=56718&view=artifacts&type=publishedArtifacts
- Powershell 7 uses "NET 5 preview 6" so the fix should be there

yet, I'm still getting the same error. I'm missing something?

Nvm, it appears to be fixed with the recent daily build. Also derek-xia sorry for wrong mention.

[iSazonov]

@ALIENQuake PowerShell daily is on .Net 5 Preview 6 (see global.json file in the root of the repository).

[ALIENQuake]

@iSazonov Thanks for pointing this, still, Net 5 Preview 6 has the necessary fix.

[ALIENQuake]

I noticed that I mention the wrong person when I did my initial tests. 🤦‍♂️

Lucky for me, I'm testing this issue using this daily build: https://dev.azure.com/powershell/PowerShell/_build/results?buildId=60097&view=artifacts&type=publishedArtifacts and it appears to be working! 😍

@daxian-dbw Many thanks for investigating and handling this issue! Thank you very much!