Powershell: C# TaskCompletionSource is null when used as part of a PSObject InputObject to Start-ThreadJob

[iUnknwn]

Steps to reproduce

$ErrorActionPreference = "Stop"

# launch process with the appropriate args. 
$p = [System.Diagnostics.Process]::new() 
$p.StartInfo.FileName = "notepad.exe" 
$p.StartInfo.Arguments = $null 
$p.EnableRaisingEvents = $true 
$p.Start()

# create a task completion source
$tcs = [System.Threading.Tasks.TaskCompletionSource[bool]]::new()

# bundle the process object and the task completion source into 
# a single input object
$inputObj = New-Object psobject -Property @{tcs = $tcs; p = $p}

$job = Start-ThreadJob -InputObject $inputObj -ScriptBlock {
    $input.p.WaitForExit()
    $input.tcs.SetResult($true)
}

# wait for the program to exit 
$processTask = $tcs.Task

# this deadlocks
$processTask.Wait()

Expected behavior

The task completion event to fire after the program closes.

Actual behavior

The following error is visible if you close the program and read the output from the job:

You cannot call a method on a null-valued expression.
At line:3 char:5
+     $input.tcs.SetResult($true)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull

Environment data

Name                           Value
----                           -----
PSVersion                      6.2.3
PSEdition                      Core
GitCommitId                    6.2.3
OS                             Microsoft Windows 10.0.18363
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

As a side note, passing the task completion source directly (not wrapping it in an object), does appear to work (though then you can't wait on the process).

[iSazonov]

You should use ArgumentList to send arguments

[SeeminglyScience]

@iUnknwn $input is an enumerator. After the first time you use it the enumeration is completed so when you call it the second time it is empty. Use $_ or $PSItem instead, or the ArgumentList parameter as @iSazonov suggested.

If you really want to stick with $input you could call $input.Reset() between uses, but the other options will better serve your use case.

[iUnknwn]

This looks like a potential documentation bug for Start-ThreadJob then, since ArgumentList clearly says it's only to be used with a script file (not a script block). I'll open a new issue.

Manually manipulating the iterator works, as does using ArgumentList. However, it doesn't look like $PSItem is populated in the script block. Is this expected?

$job = Start-ThreadJob -InputObject $inputObj -ScriptBlock {
    echo $PSItem #outputs nothing
    $PSItem.p.WaitForExit() #fails with null valued expression
    $PSItem.tcs.SetResult($true) #fails with null valued expression.
}

[iSazonov]

However, it doesn't look like $PSItem is populated in the script block. Is this expected?

@PaulHigin Is it by-design?

[mklement0]

@iSazonov, I think it is; the documentation even states for the -InputObject parameter:

In the value of the ScriptBlock parameter, use the $Input automatic variable to represent the input objects.

While something like 1, 2 | Start-ThreadJob { ... } (implicit -InputObject use) _looks like_ a per-input-object script-block-invocation scenario such as 1, 2 | ForEach-Object { $_ + 1 } ($_ being short for $PSItem), it isn't:

In the Start-ThreadJob scenario (as with Start-Job, the CLI, and Invoke-Command - with or without remoting), the script block is only invoked _once_, in a different runspace (which is in a different thread in the same process for Start-ThreadJob, and in a different process for the other scenarios); it is like invoking the script block with & without input from a pipeline in the target runspace, and in a stand-alone invocation such as & { $PSItem } $PSItem isn't defined.

Instead, such an invoked-once script block needs to use a _nested_ pipeline with explicit use of $Input in order to access and enumerate the input objects (or move through the iterator manually, but that's obviously cumbersome).

To adapt the example above:

PS> 1, 2 | Start-ThreadJob { $input | ForEach-Object { $_ + 1 } } | Receive-Job -Wait -AutoRemove
3
4

@SeeminglyScience: Curiously, calling $input.Reset() in these scenarios _fails_, unlike with intra-runspace use of $input; in the cross-runspace scenarios, the type of $input seems to change from System.Collections.ArrayList+ArrayListEnumeratorSimple to System.Management.Automation.Runspaces.PipelineReader<object> (the object returned from the internal .GetReadEnumerator() method).

This smells like a bug, right?

Repro (Windows, using "loopback remoting"):

1, 2 | Invoke-Command -ComputerName . { $input; $input.reset() }
1
2
Exception calling "Reset" with "0" argument(s): "Specified method is not supported."

[iSazonov]

I think it is;

@mklement0 In the case PowerShell shouldn't be silent and throw on $PSItem, yes?

[mklement0]

Just like with local use, it does complain, but only with Set-StrictMode -Version 1 or higher (note that, curiously, $PSItem is translated to $_ in the error message):

PS> 1, 2 | Invoke-Command { Set-StrictMode -Version 1; $PSItem } -ComputerName .
The variable '$_' cannot be retrieved because it has not been set

---

Taking a step back:

While documented, the current behavior is both non-obvious and cumbersome.

Piping objects in job / remoting scenarios isn't common, but perhaps that's owed to the above.

Especially in the context of the CLI the behavior is unexpected, as I've pointed out in #9497

However, changing the behavior for script blocks passed to jobs and remoting commands to per-input-object invocation with $_ / $PSItem bound would be a breaking change, because $Input would then turn into an effective alias of $_ instead of representing _all_ input:

# Currently: 1 invocation, explicit use of $input, which represents all input.
PS>  1, 2 | Invoke-Command -ComputerName . { $input | % { '!' + $_ } }
!1
!2

If we changed to per-input-object execution, we could more conveniently write:

# With the change: 2 invocations, $_ usable.
PS>  1, 2 | Invoke-Command -ComputerName . { '!' + $_ }
!1
!2

However, the semantics of $input would change:

# Currently: 1 invocation, explicit use of $input, which represents all input.
PS>  1, 2 | Invoke-Command -ComputerName . { $input | % { '!' + $_ }; 'after' }
!1
!2
after

With per-input-object execution, you'd get:

# With the change: 2 invocations, and $input is now the same as $_
PS>  1, 2 | Invoke-Command -ComputerName . { $input | % { '!' + $_ }; 'after' }
!1
after
!2
after

[msftbot[bot]]

This issue has been marked as answered and has not had any activity for 1 day. It has been closed for housekeeping purposes.