Powershell: Add operator to create a thread job from a command

[KirkMunro]

Summary of the new feature/enhancement

As a scripter,
I want an operator that allows me to create a thread job from a single command
so that I can run commands on threads more easily in PowerShell.

This request is similar to the need for the recently added "ampersand background operator" (&) that allows you to run a command in a background job, but instead of running a command in a separate process, this operator would run the command in a separate thread.

To differentiate this way of running commands from background jobs, a different sigil would be used for the operator at the end of a command. The sigil that is chosen must be something that the parser currently does not support so that it wouldn't break existing scripts.

Proposed technical implementation details (optional)

To be consistent with the existing ampersand background operator, a sigil of &~ could be used. This sigil results in parser errors in versions of PowerShell that don't support the ampersand background operator.

In PowerShell 6.2, it results in a command not found error, because PowerShell runs everything in the statement up to & as a background job and then looks to run everything after that operator as a subsequent command. Technically this means it would be a breaking change for anyone with a command called ~, but only if they invoke that command immediately following a & operator in a script.

At any rate, here is what invocation might look like with a new glyph:

# Run a command in a threadjob
$job = Get-Process -Id 12345678 2>&1 &~
# Get the results from the threadjob
$job | Receive-Job

[BrucePay]

@KirkMunro As I mentioned in the other issue, the handling of a trailing & is by design and matches Unix shell behaviour. As far as simple dispatch on a runspace or runspace pool, I always figured we'd do it with

& $runspacePool invoke-stuff -arg1 -arg2 abc

You do have to create the runspace or pool but that's not a huge deal.

[KirkMunro]

I see, so more like using the call operator with some thing where you want to run commands, like you can do with modules today (i.e. & (gmo SomeModule) invoke-stuff -arg1 -arg2 abc). Are there other "things" where you might want to invoke commands that would make it more appropriate to use this techinque?

Without thinking too hard about it, right now I like the simplicity of something like the & background operator better, and I also like the consistency between the two, allowing users to choose multi-process or multi-threaded job-based execution of statements by simply changing an operator. I think that will resonate well with the PowerShell community.

[rkeithhill]

Do you think thread job usage will be great enough to warrant dedicated syntax?

create a thread job from a single command

But you can do this today without a dedicated operator: $job = Start-ThreadJob { Get-Process -Id $pid }

To me, the primary advantage of & for regular jobs is it provides a known, comfortable mechanism for old *nix scripters like @BrucePay . 😜

[KirkMunro]

@rkeithhill I don't have usage data to answer that question.

My argument for this functionality comes more from some of the core tenets of PowerShell: consistency and discoverability. I understand how & came to PowerShell because of common use in bash, just like how && and || are coming to PowerShell. Now that it's here though you may have users who don't come from bash discovering and using it, and it seems natural (at least to me) for PowerShell to give them a consistent model that is easily discoverable to run a command in a thread job as well.

[KirkMunro]

Speaking of consistency, @BrucePay, there are several inconsistencies between the & background operator in PowerShell and the & control operator in bash that I'm curious about:

1. The former just returns a job object, and all output is hidden behind that object, while the latter returns something similar (the job id and process id), but then it also shows output in real-time in the current terminal. Was there any discussion about having & be more than a simplistic invocation of Start-Job, and instead having it set up event handlers on the data stream collections in the job object to capture data as it is added and output that data it in the current terminal (ideally with the job id in front of that data), to more closely mirror how the bash counterpart works? I've also read somewhere (but I can't find it now) about a &! operator which functions like the & control operator at the end of a command, but without showing the job streams in the terminal. This isn't a bash operator, but I like the distinction and the flexibility that having both would give scripters.

2. In bash, when you run a command in the background using the & control operator, the process ID is stored in $!. PowerShell should do something similar, shouldn't it? One challenge PowerShell has in this area is that $! is currently parsed as a command name when used by itself (What? That certainly seems like a parser bug). If $! was properly parsed as a variable, if it returned a job object, and if job objects in PowerShell included the process ID in their output (which they really should -- that's probably PR material), then I think we'd have better parity with bash while making it easier for scripters at the same time.

[vexx32]

$! being parsed as a command name isn't really a bug per se, I don't think. Most of those shorthand variables ($^, $$, etc) seem to be special-cased in the tokenizer, because some of the characters they use aren't normally valid variable name characters. (e.g., try entering $a$bb$$b and look at the multiple errors you get for your efforts).

As a result, if the character sequence isn't part of the defined exceptions and isn't able to be parsed as a regular variable name, they're treated like other bareword tokens and PS attempts to find command names that match as with all other generic bareword tokens.

But yeah, that would be pretty useful. Wouldn't take a lot to add an extra special variable, just an extra tokenizer case and follow the trail the other special variables have laid out there.

[mklement0]

Great suggestion, @KirkMunro: Given that the postpositional & is so much less ceremony than an explicit Start-Job call, having a something similarly concise syntax for _thread_ jobs (Start-ThreadJob) is desirable.

@rkeithhill:

Do you think thread job usage will be great enough

I wouldn't think about _current_ usage with respect to thread jobs, because I think that thread jobs merely have a PR problem / in-box-availability problem (in older PS versions):

If everyone knew that for most use cases they are _functionally equivalent_ to child-process-based background jobs _yet perform significantly better_ - while seamlessly integrating with the other *-Job cmdlets - I think thread jobs would see significantly more use - they certainly deserve to.