Powershell: Start-Job needs better discovery/support for pwsh.exe (and possibly powershell.exe)

[KirkMunro]

Summary of the new feature/enhancement

As a scripter,
I want to be able to use Start-Job from within a C# application that hosts PowerShell
so that I can start background jobs easily when it is appropriate to do so.

As a scripter,
I want to be able to use Start-Job to start a background job in a specific version of PowerShell
so that I can control the type of background job that is run.

Proposed technical implementation details (optional)

Today if you invoke Start-Job from a C# application that hosts PowerShell, you may be presented with an error message that states the following:

   System.Management.Automation.CmdletInvocationException : The pwsh executable cannot be found at "C:\myApplicationPath\pwsh.exe".
Note that 'Start-Job' is not supported by design in scenarios where PowerShell is being hosted in other applications. Instead, usage of the 'ThreadJob' module is recommended in such scenarios.

This "by design" limitation is necessary because there are scenarios where you simply do not have access to the PowerShell executable that is required to launch a background job; however, the error message is misleading, because it can occur in in environments where you do have full access to the PowerShell executable that is required to launch a background job. In those scenarios, the error shows up simply because the executable is not in the same location as the process that is running, and the logic internal to Start-Job isn't looking for it elsewhere.

To correct this, the following enhancements should be considered for Start-Job:

1. If Start-Job is invoked from an application other than pwsh.exe, it should internally look for pwsh.exe using command discovery. If it finds the executable, then it should start the job using that executable.
2. Start-Job should have a -PSExecutable parameter of type string that accepts a path to a PowerShell executable. If this parameter is supplied as part of the invocation of Start-Job, Start-Job will attempt to start the job using that executable in place of anything else.

~In addition, Start-Job has two parameters that are a parameter that is no longer supported today: -RunAs32 and -PSVersion. Both of those apply for Windows PowerShell only. Someone (@SteveL-MSFT or the committee) needs to decide if these need to be kept around (they only produce errors right now) or if they can be removed. I'd prefer them be removed -- better to catch use of them early and report errors that clearly indicate they are not supported than to leave them kicking around.~

Update that reflects what @SteveL-MSFT explained below:
In addition, Start-Job has a -RunAs32 parameter that is no longer supported today. -RunAs32 would be supported by providing a path to the the 32-bit version of the executable in the new -PSExecutable parameter, so -RunAs32 could be deprecated as part of that change.

[SteveL-MSFT]

For discovering pwsh executable, it can just rely on command discovery. If it's already cached, it'll be pretty quick. -RunAs32 usage would be replaced by specifying a path to the x86 exe. -PSVersion actually works. You can use Start-Job from PS7 and specify -PSVersion 5.1 to have the job start with powershell.exe.

[daxian-dbw]

When Start-Job requires pwsh to work, I think it should check for the version of pwsh as part of the command discovery, and only automatically works when the version matches the powershell sdk used in the hosting application. Otherwise, it should fail and require -PSExecutable to be explicitly specified. This will make the script execution more predictable.

[ALIENQuake]

@KirkMunro Can you tell a bit more about the 'deployment context' of the application that hosts PowerShell specific version? Are you deploying the application into the Windows machines which do not have Powershell 7 or even 5.1 (win 7, 8, 2008, 2012)?

[jmiller76]

I wanted to do something like:
Start-Job { [System.Environment]::Is64BitProcess } -Runas32 -PSVersion 5.1
from 7.0.x and Id expect it to see 5.1 and RunAs32 and start the SysWow64 PowerShell. Instead it tells me that I don't have 32bit pwsh 7 installed

Is this really the same "Find the right, exe" with a different use case or would this be considered a different issue, as the two switch behave counterintuitive.

Id happily, specify the 32bit PowerShell 5.1 as the workaround.

[KirkMunro]

@ALIENQuake: Cloud-based invocations. CI/CD using Pester in Azure DevOps. Places where you do not directly manage the infrastructure where you are running.

[KirkMunro]

When Start-Job requires pwsh to work, I think it should check for the version of pwsh as part of the command discovery, and only automatically works when the version matches the powershell sdk used in the hosting application. Otherwise, it should fail and require -PSExecutable to be explicitly specified. This will make the script execution more predictable.

@daxian-dbw: Can you explain this a little bit more? If I build an application using https://www.nuget.org/packages/Microsoft.PowerShell.SDK/7.0.3, and then later that application is installed on a system that has PowerShell 7.1 installed, are you indicating that Start-Job should fail and require -PSExecutable in that scenario? If so, why? I can understand for downlevel versions, but uplevel versions should be backwards compatible, so I would expect that to just work.

Microsoft.PowerShell.SDK 7.0.3

Runtime for hosting PowerShell