Powershell: Async cmdlets

[felixfbecker]

I've recently tried to write a cmdlet that had to use async APIs. Doing so turned out to be incredibly difficult, and with help I found an AsyncCmdlet class that lets you override processRecordAsync etc. as async functions that get CancellationTokens:
https://github.com/felixfbecker/PSKubectl/blob/master/src/AsyncCmdlet.cs
https://github.com/felixfbecker/PSKubectl/blob/master/src/ThreadAffinitiveSynchronizationContext.cs

Could PowerShell provide this out of the box?

[BrucePay]

@felixfbecker So if I understand you correctly, you'd like to see an in-box variant of the PSCmdlet base class possibly called PSAsyncCmdlet that lets you write synchronous cmdlets that use asynchronous APIs and the await keyword including built-in cancellation support? (Note: cmdlets are always synchronous WRT the pipeline but may have asynchronous implementations). I'm currently working on some stuff to let you do write asynchronous task-based _scripts_ but I don't think this overlaps that. This is really an SDK request instead of an end-user feature - right?

[felixfbecker]

Yep

[jackbond]

I found this thread after looking at some .Result instances in our Powershell library. I'm actually amazed that PSAsyncCmdlet doesn't already exist, and that it isn't the default / recommended base class.

[michsco]

I just ran into the exact same thing. When I tried to implement a solution, it got pretty messy with Tasks blocking when awaiting.

[henning-krause]

I'd like to see this feature as well since more and more libraries are async-only...

[pcgeek86]

Just came across this issue today. Thanks for posting @felixfbecker.

[tig]

Ditto. In my case I am finding weird situations where the pipeline exits when I invoke a method with await.

[tig]

This seems to work well. I just submitted a PR to it to use dotnetcore.

https://github.com/ttrider/PowerShellAsync

Maybe this could be a starting point for official support in PS?

GitHub

ttrider/PowerShellAsync

create async cmdlets for PowerShell. Contribute to ttrider/PowerShellAsync development by creating an account on GitHub.

[tig]

This seems to work well. I just submitted a PR to it to use dotnetcore.

https://github.com/ttrider/PowerShellAsync

Maybe this could be a starting point for official support in PS?

GitHubttrider/PowerShellAsynccreate async cmdlets for PowerShell. Contribute to ttrider/PowerShellAsync development by creating an account on GitHub.

I take it back. I'm finding ttrider/PowerShellAsync actually doesn't work as expected. I'm not savvy enough to debug it. It's pretty easy to repro what I'm experiencing by doing something like this:

        protected override async Task EndProcessingAsync() {
            _cancellationToken = new CancellationTokenSource();
            var version = await ServiceLocator.Current.UpdateService.GetLatestStableVersionAsync(_cancellationToken.Token);
            Debug.WriteLine($"Version: {version}");
            var path = await ServiceLocator.Current.UpdateService.StartUpgradeAsync();
            Debug.WriteLine($"path: {path}");
            ShouldContinue("my message.", "Exit this Powershell instance?");
            Debug.WriteLine($"done.");
            await base.EndProcessingAsync();
        }

ShouldContinue wedges and "done" is never written to the debug console.

The two async methods I call are really just wrappers around WebClient, so nothing fancy.

GitHub

ttrider/PowerShellAsync

create async cmdlets for PowerShell. Contribute to ttrider/PowerShellAsync development by creating an account on GitHub.

[felixfbecker]

@tig are you seeing the same problem with https://github.com/felixfbecker/PSKubectl/blob/master/src/AsyncCmdlet.cs too? If not, maybe we could use that as a starting point instead.

GitHub

felixfbecker/PSKubectl

kubectl with the power of the object pipeline. Contribute to felixfbecker/PSKubectl development by creating an account on GitHub.

[tig]

@tig are you seeing the same problem with https://github.com/felixfbecker/PSKubectl/blob/master/src/AsyncCmdlet.cs too? If not, maybe we could use that as a starting point instead.

I'll take a look at.

For now, FWIW, a bunch of debugging last night gave me SOME insight into the problem with PowerShellAsync, but not enough for me to actually suggest how to fix it.

If I do this:

 var path = await ServiceLocator.Current.UpdateService.StartUpgradeAsync();
 ShouldContinue("my message.", "Exit this Powershell instance?");

The deadlock happens here:

https://github.com/ttrider/PowerShellAsync/blob/66eb99e33516532749cdd81ced3faf210361a9ee/PowerShellAsync/AsyncCmdlet.cs#L294

However, if I do this:

 var path = await ServiceLocator.Current.UpdateService.StartUpgradeAsync();
 await Task.Run(() => ShouldContinue("my message.", "Exit this Powershell instance?"));

That Wait doesn't block, but instead the task gets pulled off the queue and invoked via:

https://github.com/ttrider/PowerShellAsync/blob/66eb99e33516532749cdd81ced3faf210361a9ee/PowerShellAsync/AsyncCmdlet.cs#L200

I think there's probably something that could be done to how methods like ShouldContinue are wrapped by PowerShellAsync to fix this:

https://github.com/ttrider/PowerShellAsync/blob/66eb99e33516532749cdd81ced3faf210361a9ee/PowerShellAsync/AsyncCmdlet.cs#L117

For now, I'm moving forward by just wrapping CmdLet calls like ShouldContinue and WriteProgress which appear to not be correctlyl wrapped by PowerShellAsync with Task.Run calls.

[tig]

@felixfbecker I need help. I just can't get my head around this stuff enough to actually fix the problem. I'm currently using your version and things are mostly working. However, WriteDebug/Error etc... need to run on the main cmdlet thread and I'm not smart enough to modify your code to do so.

Not sure if this Issue is the right place for me to get educated. Please send me elsewhere if there's a better forum for this topic. I'm putting it here for now because a) you are here and b) this really is all about building async cmdlets....

To set context, I've built a Serilog sink for PowerShell CmdLets. My libraries use Serilog logging throughout for other clients (mainly a Winforms app) and I want to redirect Log.Debug -> WriteDebug, Log.Information -> WriteVerbose, etc... in my PowerShell cmdlet.

I discovered along the way that Serilog's singleton pattern makes running within a shared host like Powershell challenging, but I've worked around it via a registration model from my CmdLet.

See https://github.com/tig/winprint/blob/master/src/WinPrint.Console/PowerShellSink.cs

Emit is often called from threads other than the cmdlet thread. Thus I get exceptions like:

Exception thrown: 'System.Management.Automation.PSInvalidOperationException' in System.Management.Automation.dll
The WriteObject and WriteError methods cannot be called from outside the overrides of the BeginProcessing, ProcessRecord, and EndProcessing methods, and they can only be called from within the same thread. Validate that the cmdlet makes these calls correctly, or contact Microsoft Customer Support Services.

And WriteDebug just fails silently when on the wrong thread (apparently).

It seems the solution should be simple:

Wrap PSCmdLet.WriteDebug et. al. in SynchronizationContext.Post calls. If I'm understanding this right, the message pump in ThreadAffinitiveSynchronizationContext will pick these up and run them on the right thread.

I don't think the combination of your AsyncCmdlet and ThreadAffinitiveSynchronizationContext are fully baked because when I do this:

    public new void WriteDebug(string text) {

        ThreadAffinitiveSynchronizationContext.Current.Post(new SendOrPostCallback((o) => {
            base.WriteDebug(text);
        }), null);
    }

Current is null on those calls and I obviously need to prime the pump somehow, this is where I need a hint.

I won't feel too bad if y'all just tell me to go really study how to do threading right. I'm actually doing all this specifically to learn. So...

GitHub

tig/winprint

WinPrint 2.0 - A modern take on the the classic source code printing app from 1988. WinPrint is the perfect tool for printing source code, web pages, reports generated by legacy systems, documenta...