Powershell: Executing commands that run for longer than one minute in a remote runspace (from a Linux client to a Windows server) causes the client to hang forever when OperationTimeout = 0

Created on 8 Apr 2019  路  4Comments  路  Source: PowerShell/PowerShell

Compile the following program against 6.2.0 or 6.1.3 (I've tried both) and run it on a Linux machine (I've been using an Docker image based on Ubuntu), with the appropriate arguments (e.g. mywindowsserver myusername mypassword).

```c#
public static void Main(string[] args)
{
var connectionInfo = new WSManConnectionInfo
{
ComputerName = args[0],
Credential = new PSCredential($@"{args[0]}{args[1]}", new NetworkCredential(string.Empty, args[2]).SecurePassword),
AuthenticationMechanism = AuthenticationMechanism.Negotiate,
OperationTimeout = 0,
IdleTimeout = (int)TimeSpan.FromMinutes(20).TotalMilliseconds
};
using (var runspace = RunspaceFactory.CreateRunspace(connectionInfo))
{
Console.WriteLine("Connecting...");
runspace.Open();
Console.WriteLine("Connected; running script...");

    using (var shell = PowerShell.Create())
    {
        shell.Runspace = runspace;
        shell.Commands.AddScript(@"

Write-Host ""Sleeping for 90 seconds...""
Sleep 90
Write-Host ""Done!""
");
shell.Invoke();
}
Console.WriteLine("Finished.");
}
}


The program should exit after 90 seconds, but instead it hangs forever (even if you reboot the server!). On Windows, the program does exit as expected after 90 seconds.

# Analysis
I've enabled and reviewed the OMI client logs, which show that the client socket is timing out after one minute:

DEBUG: null(0): _CreateConnectorSocket - OK exit. socket: 76, secure: 0, timeout: 01:00.000
ERROR: null(0): _RequestCallback - Timed out. socket: 76, result: 1004 (MI_RESULT_TIME_OUT)
```

That one minute timeout appears to be a default value hard coded in the OMI client:
https://github.com/Microsoft/omi/blob/4a0b6ebabdce442c3e526cd47fb326425aeed79a/Unix/http/httpclient_private.h#L24
https://github.com/Microsoft/omi/blob/049c361978731425549f35067ab25b0b14febd01/Unix/http/httpclient.c#L1868

The WSManConnectionInfo.OperationTimeout value is passed through various layers to the server, but does not appear to be used anywhere else (e.g. to configure client timeouts in libmi).

So there are likely at least two issues here:

  1. PowerShell does not correctly handle client timeouts when they do occur.
  2. PowerShell does not attempt to configure client timeouts in libmi.
Issue-Bug WG-Remoting

All 4 comments

@PaulHigin Could you please comment the Issue?

I've dug further into this and it appears the problem is a result of setting OperationTimeout = 0. libmi only sets its internal socket timeout if one is specified by the client; otherwise it uses its default of one minute. So I think this is still a bug (the PowerShell stack shouldn't hang in this situation), but it's a pretty specific edge case, and can be easily worked around by setting OperationTimeout to a non-zero value.

It looks like this has already been analyzed (thanks @danports !).

I know that when this OMI 'bridge' was written, many WSMan details were left out or not fully implemented and it looks like this is one of them.

I agree that the workaround is to just not set the OperationTimeout value to zero.

Related #8233

Was this page helpful?
0 / 5 - 0 ratings