Terminal: Job control and the Console API

[al45tair]

The Console API is presently lacking any notion of the foreground process/process group; that is, there is no equivalent to the UNIX tcsetpgrp() call, and indeed the API documentation notes that in the presence of multiple processes there is no guarantee which process will receive input read from the console.

This is really quite a serious bug — it prevents the use of any kind of job control from the shell, because the shell has no way to cause a process to stop if it tries to read from the attached console. Yes, it's possible that a shell could create pseudo consoles for every subprocess and attempt to manage this itself, but that doesn't work if the subprocesses themselves spawn subprocesses.

Please take a look at https://ftp.gnu.org/old-gnu/Manuals/glibc/html_node/Access-to-the-Terminal.html#Access%20to%20the%20Terminal which describes the UNIX behaviour; you really want to make sure that it's possible to do similar things in the Windows API. In particular, it must be possible to send a console control message to a process if it tries to read from a console for which it is not the foreground process or in the foreground process group; the default behaviour for the message, if unhandled, should be to suspend the process. It must also be possible, optionally, depending on the console configuration, to cause output to behave the same way.

I'd suggest adding something like

BOOL ConsoleSetForegroundProcessGroup(HANDLE hConsole, HANDLE hProcessGroup);
HANDLE ConsoleGetForegroundProcessGroup(HANDLE hConsole);

Then I'd probably add an extra console mode flag to control the suspension of processes on output, maybe ENABLE_SUSPEND_ON_OUTPUT; you could also add ENABLE_SUSPEND_ON_INPUT and leave that off by default to maintain the present behaviour.

Finally, the console control events CTRL_SUSPEND_FOR_INPUT and CTRL_SUSPEND_FOR_OUTPUT need adding. It would also be a good idea to add, while you're about it, CTRL_SUSPEND, the default behaviour of which should be to suspend the process unless it's handled.

[oising]

Interestingly there's a pseudo job control infrastructure in PowerShell (and pwsh) that allows the same unix suffix of & to create background jobs, but it's not quite the same. PowerShell commands (cmdlets) are in-process, though it can still exec out-of-process windows executables and background them in the same way. PowerShell it uses its own background jobs infrastructure and jobs (inproc or not) are never really suspended; instead their output is buffered, ready for collection with the receive-job cmdlet.

That said, cmd has nothing. But really, is it such a big deal for cmd? It's effectively deprecated in favour of pwsh/powershell. And for WSL, you've already got job suspension with fg, bg, jobs and & because, well, it _is_ unix.

So, do we really need this? I agree that there's a bug of sorts with the non-deterministic way that multiple processes bound the same stdin currently behave, but generally speaking, it's not something that is a common paradigm in Windows.

[PhMajerus]

I don't think this is necessary, a Windows CUI app that launches another CUI process knows that it can access the same console. To me, in the Win32 world, it is an app's responsibility to launch a CUI process sharing the console, or providing pipes to process its stdio. The ConPTY now makes it possible to redirect even I/Os from processes that tries to use the console API to take over the console, so a shell can easily provide the background tasks facility in a reliable way.

[al45tair]

@PhMajerus It's certainly true at present, but I don't think the fact that that is currently the case is an argument against making things better. And I think that the features I mentioned above would go a long way towards that goal — I don't think they'd be that complicated to implement, either, and since they'd be off by default, unless the user was using a job control shell that enabled them, I can't see much downside.

@oising One of my main motivations for asking for this support is that it would allow us to fix Cygwin, Msys, Git for Windows et al, such that there is reasonable behaviour in cases that are presently really very broken. Right now, Cygwin's DLL has to emulate a lot of this functionality for Cygwin processes, but as a result it has no way to interact properly with Win32 console mode programs, the result of which is a lot of unpleasant hackery (winpty, for instance), and even then it's very easy to get things to hang or misbehave. Even with pseudo consoles, this problem won't go away because the UNIX functionality is still needed by Cygwin et al, and simply isn't present on the Win32 side - though at least pseudo consoles get us away from having to scrape a Console for I/O.

[oising]

@al45tair In this day and age of WSL, why would people still use Cygwin? Whatever effort it might take to add this feature, it's several orders of magnitude more than porting a few scripts from cygwin/msys to WSL: A native, full-fidelity Linux. And with a de-facto kernel coming to Windows in June, I really don't see how and why anyone within Microsoft would spend money on this.

[al45tair]

@oising Plenty of reasons; Cygwin, Msys, Mingw et al let your programs have a UNIX-like base but can still access the Win32 API (unlike WSL). Also, Git for Windows, the official version of Git on Windows, is based on Msys2. And, for better or worse, lots of us have build systems built on top of Cygwin; moving those to run via WSL is not necessarily a trivial exercise.

Plus there's the issue of those of us who prefer Bourne shell and a set of Unix-like utilities to Powershell or the somewhat less powerful cmd, but are mainly running Win32 software rather than wanting specifically to run Linux stuff.

Oh, and for some people, using WSL potentially requires buy-in from IT and management, who might object to it being installed on machines; we've already fought those battles to get Cygwin/Msys/Mingw.

So, you've got a point, but I still think this is worth doing.

[oising]

@al45tair I get you, absolutely, and sympathize. Regardless, Cygwin et al grew to fill a void and were always stopgap solutions in my mind, albeit impressive ones. With WSL, there's no more gap and these things are in essence, obviated. People will move to WSL over time, and will be glad of it when they find they can run and/or compile scripts/code directly on Windows 10 instead of having to dual boot or rely on gotcha-laden stopgap tooling. If you prefer gnu utils and want to use win32 you can mix and match your executables in a single pipeline, e.g. cat c:\windows\system32\log.txt | wsl grep -i foo -- the LHS runs on win32, the RHS of the pipe runs on WSL. Pretty sick, no?

[al45tair]

@oising :-) Oh, I quite agree, WSL is clearly a great thing — I'm certainly looking forward to the next version.

[Congee]

In cmd.exe you can use mouse to select some area to suspend the process running in foreground, and then you can press Esc key to resume the job. I am surprised it is not possible in the new Terminal.

[oising]

In cmd.exe you can use mouse to select some area to suspend the process running in foreground, and then you can press Esc key to resume the job. I am surprised it is not possible in the new Terminal.

It was also possible with the legacy console to use ctrl+q and ctrl+s (XOFF / XON - https://vt100.net/docs/vt100-ug/chapter3.html ) but sadly this no longer works in the "new" console, nor in windows terminal.