Sdk: Hiding process console with no parent console to inherit from

[idraper]

Dart SDK Version: Dart 2.7.0
OS: Windows x64

I ran into this issue using flutter but was referred to here since it seems like the issue is related to something with the dart:io library.

Flutter upgraded its windows embedding project to run from wWinMain instead of main, which does not spawn a console. This is obviously desirable for a GUI application such as flutter. However, the issue is that whenever the Process class is used from the dart:io library there is no console to inherit from, resulting in the cmd window popping up until the process is closed/completed.

I am wondering how I can spawn a process and hide the console so it doesn't appear. I understand it will have to create a console for itself, but surely there is some way to hide it? Here is a reference to the issue I created in the flutter repo. It has some example code demonstrating the problem as well as the discussion thus far, explaining why this appears to be a problem with Dart and not the flutter library.

Thanks!

[devoncarew]

@idraper, thanks for the report. From discussion on the related flutter issue (https://github.com/flutter/flutter/issues/47891#issuecomment-569814063), it seems like there may be a way to resolve this via modifying the flutter embedder for windows.

I'm triaging to the dart-io library for this repo, but please close the issue of you think there are no steps here from the Dart side; thanks!

[stuartmorgan]

it seems like there may be a way to resolve this via modifying the flutter embedder for windows

That's a hack though, requiring the embedder to create a console that it doesn't need. There are, IIUC, Win32 flags that can be passed to process creation to avoid this problem when spawning processes, and it seems like Dart should provide a way to get that behavior.

[idraper]

I was able to get a workaround with the suggestions of @stuartmorgan (see other thread). However, as he states above, this is a hacky sort of solution that requires creating another c++ project to give the child processes something to attach to and hiding that start-up window. It would be much simpler (from my perspective) to be able to pass flags to the Process class on startup to hide the console on spawn.

[a-siva]

/cc @ZichangG

[Sunbreak]

Any progress?

[stuartmorgan]

There are, IIUC, Win32 flags that can be passed to process creation to avoid this problem when spawning processes

Specifically, now that I'm more familiar with the Dart code involved here, what we want here is presumably something that can be provided at the Dart code level that will cause this code to pass CREATE_NO_WINDOW when spawning in attached mode.

[zichangg]

To support it, an additional parameter/method is needed in the class process.
Since this is only valid on Windows, I actually doubt that it is worthy making a breaking change.

The other way is to make CREATE_NO_WINDOW by default. I'm not sure whether this will be a problem.

@lrhn @stuartmorgan @sortie Any idea?

[stuartmorgan]

Since this is only valid on Windows, I actually doubt that it is worthy making a breaking change.

Adding a new optional parameter is a breaking change?

As for whether it's worthwhile, if we don't solve this it's very difficult to use Process in a Flutter Windows application without having an unacceptable user experience. Not being able to sub-run processes is severe limitation for a desktop application development framework.

The other way is to make CREATE_NO_WINDOW by default. I'm not sure whether this will be a problem.

I'm not entirely sure, since I don't have a Windows background. My suspicion is that it's the right default for Flutter, but not necessarily for command line Dart. It would need experimentation both ways to see exactly what the impact would be.

[zichangg]

Adding a new optional parameter is a breaking change?

This is an abstract class. Any class implementing process will be affected by a function signature change.

As for whether it's worthwhile, if we don't solve this it's very difficult to use Process in a Flutter Windows application without having an unacceptable user experience.

Agree! I'm wondering whether there are some other good ways instead of adding an optional parameter. If flutter is OK to turn it on by default, is it possible to allow embedder enable "CREATE_NO_WINDOW"? @a-siva

[a-siva]

I am not sure if turning it on by default would be the right thing, what if a flutter app wants to create a process that requires a console.

Would it be fine to define a variable in the environment parameter that is passed to Process.start and control it via that setting?

[idraper]

I think for flutter it does make more sense to have CREATE_NO_WINDOW on by default since it is UI based. Perhaps there is some other case but, off the top of my head, the only reason you'd need to have a console is for input/output. In the context of flutter (UI), you'd do this by piping stdin/stdout to display it in your UI which is what I do currently. In contrast, I think it makes more sense having a console appear by default for dart in general by the same logic, input/output but with no UI. Thus, if an environment variable is possible I think it would work well since flutter can simply set that environment variable as a default.

Since this is only valid on Windows

Is this a Windows-only problem? I plan to eventually have the application I am working on Windows, Mac, and Linux but am currently only working in Windows and thus haven't tested this on the other platforms. It seems to me this same issue would arise by spawning processes in other OS's if they implement the abstract process class but I am not very familiar with them.

[stuartmorgan]

Is this a Windows-only problem?

Yes.

[YazeedAlKhalaf]

Any updates? What is the progress?

[stuartmorgan]

There is some discussion about solutions in https://dart-review.googlesource.com/c/sdk/+/153067

[YazeedAlKhalaf]

@stuartmorgan I am wondering what is the problem that is making the fix take so long?

[stuartmorgan]

I'm not sure why that question is directed at me. I'm not the author of the patch, nor do I make API decisions for the Dart project.

[YazeedAlKhalaf]

@stuartmorgan because I saw your name in here as a reviewer https://dart-review.googlesource.com/c/sdk/+/153067

[stuartmorgan]

The CL hasn't changed since my last comment.

[YazeedAlKhalaf]

Ah okay thanks 👍🙌

[imReker]

@YazeedAlKhalaf
Try this hack: https://github.com/flutter/flutter/issues/47891#issuecomment-708850435

[YazeedAlKhalaf]

@YazeedAlKhalaf
Try this hack: flutter/flutter#47891 (comment)

Well thanks a lot, @imReker, it works fine for me now! I really appreciate it 🙈🔥🚀
I used it in my Flutter Installer here

Anybody wondering how, this is my main.cpp file
comments surrounding the solution by @imReker, his comment: flutter/flutter#47891 (comment)

#include <flutter/dart_project.h>
#include <flutter/flutter_view_controller.h>
#include <windows.h>

#include "flutter_window.h"
#include "run_loop.h"
#include "utils.h"

int APIENTRY wWinMain(_In_ HINSTANCE instance, _In_opt_ HINSTANCE prev,
                      _In_ wchar_t *command_line, _In_ int show_command) {
  // Attach to console when present (e.g., 'flutter run') or create a
  // new console when running with a debugger.
  // if (!::AttachConsole(ATTACH_PARENT_PROCESS) && ::IsDebuggerPresent()) {
  //   CreateAndAttachConsole();
  // }
  // Workaround from: https://github.com/flutter/flutter/issues/47891#issuecomment-708850435
   if (!::AttachConsole(ATTACH_PARENT_PROCESS) && ::IsDebuggerPresent()) {
    CreateAndAttachConsole();
  } else {
    STARTUPINFO si = { 0 };
    si.cb = sizeof(si);
    si.dwFlags = STARTF_USESHOWWINDOW;
    si.wShowWindow = SW_HIDE;

    PROCESS_INFORMATION pi = { 0 };
    WCHAR lpszCmd[MAX_PATH] = L"cmd.exe";
    if (::CreateProcess(NULL, lpszCmd, NULL, NULL, FALSE, CREATE_NEW_CONSOLE | CREATE_NO_WINDOW, NULL, NULL, &si, &pi)) {
      do {
        if (::AttachConsole(pi.dwProcessId)) {
          ::TerminateProcess(pi.hProcess, 0);
          break;
        }
      } while (ERROR_INVALID_HANDLE == GetLastError());
      ::CloseHandle(pi.hProcess);
      ::CloseHandle(pi.hThread);
    }
  }
  // Workaround end

  // Initialize COM, so that it is available for use in the library and/or
  // plugins.
  ::CoInitializeEx(nullptr, COINIT_APARTMENTTHREADED);

  RunLoop run_loop;

  flutter::DartProject project(L"data");
  FlutterWindow window(&run_loop, project);
  Win32Window::Point origin(10, 10);
  Win32Window::Size size(1280, 720);
  if (!window.CreateAndShow(L"flutter_installer", origin, size)) {
    return EXIT_FAILURE;
  }
  window.SetQuitOnClose(true);

  run_loop.Run();

  ::CoUninitialize();
  return EXIT_SUCCESS;
}