Godot-proposals: Improve `OS.execute()` (run a callback with the line contents every time a line is printed)

[NHodgesVFX]

Describe the project you are working on:
A application that calls a console executable to encode video

Describe the problem or limitation you are having in your project:

When using os.execute you can get the data back using blocking mode or non-blocking when using threads but only when the application is closed/completed. At least as far as I know there is no way to get the data line by line while the application is running.

Describe how this feature / enhancement will help you overcome this problem or limitation:

It would be great if in either blocking or non blocking for every line printed by the exe call a function with the new line as its printed in realtime.

this would make it super easy to call and check progress on external executables

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
for example it should work like

os.execute("res://example.exe", ["-w", "example"], true, "callback")

func callback(new_line):
    #called every time a new line is printed by the called exe in real time
     print(new_line)

Describe implementation detail for your proposal (in code), if possible:

If this enhancement will not be used often, can it be worked around with a few lines of script?:

no this would need to be implemented in core as it not something that can be changed in gdscript

Is there a reason why this should be core and not an add-on in the asset library?:
as noted above this is already a core feature that would be enhanced

[aspenforest]

Have you tried creating a TCP server on Godot and then OS.execute a client (python) script which can then talk with godot through the server instance? I tried it and it works.

EDIT: Though, if you're not used to socket programming, then this might be difficult for you.

[NHodgesVFX]

I am not to familer with socket programming but I'm sure you could probably also run a simple http server on the python side then post to it from godot. the point of this proposal, is at the moment it requires alot of extra work and work arounds to make a usable solution to robustly call an outside executable.

Thanks for the suggestion, it inspired using a http server . Which should be a good work around till this is hopefully improved upon.

[nobuyukinyuu]

For some reason TCP is preferred over standard pipes, but as someone who barely knows how to use C++, communication with console apps written in this language becomes less than trivial, requiring argument processing (in my case, 3rd party string functions to handle splitting and type conversion) or a sockets implementation on the backend.

I support proposals that amend OS.execute() to have arguments for non blocking communication via stdin/stdout.

[aspenforest]

Okay so I took a look at how, if even, it can be implemented and I have to be honest here, I could not find where the OS.execute is defined in the source code. You might think "ok", its probably in "OS" -> "os.cpp" file, but you'd be wrong. The header file "os.h" has a definition of "execute", which is not defined in the os.cpp" file. If the source code would make any fucking sense, then MAYBE I could contribute something myself.

[NHodgesVFX]

I think its defined per platform. I found the windows version here https://github.com/godotengine/godot/blob/master/platform/windows/os_windows.cpp

[bruvzg]

It's platform specific, plus there is additional wrapper for the gdscript.

• core/bind/core_bind.cpp (gdscript interface for C++ implementation, with the different arguments!).
• core/os/os.h virtual definition of the C++ function.
• drivers/unix/os_unix.cpp generic Unix implementation, used on Linux, Android, Haiku, iOS and macOS.
• platform\windows\os_windows.cpp implementation for Windows.
• platform\javascript\os_javascript.cpp and platform\uwp\os_uwp.cpp dummy implementations, execute is not supported on these platforms.

[aspenforest]

Okay, thanks. That's the code I was looking for. Now that I've realized that I can't understand a single line of that gibberish, I'm dropping any attempt to improve Godot by myself.

[neikeq]

There's godotengine/godot#8310 if any one wants to retake it.

[silverkorn]

For some reason TCP is preferred over standard pipes, but as someone who barely knows how to use C++, communication with console apps written in this language becomes less than trivial, requiring argument processing (in my case, 3rd party string functions to handle splitting and type conversion) or a sockets implementation on the backend.

I support proposals that amend OS.execute() to have arguments for non blocking communication via stdin/stdout.

+ stderr, and if possible, custom file descriptors for some kind of parallelism or message compartmentalization.

But at least stdin _(write)_, stdout _(read)_ & stderr _(read)_ would be the must and it's pretty much the standard everywhere.