Julia: Add STDERR pipe to `readandwrite`

[mbaz]

Sometimes it is necessary to have access to a process' STDIN, STDOUT and STDERR. In my case, I need to open pipes to an external command (specifically, gnuplot) that outputs text to both STDOUT and STDERR. With current command readandwrite(), I can't read STDERR. This is on both 0.3 and master.

I can redirect the process' STDERR to a file with pipe(), but that is not good enough in my case. The reason is that the process is long-lived; I want to issue a command to it by writing to its STDIN, and then read both STDOUT and STDERR to see if the command was successful. This may be repeated any number of times.

I have tried redirecting STDERR to a PipeBuffer, but it doesn't work:

julia> S=PipeBuffer(1024)
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=false, append=true, size=0, maxsize=1024, ptr=1, mark=-1)
julia> readandwrite(pipe(`gnuplot`, stderr=S))
ERROR: MethodError: `redir_err` has no method matching redir_err(::Cmd, ::Base.AbstractIOBuffer{Array{UInt8,1}})

[mbaz]

In the mean time, this function does what I need:

function popen3(cmd::Cmd)
    pin = Base.Pipe(C_NULL)
    cmd_pin = Base.Pipe(C_NULL)
    out = Base.Pipe(C_NULL)
    cmd_out = Base.Pipe(C_NULL)
    err = Base.Pipe(C_NULL)
    cmd_err = Base.Pipe(C_NULL)
    Base.link_pipe(pin, false, cmd_pin, true)
    Base.link_pipe(out, true, cmd_out, false)
    Base.link_pipe(err, true, cmd_err, false)
    r = spawn(false, cmd, (pin, cmd_out, cmd_err))
    Base.close_pipe_sync(cmd_out)
    Base.close_pipe_sync(cmd_err)
    Base.close_pipe_sync(pin)
    Base.start_reading(out)
    Base.start_reading(err)
    return (cmd_pin, out, err, r)
end

However, i don't see a clear way to refactor this to use the @tmp_wpipe and @tmp_rpipe macros, or how the work in https://github.com/JuliaLang/julia/pull/11919 will affect this.

[mbaz]

@vtjnash Sorry to resurrect this old issue... however, I get the feeling that 0.5 is getting close to release, and readandwrite() still does not support reading from STDERR, and I don't see any support for this use case in the documentation.

I know about pipeline(), but it only allows redirection to files, but I need to read and write to the pipes directly.

[vtjnash]

questions should be asked on julia-users or stack overflow. readandwrite is slated for deprecation as it is redundant with open. also, pipeline has no such restriction as you claim.

But just to make sure my cleanup was successful, I tested the cleaned up version of your above function. Although, if you only need to run a single command, then using open or pipeline provides easier wrappers for the same.

julia> function popen3(cmd::Cmd)
           in = Pipe()
           out = Pipe()
           err = Pipe()
           r = spawn(cmd, (in, out, err))
           return (in, out, err, r)
       end

julia> in, out, err, r = popen3(`ls`)

julia> r = spawn(`ls -l VERSION`, (in, out, err))

julia> r = spawn(`ls -l VERSION`, (in, out, err))

julia> r = spawn(`ls -l VERSION`, (in, out, err))

julia> close(in); close(out.in); close(err.in);

julia> readstring(out)
"CONTRIBUTING.md\nDEBUGGER.md\nDISTRIBUTING.md\nHISTORY.md\nLICENSE.md\nMake.inc\nMake.user\nMakefile\nNEWS.md\nREADME.arm.md\nREADME.md\nREADME.windows.md\nVERSION\nui\nusr\nusr-staging\n-rw-r--r--  1 jameson  staff  10 Nov 10  2015 VERSION\n-rw-r--r--  1 jameson  staff  10 Nov 10  2015 VERSION\n-rw-r--r--  1 jameson  staff  10 Nov 10  2015 VERSION\n"

[mbaz]

Thanks, and sorry for the noise. The source of my problem here is that I don't see a documented way to access stderr from open(), and that both the spawn method you used and Pipe() are also undocumented.

[FlorianFranzen]

Almost 4 years later, it is still not documented. Could we at least document Pipe, including the fact that calling close(p) on a pipe deletes all input data, while close(p.in) preserves it? That would have saved me a lot of time.