Crystal: process terminated not reporting correctly

[xmonader]

process terminated not reporting correctly

out1, in1 = IO.pipe(read_blocking = false, write_blocking = false)
out2, in2 = IO.pipe(read_blocking = false, write_blocking = false)

s = Process.new("find /tmp/", shell: true, output: in1, error: in2)
until s.terminated?
  # pp out1.closed?
  # res = out1.gets(1)
  # res = out1.gets
  puts "before readline, s is terminated? #{s.terminated?}"
  res = out1.read_line
  # puts "res is >#{res}<" <--- need to have this line here or it hangs
  break if res.nil?
end

Versions

Crystal 0.34.0 [4401e90f0] (2020-04-06)

LLVM: 8.0.0
Default target: x86_64-unknown-linux-gnu

expected

this is the output i expect without the commented line

➜  subprocess git:(master) ✗ crystal run src/cmd2.cr
before readline, s is terminated? false
res is >/tmp/<
before readline, s is terminated? false
res is >/tmp/.com.vivaldi.Vivaldi.gy4Cd9<
before readline, s is terminated? false
res is >/tmp/.com.vivaldi.Vivaldi.gy4Cd9/SingletonCookie<
before readline, s is terminated? false
res is >/tmp/net-export<
before readline, s is terminated? false
...
...
res is >/tmp/crystal_tests/mosquito/.travis.yml<
before readline, s is terminated? false
res is >/tmp/crystal_tests/mosquito/LICENSE<
before readline, s is terminated? false
res is >/tmp/crystal_tests/mosquito/README.md<
reached here..

this with the commented line

➜  subprocess git:(master) ✗ crystal run src/cmd2.cr
before readline, s is terminated? false
res is >/tmp/crystal_tests/.git/logs/refs/remotes<
res is >/tmp/crystal_tests/.git/config<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.git/index<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/base_container_delete<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/base_container_delete/log<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/base_container_delete/log/log.err<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/base_container_delete/log/log.out<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/main<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/main/log<
before readline, s is terminated? false
res is >/tmp/crystal_tests/.neph/main/log/log.err<
before readline, s is terminated? false
....bunch more....
res is >/tmp/crystal_tests/mosquito/LICENSE<
before readline, s is terminated? false
res is >/tmp/crystal_tests/mosquito/README.md<
reached here..
➜  subprocess git:(master) ✗ crystal run src/cmd2.cr
➜  subprocess git:(master) ✗ crystal run src/cmd2.cr
before readline, s is terminated? false
before readline, s is terminated? false
before readline, s is terminated? false
before readline, s is terminated? false

...hangs here with the same message lots more....

[HankAnderson]

When you use pipes R and W, R never ends unless W is closed. Since you never close W, this is expected.

Here's something that works, no need to use pipes (because Process.run already uses pipes):

Process.run("find /tmp/", shell: true) do |process|
  while line = process.output.gets
    puts line
  end
end

Also, I don't think that long trace is really needed.

Maybe you could also explain what are you trying to do. Many times it doesn't matter that one code you write doesn't work, it matters what you want to accomplish it works (and it might be written in a completely different way).

[jhass]

Duplicate of #9404 ?

[despiegk]

we are trying to get it line by line so we can process the stdout coming from the multiple subprocesses, the find example is just an example.
I've tried many combinations but never succeeded, that's why we came with the readline solution on the pipe which works well appart from the lock, I understand to have to close the in, but how do we know when to close it and not block? any ideas much appreciated, thanks so much for your help.

[asterite]

@despiegk Did you see @HankAnderson 's answer? It looks like the solution.

Otherwise, unless you tell us exactly what you need to do (show us more code!) there's nothing we can do to help you.

[despiegk]

solution from @HankAnderson is not really what we are looking for, it will wait till the command exits.

• what we need is ability to execute a command and then line by line fetch stdout/sterr while the command is running (e.g. a long find)
• when commands stops we need to get out of the fetching of the lines.

we tried quite some tricks already, no real solution yet,
hope this helps and thank you

@xmonader found that if you print before leaving it does not block, this shows there is an issue somewhere

[jhass]

Did you try it? I'm pretty confident that's what @HankAnderson's solution does, even without trying it.