Crystal: Reading from FiFo pipe blocks execution of other fibers

[mavu]

steps to reproduce:

create a pipe with
mkfifo piope
you can put stuff into the pipe from a different terminal with for example
cat > piope and if you use cat pipe to read the pipe, it works as expected. But the following Crystal program hangs, and never executes the second fiber.

f = File.open("piope","a+")
spawn do
  loop do
    f.gets
  end
end


spawn do
  puts "-"
  loop do
    puts "+"
    sleep(1)
  end
end

sleep()

Crystal version 0.30.1 on Debian testing

[jhass]

Somebody wants to try on master with and without MT? :)

[asterite]

I already did: the other fiber runs well with MT. But I think the pipe fiber shouldn't block. I debugged it and it seems the LibC.read call just hangs. But I know nothing about pipes to understand why that happens.

[waj]

Can you try setting f.blocking = false? File.open currently opens the files in blocking mode since it makes no sense to use non-blocking for regular files (the calls to read/write are always blocking in POSIX).

[mavu]

Can you try setting f.blocking = false? File.open currently opens the files in blocking mode since it makes no sense to use non-blocking for regular files (the calls to read/write are always blocking in POSIX).

Yes, that does in fact make it work.
Thats not a bug after all then I guess?

[asterite]

@waj Is there a problem if we open files in non-blocking mode by default?

[waj]

@asterite I was about to say no but then I found this commit: https://github.com/crystal-lang/crystal/commit/6943331da87c240c719d04fb8281f9b959586c38

[Sija]

@waj Is this is still an issue, 4 yrs l8r?

[waj]

@Sija what do you mean by "this"? non-blocking IO for files? Standard IO operations over filesystem descriptors are always blocking, no matter what you do. They are always "ready" to read and write. To solve this maybe we should use aio APIs for files: http://man7.org/linux/man-pages/man7/aio.7.html

[Sija]

@waj By _this_ I was referring to your commit comment (epoll raises warnings in Linux)

[mavu]

@waj Is this is still an issue, 4 yrs l8r?

I did not get any warnings when I add file.blocking = false.
This is on Kernel 5.2

[asterite]

@mavu Is that for the pipe or just for any regular files? I think the warning happened on regular files.

[mavu]

works for me with regular files too, but needs an additional sleep(1) in the first fiber after the f.gets or the output seems only to be the :yes from the first loop, but I think that is to be expected?

[waj]

@mavu gets returns nil when it reaches the EOF, so you should break the loop in that case

[waj]

Regarding this issue, I've been discussing with @asterite and we think that it would be great if Crystal sets the mode to non-blocking if the opened file is a FIFO (or any other fd type that can be used in non-blocking mode).

We could use fstat to check the type of file right after open and set the blocking mode accordingly.

One problem I found is that if "r" mode is used while opening the FIFO then the open call blocks until there is another process at the other end of the pipe. That behaviour seems to be quite difficult to mimic with non-blocking operations, because until open returns there is no fd to use in libevent (epoll and kevent). If O_NONBLOCK is used in the open call it returns immediately without waiting for a writer on the other end.

In other words we always try to make these APIs to behave like blocking from the fibers point of view but non-blocking for the entire process, but seems like it might not be possible for these.

Any ideas?

[wontruefree]

I would rather File could handle all file related cases but we could add a Pipe class for this.

Pipe.open("piope", "a+")

[yxhuvud]

@wontruefree Would sockets be Pipes too?

Eventually, uring could be interesting as a general solution in the area, but it will probably take a while before the user space support for that in libc is good enough. See https://lwn.net/Articles/776703/ .