Swoole-src: WARNING swPipeUnsock_create(:83): socketpair() failed, Error: Too many open files[24]

[asheroto]

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a simple script for reproducing the error.

I'm downloading many small files from the Internet using file_get_contents, then writing using file_put_contents. I have these inside a swoole process function. At around the 500 file mark, I get this error:

WARNING swPipeUnsock_create(:83): socketpair() failed, Error: Too many open files[24]

Example script:

$data = array(<URLs of over 500 files>);

foreach($data as $abc) {

    $processes[ $abc ] = new \Swoole\Process(function () use ($abc) {

        $bca = file_get_contents($abc);

        file_put_contents("dir/path/to/new/file.abc",$bca);

    }

    $processes[ $data ]->start();

}

I even tried using the wait command after 100 processes at a time. Do I need to allow more time for swoole to close the process?

1. What did you expect to see?

No error

1. What did you see instead?

WARNING swPipeUnsock_create(:83): socketpair() failed, Error: Too many open files[24]

1. What version of Swoole are you using (show your php --ri swoole)?

Latest

1. What is your machine environment used (show your uname -a & php -v & gcc -v) ?

Debian 10 Buster

[matyhtf]

@asheroto

You need to adjust the maximum number of handles, (ulimit -n 100000) or set the third parameter to 0 to close the process pipe.

new Swoole\Process($fn, false, 0);

[asheroto]

Thanks. Just curious, why would anyone want to leave the process pipe open?

Also, does closing the process pipe kill all of Swoole's process pipe or just for that file? For example, if I have two PHP scripts running at the same time, both using the process pipe, will specifying 0 on the third parameter kill the pipe for both scripts? Also, if I use a channel to communicate, do other scripts have access to that same channel, or is it only for the current instance of the php script. So if I have test1.php and test2.php running, can they see each other's channels?

Thanks again, very much appreciated. :)

[huanghantao]

@asheroto

1. Pipes are used to communicate between processes. If processes need to communicate with each other, then you need to keep the pipes.
2. Closing the pipes won't kill the process, just like opening a file and then closing the file. In fact, you can also use files for process communication.
3. A Channel can be accessed between two processes, for example:

<?php

use Swoole\Coroutine;
use Swoole\Coroutine\Channel;
use Swoole\Process;

$chan = new Channel();

$process = new Process(function () use (&$chan) {
    Coroutine::create(function () use (&$chan) {
        var_dump('process: ' . getmypid());
        $chan->push('data');
    });
});

Coroutine::create(function () use (&$chan) {
    var_dump('process: ' . getmypid());
    $data = $chan->pop();
    var_dump($data);
});

$process->start();
$process->wait();

You can see the following output on my machine:

string(14) "process: 94399"
string(14) "process: 94400"
string(4) "data"

However, this approach relies on the operating system's write-time replication feature, where memory can be shared between multiple processes. However, there is no guarantee that all operating system versions will support this. It is recommended that a channel be used within a process's coroutine.

[asheroto]

I see, okay, thank you for your help!