Php-rdkafka: In the FPM environment, the response is very slow,

[QQ2021]

• PHP version: 7.0.10
• librdkafka version: 0.9.2
• php-rdkfaka version: 3.0.0

$_st  = microtime(TRUE);
$producer = new \RdKafka\Producer();
$producer->addBrokers('kafka001:9092,kafka002:9092');
$topic = $producer->newTopic('PHPTest');
$topic->produce(\RD_KAFKA_PARTITION_UA, 0, date(DATE_W3C));
var_dump(microtime(TRUE) - $_st);

output: float(0.00036001205444336)

But in the nginx log is mostly 100 to 200ms

I would like to ask FPM how to optimize it?

[arnaud-lb]

Hi @zzuliqiu

Setting the internal.termination.signal and socket.blocking.max.ms can reduce this drastically: https://github.com/arnaud-lb/php-rdkafka#internalterminationsignal

[QQ2021]

Thank you, I will use it as soon as possible

[QQ2021]

`pcntl_sigprocmask(SIG_BLOCK, [SIGIO]);
$conf = new RdKafkaConf();
$conf->set('socket.blocking.max.ms', 1);
$conf->set('internal.termination.signal', SIGIO);

$rk = new RdKafkaProducer();
$rk->setLogLevel(LOG_INFO);
$rk->addBrokers('kafka001:9092,kafka002:9092');
$topic = $rk->newTopic('PHPTest');
$topic->produce(RD_KAFKA_PARTITION_UA, 0, date(DATE_W3C));`

I used like this, but the problem has not improved, please help me check，Thank you！

[cpats007]

I have that same issues as @zzuliqiu has reported, although I see a massive increase in response times even before using it. If I create a class which defines and sets up Kafka my site response time doubles - if I then start producing with it it increases even more.

You can't use the pcntl_sigprocmask(SIG_BLOCK, [SIGIO]) command in PHP-FPM as it's a CLI php only command from what I read, which means we need to find some other way to optimise otherwise this is completely unusable 😞

my response times have gone from 48ms average, up to around 79ms - and thats simply instantiating the class, not producing

[cpats007]

Also, the documentation states that socket.blocking.max.ms is deprecated so not sure whether this would have any effect anyway

[arnaud-lb]

Hi @cpats007

socket.blocking.max.ms seems to still be used during librdkafka shutdown, so it's still relevant to reduce its value: https://github.com/edenhill/librdkafka/blob/f904298e774aae4776d25f074a071048b51dd53c/src/rdkafka_broker.c#L3310

pcntl_sigprocmask() can be used in fpm. Make sure that the PCNTL extension is loaded, and that the function is not disabled in php.ini.

[cpats007]

hi @arnaud-lb - thanks for the reply - I'll set that in the config then, but from what I can see the pcntl_sigprocmask() function doesn't work in FPM. It is loaded and enabled, but is not available in FPM, only via CLI commands.

[arnaud-lb]

Is it in php.init's disable_functions setting ?

[cpats007]

No, disabled functions is empty. Information I got from here:

http://stackoverflow.com/questions/35026153/call-to-undefined-function-pcntl-fork-php-fpm-nginx

edit: and here: https://forum.nginx.org/read.php?3,193092

[arnaud-lb]

I'm using it with php-fpm, so I can say 100% sure that this is not true :)

Maybe your PHP configuration is loading the extension only in CLI ? I know that some PHP packages have different php.inis for different SAPIs. Could you check in the output of phpinfo() ?

[arnaud-lb]

Note that phpinfo() may mention PCNTL in the credits, but this doesn't mean that the extension is actually loaded.

If the extension is loaded, you should see this:

[K4ST0R]

Hello,
I have the same slow issue with php-fpm event with

$conf->set("socket.blocking.max.ms", 1);
$conf->set('internal.termination.signal', SIGIO);

I'm using
PHP Version 5.6.20
php-rdkafka: 3.0.1
librdkafka : 0.9.1.0

[Wangwei057]

Hi,
I have met the same issue. The kafka producer finished within 1ms, but the nginx's log shows it still took nearly 1000 ms. Does the extension do some background work when the process end?

The configuration below has already been used.
$conf->set("socket.blocking.max.ms", 1);
$conf->set('internal.termination.signal', SIGIO);

php-fpm : 5.6.13
php-rdkafka: 3.0.1
librdkafka: 0.9.4

[arnaud-lb]

Hi,

librdkafka is async, this means that calling ->produce() sends messages in the background. When produce() returns, the messages are probably not sent yet.

When the RdKafka\Producer object is destroyed, it waits for all messages to be really sent to the server.

After that, it also waits for all librdkafka threads to terminate. This is where socket.blocking.max.ms + internal.termination.signal + pcntl_sigprocmask() help. By setting these properly, the delay should be minimal.

You can force messages to be sent earlier by calling poll() repeatedly until getOutQLen() returns 0 :

while ($producer->getOutQLen() > 0) {
    $producer->poll(50);
}

This way, messages are sent before RdKafka\Producer, and you can tell which part is being slow exactly: is it pushing messages, or is it waiting for threads to terminate ?

@zzuliqiu @cpats007 @K4ST0R @Wangwei057, could you try that and confirm which part is being slow for you ?

[Wangwei057]

I saw that in the producer example, and removed it from my code when I found it took a long time waiting for the $producer->getOutQLen() returns 0. I didn't realize what actually happened when produce()was called then. Thanks a lot.

@arnaud-lb

[cpats007]

Hi @arnaud-lb. I know this is an outdated thread possibly - but I was wondering if there were any speed improvements - currently, New Relic is reporting it's taking around 34ms to add the data to Kafka in my application, would that seem about right? Seems slow to me - would enabling the internal.termination.signal + pcntl_sigprocmask() help to get this lower? I was hoping I would be able to achieve this in less than 10ms for just this part of my application

Many thanks

[indieyourself]

@arnaud-lb php-fpm returns turns slow after creating new Producer, even though did nothing except creating a Producer.

[indieyourself]

@arnaud-lb and set this https://github.com/arnaud-lb/php-rdkafka#internalterminationsignal didn't work.;(

[paulm17]

@arnaud-lb seems out of all the php libs, this is the last man standing. What a sad state of affairs!

I'm having the same problem.

When will persistent connections and better performance be forth-coming?

Thanks for the work on the lib, much appreciated!

[mamciek]

I can confirm the same problem with PHP 7.1.
Setting socket.blocking.max.ms + internal.termination.signal + pcntl_sigprocmask() does not work

[arnaud-lb]

@no1youknowz @mamciek could you provide some code that reproduces your issue ? Also, which version of php-rdkafka, librdkafka, and kafka are you using ?

[paulm17]

Sorry, now using a different library and now moved onto another phase of the project. Won't be returning to this point again.

[cpats007]

Different library for Kafka @no1youknowz ? If you are, which one are you using if you don't mind me asking?

[paulm17]

@cpats007. I'm using weiboad/kafka-php.

[cpats007]

thanks - how are you finding it? which library does it use etc?

[arnaud-lb]

@no1youknowz great to know that you found a solution to your issues. Note that this library seems to be rather experimental, and right now it doesn't yet handle error conditions (leader changes, temporary network failures, etc happening quite often in a live cluster), and will silently lose messages when this happens (by default require ack = 0).

I'm still willing to look at your issue if you could provide some code reproducing it.

[paulm17]

@arnaud-lb thanks for the feedback. Although I have put steps in to mitigate such occurrences.

[matthewgoslett]

I'm experiencing the same issue my side.

Docker Image: php:7.0-fpm
librdkafka version (runtime) => 0.9.4-174-g980cea
librdkafka version (build) => 0.9.5.0
php-rdkafka 3.0.1 (stable)

test.php

<?php

$broker = '192.168.99.100';

$producer = new \RdKafka\Producer();
$producer->addBrokers($broker);

$topic = $producer->newTopic('test');
$topic->produce(RD_KAFKA_PARTITION_UA, 0, 'Hello World');

root@29bafaaea1a3:/opt/test# for run in {1..10}; do time php test.php; done

real    0m1.178s
user    0m0.010s
sys 0m0.030s

real    0m1.127s
user    0m0.000s
sys 0m0.010s

real    0m1.133s
user    0m0.000s
sys 0m0.020s

real    0m1.142s
user    0m0.030s
sys 0m0.000s

real    0m1.135s
user    0m0.010s
sys 0m0.010s

real    0m1.143s
user    0m0.020s
sys 0m0.000s

real    0m1.145s
user    0m0.020s
sys 0m0.010s

real    0m1.190s
user    0m0.010s
sys 0m0.010s

real    0m1.147s
user    0m0.020s
sys 0m0.010s

real    0m1.185s
user    0m0.040s
sys 0m0.020s

[matthewgoslett]

re my previous test:

Without socket.blocking.max.ms set to 1, it averages +- 0m2.226s
With socket.blocking.max.ms set to 1, it averages +- 0m1.142s
If I add queue.buffering.max.ms with a value of 1, it averages +- 0m0.629s

edit
If I raise socket.blocking.max.ms to 50 or 100 (ms), I can consistently achieve 0m0.284s

[arnaud-lb]

Thanks for your feedbacks @matthewgoslett. I can confirm these numbers.

By also adding a poll() loop after producing, I can reduce the time to 0.030s - 0.060s (actual code taking only 12-14ms of that) :

$topic->produce(RD_KAFKA_PARTITION_UA, 0, 'Hello World');

while ($producer->getOutQLen() > 0) {
    $producer->poll(5);
}

[arnaud-lb]

Closing because it seems to be a configuration issue.

[sjp00556]

"under fpm it is possible to call fastcgi_finish_request before destroying the RdkafkaProducer object." it work,thank you arnaud-lb!

[lzyRun]

@arnaud-lb i also met this issue.
the version i use:
rdkafka: 3.0.4
librdkafka: 0.11.0
And i have try these ways，but it doesn`t work. the nginx log is mostly 100 to 200ms.
1、
while ($producer->getOutQLen() > 0) {
$producer->poll(5);
}

2、
socket.blocking.max.ms + internal.termination.signal + pcntl_sigprocmask()

[pinguo-shenqiang]

@lzyRun did u resolve it?

[pinguo-shenqiang]

@sjp00556 could you show me the code how to resolve it?

[h1z3y3]

I met the same issue when I used php5.5.24 with docker.
I resolved it by upgrading php to 7.1.3.

[jiangxiaocan]

I strong suggestiong use to these code to demo ，when i begin used rdkafka to produce message that kafka can not receive message if not these code
while ($producer->getOutQLen() > 0) { $producer->poll(5); }