Php-rdkafka: Consumption problems on upstream versions

[helmer]

Encountered some problems while trying to setup a PHP Kafka consumer. Here be my environment:

• PHP version: 7.4.16 (NTS)
• librdkafka version: 1.6.1
• php-rdkafka version: 5.0.0
• kafka version: 2.7.0

Following the upstream High-level consumer
guide, I am getting RdKafka\KafkaConsumer->subscribe() does not exist:
PHP Fatal error: Uncaught Error: Call to undefined method RdKafka\Consumer::subscribe() in /tmp/consumer/metrics.php:59

Reverting to Low-level consuming (legacy) seems to work,
although now
%4|1615245695.751|CONFWARN|rdkafka#consumer-1| [thrd:app]: Configuration property dr_msg_cb is a producer property and will be ignored by this consumer instance.

I'm not entirely sure why dr_msb_cb is problematic - pretty sure I am not providing any invalid callbacks:

$conf = new RdKafka\Conf();
$conf->set('metadata.broker.list', 'kafka-server:9092');
$consumer = new RdKafka\Consumer($conf);

The guidelines seem(?) to be against an older version? Or am I missing something else?

Additionally the RD_KAFKA_OFFSET_STORED constant when used during consumeStart() throws PHP Fatal error: Uncaught RdKafka\Exception: Local: Invalid argument or configuration in /tmp/consumer/metrics.php:63 ..

[Steveb-p]

Hello @helmer.

Could you please make sure that the librdkafka being used in phprdkafka is the same as the one you declared? You can check it in phpinfo()/php -i. Sometimes build and runtime libraries differ which causes issues.

[helmer]

Hey @Steveb-p, yes, what I declared I actually extracted from php -i, but just in case:

# php -i | grep -B2 -A4 'rdkafka support'
rdkafka

rdkafka support => enabled
version => 5.0.0
build date => Mar  8 2021 13:29:22
librdkafka version (runtime) => 1.6.1
librdkafka version (build) => 1.6.1.255

[nick-zh]

Please make sure you are using the correct consumer, because you state you are using the high level consumer RdKafka\KafkaConsumer->subscribe() but your error indicates you are using the low level consumer RdKafka\Consumer::subscribe(), just to avoid any confusion

[helmer]

@nick-zh Whoops, that was it 😊 I started with librdkafka 0.11.6 + low level consumer and did not realise there is a different base class for consumption now. The classnames do look very similar upon first glance!

Since the documentation of low level consumer hints that it is legacy, perhaps adding a @deprecated annotation to RdKafka\Consumer in kwn/php-rdkafka-stubs could be worth considering?

Closing this, thanks @Steveb-p and @nick-zh

[nick-zh]

@helmer i totally agree, there was a discussion about deprecating it, but for now the decision was not to do it, but maybe i could add a note in the docs as well :v:
Glad it is working now :+1: