Php-rdkafka: Call to undefined method RdKafka\ProducerTopic::producev()

[daalvand]

Dear All,
in windows os, I get Call to undefined method RdKafka\ProducerTopic::producev() error, but it works in Linux well.

• PHP version: 7.4
• librdkafka version: ...
• php-rdkafka version: 4.1.1
• kafka version: 2.13

[nick-zh]

This can happen if you use a librdkafka version that doesn't support headers (is below 0.11.4)

[daalvand]

This can happen if you use a librdkafka version that doesn't support headers (is below 0.11.4)

I just downloaded zip file php_rdkafka-4.1.1-7.4-ts-vc15-x64 from following link
https://pecl.php.net/get/rdkafka-4.1.1.tgz
and put php_rdkafka.dll in the ‍ext‍ folder
and librdkafka.dll in main php folder
and added extension=rdkafka in php.ini
Do I have to do anything else?

[nick-zh]

I am uncertain, @arnaud-lb you are maybe more familiar with the w32 build, do we need to add HAVE_RD_KAFKA_MESSAGE_HEADERS there as well, fixed to 1 similar to other values we have there?
If so, probably few other functions are missing, like purge and all the transaction functions

[arnaud-lb]

Yes, it appears that the DLLs are built with librdkafka 1.2, and that defining HAVE_RD_KAFKA_MESSAGE_HEADERS, HAS_RD_KAFKA_PURGE, and HAS_RD_KAFKA_PARTITIONER_MURMUR2 would work.

@cmb69 do you confirm ?

[nick-zh]

Can we adjust it somwhere that it is built with 1.5.3 or 1.4.4 then we also have transactions. If not i can adjust my PRs to remove transactions again

[cmb69]

I just submitted PR #414 which adds AppVeyor CI integration for building the extension only. That might be helpful to see what is supported on Windows and what is not.

Anyhow, trying to build PR #411 locally fails, because rd_kafka_init_transactions is not defined. Apparently, that is not supported by librdkafka 1.2.1. I'll try to update to a newer librdkafka.

[nick-zh]

@cmb69 correct, transactions are available from librdkafka:1.4 onward. Sry i am not too familiar with the w32 build process, so not sure where we would need to update this for pecl :sweat_smile:
and thx for your PR :+1:

[cmb69]

The Windows builds rely on packages which have been build by the PHPonWindows teams, and these are available from https://windows.php.net/downloads/pecl/deps/. I can deploy librdkafka 1.5.3 there and in combination with PR #411 (plus the minor #415) the build would succeed (I have no testing enviroment available, though), but for users who wanted to build against librdkafka 1.2.1 the build would then fail. Unfortunately, checking for the installed librdkafka version isn't easy (one would have to grep rdkafka.h for RD_KAFKA_VERSION, since the Windows build system is lacking respective autotools features). On the other hand, that might not be an issue in practice – don't know if anyone builds PECL/rdkafka on Windows. Then again, it might not be the best idea to upgrade librdkafka for stable (4.1.x) releases, so maybe do that only for 5.x?

[nick-zh]

@cmb69 thx a lot for the insight 🙏
@arnaud-lb what is your opinion on this? personally i think librdkafka:1.5.3 should be fine, it is also just a minor upgrade and i have been using it with no problems so far.
Let me know, if we ant to stick with librdkafka:1.2.1 for w32 for rdkafka:4.1.x, if so i will adjust my 4.x pr to exclude the transaction flag

[arnaud-lb]

Thanks @cmb69

I would be ok for upgrading librdkafka for 5.x and enabling transaction support by default. People would still be able to build against older versions by editing only config.w32, which seems reasonable to me.

[nick-zh]

What about 4.x?

[arnaud-lb]

I think that we should stick with 1.2 in this version, because users wouldn't expect a librdkafka upgrade in a minor 4.x

[nick-zh]

I have updated the 4.x PR to reflect this (transactions flag removed)

[arnaud-lb]

@daalvand: thanks to @nick-zh and @cmb69, this is fixed in version 4.1.2 - the DLLs for version 4.1.2 have the producev() method: https://pecl.php.net/package/rdkafka/4.1.2/windows