Php-rdkafka: PHP7.4 $producer->newTopic results in Couldn't execute method Exception::__tostring

Created on 18 Mar 2020  路  15Comments  路  Source: arnaud-lb/php-rdkafka
  • PHP version: 7.4.3
  • librdkafka version: 1.3.0
  • php-rdkafka version: 4.0.3
  • kafka version: 2.4.1

(Feel free to suggest a better issue title)

We're upgrading from PHP 7.3.8 to 7.4.3 and we're running into this problem that we're having some trouble with debugging.

Before this upgrade we were running librdkafka ^1, kafka ^2, php-rdkafka ^3.

The error occurs when we call newTopic() on a producer. E.g.

$config = new \RdKafka\Conf();
$this->producer = new \RdKafka\Producer($config);
$this->producer->addBrokers($brokers);
$topic = $this->producer->newTopic($topicName);

It results in a 

Fatal error: Couldn't execute method Exception::__tostring in Unknown on line 0

which we assume to be somewhere in a C binding that I don't know how to debug.

The full debug log is as follows, where the (inexistent) topic I am trying to publish to is dev.a.

It feels like there is some weird stuff going on there, but I don't know why. If we downgrade the php-rdkafka version to 3.1.3 (from 4.0.3) things work again.

Any idea what this could be?

18-Mar-2020 13:37:16] NOTICE: fpm is running, pid 9
[18-Mar-2020 13:37:16] NOTICE: ready to handle connections
%7|1584538705.951|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1584538705.951|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.3.0 (0x10300ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,sasl,regex,lz4,sasl_plain,plugins, GCC GXX PKGCONFIG INSTALL
 GNULD LDS C11THREADS LIBDL PLUGINS ZLIB HDRHISTOGRAM SNAPPY SOCKEM CRC32C_HW, debug 0xfffff)
%7|1584538705.952|WAKEUPFD|rdkafka#producer-1| [thrd:app]: kafka:9092/bootstrap: Enabled low-latency ops queue wake-ups
%7|1584538705.952|BROKER|rdkafka#producer-1| [thrd:app]: kafka:9092/bootstrap: Added new broker with NodeId -1
%7|1584538705.952|CONNECT|rdkafka#producer-1| [thrd:app]: kafka:9092/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1584538705.953|BRKMAIN|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enter main broker thread
%7|1584538705.954|CONNECT|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received CONNECT op
%7|1584538705.954|STATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1584538705.954|BROADCAST|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: Broadcasting state change
%7|1584538705.954|CONNECT|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: broker in state TRY_CONNECT connecting
%7|1584538705.954|STATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1584538705.954|BROADCAST|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: Broadcasting state change
%7|1584538705.957|CONNECT|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connecting to ipv4#172.19.0.3:9092 (plaintext) with socket 13
%7|1584538705.959|CONNECT|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connected to ipv4#172.19.0.3:9092
%7|1584538705.959|CONNECTED|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connected (#1)
%7|1584538705.959|FEATURE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1584538705.959|STATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1584538705.959|BROADCAST|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: Broadcasting state change
%7|1584538705.959|SEND|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Sent ApiVersionRequest (v0, 25 bytes @ 0, CorrId 1)
%7|1584538705.967|RECV|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received ApiVersionResponse (v0, 294 bytes, CorrId 1, rtt 8.20ms)
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker API support:
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Produce (0) Versions 0..8
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Fetch (1) Versions 0..11
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Offset (2) Versions 0..5
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Metadata (3) Versions 0..9
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey LeaderAndIsr (4) Versions 0..4
%7|1584538705.967|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey StopReplica (5) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey UpdateMetadata (6) Versions 0..6
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey ControlledShutdown (7) Versions 0..3
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey OffsetCommit (8) Versions 0..8
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey OffsetFetch (9) Versions 0..6
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey FindCoordinator (10) Versions 0..3
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey JoinGroup (11) Versions 0..6
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Heartbeat (12) Versions 0..4
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey LeaveGroup (13) Versions 0..4
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey SyncGroup (14) Versions 0..4
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DescribeGroups (15) Versions 0..5
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey ListGroups (16) Versions 0..3
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey SaslHandshake (17) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey ApiVersion (18) Versions 0..3
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey CreateTopics (19) Versions 0..5
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DeleteTopics (20) Versions 0..4
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DeleteRecords (21) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey InitProducerId (22) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey OffsetForLeaderEpoch (23) Versions 0..3
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey EndTxn (26) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DescribeAcls (29) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey CreateAcls (30) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DeleteAcls (31) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DescribeConfigs (32) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey AlterConfigs (33) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DescribeLogDirs (35) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey SaslAuthenticate (36) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey CreatePartitions (37) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey CreateDelegationToken (38) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey RenewDelegationToken (39) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey DeleteGroups (42) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Unknown-43? (43) Versions 0..2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Unknown-44? (44) Versions 0..1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Unknown-45? (45) Versions 0..0
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Unknown-46? (46) Versions 0..0
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:   ApiKey Unknown-47? (47) Versions 0..0
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature MsgVer1: Produce (2..2) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature MsgVer1: Fetch (2..2) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature MsgVer1
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature MsgVer2: Produce (3..3) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature MsgVer2: Fetch (4..4) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature MsgVer2
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature ApiVersion
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1584538705.968|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1584538705.969|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1584538705.973|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1584538705.973|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1584538705.973|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1584538705.975|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1584538705.975|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1584538705.975|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1584538705.977|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1584538705.977|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature ThrottleTime: Produce (1..2) supported by broker
%7|1584538705.978|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1584538705.978|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature ThrottleTime
%7|1584538705.979|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature Sasl: JoinGroup (0..0) supported by broker
%7|1584538705.979|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature Sasl
%7|1584538705.980|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1584538705.981|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature SaslHandshake
%7|1584538705.981|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1584538705.982|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature LZ4
%7|1584538705.982|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature OffsetTime: Offset (1..1) supported by broker
%7|1584538705.983|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature OffsetTime
%7|1584538705.984|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1584538705.984|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature IdempotentProducer
%7|1584538705.985|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature ZSTD: Produce (7..7) supported by broker
%7|1584538705.985|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature ZSTD: Fetch (10..10) supported by broker
%7|1584538705.985|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature ZSTD
%7|1584538705.986|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature UnitTest: SaslHandshake (1..1) supported by broker
%7|1584538705.986|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap:  Feature UnitTest: SaslAuthenticate (0..0) supported by broker
%7|1584538705.986|APIVERSION|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Enabling feature UnitTest
%7|1584538705.987|FEATURE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshak
e,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,UnitTest
%7|1584538705.987|STATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker changed state APIVERSION_QUERY -> UP
%7|1584538705.987|BROADCAST|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: Broadcasting state change
%7|1584538705.987|METADATA|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Request metadata for brokers only: connected
%7|1584538705.988|SEND|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Sent MetadataRequest (v2, 25 bytes @ 0, CorrId 2)
%7|1584538705.990|RECV|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received MetadataResponse (v2, 63 bytes, CorrId 2, rtt 2.80ms)
%7|1584538705.991|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ===== Received metadata: connected =====
%7|1584538705.991|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ClusterId: _sUFJdjPS0ijpVJjsqYbcg, ControllerId: 1001
%7|1584538705.991|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: 1 brokers, 0 topics
%7|1584538705.991|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap:   Broker #0/1: 192.168.178.157:9092 NodeId 1001
%7|1584538705.991|WAKEUPFD|rdkafka#producer-1| [thrd:main]: 192.168.178.157:9092/1001: Enabled low-latency ops queue wake-ups
%7|1584538705.991|BROKER|rdkafka#producer-1| [thrd:main]: 192.168.178.157:9092/1001: Added new broker with NodeId 1001
%7|1584538705.991|CLUSTERID|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ClusterId update "" -> "_sUFJdjPS0ijpVJjsqYbcg"
%7|1584538705.991|CONTROLLERID|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ControllerId update -1 -> 1001
%7|1584538705.991|BROADCAST|rdkafka#producer-1| [thrd:main]: Broadcasting state change
%7|1584538705.991|BRKMAIN|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enter main broker thread
%7|1584538706.011|TOPIC|rdkafka#producer-1| [thrd:app]: New local topic: dev.a
%7|1584538706.011|TOPPARNEW|rdkafka#producer-1| [thrd:app]: NEW dev.a [-1] 0x55666faafaa0 (at rd_kafka_topic_new0:393)
%7|1584538706.011|METADATA|rdkafka#producer-1| [thrd:app]: Hinted cache of 1/1 topic(s) being queried
%7|1584538706.011|METADATA|rdkafka#producer-1| [thrd:app]: Requesting metadata for 1/1 topics: leader query
%7|1584538706.011|METADATA|rdkafka#producer-1| [thrd:app]: kafka:9092/bootstrap: Request metadata for 1 topic(s): leader query
%7|1584538706.011|SEND|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Sent MetadataRequest (v2, 32 bytes @ 0, CorrId 3)
%7|1584538706.059|RECV|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received MetadataResponse (v2, 77 bytes, CorrId 3, rtt 47.72ms)
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ===== Received metadata (for 1 requested topics): leader query =====
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ClusterId: _sUFJdjPS0ijpVJjsqYbcg, ControllerId: 1001
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: 1 brokers, 1 topics
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap:   Broker #0/1: 192.168.178.157:9092 NodeId 1001
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap:   Topic #0/1: dev.a with 0 partitions: Broker: Leader not available
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: Temporary error in metadata reply for topic dev.a (PartCnt 0): Broker: Leader not available: ignoring
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: 1/1 requested topic(s) seen in metadata
%7|1584538706.059|METADATA|rdkafka#producer-1| [thrd:main]: Purged 1/1 cached topic hint(s)
%7|1584538706.951|NOINFO|rdkafka#producer-1| [thrd:main]: Topic dev.a metadata information unknown
%7|1584538706.951|NOINFO|rdkafka#producer-1| [thrd:main]: Topic dev.a partition count is zero: should refresh metadata
%7|1584538706.951|METADATA|rdkafka#producer-1| [thrd:main]: Requesting metadata for 1/1 topics: refresh unavailable topics
%7|1584538706.951|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: Request metadata for 1 topic(s): refresh unavailable topics
%7|1584538706.953|SEND|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Sent MetadataRequest (v2, 32 bytes @ 0, CorrId 4)
%7|1584538706.961|RECV|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received MetadataResponse (v2, 103 bytes, CorrId 4, rtt 8.44ms)
%7|1584538706.961|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ===== Received metadata (for 1 requested topics): refresh unavailable topics =====
%7|1584538706.961|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: ClusterId: _sUFJdjPS0ijpVJjsqYbcg, ControllerId: 1001
%7|1584538706.961|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: 1 brokers, 1 topics
%7|1584538706.962|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap:   Broker #0/1: 192.168.178.157:9092 NodeId 1001
%7|1584538706.962|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap:   Topic #0/1: dev.a with 1 partitions
%7|1584538706.962|STATE|rdkafka#producer-1| [thrd:main]: Topic dev.a changed state unknown -> exists
%7|1584538706.962|PARTCNT|rdkafka#producer-1| [thrd:main]: Topic dev.a partition count changed from 0 to 1
%7|1584538706.963|TOPPARNEW|rdkafka#producer-1| [thrd:main]: NEW dev.a [0] 0x55666fab0640 (at rd_kafka_topic_partition_cnt_update:703)
%7|1584538706.963|METADATA|rdkafka#producer-1| [thrd:main]:   Topic dev.a partition 0 Leader 1001
%7|1584538706.963|BRKDELGT|rdkafka#producer-1| [thrd:main]: dev.a [0]: delegate to broker 192.168.178.157:9092/1001 (rktp 0x55666fab0640, term 0, ref 2)
%7|1584538706.963|BRKDELGT|rdkafka#producer-1| [thrd:main]: dev.a [0]: delegating to broker 192.168.178.157:9092/1001 for partition with 0 messages (0 bytes) queued
%7|1584538706.963|BRKMIGR|rdkafka#producer-1| [thrd:main]: Migrating topic dev.a [0] 0x55666fab0640 from (none) to 192.168.178.157:9092/1001 (sending PARTITION_JOIN to 192.168.178.157:9092/1001)
%7|1584538706.963|PARTCNT|rdkafka#producer-1| [thrd:main]: Partitioning 1 unassigned messages in topic dev.a to 1 partitions
%7|1584538706.963|TOPBRK|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Topic dev.a [0]: joining broker (rktp 0x55666fab0640, 0 message(s) queued)
%7|1584538706.964|FETCHADD|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Added dev.a [0] to active list (1 entries, opv 0, 0 messages queued)
%7|1584538706.964|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538706.964|UAS|rdkafka#producer-1| [thrd:main]: 1/1 messages were partitioned in topic dev.a
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) in xmit queue (1 added from partition queue)
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) queued but broker not up
%7|1584538706.964|STATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker changed state INIT -> TRY_CONNECT
%7|1584538706.964|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538706.964|CONNECT|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: broker in state TRY_CONNECT connecting
%7|1584538706.964|STATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker changed state TRY_CONNECT -> CONNECT
%7|1584538706.964|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538706.964|METADATA|rdkafka#producer-1| [thrd:main]: kafka:9092/bootstrap: 1/1 requested topic(s) seen in metadata
%7|1584538706.964|CONNECT|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Connecting to ipv4#192.168.178.157:9092 (plaintext) with socket 16
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) queued but broker not up
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1584538706.964|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) queued but broker not up
%7|1584538706.965|CONNECT|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Connected to ipv4#192.168.178.157:9092
%7|1584538706.965|CONNECTED|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Connected (#1)
%7|1584538706.965|FEATURE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1584538706.965|STATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1584538706.965|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538706.965|SEND|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Sent ApiVersionRequest (v0, 25 bytes @ 0, CorrId 1)
%7|1584538706.965|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1584538706.965|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) queued but broker not up
%7|1584538706.981|RECV|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Received ApiVersionResponse (v0, 294 bytes, CorrId 1, rtt 15.76ms)
%7|1584538706.981|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker API support:
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Produce (0) Versions 0..8
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Fetch (1) Versions 0..11
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Offset (2) Versions 0..5
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Metadata (3) Versions 0..9
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey LeaderAndIsr (4) Versions 0..4
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey StopReplica (5) Versions 0..2
%7|1584538706.983|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey UpdateMetadata (6) Versions 0..6
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey ControlledShutdown (7) Versions 0..3
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey OffsetCommit (8) Versions 0..8
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey OffsetFetch (9) Versions 0..6
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey FindCoordinator (10) Versions 0..3
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey JoinGroup (11) Versions 0..6
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Heartbeat (12) Versions 0..4
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey LeaveGroup (13) Versions 0..4
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey SyncGroup (14) Versions 0..4
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DescribeGroups (15) Versions 0..5
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey ListGroups (16) Versions 0..3
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey SaslHandshake (17) Versions 0..1
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey ApiVersion (18) Versions 0..3
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey CreateTopics (19) Versions 0..5
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DeleteTopics (20) Versions 0..4
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DeleteRecords (21) Versions 0..1
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey InitProducerId (22) Versions 0..2
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey OffsetForLeaderEpoch (23) Versions 0..3
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey AddPartitionsToTxn (24) Versions 0..1
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey AddOffsetsToTxn (25) Versions 0..1
%7|1584538706.984|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey EndTxn (26) Versions 0..1
%7|1584538706.986|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey WriteTxnMarkers (27) Versions 0..0
%7|1584538706.986|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey TxnOffsetCommit (28) Versions 0..2
%7|1584538706.986|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DescribeAcls (29) Versions 0..1
%7|1584538706.986|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey CreateAcls (30) Versions 0..1
%7|1584538706.987|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DeleteAcls (31) Versions 0..1
%7|1584538706.987|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DescribeConfigs (32) Versions 0..2
%7|1584538706.987|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey AlterConfigs (33) Versions 0..1
%7|1584538706.987|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey AlterReplicaLogDirs (34) Versions 0..1
%7|1584538706.987|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DescribeLogDirs (35) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey SaslAuthenticate (36) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey CreatePartitions (37) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey CreateDelegationToken (38) Versions 0..2
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey RenewDelegationToken (39) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey ExpireDelegationToken (40) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DescribeDelegationToken (41) Versions 0..1
%7|1584538706.989|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey DeleteGroups (42) Versions 0..2
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Unknown-43? (43) Versions 0..2
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Unknown-44? (44) Versions 0..1
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Unknown-45? (45) Versions 0..0
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Unknown-46? (46) Versions 0..0
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:   ApiKey Unknown-47? (47) Versions 0..0
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature MsgVer1: Produce (2..2) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature MsgVer1: Fetch (2..2) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature MsgVer1
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature MsgVer2: Produce (3..3) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature MsgVer2: Fetch (4..4) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature MsgVer2
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature ApiVersion
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerGroupCoordinator: FindCoordinator (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature BrokerGroupCoordinator
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: FindCoordinator (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature BrokerBalancedConsumer
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature ThrottleTime: Produce (1..2) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature ThrottleTime
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature Sasl: JoinGroup (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature Sasl
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature SaslHandshake
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature LZ4: FindCoordinator (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature LZ4
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature OffsetTime: Offset (1..1) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature OffsetTime
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature IdempotentProducer: InitProducerId (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature IdempotentProducer
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature ZSTD: Produce (7..7) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature ZSTD: Fetch (10..10) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature ZSTD
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature UnitTest: SaslHandshake (1..1) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001:  Feature UnitTest: SaslAuthenticate (0..0) supported by broker
%7|1584538706.990|APIVERSION|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Enabling feature UnitTest
%7|1584538706.990|FEATURE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,Sa
slHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,ZSTD,UnitTest
%7|1584538706.990|STATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker changed state APIVERSION_QUERY -> UP
%7|1584538706.990|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538706.990|METADATA|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Skipping metadata refresh of 1 topic(s): connected: already being requested
%7|1584538706.990|TOPPAR|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0] 1 message(s) in xmit queue (0 added from partition queue)
%7|1584538706.990|PRODUCE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0]: Produce MessageSet with 1 message(s) (78 bytes, ApiVersion 7, MsgVersion 2, MsgId 0, Bas
eSeq -1, PID{Invalid}, uncompressed)
%7|1584538706.991|SEND|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Sent ProduceRequest (v7, 130 bytes @ 0, CorrId 2)
%7|1584538707.008|RECV|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Received ProduceResponse (v7, 49 bytes, CorrId 2, rtt 17.64ms)
%7|1584538707.008|MSGSET|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: dev.a [0]: MessageSet with 1 message(s) (MsgId 0, BaseSeq -1) delivered
%7|1584538707.140|DESTROY|rdkafka#producer-1| [thrd:app]: Terminating instance (destroy flags none (0x0))
%7|1584538707.141|TERMINATE|rdkafka#producer-1| [thrd:app]: Interrupting timers
%7|1584538707.141|TERMINATE|rdkafka#producer-1| [thrd:app]: Sending TERMINATE to internal main thread
%7|1584538707.141|TERMINATE|rdkafka#producer-1| [thrd:app]: Joining internal main thread
%7|1584538707.142|TERMINATE|rdkafka#producer-1| [thrd:main]: Internal main thread terminating
%7|1584538707.142|DESTROY|rdkafka#producer-1| [thrd:main]: Destroy internal
%7|1584538707.142|BROADCAST|rdkafka#producer-1| [thrd:main]: Broadcasting state change
%7|1584538707.142|DESTROY|rdkafka#producer-1| [thrd:main]: Removing all topics
%7|1584538707.142|PARTCNT|rdkafka#producer-1| [thrd:main]: Topic dev.a partition count changed from 1 to 0
%7|1584538707.143|REMOVE|rdkafka#producer-1| [thrd:main]: dev.a [0] no longer reported in metadata
%7|1584538707.144|BRKMIGR|rdkafka#producer-1| [thrd:main]: dev.a [0] 0x55666fab0640 sending final LEAVE for removal by 192.168.178.157:9092/1001
%7|1584538707.145|TOPPARREMOVE|rdkafka#producer-1| [thrd:main]: Removing toppar dev.a [-1] 0x55666faafaa0
%7|1584538707.146|DESTROY|rdkafka#producer-1| [thrd:main]: dev.a [-1]: 0x55666faafaa0 DESTROY_FINAL
%7|1584538707.146|DESTROY|rdkafka#producer-1| [thrd:main]: Sending TERMINATE to 192.168.178.157:9092/1001
%7|1584538707.147|DESTROY|rdkafka#producer-1| [thrd:main]: Sending TERMINATE to kafka:9092/bootstrap
%7|1584538707.147|TERMINATE|rdkafka#producer-1| [thrd:main]: Purging reply queue
%7|1584538707.145|TOPBRK|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Topic dev.a [0]: leaving broker (0 messages in xmitq, next broker (none), rktp 0x55666fab0640)
%7|1584538707.147|FETCHADD|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Removed dev.a [0] from active list (0 entries, opv 0)
%7|1584538707.147|TOPBRK|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Topic dev.a [0]: no next broker, failing 0 message(s) in partition queue
%7|1584538707.147|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538707.147|TOPPARREMOVE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Removing toppar dev.a [0] 0x55666fab0640
%7|1584538707.147|DESTROY|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: dev.a [0]: 0x55666fab0640 DESTROY_FINAL
%7|1584538707.147|TERM|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Received TERMINATE op in state UP: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs
%7|1584538707.147|BROKERFAIL|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: failed: err: Local: Broker handle destroyed: (errno: Resource temporarily unavailable)
%7|1584538707.147|TERMINATE|rdkafka#producer-1| [thrd:main]: Decommissioning internal broker
%7|1584538707.147|TERMINATE|rdkafka#producer-1| [thrd:main]: Join 3 broker thread(s)
%7|1584538707.147|TERM|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Received TERMINATE op in state INIT: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs
%7|1584538707.148|BROKERFAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: failed: err: Local: Broker handle destroyed: (errno: No error information)
%7|1584538707.148|FAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Client is terminating (after 1196ms in state INIT)
%7|1584538707.148|STATE|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Broker changed state INIT -> DOWN
%7|1584538707.148|BROADCAST|rdkafka#producer-1| [thrd::0/internal]: Broadcasting state change
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Purging bufq with 0 buffers
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Purging bufq with 0 buffers
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Updating 0 buffers on connection reset
%7|1584538707.148|TERMINATE|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Handle is terminating in state DOWN: 1 refcnts (0x55666faad750), 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs: failed 0 request(s) in retry+outbuf
%7|1584538707.148|BROKERFAIL|rdkafka#producer-1| [thrd::0/internal]: :0/internal: failed: err: Local: Broker handle destroyed: (errno: No error information)
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Purging bufq with 0 buffers
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Purging bufq with 0 buffers
%7|1584538707.148|BUFQ|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Updating 0 buffers on connection reset
%7|1584538707.149|TERM|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Received TERMINATE op in state UP: 1 refcnts, 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs
%7|1584538707.149|BROKERFAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: failed: err: Local: Broker handle destroyed: (errno: Resource temporarily unavailable)
%7|1584538707.158|FAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Client is terminating (after 1170ms in state UP)
%7|1584538707.158|STATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Broker changed state UP -> DOWN
%7|1584538707.158|BROADCAST|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: Broadcasting state change
%7|1584538707.158|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Purging bufq with 0 buffers
%7|1584538707.158|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Purging bufq with 0 buffers
%7|1584538707.158|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Updating 0 buffers on connection reset
%7|1584538707.158|TERMINATE|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Handle is terminating in state DOWN: 1 refcnts (0x55666faae350), 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs: failed 0 request(s) in retry+outbuf
%7|1584538707.158|BROKERFAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: failed: err: Local: Broker handle destroyed: (errno: Resource temporarily unavailable)
%7|1584538707.158|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Purging bufq with 0 buffers
%7|1584538707.159|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Purging bufq with 0 buffers
%7|1584538707.159|BUFQ|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Updating 0 buffers on connection reset
%7|1584538707.164|FAIL|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Client is terminating (after 173ms in state UP)
%7|1584538707.164|STATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Broker changed state UP -> DOWN
%7|1584538707.164|BROADCAST|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: Broadcasting state change
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Purging bufq with 0 buffers
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Purging bufq with 0 buffers
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Updating 0 buffers on connection reset
%7|1584538707.164|TERMINATE|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Handle is terminating in state DOWN: 1 refcnts (0x55666faaf230), 0 toppar(s), 0 active toppar(s), 0 outbufs, 0 waitresps, 0 retrybufs: failed 0 request(s) in retry+outbuf
%7|1584538707.164|BROKERFAIL|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: failed: err: Local: Broker handle destroyed: (errno: Resource temporarily unavailable)
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Purging bufq with 0 buffers
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Purging bufq with 0 buffers
%7|1584538707.164|BUFQ|rdkafka#producer-1| [thrd:192.168.178.157:9092/1001]: 192.168.178.157:9092/1001: Updating 0 buffers on connection reset
%7|1584538707.165|TERMINATE|rdkafka#producer-1| [thrd:main]: Internal main thread termination done
%7|1584538707.165|TERMINATE|rdkafka#producer-1| [thrd:app]: Destroying op queues
%7|1584538707.166|TERMINATE|rdkafka#producer-1| [thrd:app]: Termination done: freeing resources
Fatal error: Couldn't execute method Exception::__tostring in Unknown on line 0
[18-Mar-2020 13:38:27] WARNING: [pool www] child 10 exited with code 70 after 70.591586 seconds from start
[18-Mar-2020 13:38:27] NOTICE: [pool www] child 29 started
question wait-info
Source
picture minitauros
👍1

Most helpful comment

Guys, thanks for the swift replies.

I've updated a few other packages and stripped down the environment to the bare required minimum. I did not expect anything that was now removed could be interfering with rdkafka. Maybe I've missed something. Exception is not happening. Please pause putting effort into this. I will update once I find out more.

picture minitauros on 18 Mar 2020
👍2

All 15 comments

At first glance, i was unable to deduct what happens in the log.
Also i was unable to reproduce this:

PHP: php7.4.3
lib: librdkafka:1.3.0
system: alpine:3.11

i used the same code as you (some replacements):

<?php

$config = new \RdKafka\Conf();
$producer = new \RdKafka\Producer($config);
$producer->addBrokers('kafka:9092');
$topic = $producer->newTopic('test');
nick-zh picture nick-zh on 18 Mar 2020

@minitauros did it by any chance produce a core file?

nick-zh picture nick-zh on 18 Mar 2020

Yeah what looked interesting to me (but honestly I know not a lot about the Kafka internals) is

%7|1584538707.142|DESTROY|rdkafka#producer-1| [thrd:main]: Removing all topics
%7|1584538707.142|PARTCNT|rdkafka#producer-1| [thrd:main]: Topic dev.a partition count changed from 1 to 0

"Removing all topics", it sounds quite extreme. While this script is running I'm listening to topic dev.a using Kafkacat and both consumption from and publication to it work fine using Kafkacat.

@minitauros did it by any chance produce a core file?

Where would I find such a core file?

minitauros picture minitauros on 18 Mar 2020

"Removing all topics"

It's just librdkafka internal queue purging. Client de-registers from Kafka broker to let it know that it won't handle more messages and other consumers can pick up that partition.

Where would I find such a core file?

PHP-FPM has to be running through gdb.

https://gist.github.com/guilhermeblanco/894d3288e17d4cd11f19f58d9c0fd253

Do not run on production, obviously.

EDIT: It's not really a segfault, PHP recovered by emitting a fatal error. We should probably look up all exception throws in phprdkafka to see if maybe they're missing something.

Steveb-p picture Steveb-p on 18 Mar 2020

@Steveb-p yeah that's what i thought too, there are quite a few, but so far i was unable to spot the odd one :(

nick-zh picture nick-zh on 18 Mar 2020

@Steveb-p yeah that's what i thought too, there are quite a few, but so far i was unable to spot the odd one :(

What about https://github.com/arnaud-lb/php-rdkafka/blob/cc11637e4530caca730f8a32a8d36fc1998eee8e/kafka_consumer.c#L137 ?

zend_replace_error_handling(EH_THROW, spl_ce_InvalidArgumentException, &error_handling TSRMLS_CC);

You did try just creating a method throwing an exception under PHP 7.4 and it worked ok? Maybe some macros changed?

Steveb-p picture Steveb-p on 18 Mar 2020

@Steveb-p that's code from the consumer, so probably can't be it.
good point, i can try to force an exception :+1:
Update: seems to work ok, i tried to set a config that doesn't exist...

nick-zh picture nick-zh on 18 Mar 2020

Guys, thanks for the swift replies.

I've updated a few other packages and stripped down the environment to the bare required minimum. I did not expect anything that was now removed could be interfering with rdkafka. Maybe I've missed something. Exception is not happening. Please pause putting effort into this. I will update once I find out more.

minitauros picture minitauros on 18 Mar 2020
👍2

@minitauros thanks for the update, keep us posted

nick-zh picture nick-zh on 18 Mar 2020

I replied too fast

Stop reading, this appears to only occur within the framework that we're using, I've been too fast with my answer 馃槵.

Will let you know when I find out more and either delete/update this message.

Ok, I have no idea why this is an issue, but I've managed to reproduce it.

The issue appears to happen only when the producer itself is constructed within the constructor of a class, then saved in a property, and then a topic is created in another method of that class and saved in another property.

Example:

Will cause fatal

Producer is constructed within constructor. Topic is created and added to property of class within publish method.

class Bla {
  /**
   * @var \RdKafka\ProducerTopic[]
   */
  private array $topics;

  private \RdKafka\Producer $producer;

  public function __construct() {
    $conf = new \RdKafka\Conf();
    $conf->set('debug', 'all');
    $producer = new \RdKafka\Producer($conf);
    $producer->addBrokers('kafka:9092');
    $this->producer = $producer;
  }

  public function publish(string $topic, string $distKey, array $payload): void {
    if (!isset($this->topics[$topic])) {
      $t = $this->producer->newTopic($topic);
      $this->topics[$topic] = $t;
    }

    $this->topics[$topic]->produce(RD_KAFKA_PARTITION_UA, 0, \json_encode($payload), $distKey);
    $this->producer->poll(1);
  }
}

$bla = new Bla();
$bla->publish('topic', 'a', ['a' => 'b']);

Will not cause fatal

Producer is constructed within constructor. Topic is created in publish method but not added to a class property.

class Bla {
  /**
   * @var \RdKafka\ProducerTopic[]
   */
  private array $topics;

  private \RdKafka\Producer $producer;

  public function __construct() {
    $conf = new \RdKafka\Conf();
    $conf->set('debug', 'all');
    $producer = new \RdKafka\Producer($conf);
    $producer->addBrokers('kafka:9092');
    $this->producer = $producer;
  }

  public function publish(string $topic, string $distKey, array $payload): void {
    $t = $this->producer->newTopic($topic);
    $t->produce(RD_KAFKA_PARTITION_UA, 0, \json_encode($payload), $distKey);
  }
}

$bla = new Bla();
$bla->publish('topic', 'a', ['a' => 'b']);

Will also not cause fatal

Producer is injected into constructor. Topic is added to class property in publish method.

class Bla {
  /**
   * @var \RdKafka\ProducerTopic[]
   */
  private array $topics;

  private \RdKafka\Producer $producer;

  public function __construct(\RdKafka\Producer $producer) {
    $this->producer = $producer;
  }

  public function publish(string $topic, string $distKey, array $payload): void {
    if (!isset($this->topics[$topic])) {
      $t = $this->producer->newTopic($topic);
      $this->topics[$topic] = $t;
    }

    $this->topics[$topic]->produce(RD_KAFKA_PARTITION_UA, 0, \json_encode($payload), $distKey);
    $this->producer->poll(1);
  }
}

$conf = new \RdKafka\Conf();
$conf->set('debug', 'all');
$producer = new \RdKafka\Producer($conf);
$producer->addBrokers('kafka:9092');

$bla = new Bla($producer);
$bla->publish('topic', 'a', ['a' => 'b']);

The examples were run within a PHP framework, but since I've been testing within the framework on and off I'm fairly sure that this will also happen outside of the framework.

minitauros picture minitauros on 18 Mar 2020

@minitauros haha no worries, i was just about to write, that i still can't reproduce it :smile:
thx for keeping us in the loop :+1:

nick-zh picture nick-zh on 18 Mar 2020

Yet Another Custom Framework, eh? :smiling_imp:

@minitauros does it happen when you instantiate RdKafka\Producer twice for the same broker maybe?

Stop reading, this appears to only occur within the framework that we're using, I've been too fast with my answer .

Curiosity got the better of me :smile:

Steveb-p picture Steveb-p on 18 Mar 2020

Yet Another Custom Framework, eh?

Haha, if only, but no, it's an old version of CodeIgniter.

I've spent a few more hours trying to find out what the hell is causing this issue, but I have still not found it and cannot allocate the time now to dig into it any further.

@minitauros does it happen when you instantiate RdKafka\Producer twice for the same broker maybe?

I thought the same, but no problems occur when I do this outside of the framework. Even if I create a custom class that is only loaded from my test file (within the framework) that constructs the producer within the class constructor (only once), code breaks. If I do the same thing outside the framework, it goes ok. If in the framework instead of constructing the producer inside the constructor of the test class, I pass it to it as an argument, all goes fine. I'm extremely confused, but, as I said, I can't spend more time on it now.

Hope you didn't spend/waste too much time on it. Thanks for the support! Please feel free to close this issue if you are done with it.

minitauros picture minitauros on 19 Mar 2020

Haha, if only, but no, it's an old version of CodeIgniter.

God have mercy on your soul.

I actually converted a legacy application in CodeIgniter to a Symfony hybrid: CodeIgniter starting up and booting Symfony - base boot after initialization, so I can access dependency injection container if needed, and full request lifecycle in cases where I've had rewritten controllers. This allowed me to transition from one to the other and now only a shell of CodeIgniter remains in my project.
I can share hooks if you like, but maybe not on github issue, but on Slack/Gitter instead :smile:

Steveb-p picture Steveb-p on 19 Mar 2020

Hah, sounds like one intense job 馃槄 . Thanks for the offer! But it's all fine, we're slowly migrating to Go services, so there's no need to rewrite the whole thing.

And thanks again for the support!

minitauros picture minitauros on 19 Mar 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

godtou picture godtou  路  10Comments
maks-rafalko picture maks-rafalko  路  9Comments
PopovAnatolii picture PopovAnatolii  路  8Comments
godtou picture godtou  路  5Comments
aand18 picture aand18  路  10Comments