Strimzi-kafka-operator: testing strimzi

[thbeh]

Hi, I did a quick test with Strimzi (following the docs) in kubernetes and use the following to test - _kubectl exec -it my-cluster-kafka-0 -- bin/kafka-topics.sh --zookeeper my-cluster-zookeeper-client:2181 --list_ but showing errors like below:

_[2018-08-05 06:38:22,559] WARN Session 0x0 for server my-cluster-zookeeper-client.default.svc.cluster.local/10.108.93.48:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
[2018-08-05 06:38:24,403] WARN Session 0x0 for server my-cluster-zookeeper-client.default.svc.cluster.local/10.108.93.48:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
....._

My k8s services -

Name: my-cluster-kafka-bootstrap
Namespace: default
Labels: strimzi.io/cluster=my-cluster
strimzi.io/kind=Kafka
strimzi.io/name=my-cluster-kafka-bootstrap
Annotations: prometheus.io/path=/metrics
prometheus.io/port=9404
prometheus.io/scrape=true
Selector: strimzi.io/cluster=my-cluster,strimzi.io/kind=Kafka,strimzi.io/name=my-cluster-kafka
Type: ClusterIP
IP: 10.107.97.230
Port: clients 9092/TCP
TargetPort: 9092/TCP
Endpoints: 10.244.1.28:9092,10.244.2.55:9092,10.244.3.56:9092
Port: clientstls 9093/TCP
TargetPort: 9093/TCP
Endpoints: 10.244.1.28:9093,10.244.2.55:9093,10.244.3.56:9093
Port: replication 9091/TCP
TargetPort: 9091/TCP
Endpoints: 10.244.1.28:9091,10.244.2.55:9091,10.244.3.56:9091
Port: kafkametrics 9404/TCP
TargetPort: 9404/TCP
Endpoints: 10.244.1.28:9404,10.244.2.55:9404,10.244.3.56:9404
Session Affinity: None
Events:

Name: my-cluster-kafka-brokers
Namespace: default
Labels: strimzi.io/cluster=my-cluster
strimzi.io/kind=Kafka
strimzi.io/name=my-cluster-kafka-brokers
Annotations: service.alpha.kubernetes.io/tolerate-unready-endpoints=true
Selector: strimzi.io/cluster=my-cluster,strimzi.io/kind=Kafka,strimzi.io/name=my-cluster-kafka
Type: ClusterIP
IP: None
Port: clients 9092/TCP
TargetPort: 9092/TCP
Endpoints: 10.244.1.28:9092,10.244.2.55:9092,10.244.3.56:9092
Port: replication 9091/TCP
TargetPort: 9091/TCP
Endpoints: 10.244.1.28:9091,10.244.2.55:9091,10.244.3.56:9091
Port: clientstls 9093/TCP
TargetPort: 9093/TCP
Endpoints: 10.244.1.28:9093,10.244.2.55:9093,10.244.3.56:9093
Session Affinity: None
Events:

Name: my-cluster-zookeeper-client
Namespace: default
Labels: strimzi.io/cluster=my-cluster
strimzi.io/kind=Kafka
strimzi.io/name=my-cluster-zookeeper-client
Annotations: prometheus.io/path=/metrics
prometheus.io/port=9404
prometheus.io/scrape=true
Selector: strimzi.io/cluster=my-cluster,strimzi.io/kind=Kafka,strimzi.io/name=my-cluster-zookeeper
Type: ClusterIP
IP: 10.108.93.48
Port: metrics 9404/TCP
TargetPort: 9404/TCP
Endpoints: 10.244.1.27:9404,10.244.2.54:9404,10.244.3.55:9404
Port: clients 2181/TCP
TargetPort: 2181/TCP
Endpoints: 10.244.1.27:2181,10.244.2.54:2181,10.244.3.55:2181
Session Affinity: None
Events:

Name: my-cluster-zookeeper-nodes
Namespace: default
Labels: strimzi.io/cluster=my-cluster
strimzi.io/kind=Kafka
strimzi.io/name=my-cluster-zookeeper-nodes
Annotations: service.alpha.kubernetes.io/tolerate-unready-endpoints=true
Selector: strimzi.io/cluster=my-cluster,strimzi.io/kind=Kafka,strimzi.io/name=my-cluster-zookeeper
Type: ClusterIP
IP: None
Port: clients 2181/TCP
TargetPort: 2181/TCP
Endpoints: 10.244.1.27:2181,10.244.2.54:2181,10.244.3.55:2181
Port: clustering 2888/TCP
TargetPort: 2888/TCP
Endpoints: 10.244.1.27:2888,10.244.2.54:2888,10.244.3.55:2888
Port: leader-election 3888/TCP
TargetPort: 3888/TCP
Endpoints: 10.244.1.27:3888,10.244.2.54:3888,10.244.3.55:3888
Session Affinity: None
Events:

[ppatierno]

Hi, are you working with master or 0.5.0? In both cases, the Zookeeper connection on port 2181 is encrypted. Due to the fact that the Zookeeper version distributed with Kafka doesn't support TLS, it's provided with an "stunnel" sidecar container running alongside the Zookeeper one. The sidecar is listening on port 2181.
For using the kafka-topics.sh script I suggest to use the following command:

kubectl exec -it my-cluster-zookeeper-0 -- bin/kafka-topics.sh --zookeeper localhost:21810 --list

The command runs directly on one of the Zookeeper nodes and more specifically on the 0 node which listens on port 21810 locally.
In general if you want to use a different Zookeeper node with index "X" the port to use is determined by this formula 2181 * 10 + X.

[thbeh]

Thanks, that works. I guess creating producer/consumer will be the same using kubectl as I couldn't find any examples. Or is there a different way?

[thbeh]

Regarding the Zookeeper above, in that case would using an external zookeeper cluster makes more sense or using --zookeeper zk1:21810,zk2:21811,zk:3:21812 ?

[ppatierno]

For Kafka clients connecting from inside the OpenShift/Kubernetes cluster (because Kafka isn't exposed outside yet) you can follow the guidelines here :

http://strimzi.io/docs/0.5.0/#kafka_client_connections_via_tls

[thbeh]

is it possible to share strimzi/kafka-connect:0.5.0 Dockerfile?

[ppatierno]

what do you mean by "sharing". The source is in the repo at https://github.com/strimzi/strimzi-kafka-operator/blob/master/docker-images/kafka-connect/Dockerfile

[thbeh]

my apologies, I missed that.
i see in the documentation in OpenShift there's OpenShift Build to add additional connectors, I guess to add connectors in k8s is through re building the Dockerfile, am I correct?

[ppatierno]

Yes it's mostly explained in the documentation http://strimzi.io/docs/0.5.0/#using_kafka_connect_with_additional_plugins

[thbeh]

Strimzi development being part of Redhat , would there be features bias on OpenShift compare to native Kubernetes. Advice?

[ppatierno]

No, it will not be biased. You can also notice that all system tests on Travis CI run on minikube :-) and now we have Helm charts integration as well.
The only part which is related to OpenShift is the possibility to add more connectors to Kafka Connect image using the S2I feature provided by OpenShift.