Clickhouse: How to configure different Kafka clusters in clickhouse ?

[hoseiney]

we have 2 Kafka clusters, in one of them we are setting up SASL security and another one does not have it.
The Kafka configuration is global, how we should deal with different Kafka cluster configuration ?

[filimonov]

You can make different configurations per topic in config.xml, like that:

<yandex>
    <kafka>
      <!-- global configuration -->
    </kafka>

    <kafka_events>
     <!-- configuration for topic 'events' -->
    </kafka_events>

    <kafka_users>
     <!-- configuration for topic 'users' -->
    </kafka_users>
</yandex>

Bit more flexible solution for that (passing any rdkafka settings during table creation) is planned.

[hoseiney]

it is good to have customized config for each topic
but we are using Debezium and topics name contain dot
for example : mysql.users
can we have this config ?

<kafka_mysql.users>
</kafka_mysql.users>

[filimonov]

No i think it will not work :\ Dot has a special meaning in used XML parser.