Ksql: VALUE_AVRO_SCHEMA_FULL_NAME does not appear to work

[gpaniagua013]

I am trying to set up the schema namespace and name of a topic created by KSQL. Base in the documentation I can do this in the following way:

"
By default, KSQL registered avro schemas have the same name (KsqlDataSourceSchema) and the same namespace (io.confluent.ksql.avro_schemas). You can override this behaviour by providing an VALUE_AVRO_SCHEMA_FULL_NAME property in the WITH clause where you set the VALUE_FORMAT to 'AVRO'. As the name suggests, this property will override the default name/namespace with the provided one. For Example, com.mycompany.MySchema registers a schema with the MySchema name and the com.mycompany namespace.
"

In am trying to do this:

CREATE STREAM Test_GameStart_Rekeyed_KS WITH (VALUE_FORMAT='AVRO', VALUE_AVRO_SCHEMA_FULL_NAME='com.mycompany.MySchema', PARTITIONS=128) AS SELECT column_sample AS column_sample FROM [TABLE] PARTITION BY key;

And I'm getting this error:

io.confluent.ksql.util.KsqlException: Invalid config variable in the WITH clause: VALUE_AVRO_SCHEMA_FULL_NAME.

[xs005]

I just tried the same command on ksql-server 5.2.2 and it works well. Maybe you need to update ksql-server. Are you familiar with KSQL logging, I am trying to include the row data but cannot make it, need some help.

And I am curious why do you need your own namespace for the avro schema?

[gpaniagua013]

Well, I am using the latest docker image of KSQL. I am not familiar with KSQL logging.

I have a .net consumer and at the moment of the deserialization, I am getting a schema mismatch error, because we have our own namespace for the .net Avro entities. Like a workaround, we had to create the entities in the that KSQL namespace and the same name. But I don't like it.

[xs005]

Try 5.2.2 for testing. I always use the version a little bit earlier.

[gpaniagua013]

Yes in that version is working. Thanks.

[marghidanu]

I tried this option in 5.3.1 and it doesn't seem to work. Is there a way to specify the schema subject name when creating a stream?

[rmoff]

Trying this out in ksqlDB 0.6.

Create two new streams, one using the config, one not:

ksql> CREATE STREAM TEST_AVRO WITH (VALUE_AVRO_SCHEMA_FULL_NAME='FOO', VALUE_FORMAT='AVRO')
AS SELECT * FROM TEST;


 Message
----------------------------------------------------------------------------------------
 Stream TEST_AVRO created and running. Created by query with query ID: CSAS_TEST_AVRO_4
----------------------------------------------------------------------------------------
ksql>
ksql> CREATE STREAM TEST_AVRO2 WITH (VALUE_FORMAT='AVRO') AS SELECT * FROM TEST;

 Message
------------------------------------------------------------------------------------------
 Stream TEST_AVRO2 created and running. Created by query with query ID: CSAS_TEST_AVRO2_5
------------------------------------------------------------------------------------------
ksql>

Both get created with the standard subject name, and VALUE_AVRO_SCHEMA_FULL_NAME is not used:

$ curl -s -XGET localhost:8081/subjects/
["TEST_AVRO2-value","TEST_AVRO-value"]

[big-andy-coates]

FYI, VALUE_AVRO_SCHEMA_FULL_NAME doesn't change the name of the subject in the schema registry, so TEST_AVR02-value and TEST_AVRO-value are what I'd expect to see.

What VALUE_AVRO_SCHEMA_FULL_NAME changes is the name of the record _within_ the schema. So you'd need to pull back the content of both of those schemas to see if there is an actual issue or not.