Ksql: Support a VARBINARY, BINARY, or BYTES data type

[miguno]

The idea is to provide sth similar to a byte[] (byte array) data type.

Example of SQL Server:

• binary [ ( n ) ] Fixed-length binary data with a length of n bytes, where n is a value from 1 through 8,000. The storage size is n bytes.
• varbinary [ ( n | max) ] Variable-length binary data. n can be a value from 1 through 8,000. max indicates that the maximum storage size is 2^31-1 bytes (2 GB). The storage size is the actual length of the data entered + 2 bytes. The data that is entered can be 0 bytes in length. The ANSI SQL synonym for varbinary is binary varying.

Use cases

• To store and work with BLOBs of data.
• To read and/or write a data field without requiring (de)serialization
  
  
  
  • Example: to efficiently and directly copy a data field 1:1 from an input stream to an output stream, where efficiency is achieved by not requiring (de)serialization of the field
  
  

[apurvam]

We already have #1282

[miguno]

For reference, https://github.com/confluentinc/ksql/issues/1282 is "Add support for the "bytes" type in avro".

[apurvam]

Should we close one of them as a duplicate? This one seems to have more context and is more general.

[miguno]

Yes, I closed https://github.com/confluentinc/ksql/issues/1282 as a duplicate of this one.

[DW-Pete]

This would be help for me when creating a stream over an Oracle Golden Gate topic containing image data - I don't need to process it, just pass it as is to the consumer

[ehrlichja]

+1 for this. Use case: we would like to have the key as a binary, and then have the JDBC connector sink the topic into MySQL, where the PK is a binary.

[muirandy]

+1 I've got a Debezium connector reading stuff in from Oracle. Some data comes through onto the topic as a byte array. I'd like to have a custom UDF to turn this into something more useful. However, I can't see the data and I believe it is because KSQL doesn't yet play with byte arrays.

[pavan-choudhary]

@apurvam @miguno Do we have any update for using binaries such as JSONB in KSQL or KSQLDB ?

[aajacobs]

+1 I've got Oracle data coming in and want to easily separate out the BLOB data from the rest of the record.