Schema-registry: Nullable string field errors out with "Non-string default value for string: null"

[benjaminoakes]

Version: schema-registry v3.3.0

Use Case

I wanted to add a field like this:

{ "name": "foobar", "type": ["string", "null"], "default": null },

Expected Behavior

The field would pass compatibility checking without errors. Or if this is actually invalid, a HTTP 4XX response with an explanation. (Related documentation: default fields with union types and unions.)

Actual Behavior

I received an HTTP 500.

This is the response body I received when compatibility checking:

{"error_code":500,"message":"Internal Server Error"}

And this was found in the logs:

org.apache.avro.AvroTypeException: Non-string default value for string: null

Workaround

Reversing the types allows a default of null to be used:

{ "name": "foobar", "type": ["null", "string"], "default": null },

Another possible workaround is using the empty string as the default, but our preference is null because it's a nullable field.

{ "name": "foobar", "type": ["string", "null"], "default": "" },

[benjaminoakes]

This may be a difficult bug to reproduce. I was attempting to put together a minimal example and that was proving to be more challenging than expected. I'm considering other options.

[tjwp]

The workarounds listed are what you are required to use based on the Avro spec:

_default: A default value for this field, used when reading instances that lack this field (optional).
Permitted values depend on the field's schema type, according to the table below. Default values
for union fields correspond to the first schema in the union._

[benjaminoakes]

Understood.

However, it probably shouldn't respond with "HTTP 500 Internal Server Error". I would expect a more helpful response instead, like "HTTP 422" with a message like "Non-string default value for string: null".

[wernerdaehn]

I would simply state, if there is an Avro error like this, the error message should not only go into the /logs/schema-registry.log but also be returned in the http request.

Scenario is simple to reproduce: Extend an existing schema with the global compatibility set to none or extend it with a not-null column when compatibility is forward/full.

[mkwhitacre]

Seems like it should be getting mapped to a 422(status)/42201(error code) based on the API contract.
https://docs.confluent.io/current/schema-registry/docs/api.html#post--subjects-(string-%20subject)-versions

[xiaoyali97]

Closing it because it will be fixed in 6.1.0.