Azure-sdk-for-java: Azure Core Custom Json Deserialization

[navalev]

Deserialization of REST API responses using azure-core JacksonAdapter does not allow deserializing into strongly typed class. Currently when using RestProxy service, the AdditionalPropertiesDeserializer automatically converts all "additionalProperties" fields into a property bag (ie a propery:value map).

Currently, users of the response have to convert this additionalProperties map to the desired object type, thus adding a second step to the deserilization (with potential impact of performance).

It would be great to be able to inject our own custom deserializer, maybe when the RestProxy service is created (when initializing the JacksonAdapter).
And the optimal solution would be to allow the additionalProperties type to be determined by the requester, and not set to map type (possibly use generics)

[anuchandy]

I want to share a possible approach to solve this.

Background:

So far the "additionalProperties" attribute were used to accommodate new wire properties from service response those does not have mapping property defined in the swagger model yet. This enable user to still access such properties while waiting for next version of SDK based on an updated-swagger that has a real mapping properties defined.

An example for Azure search entity with "additionalProperties" is SearchResult. Today the generated code looks like:

public final class SearchResult {
    // additionalProperties
    @JsonProperty(value = "")
    private Map<String, Object> additionalProperties;
 
    public Map<String, Object> additionalProperties() {
        return this.additionalProperties;
    }

    // score
    @JsonProperty(value = "@search.score", access = JsonProperty.Access.WRITE_ONLY)
    private double score;
 
    public double score() {
        return this.score;
    }
 
    // highlights
    @JsonProperty(value = "@search.highlights", access = JsonProperty.Access.WRITE_ONLY)
    private Map<String, List<String>> highlights;
 
    public Map<String, List<String>> highlights() {
        return this.highlights;
    }
}

In the context of Search SDK, the "additionalProperties" has special meaning, it is not simply extra properties from the service but the group of these extra properties has a "Runtime Defined Shape". In case of SearchResult it's the Document.

The problem to solve can be defined as: if SDK returns a Model instance that has extra properties with "runtime shape" then we need a way for user to provide the runtime shape and have those extra properties deserialized to an instance of provided shape.

One approach to solve:

We let code-generator knows about these type of Swagger Models having properties with runtime shape. This could be done using swagger readme file. This result in code generator to emit model with some state information, for e.g. the SearchResult model from Azure search looks like:

public final class SearchResult {
 
    // Begin: Effect of hasRuntimeShape attribute in swagger readme
    private String additionalProperties;
    private com.azure.core.implementation.serializer.SerializerAdapter adapter;
 
    public <T> T getValue(Class<T> class) {
        return (T) this.adapter.deserialize(this.additionalProperties, class);
    }
    // End:
 
    // score
    @JsonProperty(value = "@search.score", access = JsonProperty.Access.WRITE_ONLY)
    private double score;
 
    public double score() {
        return this.score;
    }
 
    // highlights
    @JsonProperty(value = "@search.highlights", access = JsonProperty.Access.WRITE_ONLY)
    private Map<String, List<String>> highlights;
 
    public Map<String, List<String>> highlights() {
        return this.highlights;
    }
}

Here type of additionalProperties field is String instead of Map<> and is private and has no getter. In addition there is a private field of azure-core SerializerAdapter type.

When azure-core deserializes APICall result to SearchResult,

(a). It inject the Serializer adapter that core uses and.
(b). Over the wire extra properties become a json string and become value of additionalProperties member variable.

There is this generic method getValue() whose type can be specified during runtime. e.g.:

SearchResult result = apiCall();
Hotel hotel = result.<>getValue(Hotel.class);

Where Hotel is the runtime shape that only user knows.

This should address the first requirement that @navalev mentioned: "optimal solution would be to allow the additionalProperties type to be determined by the requester"

The second requirement - the support for custom deserializer is bit involved as it may requires us to define a Jason/xml parser & annotations, because we don't want to leak Jackson. @navalev how important/urgent this use case is for search customer?

[anuchandy]

\\cc @JonathanGiles @joshfree FYI.

[anuchandy]

In the above SearchResult type I really like to see:

public <T> T getValue() {...}

instead of:

public <T> T getValue(Class<T> cls) {...}

but that seems not possible in Java due to type-erasure.

[navalev]

@anuchandy custom deserializer is not an P1 requirement, and can be easily resolved if we will provide the user the original additionalProperties (as a string/property bag). In this case the user will be able to deserialise it manually and create any customisation they require. Also it will keep the data integrity - in case the class is different than the response json we will not lose any data.
Ideally if we will be able to provide the 2 options - one with strongly typed response, second with the full property bag

[anuchandy]

I will pick this after v1 GA, parking it for now.

[alzimmermsft]

Pluggable serialization for Search has been completed in #13496