Axonframework: Query a list of DTO (pojo class) returns a list of LinkedHashMap

[Blackdread]

Kind of very weird issue, I am still trying to investigate.

I use Kotlin 1.3.61, spring boot 2.2.4, jackson 2.10.2, axon 4.2.1, jdk 12, axon server.
Multi module maven project, let's call them "query" for read side, "file-context" for file aggregate and "rest" for rest controllers.

All modules are configured to use jackson (messages, events, etc).

Payload from debug:

payload {
  type: "java.util.ArrayList"
  data: "[{\"fileId\":\"e2f8fe8d-511b-4beb-865f-da1c71e52a23\",\"url\":\"http://localhost:8080/myKye\",\"savedFilenameKey\":\"myKye\",\"extension\":\"png\",\"originalFilename\":\"png\",\"description\":\"png\",\"tags\":[\"tag1\",\"tag2\"],\"metadata\":{\"key1\":\"val1\",\"key2\":\"val2\"}},{\"fileId\":\"d8b6ecf9-042c-4f5e-926e-a13736efb98c\",\"url\":\"http://localhost:8080/myKye\",\"savedFilenameKey\":\"myKye\",\"extension\":\"png\",\"originalFilename\":\"png\",\"description\":\"png\",\"tags\":[\"tag1\",\"tag2\"],\"metadata\":{\"key1\":\"val1\",\"key2\":\"val2\"}}]"
}

Switched to use data class for Ids instead of inline

interface Id {
    val identifier: String
}

class IdSerializer : JsonSerializer<Id>() {
    override fun serialize(value: Id, gen: JsonGenerator, serializers: SerializerProvider) {
        gen.writeString(value.identifier)
    }
}

@JsonDeserialize(using = FileIdDeserializer::class)
@JsonSerialize(using = IdSerializer::class)
data class FileId(override val identifier: String = IdentifierFactory.getInstance().generateIdentifier()) : Id {
    override fun toString(): String {
        return identifier
    }
}

class FileIdDeserializer : JsonDeserializer<FileId>() {
    override fun deserialize(p: JsonParser, ctxt: DeserializationContext): FileId {
        return FileId(p.valueAsString)
    }
}

Some DTO:

data class FileDTO(
    val fileId: FileId,
    val url: URL,
    val savedFilenameKey: String,
    val extension: String,
    val originalFilename: String,
    val description: String,
    val tags: Set<String>,
    val metadata: Map<String, String>
)

In query module:

    @QueryHandler
    fun handle(query: FindAllFileQuery): List<FileDTO> {
        return fileRepository.findAll()
            .map { toDTO(it) } // toDTO returns a FileDTO class (not a LinkedHashMap)
    }

In rest module

fun getAllFiles() : CompletableFuture<List<FileDTO>> {
val future = queryGateway.query(FindAllFileQuery(), ResponseTypes.multipleInstancesOf(FileDTO::class.java))

val join = future.join()

// this next line will throw as list contains LinkedHashMap objects
val first = join.first()

// ...
}

Screenshot of what is contained in the join variable:

[smcvb]

I've been in the practice of "showering" the DTO with Jackson annotations to make it work to be honest. In absence of any type knowledge, I am accustomed to receive a LinkedHashMap from the ObjectMapper. I typically get by with the @JsonProperty annotation, providing the property name in the annotation itself.

That said, I don't think this is overly Axon-specific. As such I will close the issue preemptively as a "question". Feel free to keep responding on the matter by the way; closed doesn't mean "never reply here again" if you ask me. :-)

[Blackdread]

I am not so sure, I usually (I mean never unless needed) do not use "showering" the DTO with Jackson annotations as it works by default as expected.
Example (I used REST code but a simple test case would work as well):
Given in a post:
[{"fileId":"372b37ab-8a64-415a-8570-57df92a14b6b","url":"http://localhost","savedFilenameKey":"asdasdwdwda.png","extension":"png","originalFilename":"dawdawd.png","description":"desc","tags":[],"metadata":{}},{"fileId":"2a780e88-f947-4a0d-8ace-c1e3064f9d5d","url":"http://localhost","savedFilenameKey":"asdasdwdwda.png","extension":"png","originalFilename":"dawdawd.png","description":"desc","tags":[],"metadata":{}}]

The REST handling:

    @PostMapping("/deserialize-file-dto-list")
    fun deserializeListFileDTO(@RequestBody data: String): String {
        val readValue = objectMapper.readValue(data, object : TypeReference<List<FileDTO>>() {})
         // readValue will actually be a list of 'List<FileDTO>' and not 'List<LinkedHasMap>'
        return objectMapper.writeValueAsString(readValue) // the string returned will be same as the request body
    }

---

It seems weird to me that Serializer/deserializer from Axon framework returns wrong type.
Furthermore if I query "FileDTO" as a single response from QueryGateway I get "FileDTO" (not a LinkedHashMap).
Using:
query(query, FileDTO::class.java) or query(query, ResponseTypes.optionalInstanceOf(FileDTO::class.java))

When querying with query(query, ResponseTypes.multipleInstancesOf(FileDTO::class.java)), axon knows the type contained in the list, so there seem to be an issue how the deserializer works in axon.

[Blackdread]

It feels a bit silly to need to do:
class FileDtoList(c: Collection<FileDTO>) : ArrayList<FileDTO>(c) and queryGateway.query(FindAllFileQuery(), ResponseTypes.instanceOf(FileDtoList::class.java)) to get the query gateway return the correct objects in the list.

---

I tried to use @JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, visible = true, include = JsonTypeInfo.As.PROPERTY, property = "@class") but did not work with JacksonSerializer in Axon; But worked when autowiring the ObjectMapper and doing serialize/deserialize. (Weird since supposed to have same autowired ObjectMapper in axon and in my tests)

I think the JacksonSerializer in org.axonframework.serialization.json may do more, since you already have (type is not perfect since it lost the generic part, could the generic missing be added?):

payload {
  type: "java.util.ArrayList"
  data: "..."
}

I am not sure why the generic part is not used from the "ResponseTypes.multipleInstancesOf(...)" when specified in the query.

Furthermore in the axon dashboard, queries and response types table display correctly generics parts of methods.
I guess it uses reflection and get type with that, then why is it not used when the query handler returns ? At that point in time, axon already knows everything since it routed the query based on query name and return type.

---

The actual data de-serialization, and generic lost as objectMapper.readerFor(type) received java.util.ArrayList

---

EDIT: I know this issue is mostly due to JSON serialize/deserialize when using a generic type as root object (List, Map, etc). Maybe some extra information in the payload or somewhere could solve this issue. Otherwise using class FileDtoList is the easiest solution. (Issue for JsonTypeInfo is also due to root, would work on a non-root generic object serialized/deserialized)

[abuijze]

Hi @Blackdread ,
adding this also helps:
objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_CONCRETE_AND_ARRAYS, JsonTypeInfo.As.PROPERTY);

I agree that it would be nicer to "detect" the contents by using the ResponseType passed in the query. Unfortunately, the Serialization mechanism in Axon is abstracted away from the QueryBus. The QueryBus doesn't influence the Serializer, and the Serializer doesn't know about the context for which it needs to (de)serialize. We do have some ideas for changing this, but that will take time to implement. And we need to validate the impact on compatibility.

[Rafaesp]

I'm facing the same issue and I've reached the same conclusion than @Blackdread.

I am not sure why the generic part is not used from the "ResponseTypes.multipleInstancesOf(...)" when specified in the query.

The fact that Jackson is told to deserialize into an ArrayList doesn't sound right to me.

Now I'm not sure what's better: to configure two ObjectMappers (one for axon with enableDefaultTyping() and another for the my service) or to have only one default ObjectMapper but having QueryHandlers returning JsonNode and doing the serialization/deserialization myself doing something like this:

JsonNode jsonNode = query.get();
CollectionType javaType = objectMapper.getTypeFactory()
          .constructCollectionType(List.class, MyClass.class);
ObjectReader reader = objectMapper.readerFor(javaType);
return reader.readValue(jsonNode);

@abuijze Have you looked into this in the last few months?

Thanks

[Blackdread]

@Rafaesp I went for
class FileDtoList(c: Collection<FileDTO>) : ArrayList<FileDTO>(c)
and
queryGateway.query(FindAllFileQuery(), ResponseTypes.instanceOf(FileDtoList::class.java)).

It is the easiest solution in kotlin, just one line of code

[smcvb]

We've recently done a PR, #1421 to be exact, which provides you a short hand to enable default type information on the ObjectMapper used in the JacksonSerializer.

That way you do not have to provide this ObjectMapper instance yourself _and_ be mindful of the fact you do not accidentally use it in other portion of your application where you do not want default type information.

It'll be part of 4.4, so I hope that helps you out @Rafaesp. In the mean time, the suggested solution of @Blackdread does the job too of course.

[Controlix]

I still get the error Retrieved response [class java.util.ArrayList] is not convertible to a List of the expected response type in version 4.4.6.

How can I make sure that I am using the fix? Is there some configuration required?

[smcvb]

Default typing as constructed in issue #1421 is not the default in the JacksonSerializer.
Simply because that would constitute a breaking change and adjust the ObjectMapper which is set on the JacksonSerializer.
Possibly an ObjectMapper which is used somewhere else too.

So, you will have to trigger this yourself.
You can either do this on the JacksonSerializer's builder when constructing the serializer, or you set default typing on the ObjectMapper yourself.
The first would look like so:

JacksonSerializer.builder()
                 // Your custom configuration
                 .defaultTyping()
                 .build();

The second would manipulate the ObjectMapper directly, and would be something like this:

ObjectMapper objectMapper = new ObjectMapper();
objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_CONCRETE_AND_ARRAYS);

JacksonSerializer.builder()
                 .objectMapper(objectMapper)
                 .build();

Option two's change on the ObjectMapper is what defaultTyping() would invoke too.