Quarkus: Quarkus Native Jackson InvalidDefinitionException Cannot find a (Map) Key deserializer for type [simple type, class java.math.BigDecimal]

[stephane-mori]

Hello,

I tried to use Quarkus to develop a Rest API to generate a PDF from datas from POST payload only. And I have the following exception only when the app run in native mode :

com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot find a (Map) Key deserializer for type [simple type, class java.math.BigDecimal]
 at [Source: (SequenceInputStream); line: 1, column: 1]
    at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1589)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._handleUnknownKeyDeserializer(DeserializerCache.java:599)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findKeyDeserializer(DeserializerCache.java:168)
    at com.fasterxml.jackson.databind.DeserializationContext.findKeyDeserializer(DeserializationContext.java:499)
    at com.fasterxml.jackson.databind.deser.std.MapDeserializer.createContextual(MapDeserializer.java:248)
    at com.fasterxml.jackson.databind.DeserializationContext.handlePrimaryContextualization(DeserializationContext.java:650)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:484)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findNonContextualValueDeserializer(DeserializationContext.java:466)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:473)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:293)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:476)
    at com.fasterxml.jackson.databind.ObjectReader._findRootDeserializer(ObjectReader.java:2050)
    at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1677)
    at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:977)
    at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.readFrom(ResteasyJackson2Provider.java:191)
    at org.jboss.resteasy.plugins.providers.multipart.MultipartInputImpl$PartImpl.getBody(MultipartInputImpl.java:218)
    at org.jboss.resteasy.plugins.providers.multipart.MultipartFormAnnotationReader.setFields(MultipartFormAnnotationReader.java:189)
    at org.jboss.resteasy.plugins.providers.multipart.MultipartFormAnnotationReader.readFrom(MultipartFormAnnotationReader.java:79)
    at org.jboss.resteasy.core.interception.jaxrs.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:101)
    at org.jboss.resteasy.core.interception.jaxrs.ServerReaderInterceptorContext.readFrom(ServerReaderInterceptorContext.java:63)
    at org.jboss.resteasy.core.interception.jaxrs.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:80)
    at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:213)
    at org.jboss.resteasy.core.MethodInjectorImpl.injectArguments(MethodInjectorImpl.java:95)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:128)
    at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:621)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:487)
    at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:437)
    at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:439)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:400)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:374)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:67)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:488)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:259)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:160)
    at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:362)
    at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:163)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:245)
    at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:73)
    at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:122)
    at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.access$000(VertxRequestHandler.java:36)
    at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:87)
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2027)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1551)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1442)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
    at java.lang.Thread.run(Thread.java:834)
    at org.jboss.threads.JBossThread.run(JBossThread.java:479)
    at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:497)
    at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:193)

My datas look like this :

@Getter
@Setter
public class Invoice {

    @NotBlank
    private String number;

    @NotNull
    private LocalDate date;

    @NotNull
    private Currency currency;

    @NotNull
    @AllowedLocale
    private Locale locale;

    @NotNull
    @Valid
    private Provider provider;

    @NotNull
    @Valid
    private Customer customer;

    @NotNull
    @Valid
    private Order order;

    @Valid
    private ConsolidatedTaxes consolidatedTaxes;

    @NotNull
    @Valid
    private PaymentInstructions paymentInstructions;

}

@Getter
@Setter
public class ConsolidatedTaxes {

    @NotNull
    @Size(min = 1)
    @Valid
    private Map<BigDecimal, ValueAddedTax> byAmount;

    @NotNull
    @Valid
    private ValueAddedTax total;

}

@Getter
@Setter
public class ValueAddedTax {

    @NotNull
    private BigDecimal baseAmount;

    @NotNull
    private BigDecimal taxAmount;

    @NotNull
    private BigDecimal includingTaxAmount;

}

The problem is only when i try to deserialize BigDecimal in the Map.
The problem don't exists in development mode.

How I make my deserialization :

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-resteasy</artifactId>
</dependency>
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-resteasy-jackson</artifactId>
</dependency>
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-hibernate-validator</artifactId>
</dependency>
<dependency>
    <groupId>org.jboss.resteasy</groupId>
    <artifactId>resteasy-multipart-provider</artifactId>
</dependency>

@Path("/v1/invoices")
public class InvoiceController {

    private InvoiceService invoiceService;

    InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @POST
    @Consumes(MediaType.MULTIPART_FORM_DATA)
    @Produces("application/pdf")
    public Response post(@MultipartForm InvoiceResource invoiceMultipartBody) throws IOException {
        if (Objects.nonNull(invoiceMultipartBody.getLogo())) {
            byte[] logo = invoiceMultipartBody.getLogo().readAllBytes();
            invoiceMultipartBody.getInvoice().getProvider().setLogo(logo);
        }

        Response.ResponseBuilder response = Response.ok(invoiceService.generate(invoiceMultipartBody.getInvoice()));
        response.header("Content-Disposition", "attachment; filename=" + invoiceMultipartBody.getInvoice().getNumber() + ".pdf");
        return response.build();
    }

}

@Getter
@Setter
public class InvoiceResource {

    @FormParam("logo")
    @PartType(MediaType.APPLICATION_OCTET_STREAM)
    public InputStream logo;

    @FormParam("invoice")
    @PartType(MediaType.APPLICATION_JSON)
    public Invoice invoice;

}

JSON payload example :

{
    "number": "#Invoice-Number",
    "date": "2019-06-25",
    "currency": "EUR",
    "locale": "fr_FR",

    "provider": {
        "corporateName": "corporateName",
        "address": {
            "identification": "identificationAddressSender"
        }
    },

    "customer": {
        "address": {
            "identification": "identificationAddressRecipient"
        }
    },

    "order" : {
    "description": "description with accents special characters &é'(§è!çà)-\""
    },

    "paymentInstructions": {
        "amount": 1386.26,
      "dueDate": "2019-07-25"
    }
}

My configuration :

    <quarkus.version>1.3.2.Final</quarkus.version>
    <lombok.version>1.18.12</lombok.version>

I quickly tried with quarkus version 1.4.1.Final but same result.
GraalVM CE 19.3.1

What is append only during native compile that can produce this kind of behavior ? Thank you for your help.

I make this issue as asked by @stuartwdouglas

Thank you very much @stuartwdouglas

[stuartwdouglas]

Ah, I did not notice that you were returning a Response rather than the object itself.

Can you put an @RegisterForReflection on Invoice, and see if this works?

If you return an Invoice from the controller we will recognise that this needs to be registered for reflection, and we will automatically handle it, if you use response we can't figure this out for you so you need to use RegisterForReflection to tell graal to make reflection work.

[stephane-mori]

@RegisterForReflection don't work.

Moreover, I dirty tried to modify my controller :

@Path("/v1/invoices")
public class InvoiceController {

    private InvoiceService invoiceService;

    InvoiceController(InvoiceService invoiceService) {
        this.invoiceService = invoiceService;
    }

    @POST
    @Consumes(MediaType.MULTIPART_FORM_DATA)
    @Produces("application/pdf")
    public Invoice post(@MultipartForm InvoiceResource invoiceMultipartBody) throws IOException {
        return invoiceMultipartBody.getInvoice();

        /*
            if (Objects.nonNull(invoiceMultipartBody.getLogo())) {
                byte[] logo = invoiceMultipartBody.getLogo().readAllBytes();
                invoiceMultipartBody.getInvoice().getProvider().setLogo(logo);
            }

            Response.ResponseBuilder response = Response.ok(invoiceService.generate(invoiceMultipartBody.getInvoice()));
            response.header("Content-Disposition", "attachment; filename=" + invoiceMultipartBody.getInvoice().getNumber() + ".pdf");
            return response.build();
         */
    }

}

I'll create a project with minimum code to reproduce this problem and share it through GitHub.

[stuartwdouglas]

thanks, I just tried to add a BigDecimal to one of our existing integration tests and it appeared to work, which is why I thought it might just be RegisterForReflection.

[geoand]

I'll create a project with minimum code to reproduce this problem and share it through GitHub.

That would be great, thanks.

[stephane-mori]

I just created a project with minimum code and I reproduced the undesired behavior.
In the example, if we replace BigDecimal by String, all is ok. But BigDecimal still not work.

_https://github.com/stephane-mori/QuarkusJacksonMapBigDecimalKey_ (repo removed)

[stephane-mori]

Of course, I tried @RegisterForReflection but it still not work.

[stephane-mori]

Environment :

Output of uname -a or ver:
Darwin MBP-de-Stephane 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64

Output of java -version:
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment GraalVM CE 19.3.1 (build 11.0.6+9-jvmci-19.3-b07)
OpenJDK 64-Bit Server VM GraalVM CE 19.3.1 (build 11.0.6+9-jvmci-19.3-b07, mixed mode, sharing)

Build tool (mvn --version):
Maven home: /usr/local/Cellar/maven/3.6.3_1/libexec
Java version: 11.0.6, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/graalvm-ce-java11-19.3.1/Contents/Home
Default locale: fr_FR, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.4", arch: "x86_64", family: "mac

[stuartwdouglas]

Using @RegisterForReflection(targets = {Payload.class, BigDecimal.class}) will fix it.

@geoand At the moment RegisterForReflection does not register the heirachy, just the class. Maybe we need a RegisterForSerialization or something that just registers everything.

[geoand]

I am surprised we don't have something like that exposed TBH - since we do have the backing code to take care of that in https://github.com/quarkusio/quarkus/blob/d86275e89dcea1d8aab7fd6eaaff85f69a873665/core/deployment/src/main/java/io/quarkus/deployment/builditem/nativeimage/ReflectiveHierarchyBuildItem.java (I haven't tried it for this case, but I hope it's smart enough to work out the generics - if not, it has to be updated)

@stuartwdouglas how about instead of @RegisterForSerialization we add a entireHierarchy (or something like that) field to @RegisterForReflection which would be false by default (to keep the current behavior)?

[stuartwdouglas]

That seems like I good idea. I am really surprised this has not come up before.

[geoand]

Indeed yes. I'll take care of it later on today, I need to look at the liquibase issue first.

[geoand]

Just looking at the code, it seems like generics should be handled properly as well, so I think it should be a super simple change.
I'll do that later and add a test similar to the reproducer.

[stephane-mori]

Using @RegisterForReflection(targets = {Payload.class, BigDecimal.class}) will fix it.

@geoand At the moment RegisterForReflection does not register the heirachy, just the class. Maybe we need a RegisterForSerialization or something that just registers everything.

Thank you very much :)

[machi1990]

Just looking at the code, it seems like generics should be handled properly as well,

This is what I thought also, could it be because java. classes are ignored ?
https://github.com/quarkusio/quarkus/blob/d86275e89dcea1d8aab7fd6eaaff85f69a873665/core/deployment/src/main/java/io/quarkus/deployment/builditem/nativeimage/ReflectiveHierarchyBuildItem.java#L71 and https://github.com/quarkusio/quarkus/blob/a3ee2d89c27b041683e1413258be67a82ae999cc/core/deployment/src/main/java/io/quarkus/deployment/steps/ReflectiveHierarchyStep.java#L93

[geoand]

That's probably an additional thing to deal, however the basic issue is that @RegisterForReflection doesn't involve the hierarchy at all: https://github.com/quarkusio/quarkus/blob/5257fd855f44d1389e110bfc74303d835d0a3c05/core/deployment/src/main/java/io/quarkus/deployment/steps/RegisterForReflectionBuildStep.java#L30

Adding the property we discussed about above is the first order of business. Then updating the hierarchy registration code if needed.

[machi1990]

That's probably an additional thing to deal, however the basic issue is that @RegisterForReflection doesn't involve the hierarchy at all:

I agree with you here.

Also, another interesting thing, using the quarkus-resteasy-jsonb extension instead of quarkus-resteasy-jackson seems to work fine without the need for @RegisterForReflection.

[geoand]

For the same application you mean? That is very odd. Do you want to look at that @machi1990 ?

[geoand]

9007 should take care of it. I didn't need to update @RegisterForReflection (although we might still want to think about it in the future).

[machi1990]

For the same application you mean? That is very odd.

Yes, for the same application.

[geoand]

Yeah, I checked. It's just a different way of handling keys between JsonB and Jackson. I would say it's not worth spending time on, we just register BigDecimal and BigInteger for reflection and like I did in #9007 and be done with it :)

[machi1990]

Nice, thanks.