Quarkus: Quarkus ignores mapped jackson annotations of 3rd party libraries

[nimo23]

Describe the bug
I included a 3rd party library in my application. This library makes use of Jackson and Jersey for its rest webservices and json-(un)marshallings. My main application uses Json-B annotated classes and I included the quarkus-resteasy-jsonb in my pom.xml. However, with such configuration, quarkus totally ignores the jackson-mappings from the 3rd-party library and tries to unmarshall it by Json-B even if these classes does not contain any Json-B annotations. The bug is also described in https://github.com/quarkusio/quarkus/issues/5906.

To sum up:

• If you use a 3rd party dependency which uses jackson under the hood, then your application must also use jackson.
• If you use json-b in your application and one of your dependencies uses jackson, then these dependencies will not work in Quarkus and can lead to wrong (un)marshallings or unexpected error logs
• It will also not work, if you use two dependencies, one using jackson and the other using json-b internally even if your main application does not use jackson or json-b.
• It only works if both your application or any dependency has the same json-provider, either json-b or jackson.

Expected behavior
The mapped jackson-annotations (or jsonb-annotations) of a 3rd party library must be considered instead of ignoring them and trying to (un)marshalling it with the json-provider of the main application.

Actual behavior
All 3rd party libraries using jackson or jsonb under the hood and which does not comply with the json-provider of the main application can lead to wrong (un)marshallings or unexpected error logs. Quarkus totally ignores the mappings of any 3rd party dependency - for the user this is not obviously recognizable because the mappings are hidden in the 3rd party library.

To Reproduce
Steps to reproduce the behavior:

1. include any library which uses jersey with jackson annotated classes to unmarshall a jackson annotated class
2. use jsonb for your main application
3. try to use the 3rd party library - you will see that your jackson annotations are ignored and the mapping is tried by using jsonb (even if the author of the 3rd party library does not intended the use of jsonb for its classes)

Environment (please complete the following information):

• Output of java -version: 12
• Quarkus version or git rev: 1.0 Final

Additional context
There are a lot of 3rd party dependencies making use of jackson as its json-provider. There are also more and more 3rd party dependencies making use of jsonb as its json-provider. Actually, in Quarkus you cannot include them togheter in pom.xml.

[gsmet]

This is not a bug. The fact that you have annotations or not does not enter into account.

The annotations are hints on how to serialize things but each serializer is perfectly capable of serializing without those hints.

[nimo23]

This is not a bug. The fact that you have annotations or not does not enter into account.

@gsmet It is a fact that I cannot use the described use cases in quarkus because quarkus ignores jackson mappings of a third party library. If quarkus did not ignore these annotations, then the serializer would be capable of serializing it. The third party library has no bugs, but quarkus has a bug because it cannot handle it right.

The annotations are hints on how to serialize things but each serializer is perfectly capable of serializing without those hints.

@gsmet Really? I dont think annotations are only hints. It makes a big difference in (de)serializing if I include for example @JsonBTransientor @JsonCreator. If quarkus did not ignore these annotations, then the serializer would be capable of serializing it.

[rmannibucau]

@gsmet this issue is accurate and quarkus behaves against JAX-RS spec it seems, see https://github.com/eclipse-ee4j/jsonb-api/issues/214#issuecomment-565832130 for the background. JAX-RS is designed to support that and annotations are hints the JAX-RS spec enables to exploit to properly handle the serialization in all cases. If quarkus does not handle that it means it does not support JAX-RS which should likely be highlighted a bit more on the website IMHO

[gsmet]

If you include resteasy-jsonb then RESTEasy will use JSON-B. The fact that you have Jackson annotations on your classes doesn’t matter.

[gsmet]

And yes, we support JAX-RS.

[rmannibucau]

@gsmet if the previous mapper is included and as reported @nimo23 it does not use jsonb or jackson as needed then quarkus (resteasy?) does not support jaxrs.

[gsmet]

BTW opening random issues everywhere without even providing a reproducer and stating incorrect assumptions doesn’t really help your case.

It seems you are mixing a lot of incompatible things and hoping it will work and it won’t.

Our JAX-RS implementation is RESTEasy, we don’t support Jersey.

[nimo23]

Our JAX-RS implementation is RESTEasy, we don’t support Jersey.

I do not use jersey directly in my application. The 3rd party library uses jersey. It should be isolated from quarkus.

[gsmet]

Well as I said, I think you are trying to mix things that won’t work together.

But again, instead of stating things and making incorrect assumptions, provide a reproducer and then we will be able to talk about something concrete.

Until then, I don’t see us being able to do anything about that. And certainly not being sure it’s a valid issue.

[gsmet]

And no things won’t be isolated. There’s no modular class loading in Quarkus.

Don’t make assumptions please. Provide us a reproducer and we’ll talk then.

[rmannibucau]

@gsmet maybe push an app with a custom provider routing to jackson or jsonb depending the class on a github repo, this is plain jaxrs spec and must work with quarkus (if resteasy code handling that is not substituted Im pretty confident it works, if it is substituted quarkus does not support jaxrs and should be enhanced). Will enable to close the topic faster probably.

Just my 2cts

[gsmet]

Well, you apparently have the time to open a lot of issues. So please take the time to provide a reproducer if you want us to investigate. If not, that’s your choice.

It’s typically the type of issue where we will spend hours shooting in the dark.

[maxandersen]

I'm trying to spot what the exact issue is here and still struggling to see what the 3rd party dep is doing that quarkus is somehow "breaking" - a reproducer from either @rmannibucau or @nimo23 would be great as like @gsmet says we could be guessing at very different things here.

[aguibert]

I think the basic scenario is that @nimo23 is trying to use JAX-RS and JSON-B in their application, but one of their 3rd party deps is using Jackson. To try and satisfy the 3rd party dependency, they are adding Jackson in some way to their pom.

Although it's not explicitly stated, I assume @nimo23 is adding 2 JAX-RS message body reader/writers -- one for JSON-B via quarkus-resteasy-jsonb and another one for Jackson via some other dependency such as quarkus-resteasy-jackson.

Within the application some classes need to use JSON-B for POJO<-->JSON and for some of the 3rd party classes Jackson needs to be used for POJO<-->JSON (because the 3rd party classes have special Jackson annotations on them for instructions on how to perform the [un]marshalling).

If the above assumptions are correct, then there isn't anything wrong with Quarkus, Jackson, JSONB, etc. Rather, the user will need to jump through some extra hoops to use JSONB and Jackson together. Perhaps something along the lines of a JAX-RS message body reader/writer that knows about JSONB and Jackson, such as:

public class JsonbJacksonMessageAdapter implements MessageBodyReader<Object>, MessageBodyWriter<Object> {

  @Override
  public Object readFrom(Class<Object> type, /* lots of other args */) {
    // check 'type' to see if it has any Jackson annotations on it. 
    // If yes, deserialize with Jackson. If not, deserialize with JSONB
  }

  @Override
  public void writeTo(Object t, /* lots of other args */) {
    // check class of 't' to see if it has Jackson annotations on it. 
    // If yes, serialize with Jackson. If not, serialize with JSONB
  }
}

[rmannibucau]

Ok, took time to test the code from the jsonb issue and it works well in jvm mode. So it can likely just be some @RegisterForReflection or META-INF/services/Providers missing file in native mode. // cc @nimo23

[maxandersen]

Thanks @rmannibucau, I assume your example is https://github.com/rmannibucau/jaxrs-router and with yours and @aguibert additional examples I think we can agree that we have at least two ways that works.
Thus @nimo23 please - if you think there is still an issue here please make a small project showing what is failing for you. Thank you!

[gsmet]

That being said, if something is working in JVM mode and not working in native mode, we have a bug.

@rmannibucau can you post a reproducer of this issue here? Or point to a GH repository with instructions on how to reproduce it?

[rmannibucau]

@gsmet ^^ ;), i didnt manage to build with the graalvm I had locally but it does not mean it does not work, guess it is just not yet tested (until nimo confirms he did).

[gsmet]

@rmannibucau I see @maxandersen assuming it might be the reproducer so yeah a confirmation helps.

I see no instructions on how to reproduce the issue though and what we should look for.

But from what you say, I don't understand if we have an issue or not?

[nimo23]

Sorry for the late response. I have spend hours to look for the reason why it fails in my project.

I have made an _example project_ which confirms that there is indeed no error apparent. It also works nice for native mode.

I cannot see any issues when marshalling json-b and jackson within the attached example project. That is fine.

The strange thing is: I found out that my original project also works without problems only if I remove the following dependency from my pom.xml:

<!--  this makes trouble with 3rd party json processings, but only under unknown circumstances of "org.jboss.resteasy.resteasy-client" -->
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-rest-client</artifactId>
</dependency>

However, including quarkus-rest-client within the _example project_ works without problems. Strangely, if I include quarkus-rest-client in my original project it does not work:

// the Task.class is processed within a 3rd-party lib and it normally should be processed by jackson (with jersey) and not by json-b (with resteasy). 
Caused by: javax.json.bind.JsonbException: Can't create instance of a class: class org.api.data.Task, No default constructor found.
    at org.eclipse.yasson.internal.serializer.ObjectDeserializer.getInstance(ObjectDeserializer.java:95)
    at org.eclipse.yasson.internal.serializer.AbstractContainerDeserializer.deserialize(AbstractContainerDeserializer.java:61)
    at org.eclipse.yasson.internal.serializer.CollectionDeserializer.deserializeNext(CollectionDeserializer.java:106)
    at org.eclipse.yasson.internal.serializer.AbstractContainerDeserializer.deserializeInternal(AbstractContainerDeserializer.java:84)
    at org.eclipse.yasson.internal.serializer.AbstractContainerDeserializer.deserialize(AbstractContainerDeserializer.java:60)
    at org.eclipse.yasson.internal.Unmarshaller.deserializeItem(Unmarshaller.java:68)
    at org.eclipse.yasson.internal.Unmarshaller.deserialize(Unmarshaller.java:54)
    at org.eclipse.yasson.internal.JsonBinding.deserialize(JsonBinding.java:53)
    at org.eclipse.yasson.internal.JsonBinding.fromJson(JsonBinding.java:93)
    at org.jboss.resteasy.plugins.providers.jsonb.JsonBindingProvider.readFrom(JsonBindingProvider.java:70)
    ... 48 more

But it works without problems if I remove quarkus-rest-client from my pom.xml.

I dont know why.

However, I cannot confirm that this is a quarkus issue as the example project works. And my project also works but only if I remove quarkus-rest-client from my pom.xml.

[aguibert]

@nimo23 in your actual project, can you post the dependencies by posting the results of mvn dependency:tree

[nimo23]

@aguibert @rmannibucau @gsmet I was able to reproduce the error also within the new _example project_. Please ./mvnw compile quarkus:dev (in jvm-mode) with

I think quarkus-rest-client makes trouble because of its provided resteasy-client..

[aguibert]

thanks for uploading a sample @nimo23. I tried it out and the issue was that in the method where you build a client for the application to call itself, you need to register the provider manually, since you are building the WebTarget from scratch.

Fixed code:

public static TaskJackson doPostRequestJackson(String uri, Entity<?> entity) {
    WebTarget target = ClientBuilder.newClient()
            .register(JsonRoutingProvider.class) // <<< added this line to fix
            .target(uri);
    Invocation request = target.request(MediaType.APPLICATION_JSON).buildPost(entity);
    return request.invoke(TaskJackson.class);
}

[nimo23]

Thanks @aguibert one little problem: The 3rd party dependency uses WebTarget without .register(JsonRoutingProvider.class) because it defaults to jackson. All the jackson stuff is only done within the 3rd party lib and not in my app code. How should I place the .register(JsonRoutingProvider.class) within the 3rd party library? I have no influence to the 3rd party code.

[aguibert]

It sounds like a bug in the 3rd party library then. If they are providing integration with classes that _only_ work with Jackson (i.e. annotated with Jackson stuff) then the third party lib should be explicitly registering a Jackson-compatible provider in the WebTarget they use, rather than assuming that the JAX-RS container being used happens to have Jackson installed as some sort of default provider.

Unless there is some interaction between CDI+JAXRS for auto-registration that I'm missing here, the only way to resolve this is have the 3rd party dependency change.

[rmannibucau]

@nimo23 implement your own ClientBuilder (it is a SPI), exclude default one (only the registration file from meta-inf) and register the default providers you want there delegating the logic to your default impl. Will be a workaround for such hybrid cases.

[nimo23]

then the third party lib should be explicitly registering a Jackson-compatible provider in the WebTarget they use

@aguibert thanks for the solution. that makes sense. The 3rd-lib has already

        <dependency>
            <groupId>org.glassfish.jersey.media</groupId>
            <artifactId>jersey-media-json-jackson</artifactId>
        </dependency>

So it must also register jerseys JacksonJsonProvider.class

Client client = ClientBuilder.newClient().register(JacksonJsonProvider.class);

Such simple things need sometimes so much time,)

With your solution, I can still use json-b instead of switching to jackson.

Thanks for your help.

@gsmet _I guess, such things worked in jee-containers before because they have a module system, but not in quarkus..therefore such issues.._

[nimo23]

I registered the official JacksonJsonProvider.class from jackson, but it does not work.

WebTarget target = ClientBuilder.newClient()
                                // does not work
                .register(JacksonJsonProvider.class)
                .target(uri);

It only works if I extend JacksonJsonProvider within my application

package org.quark;

import javax.ws.rs.Consumes;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.ext.Provider;

import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@Provider
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class MyJacksonProvider extends JacksonJsonProvider {
    // is empty
}

and use the extended class, then it works:

WebTarget target = ClientBuilder.newClient()
                                // does work
                .register(MyJacksonProvider.class)
                .target(uri);

updated reproducer:

quark.zip

[aguibert]

The app you are using to reproduce the issue is a bit misleading because it is making a web request to itself using a ClientBuilder, so it's conflating where the actual issue is occurring.

In the previous reproducer the quarkus JAX-RS endpoints were operating fine, but it was the unmanaged WebTarget you were creating that was having issues (which Quarkus or any JEE container will not interfere with).

So when you say "it does not work", please be specific about which chain of the request is not working. Is it:

• The initial JAX-RS request going into the application from the web browser
• Serializing JSON data using the custom web target
• Deserializing JSON data using Quarkus's built-in providers

Normally JEE containers do not scan third-party libraries for @Provider annotations, which would explain why JacksonJsonProvider is not found but MyJacksonProvider is

[nimo23]

Normally JEE containers do not scan third-party libraries for Provider annotations

Ok, this explains the reason why I need MyJacksonProvider.

With "it does not work", I mean the registration by ClientBuilder.newClient().register(JacksonJsonProvider.class) for marshalling does not work. Only when I register it by MyJacksonProvider, the marshalling works.

It is not obvious why register(JacksonJsonProvider.class) does not work for marshalling because client.getConfiguration().isRegistered(JacksonJsonProvider.class) returns true, so I assumed that the marshalling provider is indeed registered and ready for marshallings..such a pitfall:)

However, I can solve that by using MyJacksonProvider so I am happy with it. Thanks.