Quarkus: ObjectMapperCustomizer with Hibernate5Module for lazyloading

[cerias]

Describe the bug
Jackson ObjectMapperCustomizer not working as expected.

Expected behavior
If the Hibernate5Module would be set with the Hibernate5Module.Feature.SERIALIZE_IDENTIFIER_FOR_LAZY_NOT_LOADED_OBJECTS feature, i would expect that on every jax-rs endpoint with the jackson module would use the given Objectmapper feature.
So that lazy loaded relations would shown only as id lists.

Actual behavior
Getting Lazyloading execptions.

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/cerias/quarkus-debug-jackson-objectmapper
2. setup database with one entity of TestA and TestB
3. Open /swagger-ui and trigger GET

Configuration

also in the example repo.

Environment (please complete the following information):

• Output of uname -a or ver: Linux huffelpuff 5.1.1-050101-generic #201905110631 SMP Sat May 11 06:33:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

• Output of java -version: OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3, mixed mode, sharing)

• GraalVM version (if different from Java): 19.0.2

• Quarkus version or git rev: 0.25

It is also possible im not using the hibernate feature wrong. ;)

[Sanne]

Hibernate ORM as configured in Quarkus used Enhanced Proxies. This is similar to the HibernateProxy, but they are generated at build time.

I suspect this issue might be related to #3261

[Sanne]

This was actually not related to #3261 I think, we'll investigate it separately. CC @FroMage could you have a look?

[FroMage]

OK, so, what happens is that the serialisation of TestA which has a @OneToMany collection of TestB causes a lazy-loading exception:

Caused by: org.hibernate.LazyInitializationException: Unable to perform requested lazy initialization [wtf.heck.spqg.core.enviroment.entity.TestA.resources] - no session and settings disallow loading outside the Session
    at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.throwLazyInitializationException(EnhancementHelper.java:196)
    at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.performWork(EnhancementHelper.java:86)
    at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.loadAttribute(LazyAttributeLoadingInterceptor.java:76)
    at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.fetchAttribute(LazyAttributeLoadingInterceptor.java:72)
    at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.handleRead(LazyAttributeLoadingInterceptor.java:53)
    at org.hibernate.bytecode.enhance.spi.interceptor.AbstractInterceptor.readObject(AbstractInterceptor.java:153)
    at wtf.heck.spqg.core.enviroment.entity.TestA.$$_hibernate_read_resources(TestA.java)
    at wtf.heck.spqg.core.enviroment.entity.TestA.getResources(TestA.java:38)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:688)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)

Now, this exception is quite normal, _except_ that the jackson-hibernate module has some code to deal with lazy collections by opening sessions prior to reading them. Now, I don't know if that code would work, because it appears that code can't even get reached, because just reading the field/getter holding the collection will cause it to be loaded.

I believe that for non-enhanced entities you would probably be able to read the field/getter and get a PersistentBag which is not yet lazy-loaded and then Jackson-hibernate would try tricks with restoring a session to load it.

But in our case, just loading the field/getter triggers lazy-loading (even though in the end we do get a PersistentBag, but it's loaded) so Jackson-hibernate can't even look at its type to determine if it should load a session or not.

@Sanne is that a known behaviour for enhanced lazy collections?

Note that even if we were able to get the unloaded lazy collection it's really not clear if Jackson-hibernate would actually succeed in restoring a session.

Now, there's also code in Jackson-Hibernate that's dealing with HibernateProxy which we could rewrite to deal with PersistentAttributeInterceptable (as I see in PersistenceUtilHelper.isLoaded that it appears to be the type of bytecode-enhanced lazy proxies) but I haven't met such an instance in this test yet. I could expand the test to cause such an instance to be loaded by extending the model, but that does not appear to be the current issue's cause, even though future users are bound to run into it.

So, to recap, several issues:

• Calling TestA.getResources() (the lazy collection) causes the lazy collection to be loaded, where Jackson-Hibernate expects it to not be loaded. This may be different in non-enhanced entities.
• Jackson-Hibernate has code to restore sessions for lazy-loading but I've no idea if it would even work in Quarkus.
• Jackson-Hibernate has specific code to deal with HibernateProxy but not bytecode-enhanced proxies.

We could (well, really it would be better if someone from Hibernate would) participate upstream to Jackson-Hibernate to deal with these things in a more systematic way, but that appears to be a non-short-term task.

Not sure we can do something to make this work in time for 1.0, and while it's not ideal, there's a clear workaround which is to serialise using separate data classes. I would not call this a blocker, because there's a clear and complete workaround (even if verbose/inconvenient).

[Sanne]

thanks for the thorough analysis @FroMage !

Agreed this is not something we can fix in short time, as it requires at least changes to Jackson, and possibly to Hibernate ORM as well - to support a questionable use case.

@cerias please do consider that attempting to load objects in this way implies that the state you retrieve is defined by reading from multiple, independent read transactions. That implies the overall state of the retrieved object graph could be inconsistent.

Ensuring the necessary data is loaded within the scope of a single transaction would clearly be a better design, and avoid this problem. Please do that at least until the Jackson/Hibernate extension is improved to deal with build time enhanced proxies.

Thanks

[EfraimLA]

Temporally... Could it be fixed with hibernate.enable_lazy_load_no_trans=true? Also i suppose it's necessary configuration with persistence.xml as the quarkus property doesn't exist

When will be fixed? Approximately?

[Sanne]

That might help you pass some tests, but you're still open to exceptions in case of a concurrent transaction making a conflicting change on the db.

N.B. the main issue is that Jackson's Hibernate5Module is not integrated with Quarkus, and also when I checked its code we noticed it hasn't been updated to support the latest proxy types that Hibernate ORM now uses.. it's fixable but I had never heard of that library before, and this will require changes in various components.

Generally speaking, don't expect that any 3rd party library will "just work" by just tosssing it on your classpath, especially if it has a high degree of coupling to other libraries and does stuff with reflection and proxies.

[EfraimLA]

so don't wait fixes for now... is there some way to avoid the json serialization of those fields with lazy loading? i tried with @JsonIgnore and others annotations/properties but didn't work, i'm using resteasy-jsonb

[FroMage]

Try @JsonbTransient: https://javaee.github.io/jsonb-spec/users-guide.html#ignoring-properties

[EfraimLA]

It works, thanks & waiting for the solution :D

[EfraimLA]

Is there some plan to resolve this?

[vteferrer]

I have the same issue. The @JsonbTransient workaround does not work for me because sometimes I need to avoid getting the property and sometimes not (I would use join fetch for discriminate between one and the other). Is there some date planned to solve this?

[arielcarrera]

It is a pity that this module does not work. Maybe a temporary workaround can be work with DTOs and some mapper tool at compile time like MapStruct.

[EfraimLA]

@arielcarrera have a sample?

[rodrigorodrigues]

@arielcarrera have a sample?

https://github.com/rodrigorodrigues/quarkus-vs-springboot-reactive-rest-api/blob/master/quarkus/src/main/java/com/github/quarkus/CompanyResource.java#L161-L165

I'm trying to do use ObjectMapperCustomizer but the customize method has not been invoked, does anyone have some clue please?

@Singleton
public class JacksonCustomizer implements ObjectMapperCustomizer {
    @Override
    public void customize(ObjectMapper objectMapper) {
        objectMapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
    }
}

[arielcarrera]

@arielcarrera have a sample?

Hi @EfraimLA, do you need a sample to transform your entities to DTOs during your request? Here is an example of mapstruct in Quarkus https://github.com/mapstruct/mapstruct-examples/tree/master/mapstruct-quarkus/src/main/java/org/mapstruct/example/quarkus that shows how to use mapstruct, you can use it to map from entities to your Api's contract.

[EfraimLA]

@arielcarrera have a sample?

Hi @EfraimLA, do you need a sample to transform your entities to DTOs during your request? Here is an example of mapstruct in Quarkus https://github.com/mapstruct/mapstruct-examples/tree/master/mapstruct-quarkus/src/main/java/org/mapstruct/example/quarkus that shows how to use mapstruct, you can use it to map from entities to your Api's contract.

For lazy object data? How is it related?

[darmanovic]

It would be really nice if this module could be made to work.