Jackson-databind: ObjectMapper throws InvalidDefinitionException with JDK 9

[yrachid]

Jackson-databind JDK9 issue

When used with JDK 9, Jackson's ObjectMapper launches a com.fasterxml.jackson.databind.exc.InvalidDefinitionException while trying to read JSON content into a class that declares constructors with arguments. It works when the same is attempted with a class that declares a constructor with no arguments.

The issue 393 shows a very similar problem. The only difference is that in such issue the problem was related to Lombok annotations whereas in the current scenario the problem happens in __both Pojos and Lombok annotated classes.__

At the aforementioned issue's discussions someone suggested disabling the INFER_CREATOR_FROM_CONSTRUCTOR_PROPERTIES feature from the Mapper but that did not solve the current
problem and the exception is still being thrown.

Still in that discussion, the @JsonCreator annotation was mentioned as a possible way to solve the
problem. Such annotation was added on the cases with constructor arguments but the problem still persists (with or without @JsonCreator).

Test Scenarios

This repository was created as an attempt to reproduce the error.

There are four scenarios reproduced in the given repository:

__Pojos:__

• A Pojo class with constructor arguments (__FAIL__)
• A Pojo class with empty constructor (__PASS__)

__Lombok generated constructors:__

• A Lombok based class with constructor arguments (__FAIL__)
• A Lombok based class with empty constructor (__PASS__)

Given scenarios were run with and without applying the @JsonCreator annotation to the classes.

When running the tests, the following output is presented:

When @JsonCreator is applied:

ObjectMapperTest > lombok_all_args_fails FAILED
    com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Invalid type definition for type `lombok.PersonAllArgs`: Argument #0 has
no property name, is not Injectable: can not use as Creator [constructor for lombok.PersonAllArgs, annotations: {interface
com.fasterxml.jackson.annotation.JsonCreator=@com.fasterxml.jackson.annotation.JsonCreator(mode=DEFAULT)}]
     at [Source: (String)"{"name": "Joe", "age": 10}"; line: 1, column: 1]
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:62)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadTypeDefinition(DeserializationContext.java:1429)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addExplicitPropertyCreator(BasicDeserializerFactory.java:684)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addExplicitAnyCreator(BasicDeserializerFactory.java:715)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerConstructors(BasicDeserializerFactory.java:465)
        at
com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:337)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:255)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:214)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:137)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:411)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
        at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
        at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:477)
        at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4178)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3997)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
        at ObjectMapperTest.lombok_all_args_fails(ObjectMapperTest.java:46)

ObjectMapperTest > pojo_all_args_fails FAILED
    com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Invalid type definition for type `pojo.PersonAllArgs`: Argument #0 has no
property name, is not Injectable: can not use as Creator [constructor for pojo.PersonAllArgs, annotations: {interface
com.fasterxml.jackson.annotation.JsonCreator=@com.fasterxml.jackson.annotation.JsonCreator(mode=DEFAULT)}]
     at [Source: (String)"{"name": "Joe", "age": 10}"; line: 1, column: 1]
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:62)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadTypeDefinition(DeserializationContext.java:1429)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addExplicitPropertyCreator(BasicDeserializerFactory.java:684)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addExplicitAnyCreator(BasicDeserializerFactory.java:715)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerConstructors(BasicDeserializerFactory.java:465)
        at
com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:337)
        at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:255)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:214)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:137)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:411)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:349)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264)
        at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
        at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
        at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:477)
        at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:4178)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3997)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
        at ObjectMapperTest.pojo_all_args_fails(ObjectMapperTest.java:55)

When @JsonCreator is absent:

ObjectMapperTest > lombok_all_args_fails FAILED
    com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `lombok.PersonAllArgs` (no Creators, like
default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
     at [Source: (String)"{"name": "Joe", "age": 10}"; line: 1, column: 2]
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1451)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1027)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1290)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
        at ObjectMapperTest.lombok_all_args_fails(ObjectMapperTest.java:46)

ObjectMapperTest > pojo_all_args_fails FAILED
    com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `pojo.PersonAllArgs` (no Creators, like
default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
     at [Source: (String)"{"name": "Joe", "age": 10}"; line: 1, column: 2]
        at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1451)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1027)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1290)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4001)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2992)
        at ObjectMapperTest.pojo_all_args_fails(ObjectMapperTest.java:55)

Reproducing

There are two ways to reproduce the error in the aforementioned repository:

__Without Docker:__

./gradlew test --info

__With Docker:__

Using OpenJDK 9.0.1 image

./reproduce-with-docker.sh

Information

The problem was produced under the following circumstances:

__Jackson versions tested:__

• 2.7.4
• 2.9.1
• 2.9.4

__Operating System:__

Linux Ubuntu 17.10
Kernel 4.13.0-32-generic

__JDK Version:__

$ java -version

java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)

__Gradle version:__

------------------------------------------------------------
Gradle 4.2.1
------------------------------------------------------------

Build time:   2017-10-02 15:36:21 UTC
Revision:     a88ebd6be7840c2e59ae4782eb0f27fbe3405ddf

Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.6 compiled on June 29 2015
JVM:          9.0.4 (Oracle Corporation 9.0.4+11)
OS:           Linux 4.13.0-32-generic amd64

Sorry if this is the wrong place or if this issue was already reported. Would you have any advice on that?

Thanks in advance!

[yrachid]

__Updates__

After some further testing I found out that using the @JsonProperty annotation might solve the problem for the Pojo scenarios given that it will only work when applied to constructor parameters instead of class fields:

• When applied to class fields, the problem persists
• When applied to constructor params, the problem is gone

[cowtowncoder]

Just one note: although use of Jackson with Lombok should be fine as things are, all reproductions of problems need to be without use of Lombok itself -- just using annotations like Lombok pre-processor adds into byte code.

There are no known issues with Java 9 at this point (wrt 2.9.4, unit test suite) that I am aware of.

But one thing to note, wrt constructor/field/method annotations -- unless you also register Java 8 parameter names module, constructor parameter names are not available, and as such linkage can not be established. This is probably relevant here. So whereas fields have "implicit" name (name of field in bytecode) and possibly "explicit" name, override (from annotations), constructor parameters do not. This matters when combining individual accessors (ctor parameter, field, getter, setter) into single logical property, and combining information from all annotations, to avoid having to add multiple instances of same annotation.

[yrachid]

Thank you for the details. I have a few questions:

all reproductions of problems need to be without use of Lombok itself

In the repository I sent previously there are test cases where the classes do not use Lombok at all (please see pojo.PersonAllArgs class) but Lombok is still in the classpath of that project, do you believe that can be an issue?

just using annotations like Lombok pre-processor adds into byte code.

Do you mean that just by having Lombok in the classpath the results can be affected? Sorry for asking this kind of unrelated question, I was just wondering if you would know anything about it.

unless you also register Java 8 parameter names module, constructor parameter names are not available, and as such linkage can not be established.

In that same repository the Jdk8Module is registered in the ObjectMapper (see here), are you referring to that module specifically? Or is there another module to configure Java 8 parameter names?

I have another project (using JDK 8) that uses Lombok and Jacksons' JDK8Module and in that project the use of JsonCreator and JsonProperty annotations are not required, Jackson works fine without them even in conjunction with Lombok annotations like @RequiredArgsConstructor or @AllArgsConstructor and after trying to migrate that project to JDK 9, things stopped working. That's the reason why I believe this could be JDK 9 related.

Once again thank you very much for your attention! I will create another repository without having Lombok in the classpath to try reproducing the issue.

[yrachid]

__Update__

I was able to reproduce the problem in a project without Lombok as a dependency:

Please see the project here.

[cowtowncoder]

What I meant was not that Lombok couldn't be somewhere, but rather that it can not be a required dependency by test(s).
This because as far as I know, it is not enough to add a dependency, but additional jar has to be deployed with jdk, and I want to keep project free of any external dependencies beyond Maven. That is, when cloning git repo, all you need is Maven (and JDK for it to use), after which build should should succeed from clean slate.
(I also in general try to minimize lib dependencies by core components, but in some cases new test dependencies may be added)

Thank you for reproduction!

[MiWeiss]

For reference: This seems to be the related issue in the Lombok project: https://github.com/rzwitserloot/lombok/issues/1563

[maffe]

unless you also register Java 8 parameter names module, constructor parameter names are not available, and as such linkage can not be established.

In that same repository the Jdk8Module is registered in the ObjectMapper (see here), are you referring to that module specifically? Or is there another module to configure Java 8 parameter names?

Yes, namely ParameterNamesModule.

[tup916]

It seems to be a problem in java 11 as well.

[sergey-morenets]

It seems to be a problem in java 11 as well.

I was able to fix this issue with JDK 14 by adding @ConstructorProperties annotation.

[cowtowncoder]

Since core jackson-databind does not automatically find parameter names -- jackson-module-parameter-names (from jackson-modules-java8) is needed -- then one of following is needed:

1. Use of @JsonCreator to indicate constructor to use, along with @JsonProperty -- or, @ConstructorProperties
2. Move/re-create this issue for jackson-modules-java8, along with reproduction (possibly one that was linked to earlier -- but if possible, just inline minimal repro here)

As to jackson-databind, I think a better exception would make sense, if possible, to try to indicate problem better. But that may be challenging too, depending on where all information is available (wrt. potential candidates)

[cowtowncoder]

At this point not sure what could be done here: closing for now; may be re-filed with up-to-date reproduction with 2.11.x (or later).

[xenoterracide]

unless you also register Java 8 parameter names module

I just spent so much time on this... so much time... because I are dumb (or rather spoiled by spring boot) I was missing both the databind and the parameters module, is there any way the thing that throws this error could detect that they, or at least the latter is missing and throw a better error?

[cowtowncoder]

@xenoterracide I am open to improvements, but it is typically quite difficult to figure out something that might be needed is missing. This is problematic for another common problem case: javac not being configured to include parameter names in bytecode -- but since that alone may be intentional (older code compiled with older jdk; JDK itself I think leaves names out on purpose), missing names is not an error condition.

For 2.13 I will probably suggest upgrade of baseline to be Java 8 (same as what 3.0/master already requires) and if so, parameter names functionality could be included directly and not as module. That would probably reduce these issues.

[xenoterracide]

maybe the error message could just be improved? like "make sure you have -parameters enabled, and have the parameters-name-module` registered. I want paragraph long error messages! (or like, links to them?)