Jackson-databind: com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope) at [Source: {"id":"0","name":"textFile"}; line: 1, column: 1]

[dbl001]

Hi,

I'm getting this error in incubator-zeppellin:

com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
at [Source: {"id":"0","name":"textFile"}; line: 1, column: 1]

zeppellin has 5 different versions of jackson-databind as external dependencies:

Here's the stack trace:

"msg": "com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name \u0027id\u0027 (in class org.apache.spark.rdd.RDDOperationScope)\n
at [Source: {\"id\":\"1\",\"name\":\"textFile\"}; line: 1, column: 1]\n\t
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)\n\t
at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:843)\n\t
at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:533)\n\t
at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:220)\n\t
at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)\n\t
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409)\n\t
at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358)\n\t
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265)\n\t
at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245)\n\t
at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143)\n\t
at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439)\n\t
at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666)\n\t
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558)\n\t
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2578)\n\t
at org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:85)\n\t
at org.apache.spark.rdd.RDDOperationScope$$anonfun$5.apply(RDDOperationScope.scala:136)\n\t
at org.apache.spark.rdd.RDDOperationScope$$anonfun$5.apply(RDDOperationScope.scala:136)\n\t
at scala.Option.map(Option.scala:145)\n\t
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:136)\n\t
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)\n\tat org.apache.spark.SparkContext.withScope(SparkContext.scala:714)\n\t
at org.apache.spark.SparkContext.hadoopFile(SparkContext.scala:1011)\n\tat org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:832)\n\tat org.apache.spark.SparkContext$$anonfun$textFile$1.apply(SparkContext.scala:830)\n\t
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)\n\t
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)\n\t
at org.apache.spark.SparkContext.withScope(SparkContext.scala:714)\n\t
at org.apache.spark.SparkContext.textFile(SparkContext.scala:830)\n\tat $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:32)\n\t
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:37)\n\t
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:39)\n\t
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:41)\n\t
at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:43)\n\tat $iwC$$iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:45)\n\t
at $iwC$$iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:47)\n\tat $iwC$$iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:49)\n\t
at $iwC$$iwC.\u003cinit\u003e(\u003cconsole\u003e:51)\n\tat $iwC.\u003cinit\u003e(\u003cconsole\u003e:53)\n\tat \u003cinit\u003e(\u003cconsole\u003e:55)\n\t
at .\u003cinit\u003e(\u003cconsole\u003e:59)\n\tat .\u003cclinit\u003e(\u003cconsole\u003e)\n\t
at .\u003cinit\u003e(\u003cconsole\u003e:7)\n\t
at .\u003cclinit\u003e(\u003cconsole\u003e)\n\t
at $print(\u003cconsole\u003e)\n\t
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\t
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\t
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:483)\n\tat org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065)\n\tat org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1346)\n\t
at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840)\n\tat org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)\n\tat org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)\n\t
at org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:812)\n\tat org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:755)\n\t
at org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:748)\n\tat org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)\n\tat org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)\n\t
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:345)\n\t
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)\n\tat org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)\n\t
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\t
at java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)\n\t
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)\n\t
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\t
at java.lang.Thread.run(Thread.java:745)\n\n"

[dbl001]

OS X 10.11.4
$ mvn --version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /opt/local/share/java/maven3
Java version: 1.8.0_05, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.4", arch: "x86_64", family: "mac"
David-Laxers-MacBook-Pro:Chapter08 davidlaxer$

[cowtowncoder]

@dbl001 I would need the class definition at minimum; json and exception are not enough to tell whether this is a valid failure or not. Ideally a stand-alone unit test could be used.

Also Jackson version is needed; screenshot seems to include many different versions. Note that reproduction would be needed for a relatively new version, ideally 2.7(.4). Older versions are not maintained so even if reproducible, it is unlikely there would be fixes for 2.6 or older.

[cowtowncoder]

Can not reproduce at this point; may be reopened with reproducible test case.

[dominik42]

getting the same error after update from Spring Boot 1.5.1 (jackson-databind 2.8.6) to Spring Boot 1.5.2 (jackson-databind 2.8.7). It's raised for the deserialization of an empty array. After overwrite the version back to 2.8.6 in pom.xml, this error is gone.

[jjzabkar]

I can confirm that @dominik42 's solution to roll back to jackson-databind-2.8.6 from jackson-databind-2.8.7 worked for me. (big hat tip!!!)
My jackson dependencies are pulled in by spring-boot-1.5.2.RELEASE and aws-sdk-core-1.11.18.

[cowtowncoder]

@dominik42 Since there apparently is a problem of some kind, a reproduction (unit test) would be highly useful. Empty array deserialization is tested by dozens if not hundreds of existing tests so this has to be in some specific context or configuration.
From exception it appears that a @JsonCreator is involved for example; but specific POJO on which problem is reported is for some reason not included here.

If anyone has a reproduction I'd be happy to have a look.

[jjzabkar]

@cowtowncoder My issue also occurred on deserialization.

MappingJackson2HttpMessageConverter : Failed to evaluate Jackson deserialization 
  for type [[simple type, class com.mycompany.MyEntity]]:
  com.fasterxml.jackson.databind.JsonMappingException: Could not find creator 
  property with name 'id' (in class com.mycompany.MyEntity)

This resulted in Spring MVC catching the JsonMappingException thrown by BeanDeserializerFactory:364 and returning a bogus 405 Method not supported.

MyEntity is just a Hibernate & Spring data-owned DAO with a Long primary key:

@AllArgsConstructor
@Builder
@Entity
@EqualsAndHashCode(of = {"id"})
@Getter
@NoArgsConstructor // required by Hibernate deserialization
@Table
@ToString(callSuper = true)
public class MyEntity extends AbstractAuditingEntity implements Serializable {

    @Id
    @Column(name = "id", insertable = false, updatable = false, nullable = false)
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
...
}

One thing to note is that my "leaf" entities (those without child relationships) do not experience this deserialization issue. MyEntity _does_ have child relationships (@ManyToMany, @OneToMany).

[cowtowncoder]

@jjzabkar Looks like you might be using something like Lombok? That seems to be particularly prone to all kinds of problems unfortunately. Big problem form my perspective is just that what I need is expanded class definition to see how Jackson sees the type after post-processing (or recompilation? I forget where Lombok does its magic), but obviously from user perspective it is irrelevant (should just work). So reproducing the issue is trickier.

[jjzabkar]

Indeed; I am using Lombok. That would certainly effect your reflection requirements.

On Mar 29, 2017, at 2:28 PM, Tatu Saloranta [e-mail] wrote:

@jjzabkar Looks like you might be using something like Lombok? That seems to be particularly prone to all kinds of problems unfortunately. Big problem form my perspective is just that what I need is expanded class definition to see how Jackson sees the type after post-processing (or recompilation? I forget where Lombok does its magic), but obviously from user perspective it is irrelevant (should just work). So reproducing the issue is trickier.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[cowtowncoder]

@jjzabkar yes. Same is true of a few other auto-bean frameworkers. The trick is to find what is the actual class representation Jackson sees, with annotations, and configuration settings. Similar challenges occur with JVM languages like Kotlin and Scala; both of which are supported without databind having any knowledge of their peculiarities (although additional testing obviously is done by actual Scala and Kotlin modules, where respective sources are used).

Anyway: for verification and regression post-processed types would be needed.

[jjzabkar]

@cowtowncoder Sounds like the workaround here (to perhaps save other Googlers) is to remove Lombok and add in the vanilla getters/setters so that Jackson can see the _actual class representation_.

[cowtowncoder]

@jjzabkar Yes, for purpose of reproduction. I do think it is possible to understand the issue and either fix it or at least see which lombok annotations have issues in what situations.

[jjzabkar]

@cowtowncoder I'm worried that I might've hijacked this thread enough :). But, if I haven't, I would love to investigate:

The trick is to find what is the actual class representation Jackson sees

Even though I've been using Jackson for years, I've never really dug into how databind works. If you have a pertinent link to point my investigation in the right direction, please share.

[cowtowncoder]

@jjzabkar What I really mean is that I'd love to know exactly what kind of class exists after Lombok is done with its thing.

But within Jackson if you would like to understand more, much of handling is via classes in com.fasterxml.jackson.databind.introspect, in particular:

• POJOPropertiesCollector (handles all properties of a POJO)
• POJOPropertyBuilder (collects info on individual logical property)

Handling of Creators, however, is bit more scattered and I think mostly in:

• BeanSerializerFactory for serialization
• BeanDeserializerFactory for deserialization

Access to annotations is through AnnotationIntrospector (and in case of Jackson annotations, JacksonAnnotationIntrospector); it may make sense to see who calls which methods.

Workflow itself starts from SerializerProvider / DeserializationContext, when (de)serializer is needed and lookup starts. Most information is initially collected using:

• AnnotatedClass for basic field/method/ctor, type hierarchy info (including mixing in of mix-ins)
• POJOPropertiesCollector (mentioned earlier) combines various accessors into logical properties
• BasicBeanDescription is constructed from this info.

I wish I had better documentation to offer, but I hope this gives some idea where to look if you are interested.

One last tidbit: I have noticed that in many Lombok/constructor problem cases, @JsonBackReference has been part of the problem.

[cowtowncoder]

FWTW, I think this is the same as #1516. Improvements have been made for 2.8.8 / 2.9.0 so that the problem is described better in exception, but fundamentally the problem is that @JsonBackReference may not necessarily be passed via constructor: sometimes it is legal (basically if parent object does not use argument-taking constructor, and is thus available to be passed), othertimes not.

Work-arounds include:

1. Removal of @AllArgsConstructor for child class (but do need no-args ctor!)
2. Disabling of MapperFeature.INFER_CREATOR_FROM_CONSTRUCTOR_PROPERTIES (only added in 2.9, however)

Anyway: since #1516 is tracking further progress, future discussion should occur there.

[ManivannanMurugavel]

Remove the Three jar file from zebblein lib folder(/usr/local/zebblein/lib/)


Start with jackson-annotations-2.5.3.jar,jackson-core-2.5.3.jar,jackson-databind-2.5.3.jar


It's Worked

[OndraZizka]

How do I find which class causes this? What I see is:

` com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Duplicate creator property "id" (index 0 vs 1) at [Source: (String)"{ "description": "patched description"}"; line: 1, column: 1] at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:62) at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:221)

[cowtowncoder]

@OndraZizka please do not comment on closed issues.

As to how: first make sure to upgrade to as recent version as possible; more information has been added on exceptions. Given source document, however, it seems it should be target type you pass to readValue().