Jackson-databind: FeatureRequest: Deserialize null to empty collection

[leifhanack]

He there,

Hi there,
what to you think about adding the possibility to map null values to empty collections?

If I have a object

// pseudo code
class Person {
  String name
  List<String> nicknames = new ArrayList()
}

and a json

{
  "name": "jackson",
  "nicknames": null
}

it would be cool to get an instance where the nicknames collection is empty instead of null

  assert person.nicknames.size() == 0

Best regards, Leif

[saden1]

+1
...nulls are really evil!

[cowtowncoder]

I am not super happy adding all various coercions and transformations -- if JSON says 'null', simplest way is to match to null. If empty Collection was wanted, express it as such.

Having said that I am not opposed to this if anyone wants to take a stab at implementation.

[tadhgpearson]

+1
Would the best way to do it be a DeserializationFeature? For example DeserializationFeature.READ_NULL_ARRAY_AS_EMPTY_COLLECTION ?

[cowtowncoder]

Minor nitpick: there is no "null array" in JSON, just null and empty array "[ ]". Expected type of null depends on data-binding target. So name should not refer to 'array' in this case. Empty JSON arrays may of course be read as arrays or java.util.Collections.

[tadhgpearson]

Makes sense. I guess a better way of expressing it would be, when data binding to an instance of Collection, the deserialization mechanism should produce an empty collection rather than a null object if the corresponding key in the JSON being deserialized does not exist or is null (is that possible?).

So maybe DeserialiaztionFeature.READ_NULL_COLLECTION_AS_EMPTY?

I also just realized there's a SerializationFeature.WRITE_EMPTY_JSON_ARRAYS so this is just the reverse, I don't know if the names make that obvious.

Who was it that said the hardest thing about software development was naming things? :)

[bugs84]

+1
handling nulls is evil

[tadhgpearson]

I made a brief attempt to implement this on Thursday evening.
I did a spec and built some unit tests, but in the 3 hours I spent, I didn't manage to close all the edge cases I defined in my tests.
Specifically, I had difficulty with cases when the parses switches between when vanilla processing becomes non-standard processing. I might come back to it in the future, but I don't have time right now :(

[cowtowncoder]

Nulls are great. It seems odd to have all these coercions from existing nicely mapping concepts into something different.

Maybe I should add a feature that allows converting empty Collections to nulls. :-D

[tadhgpearson]

haha - why not? It's good for a framework to be opinionated :)

[cowtowncoder]

@tadhgpearson :)

To be honest, the biggest practical challenge is that handling of null values is quite deeply ingrained, so that JsonDeserializers' options are limited.

But this does point out one practical problem: while there is JsonDeserializer.getNullValue(), it does not take arguments. Because of this, it does not have access to active set of DeserializationFeatures. It should.

So I will probably need to first deprecate existing version, add a new one that takes DeserializationContext, and then this problem could be solved. Or perhaps add even more general facility to do other null co(nv)ercions.

[cowtowncoder]

Good news everybody! :)

So, #741 was implemented (for 2.6), so configuration settings are now accessible from getNullValue(...) so that one practical issue is now resolved.
Given this, configurable null/empty Collection handling could be implemented, I think. Scope of changes is non-trivial, since there are many deserializers involved. But it should not require complicated code, just bit of it in quite a few places. If anyone wants to try, I can help. If not, I'll eventually circle back, it's just that the pile of things to do is quite big.

[orybnikov]

+1

[smaudet]

Cross posting from: https://github.com/FasterXML/jackson-databind/issues/741

I took a hack at this, however I got confused amid the (overgrown, unmarked, and unweeded) garden of various Factories, Modules, Providers, Delegators, and ClassUtils, and Abstract implementations with no direct implementation other than in tests...

Is there a road map to Jackson somewhere that I'm missing?

Or a 'right direction' to travel to get this sort of thing working?

I solved my original problem, incidentally, with a JsonSerialize annotation and a private class on a DTO, but feeling like I nearly got it right doing it the other way, I'm curious how I could/should have been going about using the fruits of this change...

For shiz & giggles:

https://gist.github.com/smaudet/1db10990d8cac00f369d

Basically, I tried to add a Deserializer impl with an overridden getNullValue, and got stuck trying to find a base class that I could easily override.

I came across StdConverter, but as this is abstract, and the only examples of usage are in Tests cases, and the only reason I was even trying that was to subclass StdDelegatingConverter, I decided maybe I was barking up the wrong tree.

[tadhgpearson]

@smaudet Well - thanks for trying, it's reassuring to know that I'm not the only one who didn't manage to do this :p

[obozek]

+1

[hexonxons]

+1

[davinkevin]

+1

[shyiko]

For what it worth, requested behavior can be achieved with

ObjectMapper().registerModule(object : SimpleModule() { init {
    setDeserializerModifier(object: BeanDeserializerModifier() {

    // null as Map -> emptyMap
    override fun modifyMapDeserializer(config: DeserializationConfig?, type: MapType?,
        beanDesc: BeanDescription?, deserializer: JsonDeserializer<*>):
        JsonDeserializer<*>? =
        object : JsonDeserializer<Map<Any, Any>>(), ContextualDeserializer,
            ResolvableDeserializer {
            @Suppress("UNCHECKED_CAST")
            override fun deserialize(jp: JsonParser, ctx: DeserializationContext?):
                Map<Any, Any>? =
                deserializer.deserialize(jp, ctx) as Map<Any, Any>?

            override fun createContextual(ctx: DeserializationContext?,
                property: BeanProperty?):
                JsonDeserializer<*>? =
                modifyMapDeserializer(config, type, beanDesc,
                    (deserializer as ContextualDeserializer)
                        .createContextual(ctx, property))

            override fun resolve(ctx: DeserializationContext?) {
                (deserializer as? ResolvableDeserializer)?.resolve(ctx)
            }

            override fun getNullValue(ctx: DeserializationContext?): Map<Any, Any>? =
                emptyMap()
        }

    // null as List -> emptyList
    override fun modifyCollectionDeserializer(config: DeserializationConfig?, 
        type: CollectionType?, beanDesc: BeanDescription?, 
        deserializer: JsonDeserializer<*>):
        JsonDeserializer<*>? =
        object : JsonDeserializer<List<Any>>(), ContextualDeserializer {
            @Suppress("UNCHECKED_CAST")
            override fun deserialize(jp: JsonParser, ctx: DeserializationContext?):
                List<Any>? =
                deserializer.deserialize(jp, ctx) as List<Any>?

            override fun createContextual(ctx: DeserializationContext?,
                property: BeanProperty?): 
                JsonDeserializer<*>? =
                modifyCollectionDeserializer(config, type, beanDesc,
                    (deserializer as ContextualDeserializer)
                        .createContextual(ctx, property))

            override fun getNullValue(ctx: DeserializationContext?): List<Any>? =
                emptyList()
        }

    })
}})

(code is in Kotlin but the idea should be clear)

[cowtowncoder]

@shyiko thank you for sharing this!

[cowtowncoder]

Quick note: will tackle this to some degree with #1269: at very least will allow "skipping" of input nulls (so default value of a property can stay as is); may further push into converting nulls into "empty values" as reported by JsonDeserializer.

[cowtowncoder]

Minor correction: issue that replaces this is #1402. It should allow both skipping (to avoid override, allow retaining of original value) and some form of "use default/empty". So, closing this one.