Jackson-databind: [feature request] javatuples (immutable types) support (moved to FasterXML/jackson-future-ideas#25)

[paulvi]

Please support javatuples https://www.javatuples.org/ as special case.

Jackson usually requires default (empty) Constructor and setters.
But javatuples are by design immutable, so adding setters would violate immutability contract.

Possibly there was already similar issue for other immutable types, and possible solution to try.

[cowtowncoder]

Ok: this would actually belong in:

https://github.com/FasterXML/jackson-future-ideas/issues

So, to do this, someone would need to write a datatype module implementation, similar to jackson-datatype-joda and jackson-datatype-guava. I hope someone with an itch is interested since I don't have time myself to do this (nor have personal use case).

Immutable types themselves are fine: what deserializers would need to do is simply use builders and/or argument-taking constructors.

[paulvi]

Is it correct that there are only 2 ways possible:

1. Raise an issue at https://github.com/FasterXML/jackson-future-ideas/issues for new jackson-datatype-javatuples and most likely still would those interested would need to do themselves
2. use builders and/or argument-taking constructors.
   Would use of @JsonCreator be right direction?
   https://github.com/javatuples/javatuples/issues/3#issuecomment-386324617

Is it possible to apply @JsonCreator for (final) classes defined in library jar?

[paulvi]

Also seems to be way 3: static @JsonCreator factory methods , ref https://github.com/FasterXML/jackson-future-ideas/issues/10, https://github.com/FasterXML/jackson-databind/issues/1820

[cowtowncoder]

@paulvi What is the canonical way of constructing instances? Builder pattern might work, and use of mix-in annotations does allow assigning all kinds of annotations to 3rd party types. Some datatype libraries use mix-ins to reduce amount of code to write.

As to part of "those interested would need to do themselves" -- yes, but the whole idea of datatype modules is that this only needs to happen once and then others can make use of functionality. When using other mechanisms what often happens is that everyone needs to discover & cut-n-paste bunches of configurations, or half-ready (de)serializers, which work for their particular usage and not others. So in that sense I think a proper datatype module is the right way and saves everyone time.

But as to good way to go: what is the usual way of constructing instances? Is there some sample/example page for usage? I don't have time to work on this myself, but you could raise this also on mailing list(s) (jackson-user and/or jackson-dev) to maybe find interested collaborators. This seems like a really good project for someone who would like to get start with a cool OSS project I think.

[paulvi]

https://www.javatuples.org/using.html is to be said to list canonical ways.

And it says:

Pair.with("hello", Integer.valueOf(23));

or

new Pair("hello", Integer.valueOf(23));

or

Pair.fromCollection(myCollectionWith5Elements);

[cowtowncoder]

Ok. So both constructor and of factory method would be eligible. It would probably be possible to generate matching ValueInstantiator (StdValueInstantiator might work fine) -- this is what is created from @JsonCreator anyway.

One obvious question here is that of what should be default JSON representation: looks like there are at least 2 general approaches that would work:

1. Use JSON Array, positionally. Compact.
2. Use JSON Object, with name representing position ("value1", .... ?). Possibly more readable? But more verbose certainly.

For some of types, specifically Unit and Pair there would be alternatives too: Unit could just map from value itself ("delegating" creator), and Pair could, at least in some cases, use single-entry Object (key / value).
But whether having optimizations like that make sense... probably not.

Serialization side should produce matching output.

Also: if both alternatives are to be supported, datatype would then allow use of @JsonFormat(shape = ...). It could, for example, default to Shape.ARRAY but allow alternative of Shape.OBJECT.

I am actually getting bit more interested in this... maybe I should find time to work on it, at least creating skeleton?

Finally: one thing that could be helpful would be set of unit tests showing expected working behavior. That would be helpful for verifying funcionality.

[paulvi]

I wonder if it is possible for end users to solve with current versions of javatuples/jackson?

The idea of #1820 was great: some other code, that is factory with @jsoncreator annotation applied.

And in general I would favor more general (and minimal) solution.

But as you said

This seems like a really good project for someone who would like to get start with a cool OSS project

[cowtowncoder]

@paulvi Before considering solution, it is necessary to consider what the specific problem is. It is not enough to simply state that tuples should be serialized and deserializer, but also consider representation to use. And to support full range of features contained types (including polymorphic types, custom (de)serialization), it is necessary to make them handle as containers of some sort.
So the answer depends on specific goals.

If representation to use is JSON Arrays, I suspect that it is not easy for end users to configure things to work. If JSON Objects it might be possible, possibly using @JsonCreator annotations via mix-ins -- basically, associate @JsonCreator (and @JsonProperty for arguments, I think) with all tuple types. This might work for deserialization. For serialization I am not sure, it depends on whether there are named getter-methods.
Writing custom serializer would probably quite a bit simpler than deserializer, however, so perhaps that kind of hybrid solution would be simplest approach (not simple, but simplest...)

[paulvi]

From my understanding tuples are feature of some computer languages (Python, .NET: C#, F#, TypeScript ), but not in Java or JS and so not in JSON.
That would lead to long discussions on language design.
While it is not possible to decide for all: some users would want one way, others other.
In the end both options may be asked for.
But as of now it is mostly of blocker to use tuples libarary, as it is not supported (or it is unknown how to use) with default JSON library that Jackson is.
So simple 1st implementation is solution will be enalber,
otherwise it is just not possible at all.

I just think JSON serialization should be first convenient for operations within JS.
And I bet array is better.
It may help to check what is default JSON serialization of tuples in languages that have tuples. (But I don't know those languages)

Check also performance testing https://jsperf.com/array-vs-object-tuple

Array is faster for small tuples:

See also interesting case when object key is also tuple

https://stackoverflow.com/questions/715550/best-way-to-encode-tuples-with-json

Other links from quick search:

• https://www.tutorialspoint.com/typescript/typescript_tuples.htm
• http://gorodinski.com/blog/2013/01/05/json-dot-net-type-converters-for-f-option-list-tuple/
• https://github.com/Olical/tuple

Tuples are array-like objects

• https://stackoverflow.com/questions/4512405/javascript-variable-assignments-from-tuples

[cowtowncoder]

Right, what I mean is this: how Tuples work in a language does not necessarily dictate how they are express in JSON. So although it is good to make things "look the same", sometimes mapping has to either change the representation, or makes more sense. Or just need to select one way over another.

I am fine with mapping them to JSON Array by default; and if someone wants alternative, offer that if there is interest.
I just wanted to bring this up since like I said it may not be easy to achieve that option without custom serializers/deserializers.

Part of the problem here is that Tuples really are not quite array/List/Collections, since they are heterogenous (types of elements vary by index, with no shared base type).
While serialization would be possible to make work (at least for non-polymorphic element types), deserialization wouldn't.

If types were same, existing functionality used with Java arrays, Collections might work to allow mix-in @JsonCreator / @JsonValue annotations.

[paulvi]

Yes, indeed.
While serialization to JSON Array is the most widely used and should be default,
deserialization is more problematic:

However for an example, that shows that tuples being List/Collection of Object is not really super special case:

List<Vehicle> = new ArrayList<Vehicle>() {{
    add( new Bicycle() );
    add( new Car() );
    add( new Train() );
}};

So JSON array (or Java Collection) may be defined with element of base type,
while the content is instance of more specific (deeper in hierarchy) type.
I wonder would this deserialization case can work in Jackson now.

This particular tuples implementation has add method, that is feels like Collection.add(),
but returned next time

e.g. Pair has add() that make new Triplet (and so on)

    public <X0> Triplet<A,B,X0> add(final X0 value0) {
        return addAt2(value0);
    }

So while type is immutable, it is still possibly to modify it by creating new (just like Strings)

The bigger problem is that implementation is limited to 10: Decade, and if one needs more than 10 elements in tuple (e.g. when receiving tuple from other language), that will become
Object[] or Collection<Object> or should be Exception ?

[cowtowncoder]

I don't think Tuples are Collections at all, as per Javadocs. That makes sense to me sense they really aren't Collections logically -- due to heterogenity, having fixed size and so forth -- and making them extend that type would seem logically wrong (even if convenient).
That would just be bad OOP design.

This being the case, existence of add method does not matter since it's not part of interface jackson-databind knows or can have dependency.
As to size limitation, yes, beyond 10 Tuple deserializer should throw an exception. And for specific subtypes, exact size should match otherwise exception should be thrown as well.

But at this point I do not see any benefit speculating on work-arounds that might or might not help hack something together without datatype module.

[cowtowncoder]

Moved to

https://github.com/FasterXML/jackson-future-ideas/issues/25

since support can not be added by any simple changes, and as a matter of policy no dependencies to external 3rd party libraries should be added.

[paulvi]

So support only can be some new special Jackson data-binding extension module

[cowtowncoder]

@paulvi or custom (de)serializers. I can not think of a way using just configuration or mix-in annotations.

[paulvi]

As commented in https://github.com/FasterXML/jackson-databind/issues/1820#issuecomment-388517251

Just to make sure: mix-in annotations can already be attached to static factory methods without problems.

Just tried with Jackson 2.9

@JsonIgnoreProperties("size")
abstract class MyPairMixIn6 { 

    @JsonCreator
    public static Pair factory(@JsonProperty("value0") Long value0, @JsonProperty("value0") String value1) {
        return Pair.with(value0, value1);
    }
}

and

    public static void main(String[] args) throws JsonGenerationException, JsonMappingException, IOException {
        //for Pair
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.addMixIn(Pair.class, MyPairMixIn6.class);

        Pair<Long, String> pair = Pair.with(789L,"Borris");

        objectMapper.writerWithDefaultPrettyPrinter().writeValue(new File("pair6.json"), pair);
        String json2 = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(pair);
        System.out.println(json2);


        String s = "{\"value0\" : 456, \"value1\" : \"John\"}";
        Pair<Long, String> pair2 = objectMapper.readValue(s, Pair.class);
//        Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.javatuples.Pair` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
//        at [Source: (String)"{"value0" : 456, "value1" : "John"}"; line: 1, column: 2]
//          at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
        System.out.println(pair2);

    }

result is while @JsonIgnoreProperties("size") works for serialization,
for deseralization @JsonCreator is just IMHO not visible/ not noticed

{
  "value0" : 789,
  "value1" : "Borris"
}
Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.javatuples.Pair` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (String)"{"value0" : 456, "value1" : "John"}"; line: 1, column: 2]

[cowtowncoder]

@paulvi Without definition of Pair, it's hard to verify, but one thing to make sure is that signature must match 100% regarding types (that is, can not use sub-class substitution for arguments) and name. If so, mix-in annotations do work for static factory methods too.

But the contents of the method are fully ignored -- code in mix-in class is never used for anything, ever. So mix-in can only cause code in target class to be called, not code in mix-in.

Ability to actually call static method of mixin class is what that future improvement idea talks about.

[paulvi]

javatuples via maven

    <dependencies>
        <dependency>
            <groupId>org.javatuples</groupId>
            <artifactId>javatuples</artifactId>
            <version>1.2</version>
        </dependency>

public final class Pair<A,B> 
        extends Tuple
        implements IValue0<A>,
                   IValue1<B> {

    private static final long serialVersionUID = 2438099850625502138L;

    private static final int SIZE = 2;

    private final A val0;
    private final B val1;

    public static <A,B> Pair<A,B> with(final A value0, final B value1) {
        return new Pair<A,B>(value0,value1);
    }
     ...
    public Pair(
            final A value0, 
            final B value1) {
        super(value0, value1);
        this.val0 = value0;
        this.val1 = value1;
    }

I think the problem is objectMapper.readValue(s, Pair.class);

as Pair.class is not the same as class of Pair<Long, String>

P.S. I tried many variations of MixIn, none could fit

@JsonIgnoreProperties("size")
abstract class MyPairMixIn7<Long, String> {  // with and without <Long, String>
    @JsonCreator
    public Pair<Long, String> with( // with and without static or <Long, String>
            @JsonProperty("value0") final Long value0, // with and without final 
            @JsonProperty("value1") final String value1) // with and without final 
    {// body does not matter, only signature
        return Pair.with(value0, value1);
    }
}

[cowtowncoder]

@paulvi Try declaration as:

    @JsonCreator
    public Pair<Long, String> with(
            @JsonProperty("value0") Object value0,
            @JsonProperty("value1") Object value1

the thing is that as far as bytecode is concerned, type erasure will make type variable A and B look much like java.lang.Object (if there was "extends", it'd be raw class after "extends" etc).
So those raw types have to match; generic parameters, if any, are ignored. This is how JVM matches method signatures.

As to return type, only raw type (Pair) matches as well: you can use type parameters or ignore, should not matter.

[paulvi]

Thanks. Still cannot manage to let it be seen

@JsonIgnoreProperties("size")
abstract class MyPairMixIn8 { // with and without <Long, String>
    @JsonCreator
    public Pair with( // with and without <Long, String>
            @JsonProperty("value0") Object value0, 
            @JsonProperty("value1") Object value1) 
    {// body does not matter, only signature
        return Pair.with( (Long)value0, (String)value1);
    }
}

        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.addMixIn(Pair.class, MyPairMixIn8.class);

        String s = "{\"value0\" : 456, \"value1\" : \"John\"}";
        Pair<Long, String> pair2 = objectMapper.readValue(s, Pair.class);
        System.out.println(pair2);

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.javatuples.Pair` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (String)"{"value0" : 456, "value1" : "John"}"; line: 1, column: 2]
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)

[cowtowncoder]

@paulvi Ok. I'll have a look. This should work

[cowtowncoder]

@paulvi Interesting. I can reproduce the problem. Existing tests cover delegating case only it seems, so guessing this might relate to missing @JsonProperty mix-in... but we'll see.

[cowtowncoder]

@paulvi Ah! It does work. But you had forgotten static modifier from with(...) method. That must match; it is considered part of matching. Once mix-in method is made static to match target, test finally works.

[paulvi]

Indeed, it worked . Even though I played with static before, as shown in https://github.com/FasterXML/jackson-databind/issues/2020#issuecomment-389079413

Anyways, what works looks like:

Mixin

import org.javatuples.Pair;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.annotation.JsonProperty;

@JsonIgnoreProperties("size")
abstract class MyPairMixIn9 { 
    @JsonCreator
     public static Pair with( 
            @JsonProperty("value0") Object value0, 
            @JsonProperty("value1") Object value1) 
    {// body does not matter, only signature
        return Pair.with( (Long)value0, (String)value1);
    }
}

Usage Code

    public static void main(String[] args) throws JsonGenerationException, JsonMappingException, IOException {
        // for Pair
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.addMixIn(Pair.class, MyPairMixIn9.class);

        String s = "{\"value0\" : 456, \"value1\" : \"John\"}";
        Pair<Long, String> pair2 = objectMapper.readValue(s, Pair.class);
        System.out.println(pair2);

        Pair<Long, String> pair = Pair.with(789L, "Boris");
        //objectMapper.writerWithDefaultPrettyPrinter().writeValue(new File("pair6.json"), pair);
        String json2 = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(pair);
        System.out.println(json2);

    }

Result:

[456, John]
{
  "value0" : 789,
  "value1" : "Boris"
}

[paulvi]

And thanks for blogging long time ago http://www.cowtowncoder.com/blog/archives/2009/08/entry_305.html