Gson: Feature Request: Add deserializeNulls

[julianstokkur]

There's already a serializeNulls option, and with the introduction of Kotlin I think it would be good to have a deserializeNulls, right now serializeNulls is only a method that sets a boolean to true, but probably that can be a method that receives a boolean, this way you can reuse the same builder to construct different gsons, the reasoning behind adding deserializeNulls is that since you can mark some fields in Kotlin as non null you expect to be able to use the field with no problem, but if the field in the json string comes as null then you will get an exception at runtime when using that property since gson set that field to null, this is the same as using NonNull annotations for Java where you can "safely use the field", there are some cases where someone can think of something being null as a default case for example if a class has a list of items it's technically the same thing having a null or an empty list in both cases you have no items.

Right now if you define a class with a default constructor and default values for it's fields, gson will make the field null if the json string has it as null, but since you already defined a default value this will "break" the null safety you had, no matter if it's Java or Kotlin. We can always define type adapters for this, but why would all the consumers of the library have to do that if gson could only provide a deserializeNulls option?

[lyubomyr-shaydariv]

We can always define type adapters for this, but why would all the consumers of the library have to do that if gson could only provide a deserializeNulls option?

I don't think it's possible due to several reasons:

• Default values that might replace nulls are not first-class citizens in JVM, and the respective fields are initialized in constructor bodies (implicitly generated and copied to constructors by javac).
• There is no way to get field default values using Java Reflection.
• Gson uses either the default default constructor or an "unsafe allocator" if the default constructor does not exist, therefore omitting field initializers completely if there's no default constructor.
• Kotlin does not seem to provide any default arguments (at least for data classes) information as well.

Simply speaking, there is no a way of simple mapping from nulls to default values. Combining #1377 and https://github.com/google/gson/tree/master/extras/src/main/java/com/google/gson/interceptors might result in something like:

final class KotlinPostProcessor
        implements Function<TypeToken<?>, Consumer<?>> {

    private final Function<? super Field, ?> fieldToDefaultValue;

    private KotlinPostProcessor(final Function<? super Field, ?> fieldToDefaultValue) {
        this.fieldToDefaultValue = fieldToDefaultValue;
    }

    static Function<TypeToken<?>, Consumer<?>> of(final Function<? super Field, ?> fieldToDefaultValue) {
        return new KotlinPostProcessor(fieldToDefaultValue);
    }

    @Override
    @Nullable
    public Consumer<?> apply(final TypeToken<?> typeToken) {
        final Class<?> clazz = typeToken.getRawType();
        if ( !isKotlinClass(clazz) ) {
            return null;
        }
        return o -> Defaults.resolveNonnulls(o, fieldToDefaultValue);
    }

    private static boolean isKotlinClass(final AnnotatedElement annotatedElement) {
        return annotatedElement.getDeclaredAnnotation(Metadata.class) != null;
    }

}

final class Defaults {

    private Defaults() {
    }

    static void resolveNonnulls(final Object object, final Function<? super Field, ?> fieldToDefaultValue) {
        for ( Class<?> clazz = object.getClass(); clazz != null && clazz != Object.class; clazz = clazz.getSuperclass() ) {
            for ( final Field field : clazz.getDeclaredFields() ) {
                if ( !Modifier.isStatic(field.getModifiers())
                        && !Modifier.isTransient(field.getModifiers())
                        && !field.getType().isPrimitive() ) {
                    field.setAccessible(true);
                    try {
                        @Nullable
                        final Object value = field.get(object);
                        if ( value == null ) {
                            field.set(object, fieldToDefaultValue.apply(field));
                        }
                    } catch ( final IllegalAccessException ex ) {
                        throw new IllegalArgumentException(ex);
                    }
                }
            }
        }
    }

}

data class Person(
        val firstName: String = "<NO FIRST NAME>",
        val lastName: String = "<NO LAST NAME>"
);

fun printToStdOut(message: String) {
    System.out.println(message);
}

fun main(args: Array<String>) {
    val gson = GsonBuilder()
            .registerTypeAdapterFactory(PostProcessorTypeAdapterFactory.of(KotlinPostProcessor.of({ field ->
                when (field.type) {
                    String::class.java -> "" // for example
                    else -> throw UnsupportedOperationException(field.type.toString());
                }
            })))
            .create();
    val jsonReader = ... { "firstName": "John" } ...; // no `lastName`
    jsonReader.use {
        val person = gson.fromJson<Person>(jsonReader, Person::class.java);
        printToStdOut(person.firstName);
        printToStdOut(person.lastName);
    }
}

The result is as follows:

<NO LAST NAME>

Omitting the Person class fields default values will print

/* empty string */

[julianstokkur]

I'm not saying that gson needs to insert default values, what I'm asking is the possibility to add deserializeNull, meaning that gson will either ignore or listen to any null value inside a json string while deserializing, using the same example you have when you deserialize the following json

{"firstName":"Someone","lastName":null}

The resulting object will be

person.firstName = "Someone"
person.lastName = null

What I'm asking is if gson could add something to ignore that null in the lastName or listen to it depending on the flag, I debugged the library and got to the class ReflectiveTypeAdapterFactory.java, around line 132, there's this block of code inside the function private BoundField createBoundField(...)

if (fieldValue != null || !isPrimitive) {
  field.set(value, fieldValue);
}

As you can see, if this factory class would have something to ignore nulls while deserializing then the default values that you set in your class (either be data class in Kotlin or just assigned values when declaring a field in Java) gson will not override them with null. So what I'm asking is if gson could add something in the GsonBuilder similar to serializeNulls that gets propagated to the writer, but this case would be to the gson reader so the above block would end up something like

if (fieldValue != null || !isPrimitive || deserializeNulls) {
  field.set(value, fieldValue);
}

By the way, Kotlin do generates a non arg constructor when you declare a data class that all fields have default values, if you decompile the Person data class you wrote, you will get these constructors

public Person(@NotNull String firstName, @NotNull String lastName) {
      Intrinsics.checkParameterIsNotNull(firstName, "firstName");
      Intrinsics.checkParameterIsNotNull(lastName, "lastName");
      super();
      this.firstName = firstName;
      this.lastName = lastName;
   }

   // $FF: synthetic method
   public Person(String var1, String var2, int var3, DefaultConstructorMarker var4) {
      if ((var3 & 1) != 0) {
         var1 = "<NO FIRST NAME>";
      }

      if ((var3 & 2) != 0) {
         var2 = "<NO LAST NAME>";
      }

      this(var1, var2);
   }

   public Person() {
      this((String)null, (String)null, 3, (DefaultConstructorMarker)null);
   }

You can see that the no args constructor calls $FF: synthetic method which sets the default values.

[julianstokkur]

I tried with the approach you said, adding the post processing type adapter factory, and it did work, but it's obviously slower and tedious having to write all the types you want to have default values, it's also hard because it's not really simple to do a List::class.java, besides default values could not be the same in one class or another, maybe you want one class to have an empty phone number but an emergency class to have 911 as default phone number, both are strings but with the type adapter factory approach becomes harder, I tried to add the deserializeNulls in the gson source code, and I got it to work, at least for the reflective type adapter factory, see in the attached file the git diff I got after doing the changes:

deserializeNulls.txt

I didn't do the whole thing for all type adapter factories because I only went and check out the reflective one, so I have no clue what the others might do, but the diff probably is a start?

[leleliu008]

@julianstokkur highly recommend you try fastJSON

[lyubomyr-shaydariv]

@julianstokkur

but the diff probably is a start?

No reason. Follow the extremely helpful comment by @leleliu008.

[lyubomyr-shaydariv]

1550