Moshi: Incomplete hierarchy error on nested Kotlin Sealed Classes when obfuscated

[kgbier]

Upgrading to moshi v1.10.0 has broken deserialisation of sealed classes when they appear to be nested inside other structures, and the code has been minified.

The precise exception and callstack is as follows:

java.lang.RuntimeException: Unable to create application com.example.MainApplication: java.lang.IllegalStateException: Incomplete hierarchy for class OptionOne, unresolved classes [com.example.WrappingType$OptionType]
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6465)
        at android.app.ActivityThread.access$1300(ActivityThread.java:219)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1859)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:214)
        at android.app.ActivityThread.main(ActivityThread.java:7356)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
     Caused by: java.lang.IllegalStateException: Incomplete hierarchy for class OptionOne, unresolved classes [com.example.WrappingType$OptionType]
        at kotlin.reflect.jvm.internal.impl.descriptors.runtime.components.RuntimeErrorReporter.reportIncompleteHierarchy(:26)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassTypeConstructor.computeSupertypes(:198)
        at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$1.invoke(:80)
        at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$1.invoke(:26)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(:370)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.invoke(:443)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValueWithPostCompute.invoke(:474)
        at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor.getSupertypes(:27)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope.getNonDeclaredVariableNames(:306)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$variableNamesLazy$2.invoke(:77)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope$variableNamesLazy$2.invoke(:40)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(:370)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(:489)
        at kotlin.reflect.jvm.internal.impl.storage.StorageKt.getValue(:42)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.getVariableNamesLazy(Unknown Source:7)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.getVariableNames(:91)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.addFunctionsAndProperties(:216)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedMemberScope.computeDescriptors(:187)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope$allDescriptors$1.invoke(:227)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope$allDescriptors$1.invoke(:220)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(:370)
        at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(:489)
        at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$DeserializedClassMemberScope.getContributedDescriptors(:237)
        at kotlin.reflect.jvm.internal.impl.resolve.scopes.ResolutionScope$DefaultImpls.getContributedDescriptors$default(:52)
2020-08-29 11:33:47.451 30854-30854/com.example E/AndroidRuntime:     at kotlin.reflect.jvm.internal.KDeclarationContainerImpl.getMembers(:57)
        at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(:161)
        at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(:46)
        at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(:92)
        at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(:31)
        at kotlin.reflect.jvm.internal.KClassImpl$Data.getDeclaredNonStaticMembers(Unknown Source:8)
        at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(:170)
        at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(:46)
        at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(:92)
        at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(:31)
        at kotlin.reflect.jvm.internal.KClassImpl$Data.getAllNonStaticMembers(Unknown Source:8)
        at kotlin.reflect.full.KClasses.getMemberProperties(:149)
        at com.squareup.moshi.kotlin.reflect.KotlinJsonAdapterFactory.create(:227)
        at com.squareup.moshi.Moshi.adapter(:141)
        at com.squareup.moshi.Moshi.adapter(:101)
        at com.squareup.moshi.Moshi.adapter(:75)
...

The data structure in question:

@Keep
data class WrappingType(val option: OptionType) {
    sealed class OptionType {
        @Keep
        data class OptionOne(val first: String) : OptionType()
    }
}

Minimal code to reproduce:

val moshi: Moshi = Moshi.Builder()
    .add(KotlinJsonAdapterFactory())
    .build()

val jsonString = "{ \"first\": \"abcde\" }"

val adapter = moshi.adapter(WrappingType.OptionType.OptionOne::class.java)

adapter.fromJson(jsonString)

Removing the nesting solves this:

@Keep
data class WrappingType(val option: OptionType)

sealed class OptionType {
    @Keep
    data class OptionOne(val first: String) : OptionType()
}

Making the parent non-sealed solves this:

@Keep
data class WrappingType(val option: OptionType) {
    class OptionType {
        @Keep
        data class OptionOne(val first: String)
    }
}

Adding the following line to the proguard config solves this:

Adding the following dreaded line to the proguard config does not solve this:

-keep class **.* { *; }

[ZacSweers]

And what happens if you @Keep the sealed class itself? OptionType is referenced from OptionOne but @Metadata probably is unaware of your obfuscation. In any case, I don't think this is a Moshi problem, the previous proguard rules tried to haphazardly keep certain things and with Kotlin 1.4 you should keep things like any non-kotlin reflective serialization would. This likely includes sealed base types

[kgbier]

@Keep on the sealed class itself results in the same problem

@Keep
data class WrappingType(val option: OptionType) {
    @Keep
    sealed class OptionType {
        @Keep
        data class OptionOne(val first: String) : OptionType()
    }
}

[ZacSweers]

Not sure then, you'll need to figure out what the right proguard rules are for your project then. Probably worth an issue on the kotlin issue tracker if you get to the bottom of it. It's possible it automagically worked before because moshi packaged incomplete and often overly conservative rules. Now Moshi defers to kotlin-reflect for reflect machinery rules and the rest is down to consuming projects to keep the appropriate classes for reflective serialization. As such, I don't think this is a bug in moshi and we should close this.

You should really use code gen in production though if you care about easy compatibility with proguard/R8.

[hectorups]

@kgbier we are still on 1.9.3 and had the same issue. We narrowed it down to the release where we bumped up AGP to 4.0.1 (and Gradle to 6.1.1)

[kgbier]

Experimentation on my end is showing something similar, it's some weird combo of those two other variables.
Thanks @hectorups, thanks for your time @ZacSweers.
Closing as there isn't much to take action on.

[OlivierGenez]

In case you're still looking into this, it turns out this is an R8 issue: https://issuetracker.google.com/issues/169264693. In particular:

The kotlin metadata-processing in AGP 4.0 (R8 version 2.0) is not working correctly and we only started fully supporting kotlin metadata from AGP version 4.1 (R8 version 2.1).

Android Studio 4.1 stable should come with a version of AGP/R8 which supports this use case correctly. In the meantime, you can manually configure your top-level build.gradle file to use a more recent version of R8:

buildscript {

    repositories {
        maven {
            url 'https://storage.googleapis.com/r8-releases/raw'
        }
    }

    dependencies {
        classpath 'com.android.tools:r8:2.1.68'          // Must be before the Gradle Plugin for Android.
        classpath 'com.android.tools.build:gradle:X.Y.Z' // Your current AGP version.
     }
}