Kotlinx.serialization: @SerialId does not affect enum members

[jamesachn]

There appears to be an incorrect enum index/value check during protobuf enum decoding (kotlin: 1.3.20, kotlinx.serialization: 0.10.0)

For example using the following .proto definition to generate the incoming serialized message:

syntax = "proto3";
package example;

option java_package = "example.java";
option java_multiple_files = true;

enum EnumA {
    reserved 2 to 9;
    A_ZERO = 0;
    A_ONE = 1;
    A_TEN = 10;
}

enum EnumB {
    B_ZERO = 0;
    B_ONE = 1;
    B_TWO = 2;
}

message ExampleEnumMessage {
    EnumA a = 1;
    EnumB b = 2;
}

Deserializing to the following data class:

enum class EnumA {
    @SerialId(0) A_ZERO,
    @SerialId(1) A_ONE,
    @SerialId(10) A_TEN;

    companion object {
        val defaultValue: EnumA = EnumA.values()[0]
    }
}

enum class EnumB {
    @SerialId(0) B_ZERO,
    @SerialId(1) B_ONE,
    @SerialId(2) B_TWO;

    companion object {
        val defaultValue: EnumB = EnumB.values()[0]
    }
}

@Serializable
data class ExampleEnumMessage(
    @SerialId(1) @Optional val a: EnumA = EnumA.defaultValue,
    @SerialId(2) @Optional val b: EnumB = EnumB.defaultValue) {
}

The following example uses protobuf-java runtime to generate the encoded data to be decoded by kotlinx.serialization:

fun example() {
    val parser = ExampleEnumMessageJava.parser()
    val serializer = ExampleEnumMessage.serializer()

    val message = ExampleEnumMessageJava.newBuilder().setA(EnumAJava.A_TEN).build()
    // Replacing previous line with the following will not have an error.
    // val message = ExampleEnumMessageJava.newBuilder().setB(EnumBJava.B_TWO).build()
    val encodedWithJava = message.toByteArray()
    val decodedWithKotlin = ProtoBuf.load(serializer, encodedWithJava)
}

will fail with:

java.lang.IllegalStateException: 10 is not among valid example.EnumA choices, choices size is 3

    at kotlinx.serialization.internal.CommonEnumSerializer.deserialize(Enums.kt:46)
    at kotlinx.serialization.protobuf.ProtoBuf$ProtobufReader.decodeSerializableValue(ProtoBuf.kt:204)
    at kotlinx.serialization.TaggedDecoder$decodeSerializableElement$1.invoke(Tagged.kt:254)
    at kotlinx.serialization.TaggedDecoder.tagBlock(Tagged.kt:267)
    at kotlinx.serialization.TaggedDecoder.decodeSerializableElement(Tagged.kt:254)

It looks like the decoded value is directly used as the index value into an enum-value choice array, which is causing the range-check condition to fail.

What is the correct way to pass in enum constant information such that it matches the analogous protobuf IDL declaration?

Edited:

• Removed red-herring information on a application-supplied 'value' property in the enum class definitions. The example now attempts to tag each enum value with a @SerialId (though probably incorrectly used).

[sandwwraith]

This is the current enums limitation: no serial annotation has any effect on them. We are going to fix this at some point in the future.

[rickygao]

Looking forward it. Actually, values in enums should be easier to be customized any way you like, instead of just in sequence.

I noticed that Int may replace the enum type, however, to make a workaround.

[sandwwraith]

Fixed with https://github.com/Kotlin/kotlinx.serialization/pull/572 . See https://github.com/Kotlin/kotlinx.serialization/issues/31#issuecomment-559571106 for detailed explanation

[FerhatBahceci]

Hey guys, is it possible to retrieve the Enum int from the ProtobufDecoder somehow when using the kotlinx.serialization.protobuf.ProtoBuf.decodeFromByteArray and implementing a custom kotlin serializer?
I've been trying to use decodeEnum(descriptor) and some other stuff but with no luck. So my question is, if it is even possible to retrieve the enum int from the ProtobufDecoder? Thanks

[sandwwraith]

@FerhatBahceci You can use decodeInt and then SerialDescriptor.getElementName with your enum descriptor

[FerhatBahceci]

Protobuf spec:

enum DayOfWeek {
    MONDAY = 0;
    TUESDAY = 1;
    WEDNESDAY = 2;
    THURSDAY = 3;
    FRIDAY = 4;
    SATURDAY = 5;
    SUNDAY = 6;
}

Kotlin enum:

 @Serializable
    enum class DayOfWeek {
        MONDAY,
        TUESDAY,
        WEDNESDAY,
        THURSDAY,
        FRIDAY,
        SATURDAY,
        SUNDAY;

        @ExperimentalSerializationApi
        @Serializer(forClass = DayOfWeek::class)
        object DayOfWeekSerializer : DeserializationStrategy<DayOfWeek> {

            override fun deserialize(decoder: Decoder): DayOfWeek {

                        val enumValue = decoder.decodeInt()
                return valueOf(descriptor.getElementName(enumValue))
            }
        }
    }

1. ProtoBuf.decodeFromByteArray(request.toByteArray()).
   // CreateStoreReq contains a Store which has a DayOfWeek instance field. I don't think this toByteArray() is actually valid for the proto generated classes.

2.       override fun deserialize(decoder: Decoder): DayOfWeek {


                val enumValue = decoder.decodeInt()
                return valueOf(descriptor.getElementName(enumValue))

//Here, the DayOfWeek proto are arriving as a ByteArray. I assume that it works in the same way as it does with Java, java.io.ObjectInputStream. readLine() would read one line and return a String from the stream and it could be invoked to the EOF.

This actually works back and forth:

ProtoBuf.decodeFromByteArray<DayOfWeek>(ProtoBuf.encodeToByteArray(proto.store.service.DayOfWeek.forNumber(2)))
but we don't have Serializers for the proto classes, we cannot encodeToByteArray and Im sticking with the toByteArray instead.

decoder.decodeInt() should return the int of that proto.DayOfWeek but it does not. This is the behaviour that I'm having for both decodeEnum(descriptor) as well as decodeInt()

         1. decoder.decodeInt() ---> Expected wire type 0, but found 2
         2. decoder.decodeInt() ---> 2
         3. decoder.decodeInt() ---> 32
         4. decoder.decodeInt() ---> 5 // This one is correct!
         5. decoder.decodeInt() ---> throwing EOF


         1. decoder.decodeEnum(descriptor) ---> Expected wire type 0, but found 2
         2. decoder.decodeEnum(descriptor) ---> 2
         3. decoder.decodeEnum(descriptor) ---> -1
         4. decoder.decodeEnum(descriptor) ---> 5  // This one is correct!
         5. decoder.decodeInt() ---> throwing EOF

And if Im trying to use the:

decoder.beginStructure(descriptor) ---> throwing Primitives are not supported at top-level

I think the issue here is that toByteArray() is not actually returning the same as ProtoBuf.encodeToByteArray and the entire ProtobufReader is populated with wrong structure

Since ProtoMessages are not implementing Kotlin Serializable we cannot do the following:

ProtoBuf.encodeToByteArray(ANY_PROTO_GENERATED_CLASS)

[FerhatBahceci]

Never mind. Got it working with protos toByteArray now. Thank you!