Kotlinx.serialization: Serializing generics is not working

Created on 13 Dec 2017 · 15Comments · Source: Kotlin/kotlinx.serialization

Serializing the following Container<Data> class fails with:

java.lang.NoSuchFieldException: INSTANCE

@Serializable
class Container<T>(val data: T)

@Serializable
class Data(val text: String)

question

Source

czeidler

Most helpful comment

If you have non-generic class (say, MyData(val a: Container<Data>), you can use simple MyData.serializer(). However, if your class has type arguments, you have to provide serializers explicitly for each generic type.

In some observable future, we are going to invent mechanism such compiler could also insert serializers on call-site, not only in internal serialization calls. This should make things easier.

sandwwraith on 21 Dec 2017

👍3 👎1

All 15 comments

Error seems like you're trying to serialize container by simple JSON.stringify(Container(Data(..))). Because of generic type erasure, this wouldn't work: On runtime, framework can't determine how to serialize data in container. Thus Container class does not have default (object) serializer, and thus there is no field INSTANCE.

Right way to do it is to create serializer explicitly: JSON.stringify(Container.serializer(Data.serializer()), Container(Data(...)))

You can learn more here

sandwwraith on 13 Dec 2017

👍1

Ok thanks, I tried it (see below) but it still fails with the same error. What am I doing wrong? I'm using version 0.3.

Btw. IDEA is not recognising Container.serializer but only Container::class.serializer. However, if I use the later I can't do Container::class.serializer(Data::class.serializer()) Why is that?

@Serializable
class Container<T>(val data: T)

@Serializable
class Data(val text: String)

@Test
fun testGenerics() {
    val json = JSON.stringify(Container.serializer(Data.serializer()), Container(Data("test")))
    val parsed = JSON.parse<Container<Data>>(json)
    assertEquals("test", parsed.data.text)
}

czeidler on 13 Dec 2017

JSON.parse also requires explicit serializer in your case - generic arguments are not reified, even if the function reified itself :( So in your case you should end with something like

val boxedDataSerial = Container.serializer(Data.serializer())
val json = JSON.stringify(boxedDataSerial, Container(Data("test")))
val parsed = JSON.parse(boxedDataSerial, json)
assertEquals("test", parsed.data.text) // ok

Yes, for now IDEA experience is slightly poor because it don't recognize some synthetic functions. You can improve it by following this instructions: https://github.com/Kotlin/kotlinx.serialization#working-in-intellij-idea

sandwwraith on 14 Dec 2017

Ok thanks this works now!

I don't understand the internals but it still looks a bit cumbersome, i.e. since I can't do T.serializer() I have to provide the correct serializer for each concrete type. Or is the there a generic way to do it?

czeidler on 14 Dec 2017

In some observable future, we are going to invent mechanism such compiler could also insert serializers on call-site, not only in internal serialization calls. This should make things easier.

sandwwraith on 21 Dec 2017

👍3 👎1

Are the serializer classes threadsafe?

coder82 on 7 Jun 2018

@coder82 They must be stateless and immutable. How this question is related to the issue?

sandwwraith on 7 Jun 2018

😄1

Hi,
I also concerned about thread safety of serializers? Especially for
PolymorphicSerializer. We are getting strange data corruptions during
serialization when parallel threads do serialization simultaniously. And
always getting corrupted json just after generic field where
PolymorphicSerializer kicks in. Anyone have similiar issues?

On Thu, 7 Jun 2018 at 23:02, Leonid Startsev notifications@github.com
wrote:

@coder82 https://github.com/coder82 They must be stateless. How this
question is related to the issue?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Kotlin/kotlinx.serialization/issues/53#issuecomment-395546736,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEAcnAxNTVF6JKrRSuVgQvvIY92lNyuyks5t6YbCgaJpZM4RABC4
.

antanas-arvasevicius on 7 Jun 2018

The JSON serializer is stateful. You cannot really reuse it and it doesn't have synchronization to coordinate access (writing from different threads). You want to create one at least per thread, but probably per operation. Also remember that serialization is not atomic as outputstreams are not (if they are threadsafe to begin with) but also due to the fact that the json serializer writes in fragments smaller than a full element.

pdvrieze on 8 Jun 2018

Problem is that "PolymorphicSerializer" is singleton object... and I cannot have multi instances of it.
Now It's possible to create new serializers for each serialization action but impossible for generics (i.e. EventLog.serializer(PolymorphicSerializer)).
Or we didn't understand kotlinx.serialization design. Shouldn't (Is?) KSerializer be thread-safe and immutable?

antanas-arvasevicius on 8 Jun 2018

PolymorphicSerializer has no state so is threadsafe, but the generated serializers for classes have no specific thread support. That is, if you change the object while it is being serialized this will likely lead to incorrect values. Custom serializer code can of course add locks where appropriate or be externally synchronized.

Remember there are two kinds of "serializers" there is the code specific to types (which includes PolymorphicSerializer) and the output/input such as JSON. The JSON class has state and no synchronization so isn't threadsafe, the serializers providing the information to json themselves have the limited threadsafety I described above..

pdvrieze on 8 Jun 2018

👍1

This is the type of garbage returning.
I am using:

ret = JSON.nonstrict.stringify(DeSerializers.klubschuleDeSerializer, KlubschuleCalendarDataRandomizer.create())

See at the end, it's not valid Json, a bunch of appointments have not been serialzied.

{"AppointmentList":[{"center_id":0,"kurs_bezeichnung":"Deutsch Semi-Intensiv Niveau B1 (3/3)","raum_bezeichnung_kurz":"Desk","kursleiter_vorname":"Sepp","kursleiter_name":"Keller","event_datum":"2018-06-09","event_beginn":"08:25:00","event_ende":"09:00:00"},{"center_id":0,"kurs_bezeichnung":"Hebräisch Niveau B1 (Kleingruppe)","raum_bezeichnung_kurz":"Bern","kursleiter_vorname":"Peter","kursleiter_name":"Meier","event_datum":"2018-06-09","event_beginn":"09:40:00","event_ende":"12:40:00"},{"center_id":1016,"kurs_bezeichnung":"Deutsch Intensiv Niveau A2 (2/2)","raum_bezeichnung_kurz":"Bern","kursleiter_vorname":"Fritz","kursleiter_name":"Keller","event_datum":"2018-06-11","event_beginn":"14:25:00","event_ende":"18:25:00"},{"center_id":1017,"kurs_bezeichnung":"Deutsch Semi-Intensiv Niveau B1 (3/3)","raum_bezeichnung_kurz":"Zurich","kursleiter_vorname":"Peter","kursleiter_name":"Müller","event_datum":"2018-06-09","event_beginn":"19:00:00","event_ende":"21:25:00"},{"center_id":0,"kurs_bezeichnung":"Jedi arts for experts.","raum_bezeichnung_kurz":"Düsseldorf","kursleiter_vorname":"Sepp","kursleiter_name":"Müller","event_datum":"2018-06-10","event_beginn":"09:00:00","event_ende":"12:25:00"},{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{,{]}

coder82 on 8 Jun 2018

@antanas-arvasevicius @coder82 see #139, there is a discussion about thread-safety in json. Try to replace JSON.nonstrict with JSON(nonstrict = true) and check if it works correctly.

sandwwraith on 8 Jun 2018

hi Sand, it worked... I saw you committed a fix on master, may I use that directly?

coder82 on 8 Jun 2018

If you have non-generic class (say, MyData(val a: Container<Data>), you can use simple MyData.serializer(). However, if your class has type arguments, you have to provide serializers explicitly for each generic type.

In some observable future, we are going to invent mechanism such compiler could also insert serializers on call-site, not only in internal serialization calls. This should make things easier.

Can you give an example, please?