Javalin: Add Kotlin Serialization JSON compatibility

[LooserName404]

The kotlinx.serialization module for JSON doesn't have easy support or tutorials in Javalin docs.

If you could, please add it for integration with this library.

[tipsy]

@LooserName404 you can follow the instructions for enabling GSON, but instead use kotlinx.serialization: https://javalin.io/documentation#configuring-the-json-mapper

[Freezystem]

I partially succeed to set-up kotlinx.serialization but it took me a lot of research and some help from SO
As a beginner in Kotlin it was very frustrating.
The procedure is not that easy and require setting a lot of @ExperimentalSerializationApi annotations everywhere my Javalin server was called making the code hard to read.
IMO this is more a problem with how the decode/encode functions are implemented in kotlinx.serialization that does not play well with Javalin json mapper interface. I'm opening a ticket into kotlinx.serialization repo for that matter and to know if it can be improve.

[tipsy]

Thanks @Freezystem! I've never used kotlinx.serialization, and it seems I wrongly assumed it would be plug and play (meaning that they would have similar signatures to the big Java libraries).

Could you add your solution to https://github.com/javalin/javalin.github.io/blob/master/pages/docs/docs.md#configuring-the-json-mapper so others can benefit from your research?

[Freezystem]

Hi @tipsy. Yes I will once I get an answer from kotlinx.serialization team cause the integration is a bit _hacky_ for the moment and rely on experimental features.
If it's good for you, I'll begin to write a draft PR this week to start the review process.

[tipsy]

Sounds good!

[Freezystem]

Hi @tipsy, according to this comment the Javalin interface for serialization is not optimal.
I was wondering if it was conceivable to improve this interface to be usable with both _class parameter_ and _reifed class_ ways.
Of course I'd be glad to hear from you on that subject and to help as much as I can to make it happen if it's a feature that you'll be willing to approve.

[rbygrave]

Hmmm, my 2c of thoughts. If I'm designing a rest API using JSON payloads I would never choose to have an endpoint with a POST/PUT request body that is a json array (as the top level of the request body). That is, I'm not expecting the use of generics in this use case. I'd always expect to POST/PUT a _json object_ as the top level even if it only contains a single property that is a json array.

The thinking there is that design wise we can't nicely extend/enhance the request payload (add a property) to handle future requirements when the top level is a json array so if I saw an API that POSTed a json array I'd think that was bad form.

So while the Javalin interface might not be as flexible (deemed optimal) I think in the situation it's used (JSON request payload) I personally don't think generics is an expected requirement.

This is just design opinion so I'm sure others don't agree on what I've said.

[Freezystem]

If I'm designing a rest API using JSON payloads I would never choose to have an endpoint with a POST/PUT request body that is a json array

@rbygrave I totally agree with that but It's opinionated. This implementation is not necessarily universal and enforcing such a thing is not the role of a library.

I'm just saying here that kotlinx.serialization has strong potential performance-wise (according to this benchmark and kmongo) with a growing community. It would be a good thing to make the boarding of new comers who want to use it easier.

[Freezystem]

@rbygrave I stumble upon something interesting yesterday when I was building my Javalin App that will, I think, advocate in favor of the necessity of a reifed interface for JsonMapper serialization.

Here is my use case:
I wanted to create a GenericResponse _data class_ dedicated to return serializable items of different types.
So I made these classes:

@Serializable
data class ResponseMeta(
    val count: Int? = null,
    val total: Int? = null,
    val page: Int? = null
)

@Serializable
data class GenericResponse<T>(
    val _meta: ResponseMeta,
    val _items: List<T> = emptyList()
)

When I use Kotlinx.serialization directly everything is fine:

return ctx.status(200)
            .result(kotlinx.encodeToString(res)) // "res" is a considered here as a GenericResponse<T>
            .contentType("application/json")

But when I try to use the JsonMapper interface with the _proper_ implementation for kotlinx by simply doing:

return ctx.status(200).json(res) // "res" is a considered here as a GenericResponse

I get this error: kotlinx.serialization.SerializationException: Serializer for class 'GenericResponse' is not found
And it's normal (I think) because the JsonMapper function erase information about GenericResponse real type when passing the object to the serializer function.

The only workaround would be to declare a new derived _data class_ for every GenericResponse subtype and that would be a bit tedious.

[rbygrave]

That is interesting and to me unexpected in this "serialise to json" direction. That is, Jackson would serialise that fine and it's the deserialisation direction (json to generic type) that would be the issue from a Jackson perspective. My thought is that Jackson is dynamically determining the type of the list elements and that kotlinx.serialization does not work in that manor.

Hmmm.

[tipsy]

@Freezystem do you have a suggestion for how this could coexist with the Java-oriented interfaces without causing confusion?

[rbygrave]

I _think_ in java this means having an extra/overloaded method that takes Type?

[Freezystem]

As a beginner in Kotlin (and Java) I have some troubles wrapping my head around it. I search for reified interface in Java but find nothing concluding.

Maybe it is possible to:

• create a second interface dedicated to koltin
• create a kotlin wrapping interface that is in charge to call java method or the other way around ?

These are just wild guesses for the moment but I'm still documenting.
I found this blog post which sound interesting.
I wonder if Java can use a kotlin reified interface flawlessly, I'll clone the Javalin repo and run some tests by myself.
Feel free to keep me updated if some ideas come to your mind.

[rbygrave]

In terms of writing a generic object to JSON with Gson or Jackson we see that they have options for using Class<?> and an additional option for using java.lang.reflect.Type (Gson) or for Jackson it's own JavaType or TypeReference<?>.

Gson:

java.lang.String toJson​(java.lang.Object src)
java.lang.String toJson​(java.lang.Object src, java.lang.reflect.Type typeOfSrc)

https://www.javadoc.io/static/com.google.code.gson/gson/2.8.6/com.google.gson/com/google/gson/Gson.html#toJson(java.lang.Object,java.lang.reflect.Type)

Jackson:

ObjectWriter    writerFor(Class<?> rootType)
ObjectWriter    writerFor(JavaType rootType)
ObjectWriter    writerFor(TypeReference<?> rootType)

http://fasterxml.github.io/jackson-databind/javadoc/2.10/com/fasterxml/jackson/databind/ObjectMapper.html#writerFor-java.lang.Class-

Conceptually in Java land for Gson it would suggest having an additional/overloaded toJson() that took java.lang.reflect.Type - but that wouldn't work for Jackson which has it's own JavaType and TypeReference (but for myself I don't _think_ we need this for Jackson and it would be worth checking Gson - as in, is this more a kotlin/kotlinx.serialization issue/requirement? I should confirm when Jackson and Gson _need_ generic type on serialisation to json).

Hmmm.

[tipsy]

If we introduce a type parameter, it has to be completely independent of any third party (like java.lang.reflect.Type). This is the first time this issue has come up in three years though, so I'm not convinced it's necessary.

Since this is mostly (exlusively?) a Kotlin problem, maybe an extension function is an acceptable solution @Freezystem @LooserName404 ?

[Freezystem]

As it's a Kotlin related problem I think that it's OK to build a workaround just for Kotlin and keep Java implementation unchanged to avoid any breaking changes.
Thus, an extension function seems to be an acceptable solution. :blush:

[tipsy]

@Freezystem any update on this? Do you need hep creating the extension function?

[Freezystem]

Hi @tipsy thanks for asking. I've tried to implement it correctly but didn't succeed. I guess I'm doing it wrong but I'm too novice in kotlin to understand what. I'd like to help and understand though so if you have any suggestions I'd be glad to exchange with you on this subject.

[tipsy]

Sorry @Freezystem, I completely missed your reply!

You can read a bit about extension functions here: https://kotlinlang.org/docs/reference/extensions.html. You would have to do something like:

fun Context.kotlinxJson() {

}

With your desired signatures.

Then you can import these when you want to use it, and call them like ctx.kotlinxJson().

[Freezystem]

Thanks @tipsy. I'll try it in the following days and keep you updated.