Kotlinx.serialization: Composition of JSON.stringify() and DynamicObjectParser.parse() is not idempotent when there are properties of type Long

[fatjoem]

It would be nice if we could serialize an object with JSON.stringify() and then deserialize the result with DynamicObjectParser.parse(), and get back the original result. Right now this is not always the case. When properties of type Long are involved, this will yield a cast exception.

Example

@Serializable
class FooModel(val id: Long)

JSON.stringify(FooModel.serializer(), FooModel(1)) // yields { "id": 1 }

DynamicObjectParser().parse(js("""{ "id": 1 }"""), FooModel.serializer()) // yields a cast exception

This does not work. The parse() function throws a cast exception.

When the type of the "id" property is changed to String, it will work fine.

[anton-bannykh]

Here is a little discussion me and @sandwwraith had recently, which is somewhat related to this topic: how should JSON.stringify and DynamicObjectParser behave in case of large long values?

Take for example 2^60 - 1. At the moment it will be serialized to { "id": 1152921504606846975 }, which is correct. Problem is this number is converted to a 64 bit floating point number as soon as it is evaluated in JavaScript. So js(""" { "id": 1152921504606846975 }""") yields {id: 1152921504606847000}. That means there is no way to recover the original long value in this particular scenario with this particular serialization scheme.

That leads to a number of questions:

• What should DynamicObjectParser do in this case?
  
  
  
  1. Just return the closest Long value?
     
     
     
     
     
     • (+) Just works
     
     
     
     • (-) User might be getting incorrect values and have no clue why...
     
     
     
  
  
  2. Throw an exception telling "Value too large to be deserialized into Long without precision loss"?
     
     
     
     
     
     • (+) Fail fast - able to detect a problem earlier on
     
     
     
     • (-) Runtime checks might be missed during testing => some might experience a nasty surprise
     
     
     
  
  
• Is current serialization scheme even correct? Why not serialize Long values as string for example?
  
  
  
  • Breaks compatibility with other platforms
  
  
• Could we prevent this by warnings/inspections/sensible defaults?

AFAIK nothing is set in stone yet and it's a perfect time to chime in your opinion. In particular we are in dire need for the use cases. When do you need to deserialize Long's?

So far I am aware of two cases:

• Timestamps in milliseconds. Any sensible date fits into 64bit fp number without a problem
• Hashes/ID's. Getting a "somewhat close" number is unnacceptable

[fatjoem]

These are the use cases in which we have long properties:

• timestamp in milliseconds
• IDs
• price values that are stored in cents (or even smaller units) and can get very big

Our current workaround is using String properties. The only disadvantage that we face with this workaround is the added code complexity when we want to use them as Longs for better type safety.

An optional annotation that allows Long properties to be serialized as Strings would perfectly fit our use case.

[fatjoem]

A good first step would be to throw a more telling error instead of the generic cast error that is currently thrown. I wasted much time figuring out why the cast exception was thrown.

[sandwwraith]

After an internal discussion, we came up to the following solution:

• Deserialize JS Number to kotlin.Long only if its absolute value does not exceed Number.MAX_SAFE_INTEGER, otherwise throw an error. This will help avoid a precision loss for prices and ids, and will not affect timestamps.
• For all other cases, provide LongAsStringSerializer to be used with @Serializable(with=...) annotation.

I think this design can cover all the cases. If you have some considerations, feel free to share them.