Fable: Breaking changes in JSON serialization introduced in 3.1.6

[JohnTheGr8]

First things first, thank you all for your work in this project.. it has been a delight using Fable for the last ~3 years!

Description

I caught a regression after upgrading a project from Fable 3.1.5. I use Fable alongside Thoth.Json (and Thoth.Fetch), and upgrading from 3.1.5 caused data serialized on the client-side to fail to deserialize on the server-side.

After some (admittedly not very thorough) digging, I have found that the behavior of Fable.Core.JS.JSON.stringify - when used with lists specifically - was changed in Fable 3.1.6 and probably in this commit. This change is not mentioned in the release notes as intended, so I assume it is not.

Repro code

As a quick example, the following code:

Fable.Core.JS.JSON.stringify [ "a"; "b"; "c" ]

when ran with Fable 3.1.5 gives:

["a","b","c"]

whereas with Fable 3.1.6 - 3.1.11 it gives:

{"head":"a","tail":{"head":"b","tail":{"head":"c","tail":{"head":null}}}}

Related information

• Fable 3.1.6 up to 3.1.11
• Fable.Core 3.2.5

(I am not familiar with this codebase.. hopefully the above information is enough to get an idea for what is going on)

[ncave]

@alfonsogarciacaro We're missing the toJSON attached method on the List object, but I'm not sure what's the best way to add that. Is there a way to force a single method to be attached? See also #1678.

[alfonsogarciacaro]

Thanks for report and the nice words @JohnTheGr8! Ah, damn, forgot about the "native" JSON serialization with toJSON. Yes, we need to add it back to list to avoid breaking changes and it's probably a good idea to add it to other fable-library types already coded in F# like BigInt, maps or sets. @ncave I think there's no way to explicitly mark a method as attached, but we can just declare an IJsonSerializable interface with a toJSON method and make list (and other types) implement it.

[alfonsogarciacaro]

I've added back toJSON to lists, and also bigints, maps and sets. I had some doubts particularly in the case of MutableMap and MutableSet because the native JS Map and Set (which are used for primitive keys) don't work with JSON.stringify but I finally did it for consistency with the types within fable-library. Hopefully this doesn't cause other problems.

[JohnTheGr8]

Thank you for the quick response and fix, looking forward to the new release.

[MangelMaxime]

I am a bit surprised that you add this kind of problem with Thoth.Json.

I would have expected Thoth.Json to not be affected by this problem.

@JohnTheGr8 Could you please share with us how you use Thoth.Json to encode list?

[JohnTheGr8]

I am a bit surprised that you add this kind of problem with Thoth.Json.

I would have expected Thoth.Json to not be affected by this problem.

@JohnTheGr8 Could you please share with us how you use Thoth.Json to encode list?

I use it with Thoth.Fetch like I mentioned, nothing special really... I see that Thoth.Json's Encode.toString uses JS.JSON.stringify which should explain it.

@MangelMaxime If you want me to expand on this any further, let me know and I'll be happy to.

[MangelMaxime]

Thoth.Json use it as the last moment, at that moment all the list should have already been converted into a JSON compatible type.

I am interested in see how you write the Encoder in the case of a list or to know if you use auto encoder.

[JohnTheGr8]

Okay, I see what you mean. I have done some more testing, removing any extra coders I had to try to boil it down. I am getting some strange behavior...

I have a helper module that looks like this:

[<RequireQualifiedAccess>]
module F =
    let inline post<'a> (url, data) : Fable.Core.JS.Promise<'a> =
        Thoth.Fetch.Fetch.post<_,'a> (url, data = data)

which I typically use as a shorthand with elmish like so:

Cmd.OfPromise.either F.post ("/endpoint", [ "a"; "b"; "c" ]) onSuccess onFailure

Using the above in Fable 3.1.6 sends the bad json payload I mentioned above. However, if I move this module to the same file as the page that calls it, it produces the expected json and everything works... 🤷‍♂️

I have tested the exact same situations (putting F in same/separate file) with Fable 3.1.12 and both cases work just fine.

Does this make any sense?

[alfonsogarciacaro]

Hmm, it's difficult to say without seeing the code but my guess is somehow the generic parameters were not inferred correctly and serialization was just defaulting to toJSON which was removed in 3.1.6 for lists and brought back in 3.1.12.

I have a similar inlined helper for different http methods (get, put, post, delete...). Can you please check what the IDE says and whether the generic parameters are filled with the expected types?

[JohnTheGr8]

@alfonsogarciacaro good call. I found some time to look into this some more:

When the F module (same as above) is in its own file, data is inferred to be of type obj. If I set it to be a generic type like 'Request. it works as expected with fable 3.1.6

When the module is in the same file, data is not treated as an obj. (I only called F.post from one place to test it, and so it picked data to be of type string list, which explains why it worked.) If I try to use F.post elsewhere I get a compilation error.

So, when having the module in the same file, the compiler is forcing me to add a generic type annotation. Do you have any idea if this is normal/expected?

@MangelMaxime tldr: thoth had a problem in fable 3.1.6-3.1.11 because the request object was treated as obj. Does it make sense now?

[alfonsogarciacaro]

Thanks for checking @JohnTheGr8! It's difficult to say without opening the full project in an F# IDE, but it's true type inference sometimes can yield unexpected results, particularly in inlined functions or recursive modules. The safest bet is always to use type annotations in the function signature, this is also a good practice for public utility methods because it makes the function better documented and avoid unexpected changes (when type inference changes for some reason).

[MangelMaxime]

@MangelMaxime tldr: thoth had a problem in fable 3.1.6-3.1.11 because the request object was treated as obj. Does it make sense now?

@JohnTheGr8 Yes think the cause was due to what Alfonso explained. And indeed, in the case of obj it's possible that we fallback to the default serialiser to support some specific case in the case of the Auto module..

[JohnTheGr8]

Thank you, guys!