Azure-cosmos-dotnet-v3: Eliminate Newtonsoft.Json

[fuocor]

Your dependency on Newtonsoft.Json and use of Stream goes counter to everyone's use of System.Text.Json and IBufferWriter<>

Describe the solution you'd like
Now that System.Text.Json is netstandard2.0 you should get rid of Newtonsoft.Json

Additional context
it creates dangling references that I do not want to carry around

The core cosmos API should limit itself to IBufferWriter<> and Span<T> and expose typed serialization as extensions

[kirankumarkolli]

System.Test.Json is not yet released. We will evaluate once its ready.
CosmosJsonSerializer implementation can be provided to CosmosClient to use even today to leverage the power of those libraries.

[fuocor]

It cannot be used implicitly.

There is no public setter for the default serializer, and not all the methods have stream equivalents. See CosmosStoredProcedure.ExecuteAsync

IMHO - the core Cosmos API should de devoid of object serialization. Any and all object serialization should be implemented in abstractions or extensions. I think it will make the whole thing lighter and more flexible.

[j82w]

You can set the CosmosJsonSerializer via the CosmosClientBuilder.

CosmosClient cosmosClient = new CosmosClientBuilder("http://localhost", "SuperSecretKey").UseCustomJsonSerializer(new CustomJsonSerializer).Build()

[DanielLarsenNZ]

When using the Cosmos SDK in ASP.NET Core 3.0 and .NET Core 3.0 today, we have to use Newtonsoft.Json, for its JsonPropertyAttribute to rename Id to id. Using System.Text.Json.Serialization attributes does not work. For example:

[Newtonsoft.Json.JsonProperty(PropertyName = "id")]
public string Id { get; set; }

IMO we should be able to use the native System.Text.Json.Serialization.JsonPropertyNameAttribute to rename properties for serialization. Same goes for Ignore, etc.

[Davy-F]

This still seems an issue in 3.5.1.

When using System.Text.Json, as so:

[JsonPropertyName("id")]
public string Id { get; set; }

I get the following error from CosmosDB:

{"Errors":["The input content is invalid because the required properties - id; - are missing"]}

[j82w]

System.Text.Json will not be the default in the v3 SDK. That is a breaking change. You can try out the v4 preview SDK which is in the process of removing all newtonsoft dependencies.

In v3 if you want System.Text.Json support you can add it by setting a custom serializer.

[tmakin]

What is the long term plan for the Newtonsoft dependency? I can see it was removed in v4 and then put back in via https://github.com/Azure/azure-cosmos-dotnet-v3/pull/1144. This seems like a bit of a backward step given that a tidy dependency tree is likely to be a key motivation to upgrade.

[j82w]

For v4 it's following the Azure Core SDK guidelines which removes all newtonsoft dependencies. @ealsur please correct me if I'm wrong.

[ealsur]

Correct, the idea is for it to be completely removed.

Currently V4 still has it due to some internal processes still depending on it, but it's not used while interacting with user data.

[Camios]

We have swagger generated c# classes that use Newtonsoft json attributes like [JsonProperty(PropertyName = "id")] mainly and converters like [JsonConverter(typeof(StringEnumConverter))]

The serializer in System.Text doesn't handle nullable enums. And appeared to have issues with StringEnum conversion (seemed to generate a string representation that was off-by-one enum, but I need to look more into this one).

Anyone know when System.Text serializer will work as smoothly as Newtonsoft wrt (nullable) StringEnums? If that is resolved, then we could consider moving to System.Text serializers, but we still have reliance on generated classes that use newtonsoft attributes, so we'd have to transform all the associated json attributes too.

To avoid the above concerns, is there consideration for how easy will it be for us to have Cosmos continue to use Newtonsoft serializer in Cosmos SDK v4?

Will the 'legacy' Newtonsoft-based serializer be available as a CosmosSerializer in SDK v4 or as an external package? It sounds like it won't, because it is to be eliminated, correct? So going from SDK v3 to v4 would really be a breaking change. Will we need to implement a CosmosSerializer which uses Newtonsoft under the covers (copy and rename the one from SDK v3)?

[j82w]

In Cosmos SDK v4 there will be a client option that will let you set a custom Cosmos Serializer. This will allow you to set Newtonsoft as the serializer. The benefit of this is you can pass in any version of Newtonsoft, and there won't be any version conflicts.

[AndriySvyryd]

Even if the main Azure.Cosmos v4 package uses Newtonsoft at runtime dynamically it shouldn't have a package dependency on it as that brings it in even when not used, see https://github.com/Azure/azure-cosmos-dotnet-v3/issues/37

[ealsur]

The goal is for V4 not to have any Newtonsoft.Json dependency.