Runtime: System.Text.Json forces double quote on name

[unruledboy]

The following json works fine with Newtonsoft.Json, but does not work with System.Text.Json

{
  name: "value"
}

It says:

System.Text.Json.JsonException: ''m' is an invalid start of a property name. Expected a '"'. Path: $ | LineNumber: 1 | BytePositionInLine: 0.'

I tried to find if there is anything in JsonSerializerOptionsthat can help me to support this, but I could not find one.

I know System.Text.Json has better performance that's why I switched to it. I don't want to switch back if there is solution.

There are cases that we have no control over the source json to be in double-quoted name format.

[ExplodingCabbage]

If the keys are not quoted, it's not valid JSON; it's a JavaScript object.

Is a parser for JavaScript object literals a valid and useful thing to have? Perhaps. But it seems like such a thing is a separate tool to a JSON parser. And there are plenty more things you might want to add in that case, like support for JavaScript regex literals, or single-quoted strings, or Infinity and NaN. Doesn't make much sense to me to add support for just this one specific thing that isn't JSON.

[caihongxu]

I think this is not about supporting JS objects. It is more about robustness as well as feature parity with Json.NET for its users who want to migrate their code over. Like the author indicated, the source of the json might be from some web api that did not follow JSON standard well which we don't have control, but Json.NET had been robust enough to be able to handle this.

Like the author, I will have to stick to Json.NET because of this. And more broadly, any Json.NET user should be really careful migrating their code over because it is a breaking change (and this difference should probably be explicitly called out somewhere in System.Text.Json's documentation).

[poke]

It is more about robustness as well as feature parity with Json.NET

Feature parity with Newtonsoft.Json was never a goal for the new library.

I will have to stick to Json.NET because of this

And that is totally fine. That's why you have a choice there.

[SimonCropp]

@unruledboy what are you using json for? config files, payload serialization (eg web request body or messages), or something else?

[SimonCropp]

So there are several scenarios here, including:

• moving serlializers (newtonsoft to text.json)
• moving formats (xml to json)
• evolving contracts eg renaming properties

All of these are legitimate (and often common) scenarios in the lifetime of an app. An app should be build in such a way that these can be handled in a graceful way. Often this is facilitated via features in the selected frameworks/libraries.

On specific example is with NServiceBus when evolving contracts/serializers on message contracts that are always on the wire. https://docs.particular.net/samples/serializers/transitioning-formats/. Essentially all payloads have a header what identifies the serialization format (an possibly version). Then all nodes of the app keep the config/setup for those historical formats for a period of time. New messages are always send with latest config/settings. but older messages can still be deserialized.

The same approach can be used in a web client/server situation.

So while "forces double quote on name" is a breaking change from the perspective of people transitioning, it should be possible to handle that change using the same approach that is (or should be) in place to handle other changes in the serialization contract.

[caihongxu]

In my case, I just need to call a number of public web APIs whose response format is json, but without quots on property names. Should I ask the authors to change their response format to proper json? Maybe. Should I block myself on that change? Probably not. Therefore I will just stick to NewtonSoft unless System.Text.Json support unquoted names as well.

[unruledboy]

@SimonCropp

There are two scenarios:

1. The customer consumes our RESTful API by posting data in JSON, and the JSON is in the format of without names being double-quoted. I believe we can let the customer to provide double-quoted names, or have an API versioning by introducing a new version which forces double quotes, and that can use System.Text.Json.
2. We are processing customer provided json data, that we do not have any control over. I believe, if there is no work around, we will have to stick to Newtonsoft.JSON.

But, I would like to see if there is any solution, like custom format validator etc., which we can get the best of both worlds, the performance in System.Text.Json, and the flexibility of what we have in Newtonsoft.JSON.

[ExplodingCabbage]

@caihongxu

public web APIs whose response format is json, but without quots on property names

Can you point us to these APIs? Do they really claim that what they're emitting is JSON? I'm perplexed by how such a thing can even come to exist. Every language in popular use for backend web development has had a well-established JSON library for well over a decade; in most cases (PHP, Python, Ruby, JavaScript) it's long been built into the standard library of the language. It takes significantly more work to write an API that emits broken not-really-JSON than it does to write one that emits valid JSON. And it's not like it's an easy mistake to miss if you make it, because nobody who isn't using either the Newtonsoft library or JavaScript's eval will be able to easily parse it!

Therefore I will just stick to NewtonSoft

This seems like the right thing to do.

[safern]

cc: @ahsonkhan

[ahsonkhan]

The built-in JSON stack only processes and accepts valid JSON by default (according to the JSON RFC, which is called out in the underlying Utf8JsonReader's documentation which is the workhorse for JSON parsing while deserializing - https://docs.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonreader?view=netcore-3.0#remarks). There are only a couple of highly requested deviations that we support when the user explicitly opts-in which includes ignoring comments and trailing commas.

Therefore, we don't support unquoted property names/strings, or any other custom format that Newtonsoft.Json supports (such as using single quotes/etc.), and we definitely wouldn't do so by default (flexible parsers have pef implications and there is a correctness concern letting invalid JSON parse successfully). As mentioned previously on the thread, feature parity and 100% compatibility was never the goal, and you should continue to use Newtonsoft.Json for scenarios/use cases that are not provided by the built-in library (particularly if you don't have any impetus to move such as taking advantage of performance/avoid dependencies that would make it worthwhile). The built-in library is biased towards strict behavior, performance, and features that are heavily used/requested across workloads which don't affect overal usability/perf and hence intentionally limits all possible customizations. That said, if a feature request meets those constraint and is blocking commonly occurring scenarios, we'd certainly consider adding it. I don't think this feature meets the bar.

Please do provide an example web API that is returning JSON with properties not contained within quoted strings.

[SimonCropp]

my preference would be to close this issue as "no supported"

[ahsonkhan]

Similar concern from @lorranpalmeira in https://github.com/dotnet/corefx/issues/42819 (but for single quotes):

´´´
var objectExpeted = @"{
'cep': '01001-000',
'logradouro': 'Praça da Sé',
'complemento': 'lado ímpar',
'bairro': 'Sé',
'localidade': 'São Paulo',
'uf': 'SP',
'unidade': '',
'ibge': '3550308',
'gia': '1004'
}";

//var expected = JsonSerializer.Deserialize(objectExpeted);
var expected = JsonConvert.DeserializeObject(objectExpeted);
´´´

It has an exception when I try Deserialize in JsonSerializer.In NewtonSoft works perfectly.

[SimonCropp]

can we get a ruling on this one? if the decision is "not supported" at least people can base their own decisions on that and move forward. I just dont want this one sitting open as "we might one day support it" and people deciding "well we wont fix our double quote usage, instead we will wait till System.Text.Json supports it".

if you do want to support it can the status be changed to "will accept a PR"

[ahsonkhan]

Thanks for the reminder, @SimonCropp.

Given there isn't strong evidence that suggests users need this flexibility in processing JSON and given this request goes against the JSON spec (see previous comment https://github.com/dotnet/corefx/issues/41421#issuecomment-536844882), closing this issue as won't fix.

To me, introducing this flexibility to the JSON reader requires more user blocking scenarios to justify the code complexity, performance, and "correctness" trade off that affects the vast majority of usages.

[am11]

Property names without quotes is valid in CSON, which should be kept separate from JSON, IMO.

There are various other JSON-like syntax in existence, e.g. HCL supports blocks, MongoDB document supports custom keywords like ObjectId(), NumberLong() etc.; therefore if library allows some non-standard language syntax, then it should be a more overarching extensibility point. Likely not a good candidate for this library System.Text.Json - a high-performance, low-allocating, and standards-compliant capabilities to process JavaScript Object Notation (JSON).

[urzur]

I am hitting the same issue when trying to make a POST via RestSharp using
request.AddParameters({name}, {value})

Does anyone know within RestSharp how to change the request such that I am able to successfully get a response?