Runtime: Consider Json support for reading numbers as octal (prefixed with 0)

[krwq]

Some Json parsers i.e. Newtonsoft.Json are capable of reading octal numbers (which are prefixed by 0). Our Json parser does not support it and in addition throws very cryptic error message:

System.Text.Json.JsonReaderException : '1' is an invalid end of a number. Expected a delimiter. LineNumber: 0 | BytePositionInLine: 10.

Repro:

            bool found = false;
            byte[] bytes = Encoding.UTF8.GetBytes("{ \"foo\": 01 }");
            Utf8JsonReader reader = new Utf8JsonReader(bytes, true, new JsonReaderState());
            while (reader.Read())
            {
                Console.WriteLine(reader.TokenType);
                switch (reader.TokenType)
                {
                    case JsonTokenType.Number:
                    {
                        Assert.False(found);
                        found = true;
                        Assert.True(reader.TryGetInt32(out int val));
                        Assert.Equal(1, val);
                        break;
                    }
                }
            }

            Assert.True(found);

isolated issue from: https://github.com/dotnet/corefx/issues/37727

[Tornhoof]

I would be very careful with supporting octal, this is quite outside of the JSON specification, which does not allow leading zeros for numbers. This should at least be hidden behind some config flag, like extra commas.

[GSPP]

Is there any use case for this? What system produces JSON with octal numbers?

[krwq]

@GSPP unix permissions are in octal, for machine generated JSONs it doesn't matter what the format is

[ahsonkhan]

Based on previous discussions, we do not plan to support octal numbers. The reader tries to uphold following the JSON spec very closely by default with only a few high-priority deviation (under config).

Our Json parser does not support it and in addition throws very cryptic error message:

This is still something we should fix. Marking as up-for-grabs if anyone's interested.

[MarcoRossignoli]

I'm taking a look at this...I don't understand why -01 is not a valid json.
Number 01 is invalid for leading 0.
RFC:

The representation of numbers is similar to that used in most
   programming languages.  A number is represented in base 10 using
   decimal digits.  It contains an integer component that may be
   prefixed with an optional minus sign, which may be followed by a
   fraction part and/or an exponent part.  Leading zeros are not
   allowed.

Is it implicit that zero leading does not depend on sign?So also -01 is an invalid leading?

[ahsonkhan]

I don't understand why -01 is not a valid json.

If you look at the state machine (from json.org), you will see -01 is invalid:

From RFC - https://tools.ietf.org/html/rfc8259#section-6

number = [ minus ] int [ frac ] [ exp ]
int = zero / ( digit1-9 *DIGIT )

As to the rationale behind why it was done this way within the JSON spec itself, I am not sure. Maybe to avoid potential ambiguous states.

[MarcoRossignoli]

@ahsonkhan this is labeled future, does it make sense work on it or better pick-up 3.0 milestone?

[danmosemsft]

@marcorossignoli our goal is to zero out the 3.0 milestone by 1st July. So if it is possible to work on 3.0 issues somewhere in the repo that would help us the most!

[ahsonkhan]

this is labeled future, does it make sense work on it or better pick-up 3.0 milestone?

The main thing that needs to be fixed here is the exception message if we were to encounter a zero-leading number in the payload so that it is more meaningful. Since this is primarily just an exception message change, it is relatively low priority (and non-breaking), which is why its marked as future.

[MarcoRossignoli]

I'll pickup from 3.0 for now thanks