Runtime: JsonSerializer support for TimeSpan in 3.0?

[martincostello]

_Apologies if this is already answered somewhere else, but if it was my search abilities failed me._

I'm trying out the preview 6 of 3.0 with some existing production application code, and one of the issues that's come out of testing is that some of our existing objects we use in our API contracts use properties which are TimeSpan values, which are then represented as strings.

Is support for TimeSpan properties planned for 3.0 for the new System.Text.Json APIs?

If it's not going to be that gives us notice to do some refactoring ahead of September to change them to strings so we can use the new serializer, where as if it's planned but just not implemented yet then we just need to wait for a later preview to get this working.

Below is a minimal repro unit test that demonstrates the failure for TimeSpan to be handled compared to our existing JSON .NET code.

using System;
using System.Text.Json.Serialization;
using Xunit;
using JsonConvert = Newtonsoft.Json.JsonConvert;
using JsonSerializer = System.Text.Json.Serialization.JsonSerializer;

namespace JsonSerializerTimeSpanNotSupportedException
{
    public static class Repro
    {
        [Fact]
        public static void Can_Deserialize_Object_With_SystemTextJson()
        {
            // Arrange
            string json = "{\"child\":{\"value\":\"00:10:00\"}}";

            // Act (fails in preview 6, throws: System.Text.Json.JsonException : The JSON value could not be converted to System.TimeSpan. Path: $.child.value | LineNumber: 0 | BytePositionInLine: 28.)
            var actual = JsonSerializer.Parse<Parent>(json, new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase });

            // Assert
            Assert.NotNull(actual);
            Assert.NotNull(actual.Child);
            Assert.Equal(TimeSpan.FromMinutes(10), actual.Child.Value);
        }

        [Fact]
        public static void Can_Deserialize_Object_With_NewtonsoftJson()
        {
            // Arrange
            string json = "{\"child\":{\"value\":\"00:10:00\"}}";

            var actual = JsonConvert.DeserializeObject<Parent>(json);

            // Assert
            Assert.NotNull(actual);
            Assert.NotNull(actual.Child);
            Assert.Equal(TimeSpan.FromMinutes(10), actual.Child.Value);
        }

        private sealed class Parent
        {
            public Child Child { get; set; }
        }

        private sealed class Child
        {
            public TimeSpan Value { get; set; }
        }
    }
}

[ahsonkhan]

We currently do not have plans to support TimeSpan and would be added in the future but I can do some investigation to see how much work is involved. Alternatively, you could create your own JsonConverter to support TimeSpan as a workaround. I'll give an update by end of next week. Thanks.

If we were to add it, we would also want to add TimeSpan APIs to the reader/writer/JsonElement, which would have to go through api review.

[martincostello]

Thanks - a custom converter would also be an easy enough way to make it work for our app for 3.0. Are those capabilities planned to ship with preview 7?

[steveharter]

Yes the custom converter support is now in master and thus will be in preview 7.

However, since TimeSpan is a common BCL type we should still provide a default converter.

[terrajobst]

We reviewed this today:

• There is an argument in favor of it, because it's a value type. In principle we could optimize parsing to be allocation free (i.e. avoid going through a string).
• It's unclear whether there is an standard encoding for time spans.
• For now, close. This can be reopened if there is enough customer evidence that this needed.

[khellang]

• It's unclear whether there is an standard encoding for time spans.

The standard encoding for time spans would be ISO8601's "duration". This is also what XML use to represent TimeSpan and is already implemented in XmlConvert and XsdDuration:

https://github.com/dotnet/corefx/blob/6d723b8e5ae3129c0a94252292322fc19673478f/src/System.Private.Xml/src/System/Xml/XmlConvert.cs#L1128-L1146

https://github.com/dotnet/corefx/blob/6d723b8e5ae3129c0a94252292322fc19673478f/src/System.Private.Xml/src/System/Xml/Schema/XsdDuration.cs#L229-L236

Now, arguably, TimeSpan itself should support parsing and formatting ISO8601 duration formats. Maybe it would be better, as a first step, to add a new standard TimeSpan format string, similar to DateTime[Offset]'s round-trip ("O", "o") format specifier? Adding a converter after that would be really simple.

[yhvicey]

This also changed default behavior of how AspNetCore handing returning TimeSpan type in API, see dotnet/AspnetCore#11724. This might be a breaking change.

[rezabayesteh]

The simplest solution is to create a custom converter:

public class TimeSpanConverter : JsonConverter<TimeSpan>
    {
        public override TimeSpan Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            return TimeSpan.Parse(reader.GetString());
        }

        public override void Write(Utf8JsonWriter writer, TimeSpan value, JsonSerializerOptions options)
        {
            writer.WriteStringValue(value.ToString());
        }
    }

[FForthmann]

Talking about customer evidence: for our software-development support for TimeSpan is highly necessary.

[IGx89]

Ran into this today porting an app to .NET Core 3.0 as well. Because this is closed does that mean Microsoft has no plans to add native support? @khellang's comment seems a pretty convincing argument to me that it should be on a roadmap somewhere...

[steveharter]

Re-opening for 5.0 based on additional asks. The ISO8601 duration is likely the best representation although compat with Newtonsoft should also be considered.

[mfeingol]

Just ran into this issue today. The default behavior is worse than previous behavior and was entirely unexpected. We should fix it, either by using ISO8601 or being compatible with Newtonsoft.

[khellang]

The default behavior is worse than previous behavior and was entirely unexpected.

@mfeingol What behavior? That it simply fails?

We should fix it, either by using ISO8601 or being compatible with Newtonsoft.

It's really easy to just add the workaround @rezabayesteh mentioned.

[mfeingol]

@khellang: what I observed with a relatively vanilla ASP.NET Core project is that it serializes a Timespan? as a HasValue field, and then each of the properties of the TimeSpan struct.

It's really easy to just add the workaround

I did, but that shouldn't be necessary for such a commonly used type.

[bashocz]

I've just found this issue today (reported by my customer) and have to switch back all of my aspnet webapi and apps to use Newtonsoft.Json serializer instead using configuration in Startup.cs:

services.AddControllers()
.AddNewtonsoftJson(options =>
{
options.SerializerSettings.NullValueHandling = Newtonsoft.Json.NullValueHandling.Ignore;
});

In my cases I use a few nullable TimeSpan (TimeSpan?) and System.Text.Json has serialized it as:

{
"hasValue": true,
"value": {
"tick":0,
"days": 0,
"hours": 0,
"milliseconds": 0,
"minutes": 0,
"seconds": 0,
"totalDays": 0,
"totalHours": 0,
"totalMilliseconds": 0,
"totalMinutes": 0,
"totalSeconds": 0
}
}

This causes a little problem for javascript objects in web browsers, as well as for various cross-platform (means various programming languages) deserializers consuming my apis.

I would expect same serialization result as Newtonsoft.Json serializer:

{
"timeSpan": "00:00:00.0000000",
"nullTimeSpan": null
}

[khellang]

I've just found this issue today (reported by my customer) and have to switch back all of my aspnet webapi and apps to use Newtonsoft.Json serializer instead

@bashocz Why won't the workaround mentioned in https://github.com/dotnet/corefx/issues/38641#issuecomment-540200476 work for you?

[bashocz]

Why won't the workaround mentioned in #38641 (comment) work for you?

@khellang I just want to highlight TimeSpan serialization is an issue for another developers, and give it an attention. I'd very like to switch to System.Text.Json, however there are some obstacles.

I'd investigated new System.Text.Json a few weeks ago and found it is not feature complete. I've raised an issue dotnet/corefx#41747 and was pointed to other dotnet/corefx#39031, dotnet/corefx#41325, dotnet/corefx#38650 related. Because of that all of our internal microservices still use Newtonsoft.Json.

For unknown reason I forgot to manage devs to fix it on public APIs and webapps as well.

BTW: I try to avoid workarounds as much as possible in production code.. it's hard to maintain and remove it in future.

[arisewanggithub]

@khellang , it's not that a workaround won't work. It's just such a basic thing shouldn't need developers to add a workaround. As a big feature introduced for .NET core 3, it shouldn't lack such basic implementations.

[martincostello]

@arisewanggithub There are global settings available to controllers. You can configure it via AddJsonOptions(), or you can pass a JsonSerializerOptions instance to the Json() controller method.

[dev-thinks]

Is this closed cause we are having workaround?!

[khellang]

Is this closed cause we are having workaround?!

The issue is still open, pending an API/behavior proposal, review and implementation. In the meantime, it's pretty easy to work around by using the converter mentioned in https://github.com/dotnet/corefx/issues/38641#issuecomment-540200476.

[CodeBlanch]

For anyone blocked by this, here's a NuGet package with a converter (JsonTimeSpanConverter) we can use ahead of the 5.0 drop: Macross.Json.Extensions

Supports TimeSpan & Nullable<TimeSpan> types.

[alvipeo]

Oh, shit, this hurts!

[adams85]

For anyone blocked by this, here's a NuGet package with a converter (JsonTimeSpanConverter) we can use ahead of the 5.0 drop: Macross.Json.Extensions

Supports TimeSpan & Nullable types.

According to my tests, nullable value types are already handled by the framework. You don't need much more than the following:

public class DelegatedStringJsonConverter<T> : JsonConverter<T>
    where T : notnull
{
    private static readonly bool s_typeAllowsNull = !typeof(T).IsValueType || Nullable.GetUnderlyingType(typeof(T)) != null;

    private readonly Func<string, T> _parse;
    private readonly Func<T, string> _toString;

    public DelegatedStringJsonConverter(Func<string, T> parse, Func<T, string> toString)
    {
        _parse = parse;
        _toString = toString;
    }

    public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        // null tokens are handled by the framework except when the expected type is a non-nullable value type
        // https://github.com/dotnet/corefx/blob/v3.1.4/src/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.HandleNull.cs#L58
        if (!s_typeAllowsNull && reader.TokenType == JsonTokenType.Null)
            throw new JsonException($"{typeof(T)} does not accept null values.");

        return _parse(reader.GetString());
    }

    public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
    {
        // value is presumably not null here as null values are handled by the framework
        writer.WriteStringValue(_toString(value));
    }
}

(EDIT: after taking a closer look at the framework sources, it turned out that the checks for null tokens/null values are redundant. The code above was updated accordingly.)

Then configure JsonSerializerOptions like this:

options.Converters.Add(new DelegatedStringJsonConverter<TimeSpan>(
    value => TimeSpan.Parse(value, CultureInfo.InvariantCulture),
    value => value.ToString(null, CultureInfo.InvariantCulture)));

I don't think it's a good idea to add a 3rd party dependency if you can cope with such a few lines of extra code. We're not in NPM land after all. ;)

Of course, it'd be best if we didn't need a workaround for such a basic type at all.

[CodeBlanch]

@adams85 FYI Nullable value types are bugged < .NET 5 when used with JsonConverterAttribute. See #32006. I personally prefer to use the attribute style over global configuration, hence the extensions & bug fix, but your converter is excellent. Do you mind if I add it to Macross.Json.Extensions to help anyone else out that might also benefit from it and doesn't mind traveling to "NPM land" on occasion? :)

[adams85]

@CodeBlanch Thank you for pointing this quirk with JsonConverterAttribute out. It's definitely part of the full story.

And, of course, you are granted to add my converter to your lib if you like it. :)

[taspeotis]

Starting with .NET 5.0 Preview 5 I've been getting Cannot skip tokens on partial JSON errors round-tripping my entities to JSON with System.Text.Json. The offending line and character is the colon in "Ticks":

Anyway if I use the workaround and serialize TimeSpan in a sensible fashion it works fine.

+1 from me because my initial reaction upon opening the .json file to inspect it only to find the TimeSpan serialized in all of its structural glory was "surely not..."

[layomia]

From @jsedlak in https://github.com/dotnet/runtime/issues/42356:

Issue Title

The System.Text.Json.JsonSerializer class is unable to deserialize a TimeSpan property, even though it can serialize it.

General

Sample project demonstrating this issue is available here: https://github.com/jsedlak/TestTimeSpan

If you return a TimeSpan property in a WebApi, it is correctly serialized to JSON:

[ { "forecastLength": { "ticks": 36000000000, "days": 0, "hours": 1, "milliseconds": 0, "minutes": 0, "seconds": 0, "totalDays": 0.041666666666666664, "totalHours": 1, "totalMilliseconds": 3600000, "totalMinutes": 60, "totalSeconds": 3600 } } ]

But the default deserializer extensions (HttpClient.GetFromJsonAsync) cannot handle the property. It returns an empty TimeSpan. A custom converter must be used to deserialize the object.

DotNet Info

`.NET Core SDK (reflecting any global.json):
Version: 3.1.302
Commit: 41faccf259

Runtime Environment:
OS Name: Windows
OS Version: 10.0.20201
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk3.1.302\

Host (useful for support):
Version: 3.1.6
Commit: 3acd9b0cd1

.NET Core SDKs installed:
3.1.302 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]`

[layomia]

We'd like to address this in .NET 6.0 and would appreciate a community contribution here. The workaround in the meantime is to use a custom converter e.g. https://github.com/dotnet/runtime/issues/29932#issuecomment-540200476. The implementation should be based on the ISO8601 format for duration to be consistent with the behavior for DateTime and DateTimeOffset.

[taspeotis]

We'd like to address this in .NET 6.0

.NET 5 isn't released yet, I'd love if we bit the bullet in v5 (semver permits major breaking changes v3 => v5) instead of waiting for v6.

[layomia]

We couldn't fit this feature into 5.0. We had to prioritize along with the rest of the 5.0 feature work, and this one didn't fit. FWIW this was close to the cut line, thus the late move.

Just to highlight the simple workaround with a custom converter again:

public class TimeSpanConverter : JsonConverter<TimeSpan>
{
    public override TimeSpan Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        return TimeSpan.Parse(reader.GetString());
    }

    public override void Write(Utf8JsonWriter writer, TimeSpan value, JsonSerializerOptions options)
    {
        writer.WriteStringValue(value.ToString());
    }
}

var options = new JsonSerializerOptions { Converters = { new TimeSpanConverter() };
JsonSerializer.Serialize(myObj, options);

...

[YohDeadfall]

@layomia, I can take this for 6.0.

[BrunoZell]

I would recomment to use the invariant culture when opting to build a json converter yourself:

public class TimeSpanConverter : JsonConverter<TimeSpan>
{
    public override TimeSpan Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        return TimeSpan.Parse(reader.GetString(), CultureInfo.InvariantCulture);
    }

    public override void Write(Utf8JsonWriter writer, TimeSpan value, JsonSerializerOptions options)
    {
        writer.WriteStringValue(value.ToString(format: null, CultureInfo.InvariantCulture));
    }
}

[aquacat123]

I am working with asp.net core 3.1 and ran into this problem today. I am using teleriks grid with signalr and inline editing on rows that contain nullable timespans and can see that the payload contains the timespans data but deserialization fails. I ended up using the Macross.Json.Extensions library and reworking some of my javascript to make it work, not ideal though, i hope this gets a proper fix.

[tjipke]

And if you are looking for ISO8601 format, you can use this:
```c#

public class TimeSpanConverter : JsonConverter<TimeSpan>
{
    public override TimeSpan Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        var stringValue = reader.GetString();
        return System.Xml.XmlConvert.ToTimeSpan(stringValue); //8601
    }

    public override void Write(Utf8JsonWriter writer, TimeSpan value, JsonSerializerOptions options)
    {
        var stringValue = System.Xml.XmlConvert.ToString(value); //8601
        writer.WriteStringValue(stringValue);
    }
}

```

[gojanpaolo]

Please consider TimeSpan of at least 24 hours.. e.g. "24:00:00" should be equivalent to new TimeSpan(24, 0, 0)

[CodeBlanch]

@gojanpaolo

Please consider TimeSpan of at least 24 hours.. e.g. "24:00:00" should be equivalent to new TimeSpan(24, 0, 0)

That isn't really a JSON thing, it's part of the TimeSpan parse logic. Short answer: Use "1.00:00:00" for 24 hours. Long answer: I went spelunking through the runtime code a while back to figure out why "24:00:00" doesn't parse as expected and wrote a blog post about it.