Go: time: provide a complete RFC3339 time format parse function

[nim-nim]

The W3C profile of the ISO 8601 format, that the IETF formalized in RFC3339, specifies two timezone formats

• "Z", meaning that the time is given in UTC, without local timezone info
• with the local time offset to UTC, allowing to compute both UTC time and local time. The offset can be either positive or negative

The two timezone formats are clearly specified both in the W3C and the IETF document.

W3C:

TZD  = time zone designator (Z or +hh:mm or -hh:mm)

IETF RFC3339 ABNF:

time-zone       = "Z" / time-numoffset
time-numoffset  = ("+" / "-") time-hour [[":"] time-minute]  

/ means OR in ABNF, as is evident from the time-numoffset definition, that tells you that ISO 8601 timezone offset can be positive or negative.

The golang RFC3339 time format is non conformant, first because it tries to use "Z" in conjunction with timezone offsets (when the meaning of "Z" is that no timezone offset was provided), and second, because it only allows positive offsets, when the W3C ISO 8601 profile and RFC3339 are both very clear offsets can be negative.

As a result, go is unable to parse RFC3339 compliant date times such as the ones outputed by date --iso-8601=seconds

$ TZ="America/Los_Angeles" date --iso-8601=seconds
2019-03-28T08:14:26-0700
$ TZ="Europe/Paris" date --iso-8601=seconds
2019-03-28T16:15:04+0100

parsing time "2019-03-28T08:14:26-0700" as "2006-01-02T15:04:05Z07:00": cannot parse "-0700" as "Z07:00"

Please fix the RFC3339 time format in Go or, if the mistake is too old to be easily removed, add a new RFC3339std format that is conformant to the RFC 3339 and ISO 8601, as used by other software in IJSON, XML, etc.

[tv42]

The constant time.RFC3339 has a colon in the timezone offset. Your example input does not. That seems to be the only problem here.

I don't see a way to make the current time.Parse format parse timezones with both colons and without colons.

[nim-nim]

Ok, thanks, the error was clear as mud

Nevertheless RFC 3339 ABNF does specify that the colon is optional

time-numoffset  = ("+" / "-") time-hour [[":"] time-minute]  

So timestamps without the colon are conformant to RFC 3339 (and the date command, for example, will omit them by default). And Go won't parse them correctly.

[ianlancetaylor]

The documentation says RFC3339, RFC822, RFC822Z, RFC1123, and RFC1123Z are useful for formatting; when used with time.Parse they do not accept all the time formats permitted by the RFCs. I'm not sure there is anything to do here.

[nim-nim]

While I agree the optional parts of RFC3339 are a major PITA (the W3C was much smarter than the IETF when it defined a single canonical form that only varied depending on the precision of the desired timestamps), those optional parts do exist, and people do write software with their own understanding of the options to include or exclude, and this software outputs timestamps, that Go software can consume later, so the Go parsing part of RFC3339 timestamps at least should accept all the variations defined in the RFC.

Because if the parsing is not as permissive as allowed by the spec, individual Go devs will have to reinvent separately timestamp normalising rules, which is not efficient, and what the RFCs are supposed to avoid.

And please note I am not talking about all the time RFCs here. Only the most recent, that represents the state of the art in writing interoperable time. Telling projects to upgrade to RFC3339 to be sure their time strings are parsed by Go and other software stacks is perfectly fine. But the RFC3339 support needs to be solid and complete to do that.

[ianlancetaylor]

@nim-nim If I understand you correctly, you are asking for something that time.Parse is unable to implement. There is certainly nothing wrong with writing a function that can parse all valid RFC3339 time strings. But that function probably shouldn't live in the time package. The definition of RFC3339 in the time package is a minor convenience for use with time.Time.Format, not time.Parse.

[ulikunitz]

Just as info: The date version 8.28 on my system prints the colon in the offset with --iso-8601=seconds. So the format may have changed between versions.

Format with the RFC3339 constant creates a correct RFC3339 time stamp, but Parse is not able to process all valid time stamps according to section 5.6 of the RFC or the ABNF in the appendix. (Mainly because ABNF strings are case-insensitive and so lower-case t and z must be supported.) It also cannot parse all valid ISO8601 extended format time stamps since the time stamp may omit the time zone or use only a time offset providing hours.

I agree with @ianlancetaylor that Parse cannot support all variants of RFC3339 and a special parser function for that purpose would be required.

[nim-nim]

@ulikunitz @ianlancetaylor Then I suppose a separate function is needed to Go can consume RFC3339-compliant outputs produced by other software :(

[rsc]

It looks like if you want to parse an RFC3339 time without a colon, you can use:

RFC3339NoColon = "2006-01-02T15:04:05Z0700"

The broad claim in the initial report, that Go cannot parse RFC3339 times, is clearly wrong.
If you want to parse any one of a variety of possible formats, call time.Parse with each format
you want to accept.

I don't see much to do here.

[nim-nim]

But that requires to make every Go software, that consumes things with RFC3339 timestamps, to be aware of all the RFC3339 dialects, or to hope all the other software is works with keeps to a single RFC3339 variant. The second option is unrealistic: different software will output different RFC3339 timestamps, and even the same software can change its RFC3339 options depending on the deployed version.

[pcj]

What's the recommended pattern for 2019-04-09T16:38:44.743+0000? (timezone could be negative offset as well). Does this have an "out-of-the-box" solution? (serendipitiously relates to linked issue a few hours ago)

[Ronnie76er]

@nim-nim I believe you are mistaken about RFC 3339 saying the colon is optional in the timezone. If you look at section 5.6 (https://tools.ietf.org/html/rfc3339#section-5.6), where the ABNF is defined for RFC 3339, it says the colon is mandatory:

time-numoffset  = ("+" / "-") time-hour ":" time-minute

I think the section you pulled from was Appendix A (https://tools.ietf.org/html/rfc3339#appendix-A), which describes the ABNF of ISO8601. ISO8601 is more permissive, and does not require colons anywhere.

time-numoffset    = ("+" / "-") time-hour [[":"] time-minute]

[nim-nim]

Re-reading the RFC, I think you're right, I was tricked by the double ABNF in the document.

Thanks a lot for the verification.