Julia: different Julia ms epochs --- please deprecate one

[iwelch]

Julia 0.6.2:

julia> dt= Dates.now()
2018-03-10T09:40:53.757

julia> ep1, ep2= Dates.value( dt ), Dates.datetime2epochms( dt )
(63656358053757, 63687894053757)

julia> (ep2-ep1) /1000/60/60/24
365.0

Please deprecate one or the other, so we can agree on one Julia epoch.

regards, /iaw

[ViralBShah]

cc @quinnj

[JeffreySarnoff]

The Epoch least likely to be misascribed as other people write for other purposes is the one that is not the DateTime("0000-01-01").

[StefanKarpinski]

I have added the triage label as this seems like the sort of annoying little thing that is likely to cause an increasing amount of trouble the longer we let it continue.

[quinnj]

I don't even know what Dates.datetime2epochms is; @omus, is that something that was added for TimeZones.jl?

[omus]

It looks like the Dates.datetime2epochms function is exclusively used by the rounding code. From the manual:

As Julia Date and DateTime values are represented according to the ISO 8601 standard, 0000-01-01T00:00:00 was chosen as base (or "rounding epoch") from which to begin the count of days (and milliseconds) used in rounding calculations. (Note that this differs slightly from Julia's internal representation of Dates using Rata Die notation; but since the ISO 8601 standard is most visible to the end user, 0000-01-01T00:00:00 was chosen as the rounding epoch instead of the 0000-12-31T00:00:00 used internally to minimize confusion.)

julia> Dates.value(DateTime(0,12,31))
0

julia> Dates.datetime2epochms(DateTime(0))
0

@spurll originally wrote the Date rounding code and could probably elaborate more.

[spurll]

@omus is correct. Julia's internal representation uses the Rata Die count for Dates (which results in a technical epoch of 0000-12-31) and the same epoch for DateTimes. This epoch is fairly unusual, and resulted in some unexpected behaviour for some rounding corner cases (I don't have time to dig through my notes now for the details, but i'm happy to do so once I'm back from a work trip in a couple weeks).

I went back and forth quite a bit as to how to resolve this issue, and as outlined in the documentation, I eventually settled on a "rounding epoch" of 0000-01-01T00:00:00 in an effort to make the results of the rounding minimally confusing to the user.

The Dates.datetime2epochms function name might warrant renaming (it is not exported, because I worried it might cause exactly this kind of confusion, and I think I used something like datetime2roundingepoch in my original standalone rounding package). Alternatively we could retool rounding to just use the 0000-12-31T00:00:00 epoch and document the places where that might lead to unexpected behaviour. (We'll get some corner cases no matter what we do, because date and time systems are what they are.)

[StefanKarpinski]

Would it make more sense to change the DateTime epoch from 0000-12-31 to 0000-01-01?

[spurll]

It makes sense to me (I had not previously encountered a language that used a Rata Die count, so it was a bit of a surprise) and I'm not aware of any problems that would arise as a result in existing Julia code. So I don't think it would be a big headache in Base, but changing what Dates.value spits out for a given date will probably cause a headache for somebody (though it seems relatively minor compared to a lot of the language changes we've seen). I'm also not sure what initial considerations went into the selection of Rata Die and the associated epoch, so there might be something I'm missing.

[StefanKarpinski]

Can someone clarify what the epoch situations here are? Which is the standard epoch, which is the weird one? How public is this? It's tempting to consider this an implementation detail, but there is a value function that seems to expose the epoch as part of the public behavior to date times. Unless I'm misreading this, I believe the original epoch choice was yours, @quinnj, can you explain what's up?

[iamed2]

but there is a value function that seems to expose the epoch as part of the public behavior to date times

Neither value nor datetime2epochms are exported.

[JeffreySarnoff]

According to http://aa.usno.navy.mil/jdconverter, 0001-01-01T00:00:00 (Jan 1st, 1AD at midnight of the start of this day) is JD 1721423.500000. There is no calendric year 0: the calendric day preceding 0001-01-01 is in the year 1BC (Dec 31st). Astronomical dates do not use AD, BC. The astronomical year zero corresponds to the calendric year 1BC. This is at the root of many mistaken date computations. That is the reason I suggested that using 1-Jan-0001 as the Epoch is preferable to using an earlier date.

While there is some disclarity regarding the origin of RataDie dates (see Wikipedia), for most of us who have spent time with the algorithms, RataDie 1 is Jan 1, 0001. A good converter (afaik) is https://www.vcalc.com/wiki/MichaelBartmess/Rata+Die+%28RD%29

[spurll]

This is true; there are also no "negative" years:

julia> now() - Dates.Year(2019)
-0001-03-14T14:00:06.372

The "year zero" referred to in my earlier comment is a fairly standard way of representing year 1 BCE (which is not great, in my opinion, but it is standard). While ISO 8601 doesn't provide clear guidance on how to display dates prior to the advent of the Gregorian calendar, languages that use ISO 8601 display standards for dates and times (Julia, Matlab, and many others) typically use a signed value for the year, with 0001 representing 1 CE, 0000 representing 1 BCE, -0001 representing 2 BCE, etc.

Though it's light on the details, this is mentioned in the ISO 8601 Wikipedia page.

[Edit: Whoops, you actually mentioned this stuff. I'm preparing for a trip and I only skimmed your reply. Sorry!]

[JeffreySarnoff]

yes -- no disagreement. My notes were about determining a value to use for our Epoch.

[iwelch]

curiosity from wikipedia: "Year zero does not exist in the Anno Domini system usually used to number years in the Gregorian calendar and in its predecessor, the Julian calendar. In this system, the year 1 BC is followed by AD 1. However, there is a year zero in astronomical year numbering (where it coincides with the Julian year 1 BC) and in ISO 8601:2004 (where it coincides with the Gregorian year 1 BC) as well as in all Buddhist and Hindu calendars."

[simonbyrne]

We should document the year 0 thing.

[JeffreySarnoff]

It messes up so many people. Often code writers. The cleanest approach is to document it and better still to provide the conversion routines that take a calendric date (AD, BC) or (CE, BCE) and deliver the corresponding Epoch relative value and vice-versa.

[iwelch]

I think the unexported base should only have one version of time. if someone needs the other time, any conversion routines should have to be explicitly imported, giving the appropriate hint that it is unusual.

[JeffreySarnoff]

makes sense -- I meant to say that we should provide the explicitly importable thing.

[JeffreySarnoff]

Have we decided on a single Epoch?

Godot may arrive with the best approach at any time. Meanwhile,
the RataDie approach is a systematization of what shakes out
as the better algorithmic launching point for multi-calendric
handling (according to Edward M. Reingold, Nachum Dershowitz).

The RataDie approach has 1 as its Epoch indical date number.
It is an ordinal count, a rooted sequence that matches our
1-based indexing [clever, that]. They can process dates
before the common era, still the perspective adopted is
oriented to spans of dates for which the calendrics at a
place are known and recorded by secular or religious authority.

Civil calendars get mushy before anno domini 8, and porous
before "0004-01-01". Julia will spend more effort alongside
geolocated, astronomical, pulsar, and atomic timekeeping;
additional work to support civil histories follows later.

And, while I have personal preference that differs from
the RataDie way, at the same time I know it to be sound.
Also, it comports with the way our Dates are computed.

julia> using Dates

julia> rata2datetime(1)
0001-01-01T00:00:00

As the other date appears only ?twice internally in the rounding code,
with no disrespect to taking the max ISO compatible approach,

(oh, there's Godot now)