Esp-idf: time_t is 32-bit wide, please change to 64-bit to make applications Y2K38 safe (IDF-350)

[e7p]

The time_t base type on which all unix-timestamp (seconds after 1970-01-01) related functions (such as gettimeofday) do rely on, is only 4 bytes (32-bit) wide. It is suggested to use an 8 byte (64-bit) variable here, as otherwise all unix-timestamps after 2038-01-18 will be wrong. To sketch this, I coded an example (main.c), which shows the exact problem. Notably is, that not only the current wall-clock time is affected by this, also calculations on the time (for example, calendar events and the like) which result in a value higher than the noted limit in 2038, are affected. This could become at some time a serious issue, so it would be best to act soon. See also the wikipedia page on this problem.

So it would be very appreciating to change the time_t base type to an "signed long long" type.

The current output of my sample code can be seen here:

I (508) example: sizeof(time_t): 4 (32-bit)
I (1508) example: Time now: 000000007FFFFFF8 (2038-01-19 03:14:00)
I (2508) example: Time now: 000000007FFFFFF9 (2038-01-19 03:14:01)
I (3508) example: Time now: 000000007FFFFFFA (2038-01-19 03:14:02)
I (4508) example: Time now: 000000007FFFFFFB (2038-01-19 03:14:03)
I (5508) example: Time now: 000000007FFFFFFC (2038-01-19 03:14:04)
I (6508) example: Time now: 000000007FFFFFFD (2038-01-19 03:14:05)
I (7508) example: Time now: 000000007FFFFFFE (2038-01-19 03:14:06)
I (8508) example: Time now: 000000007FFFFFFF (2038-01-19 03:14:07)
I (9508) example: Time now: FFFFFFFF80000000 (1901-12-13 20:45:52)
I (10508) example: Time now: FFFFFFFF80000001 (1901-12-13 20:45:53)
I (11508) example: Time now: FFFFFFFF80000002 (1901-12-13 20:45:54)
I (12508) example: Time now: FFFFFFFF80000003 (1901-12-13 20:45:55)
I (13508) example: Time now: FFFFFFFF80000004 (1901-12-13 20:45:56)
I (14508) example: Time now: FFFFFFFF80000005 (1901-12-13 20:45:57)
I (15508) example: Time now: FFFFFFFF80000006 (1901-12-13 20:45:58)
I (16508) example: Time now: FFFFFFFF80000007 (1901-12-13 20:45:59)
I (17508) example: Time now: FFFFFFFF80000008 (1901-12-13 20:46:00)

[hwmaier]

Or alternatively make it an unsigned 32-bit integer postponing the 32-bit overflow by further 68 years.

[jack0c]

@igrr Can we close this issue?

[igrr]

@jack0c no, we will be working on it.

[igrr]

Currently we supply newlib as a set of pre-built libraries. time_t size is fixed at the time these libraries are built. We plan to also allow for doing custom builds of newlib in IDF. This would allow changing sizeof(time_t) and other things.

For the change of default time_t size, we will have to wait until one of the major releases, since that would be a breaking change for some existing applications.

[vielmetti]

I see a v2.0 milestone that looks complete at https://github.com/espressif/esp-idf/milestone/1 being tracked (and complete), but no other issue-based tracking of release milestones in Github.

https://github.com/espressif/esp-idf/releases/tag/v3.1-beta1 is the v3.1 beta.

Do you anticipate this waiting for a v4.0 major release? I know that might be some time out (maybe 2019 at your current release cadence?)

[projectgus]

Hi @vielmetti ,

Thanks for the reminder about the GitHub Milestones feature. As you can see, we haven't been using it. We plan to start publishing a version roadmap soon, but at the moment there isn't one publically accessible.

Currently the plan is for V4.0 to be in beta release by the end of the year, for a final release in early 2019.

[vielmetti]

Thanks @projectgus - hope that v4.0 is perking right along, and that you can confirm that the time_t changes will be in it.

[BarryAndrews]

Is there any further update on when this year 2038 issue will be resolved?

[projectgus]

No ETA for the newlib change & resolution of time_t width yet, I'm afraid. Although we are tracking this.

Our timeline slipped a bit, currently v3.3 is in beta and we're about to start work on v4.0 breaking features, of which breaking newlib changes like this will definitely be one.

[remcoNL]

@projectgus Can you maybe consider a major update with just this change, (since it has a deadline..) In the spirit of continuous delivery (-;

[projectgus]

@remcoNL Probably not I'm afraid, as our releases still involve a lot of manual testing (we do automated testing as well, before branches are pushed to github, but it doesn't cover everything).

Will your product have OTA capability? We guarantee that an OTA-updatable ESP-IDF version with year 2038 compliance will be available with well over a decade to spare before the problem strikes. :)

[remcoNL]

That's a relief...

I will now subtract 883612800 seconds and dates and weekdays are the same again! Also this gives you an extra 28 years to make the update!

https://math.stackexchange.com/questions/536847/how-often-in-years-do-calendars-repeat-with-the-same-day-date-combinations-juli

[projectgus]

Hi @remcoNL,

Glad you found a workaround!

We've decided to push this back from the v4.0 release, as we're already making a lot of other newlib changes. Sorry for the inconvenience. The current plan is as follows:

• 64-bit time_t will be supported in IDF v4.1 as an optional configuration. It may require building a custom toolchain where newlib is configured with 64-bit time_t (this process will be documented).
• 64-bit time_t is planned to become default in IDF v5.0

We're currently averaging a major version per year, so there should still be plenty of time between the v5.0 release and the 2038 deadline.

OTA updating a firmware with 32-bit time_t to 64-bit time_t should work fine, also (assuming the OTA update is complete before 2038.)

[remcoNL]

Really appreciate the work, and i'll schedule my updates for next year (or one year later).

[themadsens]

Please remember struct timeval from gettimeofday() As of IDF 4.0 the tv_sec member is a long (defined in lwip/sockets.h(!?)