Arrow: OverflowError on 32 bit Raspberry Pi 4

[JoshPaulie]

I moved my bot to my Rpi4. Arrow throws this error on script launch.

Issue Description

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/arrow/constants.py", line 16, in <module>
    _MAX_TIMESTAMP = datetime.max.timestamp()
OverflowError: timestamp out of range for platform time_t

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main_goonbot_3.py", line 6, in <module>
    from config import BotConfig
  File "/home/pi/Goonbot3/config.py", line 1, in <module>
    import cassiopeia as cass
  File "/usr/local/lib/python3.7/dist-packages/cassiopeia/__init__.py", line 2, in <module>
    from ._configuration import get_default_config, Settings, CassiopeiaConfiguration as _CassiopeiaConfiguration
  File "/usr/local/lib/python3.7/dist-packages/cassiopeia/_configuration/__init__.py", line 1, in <module>
    from .settings import Settings, get_default_config
  File "/usr/local/lib/python3.7/dist-packages/cassiopeia/_configuration/settings.py", line 9, in <module>
    from ..data import Region, Platform
  File "/usr/local/lib/python3.7/dist-packages/cassiopeia/data.py", line 2, in <module>
    import arrow
  File "/usr/local/lib/python3.7/dist-packages/arrow/__init__.py", line 2, in <module>
    from .api import get, now, utcnow
  File "/usr/local/lib/python3.7/dist-packages/arrow/api.py", line 12, in <module>
    from arrow.arrow import TZ_EXPR, Arrow
  File "/usr/local/lib/python3.7/dist-packages/arrow/arrow.py", line 34, in <module>
    from arrow import formatter, locales, parser, util
  File "/usr/local/lib/python3.7/dist-packages/arrow/parser.py", line 26, in <module>
    from arrow.util import next_weekday, normalize_timestamp
  File "/usr/local/lib/python3.7/dist-packages/arrow/util.py", line 6, in <module>
    from arrow.constants import (
  File "/usr/local/lib/python3.7/dist-packages/arrow/constants.py", line 25, in <module>
    else datetime(2038, 1, 18, 23, 59, 59, 999999).timestamp()
OverflowError: timestamp out of range for platform time_t

System Info

• 🖥 OS name and version:

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

• 🐍 Python version: Python 3.7.3
• 🏹 Arrow version: 1.0.1

[jadchaar]

@JoshPaulie thanks for reporting this. Are you running a 32 bit or 64 bit OS?

[jadchaar]

Also, what do you get when you type datetime.max.timestamp() into the python console? Do you get an overflow error?

[JoshPaulie]

@jadchaar

Are you running a 32 bit or 64 bit OS?

32 bit!

What do you get when you type datetime.max.timestamp() into the python console? Do you get an overflow error?

>>> import datetime
>>> datetime.max.timestamp()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'datetime' has no attribute 'max'

Let me know if ran the test incorrectly.

[krisfremen]

@JoshPaulie thanks for testing

This would be the correct invocation:

from datetime import datetime
datetime.max.timestamp()

Cheers!

[JoshPaulie]

Thanks @krisfremen.

>>> from datetime import datetime
>>> datetime.max.timestamp()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: timestamp out of range for platform time_t

Yes @jadchaar, same exception

[jadchaar]

Mind trying a few other values for me? Tell me what you get for the following:

datetime(2038, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2037, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2036, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2035, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2034, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2033, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2032, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2031, 1, 1, 23, 59, 59, 999999).timestamp()
datetime(2030, 1, 1, 23, 59, 59, 999999).timestamp()

What is the first value that does not give you an overflow error?

I am trying to figure out what a sane default max timestamp would be for 32 bit OSes. It seems like the current one we have (datetime(2038, 1, 18, 23, 59, 59, 999999).timestamp()), based on the Windows documentation, does not work for Linux.

[JoshPaulie]

Interesting outcome!
Test script

from datetime import datetime

time_stamps = [
    datetime(2038, 1, 18, 23, 59, 59, 999999),  # Reference
    datetime(2038, 1, 1, 23, 59, 59, 999999),
    datetime(2036, 1, 1, 23, 59, 59, 999999),
    datetime(2037, 1, 1, 23, 59, 59, 999999),
    datetime(2035, 1, 1, 23, 59, 59, 999999),
    datetime(2034, 1, 1, 23, 59, 59, 999999),
    datetime(2033, 1, 1, 23, 59, 59, 999999),
    datetime(2032, 1, 1, 23, 59, 59, 999999),
    datetime(2031, 1, 1, 23, 59, 59, 999999),
    datetime(2030, 1, 1, 23, 59, 59, 999999),
]

for time in time_stamps:
    try:
        time.timestamp()
    except OverflowError:
        print(f"{time} - Failed")
    else:
        print(f"{time} - Passed")

Output

(Goonbot3) pi@raspberrypi:~/Goonbot3 $ python3 datetime_test.py 
2038-01-18 23:59:59.999999 - Failed
2038-01-01 23:59:59.999999 - Passed
2036-01-01 23:59:59.999999 - Passed
2037-01-01 23:59:59.999999 - Passed
2035-01-01 23:59:59.999999 - Passed
2034-01-01 23:59:59.999999 - Passed
2033-01-01 23:59:59.999999 - Passed
2032-01-01 23:59:59.999999 - Passed
2031-01-01 23:59:59.999999 - Passed
2030-01-01 23:59:59.999999 - Passed

Very weird that a 17-day difference breaks it 🤔

[jadchaar]

Very interesting... I was using the datetime(2038, 1, 18, 23, 59, 59, 999999) object only because I expected all Linux systems to work fine with datetime.max.timestamp() (hence why I used the date referenced in the Microsoft docs). Really odd to see datetime.max.timestamp() overflow--it must be forcing the creation of a 64-bit integer, which doesn't work on a 32 bit system.

I will get a patch submitted to get this down to 2030!

[jadchaar]

Thank you so much for the feedback and assistance on this issue @JoshPaulie, we really appreciate it!

[JoshPaulie]

Wow! Glad we were able to figure this out in < 1 Day! Thanks for the quick response and tests ❤️
Excited to use my scripts on the Rpi 🥧

[jadchaar]

No problem! v1.0 was a big release, and we are not surprised some small bugs slipped through. For now, to get your bot running, you can lock the arrow dependency version to v0.17.0 until we release v1.0.2!

[jadchaar]

@JoshPaulie Arrow v1.0.2 has just been released to PyPI. Mind installing it with pip install -U arrow to see if this issue is resolved?

Thanks again for the help!

[JoshPaulie]

@jadchaar Works swell. Thanks gang 🤗