Marshmallow: The deserialization of DateTime fields doesn't match the documentation

[gdetrez]

According to the documentation for marshmallow.fields.DateTime:

Schema.load returns datetime objects that are timezone-aware.

But that doesn't seem to be the case in 3.0.0b14:

>>> marshmallow.__version__
'3.0.0b14'
>>> d = marshmallow.fields.DateTime().deserialize('2018-09-18T16:53:11.876810+00:00')
>>> d
datetime.datetime(2018, 9, 18, 16, 53, 11)
>>> repr(d.tzinfo)
'None'

I would have expected the deserialized datetime's tzinfo to be set. Or did I misunderstood the documentation?

[lafrech]

Right. Known issue but good to bring back up.

The doc is misleading. Well... wrong. See this old discussion and my comment at the end: https://github.com/marshmallow-code/marshmallow/issues/309

Besides, I think this is an important pitfall. Here's a proposal for fields that would enforce awareness: https://github.com/marshmallow-code/marshmallow/issues/520

I can't say this will be done for Marshmallow 3, but at least fixing the doc (MA2 and MA3) would be great.

I'm marking this as a documentation issue.

Please feel free to contribute to the discussion/implementation (#520) if you're interesting in enforcing awareness in the code.

[deckar01]

I can repro this in python 3.7.0, but not python 2.7.15.

# Python 2.7.15 (default, Jul 23 2018, 21:27:06) 
# [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] on darwin
# Type "help", "copyright", "credits" or "license" for more information.
import marshmallow
marshmallow.__version__
# '3.0.0b14'
d = marshmallow.fields.DateTime().deserialize('2018-09-18T16:53:11.876810+00:00')
d
# datetime.datetime(2018, 9, 18, 16, 53, 11, 876810, tzinfo=tzutc())
repr(d.tzinfo)
# 'tzutc()'

[lafrech]

Just realizing this is a bit different than what I thought. Sorry for overlooking.

My point is naive datetimes should be deseralized as aware according to the docs, and I'd like to be able to enforce awareness/naiveness whatever the source date awareness.

The example here is worst as the serialized representation is aware and awareness is lost.

[MattF-NSIDC]

Ran in to this issue today running unit tests in CircleCI which passed on my local machine. My schemas were loading timezone-aware strings to naive datetime objects, but the docs clearly say "Schema.load returns datetime objects that are timezone-aware."

Installing python-dateutil fixed the problem. The code explicitly behaves differently depending on whether python-dateutil is installed on the system:

https://github.com/marshmallow-code/marshmallow/blob/fd3f2db051cd1cec6bce9cd18571efd18231a2aa/src/marshmallow/utils.py#L227

Can the docs please be updated to reflect this? That would have saved me some time today!

Or maybe marshmallow can be made dependent on python-dateutil? Would this help with platform independence?

[MattF-NSIDC]

@deckar01 I know it's been a while but is it possible your 2.7 environment had python-dateutil installed, but it was not installed in 3.7?

[sloria]

Closing this. @lafrech is working on this as part of https://github.com/marshmallow-code/marshmallow/issues/1234 . Discussion can continue there.

[lafrech]

deckar01 I know it's been a while but is it possible your 2.7 environment had python-dateutil installed, but it was not installed in 3.7?

I'm pretty sure that was the reason.

Further refactor will happen in #1234 but I believe the issue about lost timezone info is fixed already thanks to https://github.com/marshmallow-code/marshmallow/pull/1249, reworked in https://github.com/marshmallow-code/marshmallow/pull/1265.

The issue about the docs being inaccurate should be addressed during the refactor in #1234.