Marshmallow: Does not preserve timezone naive format?

[jayennis22]

Currently, a datetime in timezone naive format (e.g. from datatime.utcnow()) is converted to timezone aware (+00:00) in serialization, but is not converted back to timezone naive in deserialization. This means that the serialization/deserialization process is not really symmetrical and you can't really use timezone naive formats. If that was intended, it should be documented. If this is a bug, I'm happy to look at producing a pull request.

Here is a test demonstrating what I am referring to:

from marshmallow import Schema, fields, post_load
from datetime import *

class Test(object):
    def __init__(self, ts):
        self.timestamp = ts

class TestSchema(Schema):
    timestamp = fields.DateTime()

    @post_load
    def make_code(self, data):
        instance = Test.__new__(Test)
        instance.__dict__.update(data)
        return instance

now = datetime.utcnow()
test = Test(now)
print test.timestamp
assert(now == test.timestamp)

dump = TestSchema().dump(test).data
load = TestSchema().load(dump).data
print load.timestamp
assert(now == load.timestamp)

Output:

$ python mmtimetest.py
2015-10-25 23:23:23.489722
2015-10-25 23:23:23.489722+00:00
Traceback (most recent call last):
  File "mmtimetest.py", line 25, in <module>
    assert(now == load.timestamp)
TypeError: can't compare offset-naive and offset-aware datetimes

[jklapuch]

from docs:

class marshmallow.fields.DateTime(format=None, **kwargs)[source]
A formatted datetime string in UTC.

Example: '2014-12-22T03:12:58.019077+00:00'

so If I set same format from example:

from marshmallow import Schema, fields, post_load
from datetime import *
import pytz

class Test(object):
    def __init__(self, ts):
        self.timestamp = ts

class TestSchema(Schema):
    timestamp = fields.DateTime()

    @post_load
    def make_code(self, data):
        instance = Test.__new__(Test)
        instance.__dict__.update(data)
        return instance


now = datetime.now(pytz.utc)
test = Test(now)
print test.timestamp
assert(now == test.timestamp)

dump = TestSchema().dump(test).data
load = TestSchema().load(dump).data
print load.timestamp
assert(now == load.timestamp)

output:

2015-10-26 09:57:07.060000+00:00
2015-10-26 09:57:07.060000+00:00

[jayennis22]

Point taken. So this is clearly expected behavior. Can I recommend that the following sentence be added to the description of fields.DateTime?

Timezone naive DateTime objects are converted to UTC (+00:00) by Schema.dump().  Schema.load() returns DataTime objects that are timezone aware.

Thanks!

[mark-anders]

I guess I'm missing the point and I'm running into the exact same issue. In my case, my DateTime's are stored in PostgreSQL and they do not have a tz. What comes out of Marshmallow always seems to have one.

In Meph's code, now = datetime.now(pytz.utc) explicitly sets a timezone. If you remove it, you get the exception that you can't compare the objects.

It seems that it would be good to have a way of converting UTC dates into dates without a timezone.

[lafrech]

Schema.load() returns DataTime objects that are timezone aware.

I find this sentence a bit misleading.

A naive input that goes through serialization and back will become aware because it is serialized as aware UTC.

But deserializing a naive datetime will produce a naive output.

def is_naive(d):
    return d.tzinfo is None or d.tzinfo.utcoffset(d) is None

dates = (
    '2014-12-22T03:12:58.019077+00:00',
    '2014-12-22T03:12:58.019077+00:10',
    '2014-12-22T03:12:58.019077Z',
    '2014-12-22T03:12:58.019077',
)

# Using field directly
dates_deserialized = [
    ma.fields.DateTime().deserialize(date)
    for date in dates]

dates_naive = [is_naive(date) for date in dates_deserialized]

assert dates_naive == [False, False, False, True]


# Or equivalently through Schema
class Test(ma.Schema):
    dt_field = ma.fields.DateTime()

dates_deserialized = [
    Test().load({'dt_field': date})[0]['dt_field']
    for date in dates]

dates_naive = [is_naive(date) for date in dates_deserialized]

assert dates_naive == [False, False, False, True]

So by itself, Schema.load() does not always return DataTime objects that are timezone aware. This is true if the datetime was serialized using Schema.dump(), but it could come from another source.