Marshmallow: `missing` and `default` values not handled as documented

[tadams42]

Hi,

given following schema:

~~~python
from datetime import datetime
from marshmallow import Schema, fields

class Foo(object):
def __init__(self, created_at=None):
self.created_at = created_at

class FooSchema(Schema):
FOO_TIME_FORMAT = '%Y%m%d%H%M%S'

created_at = fields.LocalDateTime(
    format=FOO_TIME_FORMAT,
    default=lambda: datetime(2017, 5, 4, 3, 2, 1), 
    missing=lambda: datetime(2017, 9, 8, 7, 6, 5),
    required=False, allow_none=False
)

schema = FooSchema()
~~~

I get:

~~~python
json_str, dumps_errors = schema.dumps(Foo())
json_str

=> '{"created_at": null}'

data, loads_errors = schema.loads('{}')
data

=> {}

~~~

I would expect it to work as following

(https://marshmallow.readthedocs.io/en/latest/api_reference.html#marshmallow.fields.Field):

~~~python
json_str, dumps_errors = schema.dumps(Foo())
json_str

=> '{"created_at": "20170504030201"}'

data, loads_errors = schema.loads('{}')
data

=> {'created_at': datetime.datetime(2017, 9, 8, 7, 6, 5)}

~~~

I also tried to apply workaround from #378:

~~~python
FOO_TIME_FORMAT = '%Y%m%d%H%M%S'

class FooSchema(Schema):
created_at = fields.LocalDateTime(
format=FOO_TIME_FORMAT,
default=lambda: datetime.strftime(datetime(2017, 5, 4, 3, 2, 1), FOO_TIME_FORMAT),
missing=lambda: datetime.strftime(datetime(2017, 9, 8, 7, 6, 5), FOO_TIME_FORMAT),
required=False, allow_none=False
)

schema = FooSchema()
~~~

which gets me wanted result only partially, when deserializing:

~~~python
json_str, dumps_errors = schema.dumps(Foo())
json_str

=> '{"created_at": null}'

data, loads_errors = schema.loads('{}')
data

{'created_at': datetime.datetime(2017, 9, 8, 7, 6, 5)}

~~~

Tested on Python and marshmallow versions:

~python
import marshmallow
import sys
sys.version
=> '2.7.12+ (default, Sep 17 2016, 12:08:02) n[GCC 6.2.0 20160914]'
marshmallow.__version__
=> '2.13.0'
~

and

~python
import marshmallow
import sys
sys.version
'3.5.2+ (default, Sep 22 2016, 12:18:14) n[GCC 6.2.0 20160927]'
marshmallow.__version__
=> '2.13.0'
~

[podhmo]

This is work example.

from datetime import datetime
from marshmallow import Schema, fields
from marshmallow.utils import get_value, missing


class FooSchema(Schema):
    created_at = fields.LocalDateTime(
        format=FOO_TIME_FORMAT,
        default=lambda: datetime(2017, 5, 4, 3, 2, 1),
        missing=lambda: datetime(2017, 9, 8, 7, 6, 5).strftime(FOO_TIME_FORMAT),
        required=False, allow_none=False,
    )

    @classmethod
    def get_attribute(self, attr, obj, default):
        return get_value(attr, obj, default=default) or missing

detail

(sorry, I'm not good at English, but I'm trying to desrcribe about above code)

missing and default option

missing option is using for deserialization, and default is for serialization. and the behaviors leave each field class's implementation(especially , deserialize() and serialize()).

(In code, fields.Int and fields.UUID's deserialization are OK, if passing raw value instead of string type value. But, fields.Time, fields.Date, fields.DateTime 's deserialization are not supported, so must passing string type value(If these are not specification of marshmallow, I want to make a Pull Request).)

Simply, deserialization is string to your expected type conversion, so missing option's value is string. And serialization is your expected type to string conversion, so default option's value is your expected type.

fields.LocalDateTime(missing="2000/01/02/03/04/05", default=datetime(2000,1,2,3,4,5))

utils.missing and None

Your Foo object has attribute named created_at (it's value is None), so, marsmallow treats as already have created_at, then deserialized result not initialized by missing option's value.

if your want to change behavior as None is also missing, need to define subclass.

assert None != missing

class NoneIsAlsoMissingSchema(Schema):
    @classmethod
    def get_attribute(self, attr, obj, default):
        return get_value(attr, obj, default=default) or missing

[tadams42]

Thanks for your detailed reply!

Since this behavior is a bit confusing I'll try to find time to add something about it to examples and to docs. If/when I find time for it, I'll do pull request.

[sloria]

@tadamic Thanks for offering to update the docs. I would welcome a PR. And thanks @podhmo for your answer