Marshmallow: Unenveloping None values doesn't work as expected

[tdevelioglu]

Trying to unenvelope serialized data when the contained data is None continues deserialization with the original data. This makes it impossible to deserialize nested fields with enveloped data where None is a valid value. (Related: http://jsonapi.org/format/)

Example (let's pretend our system allows for homeless employees):
https://gist.github.com/tdevelioglu/8e433355494ba022f48f
(I expect a deserialized Resource object, with an attribute 'address' with value 'None')

[sloria]

Thanks for the report. This does appear to be a limitation of the pre_/post_ methods. The decorators were meant to allow for in-place modifications of data and no return values (i.e. implicitly returning None).

We'll have to give this more thought to reach a solution.

[sloria]

@taion @lafrech @deckar01 Any ideas on this?

We could make it so that pre_/post_ methods must return the processed data and allow None to be returned. This would be a breaking change, which is why I'd like to resolve this for 3.0.

[deckar01]

The method receives the deserialized data and returns the processed data.

https://marshmallow.readthedocs.io/en/latest/api_reference.html#marshmallow.decorators.post_load

A return value of None indicating that the data was modified in place is not part of the docs. Anyone not returning a value is relying on undocumented behavior, but I can see how a lot of people could be relying on this unknowingly.

I recommend documenting the None behavior in the 2.x-line docs in addition to fixing it in 3.0.

[lafrech]

I also think this is the right thing to do.

The change in the code should be easy: just remove if_none stuff.

I like this. It fixes the use case while stripping unnecessary complication from the code.

[sloria]

Thanks guys. Let's make it so!

[taion]

This was my fault, wasn't it? Yeah, in retrospect trying to handle "not returning" from those hooks was way too cute. Sorry!

[deckar01]

It was discussed at the time, it just wasn't documented. It's never too late to add some docs. 😉

https://github.com/marshmallow-code/marshmallow/pull/191#issuecomment-91424999