Marshmallow: improve docs about snake case and camel case

[twosigmajab]

A Marshmallow user I'm supporting writes:

I'm running into an issue with a marshmallow/swagger/javascript interaction. In javascript (and serialized json in general), field names are expected to be camelCase. In python they're supposed to be snake_case. So to work around this, I need to dump/load in python to do snake_case to camelCase translation.

Here is what we came up with:

from marshmallow import Schema as OriginalSchema, fields

def camel_from_snake(s):
    parts = iter(s.split("_"))
    return next(parts) + "".join(i.title() for i in parts)

class Schema(OriginalSchema):
    """Override marshmallow.Schema for snake_case/camelCase marshalling."""

    def on_bind_field(self, field_name, field_obj):
        # field_obj.data_key = camel_from_snake(field_name)  # TODO: marshmallow v3
        field_obj.load_from = camel_from_snake(field_name)
        field_obj.dump_to = camel_from_snake(field_name)

Searching the docs for "camelCase" currently only gives a single result in the "upgrading" section (in which, to be nitpicky, the example confuses PascalCase with camelCase).

This seems like such a common need that it deserves at least its own dedicated section in the docs, if not actual dedicated API support. Would you be open to a PR addressing this?

Thanks for your work on marshmallow!

[deckar01]

https://marshmallow.readthedocs.io/en/3.0/extending.html#pre-processing-and-post-processing-methods

You can make a schema mixin that converts the data to snakecase before loading it and converts it back to camelcase after dumping it.

from marshmallow import Schema, fields, pre_load, post_dump
from stringcase import camelcase, snakecase

class JavaScriptMixin:
    @pre_load
    def to_snakecase(self, data, **kwargs):
        return {snakecase(key): value for key, value in data.items()}

    @post_dump
    def to_camelcase(self, data, **kwargs):
        return {camelcase(key): value for key, value in data.items()}

class TestSchema(Schema, JavaScriptMixin):
    foo_bar = fields.String()
    boo_baz = fields.String()

schema = TestSchema()
data = schema.load({'fooBar': 'test', 'booBaz': 'done'})
# {'foo_bar': 'test', 'boo_baz': 'done'}
schema.dump(data)
# {'fooBar': 'test', 'booBaz': 'done'}

[deckar01]

I'm not sure how common this use case is. I have always thought the backend defined the API so the front end must conform to it. Most applications I have seen that automate this process (due to scale like gitlab) do so on the front end. Smaller apps just encapsulate it away from the rest of the logic with an API module that maps the functional API to the network API.

[twosigmajab]

Thanks! Please note the issue title and description. This is about adding dedicated docs (specifically mentioning the problem of "JSON uses camelCase for fields but Python uses snake_case"), if not an actual, dedicated mixin built in, so that each user doesn't have to reinvent this almost-universally-needed wheel.

[deckar01]

JSON uses camelCase

As far as I know JSON is case agnostic. A javascript casing utility is probably outside the scope of the marshmallow core library. I would encourage you to publish a package to pypi if you think others could benefit from it.

[dLuna]

(@deckar01 It looks like your solution is both more code and requires more dependencies than overriding the schema.)

The Google style guide (https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format) requires camelCase for JSON. And for people who use Swagger to generate their API, the default javascript generation uses camelCase.

It's not a question of whether it's possible to override (since it clearly is), it's a question of whether Marshmallow should be easy to integrate with other systems without using home-baked ugly hacks. And I believe, at a minimum, there should be a section in the docs explaining this particular case in a generic fashion.

[deckar01]

This seems like such a common need that it deserves at least its own dedicated section in the docs

And I believe, at a minimum, there should be a section in the docs explaining this particular case in a generic fashion.

I think the general topic of pre/post processing data is already documented. If you think this can be improved PRs are welcome.

https://marshmallow.readthedocs.io/en/3.0/extending.html#pre-processing-and-post-processing-methods

dedicated API support

I don't think JavaScript and the Google style guide specific features are in line with the current functionality of this library.

Would you be open to a PR addressing this?

This could make a good addition to the examples folder.

https://marshmallow.readthedocs.io/en/latest/contributing.html#contributing-examples

I think it would be even better as a pypi package listed on the ecosystem page of the wiki.

https://github.com/marshmallow-code/marshmallow/wiki/Ecosystem

[sloria]

Sure, I think this would be fine to add in the examples docs. It's a fairly common use case, though there are many modern counterexamples. To Jared's point, it's often handled on the client-side.

That said, I think the OP's implementation is more correct than using pre_load. Using data_key allows for better introspection (e.g. by apispec and other schemagen libs).

[sloria]

See https://marshmallow.readthedocs.io/en/latest/examples.html#inflection-camel-casing-keys