Aiohttp: Persistent dumps setting for all web.json_response uses

Created on 11 May 2016 · 6Comments · Source: aio-libs/aiohttp

Long story short

Often times when you need a custom JSON implementation, you'll need it most times you send a JSON response (web.json_response)

Expected behaviour

User should be able to set a persistent dumps option for all json_responses to use.

Actual behaviour

User must provide the implementation each time web.json_response is called.

Steps to reproduce

#!/usr/bin/env python3

import datetime
from functools import partial
import json

from aiohttp import web


def default_json(obj):
    if isinstance(obj, datetime.datetime):
        return str(obj)
    raise TypeError('Unable to serialize {!r}'.format(obj))


async def index_handler(request):
    return web.json_response({
        'timestamp': datetime.datetime.now()
    }, dumps=request.app['dumps'])


# This is effectively the default implementation (json.dumps)
async def index_handler_bad(request):
    return web.json_response({
        'timestamp': datetime.datetime.now()
    }, dumps=json.dumps)


app = web.Application()
app['dumps'] = partial(json.dumps, default=default_json)
app.router.add_route('GET', '/', index_handler)
app.router.add_route('GET', '/bad', index_handler_bad)

if __name__ == '__main__':
    web.run_app(app)

Results:

$ http :8080/bad; http :8080
HTTP/1.1 500 Internal Server Error
CONNECTION: close
CONTENT-LENGTH: 170
CONTENT-TYPE: text/html; charset=utf-8
DATE: Wed, 11 May 2016 15:30:14 GMT
SERVER: Python/3.5 aiohttp/0.21.6

<html>
  <head>
    <title>500 Internal Server Error</title>
  </head>
  <body>
    <h1>500 Internal Server Error</h1>
    Server got itself in trouble
  </body>
</html>

HTTP/1.1 200 OK
CONTENT-LENGTH: 43
CONTENT-TYPE: application/json; charset=utf-8
DATE: Wed, 11 May 2016 15:30:14 GMT
SERVER: Python/3.5 aiohttp/0.21.6

{
    "timestamp": "2016-05-11 11:30:14.231798"
}

Your environment

$ python -c "import sys; print(sys.version)"
3.5.1 (default, Apr 19 2016, 21:44:31)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.29)]

question

Source

mrasband

👍1

All 6 comments

I'm happy to work on this if you have a direction you'd like that fits in your overall design (I know you are working on nested routing support)

mrasband on 11 May 2016

Hi @mrasband,
Please take a look at web.json_response function code, its quite simple (11 lines).
You are not forced to use aiohttp.web.json_response you can (and probably should)
make your own json_response shortcut with a dumps that suits your project
and use it project-wide.

popravich on 11 May 2016

That's what I currently do, I just like to keep the surface area down in my apps and it's nice to use the built in stuff with a small amount of config. No worries if you close the issue if you find it out of scope of the framework :)

mrasband on 11 May 2016

I am going to close this to clear some noise (hope that is ok) :) I think the "make your own" is a suitable answer and another option would be to just monkey patch json.dumps.

For now I just did the following in a util package:

import datetime
import functools
import json

from aiohttp import web


def default_json(obj):
    if isinstance(obj, datetime.datetime):
        return str(obj)
    raise TypeError('Unable to serialize {!r}'.format(obj))


json_dumps = functools.partial(json.dumps, default=default_json)
json_response = functools.partial(web.json_response, dumps=json_dumps)

mrasband on 20 May 2016

👍1

It might be interesting to add a parameter to the json_response() function for a JsonSerializer class override.

I couldn't make my handler serialize my object the way you suggested above:

class Properties(web.View):
    async def get(self):
        context = { 'measurement': 21, 'timestamp_utc': datetime.utcnow() }
        return json_response(context, dumps=json_serializer_helper.help_serialize_json)

with the following function:

def help_serialize_json:
        if isinstance(obj, datetime):
            return obj.isoformat()[:-3] + 'Z'
        else:
            raise TypeError('Unable to serialize {!r}'.format(obj))

The error I got in this way was a TypeError about the dict that is surrounding my datetime. So the serializer didn't even make it yet to the override for the datetime serialization.

But I could make the serialization work with this (based on https://stackoverflow.com/a/44631860/5433896):

class JsonSerializerHelper(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()[:-3] + 'Z'
        else:
            return super.encode(self, obj)

and by writing my own variation of json_response to be able to insert a class override:

class Properties(web.View):
    async def get(self):
        context = { 'measurement': 21, 'timestamp_utc': datetime.utcnow() }
        return Response(text=json.dumps(context, cls=JsonSerializerHelper),
                                  content_type='application/json',
                                  status=200)

Could you provide a parameter in json_response to enable overriding the jsonserializer class instead of its dump() function alone?
Or was there a way to do it with the function (help_serialize_json) alone and is it just me that didn't find that way then?

sanderzz91 on 22 Feb 2018

I believe you may be misusing the dumps argument, your implementation above will only serialize an datetime as a string (and fail for anything not a datetime, such as the context dict you are passing in).

I believe if you make it match my post above, it should work (e.g. use a partially applied function, functools.partial, to set that as the default on the json.dumps call).

import json
from functools import partial


def help_serialize_json(obj):
    """Your custom setup here"""


# note the usage of `default` here, which is made to be the kwarg for the built-in json.dumps
custom_dumps = partial(json.dumps, default=help_serialize_json)


class Properties(web.View):
    async def get(self):
        context = {}
        # this uses the full `json.dumps` that has the single kwarg defaulted to handle your custom types
        return json_response(context, dumps=custom_dumps)

Alternatively you can use the same idea to default the cls kwarg on json.dumps, if you would rather go that way (vs the default kwarg), and it would look quite similar.

mrasband on 22 Feb 2018

👍1

Was this page helpful?

0 / 5 - 0 ratings