Connexion: validate_responses fails when returning a ResponseContainer

[vimalloc]

(Sorry in advance if this has already been reported. I've been doing a terrible job of actually finding duplicate issues lately it seems)

Description

validate_response=True causes an exception to be raised when returning a connexion ResponseContainer object. For my case, I'm migrating an older API over to swagger, and I need to be able to return a flask Response object directly, so that I can use it to modify cookies, hence the returning a ResponseContainer instead of a dictionary.

Steps to reproduce

test.yaml

swagger: "2.0"

info:
  title: "Test"
  version: "1.0"

basePath: /v1.0

produces:
  - application/json

consumes:
  - application/json

paths:
  /foo:
    get:
      operationId: test.foo
      responses:
        200:
          description: A dns zones
          schema:
            type: object
            required:
              - foo
            properties:
              foo:
                type: string
                example: 'foo'

test.py

#!/usr/bin/env python3
import connexion
from connexion.decorators.decorator import ResponseContainer
from flask import jsonify

app = connexion.App(__name__, 9091)

def foo():
    resp =  jsonify({'foo': 'bar'})
    return ResponseContainer(mimetype=resp.mimetype, response=resp, status_code=200)

if __name__ == '__main__':
    app.add_api('test.yaml', validate_responses=True)
    app.run()

If you curl http://0.0.0.0:9091/v1.0/foo, you get the following stacktrace:

Traceback (most recent call last):
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/connexion/decorators/decorator.py", line 118, in wrapper
    response_container = function(*args, **kwargs)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/connexion/decorators/produces.py", line 100, in wrapper
    response = function(*args, **kwargs)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/connexion/decorators/response.py", line 92, in wrapper
    self.validate_response(response.get_data(), response.status_code, response.headers)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/connexion/decorators/response.py", line 47, in validate_response
    data = json.dumps(data)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/json.py", line 123, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/usr/lib64/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib64/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/connexion/decorators/produces.py", line 33, in default
    return json.JSONEncoder.default(self, o)
  File "/home/lgbland/code/connexion/venv/lib/python3.6/site-packages/flask/json.py", line 80, in default
    return _json.JSONEncoder.default(self, o)
  File "/usr/lib64/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'bytes' is not JSON serializable
127.0.0.1 - - [23/Mar/2017 14:17:30] "GET /v1.0/foo HTTP/1.1" 500 -

If I remove validate_responses=True, then this returns the JSON as expected.

Additional info:

• python version 3.6
• connexion version 1.1.5

[vimalloc]

It looks like the data stored in the ResponseContainer is:

b'{\n  "foo": "bar"\n}\n'

Due to response.get_data() returning already encoded data at decorator.py (on line 153).

It gets to this block of code in responses.py (lines 45 to 48) and chokes because the data was already dumped:

                # For cases of custom encoders, we need to encode and decode to
                # transform to the actual types that are going to be returned.
                data = json.dumps(data)
                data = json.loads(data)

If I update decorator.py as follows then I start getting closer. It no longer raises an exception, but it now returns the string 'foo' instead of the json {"foo": "bar"} (and I imagine if this would cause problems if someone was using a different format instead of JSON, such as XML, but based on the comment in the above block of code I think it currently wouldn't be supported anyways).

from flask import json
...
            self.data = json.loads(self._response.get_data())

[vimalloc]

It looks like that last error (returning 'foo' instead of the JSON) has to do with the rebuilding of the flask response object in decorator.py, and the fact that self.data is now a dict instead of a bytes object.

    def flask_response_object(self):
        """
        Builds an Flask response using the contained data,
        status_code, and headers.

        :rtype: flask.Response
        """
        self._response = flask.current_app.response_class(
            self.data, mimetype=self.mimetype, content_type=self.headers.get('content-type'),
            headers=self.headers)  # type: flask.Response
        self._response.status_code = self.status_code

        return self._response

One way to fix it would be to re-dump it here:

 self._response = flask.current_app.response_class(
            json.dumps(self.data), mimetype=self.mimetype, content_type=self.headers.get('content-type'),
            headers=self.headers)  # type: flask.Response

Another solution could be to just return the original response if it was given

    def flask_response_object(self):
        """
        Builds an Flask response using the contained data,
        status_code, and headers.

        :rtype: flask.Response
        """
        if not self._response:
            self._response = flask.current_app.response_class(
                self.data, mimetype=self.mimetype, content_type=self.headers.get('content-type'),
                headers=self.headers)  # type: flask.Response
            self._response.status_code = self.status_code

        return self._response

I wouldn't be surprised if these solutions introduced bugs or edge cases into the system, but I couldn't say for sure. If the general idea behind them looks alright, I would be happy to solidify these, make some tests, and submit a pull request. If there would be a better way to solve this problem then just disregard me :)

Cheers.

[vimalloc]

Fixed in 1.1.6