Dredd: Support content-types with charset in Swagger's "produces"

[castarco]

Even if I explicitly tell in the swagger specification that the API produces something like
application/json; charset=utf-8, when I run dredd and my API returns a response with this content-type, Dredd tells me that it was only expecting something like application/json and the test breaks.

A swagger.yml example:

swagger: '2.0'

info:
  title: 'Example API'
  description: "Example API"
  version: '1'

host: 'api.inbenta.com'

schemes:
  - "https"

consumes:
  - "application/json; charset=utf-8"
produces:
  - "application/json; charset=utf-8"

paths:
  /validate:
    x-summary: "Verification endpoint"
    x-description: "Verifies that a token represents an authorization to apply a concrete action on a given a resource."
    get:
      parameters:
        - name: requestBody
          in: body
          required: true
          schema:
            $ref: "#/definitions/TokenAuthorizationRequest"
      responses:
        "200":
          description: "Successfully Authorized"
          schema:
            $ref: "#/definitions/SuccessResponse"
          examples:
            application/json:
              authenticated: true
              authorized: true
        "400":
          description: "Invalid request"
          schema:
            $ref: "#/definitions/GenericErrorResponse"
          examples:
            application/json:
              authenticated: true
              authorized: false
              errorMsg: "Missing request field"
        "401":
          description: "Not authenticated"
          schema:
            $ref: "#/definitions/GenericErrorResponse"
          examples:
            application/json:
              authenticated: false
              authorized: false
              errorMsg: "Invalid Token"
        "403":
          description: "Not authorized"
          schema:
            $ref: "#/definitions/GenericErrorResponse"
          examples:
            application/json:
              authenticated: true
              authorized: false
              errorMsg: "The token is not authorized to access the requested resource"
        "404":
          description: "Not found"
          schema:
            $ref: "#/definitions/GenericErrorResponse"
          examples:
            application/json:
              authenticated: true
              authorized: false
              errorMsg: "The specified resource can't be found"
        "500":
          description: "Internal server error"
          schema:
            $ref: "#/definitions/InternalServerErrorResponse"

definitions:

  TokenAuthorizationRequest:
    description: "An object to validate that a token authorizes the usage of a particular resource."
    required:
      - token
      - rcUri
      - action
    properties:
      token:
        type: string
        description: "OAuth2.0 access token"
      rcUri:
        type: string
        pattern: "^(/[A-Za-z0-9_]{1,128}){1,5}$"
        description: "REST path, identifies a resource"
      action:
        type: string
        enum: ["HEAD", "GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"]
        description: "HTTP verb, identifies an action over a resource"

  SuccessResponse:
    type: object
    required:
      - authenticated
      - authorized
    properties:
      authenticated:
        type: boolean
      authorized:
        type: boolean

  GenericErrorResponse:
    type: object
    required:
      - authenticated
      - authorized
      - errorMsg
    properties:
      authenticated:
        type: boolean
      authorized:
        type: boolean
      errorMsg:
        type: string

  InternalServerErrorResponse:
    type: object
    required:
      - errorMsg
    properties:
      errorMsg:
        type: string

And here the Dredd result:

info: Configuration './dredd.yml' found, ignoring other arguments.
info: Starting server with command: /opt/cerberus_tests/runTestServer.sh
info: Waiting 3 seconds for server command to start...
2016-07-05T09:52:58.171Z - info: [CerberusServer] Running on http://localhost:9090
info: Beginning Dredd testing...
fail: GET /validate duration: 73ms
skip: GET /validate
skip: GET /validate
skip: GET /validate
skip: GET /validate
skip: GET /validate
info: Displaying failed tests...
fail: GET /validate duration: 73ms
fail: headers: Header 'content-type' has value 'application/json; charset=utf-8' instead of 'application/json'

request: 
method: GET
uri: /validate
headers: 
    User-Agent: Dredd/1.2.0 (Linux 4.4.0-28-generic; x64)

body: 



expected: 
headers: 
    Content-Type: application/json

body: 
{
  "authenticated": true,
  "authorized": true
}
statusCode: 200
bodySchema: {"type":"object","required":["authenticated","authorized"],"properties":{"authenticated":{"type":"boolean"},"authorized":{"type":"boolean"}}}


actual: 
statusCode: 200
headers: 
    x-powered-by: Express
    content-type: application/json; charset=utf-8
    content-length: 40
    etag: W/"28-BlBhACnHg2NvLfij4Z5t+Q"
    date: Tue, 05 Jul 2016 09:53:01 GMT
    connection: close

body: 
{"authenticated":true,"authorized":true}



info: 0 passing, 1 failing, 0 errors, 5 skipped, 6 total
info: Tests took 80ms
info: Sending SIGTERM to the backend server
info: Sending SIGTERM to the backend server
info: Killing backend server

[netmilk]

Hey @castarco! I ran into the same issue and I had to manually add a content-type header with the charset to all requests and responses and leave the produces and consumes without it as a workaround.

Thank you for reporting it, this limitation is something that we'd like to overcome very soon.

[castarco]

Hi @netmilk , I know this is a question about swagger, but how are you expressing that a particular endpoint returns a specific content-type header?

Thx!

[netmilk]

@castarco Have a look at the example below. The operation header parameter and the response header are overriding the produces and consumes.

swagger: "2.0"
info:
  version: 1.0.0
  title: Life Saving API
schemes:
  - http
consumes:
  - application/json
produces:
  - application/json
paths:
  /112:
    post:
      parameters:

        - in: header
          name: content-type
          type: string
          required: true
          default: application/json; charset=utf-8

        - in: body
          name: body
          schema:
            type: object
            required: ['emergency']
            properties:
              emergency:
                type: boolean
                default: true

      responses:
        201:
          description: "Successful response"
          headers:
            Content-Type:   
              type: string
              default: application/json; charset=utf-8

          schema:
            type: object
            required: ['message']
            properties:
              message:
                type: string
                default: You will live!

[rmharrison]

@netmilk: Any update in terms of a fix?

Still experiencing an identical bug. Working around by including explicit Content-type in every API end-point is quite cumbersome.

Even a minimum, "point in the right direction" would be useful, e.g. where folks should start looking if they wanted to PR?

[honzajavorek]

@rmharrison AFAIK there are two reasons why this happens:

• Until now, the Swagger adapter was skipping any other content type than application/json (exactly). That is something @kylef and his team are working on right now. I believe the outcome should allow any content type in the future, not only application/json; charset... or application/hal+json etc. IMHO for JSON-based content types, the adapter should provide generated examples, but at this moment I don't know exactly how the final mechanics are going to work.
• Gavel.js, the library Dredd uses for validating the HTTP response, validates the content type without any sophisticated logic. If the content type string isn't equal to what's expected, it is evaluated as not valid. I think that is correct - if you send charset, you should state it in the API description as well.

[rmharrison]

Doh. So, the take home is wait for the swagger adapter folks?

Had a look at the (thankfully small) issue backlog, and I don't see one that's obviously relevant. Which issue should I append to?

Move to that issue

Looks like the problem is somewhere between the type-input, which should detect the document wide swagger.consumes/swagger.produces: https://github.com/apiaryio/fury-adapter-swagger/blob/62a86ed319164de54678b6371120792fa0ad0e26/src/parser.js#L806

And where the headers are actually appended (just the JSON it seems): https://github.com/apiaryio/fury-adapter-swagger/blob/62a86ed319164de54678b6371120792fa0ad0e26/src/parser.js#L1033

[honzajavorek]

I'm sorry for not responding promptly (vacations). I'll check the situation around the adapter and whether it's current behavior in this regard is final or whether it's awaiting changes - related to https://github.com/apiaryio/dredd/issues/897#issuecomment-341432814

[honzajavorek]

Related: apiaryio/fury-adapter-swagger#143

[honzajavorek]

@castarco @rmharrison This should be now fixed in v4.7.1. Would you mind to confirm it's okay now?

The new behavior is described in docs: https://dredd.readthedocs.io/en/latest/how-it-works.html#choosing-http-transactions

[IRuslan]

@honzajavorek for me the issue is still relevant.

My server returns Content-Type: application/json; charset=utf-8 header.
In API docs I have (tried both on the top and response level):

produces:
  - application/json; charset=utf-8

After running a dredd test, the result doesn't match, and the expected header value is:

headers: 
Content-Type: application/json

This workaround works to me on the response definition level:

headers:
  Content-Type:   
    type: string
    default: application/json; charset=utf-8