Dredd: Test request's body is empty in the case of a multipart/form-data request with Swagger 2.0 Document

[swang-pro]

Describe your problem

Hi,

I try to use Dredd to tests our API's spec written in Swagger 2.0 format. The test generated by Dredd to test a multipart/form-data request fails to generated the body of the request. And the multipart/form-data definition is interpreted as "application/x-www-form-urlencoded". There are warnings who seem to say the the multipart is not well supported. Is it the case? Is there any work-around to this problem ?

warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Only the first scheme will be used to create a hostname on line 12
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17

request: 
method: POST
uri: /firstMethod
headers: 
    Accept: application/json
    Content-Type: application/x-www-form-urlencoded
    User-Agent: Dredd/2.2.5 (Linux 3.19.0-79-generic; x64)

body: 



expected: 
headers: 
    Content-Type: application/json

body: 
[
  {
    "success": true,
    "id": 123456
  }
]
statusCode: 200

Thank you

What command line options do you use?

$ docker run -v /home/user/Projects/swagger:/tmp/swagger 6d5c665ea96c dredd /tmp/swagger/api.yaml http://10.107.7.198:8081/

What is in your dredd.yml?

swagger: '2.0'
info:
  title: My API
  description: blabalbal
  version: "1.0.0"
host: my.api.com
basePath: /
tags:
  -
    name: "First Function"
    description: "Blablabla"
schemes:
  - https
  - http
paths:
  /firstMethod:
    post:
      summary: blablabla
      description: |
        blablablabla.
      consumes:
        - multipart/form-data
      produces:
        - application/json
      parameters:
        - name: param1
          in: formData
          description: blabla param1
          type: number
          format: int64
          required: true
        - name: param2
          in: formData
          description: blabla param2
          type: number
          format: int64
          required: true
        - name: param2
          in: formData
          description: blabla param2
          type: string
          required: true
        - name: param3
          in: formData
          description: blabla param3.
          type: file
          required: true
        - name: param4
          in: formData
          description: blabla param4.
          type: string
          format: string
          required: false
        - name: param5
          in: formData
          description: blabla param5.
          type: number
          format: int64
          required: false
        - name: param6
          in: formData
          description: blabla param6
          type: string
          required: false
        - name: param7
          in: formData
          description: blabla param7
          type: string
          required: false
      responses:
        200:
          description: Successful response
          # Use JSON Schema format
          schema:
            $ref: '#/definitions/ResultSuccess'
          examples:
            application/json:
              -
                success: true
                id: 123456
        400:
          description: |
            Bad Request response

            4001    INCORRECT PARAMETERS

            4101    TYPE ERROR 1

            4102    TYPE ERROR 2

            4103    TYPE ERROR 3

            4201    TYPE ERROR 4

            4202    TYPE ERROR 5

            4203    TYPE ERROR 6
          schema:
            $ref: '#/definitions/ResultError'
          examples:
            application/json:
              -
                success: false
                code: "4001"
                error: "INCORRECT PARAMETERS"
        500:
          description: |
            Internal Server Error response

            5000    INTERNAL ERROR
          schema:
            $ref: '#/definitions/ResultError'
          examples:
            application/json:
              -
                success: false
                code: "5000"
                error: "INTERNAL ERROR"
definitions:
  ResultSuccess:
    type: object
    properties:
      success:
        type: boolean
        description: If transaction created succesfully.
      id:
        type: number
        description: Unique identifier.
  ResultError:
    type: object
    properties:
      success:
        type: boolean
        description: If transaction created succesfully.
      code:
        type: string
        description: Code of the errors
      error:
        type: string
        description: Error message

What's your dredd --version output?

dredd v2.2.5 (Linux 3.19.0-79-generic; x64)

Does dredd --level=debug uncover something?

Nothing more interessting uncoverred.
You have the full trace below:

verbose: Loading configuration file: ./dredd.yml
debug: Dredd version: 2.2.5
debug: Node.js version: v4.4.7
debug: Node.js environment: http_parser=2.5.2, node=4.4.7, v8=4.5.103.36, uv=1.8.0, zlib=1.2.8, ares=1.10.1-DEV, modules=46, openssl=1.0.2h
debug: System version: Linux 3.19.0-79-generic x64
debug: npm version: 2.15.9
debug: Configuration: {"server":"http://10.107.7.198:8081/","options":{"_":["/tmp/swagger/api.yaml"],"level":"debug","l":"debug","dry-run":null,"y":null,"hookfiles":null,"f":null,"language":"nodejs","a":"nodejs","sandbox":false,"b":false,"server":null,"g":null,"server-wait":3,"init":false,"i":false,"custom":{},"j":[],"names":false,"n":false,"only":[],"x":[],"reporter":[],"r":[],"output":[],"o":[],"header":[],"h":[],"sorted":false,"s":false,"user":null,"u":null,"inline-errors":false,"e":false,"details":false,"d":false,"method":[],"m":[],"color":true,"c":true,"timestamp":false,"t":false,"silent":false,"q":false,"path":["/tmp/swagger/api.yaml","/tmp/swagger/api.yaml"],"p":["/tmp/swagger/api.yaml","/tmp/swagger/api.yaml"],"hooks-worker-timeout":5000,"hooks-worker-connect-timeout":1500,"hooks-worker-connect-retry":500,"hooks-worker-after-connect-wait":100,"hooks-worker-term-timeout":5000,"hooks-worker-term-retry":500,"hooks-worker-handler-host":"localhost","hooks-worker-handler-port":61321,"config":"./dredd.yml","$0":"/usr/bin/node /node_modules/.bin/dredd"},"custom":{"cwd":"/","argv":["/tmp/swagger/api.yaml","http://10.107.7.198:8081/","--level=debug"]}}
verbose: Using 'base' reporter.
verbose: Configuring reporters: []
verbose: Using 'cli' reporter.
verbose: No backend server process specified, starting testing at once.
verbose: Running Dredd instance.
verbose: Expanding glob patterns.
verbose: Reading API description files.
verbose: Parsing API description files and compiling a list of HTTP transactions to test.
verbose: Compiling HTTP transactions from API description file: /tmp/swagger/api.yaml
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Only the first scheme will be used to create a hostname on line 12
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Files in form parameters are not fully supported yet on line 43
warn: Parser warning in file '/tmp/swagger/api.yaml': (warning code 3) Unable to generate application/x-www-form-urlencoded example message body out of JSON Schema on line 17
verbose: Starting reporters and waiting until all of them are ready.
info: Beginning Dredd testing...
verbose: Starting transaction runner.
verbose: Sorting HTTP transactions
verbose: Configuring HTTP transactions
verbose: Reading hook files and registering hooks
verbose: Executing HTTP transactions
verbose: Running 'beforeAll' hooks
debug: Running hooks...
verbose: Processing transaction #1: /firstMethod > balbalbal > 200 > application/json
verbose: Running 'beforeEach' hooks
debug: Running hooks...
verbose: Running 'before' hooks
debug: Emitting to reporters: test start
verbose: About to perform an HTTP request to tested server
verbose: Handling HTTP response from tested server
debug: Response from tested server was recieved
verbose: Running 'beforeEachValidation' hooks
debug: Running hooks...
verbose: Running 'beforeValidation' hooks
verbose: Validating HTTP transaction by Gavel.js
debug: Determining whether HTTP transaction is valid (getting boolean verdict)
debug: Validating HTTP transaction (getting verbose validation result)
verbose: Running 'afterEach' hooks
debug: Running hooks...
verbose: Running 'after' hooks
debug: Evaluating results of transaction execution #1: /firstMethod > balbalbal > 200 > application/json
debug: Emitting to reporters: test fail
fail: POST /firstMethod duration: 118ms
verbose: Processing transaction #2: /firstMethod > balbalbal > 400 > application/json
verbose: Running 'beforeEach' hooks
debug: Running hooks...
verbose: Running 'before' hooks
debug: Emitting to reporters: test start
verbose: HTTP transaction was marked in hooks as to be skipped. Skipping
verbose: Running 'afterEach' hooks
debug: Running hooks...
verbose: Running 'after' hooks
debug: Evaluating results of transaction execution #2: /firstMethod > balbalbal > 400 > application/json
debug: Emitting to reporters: test skip
skip: POST /firstMethod
verbose: Processing transaction #3: /firstMethod > balbalbal > 500 > application/json
verbose: Running 'beforeEach' hooks
debug: Running hooks...
verbose: Running 'before' hooks
debug: Emitting to reporters: test start
verbose: HTTP transaction was marked in hooks as to be skipped. Skipping
verbose: Running 'afterEach' hooks
debug: Running hooks...
verbose: Running 'after' hooks
debug: Evaluating results of transaction execution #3: /firstMethod > balbalbal > 500 > application/json
debug: Emitting to reporters: test skip
skip: POST /firstMethod
verbose: Running 'afterAll' hooks
debug: Running hooks...
verbose: Wrapping up testing.
info: Displaying failed tests...
fail: POST /firstMethod duration: 118ms
fail: headers: Header 'content-type' has value 'text/plain' instead of 'application/json'
statusCode: Status code is not '200'

request: 
method: POST
uri: /firstMethod
headers: 
    Accept: application/json
    Content-Type: application/x-www-form-urlencoded
    User-Agent: Dredd/2.2.5 (Linux 3.19.0-79-generic; x64)

body: 



expected: 
headers: 
    Content-Type: application/json

body: 
[
  {
    "success": true,
    "id": 123456
  }
]
statusCode: 200
bodySchema: {"type":"object","properties":{"success":{"type":"boolean","description":"If transaction created succesfully."},"id":{"type":"number","description":"Unique identifier."}}}


actual: 
statusCode: 400
headers: 
    access-control-allow-origin: *
    content-type: text/plain
    transfer-encoding: chunked

body: 
{"code":"4001","error":"INCORRECT PARAMETERS","success":false}



complete: 0 passing, 1 failing, 0 errors, 2 skipped, 3 total
complete: Tests took 128ms
verbose: Dredd instance run finished.
verbose: Exiting Dredd process with status '1'.
debug: Using configured custom exit() method to terminate the Dredd process.
verbose: Gracefully terminating backend server process.
verbose: No backend server process.

Can you send us failing test in a Pull Request?

Is it necessary in my case? And how ?

[honzajavorek]

@swang-pro, thank you for such a great report! I think multipart/form-data isn't correctly supported as of now in the fury-adapter-swagger, which transforms Swagger to API Elements (consumed by Dredd): https://github.com/apiaryio/fury-adapter-swagger/issues/96 You can help by commenting under the linked issue or contributing your example with expected results to the swagger-zoo, which is where the test cases for fury-adapter-swagger live.

[MikeRalphson]

@honzajavorek Looks like this may have been fixed in fury-adapter-swagger v0.14.0 https://github.com/apiaryio/fury-adapter-swagger/commit/b46e10fc4d6c80a41e3754ce59549b5051f99dee - would be nice to update the dep.

[kylef]

That's correct, the PR at https://github.com/apiaryio/dredd-transactions/pull/108 should update Dredd exposing this fix. Will need some testing in Dredd to see how Dredd handles multipart form.

[MikeRalphson]

Thanks @kylef yes I patched in the new version and I'm seeing some issues with Unexpected end of multipart data, possibly a missing boundary. Not expecting me messing with deps to be supported!

[kylef]

@MikeRalphson The multi-part form generation is handled at https://github.com/apiaryio/fury-adapter-swagger/blob/master/src/generator.js#L26-L36 (tests: https://github.com/apiaryio/fury-adapter-swagger/blob/master/test/generator.js#L59-L75).

I think you are right, there should be an additional two -- at the end of the boundary identifier for the LAST part. I will try and prepare an additional patch shortly.

[honzajavorek]

This is going to be solved with the next upgrade of Dredd Transactions in Dredd. There are some changes we want to add before upgrading, so although in Dredd Transactions this is fixed, it still may take some time before it gets to Dredd.

[honzajavorek]

@MikeRalphson @swang-pro This should be now fixed in v4.7.1. Would you mind to confirm it's okay now?

[MikeRalphson]

@honzajavorek Looks good to me, my number of passing tests has increased. Thanks.