Dredd: Dredd and Apiary URI Template syntax are not compatible

[erajkovic]

Describe your problem

error: Compilation error in file 'https://apicatalog.oraclecloud.com/public/v1/orgs/oracle-public/apicollections/bics/1.0/apis/dataset-load/canonical': Failed to parse URI template: /dataload/v1/datasets/{set name}
Error: SyntaxError: Expected "(", "*", ",", ":", "}" or [a-zA-Z0-9_.%] but " " found. ( > /dataload/v1/datasets/{set name} > Create a data set)

As I was able to upload the same file to apiary, I am assuming it's a too restrictive check from dredd - see https://app.apiary.io/dreddissue/editor

What command line options do you use?

$dredd https://apicatalog.oraclecloud.com/public/v1/orgs/oracle-public/apicollections/bics/1.0/apis/dataset-load/canonical http://localhost:8080 --dry-run

What is in your dredd.yml?

not using one.

What's your dredd --version output?

dredd v5.1.6 (Darwin 17.4.0; x64)

Does dredd --level=debug uncover something?

no extra info

Can you send us failing test in a Pull Request?

[honzajavorek]

The URI Template syntax limitations are specified by the RFC6570. I agree though that Apiary and Dredd should be consistent in what URI Templates they consider as valid or erroneous. This needs verification.

[kylef]

I don't think the limitation here is RFC6570, it's likely a bug in either fury-adapter-swagger, dredd and Apiary Console, or a combination of all. Spaces should be escaped in the property name (space becomes %20).

---

It seems that there is more than one issue. I found a related problem(s) while reproducing with a query parameter instead of path parameter. Original issue is regarding spaces in a path parameter, but can also be produced with query. I believe the problem is generic and regarding all sorts of escaping of parameter key names in URI Template.

Example Swagger to Reproduce

swagger: '2.0'
info:
  version: '1.0'
  title: httpbin
host: httpbin.org
basePath: /
schemes:
- https
produces:
- application/json
paths:
  /get:
    get:
      summary: Get
      parameters:
        - name: hello world
          in: query
          type: string
          x-example: lol
      responses:
        200:
          description: Successful Response
          schema:
            type: object
            properties:
              args:
                type: object
                properties:
                  'hello world':
                    type: string
                required:
                  - hello world

Dredd

$ dredd thing.yaml https://httpbin.org
warn: Compilation warning in file 'thing.yaml': Ambiguous URI parameter in template: /get{?hello%20world}
Parameter not defined in API description document: hello%20world ( > /get > Get)
info: Beginning Dredd testing...
complete: Tests took 1ms

Apiary Console

Fury Swagger Parse Result

• href: /get{?hello%20world}
• hrefVariables:
  
  
  
  • key: hello world
    
    value: lol
  
  

Here, the URI Template seems to be correctly escaped. However hrefVariables has the key un-escaped, which I think I expect (so attributes table is rendered with space as user written). However, should it be escaped?

Perhaps we do a third variant in the element where we provide the key name escaped and the original string as the title?

{
  "element": "member",
  "content": {
    "key": {
      "element": "string",
      "attributes": {
        "title": { "element": "string", "content": "hello world" }
      },
      "content": "hello%20world"
    },
    "value": {
      "element": "string",
      "content": "lol"
    }
  }
}

Just to jot some of my incomplete thoughts quickly so I don't forget next time I look at this issue:

• I could also see it as it may be expecting that tooling handles escaping of the keys (although that is really annoying as consumer), it is kinda an implementation detail of URI Template regarding escaping. When a parameter is re-used in other sections (such as JSON or if URL format was changed) it would be broken. If escaping is a rule of where the key is used, then the escaping should be applied when parameter is applied to a URI Template. So that if/when the parameter key is applied to other formats it is down to tooling to encode it as necessary.
• There can also be problems if parser or tooling does escaping, because user could have escaped it and then there is double escaping
  
  
  
  • Could we restrict parser to require user to escape URI parameter names? Warn when not and escape for user but expect it is escaped. How do other Swagger tooling handle this?
  
  
  • How does API Blueprint parser handle this? I think it has similar problems.
  
  

[honzajavorek]

@kylef Thanks for the investigation! At first sight, it seems like apiaryio/uritemplate-js or whatever is used, should take care of it (cc @char0n). However, your second thought mentions double encoding and that could be a real problem... 🤔

[erajkovic]

So is "/{set name}" valid in the swagger or should it have been "/{set%20name}" - As I can load the file to the swagger editor, I am assuming it's valid.

[honzajavorek]

I think that's a tricky question, because "/{set name}" may not be a valid URI Template according to the RFC, but could be treated as valid by the Swagger tooling, thus being a valid part of a Swagger document.

I think the parser should not escape the parameters. In fact, I think it should unescape them in case user provides them escaped (if that's even deterministic task). Then the URI Template parsing library should unescape the parameters in the template. In that case, the result is Dredd or any other tooling ends up comparing unescaped values, which is IMHO correct, because the escaping is just an implementation detail of the URI or URI Template spec, while parameters are something going beyond the URI space, especially in Swagger.

[honzajavorek]

I am afraid this needs more investigation and is generally a part of the whole URI parameters epic.