Openapi-specification: $ref in response example relative to response example

[brennon]

I am using a $ref in a response example that, _when the response is returned_, is interpreted correctly as relative to the response itself:

responses:
  200:
    description: OK
    schema:
      "$ref": "#/definitions/CollectionOfFoos"
    examples:
      application/json:
        foos:
          - id: 00000000-0000-0000-0000-000000000000
            name: sample foo 0
            bar:
              id: 0
              "$ref": "#/refs/bars/0"
            baz:
              id: 42
              "$ref": "#/refs/bazzes/myBaz"
          - id: 00000000-0000-0000-0000-000000000001
            name: sample foo 1
            bar:
              "$ref": "#/refs/bars/0"
            baz:
              "$ref": "#/refs/bazzes/myOtherBaz"
        refs:
          bars:
            0:
              detail: "Additional detail about this bar..."
          bazzes:
            myBaz:
              detail: "Additional detail about this baz..."
            myOtherBaz:
              detail: "Additional detail about this other baz..."

So, were that example response to be sent to a consumer, the pointer in those $refs would point to the refs member on the response. This is a contrived example, but we do this for "normalization" purposes, so that more than one foo can refer to the same bar or baz.

The problem is that this doesn't comply with the spec, since those $ref elements that will later resolve correctly don't do so when the API including examples is checked for compliance. That first $ref ("$ref": "#/definitions/CollectionOfFoos") is perfectly compliant, since in the API doc there is something at definitions/CollectionOfFoos, but there isn't anything at refs/bazzes/myBaz, etc., so those lines throw an error with the description $refs must reference a valid location in the document. Is there any way around this, or a better way to approach this problem?

[webron]

This sounds like a tooling issue. Whichever tool you're using to run the validation, should not validate those references.

[brennon]

I'm using SwaggerHub to validate my schema. Pardon my ignorance, but is it abnormal to use a $ref in this way?

[brennon]

Additionally, I've tried several other validators now, and they _all_ complain about these usages of $ref. This is what leads me to believe I'm just using it incorrectly. That, or it's a "tooling issue" with every tool I try...

[nogates]

Hi Brennon!

Yes, I don't think $ref is meant to work as how you are trying to use it, since mostly it is used to reference to complements or definitions on the document and your intention is to reference siblings, which is not supported (https://swagger.io/docs/specification/using-ref/#sibling, or at least it wasn't in swagger and I think the same can be applied to openapi)

However, can't you use components, which allows you to define reusable examples? https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#componentsObject and then you can reference those using #/components/Examples/<example name> ?

[handrews]

@brennon OAS 3 referenced the definition of $ref from the standalone JSON Reference draft spec, rather than the definition in the JSON Schema defat-wright-json-schema-01 that they used for the Schema Object.

This made sense because OAS3 uses $ref outside of JSON Schema.

However, the problem with the standalone JSON Reference spec is that it has no concept of where a reference is valid. Therefore, if you use that spec, any occurrence of $ref anywhere in the document is treated as a reference and dereferenced. Even (such as in this case) when it is in an example of a response document.

This is why JSON Schema pulled the definition of $ref back into the JSON Schema spec.

The OAS3 spec tries to do something similar to what we did for JSON Schema by specifying where it is valid to use a $ref (or really, an OAS Reference Object) and where it is not. However, as best I can tell they do not explicitly state the behavior when $ref appears somewhere that an OAS Reference Object is not valid.

So it looks like many tools decided to apply the JSON Reference spec's approach of treating every $ref as a reference regardless of whether it makes sense or not.

@webron it would probably be good to clean this up in 3.1. Simply saying that all occurrences of $ref where a Reference Object is not valid MUST be ignored (or something similar) should suffice.

[brennon]

Thanks for the replies. I'm gathering from your response, @handrews, that while my usage of $ref isn't explicitly in conflict with the spec, because of the way references are treated, I'm going to get errors until something changes.

As a possible workaround, is there a plan to support relative JSON pointers supported in a future OAS version?

[handrews]

@brennon while I can't speak for the OAS TSC, as a JSON Schema spec editor and fan of Relative JSON Pointers, I can note that they are not compatible with how $ref is defined. $ref takes a URI reference, and while a complete fragment can be a URI reference (e.g. #/foo/bar), a partial fragment cannot. Relative JSON Pointers _cannot_ be used in URI fragments for this reason (it's covered in the draft spec for them). There would need to be a different keyword that specifically accepts them.

In JSON Schema, the solution is that there are two forms of fragment: JSON Pointer fragments, and plain name fragments. You can define a plain name fragment anywhere (using $id, although that might change to $anchor in this next draft for excessively complex reasons not worth discussing here), which makes it a reasonably effective replacement for relative pointers in that it lets you reference a schema without knowing its location within the document. Not _quite_ the same thing as a relative location, but it solves pretty much all use cases we've encountered.

OAS forbids $id (which is one of several reasons we may split $id into two keywords- $id and $anchor), so that doesn't work in OAS documents.

[webron]

Please refer to the documentation about example values (stress added for clarification):

Embedded literal example. The value field and externalValue field are mutually exclusive. To represent examples of media types that cannot naturally represented in JSON or YAML, use a string value to contain the example, escaping where necessary.

What we meant here (and if needed, can clarify further) is that the value given here is the actual value of the example. It is not to be further parsed by tooling. So documentation or mocking services should not try to parse the $ref's before further 'using' them.

The reason behind is exactly $ref's. We can't distinguish when the end user would want to use $ref as a resolved reference and when they'd want to use it as a literal value (which is a very valid case). As such, we decided that the examples would be non-parsed literal values. At the moment, there's no way to have parsable examples, as external examples are literal values as well.

[webron]

@brennon Forgot to mention, since I'm related to the SwaggerHub team, I can tell you that as far as I know, the fact that you get resolution errors is being worked on so that references within examples will not be resolved.

[handrews]

@webron if OAS 3.1 is going to continue to refer to the long-expired JSON Reference draft, I think there needs to be a clarification next to where that draft is cited that it only applies where OAS explicitly allows a Reference Object.

I'd actually recommend not referencing such an old spec (although I suppose OAI could re-submit it), but that's probably a discussion for a TSC call.

[webron]

@handrews sure, I understand the issue(s) with using the expired draft. Just regardless, the spec is explicit about the value of examples so there should be no confusion there.

[handrews]

@webron LOL "should" 🤣

(I say this in sympathy as a fellow spec-maintainer)

[webron]

Maybe should have used SHOULD :wink:

[handrews]

I think this has been answered. If there are unresolved side topics that got brought up along the way, please file them separately.