OpenAPI-Specification 🚀 - Make schema object compatible with JSON Schema draft4

Hi @IvanGoncharov Looks like we've been having similar issues. I'm having trouble getting documents that lack properties that are not actually required to be correctly validated. The Ruby JSON::Validator.validate_fully method takes a schema and a document and validates that the document conforms to the schema. That worked when we were using additionalProperties to define optional properties (until we realised that that's not what additionalProperties is for.) Also the Swagger Codegen ignores any property defined under additionalProperties which is either just a current limitation or a bug. see http://stackoverflow.com/questions/29714534/swagger-json-optional-fields-being-treated-as-required

davesag on 19 Apr 2015

The existing JSON Schema validators out there should have no problem validating Swagger definitions as-is. Using a subset of properties doesn't take away from that functionality.

There are explicit reasons for most excluded keywords as to why they were excluded, though some excluded keywords may have been excluded by mistake. We will not be able to provide support for most these keywords without finding proper solutions for the problems they may cause.

JSON Schema is extensible to the user's needs. We chose to extend it with our own keywords to better support the use cases that we require. There's no technical problem there. If there's a problem with a specific validator, it's a bug with the validator. By definition, the validator should ignore any additional keyword that's use that it is not aware of.

The discriminator absolutely cannot be replaced by oneOf as it is required for proper polymorphism support. oneOf will not guarantee that and could break support for strongly-typed languages.

You are partially right about the readOnly, however it significantly reduces the complexity of the schema. If anything, we may extend readOnly in future versions.

Given all the above, it does not mean we will not provide support for whichever JSON Schema version is available when we work on the next Swagger specification version. We would have to be sure that we can provide end-to-end (for producers and consumers) which we couldn't do in the last version of the spec.

webron on 19 Apr 2015

The existing JSON Schema validators out there should have no problem validating Swagger definitions as-is.

I'm not speaking about validating Swagger definitions instead my concern is about request/reply body validation.

There are explicit reasons for most excluded keywords as to why they were excluded

Can you please post link or short description, because it really affects further discussion.
It especially interesting, because competitive standards(API Blueprint, RAML) fully support JSON Schema without any problems.

JSON Schema is extensible to the user's needs. We chose to extend it with our own keywords to better support the use cases that we require. There's no technical problem there. If there's a problem with a specific validator, it's a bug with the validator. By definition, the validator should ignore any additional keyword that's use that it is not aware of.

If we speak only about JSON Schema standard than your right, validator could simply ignore non-standard keywords.

From Swagger point of view situation is different, lets consider the following Pet model as an example. If you simply ignore discriminator keyword than you will validate all objects only against Pet schema.

The discriminator absolutely cannot be replaced by oneOf as it is required for proper polymorphism support.

discriminator implement so-called ad hoc polymorphism, that is why it can be replaced with oneOf.
For example, I removed discriminator from Pet model example.
Same aproach used in Swagger schema to distinguish between different parameter types based on value of in property.

We would have to be sure that we can provide end-to-end (for producers and consumers) which we couldn't do in the last version of the spec.

Right now I'm aware only of two types of "consumers", which could be affected by such change:

request/reply validators is already broken
documentation generators(for e.g. swagger-ui) is unaffected because simply display schema

Do you now about any other affected "consumers"?

IvanGoncharov on 21 Apr 2015

I'm with @IvanGoncharov. (with respect to allowing all of json-schema). It's been a PITA to have swagger fail the spec validation because we use the same json-schema for model definition as validation. It's "thou shalt not use unblessed attributes" approach seems unnecessarily restrictive.

We don't need to add full blown json-schema support to the spec definition.
That's not what this is about.

We should relax the spec validation to allow valid json-schema in the model definitions. I've never understood what the harm is. So I include valid json schema attributes swagger should just ignore, what's the big deal?

paulhill on 26 May 2015

Folks, I do understand the concern. Historically, however, we have had invalid swagger specifications from a variety of tooling providers, which has made it very difficult for end-users to know "what is compatible with what". One of the most requested, and frankly most important features in 2.0 is a strict schema that should tell you whether or not your swagger specification is compliant with the spec.

If for example we started ignoring definitions in the schema, it would be up to the end user (i.e. consumer) to know great detail of the swagger specification, including what is allowed vs. what is not before being confident that the definition would work with the tooling ecosystem. This has proven over and over again to hurt the consumers of the spec.

To all you to do "whatever you want" you should consider using vendor extensions. They can be easy to produce or possibly more complicated depending on what you're looking to do.

fehguy on 26 May 2015

So use vendor extensions to support existing json-schema constructs?

x-anyOf ?

That seems a little... awkward, don't you think?
Not at all supported by existing json-schema validators.
Re-implement json-schema validation anyone?

:-)

paulhill on 26 May 2015

👍1

@fehguy I think we divert from original problem.
It's about JSON Schema keywords and only in scope of Schema object.

IvanGoncharov on 26 May 2015

I think we should divide problem into three parts:
1) Support for oneOf and anyOf.
As I understand main problem is codegen for strongly typed languages.
2) Rest of unsupported JSON keywords, such as: id, definitions, etc.
I don't see any reason why they should be excluded from Swagger spec.
3) Swagger specific extensions: discriminator and readOnly.
I done quick search in swagger-ui, swagger-codegen and swagger-js and it seems that no one is using them. IMHO, it's look like artifacts from 1.x, which no one really use.

@fehguy, Can you please comment?

IvanGoncharov on 26 May 2015

1) this is not an issue for the codegen only. This affects all the tooling.
2) Some of the missing keywords were missed unintentionally. That said, we cannot add them not. In a future version? Sure. As for the other ones, we need to cover them and see how we can add proper support for them. Like 1) above, it's easy to say we 'support' it, but if we don't provide support for it in the tools or unable to do so, we're providing poor service.
3) readOnly is being added to the codegen as we speak. The whole model inheritance is unfortunately underdeveloped in the tools and we're behind.

webron on 26 May 2015

Hi !

I'm just adding my 2 cents contribution as it's an important issue to me.

I second @IvanGoncharov , supporting the full JSON Schema v4 specs is essential, in my opinion, for many reason, mostly for being able to:

use standards and reusables validators
reference existing schemas
use all the primitive types (lack of null is really problematic in many cases)
describe ALL the API (legacy and new ones)

As I understand the main caveat is that existing tooling won't be able to implement all the specs.
In my opinion, the specs should allows to describe all the APIs, the tooling need to adapt as they can.
When Swagger 2.0 specifications were released, the tooling was not ready, but the specifications permitted to every developper behind a tool to make it evolve in few monthes.

I develop a tool too (flask-restplus) and the impossibility to use existing JSON schema validator is a real handicap because it means one of these:

I have to develop a new validator from scratch (which I won't)
I need to monkey patch an existing one, which is dirty and cost a lot to maintain

I don't know if this the place to debate about this, if there is already a dedicated issue or a dedicated topic on the mailing list, I'll join the existing debate.

noirbizarre on 24 Jun 2015

👍5

@noirbizarre - this is definitely the place to debate it. I'd just like to clarify that for 2.0, this will not change. I completely understand the problems this causes, but no matter which way we go, it will cause problems. I'd like to try and find a hopefully generic solution as part of this ticket when the time comes to be able to provide adequate support across the board. We should thrive to evolve, but do it in a thoughtful process.

webron on 24 Jun 2015

We should thrive to evolve, but do it in a thoughtful process.

That is why I think current model for standardisation is problematic.
2.0 was published and only then Swagger ecosystem slowly started to adapt.
And we have situations like this, de jure spec is correct but de facto many libraries can't implement it fully.

That said, I really like 2.0 standard, and think it best standard for documenting REST API.
But let's start community efforts toward 2.1 in better way:

Publish list of proposals, similar to JSON Schema Draft 5 list
Start working on alpha draft, probably in separate branch and accept PR.
Gather feedback from library owners, who provide experimental support for alpha draft.
In case of backward incompatibility. Prepare official 2.0 into 2.1 converter beforehand.

That procedure will ensure that Swagger ecosystem will be in sync with latest Swagger spec.
And some reference implementation will be available, for library authors.
IMHO, we should make 2.1 focused on resolving only critical issues, without adding new functionality.

IvanGoncharov on 24 Jun 2015

I agree we should improve on the process, and when the time is right we'll make an official announcement with a suggestion for such a process (we have something in mind). The process for 2.0 was open and practically anyone could have joined. Eventually, there was an agreement on the limitations we imposed on the spec, including the non-full JSON Schema support. I cannot say yet when we'll start the process for the next version and whether it would be 2.1 or 3.0. We do have a few issues we know we won't to address there, but I'm not sure this specific issue is one of them.

webron on 24 Jun 2015

I think that oneOf is needed for JSON patch (schema).

I can't find any examples of using JSON patch with Swagger, and the petstore example just uses POST to update resources.

From what I understand, the PATCH method is the most "correct" way to update parts of a resource. I was going to use this method with formData, but it's unclear (+ comments) as to whether or not this is acceptable usage.

We'll probably use PATCH (or even POST) + formData for now, but it would be nice if JSON patch was supported in the future.

glen-84 on 23 Jul 2015

This is what I tried to communicate previously, Swagger spec give people false hopes that they can use it together with JSON Schema.

@glen-84 You can try to use RFC7386.
It missing some features from JSON Patch.
Although in some sense it even better, because it will allow you to validate data inside PATCH and not only patch format.

IvanGoncharov on 23 Jul 2015

How does it give false hope when it explicitly states that it supports a subset of it?

webron on 23 Jul 2015

@IvanGoncharov That's interesting (thanks for the link), but I'd prefer to use something with existing client- and server-side implementations if possible.

Edit: I found some libs. We might consider this.

glen-84 on 23 Jul 2015

I also need oneOf for implementing JSON patch params in Swagger.

jamiehannaford on 21 Sep 2015

@jamiehannaford I personally solve this problem by prefixing unsupported keywords with x- prefix.
That turn them into specification extensions.
Next step is to strip x- before doing validation, it simple and straightforward process.
You can create PR or issue for libraries/frameworks which you use.

IvanGoncharov on 21 Sep 2015

@IvanGoncharov But if you're writing a Swagger file to power a frontend (e.g. swagger-ui), will they know to strip out the x- prefix? Will validators work out of the box too?

I ended up simplifying up the JSON patch schema like so :unamused: Not great but it works...

jamiehannaford on 21 Sep 2015

@jamiehannaford, it just one of options and maybe it will be useful for other people reading this issue.
Right now, there no public plans for extending official Swagger spec, so I think common protocol extensions is good alternative.

But if you're writing a Swagger file to power a frontend (e.g. swagger-ui), will they know to strip out the x- prefix? Will validators work out of the box too?

No, almost every tool will simply ignore x-oneOf. But adding such support is easy, just map 'x-oneOf' => 'oneOf' before validation.

I ended up simplifying up the JSON patch schema like so Not great but it works...

But you still need to validate the same constraints that you express with oneOf.
IMHO, it better solution than separate JSON Schema file or do validation inside code.

IvanGoncharov on 21 Sep 2015

@IvanGoncharov Would you be able to provide two samples which fully explain and show your explanation of the problem in https://github.com/swagger-api/swagger-spec/issues/333#issuecomment-94594048 please? Specifically discriminator. I'm not sure I fully understand the problem, and would like to before trying to explain to others.

Relequestual on 1 Oct 2015

@Relequestual Right now it has some problems(see #403) and they definitely could be solved.

Disclaimer: All below text is my personal opinion.
I think discriminator is broken on conceptual level.
Main problem is reinventing the wheel, JSON Schema already provide oneOf which is superset discriminator(that mean any schema using discriminator can be rewrited with oneOf).

All other problems is consequence of the first one:

I didn't saw any real support in tools and according to @webron: "The whole model inheritance is unfortunately underdeveloped in the tools and we're behind."
It less flexible than oneOf, main thing it require you to have dedicated field in your JSON payload. This mean that in majority of cases you should redesign your implementation. Which isn't what people want than they try to document existing implementations.
Moreover it break separation between data and metadata. You data become aware about of some types in JSON Schema.

And generally this feature not extend JSON Schema but replace oneOf. This lead to situation where in order to use both Swagger and JSON Schema(for example on DB level) you need to maintain two incompatible specifications.

In my particular example I tried to create API interface to collection of Swagger files, so I need to document function which return Swagger file. I naively thought that I could reuse official JSON Schema to describe payload, but it use a lot of oneOf. So in that sense Swagger 2.0 it isn't self hosted :)

I personally solve this issues using this approach https://github.com/swagger-api/swagger-spec/issues/333#issuecomment-142031569.

P.S. If you I didn't answer your question or you have some other questions we can chat here

IvanGoncharov on 1 Oct 2015

👍3

Well said.
+1

On Oct 1, 2015, at 8:31 AM, Ivan Goncharov [email protected] wrote:

@Relequestual Right now it has some problems(see #403) and they definitely could be solved.

Disclaimer: All below text is my personal opinion.
I think discriminator is broken on conceptual level.
Main problem is reinventing the wheel, JSON Schema already provide oneOf which is superset discriminator(that mean any schema using discriminator can be rewrited with oneOf).

All other problems is consequence of the first one:

It doesn't define well in spec, see #403
I didn't saw any real support in tools and according to @webron: "The whole model inheritance is unfortunately underdeveloped in the tools and we're behind."
It less flexible than oneOf, main thing it require you to have dedicated filed in your JSON payload. This mean that in majority of cases you should redesign your implementation. Which isn't what people want than they try to document existing implementations.
Moreover it break separation between data and metadata. You data become aware about of some types in JSON Schema.
And generally this feature not extend JSON Schema but replace oneOf. This lead to situation where in order to use both Swagger and JSON Schema(for example on DB level) you need to maintain two incompatible specifications.

In my particular example I tried to create API interface to collection of Swagger files, so I need to document function which return Swagger file. I naively thought that I could reuse official JSON Schema to describe payload, but it use a lot of oneOf. So in that sense Swagger 2.0 it isn't self hosted :)

I personally solve this issues using this approach #333 (comment).

P.S. If you I didn't answer your question or you have some other questions we can chat here

—
Reply to this email directly or view it on GitHub.

paulhill on 1 Oct 2015

I believe this comment is relevant for this issue so I reference it here. It basically present new problem(at least for this thread) with adoption of Swagger, that is not so oblivious.

IvanGoncharov on 5 Oct 2015

I'd like to throw my hat in the ring for _at least_ including patternProperties in the specification as the argument seems to be centering around oneOf, anyOf, and discriminator. I know everyone has their own use case, but not having patternProperties and failing validation is quite frustrating.

I guess, in the case of patternProperties, I'm not sure why it was left out -- what was the reason for this? It is _extremely_ similar to properties with the difference being that the names of the children are regular expressions. Maybe it was validating that regular expression as being properly formed?

mm-gmbd on 17 Dec 2015

@mm-gmbd Actually patternProperties is very similar to the oneOf.
You don't know type of property until you received data, so you can't generate class/structure for it.
In case of additionalProperties everything is simpler you just have map with string as key and fixed type as value.

IvanGoncharov on 17 Dec 2015

@mm-gmbd I would suggest you to use x-patternProperties as vendor extension. And convert to patternProperties before validation.

IvanGoncharov on 17 Dec 2015

@IvanGoncharov, so, essentially, the reason for leaving these out is to enforce users to better define specifications rather than leave "grey" areas in APIs. I'm okay with that :+1:

mm-gmbd on 18 Dec 2015

Glad to find I'm not the only one struggling with the "discriminator" as a poor replacement for oneOf. It's been feeling like you either have to change your API to align with Swagger's opinions, or add a lot of non-machine readable descriptions to the docs.

drewish on 20 Jan 2016

+1 on getting oneOf or an equivalent working....you simply need polymorphism support to document APIs implemented in an OO language

cabbonizio on 20 Jan 2016

Only that oneOf allow for far more than polymorphism. I'm inclined to agree that discriminator doesn't cover _everything_ and there's room for improvement, and yet, I don't think adding support for oneOf (or anyOf) is the right solution. We'll definitely be discussing this in the next version and it may get included. I'd just rather see a different solution.

webron on 20 Jan 2016

I think oneOf makes sense knowing that standard JSON Schema tooling would support it as-is without needing any special casing. In fact, the tools that I've been involved with would require no changes to support oneOf from a Swagger validation perspective. _(This is because those tools were written for a non-typed language.)_ Where I agree with @webron is that as simple as this change would seem to be from a Swagger specification perspective, this has a big impact on tooling like swagger-codegen and other integrations/tools, like those built for typed languages.

That being said, I can understand the frustration of being told we can't support something because the tooling built around it would become harder to write. But we do have to take things like this into consideration.

whitlockjc on 20 Jan 2016

I think @whitlockjc summed up the frustration pretty clearly. I can't accurately describe my existing APIs because swagger wants to limit itself to APIs that are easy to generate code for in statically typed languages. I can understand why the folks on the static side want the restriction but it feels very artificially limiting.

drewish on 20 Jan 2016

@drewish anything is on the table for the next version of the spec. The flip side of your frustration is that supporting oneOf in the response schema would make code generation for static typed languages nearly impossible, and therefore remove that whole functionality for _all_ users. We did choose--for 1.x and 2.x--to require that static typed languages could be generated. Should that constraint go away in the future?

Remember you can do anything by returning object in the schema. Then static or not, you can do whatever you like with the response. Like Burger King, you _can_ have it your way

fehguy on 20 Jan 2016

I do think that the suggested workaround is somewhat problematic because then the benefit of using Swagger is lost. If I have to use an empty schema to describe the parameters/responses that Swagger cannot support, I basically do not benefit from Swagger in those cases. No documentation, other than the existence of some parameter/response, and no validation because any value would pass validation based on the provided schema. So while this is a valid workaround for Swagger 1.x and 2.x, it is not ideal for the reasons mentioned above.

I bring this up not to question previous design decisions because what's done is done. I bring it up only because information like this might have an impact on whether it makes sense to support oneOf in the future.

whitlockjc on 20 Jan 2016

@drewish - don't get me wrong, the need is clear, the frustration is clear. I really wish it made sense to address everything, but I don't believe we can or should.

Let's say I provide you with the following blueprint for a structure:

6a00d83451b31569e2013489158321970c-800wi

Is it beautiful? Sure. Is it usable? Perhaps in some dimensions (not kidding here, just trying to convey an idea). But it's definitely not usable by many commons.

One of the goals we need to keep in mind is whether we want to just be able to describe everything, usable or not, or do we still want to say that some things just can't be described. In _both_ cases some users (either as producers or consumers) will end up not enjoying the spec. I prefer it to be the producer that 'suffers' because to it me it is more important for the consumer to know that no matter what spec is thrown at them, they could still use it.

With all that in mind, oneOf is _a_ solution to _a_ problem. It doesn't mean it has to be _the_ solution. I'd rather people described their use cases in details to see where the spec falls short so we can see if and how we can solve their issues without potentially introducing new ones. Can't promise we'll be successful, but at least it would be an educated process.

webron on 20 Jan 2016

👎12 ❤1

It's clear everything can't be supported but on the flip side it's essential to provide some sort of way to document these edge use cases. People will get frustrated and go elsewhere and nobody wants that.

cabbonizio on 21 Jan 2016

I'm afraid that's not accurate. Our goal is not to cover all use cases or edge cases (especially if those conflict with our other goals). There's simply no way to make _everyone_ happy. We realize it may not be suitable for everyone and that some may look for alternatives. On the flip side, providing support for some of these edge cases may cause frustration to other users and they'll go elsewhere. Personally, the goal of having API consumers being able to rely on the ability to implement any spec supersedes the goal of being to describe all API designs.

I'll restate though - it looks like everyone is focused on a specific solution to a problem, when there can be other solutions that would satisfy both goals. I'm afraid not many explicit use cases are shared, so it would be difficult to propose such alternatives.

webron on 21 Jan 2016

I'm sorry but these things you're calling edge cases are not edge cases. They are commonly used patterns for quite a long time. I agree some of the requests are not feasible and don't make sense but when you have people trying to evangelize your product at their organization hut we have no way to document a simple Map or polymorphism you find yourself in a corner defending something you believe in. Then we come on here and politely ask for enhancements which may or may not get shot down. It's a risk we take understandably. I want to get my point across that I'm not frustrated or have any disdain about Swagger I truly love it and the people I interact with on here. I think it's important for everyone to show patience and keep an open mind to drive people towards this, not away

cabbonizio on 21 Jan 2016

I'm sorry but you called them edge cases. I followed your wording.

It's clear everything can't be supported but on the flip side it's essential to provide some sort of way to document these edge use cases.

There's a way to document a simple map, and we've covered that (elsewhere). There is a way to document polymorphism, perhaps not _all_ polymorphism.

I agree about patience statement, but that goes both ways. I love that people have ideas and voice them. I love that people don't give up and insist. I believe that's crucial to make progress. There is absolutely _nothing_ personal here if we say X or Y.

Some people have been with the process of the spec for a long time, and some for very little. I'm not saying that the voice of the new people is less important. On the contrary, it can be even more important because it may hit things others haven't, or some thing that were just 'accepted' and shouldn't have been. That said there's history. And history is not always documented, or not always easily accessible. It's very easy to critique decisions without understanding the background. We also try to bring a fuller experience here - we don't want to attend to a specific group of users if it hurts a much larger group of users. Given where we are now, I should hope that we've proven we do have a wide view on things and the entire ecosystem. When we object to something, it's not because 'we don't like it'. We may not agree on the points, which is okay too.

What I sometimes fail in doing is giving proper background and proper details. It's especially difficult when old issues get back to life. It also takes an insane amount of time, and we all have a job to do on top of this. However, specifically here, I believe we've covered the different angles on things.

To me, the points that you and @drewish (and possibly others, it's a long thread) have made are not really about providing full support for JSON Schema, but providing support for specific cases which we may want to do, and we may be able to do without anyOf. Now, we can continue talking about the process, about the past, about semantics and all that. We can also decide that we want to be practical and explore the actual problem and not a specific possible solution. We may end up realizing that we have to add support for anyOf and we may not.

I'll be the first to say that the current solution is less than perfect and would love it if we can have a _productive_ discussion on what kind of APIs you'd expect to be able to describe and then discuss the options of _practical_ implementations. Give us a chance, we may surprise you. If we do this now, it'll greatly help the next version of the spec. If we wait, we may end up with an impractical solution again, whether by not supporting common API designs or by introducing unusable specs.

Now, I mean absolutely no offense here, but I've spent a lot of time on this topic, whether here or other places. Unless I feel this is headed into a more productive discussion, I will have to let it be until the official talks on the version begin. I think that everything 'meta' that could have been discussed and covered has been already (and probably multiple times).

webron on 21 Jan 2016

The first scenario I've run into where having oneOf would be useful in on a rails based API I've been working on documenting. All the GET routes that fetched data were pretty easy, but now that I'm trying to document the POSTs and PUTs I'm finding it very hard to describe the inputs. One example is an endpoint to save personal information which has two different formats for the JSON body:

fields for: name, address, phone, etc
a single token_id that references data sent in a separate request to a different endpoint

The former has several required fields, the later only requires token_id. This seems like an ideal case for a oneOf. For workarounds I guess I've got a couple options:

It's not being used externally yet so I could split the functionality up into two separate endpoints and document them separately. But that doesn't seem like a great answer.
Maybe I can get the discriminator working for this use case? It definitely seems like a long way to do it.
I'll probably just end up spelling out the required fields in the description in a non-machine readable manner.

I think one reason this seems so frustrating is because JSON Schema was used as a starting point for the Swagger spec and then parts of it were deliberately removed/ignored (in the spec itself it's not clear why). If Swagger was a clean slate that just never had that as a possibility, it'd feel different than looking and seeing a ready made solution explicitly ruled out.

I'm half joking when I say this: maybe you could solve this by just removing the references to JSON Schema. Then there'd be no expectation that it'll have the same level of functionality.

I'm not trying to be rude when I say this but it think if the focus is so strongly on a code gen friendly spec you should call that out in the spec itself and the other promotional material. It took me working with Swagger for a month or two to really understand how opinionated it is. Looking at the home page, getting started guide, and spec itself, I'd think it's a just general purpose format for documenting any type of APIs. Let's state these assumptions about how it's to be used clearly.

drewish on 21 Jan 2016

@webron If you haven't read it Gödel, Escher, Bach is definitely worth your time. I think his dialog on a building a record player that can play all sounds is an apt analogy for this topic: http://genius.com/Douglas-hofstadter-contracrostipunctus-annotated

drewish on 21 Jan 2016

I have looked all over, but I have not yet seen a reason _why_ "oneOf" is not supported.

Could someone please point me in the right direction to a succinct reasoning for this choice?

gravesmedical on 2 Feb 2016

Looking at the home page, getting started guide, and spec itself, I'd think it's a just general purpose format for documenting any type of APIs. Let's state these assumptions about how it's to be used clearly.

@drewish is spot on. Many of us are not developing a green field api and are looking for tools that will allow us to document existing apis that may not conform to the ideals set forth currently by this spec. Many of us are also leveraging different forms of polymorphism and have no intention of changing the design of our apis to satisfy the tools we use to document them.

If the intention is to force developers into specific patterns for their apis then that should be clearly called out and proclaimed in documentation and on the homepage.

Prestaul on 3 Feb 2016

I admit I've yet to have a chance to read @drewish's comment (sorry about that), but @Prestaul's comment caught my eye.

Many tools don't allow you to do various things. JSON Schema has things that it doesn't really support itself, or doesn't support well, and it doesn't fit all use cases. They don't go and state any assumptions or anything of that sort.

There's no intention to force developers into specific patterns, but there's also no intention to support all patterns. There's no contradiction there. Not every tool fits every job, and that's not really a problem. There's always a balance of what to do and what to support, and that's what we chose for 2.0. As such, there's no room for any kind of proclamations - the spec is available for you to read, and you can decide if it suits your needs or not. Instead of proclaiming what we _don't_ do, the spec just proclaims what it _does_ support.

Nobody is expecting you to change your API to satisfy the tool. Choose the tool that satisfies your needs.

webron on 3 Feb 2016

@webron, I just reread my comment and certainly didn't mean for it to come across argumentative. I simply think this thread reflects a common frustration from those of us who were eager adopters that later found swagger too restrictive for our needs. That frustration is magnified in those of us who already had experience with JSON-Schema, found it to be sufficient to describe any api fully, and were initially encouraged to see that swagger had adopted much of JSON-Schema.

@drewish, pointed out the frustration:

It took me working with Swagger for a month or two to really understand how opinionated it is.

He also pointed out what is probably the source of that opinionated nature: code-gen tools. It seems as though a tool designed for documentation first would never be built with such restrictions and if code-gen were offered later you might set restrictions around polymorphism depending on the destination language. I understand that polymorphic entities in an api make generating code in strongly typed languages very difficult, but many (probably most?) adopters are not using code generation and many are simply looking for a powerful set of tools for documenting their APIs.

I think what I was trying to say (and, at the risk of putting words in his mouth, I believe @drewish was saying as well) is that I read the spec, I spent time assessing it, I spent time implementing it, I wrote tools integrating it into my process, and was still surprised later by the limitations. At the least it seems we could identify limitations that have become common frustrations (e.g. polymorphism) and find ways to improve clarity around what is not supported.

Prestaul on 3 Feb 2016

👍7

My exhaustion with being unable to describe polymorphism in swagger 2.0 led me to look around for alternatives and RAML fit the bill nicely.

RAML 1.0 just crushes swagger with their rich type system.
http://docs.raml.org/specs/1.0/#raml-10-spec-types

Like @webron says

Nobody is expecting you to change your API to satisfy the tool. Choose the tool that satisfies your needs.

RAML was that tool for me

ghost on 3 Feb 2016

👍4

@Prestaul I think you've summed up my issues very clearly. The only thing I'd add is that it feels like there's a lot of momentum behind Swagger and few alternatives. If it's going to be the only choice it needs to allow for more possibilities.

@norbew I'm excited to check out RAML.

drewish on 3 Feb 2016

To be perfectly clear, swagger is _not_ designed just to be a documentation tool. It's a description of the API interface, intended to be used with tools _like_ documentation systems or code generators. It has _never_ strived to model all possible APIs, period.

If that has ever been confusing then I apologize, hopefully it is not now.

If we supported _everything_ that JSON Schema draft 4 had to offer, then the universe of tooling that can actually do something with the spec would dwindle and we would be nothing other than a modeling syntax like RAML. Then we would have the extremely limited practical usage of RAML, and that's certainly not the goal (ever wonder why there's no c++ generator for RAML?)

That said, there is not a hesitation to support more, but our goals are not simply to describe arbitrary APIs. We want to support tooling constructs as well, and codegen is one of, if not the most important of them. Adding more supported constructs that makes tooling overly difficult or even impossible is not an option.

fehguy on 3 Feb 2016

@fehguy that is all clear now but that is after months of usage and a day of reading through these GH issues... Did you have a comment about my last statement?

At the least it seems we could identify limitations that have become common frustrations (e.g. polymorphism) and find ways to improve clarity around what is not supported.

Prestaul on 3 Feb 2016

@fehguy Please put that on the front page then. I pull up https://openapis.org and first thing I read is:

The OAI is focusing on creating, evolving and promoting a vendor neutral API Description Format based on the Swagger Specification.

There's nothing on there at all about code gen.

Flipping through the pages I get to https://openapis.org/specification and I finally discover some hints at it (I added bolding for emphasis):

The goal of the OAI specification is to define a standard, language-agnostic interface to REST APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection. When properly defined, a consumer can understand and interact with the remote service with a minimal amount of implementation logic. Similar to what interfaces have done for lower-level programming, Swagger removes the guesswork in calling the service.

Again totally a worthwhile goal. It makes sense why c++ users would accept that restriction in exchange for those benefits. But as someone working primarily in untyped languages it's not helpful, and actively hobbles my usage. At this point I'm not even saying you should change it, just warn people about that trade off so they go into it with eyes open.

drewish on 3 Feb 2016

👍4

OK, if nothing else, your time on this will help with documentation for others. I do think we have tried to make it clear what is supported in the specification, which is intentionally not a time commitment like reading the bible:

https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#schema-object

The Schema Object allows the definition of input and output data types. These types can be objects, but also primitives and arrays. This object is based on the JSON Schema Specification Draft 4 and uses a predefined subset of it. On top of this subset, there are extensions provided by this specification to allow for more complete documentation.

In addition there is an online, hosted schema validator which uses the JSON Schema representing the specification that should tell you specifically what is not supported:

swagger: '2.0'
info:
  version: 1.0.1
  title: Modeling Samples
paths:
  /foo:
    get:
      parameters: []
      responses:
        default:
          description: success
definitions:
  Sample:
    oneOf:
      - type: object
        properties:
          id:
            type: string
      - type: object
        properties:
          name:
            type: string

response via online validator:

$ curl -X POST --data-binary @sample.yaml -H content-type:application/yaml https://online.swagger.io/validator/debug | pretty
{
    "messages": [
        "attribute definitions.Sample.oneOf is unexpected",
        "attribute definitions.Sample.properties is missing"
    ],
    "schemaValidationMessages": [
        {
            "domain": "validation",
            "instance": {
                "pointer": "/definitions/Sample"
            },
            "keyword": "additionalProperties",
            "level": "error",
            "message": "object instance has properties which are not allowed by the schema: [\"oneOf\"]",
            "schema": {
                "loadingURI": "http://swagger.io/v2/schema.json#",
                "pointer": "/definitions/schema"
            }
        }
    ]
}

So I'm sorry it took so long to get to this point, but I think the tooling is there to avoid this. I think docs can always be improved, of course.

fehguy on 3 Feb 2016

@Prestaul this is a pretty long issue so I'm guessing you've described your questions about polymorphism in here. I would suggest moving those discussions to a different issue as this has become really around "supporting all of JSON Schema Draft 4" and not "limitations in Polymorphism".

fehguy on 3 Feb 2016

My 2 cent contribution because I think this issue is major.

I totally agree with @drewish and @Prestaul : Swagger was described everywhere (swagger.io, openapi.org...) as a tool for describing _ANY_ API in a _Language Agnostic_ way but clearly, after using it for a very long time, developping some tooling around it: it's not.

The oneOf, anyOf... is not about polymorphism (which is by the way not language agnostic at all) but about real world usages, most of the time not related to polymorphism.

To me, JSON Schema v4 is about compatibility and reusability.

We can find JSON Schema v4 validator in any language, this is not the case for Swagger specific subset.
If some people design JSON schema to describe any models, there a reason. This is the 4th draft with a lot of maturity. Choosing a subset of this work is like saying: we know better, you won't need this.
It's standardised so it allow to import existing schemas for commonly used data types. Having a subset of JSON schema forbid model import (GeoJSON is still very complicated to describe with Swagger, I can testify, I design a lot of spatial APIs)

By the way, most of the time, my server-side can expose full v4 models, my client can consume full v4 models, but I'm struggled with the swagger models not allowing me.

My point is that Swagger is meant to describe APIs (and it does it very well) but there is some caveat on the model part.
JSON Schema provide the model part and it's doing well, there's an ecosystem (tooling, documentation, libraries, testers...). Why reinvent the wheel on the model part ?

If I'm understanding well, the main point is that tooling won't be able to support every case.
It's not bad: just announce it: "This tool don't support XXX, YYY and ZZZ". But clearly some other tools support it and need it.

It seems to me that codegen is really tiny use case: you generate code once on a project start if you are in a spec to code approach. Documentation has to be for the entire product life. What a deceptive surprise when after month working on and maintaining a swagger documentation when you discover you can't describe anymore your new features.

Please support the full JSON Schema V4 specifications for models, it will unlock the full Swagger potential !

noirbizarre on 3 Feb 2016

👍4

I'm really happy to see everyone's involvement on this ticket. It shows that people care and want to see improvements, which is great.

@noirbizarre - I can't think of a single place where it says that Swagger aims to be a tool that can describe _ANY_ API. In fact, I can point you to dozens of times that we have stated that Swagger does _not_ aim to cover all APIs, but rather provide a comprehensive ecosystem for the APIs it is able to describe.

@Prestaul - Any tool has its limitations. I was extremely frustrated with JSON Schema when I discovered that some things cannot be done (you simply cannot a JSON Shema that fully validates the Swagger object, for example - just can't be done) or it imposes an insane structure that actually forces you to do repetitions in some cases. It may be the best of what's out there, but I'll be the first one to drop support for it at all if we could find an alternative (no plans for it, no worries). However, even with those limitations, I continued working with it and worked with the limitations. It's fine. I really can't and don't expect it to cover everything.

The spec clearly states that JSON Schema is not fully supported. I'm open to feedback as to how it can be clarified further, and I've modified the spec in the past to make portions of it clearer. I get the frustration of discovering it at a later stage, but I don't see how that's different than any other tool. Was I frustrated by JSON Schema? I was, because I assumed things at first as well. But the JSON Schema specification is also documented and the limitations can be derived from the spec. That's just how things work. We don't normally document what _can't_ be done, but rather what _can_.

To me, code generation is not the real issue but it's a sign for the issue. I'd say that if you can't generate code for an API, then that API is not usable. It's not about the automated process, it's about the ability to write code that processes it. If you can write it manually, you should be able to generate it to speed up the process, that's all. Granted, this does not apply _all_ the time, but in most cases it would. And before you say that you may not necessarily care about languages that can't use the API (easily or at all), please read the next point.

In my _personal_ opinion, the specification should be such that when a user encounters the API description, they can feel confident that they can actually use it and implement it. There are many benefits for the API description - you can get documentation, validation, testing, monitoring, and yes, even code generation. The idea behind the wording of it being language agnostic is that you really should be free to use whatever you want, and still end up being able to use the API. It is my opinion that it's more important to be able to "describe some APIs that can be used by everyone" over being able to "describe all APIs that can be used by someone". When it comes to APIs, the consumers are more important than the producers - without consumers, there's no need for the API. If we can't make the consumers confident they can always use the description we are missing a major part of the goal.

Kindly remember that there are plenty of others out there who don't voice their opinion here. Some are definitely aligned with the opinions stated by users here, and some definitely feel things should stay as-is. We're trying to take everyone's opinion into account, whether voiced here or through other means.

Last, but absolutely not the least - I feel we're concentrating on solutions and not problems. That sounds off, doesn't it? We want solutions! Well, yes. However, just because everyone is sticking to a specific solution, doesn't mean it's the only solution or the right solution. I honestly believe we can solve quite a few of the issues raised with other means, given the chance to do so. For example, 'Incomplete support for polymorphism' is definitely an issue, and one that we should address. Can we solve it by supporting full JSON Schema? Sure, but there are costs. Can we solve it by doing something outside the scope of JSON Schema without those extra drawbacks? Possibly, but we haven't even given it a chance. Yes, there are numerous benefits to using JSON Schema directly, as @noirbizarre kindly mentioned. However, if we can find a solution that would satisfy both aspects (and I can think of a few) without being a direct usage of the full JSON Schema, wouldn't that be better? Please note that I'm not trying to start a discussion on that right now, but rather just raising the point that it seems many are fixated on a specific solution, which may not be the only one. And it applies to other tickets opened on the spec, not specifically the JSON Schema one. I'd ask people to take the time, isolate the problem they are trying to solve, and open tickets specifically for those issues. Once we start the process of work on the next spec, I just want us to keep that in mind and be creative when needed. I know it may seem that 'we' are against support for full JSON Schema, but that's not really the case. We're pretty much wearing multiple hats here and try to represent the voices of others as well. If we can do it in a smart way, I'd be more than happy to accommodate for more API descriptions.

webron on 3 Feb 2016

@webron (longest post ever).

fehguy on 4 Feb 2016

@fehguy challenge accepted.

webron on 4 Feb 2016

It seems as though a tool designed for documentation first would never be built with such restrictions and if code-gen were offered later you might set restrictions around polymorphism depending on the destination language.

If I'm understanding well, the main point is that tooling won't be able to support every case.
It's not bad: just announce it: "This tool don't support XXX, YYY and ZZZ".

I completely agree with the above comments. Why restrict the specification, and provide no support for common patterns like JSON patch, just for the sake of code generation?

Something with the name "Open API Specification" may not be able to cover all possible scenarios, but it surely needs to cover a large percentage of them? Patching, describing spatial APIs, multiple response schemas, etc. are not far-out crazy ideas used by 0.5% of APIs in the wild.

The specification should be as flexible as is technically possible. It should be up to the API designer to decide which features to use depending on their specific requirements for code generation and other tooling.

If there are alternative solutions, then propose them and they can be discussed, but it seems to me that if you're already using a subset of JSON Schema, you might as well use more of it to provide these features.

glen-84 on 7 Feb 2016

The specification should be as flexible as is technically possible. It should be up to the API designer to decide which features to use depending on their specific requirements for code generation and other tooling.

That's where we disagree. The ability to provide tooling (and not necessarily code-generation) is what makes the specification even relevant in the first place. If that means it's not as flexible, so be it. Yes, the API designer should decide to use whichever tool fits their needs, including the spec itself.

webron on 8 Feb 2016

Great conversation on this important topic.
One thing I think hasn't been mentioned: JSON Schema was not designed with APIs in mind, it is primarily maintained for the NoSQL range of databases. As such, there are approaches accommodated which simply don't fit well with common/practical API usage scenarios. As noted, polymorphic responses are pretty rare (because they cause pain for everyone involved)...this begs the question: should handling the <1% case make the priorities?
There are certainly other features in JSON Schema we all might benefit from supporting, my opinion is that anyOf/oneOf is full of pain and despair in practice, both in API usage and in tooling implementation.

I strongly echo Ron's opinions, and suggest the following approach:

Approach polymorphism support as a separate issue, and investigate other potential solutions to accomodate the small crowd that really needs this support. As noted discriminator is in 2.0 to address this problem, and as far as I know, has really never been implemented due to it's inherent complexity (and perhaps lack of JSON Schema compliance). We had to take this same approach in order to tackle composition and inheritance support with allOf on the tooling side a while back...polymorphism discussions really drag down other important issues.
Review features/fields in JSON Schema we should add toOAS, outside of the polymorphism issues. I think that's the original intent of this issue, and has been echoed in a number of other issues.

jharmn on 8 Feb 2016

The flexibility is required, and you'll continue to receive questions and issues regarding these problems.

Whether it's implemented using JSON Schema or BananaTech 2.0, is less important.

glen-84 on 8 Feb 2016

@jasonh-n-austin I'd love to see the data behind your assertion that:

polymorphic responses are pretty rare (because they cause pain for everyone involved)...this begs the question: should handling the <1% case make the priorities?

In the statically typed world that might be the case but in Ruby or JavaScript it's not uncommon.

drewish on 8 Feb 2016

@drewish I wasn't commenting on language-specific, I'm just saying that it's a less popular pattern in API design. It's certainly arguable it's more common, I really should have said that I doubt it passes an 80/20 rule of priorities...i.e. it hardly effects the broad spectrum of API design.
I don't argue it's important, but IMO the other issues in JSON Schema are probably more far reaching, and should be focused on in this issue, while polymorphism should be addressed distinctly.
My only real intent is to move the discussion on broader JSON Schema support forward.

jharmn on 8 Feb 2016

I agree with you @webron, the main point is usability and Full JSON Schema compliance might not be required, but some little adjustments might be necessary.

In my opinion, the following changes can be very usefull for a lot of people:

nullable properties support. It's the combination of anyOf/oneOf support and null primary type support (#229)
The anyOf/oneOf pattern is not so uncommon. GeoJSON is still my example, it's a standard output format which is best described with oneOf (example: https://github.com/fge/sample-json-schemas/blob/master/geojson/geojson.json). OAS should allow to easily describe common mainstream formats out of the box, GeoJSON being one of them.
another use case for oneOf/anyOf is mixed lists which are not so uncommon.
anyOf/oneOf might also be usefull when you API support both ID and slug as URL identifier or query parameters (but it's not in the scope of schema objects)

A wider polymorphism support is far less important to me as the current pattern works (very well) for linear inheritance. But I agree mixins are a really common pattern, in both Python and JavaScript for example (my primary stack). I understand that it may be complex sometimes, but It's up to the developper to chose whether or not it's a good pattern for his case. Sometimes it's simply necessary.

There is already an open issue for anyOf: #377 but I think oneOf is also important.

noirbizarre on 11 Feb 2016

Hopefully this is on topic. I've been using Enum's for inheritance and unions. See this Gist:
https://gist.github.com/fadeddata/32e0302d2a55e1392b9c

Specifically look at the questionType property in the Question definition.

      questionType: 
        type: object
        enum:
        - $ref: '#/definitions/ScaleQuestion'
        - $ref: '#/definitions/MultipleChoiceQuestion'
        - $ref: '#/definitions/TextQuestion'

The goal of the API is to sort of do what Google Forms does for questions.

This seems to work quite well and makes the documentation tool happy. Generated code is mostly junk but I don't use that anyways. Does this help solve anyones problems? Am I grossly misusing the tool?

fadeddata on 26 Feb 2016

@fadeddata This really looks like misusing the tool. You are defining your questionType property to contain an object with just a $ref property, which contains one of the mentioned three strings.

From the relevant JSON Schema Spec section:

5.5.1. enum

5.5.1.1. Valid values

The value of this keyword MUST be an array. This array MUST have at least one element. Elements in the array MUST be unique.

Elements in the array MAY be of any type, including null.

5.5.1.2. Conditions for successful validation

An instance validates successfully against this keyword if its value is equal
to one of the elements in this keyword's array value.

The elements of the enum array are not schemas, but JSON entities to be compared for equality.

(Also, at least in Swagger-UI, it doesn't really look like I guess you intend it to look.) Swagger-Editor shows it differently – it seems to follow the $refs, though I'm not sure the result is really what you meant here.

ePaul on 28 Feb 2016

Paolo,

Can you elaborate on how this is misusing the tool?

The value of the enum keyword is an array of objects ($ref pointers).

The array has at least one element.

The elements in the array are unique.

The elements are of the type $ref (object/pointer)

A schema is a json entity.

I've been struggling with a simiar issue and I'm looking for clarification
here in terms of how this is misusing the tool.

-Mark

On Sun, Feb 28, 2016 at 6:49 AM, Paŭlo Ebermann [email protected]
wrote:

@fadeddata https://github.com/fadeddata This really looks like misusing
the tool. You are defining your questionType property to contain an
object with just a $ref property, which contains one of the mentioned
three strings.

From the relevant JSON Schema Spec section
http://json-schema.org/latest/json-schema-validation.html#anchor76:

5.5.1. enum 5.5.1.1. Valid values

The value of this keyword MUST be an array. This array MUST have at least
one element. Elements in the array MUST be unique.

Elements in the array MAY be of any type, including null.
5.5.1.2. Conditions for successful validation

An instance validates successfully against this keyword if its value is
equal
to one of the elements in this keyword's array value.

The elements of the enum array are not schemas, but JSON entities to be
compared for equality.

(Also, at least in Swagger-UI
http://petstore.swagger.io/?base_url=https://gist.githubusercontent.com/fadeddata/32e0302d2a55e1392b9c/raw/03ea06655deb2cc126cb89ca125ffb2f0e866792/enum.yaml,
it doesn't really look like I guess you intend it to look.) Swagger-Editor
shows it differently – it seems to follow the $refs, though I'm not sure
the result is really what you meant here.

—
Reply to this email directly or view it on GitHub
https://github.com/OAI/OpenAPI-Specification/issues/333#issuecomment-189864341
.

gravesmedical on 28 Feb 2016

@webron has a lot of experience with this one. Can you please chime in?

fehguy on 1 Mar 2016

I don't think this does what you'd hope it does.

Following the gist, you're saying that the questionType property needs to have the value of (for example):

NumberValidation:
    allOf: 
    - $ref: '#/definitions/Validation'
    - type: object
    required:
    - comp1
    - comparison
    properties:
      comp1: 
        type: number
      comp2: 
        type: number
      comparison:
        type: string
        enum: [gt, gte, lt, lte, =, "!=", between, "!between", whole]

Not something that _validates_ against that schema, but the value has to be the schema itself. enum just sets possible values for the defined schema.

webron on 2 Mar 2016

To give some perspective the backend that I'll be implementing this in is Scala. I will likely use the Play JSON Derived Codecs (https://github.com/julienrf/play-json-derived-codecs) to serialize/deserialize the messages. So this is going to be used with a statically typed environment. The code will look something like:

sealed trait QuestionType {
  val kind: String
}

case class ScaleQuestion(kind: String, fromLabel: Option[String], toLabel: Option[String], from: Int, to: Int) extends QuestionType

case class MultipleChoiceQuestion(kind: String, other: Boolean, choices: List[String], display: String) extends QuestionType

etc...

The Swagger def looks like:

  ScaleQuestion: 
    type: object
    required:
    - kind
    - from
    - to
    properties:
      kind:
        type: string
      fromLabel:
        type: string
      toLabel:
        type: string
      from: 
        type: number
      to: 
        type: number
    example:
      {
        "kind": "ScaleQuestion",
        "fromLabel": "I never cough",
        "toLabel": "I cough all the time",
        "from": 1,
        "to": 5
      }

  MultipleChoiceQuestion:
    type: object
    required:
    - kind
    - choices
    properties:
      kind:
        type: string
      other:
        type: boolean
      choices: 
        type: array
        items:
          type: string
      display:
        type: string
        enum:
        - radio
        - select
    example:
      {
        "kind": "MultipleChoiceQuestion",
        "other": false,
        "choices": ["Option 1", "Option 2", "Option 3"]
      }

fadeddata on 2 Mar 2016

@webron I think I understand what you are saying. The enum, as the yaml is written, is specifying that it's value must conform to exactly as the ref is specified.

I need away to tell the reader of the documentation that the input can be one of a few things. I didn't find the discriminator functionality to be that useful in this endeavor.

What would you propose? A different path for each question type?

POST /question/multiple-choice-question
POST /question/scale-question

fadeddata on 2 Mar 2016

@gravesmedical While the example from @fadeddata is a valid OpenAPI (2.0) definition according to the Spec, it has not the same meaning as he seem to want it to have. Though maybe I misunderstood him there.

These are exactly the valid questionType values according to the definition:

{ "$ref": "#/definitions/ScaleQuestion" }
{ "$ref": "#/definitions/MultipleChoiceQuestion" }
{ "$ref": "#/definitions/TextQuestion" }

No others, specially nothing which I would take as a representation of some form of question.

If we assume that the refs are replaced here (nothing in the spec says so), these would be the valid values:

{ "allOf": [ { "$ref": "#/definitions/QuestionType" }, { "type": "object" } ], "required": [ "from", "to" ], "properties": {"fromLabel": {"type": "string"}, "toLabel": {"type": "string" }, "from": { "type": "number"}, "to": { "type": "number"} } }
{ "allOf": [ { "$ref": "#/definitions/QuestionType" }, { "type": "object" } ], "required": ["choices"], "properties": { "other": { "type": "boolean" }, "choices": { "type": "array", "items": { "type": "string" } }, "display": { "type": "string", "enum": [ "radio", "select" ] } } }
{ "allOf": [ { "$ref": "#/definitions/QuestionType" }, { "type": "object" } ], "properties": { "validation": { "type": "object", "enum": [ { "$ref": "#/definitions/NumberValidation"}, { "$ref": "#/definitions/TextValidation"}, { "$ref": "#/definitions/RegexValidation" } ] } } }

Still I don't assume one actually want your clients to send or receive one of these three objects – we want them to send/receive something which is described by this schema. But this is not what enum does – you want proper oneOf support as discussed further up in this issue.

ePaul on 2 Mar 2016

@ePaul Yes, that makes tons of sense. It seems that this is supported and valid syntax, but it is likely not accomplishing his intended purpose (presuming he is thinking of "instantiating" the references to validate them.

In the instance that it is actually validating against that schema, it would be correct.

Thanks for the clarification.

gravesmedical on 3 Mar 2016

Great thread, everyone, glad to see there is a lot of interest here. I'm a little late to the party, so apologies if I raise points that have already been noted. I've been loosely following Swagger and RAML for a few years from an architectural point of view. I am seeking a standard way to do design-first API development. Swagger was traditionally weak (IMHO) in this area and RAML rose to the challenge but was (is?) lacking in the tooling department and developer mindshare. The Swagger tooling I've used is great and I rarely encounter a fellow hacker that isn't familiar with Swagger at least by name.

I really like the spirit of OAI and I hope we will continue to see rapid evolution here. I also understand the concerns and tradeoffs necessary to satisfy the broad goals of the OAI specification. I think it would be helpful for the OAI to document and prioritize its use-cases in order to more formally guide the evolution of the spec. Certainly the ability to document any possible API and support code-generation for any possible language is too broad, but simply narrowing the possible APIs doesn't seem like the right approach to me.

When @fehguy says

ever wonder why there's no c++ generator for RAML?

I do wonder whether it was technically infeasible or there was simply no interest in having one in the first place.

Back to the issue at hand, let me share an payload example from one of the APIs I work on:

{
  "id": "85d8f8fc-15a5-40f4-93c9-6c09597fc403",
  "createdOn": "2015-10-21T18:28:06.123-04:00",
  "properties": {
    "singleValue": "value",
    "multiValue": [ "value1", "value2", "value3" ]
  }
}

In the above example, the properties id, createdOn and properties are formally declared and properties is an object which itself can have an arbitrary set of properties whose values can be either a single string, or an array of strings. We can validate this using JSON-schema as follows:

{
  "$schema": "http://json-schema.org/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "pattern": "^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-5][0-9a-fA-F]{3}-[89aAbB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}$"
    },
    "createdOn": {
      "type": "string",
      "format": "date-time"
    },
    "properties": {
      "type": "object",
      "additionalProperties": {
        "oneOf": [
          {
            "type": "string"
          },
          {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        ]
      }
    }
  },
  "additionalProperties": false
}

This certainly presents a problem for code generation - how do you infer the narrowest type that can represent either a string or an array of strings - but it is trivial for someone to manually create a mapping in the language of their choice.

So, when @webron says

I'd say that if you can't generate code for an API, then that API is not usable

I wholeheartedly disagree.

I think this in a good example where oneOf neatly solves the issue and discriminator support in Swagger does not. I want to use basic types for the polymorphism, not something I can add a discriminator to. Please correct me if there is a way to accomplish this in Swagger as it is literally my only blocker to adopting Swagger for this API.

Thanks, everyone, for the work so far (and sorry for writing a novel here).

emattheis on 5 Apr 2016

👍3

@webron says

I was extremely frustrated with JSON Schema when I discovered that some things cannot be done (you simply cannot a JSON Shema that fully validates the Swagger object, for example - just can't be done) or it imposes an insane structure that actually forces you to do repetitions in some cases.

A bit late to the party here, but simillaly to @emattheis , I've been following both Swagger, RAML, and JSON Schema for over 2 years now.
I've been trying to drive further work on JSON Schema, making draft 4 the standard, but there's lots going on behind the sceens that I can't or shouldn't talk about.
I'm pretty sure that the people involved would LOVE your feedback, and any suggestions for improvment would be welcome. Not saying you haven't given feedback, I don't know. Realistically, the spec, and further development of, is all a bit of a mess / unclear right now, and has been for some time.

Relequestual on 6 Apr 2016

@emattheis Your example could be described without oneOf, since JSON Schema Draft4 supports arrays as values of type properties:

The value of this keyword MUST be either a string or an array. If it is an array, elements of the array MUST be strings and MUST be unique.
String values MUST be one of the seven primitive types defined by the core specification.

So you can write:

{
  "type": ["string", "array"],
  "items": {
     "type": "string"
  }
}

And it validates both string and array of strings.

And de jure it valid according to OpenAPI spec:

The following properties are taken directly from the JSON Schema definition and follow the same specifications:
...

type

It also passes validation with JSON Schema for OpenAPI spec, because of this.
@fehguy @webron Was it an intentional decision to support this feature?

IvanGoncharov on 6 Apr 2016

@Relequestual Biggest problem with JSON Schema is it DSL for validation. It was meant to describing constraints on data and not data themselves. So it almost impossible to use it for something other than validation.

I think it needs more clear support for inheritance and polymorphism. And it needed not only to generate SDKs for statically typed languages but for example to generate nice UI with input forms solely out of JSON Schema. Try to represent oneOf or anyOf in such UI without totally confusing a user.

@Relequestual @webron we can continue this discussion.
But probably better to move it to different issue/forum/mailinglist since it diverge from the main topic of this issue.

IvanGoncharov on 6 Apr 2016

👍1

{
  "type": ["string", "array"],
  "items": {
     "type": "string"
  }
}

is invalid in 2.0. I know it's not clear from the spec itself, but it's not supported. type can only have a single value.

webron on 6 Apr 2016

@webron I suspected that because this behaviour is a subset of oneOf.
I think we should discuss real-life problems caused by oneOf absence.
And try to address at least most common ones.

IvanGoncharov on 6 Apr 2016

@IvanGoncharov thanks for the tip, I never realized that about JSON schema~~, but I'm not sure I like it - oneOf seems less ambiguous~~. Anyway, looks like a non-starter in 2.0, per @webron.

Per your comments:

It was meant to describing constraints on data and not data themselves. So it almost impossible to use it for something other than validation.

This is what I was getting at in my previous post - it should be made clear that the schema construct in the OAI spec is not intended solely for validation.

I think it needs more clear support for inheritance and polymorphism. And it needed not only to generate SDKs for statically typed languages but for example to generate nice UI with input forms solely out of JSON Schema.

This is a pretty tall order and clearly JSON Schema alone is not up to the task. This is the same realization the RAML folks had, I think, when they decided to make a clean break from XML/JSON Schema with their new data type system.

Try to represent oneOf or anyOf in such UI without totally confusing a user.

Doesn't discriminator impose the same challenges? Is it supported in current tooling?

emattheis on 6 Apr 2016

@IvanGoncharov @webron For the record, my real-life problem is limited to the desire to have polymorphic behavior with 'primitive' types - I don't really need or want full support for oneOf. For custom types, I am almost satisfied with the discriminator support since that's generally how we deal with polymorphism in JSON payloads, anyway.

emattheis on 6 Apr 2016

👍1

Doesn't discriminator impose the same challenges? Is it supported in current tooling?

IMHO, discriminator has many problems and only few tools support it.
But it definitely easier(for something other than validation) to support it than unconstrained oneOf.
With that said I think incompatibilities with JSON Schema is HUGE problem for OpenAPI spec.

Example of discriminator support, see petType field:
http://rebilly.github.io/ReDoc/#operation/addPet
Disclaimer: I'm working as OpenAPI consultant for ReDoc project.
So my opinion is based mostly on this project, and I don't have any background in OpenAPI code generation.

IvanGoncharov on 6 Apr 2016

My problem with the discriminator is that it needs to be 1:1 with a definition. Which makes it very hard to adapt to an existing API even if it's already returning fields with record types:

{
  "id": "cuavyyna8lg8",
  "object": "transaction",
  ...
}

{
  "error": {
    "type": "transaction",
    "message": "The security code you entered does not match. Please update the CVV and try again.",
  ...
  }
}

Since the ~~type~~ discriminator field needs to be unique. Again, a very frustrating "solution". It would be almost useable if there was some translation layer where you could distinguish how the discriminator values mapped to definitions.

drewish on 6 Apr 2016

I just looked a little closer at the discriminator spec. I agree with @drewish that it would be a lot more usable if it were possible to declare how a discriminator value maps to a schema type instead of requiring a 1:1 relationship.

Also, I just noticed that the discriminator property must be required. This could be slightly problematic. We generally only include discriminators when there is ambiquity.

If I have an endpoint /animals then I expect something like

[
  { "type": "cat", "name": "fluffy" },
  { "type": "dog", "name": "buster" }
]

But I may also have an endpoint /animals/cats where the following is sufficient:

[
  { "name": "fluffy" }
]

emattheis on 7 Apr 2016

👍1

And here is my question regarding the missing oneOf and property handling.

I have this C# code:

public class TestClass
{
    [Required] // <= not nullable
    public ReferencedClass RequiredProperty { get; set; }

    public ReferencedClass NullableProperty { get; set; }
}

public class ReferencedClass
{
    public string Test { get; set; }
}

Both properties reference the same ReferencedClass (i.e. it must be a reference in the generated schema). The generated schema would look like this (JSON Schema draft4):

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "additionalProperties": false,
  "required": [
    "RequiredProperty"
  ],
  "properties": {
    "RequiredProperty": {
      "oneOf": [
        {
          "$ref": "#/definitions/ReferencedClass"
        }
      ]
    },
    "NullableProperty": {
      "oneOf": [
        {
          "$ref": "#/definitions/ReferencedClass"
        },
        {
          "type": "null"
        }
      ]
    }
  },
  "definitions": {
    "ReferencedClass": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "Test": {
          "type": [
            "null",
            "string"
          ]
        }
      }
    }
  }
}

As you can see, the schema for ReferencedClass cannot have a type of object AND null because it would be wrong for the first property (RequiredProperty). Also the required array does not express nullability but if the field must be defined in JSON or not (it still can be null).

How would you express this with the Swagger "special" JSON Schema?

RicoSuter on 4 Jun 2016

Btw: allOf is not an option:

Schema:

{
  "type": "object",
  "required": [
    "foo"
  ],
  "properties": {
    "foo": {
      "type": [
        "null",
        "object"
      ],
      "allOf": [
        {
          "type": "object",
          "required": [
            "name"
          ],
          "properties": {
            "name": {
              "type": "string"
            }
          }
        }
      ]
    }
  }
}

This is valid:

{
  "foo": {
    "name": "test"
  }
}

This doesn't work because this not valid:

{
  "foo": null
}

And adding the property schema directly to the property is not possible because you can either have $ref or other properties and the property schema must be referenced so that it can be shared...

RicoSuter on 4 Jun 2016

@rsuter OpenAPI 2.0 doesn't support null at all (see e.g. #297 for an explanation) – the way to represent your TestClass would be by using the required property to indicate that RequiredProperty is required, and you could then omit NullableProperty from the JSON representation of your TestClass object if it is null.

TestClass:
   type: object
   properties:
     RequiredProperty:
       $ref: '#/definitions/ReferencedClass'
     NullableProperty:
       $ref: '#/definitions/ReferencedClass'
   required:
     - RequiredProperty

Nullability is a separate topic discussed in #229.

ePaul on 5 Jun 2016

Ok thanks for the heads up. I'm rewriting my library (NSwag.org) with an option to handle nullability this way... But how can I express a nullable HTTP response without oneOf? There is no required field there...

RicoSuter on 5 Jun 2016

phew that took a while to read through.

@webron I've seen you call for use cases a number of times (https://github.com/OAI/OpenAPI-Specification/issues/333#issuecomment-205989679, https://github.com/OAI/OpenAPI-Specification/issues/333#issuecomment-173475240). While a few have been presented I'm finding a serious lack of documentation/discussion on the prohibitive factors of supporting json-schema v4 in full - aside from a number of vague references to code generation. There seems to be a clearly expressed interest in supporting json schema more thoroughly here, and it also seems to me that a very easy way to drive this conversation is to get the technical hurdles out in the open quickly so we're not boiling down to preferences.

I'd like to echo @gravesmedical (specifically), and others (probably), in asking for a succinct reasoning for the exclusion of these parts of JSON schema v4. If it's just an issue with code then perhaps we can work towards a solution (@fehguy I haven't seen a C++ validator for RAML, but I suspect thats because of a smaller community. valijson supports json schema v4 validation including anyOf/oneOf for C++ as one example). If, on the other hand, you guys just don't like json schema (as it is, for this purpose, because of its hairdo etc) then that's totally reasonable as well - we should just clear that up IMHO.

Cheers!

mbroadst on 20 Jun 2016

It's not really about liking or disliking JSON Schema. JSON Schema, much like any other specification (yes, the OAS as well), is not without faults. It wasn't really written with the mindset of APIs. You can see in other tickets that we may want to include support for other formats as well, but it's unlikely that we'll drop support for JSON Schema. The challenges are not just with code generation, and they would appear in code in general in some cases. There challenges are also with validation and documentation visualization.

The need for full JSON Schema support is clear. At the end, it boils down to whether it's more important to describe more APIs or making sure there's robust support in the tooling, no matter which tool it is. Even if we choose that the latter is more important, it doesn't mean we won't lift some of the current restrictions. It's definitely one of the more challenging aspects and I'm hopeful we can improve it wisely.

webron on 20 Jun 2016

Apologies if the last bit about not liking json-schema came off like my main point, but I'm still unclear as to what specific technical hurdles surround full json-schema support.

Can we start to build a list so that developers who are interested in helping the cause could potentially chip in? (is there already some sort of documentation about the challenges?)
Is there any existing work to fully support json-schema in your tooling that you could point to?

I'm very keen to use OpenAPI for my product, however the lack of full json-schema support is limiting in my case. I'd like to help out on this effort in any way I can, and from what I'm reading here it seems like I'm not the only one.

mbroadst on 21 Jun 2016

Matt, let me know if you want to chat sometime about this. I'm interested
in supporting this endeavor.

-Mark

On Tue, Jun 21, 2016 at 6:46 AM, Matt Broadstone [email protected]
wrote:

Apologies if the last bit about not liking json-schema came off like my
main point, but I'm still unclear as to what specific technical hurdles
surround full json-schema support.

Can we start to build a list so that developers who are interested
in helping the cause could potentially chip in? (is there already some sort
of documentation about the challenges?)

Is there any existing work to fully support json-schema in your
tooling that you could point to?

I'm very keen to use OpenAPI for my product, however the lack of full
json-schema support is limiting in my case. I'd like to help out on this
effort in any way I can, and from what I'm reading here it seems like I'm
not the only one.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OAI/OpenAPI-Specification/issues/333#issuecomment-227444162,
or mute the thread
https://github.com/notifications/unsubscribe/AApkoffCS9a1iq3xFl5egsr64o0JDz1aks5qN-sZgaJpZM4EDVZG
.

gravesmedical on 22 Jun 2016

Just for another perspective, in our company we are thinking about supporting cases where OpenAPI can't describe what is actually needed (mainly for oneOf, I think) in this way:

We have the OpenAPI definition with its OpenAPI schema (which might be a looser approximation to what is actually allowed)
The OpenAPI schema has an extension property with the actual JSON schema
This is for now ignored by tooling, but can be read by humans (and copied for use by validators).

ePaul on 22 Jun 2016

I also have severe limitations for defining an API, which is based on the DicomWeb API (already existing, and the new de-facto standard for medical imaging). The thing is that, it doesn't matter if you think it is well or wrongly designed. I have a big problem trying to provide a Swagger definition file for it. At the end, if I cannot find a solution, I will be forced to switch to RAML or other standard that supports a full JSON schema definition.
I will put an example here about a typical DicomWeb response (again, this is already a standard, please don't suggest modifications to it):

{ 
  "00080005": { "vr": "CS",  "Value": ["ISO_IR 100"] },
  "00080020": { "vr": "DA",  "Value": ["20160602"] },
  "00080030": { "vr": "TM",  "Value": ["171855.7490"] },
  "00080050": { "vr": "SH",  "Value": ["1234"] },
  "00080090": { "vr": "PN",  "Value": [ {"Alphabetic":"Parikh MD^Anush^M"} ] }
}

So, I am having issues with two features here:

The first-level attribute names have not fixed names, but have a 8-digit pattern. This is supported in JSON Schema with _patternProperties_, but couldn't make it work in the Swagger editor (2.0)
The second-level _Value_ attribute has an array of items with multiple types (can be string, integers or an embedded object). This is supported in JSON Schema with explicit or implicit _anyOf_, but couldn't find a way with Swagger

What is really ironic is that the Swagger schema uses _patternProperties_ for describing the ˆx- and the http response codes. So, it is good when you use it, but bad for others? I think you should not feel entitled for lecturing people what is good or not from a well-known standard like JSON.

Let me know if I am wrong. I placed a question in StackOverflow for the first issue and found deceptive answers there for the second one: http://stackoverflow.com/questions/38088771/swagger-yaml-schema-definition-for-object-without-a-fixed-property-list/38091369#38091369

jaime-olivares on 29 Jun 2016

Please read my previous comment before. This is just the supporting evidence for my previous statement: the Swagger/OpenAPI language schema uses JSON Schema features that are not supported by Swagger. So, it is good for them, but a "bad practice" for us.

"responses": {
  "type": "object",
  "description": "Response objects names can either be any valid HTTP status code or 'default'.",
  "minProperties": 1,
  "additionalProperties": false,
  "patternProperties": {       <—— 
    "^([0-9]{3})$|^(default)$": {
      "$ref": "#/definitions/responseValue"
    },
    "^x-": {
      "$ref": "#/definitions/vendorExtension"
    }
  },
  "not": {
    "type": "object",
    "additionalProperties": false,
    "patternProperties": {     <—— 
      "^x-": {
        "$ref": "#/definitions/vendorExtension"
      }
    }
  }
},
"responseValue": {
  "oneOf": [    <—— 
    {
      "$ref": "#/definitions/response"
    },
    {
      "$ref": "#/definitions/jsonReference"
    }
  ]
},

jaime-olivares on 29 Jun 2016

👍1

@jaime-olivares You are right, the set of valid OpenAPI definitions can't be described with an OpenAPI schema.

It also can't be fully described using JSON Schema either, you have to read the specification text in addition to know whether it is actually valid. But JSON Schema allows a better approximation (like the one in the JSON schema document you cited).

There is a reason for this discrepance: JSON Schema is made for validation, while Swagger/OpenAPI is made for defining the API, and allowing to map the definition to constructs in (beside others) statically typed programming languages. It is not that easy to map e.g. patterned properties of a JSON object to e.g. a Java class, therefore it was left out.

ePaul on 29 Jun 2016

@ePaul, I haven't followed the entirety of the conversation so I apologize if I am missing something already covered, but the reasoning in the last comment seems odd (to me).

Swagger/OpenAPI is made for defining the API -- an API contains a number of routes which accept/return data, and the thought is that, without fully supporting JSON Schema, there are limitations to how that can that data can be defined.

The argument of "it is not that easy to map" seems like it should be outside of the this particular discussion -- Swagger/OpenAPI defines the API, whereas other tools like Swagger Codegen are the ones that should bear the burden of how easy/tough it is to map from the API to a particular language.

Regardless of easy/tough, fully supporting JSON Schema at least gives code generators a chance.

mm-gmbd on 29 Jun 2016

👍2

@mm-gmbd It has been my experience that when selecting features to include in OpenAPI that the viability of tooling support has been a significant consideration. Nobody wants a spec that is difficult to implement because then you end up with non-compliant and varying implementations which defeats the purpose of having a spec in the first place.

darrelmiller on 29 Jun 2016

@darrelmiller, good point. However, rather than limit _everybody_ because it is tough to implement for one, or some, languages, wouldn't the better (subjective) option be to support something that, again, gives code generators a chance?

Also, to contrast your comment, "Nobody wants a spec...", just based on the discussion in this thread, I can definitely tell that _somebody_ wants to have this supported because it can immediately begin more fully describing their data and types.

Is there going to be a schema (JSON Schema or not) that perfectly describes everything that makes it easy for all languages and tooling? Sounds like a unicorn. Rather than wait, maybe something is better than nothing?

mm-gmbd on 29 Jun 2016

👍4

@mm-gmbd I would like to see OpenAPI remove the direct dependency on json-schema and allow API creators to use whatever type of schema mechanism they want. That would mean you could use draft-4, draft-5, or something completely different like JSON Content Rules It would be the job of tooling to declare what type of schema formats they support. It would no longer be a concern of the OpenAPI spec. The OpenAPI spec would only be concerned with identifying the type of schema used and pointing to the instance document.

Having said this, I do not yet have a full understanding of the implications of taking that direction. I may be dreaming in technicolor :-)

darrelmiller on 29 Jun 2016

👍4

I have seen some biased/constrained opinions in this thread.

A schema that cannot be mapped to statically-typed languages is not bad at all. Not every schema has to be mapped to a POCO. That's vanilla programming. There are more complex cases outside the business software world. I state that as an industrial software developer.

Code autogeneration is not a mandatory feature of an API-First tool, as documentation or validation are not. Even in case of a tool is not able to auto-generate code (for whatever reason), an auto-generated documentation is still valuable for others to produce a customized code generation tool (I do that frequently with visual studio's T4 scripts), and the raw API spec file may be useful for validating messages in a test phase. In worst case, we also can just write code manually, by reading the API spec. That is still API-First development.

That said, I can confirm that RAML did the job with my 2 unsupported use-cases with Swagger: pattern properties and multi-typed arrays, in a very straight and concise way, so I am switching to it.

jaime-olivares on 29 Jun 2016

👍5

@darrelmiller - maybe there is a middle-ground here, although it may be tough to support. Consider a "schemaType" property that describes what schema is being used to describe the schemas throughout the API.

This would allow full support of JSON Schema v4, and the "schemaType" would be json-schema-v4, and when passing the API to a code-generator, it would know, based on the "schemaType", whether it did, did not, or partially support the schema definitions throughout the API.

That could satiate the request of this issue, while supporting other schemas in the future.

Maybe that's my technicolor dream :)

mm-gmbd on 29 Jun 2016

👍2

@mm-gmbd Many people want to support different media types, e.g. Xml as well as JSON, so they may want to use both JSON-Schema and XSD Schema in the same API.

darrelmiller on 29 Jun 2016

@darrelmiller - isn't there a slight difference between JSON-Schema and XSD Schema?

The difference I see, and maybe I'm getting it wrong, is that XSD Schema is _specifically_ for describing the structure of an XML element. Whereas, JSON-schema describes an element, whether it is just a number, a string, a boolean, or an array or object of those elements, which allows JSON-schema to describe general data structures that are not necessary Javascript Object specific.

However, to your point, maybe the middle ground is to allow the schemas throughout the API to each define what type of "schemaType" parsers should utilize (i.e. json-schema-v4 or xsd-schema), to allow for both to be present in the same API.

mm-gmbd on 29 Jun 2016

@mm-gmbd I don't really see any difference between the goals of XML Schema and JSON Schema. From the first paragraph in the JSON Schema spec,

JSON Schema defines the media type "application/schema+json", a JSON based format for defining the structure of JSON data

Open API does blur the line by introducing the xml property to allow the Schema Object to also describe xml documents, but that doesn't change the intent of JSON Schema.

darrelmiller on 30 Jun 2016

@darrelmiller, but, as I mentioned in my last comment JSON data can simply be a number, a string, a boolean, or an array or object of those primitives, which are basic constructs in several languages outside of just being "JSON data".

Maybe this is a capability of XML Schema as well? I figured XML Schema was solely for describing XML data/structures, which aren't as directly applicable to primitives/structures in other languages. I'm assuming here, so I could easily be wrong, but I think that gets the point across of what the "difference" is between the two (that I see).

mm-gmbd on 30 Jun 2016

I like the idea of a schemaType property, and would prefer to have this on the response level as my APIs support both JSON and Atom/XML via content negotiation. So I'd link to a JSON Schema for one content-type value, and to an XML Schema for the other content-type value.

ralfhandl on 30 Jun 2016

Tackling PR: #741

webron on 21 Jul 2016

I'd like to chime in as one of the people ok with a discriminator to be deterministic, but do not want a 1:1 mapping between discriminator values and definition model names.

I'm working with an API method that can return a list of connections to other people in the form of profiles (of other users on the site) or contacts (lightweight contact data that doesn't belong to a registered user).

{
  "links": [
    {"linkType": "profile", "id": "x2fx", "profileType": "creator", "name": "A Registered User"},
    {"linkType": "contact", "name": "An Unregistered Contact", "email": "[email protected]"}
  ]
}

The data works fine under a deterministic model (with linkType being a discriminator), however I do not want to be forced to rename #/definitions/ProfileLink to #/definitions/profile because that could conflict with definitions elsewhere like the definition for the full Profile model; nor do I like the idea of being forced to change "linkType": "profile", to "linkType": "ProfileLink".

Even worse, my full swagger.json is made by merging multiple versioned documents (ie: I have a v1.json, v2.json, etc... that merge to create a swagger.json exposing multiple /v#/ versioned endpoints), this works perfectly fine and definitions could even be handled by renaming references to include a version (ie: {"$ref": "#/definitions/ProfileLink"} would become {"$ref": "#/definitions/V1ProfileLink"} and Swagger would work fine. But discriminator is the only part of OAI/Swagger that is incompatible with that, because it would make it so only "linkType": "V1ProfileLink" will validate and next version data will be "linkType": "V2ProfileLink".

As a side suggestion; I can't offer an idea on how to deal with the non-deterministic models, but I did have an idea related to discriminator and oneOf/anyOf.

What about supporting something like the following:

{
  "definitions": {
    "CatModel": {
      "identifier": "cat",
      ...
    },
    "DogModel": {
      "identifier": "dog",
      ...
    },
    "Pet": {
      "type": "object",
      "discriminator": "type",
      "required": [
        "type"
      ],
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "cat",
            "dog"
          ]
        }
      },
      "anyOf": [
        {"$ref": "#/definitions/CatModel"},
        {"$ref": "#/definitions/DogModel"},
      ]
    }
  }
}

As I can tell this is still deterministic, and a little better than what we currently have.

discriminator still defines what property determines the model to use
identifier when paired with discriminator means that the values are no longer locked into model name; "cat" can be mapped to PetCat for a Pet model and mapped to CatHair in an Allergen model
As long as anyOf is a $ref to a definition code generators can still create named models in typed languages
When anyOf is used this way it's possible for a JSON Schema validator to validate this even if it doesn't know what discriminator is

dantman on 6 Sep 2016

👍5 🎉1

Just my $0.02.

First, I love standards and am very happy that swagger exists. Well done to the team. Thank you. But this issue needs to be fixed and full support for JSON schema v4 accepted.

Now, I am very happy to be corrected. But just to illustrate how really ridiculous and broken swagger is without this full support.

Because of this limitation you cannot actually use swagger to document an API that produces swagger JSON.

What??!! A JSON specification for documenting APIs cannot actually document an API that produces itself ...

An API that produces swagger JSON is just an API right? And swagger is for documenting APIs right? And giving other systems a means to interrogate machine-readable API specs via an API isn't so weird a notion. And as an API it should also be documented in the same way as the other APIs right? - in this case the schema for the response JSON is the swagger 2.x schema.

.... which fails validation against itself. Correct. The schema fails JSON that includes the swagger schema to describe a response.

So, using swagger it is not possible to correctly document an API response that produces swagger JSON. Swagger schema does not support documenting responses that are valid swagger JSON (according to swagger schema).

Like, whoever thought that was a good idea.

If the argument for not fully supporting schema is based around statically typed languages then just how do you get to deciding that the swagger schema is a special case and should be excluded from this.

I should be able include verbatim the swagger schema in swagger JSON to describe such an API response. And no, using a remote $ref reference to an external schema is not the answer to this.

Or am I totally wrong? Like I said, happy to be corrected here.

StrayAlien on 6 Dec 2016

👍1

There are some efforts in progress to support polymorphic data with oneOf and anyOf which I believe are going to appear in the next revision of the spec.

You touch on a topic that is close to my heart though. Way back in swagger 1.x I could use full json schema to validate the input and specify the swagger data model. This was mostly because swagger wasn't very ambitious about the data model spec and largely left that alone.
During the creation of swagger spec 2.0 it was decided that the spec would only be extensible via special naming convention x-blah. That was a decision that put an end to the ability to use full json schema which took away much of the descriptive power of swagger 1.x.
The decision to explicitly deny attributes that do not conform to the spec means it's unlikely you will see that full descriptive power return as they would need to include everything in the spec, much of which is outside of the things swagger should concern itself with.

I would love to see swagger in some incarnation allow extensibility in the spec without any naming convention restrictions, thus allowing one to use valid json-schema again.

paulhill on 7 Dec 2016

Thanks Paul,

I think a lot of people would like to see that. I'm not sure whether it is just me, but the idea of an API documentation specification that can't describe an API for itself is near on lunacy. It sounds harsh, but on that basis it can only really be described as a toy.

Imagine if JSON schema or XML schema could not validate themselves ...

It is unfortunate, but, where I am, because of this we have now got to drop swagger. In large organisations you get handed schemas you have to support. Not being able to even use a 'definitions' block in your own schemas is pretty lame.

It is a shame as this area of technology really needs good long-lived standards. This lack of flexibility means swagger can only get limited uptake on the large scale and thus will have limited life. And if it doesn't get full schema support it will be replaced by something that really does the job.

Like I said. A bit of a shame really.

StrayAlien on 7 Dec 2016

Validation and modeling are different beasts. JSON schema was designed for validation, not modeling. Thus supporting everything that it has for _validation_ does not make sense for _modeling_ in many cases.

3.0 has greatly increased support for JSON schema draft 4.

Decide what you want the spec to do. If it's just validate, then yes, there are constructs that may make sense outside the goals of design.

fehguy on 7 Dec 2016

Thanks fehguy. Where can I find out more about v 3.0? If there a published draft, or a repo?

StrayAlien on 7 Dec 2016

It's right here in this repo. From the home page there is a link to the branch where we're working on 3.0.

https://github.com/OAI/OpenAPI-Specification/blob/OpenAPI.next/README.md

The in-progress spec is here, if you don't feel like reading the README:

https://github.com/OAI/OpenAPI-Specification/blob/OpenAPI.next/versions/3.0.md

fehguy on 7 Dec 2016

Thanks fehguy - I did see that branch but wasn't sure I was in the right place. (though I did see your name on it which made me think I was . :-) )

The big question is ... can it describe itself? Off for a read.

If it can, what kind of release timeframe is v3.0 set for?

Thanks again.

StrayAlien on 7 Dec 2016

"can it describe itself?". No.

No apparent schema support for id keyword and a schema having its own definitions. As v3 is at the moment I still could not use it to describe an API that produces swagger JSON ....

It there a separate v3 discussion area that you could point me to?

StrayAlien on 7 Dec 2016

@StrayAlien

Imagine if JSON schema or XML schema could not validate themselves ...

From wikipedia's article on XML schema

Practical limitations of expressibility:
XSD cannot require a specific root element (so extra information is required to validate even the simplest documents). Content and attribute declarations cannot depend on attributes or element context (this was also listed as a central problem of DTD). [Therefore] It is not 100% self-describing ... even though that was an initial design requirement.

MikeRalphson on 7 Dec 2016

Hah! Oh that made me lough out loud! Very well done Mike. :-) I read this an hour ago and I am still smiling :-) Nice.

Though, that still doesn't change my feeling that a technology for describing APIs should be able to describe an API that produces those descriptions. Kinda makes sense I reckon.

I'll explain my situation: We're not talking about a guy here wanting to describe his shopping cart app. I am part of a API standards movement in a (very) large financial organisation. Part of those standards is that going forward, all APIs should have machine readable descriptions of themselves - part of a mindset move from SOAP/WSDL and a 'service enablement' of the org and mainframe systems so on. Indeed, there is a legislative requirement looming to publicly expose a number of financial APIs and that effort is now underway. This swagger limitation means that some APIs now have to be 'special cases'

What is also means is that when some part of the organisation that is designing an API using JSON schema to describe payloads hands over their interface spec it will not be able to be described in swagger. They'll have used ids, and #/definitions and oneOf/anyOf and friends (coming in swagger v3) and whatever they need to describe the payload / param structural correctness.

If we can't use those schemas in API docs then we're sunk with swagger.

It means that existing APIs to be exposed that have not taken swagger's limitations into account may not be able to be described.

Now, I have been heard many times saying "a bad standard is better than no standard". I do honestly believe that. We need standards. But even a bad standard has to actually achieve the functionality that you need or have decent workarounds. I dearly dearly want standards like swagger to succeed. But to succeed, it must be able to describe APIs - you simply cannot ask an organisation this large with such complex APIs to dumb down their objects and schemas to something swagger can describe.

It absolutely has to be the other way around. The API description tool has to fits people needs. It cannot be people working to fit the needs of the API description tool.

And again. Nice one. Still smiling at that. :-)

StrayAlien on 7 Dec 2016

@StrayAlien - putting aside the JSON Schema discussion itself, it was never Swagger's goal to support all APIs, nor is it the goal of the OpenAPI spec. We put a lot of effort into supporting _more_ features, more design paradigms and concerns, but it is unlikely we will ever support everything.

Not supporting _your_ needs doesn't make it a general bad solution, it makes it a bad solution for you. Just because a 2-seater can't accommodate a family of 7, doesn't make it a bad car.

Getting back to the topic - the upcoming version will not support the full draft, but that doesn't mean it won't be in the version after that.

webron on 8 Dec 2016

Thanks webron, I do get that. And it is good to see it evolving. Any timeframes for v3 release?

StrayAlien on 8 Dec 2016

We should be making an announcement regarding the timeline in the next few days.

webron on 8 Dec 2016

cool. Thanks.

StrayAlien on 8 Dec 2016

Validation and modeling are different beasts. JSON schema was designed for validation, not modeling. Thus supporting everything that it has for validation does not make sense for modeling in many cases.

I'm curious what is the idea behind OpenAPI regarding validation. If models were described using JSON Schema, then the same models could be used for validation. Which would make Swagger API description the single point of reference for both documentation and validation.

Otherwise one would have to maintain separate actual JSON schema for validation, which is a rather bad idea since schema and model would have to be kept in sync by hand.

IMO Swagger model descriptions should at least use a compatible subset of JSON schema (and ideally support full JSON schema).

PS considering code-generation I don't see a problem in allowing "hard" features of JSON schema, since it is already possible to create API for which generic code generation will not work (e.g. returning object with additionalProperties: true). So if code-generators are allowed not to work in that case, perhaps they could be allowed not to work when object contains anyOf, etc.

Vayu on 2 Jan 2017

👍3

Otherwise one would have to maintain separate actual JSON schema for validation, which is a rather bad idea since schema and model would have to be kept in sync by hand.

IMO Swagger model descriptions should at least use a compatible subset of JSON schema (and ideally support full JSON schema).

PS considering code-generation I don't see a problem in allowing "hard" features of JSON schema, since it is already possible to create API for which generic code generation will not work (e.g. returning object with additionalProperties: true). So if code-generators are allowed not to work in that case, perhaps they could be allowed not to work when object contains anyOf, etc.

I cannot agree more with this statement !
Currently my team cannot use swagger for our API since it is impossible to describe it without some JSON schema keywords not in swagger, this is a real shame since a one stop shop for documentation and validation would be so great !

I do really agree that if some language cannot generate code because of the way they are made, then they should juste not generate it, it seems wasteful to prevent using those keywords simply because some but not all languages are incompatible with them, different languages means different way of doing things, uniformity would be a bad thing in this case.

We should be making an announcement regarding the timeline in the next few days.

It's been quite some time but we are just out of the holidays, any news on that ? Is there a way to help drive this forward ? Since it seems most people agree now to add those keywords, what is the blocker currently ?

If possible we'd like to make this happens quickly as then the tooling needs time to adapt.

morsok on 3 Jan 2017

👍3

Hello, I'm the editor of the most recent update to the JSON Schema Internet-Draft. @Relequestual and @handrews are some of my partners in crime. Of course JSON Schema is quintessentially for validating JSON, but that's so boring: perhaps our top use case for JSON Schema is actually in Web APIs (it's certainly my focus).

The two important features around that are, first, hypermedia (defining link relations between documents), and second, document submission to Web APIs: letting a Web server declare the requirements that a submitted document must follow, so that clients can validate their submissions before the server ever needs to reply with an error message.

Well, one of the interesting things about using JSON Schema to define what's acceptable for submission, is first you have to _construct_ a document to submit. So the validation keywords are really a way of expressing _useful_ constraints on input that can be used for building forms and other tools for interacting with data, and it's fully anticipated that validation keywords will used for these sorts of things.

From a user interface perspective, defining a field with {"type":"number", "minimum":0} would trigger a numeric keyboard and only allow non-negative number input.

From a computing perspective, you can use these assertions to optimize database indexes, or generate client libraries like Swagger does.

Further following from this, I think it's also totally reasonable to allow a profile of JSON Schema (a subset) that makes sense for the application. For example, allowing "anyOf" would in many cases require falling back on behavior that's just as useless as {} -- allowing any possible value whatsoever.

In many cases, though, most of the keywords should be implementable, even if only at runtime. For example, "patternPropeties" would likely have to generate errors at runtime ("Error! Key foo does not match required pattern ^[0-9a-f]{40}$").

awwright on 31 Jan 2017

👍1

Given how long this issue dates back, there's bound to be many things which have changed.
I suggest this issue is locked, but not closed, and new issues created to target SPECIFIC key words as needed.

It's clear (and has been for a long time) that Swagger / OAI will not fully support JSON Schema (like RAML does) for the foreseeable future. I totally understanding their reasoning, and that's OK. The problem however, even if you ignore many issues, as the original creator of the issue points out, many libraries use existing JSON Schema validators, and therefore do not conform to the OAI specification.

I've suggested before, a few times, the solution to this problem is to provide a comprehensive official test suite for this purpose, which allows you to confirm compliance or not. Heck, you could even base it off our existing JSON Schema test suite at https://github.com/json-schema-org/JSON-Schema-Test-Suite

Relequestual on 1 Feb 2017

👍1

@awwright wrote:

For example, allowing "anyOf" would in many cases require falling back on behavior that's just as useless as {} -- allowing any possible value whatsoever.

Allowing any JSON possible value is not useless at all. We actually need that feature in our API where some API routes are just a gateway to another system that will validate the value asynchronously. This is also necessary to define API that are JSON document stores (should allow to store/retreive any raw JSON value without validation).

dolmen on 17 Feb 2017

To summarize what changed in 3.0.0:

We moved to the latest Wright Draft 00 (aka, Draft 05).
Added support for oneOf, anyOf, not.
Dropped the file type.
Added clarifications to the differences between what's supported by the spec and what variations are in it.

Thanks everyone who participated in this discussion.

webron on 2 Mar 2017

🎉16 👍15

oneOf can be used inside any schema object, including those under components/schemas.

MikeRalphson on 24 Nov 2018

Openapi-specification: Make schema object compatible with JSON Schema draft4

Most helpful comment

All 131 comments

5.5.1. enum

5.5.1.1. Valid values

5.5.1.2. Conditions for successful validation

Related issues