Openapi-specification: Version Enhancement to Compatibility Strategy

[srvance]

The current OpenAPI Specification includes a field version in the Info Object that is a required string containing a version number for the API. Currently the specification does not state the semantics or syntax of the structure of this string. Community discussion has indicated a preference for semver format and semantics.

Versioning is really just an obvious, intuitive, and commonly used approach to the deeper issue of API compatibility. From that perspective, versioning is just one of several possible compatibility strategies. There are several commonly used approaches to versioning. Many in the hypermedia community feel that we should not version resources, although that pushes potential versioning into the representation. Since the OpenAPI specification describes both the URL and the payload, representation compatibility strategies should be addressed, as well.

I would like to propose that the current info.version be elaborated to a compatibilityStrategy object functionally similar to the Security Definitions and Scheme Objects. To maintain compatibility and for forward migration, the default strategy would map to the equivalent of the current version definition.

An example might be

{
    "openapi": "3.1.0",
    "info": {
        "title": "MyAPI",
        "compatibilityStrategy": "semver"
    },
    ...
}

The details of compatibility strategies and their corresponding objects needs to be elaborated.

Other strategies might include “simple”, “date”, or “migration”. The “simple” strategy might be a single-digit strategy based on a major version or a sequence number as is most commonly represented in URL-based API versions. The “date” strategy is fairly self-evident and might address use cases like Stripe. The “migration” strategy describes an approach I am developing that obviates the need for versioning based on a limited compatibility period.

I would also like to propose that compatibility strategies could apply at smaller granularities than the API as a whole, most notably at the level of the payload entity defined in the Schema Object. This proposal will also impact the evolution of the “deprecated” fields and should be coordinated or merged with issue #782.

[fehguy]

While I certainly have opinions about versioning and versioning strategies, it is far too difficult to force people into a single or set of versioning strategies. Hence the @OAI/tdc did decide collectively to not pick one, nor propose a set of them. It is open-ended intentionally.

[srvance]

Part of what I'm trying to address here is that the spec narrows the solution space to "versioning strategies" not the broader space of "compatibility strategies" of which not all involve versioning. Additionally, issue #782 suggests that deprecation should be more tightly coupled to versioning as the only compatibility solution.

[srvance]

@fehguy Let me give two compatibility strategies that don't fit into the concept of a single global "version".

Take "hypermedia" for the first, understanding that OAS can't fully describe it yet. With hypermedia, you have changes that warrant a new resource and representation, and changes that evolve an existing representation without warranting a new resource. For the former, hypermedia deals with it by providing the link relations to the new resource, so no versioning is necessary, especially with the use of forms and a shared ontology. For the latter, the versioning problem is pushed down to the level of the representation. In either case, a single global version is unnecessary or insufficient.

For a "migration" strategy, you leverage the property that any incompatible transformation from API A to A' can be decomposed into no more than two transformations, the first of which is mutually compatible and the second is breaking. This allows you to apply the mutually compatible half of the transformation until you have passed the need for compatibility, and only then apply the breaking half of the transformation. This renders global versioning irrelevant or unnecessary while conceptually pushing the compatibility tracking all the way to the atomic level in a way that should not be managed by OAS.

In both of these cases, deprecation needs to be tied to the management of compatibility, not to a global property.

The security schema already sets a precedent for a pattern for representing multiple strategies in OAS. I propose we apply a similar pattern to compatibility strategies, ideally in an extensible way. This allows for the specification of more detailed versioning strategies that could be automatically managed and understood as well as accommodating non-versioning compatibility strategies.

[fehguy]

Then the design of your API should be encapsulated into a version concept, which may include "evolving" definitions of responses as defined in your hypermedia strategy. It really doesn't change the version strategy that we've outlined.

Adding more "types" for versions is an unbounded problem as you point out. There will be no value in OAI trying to constrain them to a fixed set. The smartest move would be to say "versioning strategies are up to the API provider" imho.

[srvance]

All versioning is a compatibility strategy, but not all compatibility strategies are versioning. Saying these approaches "should be encapsulated into a version concept" inverts the generalization relationship, and a single global version doesn't even cover the space of practical and useful versioning strategies adequately.

I'm not saying they should be constrained to a fixed set, but that the typical set should be well-known and defined in a way that is machine-manageable with provisions for extensibility.

In my understanding, this is how the security schema is defined. How is this different?

[asbjornu]

Labelling a strategy for how you want to ensure backwards compatibility and evolvability in your API as "versioning" is oversimplification.

This space is complex and consists of gradients, where each byte we change in our API surface as well as how it operates under the hood, which states it has and how these are communicated to the client, how the states transition between each other, etc., need to be thoroughly examined and analysed before being implemented.

Where "compatibility strategy" attempts to describe this as the fuzzy and complex space it is, "versioning strategy" labels it as black and white; either you have a new version or you don't. It oversimplifies the problem space and does not allow for the fine-grained compatibility strategy @srvance describes to be communicated and perhaps even inhibits its implementation.

[fehguy]

I think my opinion is pretty clear on this, perhaps @OAI/tdc members have other inputs?

I consider there to be virtually zero value in having an open-ended compatibilityStrategy. Even if we constrain it, it's an unconstrained problem. It'll never be something that can be described let alone enforced by any sort of tooling.

For that, I suggest using extensions to the spec to add more information where you feel the spec is falling short. Remember this spec is not trying to solve all problems but as a first said, the @OAI/tdc may have other input on it.

[asbjornu]

@fehguy: The problem is that the current spec requires the version field, which severely limits the strategies implementers can employ to ensure API compatibility. If it could be made optional, with extensibility in mind, I think everyone would be happy. @srvance has already suggested how that might look like. Here's another example:

{
    "openapi": "3.1.0",
    "info": {
        "title": "MyAPI",
        "compatibility": {
            "strategy": "semver",
            "version": "1.0.1"
        }
    },
}

Here's how that can be extended with something outside of the specification:

{
    "openapi": "3.1.0",
    "info": {
        "title": "MyAPI",
        "compatibility": {
            "strategy": "http://webconcepts.info/concepts/link-relation/"
        }
    },
}

This way, the compatibility field as well as its subfield strategy can be made required, but different values of strategy can require different sibling fields to exist (or not). I hope you get the idea.

[fehguy]

eh we're not seeing eye-to-eye. I'm hoping someone from @OAI/tdc can help because it just doesn't make sense to me, unless you want to imply (and I'm borrowing this quote) that "there is only one version, and that's the latest" in which case, yes, version is a useless field.

If that were the case--and I hope it's not--just set your version: 'n/a' which is technically legal with the spec. Problem solved!

[asbjornu]

Versioning is one of many possible compatibility strategies. If you choose to not version your API, it will indeed not have a version number. It might then be that "there is only one version, and that's the latest".

While version: 'n/a' could work, it is more information about what the compatibility strategy of the API is not than what it actually is, which I find rather useless. If a field is required, it would be nice if it could have values that makes sense for more than one use-case. If quite a few use-cases renders the field useless, I'd vote to make it optional or to design a new field that embraces both the covered and the previously uncovered use-cases.

[srvance]

Not only are there compatibility strategies where a global version is irrelevant, those strategies set different expectations around compatibility that are useful to be communicated. "Hypermedia" indicates it may only be an issue in the individual representations, not in the API and all its endpoints as a whole. It also tells you something about the mechanism by which compatibility is provided. "Migration" indicates that you will receive deprecation warnings at fairly small granularity that you will need to respond to. It may set an expectation around response time, but under that strategy, there are no version labels. This is also why it compatibility strategy needs to tie into the deprecation tagging.

@darrelmiller and I spoke about this at API Strat in Boston. His response seemed positive and he suggested I create the issue in the first place.

[fehguy]

ok perhaps when he has time he can lead this. I just don't think there's a practical solution within the goals of the OAI to what you're describing, but if you talked in person he probably gets it better than I do

[darrelmiller]

For better or worse, using a version identifier for an API is by far the most commonly expected declaration of compatibility. I would love to see better approaches to describing compatibility that could drive tooling, but without specific examples it's hard to make the case for adding complexity to the spec. The examples of semver, simple, date are further precision on the format of the version, not really an alternative to versioning. This information could be added by including a "x-versionFormat".

I don't think we help the average user by adding a layer of indirection to declaring a version. I think this area should be explored via specification extensions first and based on adoption we can consider enhancing the specification for more flexibility.

Security is one of those areas where the industry already has a wide range of schemes in practice and therefore it makes sense to model what is currently in common use. A version identifier is an adequate solution for 95% of users at this point in time. Having said, that, I would love to see exploration of other approaches to version compatibility. I just don't think the OpenAPI spec is a place to pioneer new approaches.

[handrews]

For better or worse, using a version identifier for an API is by far the most commonly expected declaration of compatibility.

That's a reason to support it, not a reason to require it.

[srvance]

Lets separate the problem being highlighted from the solution to that problem being proposed. If the version were called revision and described as "the revision of the API description," we wouldn't be having this discussion. I don't think anyone would argue with the value of having a unique identifier for each published revision of your API specification that carries with it some indicator of its place in a sequence. It would be much easier for OAI to define the contents of the field without controversy, and it would serve the machine-readable versioning needs of the tools space.

The problem arises because the field claims to be the version of the API itself and is required with its only extensibility being the avoidance of an internal format specification or of semantics. This makes it useless for cross-tool machine-readability and only partially useful for human consumption, requiring external documentation to properly interpret. Any further features built on it will suffer from the same ambiguity and inconsistency.

As @handrews points out, the prevalence of versioning of APIs means that the standard needs to support it. However, the fact that a significant population of API thought leaders think that versioning is a poor approach to compatibility should be a strong reason not to lock the standard into that singular solution, regardless of whether we can yet enumerate many alternatives.

The alternatives presented provide motivation to not lock the standard to versioning. If you don't like them, that's fine, but you also shouldn't ignore them if you want this standard to become what the industry needs.

[fehguy]

If version were called revision then we'd have 2 happy people and over a thousand unhappy ones. I suggest using extensions as @darrelmiller said. The concept of version is pretty widely shared (there are 10's of thousands of people using the spec with version required) and I do not see a large number of people asking for us to support what you're asking for.

[darrelmiller]

Because the specification doesn't not define what version looks like, the only thing tooling can do is identify that one OpenAPI definition is a "different version" than another. It is just an identifier. If you want to treat that identifier as a kind of revision that doesn't make any statement about compatibility, then go for it. The fact that semver even exists is a statement that versions don't naturally make compatibility statements.

If you don't want to use version, then leave it blank. Personally, I would be fine with making it not a required field.  The version attribute isn't required for a spec to be useful.

[srvance]

@darrelmiller How would you feel about defining it as the version of the API description, not of the API itself?

The word version carries connotations that steer users and tools and block alternate approaches. I understand if you don't want ideas you don't feel are fully mature to be represented in the standard yet, but do you really want to prevent them?

Frankly, @fehguy, lots of people feel the pain of versions, but few understand that there are alternatives and what they are. The ones who do usually get drowned out by the ones who insist there aren't practical alternatives or don't want to be bothered. Millions of people thought the world was flat for hundreds of years, and look where we are now. Mass belief is not a strong argument of anything other than mass belief and an acknowledged current perceived reality.

Is there any way to put it to the thousands? Counting reactions, there are five in this thread in favor of the proposal and two against.

[fehguy]

OK i'm stepping away from this issue :) I don't need to get into a tabs vs. spaces religious war, I've stated my position and I remain unconvinced to change it even after further comments. You have so many ways to work inside the spec for this I don't understand why this is still being debated. Put the damn revision in your api description or as a vendor extension. Done. We can all move on

The "millions of people thought the world was flat" is a personal first for me, at least in GitHub, so thanks for that.

[webron]

Regarding the original suggestion, it's unclear to me what tooling would benefit from that information.

At this point in time, arguing over the semantics of calling the field revision vs version but keeping it pretty much the same content - really pointless. Not saying it can't be changed, just don't think the semantic difference is important enough to do so now.

However, the main thing is who do we really benefit and why. The strongest case that was brought here was that people should learn to work better, not use versions and they do now and so on and so forth. I don't think it's our mission to educate people, especially not about things that are opinions and not fact. Sure, we recently chose to not support payloads for GET, DELETE and so on, and you can say that it's an educational decision, but it's not. In this case, while "a significant population of API thought leaders" may suggest that, in practice, it is not being used so often, and it may never be used that often. If we support it and it's not even further adopted, we ended up making the spec more complex, making it more difficult to adopt by those who just want to... version.

As for making it required or not - we can debate that. Since we're close to releasing an Implementer Draft, I'm inclined to say no.

Overall, I'm not saying 'no' to the suggestion, but rather 'not now'. Suggest pushing it for review for a future version.

[asbjornu]

@webron:

At this point in time, arguing over the semantics of calling the field revision vs version but keeping it pretty much the same content - really pointless.

That is not what is being suggested. What's being suggested is to introduce a new field called revision that would mean something entirely different from version. The version field is currently described as such:

Provides the version of the application API (not to be confused with the specification version).

While the suggested revision could be described as:

Provides the revision of the API description document. Not to be confused with the specification version.

While that description could be used for the version field as well, I think developers are hard-wired to think that version means "the version number of my API (library)". There's also the the fact that the current version field is described to mean exactly that as well.

If the version field semantics stay unaltered and the field remains required, OAI is going to be less useful (or even useless) for APIs that have different compatibility strategies. The options I feel we're left with that can still fix the issue are:

1. Change the version field semantics to apply to the API description document instead of the API itself.
2. Rename the field to revision and make it apply to the API description document instead of the API itself.
3. Keep the version field semantics as is, but make the field optional.

Of these options, I would prefer the two first, since having a version number for the API description document is valuable regardless of what compatibility strategy you employ.

[darrelmiller]

@asbjornu You bring up a distinction that I've never considered before. I like to think of the OpenAPI definition as the "interface" and the API on the backend as the "implementation". The Version property is the version of the "interface". I would never consider exposing to consumers the version of the "implementation", that just seems unnecessary and confusing.

Perhaps a solution to this problem is to make it clearer in the specification that the version property is an identifier for definition of the API. Unless of course I am wrong about that.

[srvance]

@webron Regarding tooling, even with some elaboration of version semantics, tools could improve the compatibility story. It would provide additional information for static analysis of the definition.

For example, semver has pretty well-defined rules about what constitutes which level of version rev. An OAI authoring tool could easily inform a user if a change they were making warranted a major, minor, or patch revision. Similarly, consumption of an OAI spec gives you machine-actionable information about whether compatibility is expected. You can only do that if you know the rules and semantics of the scheme being used.

For non-versioning approaches or approaches that need versioning at different levels (see the "hypermedia" ideas above), it lets the tool enforce change management at the appropriate level and in the appropriate way.

[asbjornu]

@darrelmiller:

You bring up a distinction that I've never considered before.

I'm glad you see it, because it is an important distinction, I think. 😃

I like to think of the OpenAPI definition as the "interface" and the API on the backend as the "implementation". The Version property is the version of the "interface".

That's not how I interpret it.

I would never consider exposing to consumers the version of the "implementation", that just seems unnecessary and confusing.

That's how most people implement versioning in "Web" APIs today; by plunging a version number into the URI of all of the resources. That is most definitely the "implementation" and not the "interface". By not being explicit about this, I strongly believe implementers are going to equal these two concepts and use the same version number in OAI as they do in their URIs.

Perhaps a solution to this problem is to make it clearer in the specification that the version property is an identifier for definition of the API. Unless of course I am wrong about that.

Being explicit is good and important. Choosing the right words and semantics also is. To avoid having to redefine 50 years of existing baggage in the software industry related to the term "version", I think "revision" might be better. Being explicit that it applies to the "interface", "definition" or "description", however you choose to word it, would also help.

Explaining that the version property applies to the physical OAI document itself and not the API implementation, is an important distinction that I'm not sure everyone here agrees on and that I'm certain most people reading the specification is going to misunderstand.

[webron]

We've modified the wording of the version property, but decided not to extend it with compatibility and such.

[srvance]

That's much better. Thank you. Do you know how the reference tool implementations will use it?

[asbjornu]

If I found the right document, that's much better indeed. Here's the new wording I found:

Required. The version of the API definition (which is distinct from the OpenAPI specification version or the API implementation version).