Autorest: Plan to support OpenAPI 3.0

[hellosnow]

A quick questions on whether we have plan to support OpenAPI 3.0? Thanks!

[olydis]

We are indeed discussing it and weighing it against other priorities - the main question is what OpenAPI 3.0 would enable us to do that wasn't possible before. It comes with great features, but many of them already have workarounds in Swagger 2.0 (x-nullable vs nullable, x-ms-parameterized-host vs servers, x-ms-skip-url-encoding vs allowReserved, etc.). Having a standard way to do stuff is obviously awesome and desirable, but given our resources, our efforts are mainly driven by enabling much required features rather than "refactoring" what was already possible before. 😐

Currently, one of the main features that could drive us towards OpenAPI 3.0 seems to be oneOf/anyOf, as we have encountered some popular scenarios that are not modelable in a satisfying way without those. Then again, it may be cheaper for us to bring x-oneOf/x-anyOf to Swagger 2.0. The thing is that once we advertise support for OpenAPI 3.0, we should support all of it.

So do we "plan to support OpenAPI 3.0"? Sure, but it is not a top priority. Driven by influx of customer scenarios and wishes, that priority may of course suddenly change 😉 This issue shall hereby be our way of communicating any changes to our plans/priorities and collect customer feedback!

[hellosnow]

@olydis get it, sounds like Azure REST APIs will maintain in OpenAPI 2.0, before AutoRest supports 3.0?

[olydis]

correct :-) in fact, for the Azure REST APIs repo, AutoRest isn't the only limiting factor, I think there is some other tooling consuming these specs, we'll have to make sure everything/everyone is ready before allowing 3.0 in that repo or even convert over

[hellosnow]

@olydis get it now, thanks for the info!

[olydis]

use cases encountered that call for OpenAPI 3.0

oneOf

• some services return errors of different shape, but under same status code, with no discriminator (technically: this is often because ASP.NET may complain before even reaching the service, also the service may complain, so different models can show up on the same endpoint)
• data factory has scenario of "expression" (string) vs object (=> primitive types obviously don't have a discriminator field, no way to model right now)

[jonschoning]

Request/Response bodies can vary by media type in OAS 3.0.

https://swagger.io/docs/specification/describing-request-body/

    "requestBody": {
      "description": "Pet object that needs to be added to the store",
      "content": {
        "application/json": {
          "schema": {
            "$ref": "#/components/schemas/Cat"
          }
        },
        "application/xml": {
          "schema": {
            "$ref": "#/components/schemas/Lizard"
          }
        }
      },
      "required": false
    },

[stevengum]

Use case for OAS 3.0 Feature: Callbacks

To reduce the efficiency drain from constantly polling the status of operations, it would be better to set up a webhook to update the client when an operation is complete. Some services have monthly quotas of transactions that a user can perform, and polling detracts from the monthly quota with no real positive to the user (perhaps 5 requests are sent, but only one will say if an operation is complete). Related to this, webhooks are becoming more common either as standalone offerings, or coupled with a regular REST API.

---

OpenAPI 3 Snippet on Callbacks:

_In OpenAPI 3 specs, you can define callbacks – asynchronous, out-of-band requests that your service will send to some other service in response to certain events. This helps you improve the workflow your API offers to clients._

_A typical example of a callback is a subscription functionality – users subscribe to certain events of your service and receive notification when this or that event occurs. For example, an e-shop can send a notification to the manager on each purchase. These notifications will be “out-of-band”, that is, they will go in other connection than the connection in which a visitor works, and they will be asynchronous as they will be out of the regular request-response flow._

_In OpenAPI 3, you can define the format of the “subscription” operation as well as the format of callback messages and expected responses to these messages. This description will simplify communication between different servers and will help you standardize use of webhooks in your API._

[alexeldeib]

Any update here, looks like this is still deprioritized? Have run into several situations where it would be useful to have OpenAPI 3 for client generation -- notably primitive return types with multiple possible values (oneOf) and serving the same API shape via two hosts with their own base paths (servers).

[fearthecowboy]

We've done a bunch of work under the covers to support OAI3; unfortunately, your primary ask is literally the thing that's the most challenging to implement (oneOf).

With @olydis having moved on, I'm not likely going to get to this for a while yet.

[alexeldeib]

Thanks for the update Garrett -- Awesome to hear that, wasn't sure if this was underway at all so great to know it's on the road map (if not anytime soon).

[anseljh]

@fearthecowboy Anything the community can do to help move toward OpenAPI 3.0? I'm building a new API now, and would really love to have good tooling like this around a 3.0 spec, instead of using 2.0. Thanks!

[fearthecowboy]

At this point, to accept the format (using no OAI3-specific features), I have to write a new loader pipeline. It's probably only about three days work, but it's queued up behind the critical work that I'm doing now. I'm quite sad I didn't get @olydis to do it before he left.

[amilos]

@fearthecowboy, We are building .NET based solution for PSD2 and we have been relying on Autorest parser and generator. The Berlin Group as the most prominent PSD2 API standardization effort in Europe recently published their spec in OAS3 format. Given the number of countries backing this specific API standard, hundreds of banks are going to search for a REST API solutions compatible with OAS3.

It would be a really bad thing for those on .NET ecosystem to miss out or have to switch to Java in order to meet the regulatory deadlines.

How can we help accelerate the support for OAS3 in Autorest? Is such support realistic in next 4-5 weeks?

[fearthecowboy]

I have a new hire starting soon; it's going to be his first task to enable the OAI 3.0 support in autorest.

The first cut won't have support for anyOf or oneOf, but the format should be able to be processed.

[nazneen84]

We are developing APIs right now and want to make a decision on 2.0 or 3.0. Please could we know what the timelines look like?

[shwetashukla]

Even I am looking for timeline on upgrade to 3.0

[klacol]

OAI 3.0 would be great for AutoRest, The support by other tools is getting better (e.g. we now can import OAI3.0 specs into Postman), so new API's are likely to be 3.0.

[fearthecowboy]

Initial Support for OpenAPI 3 is in the AutoRest 3.0 beta

To Upgrade to the beta:

npm install -g autorest@beta
autorest --reset

[stevengum]

@fearthecowboy is there a repo with a changelog-type file so that developers can see what's available in the current beta?

Edit: This is a beta fresh off the press, but a roadmap (if available) would also be fantastic 😄

[fearthecowboy]

No; I haven't done a changelog for it yet; so much coding, so little time 🍺

Mainly:

• we rewrote a large part of the way that AutoRest loads files and resolve $refs , which allowed us to also load OpenAPI3 files.
• some changes to allow Azure ARM specs to load all the specs for all the api versions, and then do processing on EVERYTHING.
• enabled --powershell generator
• enabled --interactive to assist in debugging the internals of AutoRest. (very cool to just watch it run :D )
• refactored the repository layout quite a bit, using rushjs to build as a monorepo
• we have a couple of pipeline steps that can radically optimize schemas (deduplication and subset-model-reduction) -- generators have to be updated to take advantage of those, so the PowerShell generator is the first one to use it. (the first deduplicates schemas that are identical, replacing them with a single common schema, and the second does analysis to find out if it can reduce duplication between schemas by making some inherit from others. )
• Moved to Node10 -- should be a bit faster than before.

It's not a lot of new visibile features, but a hell of a lot of new code so we can implement a lot of new features.

[henrikwh]

Initial Support for OpenAPI 3 is in the AutoRest 3.0 beta
To Upgrade to the beta:

npm install -g autorest@beta
autorest --reset

I am trying to run with the beta in order to generate from OAS v3:

autorest --input-file=https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml --output-folder=./generated/code --csharp

But I get this error:
Process() cancelled due to exception : Incompatible paths conflicting: /paths/path:0.get / Error: Incompatible paths conflicting: /paths/path:0.get

Looking forward to OAS v3 support in autorest!

[fearthecowboy]

Ah, I see.... I may have broken the csharp generator with a fix I did yesterday to fix something in powershell.

Lemme see if I can fix that up this morning.

[fearthecowboy]

Fixed it.

Reset it (autorest --reset) and try again.

[fearthecowboy]

Fixed it.

Reset it (autorest --reset) and try again.

[kant2002]

@fearthecowboy I try to run using autorest@beta

autorest --input-file=https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml --output-folder=./generated/code --csharp

and experience same error as #3016 which caused by using schema referencing in the responseBody. When I explicitly put object schema in the response definition, the generated CSharp still incorrect, since it attempt to pass all parameters do dictionary assuming that they are strings. Which obviously incorrect. They are could be different numbers or boolean for example.

[woodnic]

OpenAPI 3.0 was released 2 years ago and AutoRest (stable release) still doesn't support it?

(We can't use beta releases in our production environment)

[RenFontes]

So... autorest doesn't work for .net core 3.1 + swashbuckle because there is no support for openapi 3.0.

There also seems to be no way of generating a .net core 3.1 client with credentials when using openapi 3.0 since no other generator seems to support it.

Not sure why openapi 3.0 support was not considered important since it was kinda obvious more and more tools would make it the standard in the future. 😶

[scp-mb]

@RenFontes You can generate a V2 version under the current 5.0.0-rc5 release of swashbuckle with app.UseSwagger(o => o.SerializeAsV2 = true);

There are some issues outstanding with that, such as enum types not generating properly but workarounds are available. https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/1329

[kant2002]

OpenAPI 3.0 works for me when I add custom transformation

directive:
- from: swagger-document
  where: $.components.schemas.*.additionalProperties
  transform: |
    return typeof $ === "boolean"
      ? ($ ? { type: "object" } : undefined)
      : $
  reason: polyfill
- from: swagger-document
  where: $.paths.*.get.parameters[?(@.schema.type === "array")]
  transform: |    
    if ($.style === undefined) {
        $.style = "form";
        $.explode = true;
    }
  reason: polyfill

that way no need for force V2 version.

[rolfec]

You may consider the following open-source alternatives before AutoRest officially supports OpenAPI v3:

• NSWag: https://github.com/RicoSuter/NSwag
• OpenAPI Generator: https://github.com/OpenAPITools/openapi-generator

[anderejd]

@rolfec I can recommend NSwag, it's working great for me, using it for Typescript and C# generation.

[kostyrin]

do you have a roadmap? when will be the autorest 3 in release?

[RenFontes]

OpenAPI 3.0 works for me when I add custom transformation

directive:
- from: swagger-document
  where: $.components.schemas.*.additionalProperties
  transform: |
    return typeof $ === "boolean"
      ? ($ ? { type: "object" } : undefined)
      : $
  reason: polyfill
- from: swagger-document
  where: $.paths.*.get.parameters[?(@.schema.type === "array")]
  transform: |    
    if ($.style === undefined) {
        $.style = "form";
        $.explode = true;
    }
  reason: polyfill

that way no need for force V2 version.

How do you use a custom transformation?

[kant2002]

use-extension:
  "@microsoft.azure/autorest.typescript": "2.6.0"
  "@microsoft.azure/autorest.csharp": "2.3.82"

use-datetimeoffset: true
license-header: NONE
input-file: Application_private-v1.json
directive:
- from: swagger-document
  where: $.components.schemas.*.additionalProperties
  transform: |
    return typeof $ === "boolean"
      ? ($ ? { type: "object" } : undefined)
      : $
  reason: polyfill
- from: swagger-document
  where: $.paths.*.get.parameters[?(@.schema.type === "array")]
  transform: |    
    if ($.style === undefined) {
        $.style = "form";
        $.explode = true;
    }
  reason: polyfill
csharp:
  output-folder : output

this is example. A lot of stuff just regular directives. I want to show where I put these snippet. other things is not important, you can have your own .

[fearthecowboy]

We’re getting awful close to throwing the switch on AutoRest 3.0, and publishing it as the default.

To install the autorest and test it:

npm install -g  https://github.com/Azure/autorest/releases/download/autorest-3.0.6163/autorest-3.0.6163.tgz
autorest --reset

To go back to the published version

npm uninstall -g autorest
npm install -g autorest 
autorest --reset

This version should work for existing scenarios (and falls back to v2 core for that) , and supports v3 scenarios and generators.

Read the Release Notes : https://github.com/Azure/autorest/tree/v3#new-autorest-version-30
Also read about autorest-versioning : https://github.com/Azure/autorest/blob/v3/docs/autorest-versioning.md

If you have any problems with it, let me know ASAP

[fearthecowboy]

@RenFontes _How do you use a custom transformation?_

create a configuration file (config.yaml) with the configuration:

---
use-extension:
  "@microsoft.azure/autorest.typescript": "2.6.0"
  "@microsoft.azure/autorest.csharp": "2.3.82"

use-datetimeoffset: true
license-header: NONE
input-file: Application_private-v1.json
# directives are the customization work
directive:
- from: swagger-document
  where: $.components.schemas.*.additionalProperties
  transform: |
    return typeof $ === "boolean"
      ? ($ ? { type: "object" } : undefined)
      : $
  reason: polyfill
- from: swagger-document
  where: $.paths.*.get.parameters[?(@.schema.type === "array")]
  transform: |    
    if ($.style === undefined) {
        $.style = "form";
        $.explode = true;
    }
  reason: polyfill
csharp:
  output-folder : output

and run it autorest-beta config.yaml

[roshanr10]

@fearthecowboy is there an official ETA for moving to AutoRest 3.0?

[woodnic]

@roshanr10 you can track progress in this PR: https://github.com/Azure/autorest/pull/3348.

The first commit to the branch v3 was https://github.com/Azure/autorest/pull/3348/commits/e0a6ccdf9928e2cf9ee80c6e500919f23a4a36c5 (Jun 20, 2018). Hopefully it won't take more than 2 years to have this branch merged into master.

[woodnic]

@rolfec @anderejd nice suggestion!! Both openapi-generator and nswag are very good alternatives.

[fearthecowboy]

You can install the RC version

npm install -g  https://github.com/Azure/autorest/releases/download/autorest-3.0.6173/autorest-3.0.6173.tgz
autorest --reset

see the release notes

and read about autorest-versioning

[OzBob]

errors when running --v3 OAS petstore example: #3464

[RaviShankarPandey]

will this work for java as well?