Apisix-dashboard: [Proposal]: Import & Export route from OpenAPI Specification3.0

[liuxiran]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• [x] Feature or performance improvement

Refer to https://github.com/apache/apisix-dashboard/issues/611

---

Why

Sometimes user's apis have already defined based on the openapi specification and when they want to publish those apis on APISIX, they need to learn the concepts of our products, such as how to do parameter validation, how to do authentication, how to do path matching and so on. Actually, many users don't care how to publish a route, so If we can support importing the configuration of openapi to generate the route, that would reduce the user's use cost.

How

• add buttons for import and export in route page
  import button for upload the profile of route based on the openapi 3.0, and the export button for download it.

• manage-api parse the file, generate an apisix-compliant data structure, and write to the etcd

[LiteSun]

I think this feature will save a lot of time for users who need to import a lot of APIs.

According to the above description, the workload on the front-end is relatively small, and the challenge is to convert the profile into an apisix-compliant data structure.

let's cc @nic-chen @gxthrj for more suggestions.

[nic-chen]

@liuxiran @LiteSun

I think we need a detailed example of the imported and exported data so that everyone can understand the details more easily

[gxthrj]

OpenAPI is a description document used to provide quick reading and testing. The route in APISIX is also used to manage API, there are still some differences. For example, we need to consider how to convert the plugin logic in route.

We'd better limit the import / export responsibility boundary and provide a more detailed transformation example.

[liuxiran]

OpenAPI is a description document used to provide quick reading and testing. The route in APISIX is also used to manage API, there are still some differences. For example, we need to consider how to convert the plugin logic in route.

Yep, OpenApi is mainly defines some of the contents of the api itself, including: name, description, service-side address, method, path, request params, response params, authentication and authorization , etc, example configurations can refer to [1].

In addition to the basic content of the api mentioned above, Route defined in APISIX also contains powerful extension properties, these need to be done by configuring different plugins. The configuration of these plugins are not currently supported by OpenApi except for Authentication plugins.

We'd better limit the import / export responsibility boundary and provide a more detailed transformation example.

The limit I think would be as follows:

1. Import and export only support the basic information of the api: name, description, method, path, request parameters, response parameters, authentication, those are supported by OpenApi, and also the parameters required by APISIX.

2. Upstream info, which OpenApi cannot describe very specificly; and other plugins info, which cannot supported by OpenApi, I think we would use the service module to achieve this part of functions. As a kind of draft, we can supplement the OpenApi with something like serviceId field corresponding to APISIX's service_id[2], before user import their apis, they needs to create a service in APISIX, which can include upstream and necessary plugins, and add the service_id to their OpenApi configurations.

cc @gxthrj @nic-chen @LiteSun

Reference：

[juzhiyuan]

Here is the proposal for Route Export & Import feature but still under discussion.

ping @moonming

[juzhiyuan]

Once this proposal and discussion are completed, we could remove the discuss label.

[liuxiran]

For the upstream info or service info which are not supported by OAS3.0, there are two feasible options of implementation scenarios:

• Expand OAS3.0, increase upstream and service, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].

  • advantages:
    when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
    we can support batch operations.
    this also can be done only as a command-line tool.

  • disadvantages:
    Users need to modify the existing OAS refer to our doc.
    upstream and service are the only necessary expansiones, if we want to support other plugins by import operation, we also need to define new expansion.

• Combine with existing create route page, when user create a route, they can import a OAS file, which provides parts of the route's info, and for the upstream 、 service and other plugins, they can follow the page step.

  • advantages:
    We don't need to expand OAS3.0.
    It can support all features supported by APISIX

  • disadvantages:
    We need to modify the current UE of create route to support this feature.

Reference:
what do you think?

cc @juzhiyuan @gxthrj @LiteSun @nic-chen @membphis @moonming

[moonming]

For the upstream info or service info which are not supported by OAS3.0, there are two feasible options of implementation scenarios:

* Expand OAS3.0, increase `upstream` and `service`, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].

  * advantages:
    when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
    we can support batch operations.
    this also can be done only as a command-line tool.
  * disadvantages:
    Users need to modify the existing OAS refer to our doc.
    `upstream` and `service` are the only necessary expansiones, if we want to support other plugins by `import` operation, we also need to define new expansion.

* Combine with existing create route page, when user create a route, they can import a OAS file, which provides parts of the route's info, and for the upstream 、 service and other plugins, they can follow the page step.

  * advantages:
    We don't need to expand OAS3.0.
    It can support all features supported by APISIX
  * disadvantages:
    We need to modify the current UE of create route to support this feature.

Reference:
[1]: https://help.aliyun.com/document_detail/88956.html?spm=a2c4g.11174283.6.564.7be17d24shSMBS

what do you think?

cc @juzhiyuan @gxthrj @LiteSun @nic-chen @membphis @moonming

how can the user export it if using the second plan?

[liuxiran]

how can the user export it if using the second plan?

with the second plan, user can only export part of the route info which supported by OAS3.0

[moonming]

got it. So I think the first plan is good for me.

[liuxiran]

• Expand OAS3.0, increase upstream and service, which are required by APISIX, and at the same time provide a detailed documentation. we can refer to aliyun API gateway[1].

  • advantages:
    when users import OAS, they only need to pay attention to the profile and do not need to do any additional works.
    we can support batch operations.
    this also can be done only as a command-line tool.
  • disadvantages:
    Users need to modify the existing OAS refer to our doc.
    upstream and service are the only necessary expansiones, if we want to support other plugins by import operation, we also need to define new expansion.

Based on the first plan, how to transform route configuration to OAS3.0?
After further compare the configuration of OAS3.0 and routes in APISIX, we categorized the current route fields into three parts

• explicitly matched fields are:

| route field | OAS3.0 field|
| ---- | ---- |
| name | paths[url][method]operationId |
| desc | paths[url][method]summary |
| methods | paths[url]method |

• We can do the conversion to implement the corresponding fields are:

| route field | OAS3.0 field|
| ---- | ---- |
| uris | paths.url, |
| vars | |
| plugins.key-auth | OAS3.0 reference |
| plugins.basic-auth | OAS3.0 reference |
| plugins.jwt-auth | OAS3.0 reference |
| plugins.openid-connect | OAS3.0 reference |
| plugins.request-validation | header params OAS3.0 reference 、body params OAS3.0 reference |

• we need to expended fields are:
  _we recommend to follow the data structure of the route, only change the name of the key, We can define the first supported fields, prefixed with x-apisix-*._

| route field | OAS3.0 expanded field|
| ---- | ---- |
| upstream | paths[url][method]x-apisix-upstream apisix upstream ref |
| plugins(except for a few plugins mentioned above) | paths[url][method]x-apisix-plugins apisix plugins ref |
| hosts | paths[url][method]x-apisix-hosts |

---

we formed the first phase of development plan below:

1. Export routes configuration as a json/yaml file in the way we expanded OAS3.0.
   BE:
   new a manager-api
   params: routeids & filetype(json or yaml)
   returns: file
   FE:
   add select box to each of route items in route list
   add a button Export routes as OAS3.0 to the top of the route list
   select one or more routes, and click the Export button, get the OAS3.0 file

2. import the exported file to generate routes.
   BE:
   new an API
   params: file
   returns: success or failed
   FE:
   add a button Import routes from OAS3.0 at the top of the route list
   click button, upload file

3. Document: Write a document to describe our expansion configuration options based on OAS3.0, examples+description.

cc @LiteSun @juzhiyuan @moonming @membphis @nic-chen @gxthrj

[liuxiran]

After the offline meeting, we have agreed on the following points:

1. First implement import/export based on OAS 3.0, and consider implementing other formats later
2. import and export in atomic.

Todo before development：

1. need to List ignored OAS fields when import OAS3.0 @liuxiran
2. Investigate whether golang has librarys that can support the conversion.

cc @juzhiyuan @LiteSun @nic-chen @membphis @moonming

[liuxiran]

when we import routes from OAS3.0, some fields in OAS will be missed because there are not corresponding fields in APISIX's Route:

1. API General Info: used to describe the general information about your API, some times, a oas file contains a series of apis which belong to a app, so this info is different from the api's name and extra basic info.
   reference
   example:
   

2. API server and base path: upsream url + url prefix(options)
   reference
   example:
   

3. path params: api params described in path
   reference
   example:
   

4. query params: api params described in query
   reference
   example:
   

5. responses description and links: Define the responses for a API operations
   reference
   example:
   

[juzhiyuan]

as for API Documentation, here is an example for both 2.0 and 3.0.

https://editor.swagger.io/

cc @membphis

[membphis]

cc @membphis

LGTM, thx

[liuxiran]

all the related PR have been merged, close it now