Just curious about OpenAPI 3 support plans.
See related post about Swagger 2 and OpenAPI 3 introduction:
https://blog.readme.io/an-example-filled-guide-to-swagger-3-2/
I have this blog post in my read buffer. 馃槃
To be honest I haven't thought too much about the OpenAPI 3 in Connexion so far. But definitely it is time to start considering it. I will follow up in this thread later when I give this topic some more thought. Thank you for opening this issue.
If anyone have ideas/suggestions or is willing to help in this I will be glad to support.
Cool. I'm open to help in a near future, especially when in my team we can start considering OpenAPI 3 adoption across our services.
I need to get up to speed on version 3 and would then like to help.
Just went over the new OpenAPI 3.0 which looks quite nice. I'm also curious as to when we may consider supporting it in Connexion. :-)
Will we try to simultaneously support both versions 2 and 3? Although more work, my assumption is this would be the way to go, otherwise we will break a lot of APIs currently built with Connexion.
@danballance Yes, we will support both. We will try to reuse as much as possible the codebase we have.
As side note, I am working on a sketch of the support for OpenAPI v3.
Hey @rafaelcaricio just wanted to ask if you guys have made any progress on this and if you might need any form of testing.
@JuxhinDB Unfortunately no. I'm currently busy with some personal stuff and with little free time to take care of this right now. :(
That's fine, @rafaelcaricio! Feel free to ping me in the future should you need any help with this.
support for 3.0 would be awesome. Parameter and anchor references are much flexible and it can mean less repeated code to document the same API.
The OpenAPI 3.0 spec was finalized and released a couple of weeks ago: https://www.openapis.org/blog/2017/07/26/the-oai-announces-the-openapi-specification-3-0-0
Hi, if anyone is willing to guide me through with an email or two on the codebase, I may try to help with some progress on OpenAPIv3 support. If so feel free to ping me. :-)
Any roadmap for support for v3?
e.g. having oneOf on types.
Any update on this? This would be awfully handy!
I've been making some progress on a branch https://github.com/dtkav/connexion/tree/oas3
I'd love to chat about it if anyone is keen to help. There are a handful of design decisions to figure out.
My plan is to get the unit tests passing, and all of the examples, and then open a pull request for discussion.
@dtkav sounds great!
@dtkav -- Some time next week I would love to check out the work and see if I can provide you with any help, even if it's just testing related. :-)
I'm looking for feedback on ways to approach the problem of supporting swagger2 and oas3 simultaneously.
I can imagine a few approaches:
refactor the operations class into classes that deal with the different specs (resolve refs, path/global defaults), and then slim out the connexion operation class to focus on the webserver side of the operation.
convert all code to oas3 style data structures, then convert swagger2 spec -> oas3 spec immediately upon loading the spec.
I've put a little bit of time into options 1, and 2. I'll probably give 3 a shot next and report back.
The main pain points so far have been around requestBody vs. body/form parameters, and content mimetypes vs. consumes/produces.
Let me know what you all think! I'm currently leaning toward option 3 because it would delegate the responsibility of backwards compatibility and keep the code clean. Unfortunately that strategy isn't very incremental, so I imagine it will be tough to upstream.
@dtkav I would be fine with option 1 (incremental changes) as it would leverage existing tests and add new tests for OpenAPI v3. This then would be a solid base to do a bigger refactor like proposed with option 3. Maybe I'm too pragmatic and not so much into the code anymore, so take my opinion with a grain of salt :smirk:
Just to add my 2c -- given the man power for this, I also feel option 1 makes most sense where we can leverage as much of the current framework as we can to build on top of. I will dedicate some time next week to help out with implementation/testing where possible.
cool - thanks team!
I've gotten the unit tests passing again for swagger2, and I'm able to support some converted os3 examples here: https://github.com/dtkav/connexion/tree/oas3 (definitely a very messy work in progress!)
@JuxhinDB thanks - I'm keen to have someone to bounce ideas off of - little things keep coming up.
for example -- in oas3 the requestBody doesn't have a name (vs swagger2 body/form parameters).
with swagger2:
the body is passed to the handler function by name (ex. def put_pet(pet_id, pet) where pet is the deserialized/validated body).
with oas3 there are a few options, but none of them seem great:
def put_pet(pet_id, id, animal_type, name)).def put_pet(pet_id, body)).def put_pet(params, body))request object in Flask that is implicitly accessible in the request context. ex:def put_pet(pet_id):
pet = cnx.body
# do stuff
However, this is a significant workflow difference between swagger2 and oas3, and is potentially difficult to support for the different webservers (flask, aiohttp, etc).
Let me know what you think!
@dtkav I would propose passing the body as a parameter (option 2) and would try to make it smart, i.e. it should work for existing function signatures (last argument is the body parameter) and we could make it configurable via some x-.. extension in the YAML to define the body parameter name.
yeah, good idea @hjacobs ! I had a similar thought on my way home about something like x-body-name.
@dtkav -- I'm taking a look at your oas3 branch on your forked project. Care to shoot me an email, or explain here how you are running the full test suite? I would like to play around a bit (first I need to actually review OpenAPIv3 spec) but want to make sure tests remain stable so that I may push some PRs over the weekend.
hey @JuxhinDB - I've added fork notes to the README . They should help you get the test suite up and running.
Let me know if you run into any troubles!
My next step was going to try to get parameterized test fixtures working to automatically run with both the swagger.json and openapi.json files. Right now I've just converted a few tests over to use oas3 specs. Let me know if you think this is a good path forward, or what else you have in mind!
@dtkav - So far this looks like good progress, I just wanted to get your fork setup and have the test runner working. Only 5 tests in total seem to be failing for me right now which is good. I'm going to actually read this article to get a decent understanding of the new spec and will help convert over a few tests.
Hey @JuxhinDB yeah that's a great article to get started.
I've also been using https://github.com/Mermade/swagger2openapi/ to convert test fixture specs over.
I have a bunch of outstanding work that I've just pushed to a branch called "oas3_broken" if you want to get a sense for how I switched over some of the tests and what I'm working on at the moment https://github.com/dtkav/connexion/commit/bcc0ad25ff5797683213ed33d22f36c025e1774c
Unfortunately in converting the test fixtures to oas3 specs, it is reducing the swagger2 coverage, so ideally the tests would be parameterized to test with both fixtures.
update: I have all the tests passing for the oas3 branch with both the swagger2 and converted openapi3 specs.
next steps are:
1) make swagger2 tests run automatically to avoid regressions (parameterize the test fixture)
2) lots of refactor and cleanup
@dtkav -- Apologies for the late response from my end. Super busy week that I'd like to wrap up. I have taken a closer look at your PR and so far looks pretty beefy. I wanted to recommend having a public email thread of sorts to avoid cluttering this issue thread too much (unless people are OK with it).
If you wish, I will help with the refactoring of certain parts on your PR this weekend. :-)
@JuxhinDB I'm fine with commenting on this issue (or on the PR directly) --- don't see why this would clutter the thread (Kubernetes issues/PRs are usually HUGE compared to what we have here... :smirk: )
@dtkav - I may be up to help. Or at least to test things. My team just started using Swagger, and I decided to document everything with OAS 3 since it seemed to make more sense to start with OAS 3 since much of the tooling already supported it. Unfortunately, I created a big openapi.yml specification for some of our APIs before I realized that we'd be using connexion and that connexion does not support OAS 3.
I'm going to grab your oas3 branch and give it a try with my current OAS 3 YAML file. Let me know where you'd like comments / feedback. Just in this ticket? Or somewhere else?
hey @tombry! I feel your pain, I'm in a similar boat! RE: feedback - I'm a new contributor to connexion, but @hjacobs mentioned above that commenting on the issue is fine. Alternatively you can comment on PR #591, or issue a pull request against the oas3 branch.
If you want another pair of eyes on your spec feel free to send it to [email protected] and I can try to help get it working. Depending on the features you used, it might be possible to convert to swagger2 - I'm now intimately familiar with the differences between the two versions :smile:.
Another thing to note - after a detailed read-through of both specs, I think the current behavior (swagger2) is a bit off for how query parameters are exploded/repeated. I've changed the behavior to match (my interpretation of) the spec and updated the tests in tests/api/test_parameters.py on the oas3 branch. I've also split out this change into #587 . I'd like some input on this change. Unlike the rest of the oas3 work, it changes the expected behavior of swagger2 specs and _is not backwards compatible_ so it may break existing users.
@tombry @dtkav Can you give me some guidance on how to use the oas3 branch against my own openapi 3.0 spec? I'm a bit lost trying to get it working, probably because I don't actually know python.
Naturally, as soon as I gave up and requested assistance, I've gotten past errors in importing connexion and moved on to my own spec errors. Sorry for the unnecessary tag.
I am not 100% certain, but I may have found a bug.:
connexion.exceptions.InvalidSpecification:
components:
headers:
X-ABC-ResponseRef:
description: Unique identifier for the operation
schema:
type: string
used inside many response blocks like:
responses:
'200':
description: OK
headers:
X-ABC-ResponseRef:
$ref: '#/components/headers/X-ABC-ResponseRef'
From my reading of the 3.0 spec, it sounds like headers should be allowed to be either header objects or reference objects, but components.headers is not in the list of allowed items given in the error.
Adding
'components.headers': self.components.get('headers', {}),
to line 234 of operation.py appears to fix it (or at least let me move on to the next error).
Hi @viralanomaly - thanks for the note - I've been trying to get ports from swagger2.0 -> openapi3 specs working first - not every feature in the openapi3 spec has been implemented yet. I'll integrate your fix, and would be happy to test your spec if you're OK with sharing it - it's always good to have more examples in the test suite.
The only other major thing I've found so far is that I had to comment out my required attributes in a construct like the following:
SignatureCreate:
allOf:
- $ref: '#/components/schemas/Signature'
- required:
- clientId
- transactionRef
- signatureData
The error it gave was that those fields didn't exist (They do exist in Signature, but not in SignatureCreate at the time it checks, I expect).
@dtkav , I'm going to see if I can get permission to share the spec with you to try to get this working.
I've tracked down the problem I'm having with required, but it comes from openapi-spec-validator. Should I go write this up over there?
@viralanomaly yeah that's probably the best place - I'm happy to take a crack at the bug once you write it up - I've upstreamed a few things to the openapi-spec-validator repo.
There's the one I wrote up against the validator.
@dtkav @viralanomaly - I'm watching the thread, but I'm at a business conference this week. I did run into a couple of issues with the OpenAPI 3.0 support before I left. I'll grab the latest code when I get back and see whether the bugs are still there, and if so, I'll get them added to the issues.
I'm about to sign off for today, but would really appreciate your thoughts on a few things.
1) I've been working on a refactor for cleanly separating swagger2 vs. openapi3 classes.
The basic idea is to have the Operation class become openapi3 compatible, and to introduce a Swagger2Operation class that overrides some of the methods to support parsing swagger2 specs.
The apis/abstract.py::AbstractAPI class is responsible for getting the spec version, and creating the appropriate operation class.
1) Is this a decent general approach? What alternatives can you think of?
2) Thoughts on defaulting to OAS3, and calling out differences for Swagger2?
3) Is it worthwhile to follow the pattern for both apis,apps and create an abstract base class for Operation?
Please have a look here: https://github.com/dtkav/connexion/compare/18a7c28...dtkav:oas3_class_separation and let me know what you think.
2) What's a good strategy for getting this merged? I can image a few:
1) Put out a release that includes "beta" OpenAPI3 support, and is documented this way (keep examples in docs in swagger2, etc)
2) Try to implement more new features in OpenAPI3 so it feels complete on the first release.
3) There is still one backwards _incompatible_ change that I made, and would love your comments on (#587)
Thanks to everyone who is commenting, collaborating, reviewing, etc. - I really appreciate it :heart:
Slowly, but surely, this thing is coming together! :construction_worker_man:
Hey folks,
Quick update - I've been working on upstreaming some architectural changes unrelated to oas3 support that will make this PR much smaller.
Those are:
One other thing I'd like to do is to get the openapi.yaml tests fixtures into the master (not doing anything). After all that I think the PR will be large but hopefully manageable!
If you were using/reviewing the code before - I've undertaken the large refactor I mentioned above (splitting out operation classes, implement an abstract base class to define the interface). Hopefully this is a lot cleaner. It's still not as DRY as I'd like, but It's getting close!
Your comments and feedback are appreciated.
@dtkav Thank you for all the work you have been doing on this front. Please send a PR with the OAS3 tests fixtures and we can push that forward.
Thanks @rafaelcaricio - I appreciate it. Please see #616 - Cheers!
@dtkav , I just emailed you about a problem I'm running into with the latest on the oas3 branch. Essentially, when I have a construct like this (with schema as a $ref followed by an examples block), I get an error. If I delete the examples block, the error resolves. Unfortunately, I use this construct a lot so I don't want to just remove them all.
/blah/{blah1}/id/{blah2}/{blah3}:
parameters:
- $ref: '#/components/parameters/blah1'
- $ref: '#/components/parameters/blah2'
- $ref: '#/components/parameters/blah3'
get:
tags:
- Blah
summary: Get blahs
description: Get one blah for another blah
operationId: api.getBlahs
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/GetResponse'
examples:
Monthly:
value:
field1: ABCDEFG
OneTime:
value:
field1: ABCDEFG
Error:
File ".\git\connexion\connexion\apis\abstract.py", line 294, in _handle_add_operation_error
six.reraise(*exc_info)
File ".\git\api\venv\lib\site-packages\six.py", line 693, in reraise
raise value
File ".\git\connexion\connexion\apis\abstract.py", line 273, in add_paths
self.add_operation(method, path, endpoint, path_parameters)
File ".\git\connexion\connexion\apis\abstract.py", line 235, in add_operation
pythonic_params=self.pythonic_params)
File ".\git\connexion\connexion\operations\openapi.py", line 147, in __init__
self._responses = resolve_responses(operation.get('responses', {}))
File ".\git\connexion\connexion\operations\openapi.py", line 133, in resolve_responses
example = self._resolve_reference(example)
File ".\git\connexion\connexion\operations\openapi.py", line 200, in _resolve_reference
for obj in schema, schema.get('items'):
AttributeError: 'str' object has no attribute 'get'
hey @viralanomaly , thanks for testing the branch.
I misread the spec, and was expecting examples to be a list!
I've updated the branch - hopefully it fixes your issue!
My spec is much happier with that change.
Thanks!
FYI, some new features in the OpenAPI 3.0.0 spec that aren't implemented in #621.
@dtkav thanks for your great work so far! :smile:
It appears like there might be a problem in this file: https://github.com/dtkav/connexion/blob/oas3/connexion/apis/abstract.py
It complains "NameError: name 'old_style_options' is not defined" - and as far as I can tell, that is correct in this file.
It feels like a merge/rebase gone wrong.
hey @viralanomaly - you are right, I messed up a rebase. I'll put up a fix shortly.
@viralanomaly the build is fixed again.
I think #627 is the last architectural change I'd like to get in, and then I think I'll rebase and remove the WIP from the PR!
Another minor thing is waiting on @p1c2u to cut a release of openapi-spec-validator that includes the defaults validation.
Thumbs up that so much progress has been made! :+1:
I'm working on breaking up the diff further into small PRs.
Let me know if you have any ideas for how to break up the PR into more bite-sized chunks.
Thanks!
I have removed the WIP status from #639 (first half of openapi 3 diff) and am now looking for review!
If any of you have some spare time, I'd really appreciate your comments and review.
How is the progress with the openapi 3 support?
What exactly would you like coders to review? (I see many pull requests...?)
Hey @coltenkrauter
TL;DR:
Adding OpenAPI support is a big change, so we've been targeting a 2.0 release of connexion (branch dev-2.0).
The PRs that I'm actively looking for review on are:
#639
#656 (small bugfix)
The history:
I factored out several changes and merged them into master ( #613 #616 #638).
We then decided to target a connexion 2.0 release. #619 is the 2.0 release target (dev-2.0 -> master).
The future:
If you want to "beta test", you can use my branch, which I'll try to keep up to date: https://github.com/dtkav/connexion/tree/oas3
I haven't really changed it in about a month, and it has all the features I was planning to add. I've purely been working on trying to upstream the changes by breaking off smaller and smaller chunks for review.
If there's demand for it I'd be happy to publish a package of my fork. I'll still upstream the changes, but it's pretty slow going so it will likely take a while.
Here are some live filters you can use to check on the status of work for this project:
open PRs against dev-2.0 (i.e. needs review, unless WIP)
merged into dev-2.0
merged into master
Is anyone in the community available to review the above PRs (#656, #639)? I'm blocked on review.
@dtkav Big well done. Wish I could help more. Hoping the main contributors could take a closer look, especially to #639. #656 to me looked perfectly fine.
It would help if they'd enable codacy for dev-2.0, too...
Both PRs look sane to me. Can anyone approve them, or are you specifically looking for the core authors?
@JuxhinDB @viralanomaly thanks for your comments and review. I'll go ahead and merge #656
I've been added to the repo as a community maintainer, which means that I can write to dev-2.0, but not master. It's important to get a solid review before merging though or it will just push a large review burden onto the zalando team when it comes time to review the dev-2.0->master PR. We want to make sure the release candidate is as solid as possible.
I'm still really keen on finishing this project. I spent a huge amount of time on it, so it would be awesome to merge the changes eventually.
I've considered putting out a sort of bleeding-edge pip package, and having a sort of ubuntu:debian type relationship with connexion (i.e. upstream features as they become stable). That's not something to do lightly though, so I'd want to make sure that everyone would be happy with that sort of arrangement, and that it'd actually be used/useful.
Let me know what you think!
@JuxhinDB if you can click the "review changes" button next time, that would record your review and approval with the commit.
@dtkav -- Done, cool feature that I got to learn. :-)
Later today I'll go over the other bigger PR and give my thoughts.
This is probably an awful hack, but I just put this in my file that I run with pip install -r:
# connexion git+https://github.com/dtkav/connexion.git@oas3#egg=connexion
@jmcs thanks for the merge!
It looks like there are some things that are in master that haven't made it into dev-2.0. I've merged the latest master into dev-2.0. I've pushed the previous state to https://github.com/zalando/connexion/tree/dev-2.0-bak in case I messed that up.
@JuxhinDB thanks a lot for the review of #639 !
Now #621 should be a pretty reasonable size, and ready for review! Let me know if you can think of ways to pull out features to make the diff smaller.
I just found an new edge case with form parameters in OpenAPI 3 from a test case that had I added to master a while back. I need to work out array handling for form parameters that come in through the requestBody in oas3, so I'm going to put the WIP tag back on the diff. I'll update again when It's ready for review.
Sounds good @dtkav -- feel free to ping me if you would like eyes on certain diffs.
Thanks @JuxhinDB I really appreciate it.
AFAICT I've fixed up the problems related to form data in oas3, so I've removed the WIP status from #621 . ~I think it should be ready for early eyes on, but it's a bit big so I'll try to break out smaller pieces if I can in parallel.~ I would love your review!
To anyone who is able - please review #621 - it adds support for OpenAPI 3 specs!
Once this is landed I'll have much more energy to address other issues. Thanks!
@JuxhinDB @rafaelcaricio @coltenkrauter @viralanomaly @positron96 @ioggstream @jmcs @hjacobs
I went over the PR and overall I didn't notice any glaring issues. That said, it would be best if someone with Swagger2/OpenAPI3 experience went over it from a spec perspective
Thanks @JuxhinDB ! Do you mind going through the review process (click the review button, leave comments, etc)?
Wholehearted thanks for the review @JuxhinDB - I'm going to leave the PR open for the rest of the week to give others a chance. By default I'll merge this PR into dev-2.0 on Monday unless:
a) Someone else requests changes
b) Someone asks for more time to review
Some validations are not working yet.
One of the best new feature in OpenAPI spec 3.0 is the 'oneOf' which lets you do polymorphism.
In a post request body
requestBody:
description: Event
content:
application/json:
schema:
oneOf:
- $ref: '#/components/schemas/Event1'
- $ref: '#/components/schemas/Event2'
It will fail the validation when bootstrapping the app.
It prompts only recognise these two types.
{'oneOf': [
{'$ref': '#/definitions/requestBody'},
{'$ref': '#/definitions/reference'}]}
This is the issue from https://github.com/p1c2u/openapi-spec-validator/issues/46 anyway.
hey @lihan - thanks for opening that issue. I'll take a look over on the openapi-spec-validator project.
whoops, I pressed the wrong button, sorry!
@lihan which version of jsonschema are you using? I'm on jsonschema==2.6.0
The below example works for me:
# added required fields to your spec
openapi: 3.0.0
info:
version: 0.0.0
title: pets
# added required fields to your spec
paths:
/pets:
patch:
operationId: lihan.patch
requestBody:
content:
application/json:
schema:
oneOf:
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
responses:
'200':
description: Updated
components:
schemas:
Dog:
type: object
properties:
bark:
type: boolean
breed:
type: string
enum: [Dingo, Husky, Retriever, Shepherd]
required:
- bark
- breed
Cat:
type: object
properties:
hunts:
type: boolean
age:
type: integer
required:
- hunts
- age
> curl -X PATCH "http://localhost:5000/pets" -H "accept: */*" -H "Content-Type: application/json" -d "{\"age\":5,\"hunts\":true}"
"OK"
function args: ()
function kwargs: {'body': {'age': 5, 'hunts': True}}
While debugging your issue, I discovered that discriminator is not part of jsonschema, so I'll probably need to add that to the validator somehow. Feel free to contribute tests cases for important use cases that you have.
I've merged #621 (OpenAPI 3.0.0 Support) into the dev-2.0 branch :tada:
Next up will be the examples. I should be able to polish those up for review tomorrow.
Next up are:
Next step is to finalize the feature set for the 2.0 release and ship it!! (#619) :ship:
Also, I'm working on fixing up the codacy warnings for the branch, but some of them are bogus (#673).
Please comment on #678 if you think there are any major features that should be in the 2.0 release!
Closing with the release of 2.0 (https://github.com/zalando/connexion/releases/tag/2.0.0) :smile:
Thanks to @dtkav and all the team for this great release!
Thanks for everyone's hard work and collaboration!
Big well done guys, loved the collaboration!
Most helpful comment
Closing with the release of 2.0 (https://github.com/zalando/connexion/releases/tag/2.0.0) :smile: