Openapi-specification: Group multiple parameter definitions for better maintainability

[cansik]

Idea

Sometimes you have some parameters you want to provide on every path. At the moment it is possible to reference them with the $ref tag. This works great if you have a lot of different parameter combination for each route.

But we now have a complex api with a lot of _optional_ get parameters for every path. Now the problem we encounter is, that if we have to add new parameters to this _get_ parameters, it is very cumbersome and inconvenient to set the reference to them on each path. Often it happens that you forget one.

It would be great if we could group parameter definitions for a better maintainability and usability.

Example

As already presented on stackoverflow here an example of the definition (how it could look like):

parameters:
  MetaDataParameters:
    # Meta Data Properties
    - name: id
      in: query
      description: Entry identification number
      required: false
      type: integer

    - name: time_start
      in: query
      description: Start time of flare
      required: false
      type: string

    - name: nar
      in: query
      description: Active region number
      required: false
      type: string

And here is how I would like to reference it:

/test/:
  get:
    tags:
      - TEST
    operationId: routes.test
    parameters:
      - $ref: "#/parameters/MetaDataParameters"
    responses:
        200:
          description: OK

To get more on the user experience, this definition is much more readable than 20 parameters for each route.

Other Ideas?

Have you already thought about something like this or do you want us swagger users to go another way to deal with this problem?

Regards
Florian

[webron]

Thanks for opening the issue, there's just one thing that's not clear. On one hand, you're saying that you want to provide them for 'every path' which implies a global definition that won't require you to reference it, and on other hand, you go ahead and reference from a specific method to the group of parameters. So which of the two are you looking for? or is it both?

[cansik]

Ok the formulation _every path_ is maybe wrong (too open). We still have paths which do not need this parameter. We would like to use it as described in the example to reference a group of parameter with one reference. Otherwise it would look like this:

/test/:
  get:
    tags:
      - TEST
    operationId: routes.test
    parameters:
      # In path parameter
      - $ref: "#/parameters/event_type"
      - $ref: "#/parameters/appearance_date"

      # Meta Data Properties
      - $ref: "#/parameters/param1"
      - $ref: "#/parameters/param2"
      - $ref: "#/parameters/param3"
      - $ref: "#/parameters/param4"
      - $ref: "#/parameters/param5"
      - $ref: "#/parameters/param6"

      # Data Properties
      - $ref: "#/parameters/param7"
      - $ref: "#/parameters/param8"
      - $ref: "#/parameters/param9"
      - $ref: "#/parameters/param10"

      # Specific Properties
      - $ref: "#/parameters/param11"
      - $ref: "#/parameters/param12"
      - $ref: "#/parameters/param13"
      - $ref: "#/parameters/param14"
      - $ref: "#/parameters/param15"
      - $ref: "#/parameters/param16"
      - $ref: "#/parameters/param17"
    responses:
        200:
          description: OK

With the enhancement we could write it like this and manage the items of each group in one place:

/test/:
  get:
    tags:
      - TEST
    operationId: routes.test
    parameters:
      # In path parameter
      - $ref: "#/parameters/event_type"
      - $ref: "#/parameters/appearance_date"

      # Meta Data Properties
      - $ref: "#/parameters/MetaDataParamGroup"

      # Data Properties
      - $ref: "#/parameters/DataParamGroup"

      # Specific Properties
      - $ref: "#/parameters/SpecificParamGroup"
    responses:
        200:
          description: OK

[webron]

Okay, thank you for the clarification. The reason I asked is because we have a feature request for global parameters already.

People have asked for parameter groups in various places, but nobody opened a feature request, so thank you for taking the time to do so. The value of it is understandable. I'm not sure the syntax you shared works, but we can evaluate it further over time.

[mohsen1]

If $ref: "#/parameters/MetaDataParamGroup" resolves to a hash (as described above) parameters will have that hash in it which is not valid as Swagger 2.0. Unless we change the parameters array format to allow hashs (which I don't think it's a good idea) it will not work.

[cansik]

@mohsen1 Ok and what is the problem with hashes for parameters? Is there a discussion in another issue about this?

[mohsen1]

parameters is an array of parameter objects. I'm not sure if we want parameterName:parameterObject hashs in the spec also.

[dolmen]

@cansik, @mohsen1: /parameters is a hash of parameter objects but /paths/{path}/parameters and /paths/{path}/{operation}/parameters are arrays of parameters objects. There is no confusion.

[dolmen]

@cansik For your request, I suggest that you define your own vendor extension for parameter groups, and write a processing tool that would inline your group references into a pure Swagger 2.0 spec for external consumption.
This is how I work for my own API: I write my spec in YAML (that allows me to use comments) and will only publish the JSON version of it.

I already find that many tools do not fully support the current spec (for example I found issues with /parameters and /paths/{path}/parameters and I had to write a processor to inline them). We don't need now to complexify the spec if tools do not follow it.

[Project0]

+1
Want to see this feature as well.

[0x62ash]

+1
we too have predefined query params to almost all api paths

[renanmt]

+1 this would save me a lot of time and lines in my doc spec

[Growiel]

+1 this too, everything that gets me to reuse code is a huge +1.

[mkostin]

+1
would love to see it

[rafael84]

+1

[jharmn]

It's worth noting that RAML take on this, 'traits' (really not much different from the proposed structure).
http://raml.org/developers/raml-200-tutorial#traits

[ranacseruet]

+1

[fehguy]

Reference #560

[clemensberndt]

+1

[lggmonclar]

+1

[ePaul]

Just as a note for everyone wanting to vote this up: Github recently added a feature allowing a "reaction" to each post, using the button in the top left corner of each post. This will not clutter the thread with contentless comments.

[DavidBiesack]

How about allowing us to organize parameters into sets?
Then a path could reference parameter sets to automatically get all the parameters in those sets:

parameters:
    - name: id
      in: query
      description: Entry identification number
      required: false
      type: integer
      sets: [ "MetaData" ]

    - name: time_start
      in: query
      description: Start time of flare
      required: false
      type: string
      sets: [ "MetaData", "Alternate" ]

    - name: nar
      in: query
      description: Active region number
      required: false
      type: string
      sets: [ "Alternates" ]

paths:
  /test/:
    get:
      parameters:
        - $ref: "#/parameters/event_type"             <<<< $ref still works
        - $ref: "#/parameters/appearance_date"
      parameterSets: [ "MetaData", "Alternate" ] <<< yields id, time_start and nar
  /other/:
    get:
      parameters:
      parameterSets: [ "Alternate" ]         <<< yields time_start and nar

This is analogous to tags, but I used "sets" to avoid ambiguity. It's basically a shorthand for multiple $ref, but I think would be easier to read, maintain and reuse. It would not require changing the structure of parameters as in the OP, and would allow parameters to be in multiple sets, which the OP proposal does not allow (I think).

(I don't think there is a UI aspect to these sets, so no set descriptions are needed. Syntax for describing parameter sets would be added for documentation purposes, though.)

[k-omukai]

+1

[fehguy]

@OAI/tdc propose closing this with no action. The fix in #633 will help with this

[webron]

I think we still need to support this, and should be fairly easy.

633 doesn't really help with this. This is not about global parameters, and reusable parameters existed in 2.0 already.

[DavidBiesack]

FWIW, I would like OAS to use structural abstractions (such as the tags I suggest above) over $ref representation "tricks" which IMHO hide or obscure the intent. The API designer's goal is to reuse common API constructs such as sets of parameters. Implementing that through $ref and JSON schema inclusion works, but doing so adds a level of redirection (which to me equates to obfuscation.)

[earth2marsh]

I agree with @webron that sets deserve consideration on their own.

[ralusek]

This should absolutely be a feature. For API endpoints with many shared configuration parameters, it is a huge hassle to maintain those same params everywhere.

[rochifebo]

+ 1 this would save me time and lines

[jackkong]

+1

[matiman]

+1

[antoniogarcez]

+1

[dnascimento]

+1

[chenziliang]

+1

[ePaul]

As my previous note seems to not be followed anymore, here a repetition:

Just as a note for everyone wanting to vote this up: Github recently added a feature allowing a "reaction" to each post, using the button in the top left corner of each post. This will not clutter the thread with contentless comments.

Please don't add a comment with just "+1", instead just upvote the top post. Just post a new comment if you actually are adding any content.

[candlejack297]

Parameter sets seem better than the original proposal. Pretty simple QoL change that probably doesn't play badly with other proposals.

[evgeny-l]

+1

[xor-gate]

Grouping makes it much more clean, when having multiple endpoints with multiple parameters.

[xelber]

Would save quite a few lines of code and can focus on the specifics in an operation. Must have.

[webron]

Convenience is not a 'must have'. Not saying we're not in favor of supporting it, but keep it in proportion.

[ndurchx]

+1

[narayan24]

+1

[nidhind]

We strongly recommend this feature.
For example consider HMAC auth headers. If we could group the parameters then we could use them with a single ref, now I have to use many small refs.
I believe if grouping of definfitions like allOf exists then for the same reason grouping of parameters must also exist.

[fehguy]

We're past feature freeze for 3.0.0, so expect this isn't happening initially.

[fbe]

+1

[chorsnell]

2 years later and this still isn't a thing? any idea if this is going to be adopted?

Seems like plenty of positive reactions :)

@cansik did you ever get a more elegant solution than https://github.com/OAI/OpenAPI-Specification/issues/445#issuecomment-132948031? Or did you go the route @dolmen suggested with https://github.com/OAI/OpenAPI-Specification/issues/445#issuecomment-146250533

[darrelmiller]

This is definitely something we would like to do. The big question is what is the best way of doing it. This was my thought experiment on the subject https://github.com/OAI/Gluecon-Workshop/blob/master/Experiments/GroupRef.yml

[chorsnell]

@darrelmiller looks good. anything that promotes reuse and grouping of commom elements where ever possible has got to be a priority. Your implementation definitely seems fairly flexible, I will have a think on it and see if I have any further suggestions.

In API design I tend to favour having parameters rather than relying on a body object + schema definition so I can have better descriptions, examples and easier access to those properties from the swagger UI (have had a look around for best practises / knowledge on this but couldn't find much).

This means on resources with GET / PUT / POST, they usually end up exposing the same, or mostly the same params. Then this duplicated across various resources can get pretty duplicate heavy unless doing something like @cansik does here - https://github.com/OAI/OpenAPI-Specification/issues/445#issuecomment-132948031

[cansik]

@poppahorse As you mentioned, the issue is two years old. I have to admit that don not work with Swagger anymore, but I still think it is an important topic.

To solve the problem in our project, we decided to write a preprocessor (as @dolmen suggested) and to inject the parameters on startup. But I do not think that this is the right way to go. Because, as you see, there are a lot of people who would like to have a in-framework solution for that problem.

So, instead of everybody writing their own isolated tools, it would be better to have it directly in the specification.

But at the moment I do not know, what is needed to go a step further. Maybe the discussion about how it really could be implemented, should be opened again.

[wzrdtales]

@cansik When you moved away from Swagger, where did you move to?

At the current state swagger generates more work than it solves and is just not flexible enough. Many things are missing including this one from this issue. So I really don't want to use it anymore...

[darrelmiller]

@cansik @wzrdtales Many of the improvements in V3 of the spec came from people creating issues that proposed solutions to problems. Discussing how important things are and how much is missing, doesn't really move us forward. Proposing a concrete solution or commenting on solutions proposed by others does. The next version of the spec will most likely be a 3.1 and will come much quicker than 3.0 did.

[wzrdtales]

@darrelmiller Sure, I'm normally always open to such topics in general, but the question I asked is for the current state. And as @cansik also moved away I just went ahead asking him. And in this current state I just decide to not use swagger anymore, this is not intended to offend anyone though.

To give an honest opinion on this whole topic. For almost everything I miss, there is already an existing issue, so no need to open anything new though. But some of them are open for years and neither got closed nor accepted. I guess this issue here is open for two years just for the simple reason that you don't want to bloat the specification with any and everything. And this is totally ok. A solution to a problem that is not reallly solvable when you don't want to incorporate just everything, probably would be to have userland extensibility. Which I read you plan to add, but it seems not to be fully there yet, at least I did not find any docs.

[webron]

That's actually not the reason. If anything, this feature request is something we _really_ wanted to get in but it was a matter of lack of time. We've worked on the 3.0 version for about a year and a half and at some point, we had to stop pushing things in - a difficult decision on its own. Out of the features we ended up not including, this one is by far the one that was most discussed by the TDC

The length of time issues have been opened does not reflect the odds of them getting in or not. If we make an actual decision to not include something, we will close the ticket.

You're right that there's no need to open new tickets on issues that exist - but you can still help those tickets by proposing solutions that fit the spirit of the spec. We're very open to contributions and do our best to make it a constructive process.

[wzrdtales]

@webron Agreeing stuff can take time sometimes, but I and probably others also don't have the time to wait for years :)

So maybe I should put things correct again and apologize for how I commented this initially. I am not against giving my opinion on this, to actually give one on this issue here, I think basically something like yaml anchors could work, that just plainly insert stuff where you ask for it.

So let me reformulize it, currently I don't want to use swagger anymore, as I need something right now and don't see any easy entry point to just make changes that I need which I would in turn naturally make open source again for others to benefit from, and from the mass of things that I stumbled over while creating some docs with swagger it just developed the decision to drop it. At least for the current project we talk about, I will always reevaluate stuff unless the project died naturally, as I am very well aware that the project is not going to stop but maybe evolve to a state that would be usable for the concrete use case again.

[milesHero]

Hi, I'm new to swagger and I would like to ask about the parameters in swagger. I have a separate parser of swagger.json and I need to hide some parameters while parsing. It should be visible in both swagger ui and swagger.json so I can't just use @ApiIgnore but I need a way to ignore those parameters in parsing the swagger.json. I would like to ask if there's a way to add customize parameter's attribute (ex: hidden = true) in swagger.json through anotations (@ApiParam) so when my parser read that, that parameter will be ignored.

I tried @ApiParam(hidden = true) but it's not working.

Currently I'm getting this,
{
"name": "username",
"in": "query",
"description": "This is a username",
"required": false,
"type": "string"
},
Is it possible to have this?
{
"name": "username",
"in": "query",
"description": "This is a username",
"required": false,
"type": "string"
"hidden": "true";
}

Thank you so much!

[0x62ash]

Still stalled?

[caoge]

x.x 2 years...

[MikeRalphson]

Another thought-experiment here. This borrows the idea of traits from RAML (as suggested in #613), and is, to my mind, cleaner than parameterSets or groupRefs. It also addresses the desire for truly global parameters and headers.

[darrelmiller]

I like the fact that this addresses the group reuse concept for more than just parameters.

The term trait feels like a generic notion, but this concept is applicable to just operation and response. Which feels strange. I almost feel like we need operationTraits and responseTraits as distinct concepts.

These kind of capabilities are tough on tooling if you want two way fidelity. e.g. How do I parse this, manipulate it semantically and then output it whilst preserving the traits? $refs are hard enough, this is even harder because it spans multiple objects.

We still have to decide what to do about merging. Does the trait win?

I can see this is going to be a _fun_ conversation!

[MikeRalphson]

I almost feel like we need operationTraits and responseTraits as distinct concepts.

Not convinced. To allow for future expansion of the content and applicability of traits I see one kind of trait acting as a container as preferable to many subtypes. traits could also apply at the pathItem level.

These kind of capabilities are tough on tooling if you want two way fidelity. e.g. How do I parse this, manipulate it semantically and then output it whilst preserving the traits? $refs are hard enough, this is even harder because it spans multiple objects.

We still have to decide what to do about merging. Does the trait win?

I envision this working much like pathItem-level parameters and operation-level parameters, tooling must create a list of effective parameters from the two sources (combining as per the spec.) but if they need to serialise the document with that level of fidelity, they need to remember where each parameter was defined. Thus, the definition of a conflicting parameter or header within the operation would be seen as more specific than the trait (which is a global object) and thus the trait loses.

I can see this is going to be a fun conversation!

Oh, now if I've given you that impression, then I apologise. 😄

[ericraymond]

+1

[MikeRalphson]

Updated straw man gist for a trait implementation for v3.1.0

/cc @darrelmiller as we talked about this recently. Comment copied from #1794 as now closed.

[DavidBiesack]

Nice, Ralph.
Quick question: Why "each traitObject MUST have a parameters array XOR a headers map"?
This seems overly restrictive. I can see where someone may want a single trait on an operation that includes a query param in conjunction with a request header.

[MikeRalphson]

@DavidBiesack I've just looked at it again, and I suppose I can see no reason why that shouldn't work, just as I was envisioning traits being added at the operation level for parameters and then at the response level for response headers. Can you think of a real-world case for a linked query parameter and a response header?

Request headers would be part of the parameters array.

[DavidBiesack]

OK I see the distinction - missed where a trait might apply to just the response, not the operation. I don't know of any such use case for linking query params with response headers that would apply to _all_ responses in an operation. Thanks for the clarification. Suggestion: update your comment to
"each traitObject MUST have a request parameters array XOR a response headers map"

[MikeRalphson]

Will do, thanks.

[jeanniearbeau]

+1

[caladojm]

+1. This parameters grouping will help a lot!

[philsturgeon]

Can somebody (@cansik) jump on the next OpenAPI TSC call to get this moving along? @darrelmiller can we get it on the agenda? This one seems like an easy win and its on the v3.1 list (#1466)

[ioggstream]

@MikeRalphson is there a PR for https://gist.github.com/MikeRalphson/7ee0624fc1df70c0475717d89a5b6e4b#file-traits-yaml ?

[MikeRalphson]

@ioggstream no, but I could raise one. A direct one against the spec or a proposals document?

[darrelmiller]

@philsturgeon Based on past conversations there is no sign of an easy win here. I started to write the proposal PR up for this over the summer and ran out of time.

[ioggstream]

@MikeRalphson whichever you think is appropriate. I'm willing to help in the initial phase.

[creckord]

I came here looking for a way to express semantic grouping of query parameters across clients and servers using my OpenApi spec in order to preserve my JAX-RS @BeanParam usages in the spec.

To me, a reusable (named) group of parameters is really an expression of a structured data type, i.e. a schema, that gets exploded at its point of application. The proposed traits would be a great way to express that.

[adamreisnz]

Any progress on this feature? We have 3 routes for each endpoint (query, count, export) which all share the exact same query params, and rather than maintaining them for each endpoint individuall it'd be great if we could group them 👍

[jlusiardi]

I think this would be great to DRY out pagination parameters on list resources ;)

[Halkcyon]

@jlusiardi This is just the case I was trying to tackle at work this week. Any clue if recipes exist to emulate the behavior requested here where some parameters are required if others are included in a request? (e.g. if pageSize in the query string, pageNo needs to be there, too)