Kong: Applying a Plugin with a variety of global and specific properties

[subnetmarco]

Currently a plugin can be applied per api_id, and optionally per api_id and consumer_id.

We want a more flexible schema to apply plugins, that will allow us to:

1. Customize a global plugin for every API, without specifying any specific api_id.
   (Closed with #1369)
2. Customize a global plugin for every consumer, without specifying api_id and consumer_id.(Closed with #1369)
3. Customize a plugin per:
   
   
   
   • HTTP Method (method)
   
   
   • Endpoint (endpoint, with regex support)
   
   
   • By Request Header (Accept, Content-Type, User-Agent, etc...)
   
   

Lower priority:

1. I would add also whitelisting and blacklisting support for each property, including api_id, consumer_id, method, endpoint, etc.
2. Ability to enable more than once the same plugin for an API (deprecates #462)

This issue deprecates #160, #295 and #486.

[ahmadnassri]

should also be by response attributes, e.g.:

• response code / status
• response header

plugins that could leverage response attributes are ones like rate limiting, logging, etc ...

[bisoldi]

+1 My team is also in support of this. To clarify our use-case, we would like to be able to install a global rate limit that spans all API's with and without consumer authentication / differentiation. So, it doesn't matter who the consumer is or which API is being consumed, each consumption use contributes to the global use_count which is checked against the limit.

[jtmarmon]

+1. we have a user service that handles registration (unauthenticated, obviously) but also other user requests like updating and retrieving user data. the latter requests are authenticated, but the former is not. i don't have any ideas for a workaround for this, so i would love suggestions while waiting on this feature

[sonicaghi]

+1 - and per group if ACL plugin is enabled.

[thibaultcha]

Adding requirement:

1. Ability to enable more than once the same plugin for an API (deprecates #462)

[nijikokun]

Ability to enable more than once the same plugin for an API (deprecates #462)

:+1: This would allow for the following scenario (although somewhat convoluted):

I have three groups of consumers:

• Admins: 2, 4
• Applications: 6, 7, 8
• Users: 1, 3, 5

I would like:

• Admins can GET, POST, PUT, PATCH, DELETE
• Applications can GET, POST
• Users can GET

Correct me if I am mistaken, but this would require setting up three separate ACL plugins in three various ways? I.E.

• ACL on GET POST PUT PATCH DELETE (for Admins)
• ACL on GET POST (for Applications)
• ACL on GET (for Users)

So for GET requests it would hit each one and do a lookup, right?

[ahmadnassri]

as per the discussion on #462 ... if the concern is performance in allowing same plugin enabled multiple times, then at least I should be able to have multiple configs per plugin ... which is really the same end result if you think about it ...

another key point in making this happen is programmatic priority setting for plugins ... which are currently hardcoded

[subnetmarco]

After 0.6.0 we can start working on this issue, since we implemented invalidations.

[subnetmarco]

I am going to start writing black on white some ideas. To implement this feature we could introduce the concept of consumer and plugins mapping, and filters.

Both plugins and consumers can be created globally, and then they can be restricted to an API with an explicit mapping. Plugins and consumers can also be optionally grouped, and the group itself can be restricted to an API.

Each consumer and each plugin mapping can then act on certain conditions, defined by a filter.

Example

I want to divide my users in two groups, those who can make only GET requests and those who can make both POST and GET requests, and the users who can use both methods also have a different rate limiting:

Create consumers and automatically associate them to a group in one request. If the group doesn't exist, the system will create it:

POST /consumers/
  name="thefosk"
  groups=["only_get"]

POST /consumers/
  name="sinzone"
  groups=["get_and_post"]

Add the HTTP method restrictions for the mockbin API:

POST /apis/mockbin/acls/
  groups=["only_get"]
  methods=["GET"]

POST /apis/mockbin/acls/
  groups=["get_and_post"]
  methods=["GET", "POST"]

Create the rate-limiting plugin configurations, and associate them to a group if specified:

POST /apis/mockbin/plugins/
  name="rate-limiting"
  config.seconds=10

POST /apis/mockbin/plugins/
  name="rate-limiting"
  groups=["get_and_post"]
  config.seconds=1000

Note that using groups is optional, I could have used directly the consumers.

Filter

A filter is an object that determines on which conditions a certain mapping should be executed, it could be an object with the following properties:

locations=["us", "fr"]
hosts=["host.com"]
paths=["/hello", "/world"]
ips=["1.1.1.1","2.2.2.1/24"]
methods=["GET", "POST"]

And _at least_ one of these properties would be required. A filter can be associated to a mapping, for example let's create this filter:

POST /filters/
  name="north_america"
  locations=["us", "ca"]
  methods=["GET", "POST"]

and associate it to a mapping:

POST /apis/mockbin/acls/
  groups=["get_and_post"]
  filters=["north_america"]

The above operations allow the get_and_post consumer group to make GET and POST requests only from USA and Canada. If the filter is not specified, the API itself could create a filter under the hood:

POST /apis/mockbin/acls/
  groups=["get_and_post"]
  locations=["us", "ca"]
  methods=["GET", "POST"]

creates a filter with the following properties:

  name="autogenerated"
  locations=["us", "ca"]
  methods=["GET", "POST"]

Filters and APIs

A filter can also be associated with an API to determine its invocation:

POST /filters/
  name="north_america"
  locations=["us", "ca"]
  methods=["GET", "POST"]

and can be attached to an API object:

POST /apis/
  name="mockbin"
  filters=["north_america"]

Every GET or POST request coming from US or CA will be proxied to mockbin.

Endpoints

• /groups/
• /filters/
• /mappings/acls/ or /apis/{api}/acls/
• /mappings/plugins/ or /apis/{api}/plugins/

Regex and priority

The first version could not include regex and priorities, and that could be enabled on a second iteration.

This a summary of this brainstorming.

[subnetmarco]

Some attachments to the previous comment, to make it more clear.

Entities

Workflow

Plugins Execution Workflow

These workflows are not 100% complete, because they ignore groups, but they give an idea of a potential workflow.

[pradeepmurugesan]

Guys any idea on the milestone for this feature ?

[ericln]

it'll be nice to have global plugin, save a lot of configurations

[gschlect]

👍

[BrianHutchison]

+1 and eagerly awaiting this. As @pradeepmurugesan Asked, any idea on the target milestone for this?

[vitorsdcs]

+1

[bitsofinfo]

+1

[rogoman]

+1, any update on the expected release date?

[vitorleal]

+1

[BrianHutchison]

To clarify my company's interest, we're specifically interested in routing based on the Accept header. This is a fairly common REST pattern for specifying versions in the Accept header, and we would want to target the appropriate upstream server.

For example:

    GET /resource HTTP/1.1
    Host: api.example.com
    Accept: application/vnd.example.api+json;version=2

[srouthiau]

+1

[vinceferro]

I made a first implementation using headers for versioning that suit my use case, take a look if you want!

The commit history is not very good, I was short on time and unable to setup a pretty good dev environment on my mac 😅

https://github.com/vinceferro/kong/commits/versioning

[BrianHutchison]

Thanks @vinceferro.

I've found setting up a dev environment on my Mac to be fiendishly difficult. Never have been able to get the test suite to run all the way through, even on just the master branch.

[thibaultcha]

The new router in #1970 allows for API matching via HTTP methods, and thus partially addresses this, if one adds a plugin to an API matched via HTTP method.

[jacobwallace]

Is there any ETA for when a plugin could be customized per HTTP method, endpoint, or request header (number 3 in the description)? Thanks for all the work done so far on this issue.

[subnetmarco]

It can be done now: method, URI, host header or any combination of those (including multiple values)

On Nov 21, 2017, at 6:27 AM, Jacob Wallace [e-mail] wrote:

Is there any ETA for when a plugin could be customized per HTTP method, endpoint, or request header (number 3 in the description)? Thanks for all the work done so far on this issue.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[dudadornelles]

It can be done now: method, URI, host header or any combination of those (including multiple values)

@thefosk do we already have docs for this?

edit: I guess this would be done in the api resource and not in the plugin resource by defining the uris and method values? So if I want to apply a plugin to a particular endpoint but I want all other endpoints to not have plugins applied to them i'll need to register 2 apis?

[thibaultcha]

@dudadornelles The proxy reference has existed since Kong 0.5.0 and documents hosts, uris and methods: https://getkong.org/docs/0.11.x/proxy/

So if I want to apply a plugin to a particular endpoint but I want all other endpoints to not have plugins applied to them i'll need to register 2 apis?

This is correct as of today, applying plugins for particular routes means creating those routes (APIs), and applying plugins on them. We do have plans to improve this in the future, making plugins lighter and more flexible in their triggering rules.

[FBryant87]

Sorry but I can't see anywhere in those docs which supports routing on request headers (except host header).

[thibaultcha]

@FBryant87 Hi,

Routing by arbitrary request header is not yet supported. Pull requests welcome!

[timpamungkas]

Can you give me example?

I have service service-alpha with Kong route : /alpha
Inside of service-alpha, there are two endpoints

• [GET] /api/v1/read which require no authentication. With Kong, this will be http://kong/alpha/api/v1/read
• [POST] /api/v1/create which require no API Key authentication. With Kong, this will be http://kong/alpha/api/v1/create

The alpha services refer to upstream http://myserver:9001

I set two routes on service alpha:

• /alpha with no auth plugin
• /alpha/api/v1/read with api key plugin

However, accessing /alpha/api/v1/read is equals to accessing http://myserver:9001.

How can I set Kong to enable authentication only to /create What route configuration that correct?

[hishamhm]

A general update on this issue:

Regarding item 3 above, now that we have complete Routes and Services separation, and all three of method-based, endpoint-based and header-based routing, you split these different plugin configurations as different routes for the same service.

Header-based routing was the last piece of that puzzle, so this is possible since Kong 1.3.

[bungle]

@subnetmarco, I think @hishamhm explained it above that it is possible, so I am closing this. Please reopen perhaps with a more limited scope if there are still things missing.