Hydra: oauth2: allow issuing of JWT access tokens

[aeneasr]

[aeneasr]

This will no longer be pursued, instead, we will introduce Oathkeeper which is capable of translating opaque access tokens to JWT tokens.

[MOZGIII]

Wow. There should be a really big red warning: hydra no longer supports JWT as access keys.
That's a shame, because last time I checked - it did.

Too bad I'm only discovering it now. The solution I'm working on really requires JWT as access tokens.

Is it possible to we add support for JWT still?

[aeneasr]

because last time I checked - it did.

When was that? Back in 2015?

Is it possible to we add support for JWT still?

Not in Hydra because of multiple issues (I'll write a blog post on it soon). But you can use https://github.com/ory/oathkeeper for that. It converts access tokens to JWTs and can also validate the request based on scopes and access control policies. Just be aware that we're currently refactoring it.

[MOZGIII]

Yes, somewhere around 2015. I didn't use it then, but I stumbled upon and noted it.

So, how do I use Oathkeeper? Is it a replacement for Hydra?
The constraints I have right now is that I need an implicit grant returning a JWT access token. The client app code only supports standard implicit openid flow, and API only accepts RS256 JWT access tokens and the ability to specify well-known JWKS URL. I'd be so happy if it would just work with Hydra (or at least with Oathkeeper or maybe with Oathkeeper + Hydra somehow).

[aeneasr]

1. Don't use the implicit flow
2. Do you need an ID Token or an Access Token?
3. Why does the API only accept RS256 JWT despite various drawbacks?
4. Oathkeeper is not a replacement, the docs will definitely help you understand what it does and why.

[MOZGIII]

1. Don't use the implicit flow

Thanks for the links, that's something that can be addressed later. Though, currently things kind of rely on implementing things within the constraints.

1. Do you need an ID Token or an Access Token?

I need an access token extended with custom claims.

1. Why does the API only accept RS256 JWT despite various drawbacks?

What drawbacks are you referring to? The main point of accepting JWT there is to be able to verify the token without the access to auth server, or knowing which one (of many) auth servers issued the token.
Not sure if RS256 has any drawbacks.

1. Oathkeeper is not a replacement, the docs will definitely help you understand what it does and why.

Seems like it has to sit in front of the API. It's not obvious if it would work if we have not just Hydra, but also other issuers that provide JWTs. It doesn't seem like it.

[aeneasr]

What drawbacks are you referring to? The main point of accepting JWT there is to be able to verify the token without the access to auth server, or knowing which one (of many) auth servers issued the token.
Not sure if RS256 has any drawbacks.

You can't revoke tokens and the payload is transparent.

Seems like it has to sit in front of the API. It's not obvious if it would work if we have not just Hydra, but also other issuers that provide JWTs. It doesn't seem like it.

With the refactor, it could! It would even streamline the payloads

[MOZGIII]

You can't revoke tokens and the payload is transparent.

Oh, yeah, but that's no big deal. Benefits are more important.
We can blacklist tokens, I think it's as good as revoking. Transparent payload is not really a problem too.

With the refactor, it could! It would even streamline the payloads

It'd be cool to have something like that, however it looks like it wont work now. I really missed that lack of JWT support, and I basically already spent all the time I could on integrating with Hydra.

[aeneasr]

We can blacklist tokens, I think it's as good as revoking. Transparent payload is not really a problem too.

Blacklisting requires a network roundtrip which in turn negates the request-less validation.

It'd be cool to have something like that, however it looks like it wont work now. I really missed that lack of JWT support, and I basically already spent all the time I could on integrating with Hydra.

Sorry, JWTs won't be added to the project in the foreseeable future.

[MOZGIII]

That's ok, though I'd honestly recommend adding a note about the lack of JWT somewhere. It was really unpleasant to find out about it in the issues when the integration is done. That'd be at least some improvement in this area.

[aeneasr]

Yeah, we should probably have a "limitations" section in the documentation, I guess that would be really useful. Tracked as #839

[MOZGIII]

Blacklisting requires a network roundtrip which in turn negates the request-less validation.

Blacklist only has (and, in our case, can, since there's no single issuer that's responsible for all tokens) to live at the validation side, so no roundtrip is required there.

[aeneasr]

Unless you store the black list in memory, you will always have a roundtrip. Be it to the filesystem, or to the database. Sure, some dbs will perform better/faster for this specific task than a HTTP rountrip, but it adds the same possibility of failure.

I understand that you have multiple issuers which you want to trust where resource-provider-side validation makes more sense than requesting info from every issuer. It is however not according to the revokation spec, which specifically states that the token must be invalidated at the authorization server, not the resource provider. The problem is that the token will be marked as active at the authorization server, despite it not being active at the resource provider. Also keep in mind that revoking tokens is not just invalidating one token, but also "reovke tokens for all applications" or "revoke tokens for application x"

Indeed, it's quite the intriguing use case. Usually, apps accept tokens from one or two issuers, not many - so this is actually the first use case I've encountered where JWTs truly make sense. I still think it could be solved via oathkeeper, but I also understand the increase in complexity. You could obviously write something like oathkeeper yourself which is able to resolve tokens that aren't JWTs - you'd have all the benefits of revoking tokens.

Revoking tokens is a really important security mechanism that can prevent major threats if applied correctly. Not having that work properly may cause significant issues down the road.

[MOZGIII]

I need something really simple for the job. I'd use opaque tokens, but need to distinguish between the issuers to avoid verifying with all the issuers. Also typically different issuers will be only available in different environments. This authentication model is fundamentally different than the usual one, it's kind of more decentralized.

Relying on the ability to revoke a token is not always good. In some cases it's just not correct. It's not a silver bullet, and there are many ways to cover attack vectors that are typically covered by token revocation.

Seems like OAuth 1/2 had a paradigm shift, early on it was more like a framework that allowed, and, to some degree, encouraged implementing custom enhancements. Now it seems to be more like a set of some strict normatives. I'm counting that (the culture change, I guess) as the reason why support for JWT was eventually dropped.

[aeneasr]

Relying on the ability to revoke a token is not always good. In some cases it's just not correct. It's not a silver bullet, and there are many ways to cover attack vectors that are typically covered by token revocation.

Of course it's not a silver bullet, but it's a critical tool in your toolbox.

[aeijdenberg]

fyi as a drive-by comment, we were keen to find an OAuth impl to integrate with an existing product (that already has it's own OAuth server) which expects access tokens to be in the JWT format, and not supporting them will rule this out.

(as a side note, I fully agree that JWT is an inappropriate format for refresh tokens, so please read the below in the context of access tokens only)

I disagree about the drawbacks listed, and thought it might be productive to list the reasons why, in case it informs future product direction:

"You can't revoke tokens"

Correct. And that trade-off is why most access tokens are short-lived (ie 1 hour or less). I think an informed implementor can make this trade-off, which is a short period of irrevocability vs the benefits of statically verifiable tokens (which is good from both a latency and resiliency view).

"The payload is transparent"

For many resources server that is totally OK. The payload is essentially:

1. Identity of the user who granted the access.
2. Client to which it was granted.
3. Scopes
4. Expiration

These are not secret - and there is a latency and reliability benefit to being able to statically verify these.

[MOZGIII]

Btw, I think, if we really need revocation of JWT token, nothing stops us from supporting that. If we place an opaque string inside of a JWT, in a custom key, it can be effectively used to verify the session.
For my case it doesn't make sense though, because session is not controlled by issuers (by design).

[aeneasr]

Correct. And that trade-off is why most access tokens are short-lived (ie 1 hour or less). I think an informed implementor can make this trade-off, which is a short period of irrevocability vs the benefits of statically verifiable tokens (which is good from both a latency and resiliency view).

Actually, I agree. I think we could let developers make the trade off between timeliness of access revokation and avoiding network roundtrips.

For many resources server that is totally OK. The payload is essentially:

If you know what you're doing, then yes. However, developers are incredibly lazy (like mathematicians) and what happens often is that over time additional data is added to the token because "it's already there and I guess the data isn't really private", but it actually is. I'm not saying that this is the norm, just that this happens and is observable in the wild.

The question is if this project should protect you from doing this by enforcing an (opinionated) way of issuing access tokens or not. So far, my belief was that yes - we should enforce secure behavior even if it breaks the 1% use cases. I am, however, today more convinced that certain practices could be allowed if the devs know what they are doing. Having the default of issuing opaque tokens while also having a DO_YOU_KNOW_WTF_YOUR_DOING_OK_USE_JWT (jk) strategy _might_ be an option. It still goes against my beliefs of best practices in API access control which is to stay completely opaque (pass-by-value) to the outside world but transparent (pass-by-reference) on the inside (hence the Oathkeeper project). I do get the point which is integration with third party services though, and it is a valid point where the just mentioned structure does not help.

Btw, I think, if we really need revocation of JWT token, nothing stops us from supporting that. If we place an opaque string inside of a JWT, in a custom key, it can be effectively used to verify the session.
For my case it doesn't make sense though, because session is not controlled by issuers (by design).

That's negating the benefits of validating the tokens without an additional network roundtrip :)

[aeneasr]

To keep this discussion alive, I'm reopening the issue.

[mderazon]

Using jwts can be useful for service to service communication with password credentials grant.

Let's assume the common use case I have been seeing where opaque access tokens are terminated in an api gateway and transformed into JWTs internally to be consumed by various services (something like Oathkeeper is capable of)

Under this assumption, let's say service A wants to communicate with service B. It goes to Hydra and does the password flow to get an opaque access token. Then it calls service B with that access token.
Service B needs to validate that opaque token but the service already knows how to consume and validate JWTs, not opaque tokens.

One option would be to route service A --> B request through a gateway like Oathkeeper so that they will be terminated as well, but that's an overhead for internal service communication.

Also, at this point, if all services are sharing a secret, why having the overhead of calling hydra to get access token and then calling hydra again to validate that access token on the other side. They could just use a pre shared jwt as a secret and use that for authentication between them
(?)

[MOZGIII]

@arekkas, I’m catching up on this issue, but there’s an inportant point I’d like to state, regarding adding fields to jwt: it’s pointless to try to guard developers from stupidity (in the code level). There’s always a way to do things wrong. That’s why we have seniors, architects and other people that protect the dynamic systems from moving into wrong directions.
However, empowering people with reliable, robust and well-designed tools helps a lot with mitigating the possible stupidity issues. Purposely introducled limitations, that limit the options - like removed jwt access tokens support at hydra - really takes the control out of hands of users of a tool in a way that’s not helpful. Devs are lazy in general, but there are still teams that spend days designing just the interfaces, with people on keeping clean system boundaries :)
I’m not saying that the decision to remove JWT is wrong here - there is a rationale for it, but it’s limiting for sure.

[MOZGIII]

Btw, I think, if we really need revocation of JWT token, nothing stops us from supporting that. If we place an opaque string inside of a JWT, in a custom key, it can be effectively used to verify the session.
For my case it doesn't make sense though, because session is not controlled by issuers (by design).

That's negating the benefits of validating the tokens without an additional network roundtrip :)

Yeah, but what if you only need to validate the session in some cases (i.e. half of the endpoints)?

[someone1]

Just a quick plug here - I'd love for there to be JWT support for access tokens, I'd like it even better if we could plugin custom JWT signing mechanisms.

I mention this because I've effectively (read: foolishly) hacked the hydra bootstrap process to get my own database and JWT signing mechanism in place. I also had to wrap the oauth2.DefaultJWTStrategy used since it always expects a OpenID Session (See this commit) which you may run into while implementing this feature.

I know for the database part I could try the plugin route, but this doesn't make sense for my backend. There currently exist nothing for plugging in a custom signing mechanism.

If this opens the possibility to selecting a signing mechanism, maybe this can be made to be customizable? I'd suggest to use the model used by the https://github.com/dgrijalva/jwt-go package where you can dynamically register token signing types (which I did for GCP).

Just food for thought, I'd also like to see that for the database side too and would be happy to suggest a simple modification to the current codebase to enable this or even submit a PR!

I hope this all made sense, I'd be happy to discuss in chat or elaborate here if anything is unclear.

EDIT: Also, the use of oathkeeper doesn't make sense for my use case as I already plan on using an API Gateway from my cloud provider (not an equal comparison but still relevant) - I have an existing IDP to layer hydra on top of and can leverage other existing technologies for protecting my oauth2 protected endpoints- in my case an API.

[aeneasr]

Is there a particular reason for using a somewhat more exotic signing algorithm (like PS384) instead of standardised RS256? Keep in mind that most off-the-shelf libraries don't actually support those.

[aeneasr]

By the way, tampering with dynamically accepting token signing types can be a serious security vulnerability. The typical attack path is to change the alg part in the header (the easiest is setting it to none) and re-generate the signature. I remember reading that there was an attack for changing asymmetric to symmetric (e.g. HS256 -> RS256), but I'd have to find it again. Token signing and verification algorithms should be fixed (configuring this through an env var is an option) and everyone accepting those tokens should agree upon the algorithm to prevent malicious use.

[someone1]

Well I'd actually utilize RS256 myself, but leave the signing to GCP - they manage secrets and rotate keys so I don't need to worry about it. Its my preference to utilize a proper KMS.

Alternatively, maybe I can create a ticket to introduce GCP/AWS key management systems as pluggable backends for the jwk features of hydra? Keeping secrets is hard to get right, and separating keys such that not even hydra knows the private key greatly enhances it's security IMHO (separation of duties, utilizing a battle-tested system, etc.)

RE accepting token signing types: I'm aware of this and the jwt package does a good job of explaining how to mitigate this. I don't think this is a reason to prevent enabling the usage of such mechanisms - just helps those of us looking to extend the capabilities while adding arguably little-to-no overhead to the core hydra features/goals.

EDIT: Removed mention of KMS - not relevant for handling signing, only encryption. However, I know GCP offers a signing API with backend key management, not sure about AWS.

[aeneasr]

Does the Google KMS sign JWTs or just provide the keys via an API? I generally like the idea of integrating with KMS but would like to see a generic solution that spans GCP, AWS, Azure, (DigitalOcean?) as opposed to having a sizeable code base and maintenance overhead just to integrate with each proprietary system (and its breaking changes wrt to authorization and API usage, which has bitten me often - specifically the auth* part).

[aeneasr]

ps: key rotation is now a first-class citizen in hydra as that was a requirement for getting the OpenID Dynamic profile certified - just tell hydra to generate a new key hydra keys create hydra.openid.id-token -a RS256 -u sig (to be documented)

[someone1]

I don't think you'd get a generic solution that spans multiple providers. Just like you do with database backends, I'd expect a generic interface each implementation would have to satisfy for the backend wanted. I don't expect this to be a part of hydra, but something that we (as developers) could inject in a custom hydra deployment without having to do all the copy pasta as I did in my repository.

I was mistaken in thinking GCP/AWS KMS does signatures, it only does encryption/decryption. I do utilize GCP's IAM API to sign blobs and utilize them for JWTs - it uses the RS256 algorithm. I believe Vault by HashiCorp also allows secrets management with signing capabilties.

There could be other such systems out there that may be more secure than using what's baked into hydra, or just to enable separation of duties. I don't believe hydra would incur any maintenance overhead in enabling dynamic registration of backends for this, it would just ship with reasonable defaults.

To illustrate a bit on how I envision the change to be, I'll take the backend example where you can use a plugin vs SQL: Instead of a switch for the types supported, an interface would be defined for the backend registered that returns the manager type required.

handler_oauth2_factory.go becomes:

// Showing a simple interface for this specific file, probably would be a more generic interface that had functions which returned the proper backends for each of jwk/oauth2/client/consent/etc.
type FositeStoreManager interface {
     NewFositeStore(clients client.Manager, lifespan time.Duration) pkg.FositeStorer
}

var fositeBackends = make(map[string]FositeStoreManager)

func init() {
  // Load default supported backends
 fositeBackends["memory"] = oauth2.FositeMemoryStore
 fositeBackends["mysql"] = oauth2.FositeSQLStore
}

func RegisterFositeBackend(prefix string, backend FositeStoreManager) {
  fositeBackends[prefix] = backend
}

func injectFositeStore(c *config.Config, clients client.Manager) {
    var ctx = c.Context()
    var store pkg.FositeStorer

        if backend, ok := fositeBackends[ctx.DatabasePrefix]; ok {
             store = backend.NewFositeStore(clients, c.GetAccessTokenLifespan())
    } else {
        panic("Unknown connection type.")
    }

    ctx.FositeStore = store
}

Then in a package such as mine I could register such as:

import "github.com/ory/hydra/cmd/server"

func init() {
    server.RegisterBackend("datastore", MyDatastoreImplementation{})
}

[aeneasr]

That's a good idea - it's already possible to do that with plugins, so no need to compile it your on your own (at least not hydra). But it doesn't allow you to replace just one store, you have to write implementations for all of them (but you can obviously import e.g. client.SQLManager).

I'm not sure how many people compile hydra on their own and if adding this feature would benefit the 95% use cases. I do agree with you on separation of concerns and generally like the idea of integrating with cloud provider's KMS systems. The default store (SQL/Memory) will definitely not be removed as hydra should work standalone, but I don't think you're suggesting that anyways.

Maybe it would be enough to just provide jwk.manager_google_kms and jwk.manager_amazon_kms natively and make this somehow configurable?

[MOZGIII]

I'd love to see Vault support as well, and/or custom HSMs (for baremetal deployments).

[MOZGIII]

In general, an API to use external signer is better than a bunch of the built-in ones. Take a look at caddy server model of plugins - it's very easy to use and also very flexible. I think implementing a good extensibilitry model brings benefits beyond the scope of just JWT support, and would help project to move in the right direction in the long term.

[someone1]

The current plugin system for database backends isn't generic enough, it expects a *sqlx.DB implementation which doesn't make sense for all databases (e.g. any NoSQL database). Additionally, I want to inject my own middleware handlers for logging/tracing/security and other misc. things specific to my deployment environment. I don't think a plugin system is necessary for all these parts, but just exposing the bootstrap process a bit in the code to allow this would be helpful.

Either way, I'm really glad to hear that we can possibly support a pluggable system for signing here - I'd be happy to discuss this more design-wise and help with implementation if you'd like!

[aeneasr]

The current plugin system for database backends isn't generic enough, it expects a *sqlx.DB implementation which doesn't make sense for all databases (e.g. any NoSQL database).

I don't think it does. There is a *sqlx.DB reference but that is for re-using the already existing database connectivity if you want to. I admit that the code is a bit weird there but it's definitely possible to ignore *sqlx.DB and use your own connectivity.

Either way, I'm really glad to hear that we can possibly support a pluggable system for signing here - I'd be happy to discuss this more design-wise and help with implementation if you'd like!

It's not a super-high priority for us right now (first things first like getting out of beta for all services, enhancing the console, working on hive) but it's definitely something worth exploring once there's a bit of downtime.

[someone1]

Sounds good, I'll keep an eye on this issue when development starts.

The *sqlx.DB reference for plugins is required as when its loaded an immediate call to db.Ping() is made. I can open a separate issue to drop the reference altogether as the plugin should be able to manage the connection on its own and doesn't need the reference it returns to be passed into its own exposed functions as a parameter.

[aeneasr]

The *sqlx.DB reference for plugins is required as when its loaded an immediate call to db.Ping() is made. I can open a separate issue to drop the reference altogether as the plugin should be able to manage the connection on its own and doesn't need the reference it returns to be passed into its own exposed functions as a parameter.

Yeah you're right, feel free to track this whole discussion as a new issue as this one will probably be closed once JWT lands.

[aeneasr]

947