Opa: Support OpenId Connect Token Verification and Parsing

[jonmclachlanatpurestorage]

Expected Behavior

When a valid token given https://openid.net/connect/ standard arrives, we should be able to verify it and parse out the token's body in a single command.

Motive

Identity Providers sign and issue JWT tokens for auth services. These keys are publicly available.

For example, https://accounts.google.com/.well-known/openid-configuration
and following the link to the valid JWKs, we find the signing keys for all tokens issued by google's IdP, at, https://www.googleapis.com/oauth2/v3/certs,

#
# Data
#
jwks = `{
keys: [
{
kid: "b2ed0db1f661d8989969bab78d2fae7564fdc3a9",
e: "AQAB",
kty: "RSA",
alg: "RS256",
n: "qOcCdoOH4mcM7oyR6SurQ50dQLRkOjM4e4TbCZAHpBX5rdY4wK-jKjvrwJoFQ220G47BypMnnXBFjd7WbO5iQQzP6oIzwQF-7JI2hzqUUSNc44rYldunmjshl-xXplPuvRqcAQn9s3XM8NkbXcuMQ3azZO0ZL7XIhm3Zah5Hhc0MVbQxMiUSK2A7phV5JXS4fUJhfKb02RIj_ZtbgpJUo2A7IUzvgQHFiKf-AhVYiwKY2nuIdUbOyTY0JYw6KW4U8rv8Aga2H4WhGy7jlcZswwmnJMzergQP3Wuq41Ap_13Kc5XXt18sWMD6ixYCOU6lPCco0hr7iO-V1t59oDfgJw",
use: "sig"
},
{
use: "sig",
kid: "d8efea1f66e87bb36c2ea09d837338bdd810353b",
e: "AQAB",
kty: "RSA",
alg: "RS256",
n: "t6-oVSVZRRlgonK71la2PcpFitts99Jw7GiSCsxWpuqxNJjsux0PuNuQWrynDB7qdTuEDRsmrXYSemIXcqqmnWkM_UPzFkp7CW0Fyr-jLHERXzk55OUwc6X41hjgJqBYnGwWLeM3TyZmEliPYAJQIR09tABPwfo0aWFiVSPWtHrfwJbcUnX99i9YtB7fYwmpo4_ESEcwKwj10wpAMe33dfP51AulOTwv-YC2vLjw25h90yHh9ri6n7N-MG5lgqcup_1g8UGKD5KpLTNJas4fOBgmajHis-iW6UC3K2hjgerTF_d7VNX6VndBjQyPz9iRpzQ3wQZWGhng50SxS6-LXQ"
}
]
}`

So let's say you want you're building an app that trust google for authentication/authorization services, and so, you need to trust these tokens. (Though, often times it's bad form to use externally issued tokens, internally, and normally, we exchange external tokens for an internal token, using the spec, https://tools.ietf.org/html/draft-ietf-oauth-token-exchange-19 , I am just going to use Google's IdP as an example for a trusted issuer).

And then, we could verify and decode the JWT's directly, with something like,

#
# Authorization
#
default allow = false


#
# Basic Sanity Checking.
#
allow {
    input.path = []

    incoming_jwt_token := input.headers["Authorization"]
    io.jwt.verify_rs256(incoming_jwt_token, jwks)                  # Verify the token with the JWKS
    [header, payload, _] := io.jwt.decode(incoming_jwt_token)      # Decode the token

}

But if we step back and look at the bigger picture, all we want to say is,
(1) We trust the set of IdPs: ["https://accounts.google.com"]
(2) Given (1), please verify and parse this token X.

We can not use the provided functionality in OPA's JWT tooling to accomplish this (the above code is broken), because, the keys that google will signing tokens with will change (dynamic). OpenId Connect is extremely convenient, as it provides a key-resolution protocol for how to go from a JWT to the actual public JWK that signed it, for verification purposes. Once the key-discovery mechanism is built, the only other thing we need besides the token itself is the list of the trusted issuers. We never want to just accept anyone's tokens, instead, we control it with a whitelist of trusted issuers.

Description

I propose one new builtin opa action,

io.openid.verify(token, string_list_of_trusted_issuers)

Returns true iff the token is valid given the list of trusted issuers.

Note this is different from https://github.com/open-policy-agent/opa/issues/1205, as the client_id, client_secret, token_url, and scopes are not required and has nothing to do with actually issuing/exchanging tokens as an OAuth2.0 client. Given the OpenId connect standard, the only required component is the list of trusted issuers, which is a list of URL strings pointing to the OpenId Connect compatible IdP's.

Of course, we can cache the keys in the usual sense. Though, there are libraries that abstract all this out for us already, so would be very straight forward implementation through the use of open-source libs.

Open to ideas/suggestions here.

[anderseknert]

Great suggestion! Having pre-configured list of issuers would greatly simplify JWT verification. Since the list of trusted issuers is going to be more or less static I would propose it to belong in the server configuration rather than provided to the verify function directly.

Do note that OAuth2 backported the OIDC metadata spec pretty much verbatim, so it would make sense to not limit this to OIDC and ID tokens but have any type of JWT containing an issuer (iss) claim be matched against the list of trusted issuers. So I'm thinking something like this:

1. The server configuration containing a list of trusted issuers (HTTPS URLs and possibly the type of issuer (OIDC, OAuth2, etc).
2. OPA server would use the /.well-known metadata endpoints of configured issuers to obtain the location of the jwks_endpoint for each one, query that periodically and keep the set of keys in memory.
3. Incoming tokens could now be verified against the JWKS in memory, using the iss from the JWT claims and the key ID (kid) of the JWT header.

The only thing different between the OIDC and OAuth2 implementations would be the location of the .well-known endpoints, thus why the issuer type would need to be provided.

[jonmclachlanatpurestorage]

Not a bad suggestion at all. But the thing is, I have no need for non-openid compliant tokens today, and,

io.openid.verify(token, string_list_of_trusted_issuers)

seems like a great start. I am 100% open to someone extending the base functionality later when there's a real market push for OAuth too. But today, getting oidc quickly is a little more important to me than getting more general features.

Also, in terms of hiding a list of trusted issuers in a config, I am just worried that the trusted issuers list might miss out on the benefit of a single-pane-of-glass we get today in the rego policies if it's hidden. If it's off in some configure, it's not the end of the world, but now for an auditor to come in and verify compliance (for SOC2 for example), now the config file is security-critical, instead of just the rego's.

Also, not sure if list of trusted issuers would be easily managed by Styra's beautiful interface if it ends up as a config vs in a rego policy.

[anderseknert]

Sounds good to me Jon - all I was trying to say is that by building an OIDC compliant verifier, you have built one that would verify an OAuth2 JWT access token too, whether intentional or not. In that sense perhaps

io.jwt.verify(token, string_list_of_trusted_issuers)

could be considered, rather than having to add an identical io.oauth2.verify(token, string_list_of_trusted_issuers) function later just for naming.

Anyway, it's a good addition regardless.

[simongottschlag]

I really like the idea of moving the logic of validating the whole token to OPA. That would mean we don’t need that logic in the API and delegate it all to OPA instead.

[johntron]

Caching JWKS responses based on response headers (ETags, expiration, etc.) is essential here - see https://github.com/open-policy-agent/opa/issues/1925#issuecomment-654994262. Not only will rate-limiting make this feature useless without caching, but it's also important to support changing/rotating keys.

[anderseknert]

What we do, and what could possibly work for you as well @johntron is to provide the JWKS data to OPA either through the data REST API or in a bundle containing both policies and data. This allows you to control exactly when and how often you want to push updated JWKS data to your OPA instances, and allows you to keep your policy free from caching details.

[johntron]

@anderseknert yeah, I considered that but went with using Istio's RequestAuthentication instead for a variety of reasons. Pushing external data (JWKS) through the REST API or bundles would add a lot of complexity, and keeping them in sync with the source of truth on JWKS freshness/validity would be even more.

[anderseknert]

Yeah, if you aren't already pushing other data or using bundles, adding that just for JWKS objects would indeed add some complexity, but on the other hand the complexity is kept outside of your policy.

[tsandall]

@johntron adding support for caching based on standard HTTP headers makes sense. Are you sure that will work for your use case? I've seen some token generation endpoints not set the expiration information in the HTTP headers and only include them in the encoded tokens. I'm definitely in favour of improving http.send to support standard HTTP headers.

[johntron]

I've seen some token generation endpoints not set the expiration information in the HTTP headers and only include them in the encoded tokens.

I believe that would be a problem from a security perspective, because there must be a way to change the JWK Set in the case of a leak/attack. In the scenario where an unauthorized party obtained access to signing keys, you'd want to immediately revoke the JWK Set generated using the compromised keys.

I'm no expert, but I believe it's also common practice to periodically rotate signing keys to limit exposure in the event of compromised systems. For example, a JWT with an expiration date inadvertently set far in the future could be obtained and used many months after it was generated if the original JWKS is still available.

For these reasons, you really do need to check periodically for JWKS changes; however, it's not really necessary to check on every request.

Here's a few of auth0's JWKS response headers:

Access-Control-Allow-Credentials: false
Access-Control-Allow-Origin: *
Cache-Control: public, max-age=15, stale-while-revalidate=15, stale-if-error=15
Connection: keep-alive
Date: Wed, 08 Jul 2020 18:32:01 GMT
ETag: W/"bfc-sOhGJ7Sl3SHBJ0O3l4mmIznhO0g"
Strict-Transport-Security: max-age=15768000
X-Cache-Status: BYPASS
X-RateLimit-Limit: 20
X-RateLimit-Remaining: 19
X-RateLimit-Reset: 1594233122

[RichiCoder1]

The introduction of http.send caching will definitely make this less of a priority (thanks for that!), but it'd still be nice to see that logic get completely take care of by OPA since this is going to be a pretty common case.

Something like Envoy's now alpha API for configuring a number of valid providers for a given JWT would be great: https://github.com/open-policy-agent/opa/issues/2462

Eg:

io.jwt.validate(token, providers), with providers being a set of valid issuers and optionally things like audiences, explicit local or remote JWKS, and toggles for token claim checks.

Would allow maximum flexibility and still push some pretty complicated logic out of rego.

[tsandall]

Would allow maximum flexibility and still push some pretty complicated logic out of rego.

I'm not sure we want to push that much flexibility and logic into a built-in function. Built-in functions by their very nature are opaque; they exist to implement behaviour that's not feasible inside of Rego. Expressing a bunch of audience and claim validation is exactly what Rego is good at.

Since we now have http.send caching in master and users can fetch the JWKS relatively easily, I'm tempted to close this issue.

[RichiCoder1]

Since we now have http.send caching in master and users can fetch the JWKS relatively easily, I'm tempted to close this issue.

That's fair, but even reducing it down to just A) is this a well-formed token that B) comes from a trusted issuer and C) is signed with a valid key from that issuer's jwks is a fair bit a boilerplate. Even if this is something that can be done by the user, it'd greatly reduce the effort to implement this common pattern. I suppose documented rego snippet that people could copy and reuse would also accomplish that now however.

[konstantin-tkachuk]

Any updates on this issue? I am right now facing the exact same challenge - verifying the signature of a JWT using JWKS, where I need to check if the IDP is a trusted one. I can fetch the JWKS using http.send with cache enabled, but the problem is rotating keys. Should the IDP rotate the keys there is a time interval while the previous JWKS is still cached - until this cache runs out all incoming new tokens will simply be rejected.

Is there a solution or workaround that I can use? @jonmclachlanatpurestorage if I may ask, how did you solve this issue?

[anderseknert]

@konstantin-tkachuk is the IDP's rotation instantaneous? I.e. does it not keep more than one key ID (the old one plus the new) for any duration of time? And does it in any way use the cache headers to signal when a rotation is scheduled? Just curious on what a solution to this would entail.

Would something like this work in the meantime?

1. Get JWKS object using http.send with caching configured.
2. Check for presence of key ID from incoming JWT in cached JWKS object.
3. If present, use cached JWKS to verify JWT.
4. If missing, use another (i.e. wrapped in another rule or function) http.send call to fetch a new JWKS object and use that for verification.

[anderseknert]

Checking the OIDC specification, the above steps are almost identical to the rotation scheme they suggest:

Rotation of signing keys can be accomplished with the following approach. The signer publishes its keys in a JWK Set at its jwks_uri location and includes the kid of the signing key in the JOSE Header of each message to indicate to the verifier which key is to be used to validate the signature. Keys can be rolled over by periodically adding new keys to the JWK Set at the jwks_uri location. The signer can begin using a new key at its discretion and signals the change to the verifier using the kid value. The verifier knows to go back to the jwks_uri location to re-retrieve the keys when it sees an unfamiliar kid value. The JWK Set document at the jwks_uri SHOULD retain recently decommissioned signing keys for a reasonable period of time to facilitate a smooth transition.

Translated to Rego:

package oidc

jwks_request(url) = http.send({
    "url": url,
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 3600
})

verified {
    jwks_cached := jwks_request("https://oidc.example.org/jwks").body
    io.jwt.decode(input.token)[0].kid == jwks_cached.keys[_].kid
    io.jwt.verify_rs256(input.token, json.marshal(jwks_cached))
} else {
    # Add query param to second request to avoid both getting the same cache key
    jwks_rotated := jwks_request("https://oidc.example.org/jwks?r=1").body
    io.jwt.verify_rs256(input.token, json.marshal(jwks_rotated))
}

I.e. keep reusing the cached JWKS until an unknown key ID (kid) shows up in a token, at which point we assume the keys have been rotated and request a new JWKS. Cache duration etc should obviously be tweaked from case to case, but the above snippet should be a good starting point.

[konstantin-tkachuk]

@anderseknert Thank you, this really helped!
It seems my biggest problem was a misunderstanding of how exactly key rotations work. I believe with this additional information it should no longer be a problem to implement my use case.