Caddy: v2: Design docs for authentication plugins

[mholt]

1. What would you like to have changed?

Caddy 2 needs more authentication modules. Right now, it supports HTTP Basic Authentication, but there are more methods we need to support:

• OAuth 1 & 2
• OIDC
• SAML
• BeyondCorp
• LDAP
• JWT
• (Others? _What do you need?_ Please comment below!)

This issue is to discuss the design of more authentication modules.

Once a user is authenticated, the server should be able to forward/expose the user info (e.g. GitHub auth token, if authenticated via GitHub, for example) to the upstream via:

• HTTP Headers
• ... other methods? (Comment below)

so that it can know how to compute authorization (permissions).

What are your authentication needs / workflows? Be as specific as possible so that we can implement exactly what you need into Caddy. One or two sentences is probably not enough. Linking to how other software works is better, but not great, because we do not simply want to repeat what other software does -- the goal is to do it better, if possible.

The community is invited to collaborate in building these, so if you're interested, please get involved!

2. Why is this feature a useful, necessary, and/or important addition to this project?

Many companies are using service meshes primarily for authentication, when they do not actually need a whole service mesh.

If they could use Caddy as an ingress controller / reverse proxy with these authentication features, they would likely be able to replace their entire service meshes with just Caddy, and vastly simplify their infrastructure and lower their costs.

3. What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

Use more complex infrastructure to deliver authentication.

4. Please link to any relevant issues, pull requests, or other discussions.

HTTP Basic Auth was implemented in https://github.com/caddyserver/caddy/commit/f8366c2f09c77a55dc53038cae0b101263488867.

[bnfinet]

excited to see what's in store for Caddy 2!

The Vouch Proxy community would like to integrate with other web servers and ingress controllers.

To do so we'd need something similar to the Nginx auth_request module...

1. pass the headers to a sidecar such as Vouch Proxy
2. if sidecar responds with 200 continue
3. if not, execute an error block to 302 redirect to an auth flow

[1] https://github.com/vouch/vouch-proxy/

[shairozan]

Would really love to see a PAM authentication mechanism. We had a need for this recently and wrote focal to accomodate it.

It supports a typical POST to a /login endpoint (Could just as well be http auth) where validation for the user details are done in PAM and a JWT is returned which is used to authenticate anything downstream.

We used a simple library called tekmor to handle the actual PAM authentication component.

[empijei]

Some points you might want to consider:

• You say that caddy should forward authentication info to allow the backend to compute authorization. Maybe consider allowing authorization on a caddy level and forwarding roles to the backend.
• There might be a need for multilogin
• Provide a mechanism to protect debug endpoints
• Tokens like JWT might be in a cookie or in a bearer header, so there should be a way to handle both cases.
• If you have auth on caddy you might want to consider a possibility for user sharding: if the user info is exposed internally by the auth plugin it could make it available to balancing and other features.
• Potentially out of scope: even if credentials are valid caddy is in the position of detecting credentials theft: if the same credentials are used by very distant places or in shady patterns caddy could warn the backend or prevent auth altogether.

[sarge]

What am I missing?
Caddy has an awesome extension mechanism for adding plugins / middleware.

[mholt]

This is all excellent feedback so far, thank you everyone! (Feel free to keep it coming.)

I'll be working on authentication features after beta 13 is released (that's the next beta release).

@sarge Yes, you're right. :) I'm going to build an official authentication module. This is for discussion about specific features we want in that module.

[christian-korneck]

(Others? What do you need? Please comment below!)

I would like to suggest Kerberos / NTLMv2

I know, it sounds a bit like from a different age but krb5 is still the default logon and SSO method for traditional Windows / Active Directory networks when the AD isn't connected to Azure AD or self-hosted AD Federation services. All browsers and OSes support it. It's still often used for Intranet webapps and I think it will still be around for years to come. AD also supports LDAP (which you already have on the list) but Kerberos has multiple advantages over it. Examples: It's passwordless after the OS login, no auth traffic to the domain controller/ldap server for every single http request to check the password, it also works with smartcard login (at least on windows), etc.

[roblabla]

So, as mholt knows, I've been working on a port of caddy-jwt (and loginsrv) to caddy2. While doing this, I've come across a couple of things that I find are missing from the Authentication module that I'm going to document here:

1. As previously noted, User is a bit lacking. I'm preparing a PR that will expand it to have a generic metadata map store, and maybe add some common fields (at least username). Those fields will be exposed to the replacers.

2. Caddy's old JWT module had some amount of configuration on what happens on authentication failure (it could either return forbidden, redirect the user to the login page, or "passthrough" the request without setting the user authenticated headers/vars).

   Currently, when the caddy2 authentication fails, the Authentication module simply raises an error from ServeHTTP, with caddyhttp.Error(http.StatusUnauthorized, fmt.Errorf("not authenticated")). Ideally, users need to be able to configure what happens when authorization fails. I guess this is currently possible by wrapping the Authentication module in a subroute and providing an error route (like this) but it's frankly not great (I feel like it's very, very verbose)

   In my caddy-jwt fork, I just do the redirect directly within the func Authenticate, but I feel like that's not such a good idea.

---

While working on the above config, I came to the realization that replacer and vars are different concepts. IMO the vars matcher should have access to everything available in the replacer, so I can write { "matcher": "vars", "http.error.status_code": 401 } without the extra vars middleware. I'll make a separate issue for that.

EDIT: Make an issue for the matcher stuff: https://github.com/caddyserver/caddy/issues/3051

[mholt]

Great discussion so far.

@roblabla

Currently, when the caddy2 authentication fails, the Authentication module simply raises an error from ServeHTTP, with caddyhttp.Error(http.StatusUnauthorized, fmt.Errorf("not authenticated")). Ideally, users need to be able to configure what happens when authorization fails. I guess this is currently possible by wrapping the Authentication module in a subroute and providing an error route (like this) but it's frankly not great (I feel like it's very, very verbose)

The verbosity here is intentional: it allows the user to configure literally _any_ behavior that the primary handler chain is also capable of, but in response to an error. So you can handle errors by showing an error page, sure, or you can reverse proxy to a backup server, or issue a redirect, or you can write a static response. It's very powerful! (But it can be a little cumbersome sometimes.)

Caddy's HTTP error values are (supposed to be) structured, so you can get not only an error string, but an ID, the actual error value (which can be asserted to a concrete type), the recommended status code, and the trace.

In the case of a failed authentication, it may be useful to access that actual error value to make certain decisions based on the kind of error. This should be doable already, a handler just needs to be programmed to do it.

Another option is to make the Authentication handler accept an embedded route configuration to invoke if authentication fails. This could help simplify some more complex decision logic, like how to respond to certain kinds of authentication-specific errors, and keep it all in one place. The existing subroute handler does this, but in a more general sense. I'm not sure if you'd want to use an *HTTPErrorsConfig or embed a RouteList directly, just kind of depends. I can help give guidance here.

In my caddy-jwt fork, I just do the redirect directly within the func Authenticate, but I feel like that's not such a good idea.

I think I agree, we should expose a little more surface for configuring how to handle certain kinds of errors, with more flexibility where possible.

[roblabla]

I think the thing that bugs me with using the errors routes is that if an error chain exists, it will override the default behavior of logging the error and setting the status code to whatever was provided (or 500 if it wasn't a HandlerError). E.G. if those middlewares are removed, then the server will return a blank page with code 200 (instead of 401) and no error will be logged. This means that to handle one specific error, you need to add a bunch of boilerplate to recover the original behavior in other error cases.

And in our case, we'd certainly want any errors unrelated to auth to be logged! So I find the status quo rather unpleasant.

I'm unsure how to deal with it though. I couldn't even figure out what makes the server return 200 - I guess it's either go's http server doing this silently or I'm missing something (wouldn't be surprising, I'm by no means a Go expert).

Maybe the boilerplate I provided should be automatically appended to the error handling chain if it exists?

[mholt]

@roblabla Well, remember we're still in beta, so now's the time for us to iron out these inconveniences.

There's no _particular reason_ that an error chain prevents logging the error and responding with the status code. That actually sounds like a bug rather than intended behavior. Can you open a new issue to describe a minimal config that reproduces it, and I can look at it?

[roblabla]

(I got a small detail wrong: the error is logged regardless, I couldn't see it in the logs because they only happen when we return 5XX error codes, and I only tested with 4XX haha. Status code is still unset though).

I made an issue about it, here it is #3053

[roblabla]

So with #3053 on its way to being fixed and #3051 currently being discussed, the amount of boilerplate necessary for a minimal JSON doing (for the sake of the example) a redirection in case of auth failure got much smaller:

json config (click me)

{
  // Disable admin for test
  "admin": { "disabled": true },
  "apps": {
    "http": {
      // Use http to avoid having to provision an ssl cert
      "http_port": 4444,
      "servers": {
        "srv0": {
          "listen": [ ":4444" ],
          // Simple authentication route that will invariably fail (since
          // nothing is allowed)
          "routes": [{
            "handle": [{
              "handler": "authentication",
              "providers": {
                "http_basic": {}
              }
            }]
          }],
          // Based on the error code, we either want to redirect or return an
          // error.
          "errors": {
            "routes": [{
              // If status code is 401, auth failed, show a pretty error
              "match": [{
                "equal": { "{http.error.status_code}": 401 }
              }],
              "handle": [{
                "handler": "static_response",
                "status_code": "302",
                "headers": {
                  "Location": ["/login"]
                }
              }],
              "terminal": true
            }
          }
        }
      }
    }
  }
}

As far as the Caddyfile goes, I see there's a new handle_errors directive available since this commit. I think it should make it possible to write something equivalent to the above like so (again, assuming we get the compare directive from #3051):

@auth_error {
    compare {
        {http.error.status_code} = 401
    }
}
http://localhost:4444 {
    basic_auth {}
    handle_errors @auth_error {
        redir /login
    }
}

This more or less resolves all my complaints about the error handling story of the Authentication module, I think. Only thing I'd change, maybe, is some way of knowing for sure the error came from the authentication module (and not some other plugin), e.g. by introducing a new replacer item like http.middleware.authentication.error or something.

EDIT: Though in jwt, I think I'll keep doing things like jwt { redirect /login } and the other various convenience functions currently in jwt. The only difference being that it would expand to something similar to what is done in the above JSON, instead of being handled in the Authenticate type.

[greenpau]

Hi All, I propose separating auth* plugins into the following categories:

• firewall: handling authorization based on L3 info (source/dst IP address, etc.)
• authorization: handling verification of grants/tokens/grants and performing authorization
• authentication: handling authentication requests and issuing credentials/tokens/grants
• something else I did not think of ...

Principles:

• authorization should precede authentication
• authorization layer must be able to pass data to authorization layer
• both plugin types should have access to global configuration

Let's take an example of JWT token issuance/validation. We have multiple routes that require protection. So, we add "authentication" handler as the very first handle for the route. Then, we go to a different route. and do the same ... wait, do we need to redefine the configuration because it is a different instance of a plugin?

https://github.com/greenpau/caddy-auth-jwt/blob/8a4f364b34590b798d7fa228cc0992a819a7f8a8/assets/conf/Caddyfile.json#L17-L131

Yes, we do:

{"level":"info","ts":1587593245.7803967,"logger":"http.authentication.providers.jwt","msg":"provisioning plugin instance"}
{"level":"info","ts":1587593245.780405,"logger":"http.authentication.providers.jwt","msg":"found JWT token name","token_name":"JWT_TOKEN"}
{"level":"info","ts":1587593245.7804825,"logger":"http.authentication.providers.jwt","msg":"provisioning plugin instance"}
{"level":"info","ts":1587593245.7804863,"logger":"http.authentication.providers.jwt","msg":"found JWT token name","token_name":"JWT_TOKEN"}

bottom line ... access to shared configuration pool.

[greenpau]

access to shared configuration pool at the time of the initialization of the plugin

Also, if there is more than one instance of a plugin, which instance will take care of "Validation"? What about Provisioner? Who is responsible for provisioning? Are we "locking" when Provisioner runs?

[mholt]

@greenpau There is a facility in the Caddy core which can help with pooling global state for all instances of the modules: https://pkg.go.dev/github.com/caddyserver/caddy/v2?tab=doc#UsagePool - the UsagePool is currently used by Caddy's reverse proxy to keep track of the state of upstreams across multiple instances of the proxy handler, and for keeping track of log writers. You could use something similar for JWT configuration, for example.

As far as "who gets to do Validation/provisioning/etc", the UsagePool helps with that, since it's "first-come-first-serve, and everyone clean up after yourselves" kinda thing. There's no guarantee as to what order modules are loaded, so it's best to use the UsagePool for things like this where the pool knows whether something needs to be provisioned or not. (We can chat on Slack if you have technical questions about that.)

As for chaining handlers: authorization -> authentication -> authorization -- sure, but maybe a "supermodule" should do that, i.e. a simple wrapper module in which you configure the authz/authn once and then it does the chaining for you.

[greenpau]

so it's best to use the UsagePool for things like

@mholt , agreed! :+1:

[KittyKatt]

Proxying protected endpoints into something like Authelia would be nice. It has support for OTP and Duo Mobile Push for 2FA, can be configured to use backends like LDAP already. I've looked over the docs since switching fully to caddy2 this last week and don't currently see a way to do this.

Example nginx configuration: https://docs.authelia.com/deployment/supported-proxies/nginx.html

[Mohammed90]

@roblabla said elsewhere:

Thinking about this more, the caddy authentication module kind of merges several concept that could be kept distinct: the authentication flow, authentication backend and session management.

For instance, caddy-jwt is clearly a session management thing. It doesn't actually authenticate anyone, it just looks for a signed cookie and populates the caddyauth.User based on this. The user must have been authenticated somewhere else to obtain that JWT token.

basicauth is doing both authentication flow (e.g. setting up the header to make the browser show a password prompt) and authentication backend (the username/password list that's stored in the JSON). There's no reason basicauth couldn't take its username/password list from another source/backend (a database, an LDAP, etc...).

If we had a properly separated authentication backend module, it could then be shared with the authentication flow modules (e.g. ssh password, HTTP basicauth, HTTP login form...). Of course more special types of authentication flows (such as ssh pubkey) won't work with this system, but I think that's fine.

That seems like a proper separation of concerns. With that, I imagine the config would be something like:

{
    "authentication": {
        "flow": {
            "handler": "username_password"
        },
        "credentials": {
            "provider": "ldap",
            "hostname": "ad.contoso.com",
            "dn": "cn=contoso,dc=foo,dc=bar",
            "scope": "",
            "username": "of_the_ldap_server",
            "password": "to_the_ldap_server"
        },
        "session": {
            "module": "jwt"
            // ...
        }
    }
}

The basicauth could be a special handler or plugin that is in fact a shortcut which expands to something like the above.

[greenpau]

The basicauth could be a special handler or plugin that is in fact a shortcut which expands to something like the above.

@mholt , this is basically https://github.com/greenpau/caddy-auth-forms with LDAP backend.

https://github.com/greenpau/caddy-auth-forms/blob/master/assets/conf/Caddyfile.json#L48-L53

                        {
                          "type": "ldap",
              "hostname": "ad.contoso.com",
              "dn": "cn=contoso,dc=foo,dc=bar",
              "scope": "",
              "username": "of_the_ldap_server",
              "password": "to_the_ldap_server",
                          "realm": "local"
                        }

[mholt]

@Mohammed90 @roblabla I definitely like the idea of separating the list of users into various modules, and having the actual authenticating be done by their own modules.

@greenpau That's pretty slick -- shall we converge on something like that proposed?

[greenpau]

shall we converge on something like that proposed?

@mholt , I think current auth plugin structure works for me ... each plugin may abstract the auth* the way it works for that plugin.

The conversation about "flow - credentials - session" is another angle/perspective on AAA (authentication, authorization, and accounting).

        "session": {
            "module": "jwt"
            // ...
        }

In the above example, the "session" is handled by module jwt. What is being said there.
Is it a validator or grantor? Are we trying to consolidate auth* in a single plugin (monolith) or do we want modularity?

I am for modularity.

For example, when I implemented jwt plugin, I created a grantor object (and validator object).

https://github.com/greenpau/caddy-auth-jwt/blob/6cdfa3290afde2821526ec27c4f0d1f740e4c066/grantor.go#L7-L38

I also did it so that both the grantor and validator share a set of common parameters.

https://github.com/greenpau/caddy-auth-jwt/blob/6cdfa3290afde2821526ec27c4f0d1f740e4c066/common.go#L5-L13

Any developer from other plugins may implement issuing tokens using this object. Another plugin may import jwt module and access AuthzProviderPool.

Importantly, the AuthzProviderPool contains Masters for each authorization context.
Therefore, if you are a plugin developer and you want to issue a token, you copy
common parameters from the master for a context (that means you get the secret/key),
and issue tokens.

https://github.com/greenpau/caddy-auth-jwt/blob/6cdfa3290afde2821526ec27c4f0d1f740e4c066/plugin.go#L24-L31

---

Here is how forms plugin issues tokens:

1. User claim gets constructed at the backend:

https://github.com/greenpau/caddy-auth-forms/blob/99388d341f1461af7de00aa48b4c19263719635a/pkg/backends/local/backend.go#L156-L180

    claims, err := jwt.NewUserClaimsFromMap(userMap)
    if err != nil {
        return nil, 500, fmt.Errorf("failed to parse user claims: %s", err)
    }
    if claims.Subject == "" {
        claims.Subject = user.Username
    }

    guestRoles := map[string]bool{
        "guest":     false,
        "anonymous": false,
    }
...

1. Independent from a backend, the token gets issued (for clarity instead of TokenProvider I should have called it TokenGrantor; I was mocking the interface on a fly ... that's why it happened).

https://github.com/greenpau/caddy-auth-forms/blob/master/plugin.go#L446

userToken, err := userClaims.GetToken("HS512", []byte(m.TokenProvider.TokenSecret))

In this example, I do not go to the AuthzProviderPool to get secret .. I have a configuration section for that. (but I could do it). It probably makes sense when everything is in "default" context. It could be useful when you have a single instance of a server.

[hbagdi]

Hey @mholt,
Would you be open to adding an auth plugin to enable external authentication/authorization?
This goes along the lines of auth request module in Nginx and external authorization filter in envoy.

[trusch]

@hbagdi I think I might be able to write an external auth plugin (I also need it). @mholt Is there anything I should read before starting it?

[greenpau]

Would you be open to adding an auth plugin to enable external authentication/authorization?

@hbagdi , you can do things of this nature https://github.com/greenpau/caddy-auth-portal#google-identity-platform
It will not be straighthrough, but will likely follow a redirect.

[trusch]

@hbagdi You can have a look at this: https://github.com/trusch/caddy-extauth
Its working for me and there is a example compose.yaml and a Caddyfile to get you started. If you run into any problem just open an issue and ask me to add a good readme ;)

[daiaji]

HOTP/TOTP
In fact, I rely more on this simple offline Authentication Scheme.

[0az]

You can have a look at this: trusch/caddy-extauth
Its working for me and there is a example compose.yaml and a Caddyfile to get you started. If you run into any problem just open an issue and ask me to add a good readme ;)

What would it take to get something like this merged into Caddy main? The auth_request style seems to cover enough of the authentication space, through Vouch or just writing something custom.

[mholt]

What would it take to get something like this merged into Caddy main?

Unlikely to happen -- specific implementations (other than the simple standard HTTP basic auth) belong as separate plugins. The standard Caddy modules already facilitate general/abstract authentication, so it does its job already since you can add any other auth on top of it.

[lanrat]

What would it take to get something like this merged into Caddy main?

Unlikely to happen -- specific implementations (other than the simple standard HTTP basic auth) belong as separate plugins. The standard Caddy modules already facilitate general/abstract authentication, so it does its job already since you can add any other auth on top of it.

One of the biggest benefits of providing something like auth_request is that it can allow plugging into other auth frameworks without writing new plugin code. It is my understanding that with the current caddy system, adding anything other than basic auth requires code, and not just writing a configuration file.

[mholt]

I mean, even with auth_request you still have to write code: it just wouldn't be in Caddy. But that's actually counter to Caddy's philosophy, which _is_ to plug the code directly into the server: fewer moving parts.

[lanrat]

Fair point. I guess a plug-in that implements auth_request would be the best approach.

[hbagdi]

The standard Caddy modules already facilitate general/abstract authentication, so it does its job already since you can add any other auth on top of it.

Caddy does have the necessary authentication abstractions but there is no way to integrate authentication proxies without writing some code in form of a Caddy plugin and then bundling that with Caddy. That leads to a higher friction than necessary.

If there is enough demand, would you be open to bundling a module like auth_request or envoy's ext-authz into mainline Caddy? That way, users write only configuration, and not code to get up and running.

[mholt]

I am not sure what is involved to support authentication proxies, but feel free to draft up a proposal for design discussion to augment Caddy's standard http.handlers.authentication module: https://caddyserver.com/docs/modules/http.handlers.authentication

No guarantees but if it's fairly unintrusive (and introduces no new dependencies, especially), extensible, and generally useful, there's a good chance it could make it in.

[0az]

No guarantees but if it's fairly unintrusive (and introduces no new dependencies, especially), extensible, and generally useful, there's a good chance it could make it in.

Based on caddy-extauth, this should be doable without any new dependencies – the only direct non-Caddy dependency is zerolog, which should be unnecessary.

If nobody else writes something up, I'm willing to toss my hat in... eventually.†

† At least three to six months from now.

[francislavoie]

the only direct non-Caddy dependency is zerolog, which should be unnecessary.

Yeah, it should use zap which is the logger Caddy uses. https://caddyserver.com/docs/extending-caddy#logs