Pomerium: Keycloak with `idp_service_account` and `allowed_groups`

[amiral-fu]

Hello,

What happened?

From Pomerium 0.10.0 and the change :

config: add check to assert service account is required for policies with allowed_groups

Use allowed_groups policy for access to an app is not possible without configuring idp_service_account.

Unfortunately, I could'nt find in the documentation the page specifying the necessary configuration for the Keycloak identity provider.

The following documentation provides information on the different configuration of the main identity providers, but not Keycloak.\
https://www.pomerium.io/docs/identity-providers/

What did you expect to happen?

I hope there is a preciseidp_service_account configuration (base64 encoded json ?) for Keycloak that allow to use allowed_groups policy with Keycloak and Pomerieum 0.10.0 and above.

How'd it happen?

Case one (OK)

1. Configure Pomerium policy with a allowed_users
2. Went to whoami.example.com
3. Logged in to the identity provider, received pomerium session
4. Redirected to whoami.example.com -> access allowed

Case two (NOK)

1. Configure Pomerium policy with a allowed_groups
2. Configure IDP_SERVICE_ACCOUNT with clientid/clientsecret JSON Base64 encoded ( my random choice)
3. Went to whoami.example.com
4. Logged in to the identity provider, received pomerium session
5. Redirected to whoami.example.com -> access denied

What's your environment like?

• Pomerium 0.10.2
• Traefik 2.3.2
• Linux/Docker/Docker-compose
• Keycloak 11.0.3 (through OIDC)

What's your config.yaml?

policy:
  - from: https://whoami.example.com
    to: http://whoami
    timeout: 30s
    allowed_groups:
      - /admin
      - /users
    allow_websockets: true
    tls_skip_verify: true
    preserve_host_header: true
    cors_allow_preflight: true

What's your docker-compose.yaml?

(...)
  whoami:
    container_name: whoami
    image: containous/whoami
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoami.rule=Host(`whoami.example.com`)
      - traefik.http.services.whoami.loadbalancer.server.port=80
      - traefik.http.routers.whoami.entrypoints=websecure
      - traefik.http.routers.whoami.tls.certresolver=letsencrypt
      - traefik.http.routers.whoami.tls=true
      - traefik.http.routers.whoami.middlewares=whoami-auth
      - traefik.http.middlewares.whoami-auth.forwardauth.address=http://whoami-auth:443/?uri=https://whoami.example.com
      - traefik.http.middlewares.whoami-auth.forwardauth.authResponseHeaders=X-Pomerium-Jwt-Assertion, X-Pomerium-Claim-User, X-Pomerium-Claim-Email, X-Pomerium-Claim-Groups      
      - traefik.http.middlewares.whoami-auth.forwardauth.trustForwardHeader=true
    networks:
      - traefik_proxy 

  whoami:
    image: nimmis/apache-php5
    container_name: whoami
    networks:
      - traefik_proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoami.rule=Host(`whoami.example.com`)
      - traefik.http.services.whoami.loadbalancer.server.port=80
      - traefik.http.routers.whoami.entrypoints=websecure
      - traefik.http.routers.whoami.tls.certresolver=letsencrypt
      - traefik.http.routers.whoami.tls=true
      - traefik.http.routers.whoami.middlewares=whoami-auth
      - traefik.http.middlewares.whoami-auth.forwardauth.address=http://whoami-auth:443/?uri=https://whoami.example.com
      - traefik.http.middlewares.whoami-auth.forwardauth.authResponseHeaders=X-Pomerium-Jwt-Assertion, X-Pomerium-Claim-User, X-Pomerium-Claim-Email, X-Pomerium-Claim-Groups      
      - traefik.http.middlewares.whoami-auth.forwardauth.trustForwardHeader=true      
    volumes:
      - ./config/whoami/apache.conf:/etc/apache2/sites-enabled/000-default.conf:ro
      - ./www/whoami:/var/www/html
      - ./log/whoami/log:/var/log/apache2

  whoami-auth:
    image: pomerium/pomerium:v0.10.2
    container_name: whoami-auth
    restart: always
    environment:
      #- POMERIUM_DEBUG=true
      - SERVICES=all
      - FORWARD_AUTH_URL=http://whoami-auth:443/
      - INSECURE_SERVER=TRUE
      - IDP_SERVICE_ACCOUNT=ewogICJjbGllbnRfaWQiOiAibmFtZWFwcCIsCiAgImNsaWVudF9zZWNyZXQiOiAieHh4eHh4eHgteHh4eC14eHh4LXh4eHgteHh4eHh4eHh4eHh4Igp9
      - IDP_PROVIDER=oidc
      - IDP_CLIENT_ID=whoami
      - IDP_CLIENT_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      - IDP_PROVIDER_URL=https://keycloak.example.com/auth/realms/MyRealm
      - SHARED_SECRET=<shared_secret>
      - COOKIE_SECRET=<cookie_secret>
      - VIRTUAL_PROTO=http
      - VIRTUAL_HOST=whoami-auth.example.com
      - VIRTUAL_PORT=443
      - JWT_CLAIMS_HEADERS=email,user,groups
      - AUTHENTICATE_SERVICE_URL=https://whoami-auth.example.com
    volumes:
      - ./config/pomerium/config.yaml:/pomerium/config.yaml:ro
    labels:
      - traefik.enable=true
      - traefik.http.services.whoami-auth.loadbalancer.server.port=443
      - traefik.http.routers.whoami-auth.rule=Host(`whoami-auth.example.com`)
      - traefik.http.routers.whoami-auth.entrypoints=websecure
      - traefik.http.routers.whoami-auth.tls.certresolver=letsencrypt
      - traefik.http.routers.whoami-auth.tls=true
    networks:
      - traefik_proxy
(...)

What did you see in the logs?

In case with Pomerium policy with a allowed_groups and IDPSERVICEACCOUNT

7:12PM INF get id=xxxxxxx-6eed-40d3-aaf7-xxxxxxx service=databroker type=type.googleapis.com/session.Session
7:12PM WRN failed to get session from databroker error="rpc error: code = NotFound desc = record not found"
7:12PM WRN clearing session due to force sync failed error="session not found"
7:12PM INF authorize check allow=false check-request-id=xxxxxxx-9e5b-41f9-800f-xxxxxxx groups=[] host=whoami.example.com method=GET path= query= request-id=xxxxxxx-1139-47a7-a2b6-xxxxxxx service=authorize status=401 user=
7:12PM INF get id=xxxxxxx-6eed-40d3-aaf7-xxxxxxx service=databroker type=type.googleapis.com/session.Session
7:12PM INF authenticate: session not found in databroker error="error getting session from databroker: rpc error: code = NotFound desc = record not found" X-Forwarded-For=["172.19.0.1,172.19.0.5"] X-Forwarded-Host=["whoami-auth.example.com"] X-Forwarded-Port=["443"] X-Forwarded-Proto=["http"] X-Forwarded-Server=["15f14d39e987"] X-Real-Ip=["172.19.0.1"] id=xxxxxxx-6eed-40d3-aaf7-xxxxxxx ip=127.0.0.1 request-id=xxxxxxx-faa5-4b74-859f-xxxxxxx user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF get id=oidc/xxxxxxx-8f65-4833-9118-xxxxxxx service=databroker type=type.googleapis.com/user.User
7:12PM INF set id=oidc/xxxxxxx-8f65-4833-9118-xxxxxxx service=databroker type=type.googleapis.com/user.User
7:12PM INF refreshing user service=identity_manager user_id=oidc/xxxxxxx-8f65-4833-9118-xxxxxxx
7:12PM INF set id=xxxxxxx-336d-4aba-8801-xxxxxxx service=databroker type=type.googleapis.com/session.Session
7:12PM INF get id=xxxxxxx-336d-4aba-8801-xxxxxxx service=databroker type=type.googleapis.com/session.Session
7:12PM ERR proxy: could not locate session from context error="internal/sessions: session is not found"
7:12PM INF authorize check allow=false check-request-id=xxxxxxx-6a97-4618-9d84-xxxxxxx groups=[] host=whoami.example.com method=GET path= query= request-id=xxxxxxx-50bf-4f3e-b2a5-xxxxxxx service=authorize status=403 [email protected]
7:12PM INF httputil: ErrorResponse error="Forbidden: access denied" X-Forwarded-For=["172.19.0.1,172.19.0.5"] X-Forwarded-Host=["whoami.example.com"] X-Forwarded-Method=["GET"] X-Forwarded-Port=["443"] X-Forwarded-Proto=["http"] X-Forwarded-Server=["15f14d39e987"] X-Forwarded-Uri=["/"] X-Real-Ip=["172.19.0.1"] email= groups= ip=127.0.0.1 request-id=xxxxxxx-6a97-4618-9d84-xxxxxxx user= user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth:443 duration=4.133003 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/?uri=https://whoami.example.com referer= request-id=xxxxxxx-9e5b-41f9-800f-xxxxxxx response-code=302 response-code-details=via_upstream service=envoy size=349 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth.example.com duration=1.631597 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/.pomerium/sign_in?pomerium_callback_uri=https%3A%2F%2Fwhoami.example.com&pomerium_expiry=1606760249&pomerium_forward_auth=whoami-auth&pomerium_issued=1606759949&pomerium_redirect_uri=https%3A%2F%2Fwhoami.example.com&pomerium_signature=XXXXXXXXXXXXXXXXXXX%3D referer= request-id=xxxxxxx-faa5-4b74-859f-xxxxxxx response-code=302 response-code-details=via_upstream service=envoy size=885 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth.example.com duration=20.254507 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/oauth2/callback?state=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%3D&session_state=xxxxxxx-4e9b-48df-941d-xxxxxxx&code=0a458e59-6634-443f-9ddd-276353eb5047.xxxxxxx-4e9b-48df-941d-xxxxxxx.f085957f-4784-43f6-8c38-2e1b12282152 referer= request-id=xxxxxxx-13d0-4c21-bbd8-xxxxxxx response-code=302 response-code-details=via_upstream service=envoy size=349 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth.example.com duration=1.616806 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/.pomerium/sign_in?pomerium_callback_uri=https%3A%2F%2Fwhoami.example.com&pomerium_expiry=1606760249&pomerium_forward_auth=whoami-auth&pomerium_issued=1606759949&pomerium_redirect_uri=https%3A%2F%2Fwhoami.example.com&pomerium_signature=XXXXXXXXXXXXXX%3D referer= request-id=xxxxxxx-bb1f-46a7-b759-xxxxxxx response-code=302 response-code-details=via_upstream service=envoy size=998 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth:443 duration=0.52803 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/?uri=https://whoami.example.com referer= request-id=0ede1215-76e9-4afe-94b3-c9a4a8546804 response-code=302 response-code-details=via_upstream service=envoy size=0 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"
7:12PM INF http-request authority=whoami-auth:443 duration=14.320748 forwarded-for=172.19.0.1,172.19.0.5 method=GET path=/?uri=https://whoami.example.com referer= request-id=xxxxxxx-6a97-4618-9d84-xxxxxxx response-code=403 response-code-details=via_upstream service=envoy size=63451 upstream-cluster=pomerium-control-plane-http user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"

Thanks for your help,

[travisgroth]

Hi @amiral-fu,

We currently integrate with Keycloak via generic OIDC, so allowed_groups is not possible. However, in the newest release of Pomerium, we've added the ability to set policy on arbitrary claims from your identity provider. I believe you can use this to achieve the allowed_groups functionality as long as Keycloak can return your group membership in a claim.

Here's the docs: https://www.pomerium.com/reference/#allowed-idp-claims

[eh-am]

Hi, that works great!

The only problem I see, however, is that impersonation does not seem to work, since we aren't using "real" groups:
https://github.com/pomerium/pomerium/blob/091b71f12e0c27ff72442b87733f90e3625b8fdb/internal/sessions/state.go#L105-L112

But I guess that's more of an impersonation issue than the actual idp_service_account implementation? I can create a different issue if needed.

[amiral-fu]

Thanks for your help @travisgroth

Indeed, it works very well!

[desimone]

@eh-am / @amiral-fu -- Glad it is working. By the way, several folks have asked us for a guide for a generic oidc provider (#307 ). If you'd be willing to contribute one for Keycloak, I'm sure many in the community would find it useful. I've been unsuccessful in setting it up myself.

[desimone]

@eh-am

But I guess that's more of an impersonation issue than the actual idp_service_account implementation? I can create a different issue if needed.

That's right. We are looking at ways of refactoring to support this, but it would be helpful to create an issue to track it.

[dharmendrakariya]

@eh-am / @amiral-fu -- Glad it is working. By the way, several folks have asked us for a guide for a generic oidc provider (#307 ). If you'd be willing to contribute one for Keycloak, I'm sure many in the community would find it useful. I've been unsuccessful in setting it up myself.

I have tried, please let me know #2018