Kubernetes-ingress: document auth_request example

[nabheet]

Is your feature request related to a problem? Please describe.
I cannot find an example of using auth_request module http://nginx.org/en/docs/http/ngx_http_auth_request_module.html with this ingress controller. We have security set up similar to https://github.com/nginxinc/nginx-ldap-auth. We also have multiple locations exposed in the same server. And multiple domains/servers on the same port (443) with TLS.

Describe the solution you'd like
I would like to understand how to combine master/minion set up with the auth_request module.

Describe alternatives you've considered
Cannot find any documentation for the ingress controller. Currently we are using nginx docker images directly with the nginx.conf mounted into the container.

[Rulox]

Hi @nabheet !

There are a lot of directives in NGINX configuration and it is impossible for us to provide examples in the documentation for all of them, I hope you can understand.

However, your use case is valid. In your case, would location/server snippets annotations work for your Ingress (master and minion) Resources? With these snippets you can add any type of configuration to your servers (like a new /auth location) and your locations (like the auth_request directive you need).

For instance, in an Ingress Resource like this one (note annotations):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.org/location-snippets: |
      auth_request /auth;
    nginx.org/server-snippets: |
      location = /auth {
        return 200;
      }
  name: cafe-ingress
  namespace: default
spec:
  rules:
  - host: cafe.example.com
    http:
      paths:
      - backend:
          serviceName: tea-svc
          servicePort: 80
        path: /tea
      - backend:
          serviceName: coffee-svc
          servicePort: 80
        path: /coffee
  tls:
  - hosts:
    - cafe.example.com
    secretName: cafe-secret

The NGINX config in the Ingress Controller for this particular server cafe.example.com would be (I removed useless generated config for simplicity):

server {
    listen 80;
    listen 443 ssl;
        # generated config here...
    server_tokens "on";
    server_name cafe.example.com;
    status_zone cafe.example.com;

    location = /auth {
      return 200;
    }

    location /tea {
        proxy_http_version 1.1;
        auth_request /auth;
                 # generated config here...
        proxy_pass http://default-cafe-ingress-cafe.example.com-tea-svc-80;
    }
    location /coffee {
        proxy_http_version 1.1;
        auth_request /auth;
                 # generated config here...
        proxy_pass http://default-cafe-ingress-cafe.example.com-coffee-svc-80;
    }
}

Note that snippets must be valid NGINX config. If the config is not valid, NGINX will fail to reload and you will see an error in the Ingress Controller pod logs.

Let me know if this would work for you, thanks!

[nabheet]

WOW! Thank you! I think this is a really good starting point for me. Our configuration is fairly involved for me. It seems like the ingress controller has pretty much most if not all of the nginx configuration.

Is there a way to define proxy_cache_path without overriding the template?

[nabheet]

just found http-snippets - going to give them a shot

[nabheet]

I was able to complete this task as we discussed using the mergeable ingress type. But I just learned about the virtualserver and virtual server route implementation which seem to be a better way of handling our use cases.

I don't have any golang experience so I am having a hard time finding yaml equivalents to https://github.com/nginxinc/kubernetes-ingress/blob/c2bccae7a052538d2c7fc96a6f891ad45e2d9101/internal/configs/version2/templates_test.go#L142.

I can't figure out how to pass this in from the K8S manifest into the virtualserverroute object so the template can access it.

This is exactly what I need -
https://github.com/nginxinc/kubernetes-ingress/blob/master/internal/configs/version2/nginx.virtualserver.tmpl#L94

But I don't know how to define the K8S yaml to feed this template.

[Rulox]

Hi @nabheet

VirtualServers and VirtualServerRoutes are being developed right now, we try to prioritise most common use cases for our roadmap.

In this particular case, internal can't be set from the configuration YAML I am afraid. This is a workaround done for other reasons: for the ErrorPage feature of the VS/VSR and that directive should be transparent to the end user (aka you).

Please keep an eye on the repository, we are updating the VS/VSR with new features pretty much every week. In the meantime, you can use the approach you are using with mergeable ingresses.

Alternatively, you can update the template manually and build your own version of the Ingress Controller, but this requires knowledge in Go and templating, and it will cause you to have to maintain your own version.

My recommendation is that you stick to mergeable ingresses for now. We are planning to add a similar approach as the snippets for VS and VSR, that will fit your case I think.

Let me know if this makes sense!

[nabheet]

NOOOOOO!!!! BUT THEY ARE SOOOO COOOLLL!!!! I WANT THEM NOW!!!!!

Thank you for your prompt response. I will proceed the mergeable templates. But in case you need prioritization ideas, here is a simplified sample from our current nginx.conf file:

upstream upstream_api_svc {
  server upstream-api-svc-XXXX:3000;
  server upstream-api-svc-YYYY:3000;
  server upstream-api-svc-ZZZZ:3000;
}

location /auth {
  internal;

  proxy_pass http://upstream_api_svc/auth;

  proxy_pass_request_body off;
  proxy_set_header Authorization "sso-jwt $cookie_auth_realm";
  proxy_set_header X-Authorization-Type "app_admins";
  proxy_set_header Content-Length "";
  proxy_cache auth_cache;
  proxy_ignore_headers Cache-Control;
  proxy_cache_valid 200 10m;
  proxy_cache_valid 401 1m;
  proxy_cache_lock on;
  proxy_cache_key "$cookie_auth_dc1";
}

location / {
  auth_request /auth;

  auth_request_set $header_1 $upstream_http_x_api_header_1;
  auth_request_set $header_2 $upstream_http_x_api_header_2;
  auth_request_set $api_role $upstream_http_x_api_role;
  auth_request_set $api_user $upstream_http_x_api_user;

  proxy_set_header X-Header-1 $header_1;
  proxy_set_header X-Header-2 $header_2;
  proxy_set_header X-Api-Role $api_role;
  proxy_set_header X-User $api_user;
  proxy_set_header X-Request-Id $request_id;
  proxy_read_timeout 600;

  proxy_pass http://upstream_api_svc/;
}

so basically some way to define and internal location in virtualserverroute.subroute and a way to set HTTP Headers. I guess snippets would allow us to do that but it would be so much cooler if the manifest yaml exposed those fields.

Anyways, thank you for all you assistance! I think this issue has been fairly documented here. Feel free to close it or I can close it.

[Rulox]

Thanks! Yes snippets in routes and subroutes will work for you if using VS/VSR. Thanks for the ideas. Closing this 👍