Ingress-nginx: Updating crl doesn't trigger reload of nginx

[chrko]

NGINX Ingress controller version: 0.26.2

Kubernetes version (use kubectl version): v1.17.0 (server & client)

Environment: minikube v1.6.2 on ArchLinux with Virtualbox

What happened:

If the certificate revocation list (crl) within a client-ca secret is updated, the nginx isn't reloaded to reread the crl. So you cannot update e.g. an outdated crl.

What you expected to happen:

Updating the crl within a client-ca secret triggers a reload.

How to reproduce it:

excerpt of the deployment:

          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/sec-ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
            - --enable-ssl-chain-completion=false
            - --enable-dynamic-certificates=false
            - --http-port=8080
            - --https-port=8443
            - --ingress-class=sec-ingress-nginx

configuration via kustomize:

configMapGenerator:
  - name: nginx-configuration
    literals:
      - error-log-level=info
      - server-tokens=false
      - ssl-dh-param=sec-ingress-local/nginx-dhparams
      - worker-processes=3
      - hsts=false

Configure a ingress with client certificate authentication:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-secret: sec-ingress-local/client-ca
    nginx.ingress.kubernetes.io/auth-tls-verify-client: optional
    nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
spec:
  rules:
  - host: sec.k8s.local
    http:
      paths:
      - backend:
          serviceName: test-backend
          servicePort: 8080
        path: /test
  tls:
  - hosts:
    - sec.k8s.local
    secretName: snakeoil-tls

Create client-ca secret via kustomize:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
secretGenerator:
  - name: client-ca
    files:
      - ca.crt
      - ca.crl
generatorOptions:
  disableNameSuffixHash: true

E.g. put into ca.crl an expired crl or not all crl of the chain and see:

2020/01/09 08:59:07 [info] 473#473: *25619 client SSL certificate verify error: (3:unable to get certificate CRL) while reading client request headers, client: 172.17.0.1, server: _, request: "GET /test HTTP/1.1", host: "192.168.99.111:30443"

update the client-ca secret to contain valid and complete crls -> request will still fail.
kubectl exec -ti into the pod, get the nginx master pid and send HUP with e.g. kill -HUP 31. After this manual reload the requests succeed.

Anything else we need to know:

/kind bug

[zeeZ]

I have noticed this as well. Updated a CRL with new revocations. The change wasn't applied until a forced reload was triggered (via basic auth secret change).

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[chrko]

/remove-lifecycle stale

This issue still persits… updating the CRL doesn't trigger the reload of the nginx.

[kundan2707]

/ assign

[ntavares]

Just to record that this problem is still present on 0.32.0.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[chrko]

/remove-lifecycle stale

[lightsing]

Same issue.