Cert-manager: Can I have multiple Ingress resources?

[guanzo]

I had cert-manager working with a single Ingress resource. Now I have 2 Ingress resources, and I only have TLS working on the first Ingress resource that gets created. That is, the generated Certificate only contains references to the first Ingress and not the second. I tried deleting all Ingresses and Certificates, but it didn't help.

Ingress 1 (TLS works)

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: 16384m
spec:
  tls:
  - hosts:
    - storage.example.com
    secretName: letsencrypt-prod
  rules:
  - host: storage.example.com
    http:
      paths:
      - backend:
          serviceName: storage
          servicePort: 80

Ingress 2 (TLS doesn't work)

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-nginx-2
  annotations:
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  tls:
  - hosts:
    -example.com
    secretName: letsencrypt-prod
  rules:
  - host:example.com
    http:
      paths:
      - backend:
          serviceName: example
          servicePort: 80

The yaml is the same, except for one annotation:
nginx.ingress.kubernetes.io/proxy-body-size: 16384m

[tikael1011]

Would you please tell us why do you need two ingress resources? From your yaml it seems you are using exactly the same cluster issuer and you can put two domains in one ingress like

spec:
  tls:
  - hosts:
    - storage.example.com
    - example.com
    secretName: letsencrypt-prod
  rules:
  - host: storage.example.com
    http:
      paths:
      - backend:
          serviceName: storage
          servicePort: 80
  - host: example.com
    http:
      paths:
      - backend:
          serviceName: storage
          servicePort: 80

One possible answer I can think of is that you are using dns01 challenge and the hostedzoneid (route53) of the two domains are different?

[guanzo]

I mentioned it at the bottom of my first post, the second Ingress has an extra annotation. AFAIK annotations are applied to all rules within the Ingress, which is not okay for me. I only want a large proxy-body-size for storage.example.com, because it expects large file uploads. The other subdomains should use the default proxy-body-size. From googling on how to customize nginx for a specific Ingress rule (using nginx-ingress), the common answer was to define another Ingress resource, hence why I have two now.

For now I've "solved" this issue by adding storage.example.com to spec.tls.hosts in my main Ingress resource that gets created first. This gives me working TLS. But I'm also keeping the other Ingress that has the storage.example.com rule, so I can use the proxy-body-size annotation.

The root problem was, how do I customize nginx for a specific rule in a single Ingress resource? That problem led me to this problem.

[munnerz]

If you want to share a single TLS certificate between two ingress resources, only one should contain annotations to trigger cert-manager to create a Certificate resource.

I tend to manually create my Certificate resource in this case, instead of utilising annotations on the Ingress.

The root problem was, how do I customize nginx for a specific rule in a single Ingress resource? That problem led me to this problem.

What you're doing with two ingresses makes perfect sense - it's just that you don't need the annotation twice, as the annotation is ultimately what causes cert-manager to create a Certificate resource that matches your ingress. Adding the spec.tls stanza to your Ingress is what causes the ingress controller to secure your ingress. There is a subtle difference between the two 😄

I'm going to close this, as it does not seem directly related to cert-manager and you've managed to find a solution 😄

[stevo-knievo]

Thanks @munnerz - You saved my day! And thanks to @guanzo for raising this question!

[maxwedwards]

I'm also interested in multiple ingress resources as it's simpler to manage in our infra source code. I'm interested why this needs to be one cert? Why doesn't cert-manager and the ingress just create a cert for each subdomain? Could be done with SNI couldn't it?

[maxwedwards]

I've tried completely separate hostnames and two ingress resources and cert-manager seems to happily go off and get a cert for each using one external loadbalancer IP.

[davidshare]

I have multiple ingresses and I share the same cert for them specifying the annotation on only one of the ingresses. However, only the ingress that has the cert specified in the annotation works.