Kubernetes-ingress: VirtualServer and Ingress

[RazaGR]

I have few questions regarding Ingress and VirtualServer ( which I just found out )

1. I am using Ingress resource's to mange multiple domains, for each domain I have one ingress resource associated with it, my first question, is it possible to use different tls for each host? so this way I do not have to create multiple Ingress resources , example how I want to handle multiple certificates by providing secretName:

spec:
tls:

• hosts:
  
  
  
  • domain-one.com
    
    secretName: cert-domain-one
  
  
  • domai-two.com
    
    secretName: cert-domain-two
    
    rules:
  
  
• host:domain-one.com
  ....
• host:domain-two.com
  ....

while finding solution to above problem, I read about VirtualServer resource, I read in documentation that I could use different certificate for each VirtualServer, which solves my above issue by replacing my Ingress resource with VirtualServer, But I have two issues:

2. Does VirtualServer support annotations? I couldn't find any info in documentation
3. Does VirtualServer support custom annotations? for example, in my below snippet I have " custom.nginx.org/my-custom-annotation: "on"" , which I use in my custom template.
4. Does VirtualServer support custom template ? like in Ingress resource
5. Does VirtualServer support cert-manage to handle certificates? I have an annotation in my Ingress resource. "cert-manager.io/cluster-issuer: "letsencrypt-issuer"" which handles all certificate registration/renewal.

Here is an example of Ingress with custom annotation I am using with Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  namespace: nginx-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-issuer"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    custom.nginx.org/my-custom-annotation: "on"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.org/proxy-connect-timeout: "30s"
    nginx.org/proxy-read-timeout: "20s"
    nginx.org/client-max-body-size: "4m"
spec:
  tls:
  - hosts:
    - domain-one
    secretName: domain-one-cert
  rules:
  - host: domain-one.com
    http:
      paths:
      - path: /
        backend:
          serviceName: domain-one-service
          servicePort: 80

[RazaGR]

ok, felt a bit silly for question 1, I found that I could use multiple tls by using different hosts:
spec:

  tls:
  - hosts:
    - domain-one.com
      secretName: cert-domain-one
  - hosts:
    - domai-two.com
       secretName: cert-domain-two
  rules:
  - host:domain-one.com
    ....
  - host:domain-two.com
    ....

[Dean-Coakley]

Hi @RazaGR

Those are some valid questions.

However, I noticed you have the annotations nginx.ingress.kubernetes.io/rewrite-target and nginx.ingress.kubernetes.io/ssl-redirect which are annotations that are only supported by the Kubernetes community ingress controller. That is a different implementation of an ingress controller which does not support the Virtualserver resource.

Can you check the version of your ingress controller?

[RazaGR]

Hello @Dean-Coakley

Thanks for replying, actually I was testing something with cert-manager and left those annotations, I have followed this tutorial to setup https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/
and using nginx/nginx-ingress:1.6.3 image

[Dean-Coakley]

Ah, thanks for validating that.

Unfortunately these customisation options are not developed yet. They won't be considered stable for quite some time.

I'd recommend sticking with Ingress resources for the time being if that works for you?

[RazaGR]

ok then I will stick with Ingress resource. But with ingress I encountered one issue
If I create an ingress resource with :

  tls:
  - hosts:
    - domain-one.com
    - www.domain-one.com
      secretName: cert-domain-one
  - hosts:
    - domai-two.com
    - www.domain-two.com
       secretName: cert-domain-two
  rules:
  - host:domain-one.com
    ....
  - host:www.domain-two.com
    ....
  - host:domain-one.com
    ....
  - host:www.domain-two.com
    ....

it works perfectly, But if I use it with wildcard:

  tls:
  - hosts:
    - domain-one.com
    - "*.domain-one.com"
      secretName: cert-domain-one
  - hosts:
    - domai-two.com
    - "*.domain-two.com"
       secretName: cert-domain-two
  rules:
  - host:domain-one.com
    ....
  - host:www.domain-two.com
    ....
  - host:domain-one.com
    ....
  - host:www.domain-two.com
    ....

it creates 2 server blocks one with www, one with non-www, It doesn't add SSL attributes in www server block in config file (/etc/nginx/conf.d/nginx-ingress-my-ingress.conf), for 'www' hosts ( 'host:www.domain-one.com' / 'host:www.domain-two.com')

these only exist for non-www host in server block:

        listen 443 ssl;
        ssl_certificate /etc/nginx/secrets/nginx-ingress-cert-domain-one;
        ssl_certificate_key /etc/nginx/secrets/nginx-ingress-cert-domain-one;

so I am thinking just to remove {{if $server.SSL}} block in ingress-template template to cover this issue:

    {{if $server.SSL}} // I REMOVE THIS
    {{- range $port := $server.SSLPorts}}
    listen {{$port}} ssl{{if $server.HTTP2}} http2{{end}}{{if $server.ProxyProtocol}} proxy_protocol{{end}};
    {{- end}}
    ssl_certificate {{$server.SSLCertificate}};
    ssl_certificate_key {{$server.SSLCertificateKey}};
    {{if $server.SSLCiphers}}
    ssl_ciphers {{$server.SSLCiphers}};
    {{end}}
    {{end}} // I REMOVE THIS

is that right or some other way exist?

OR is there a way I do not have to provide www and non-www in rules, so I could just use one rule per domain, which defaults to www, I have to provide www and non-www, otherwise I get 404:

  rules:
  - host:www.domain-one.com
    ....
  - host:www.domain-two.com
    ....

[Dean-Coakley]

The host referenced in the TLS section must exactly match the hosts in the rules section. Otherwise, the TLS configuration won’t be generated.

To fix this, something like the following should work:

  tls:
  - hosts:
    - www.domain-one.com
    - "*.domain-one.com"
      secretName: cert-domain-one
  - hosts:
    - www.domain-two.com
    - "*.domain-two.com"
       secretName: cert-domain-two
  rules:
  - host:*.domain-one.com
    ....
  - host:www.domain-two.com
    ....
  - host:*.domain-one.com
    ....
  - host:www.domain-two.com
    ....

[RazaGR]

@Dean-Coakley that fixed it, Thank you so much