Ingress-nginx: Empty $service_name and $service_port variables

[praseodym]

NGINX Ingress controller version: 0.30.0

Kubernetes version (use kubectl version): v1.17.2 in kind

What happened:

I am trying to get Linkerd to work with ingress-nginx. Linkerd provides instructions for this, the important annotation to add to the Ingress object being:

    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
      grpc_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;

This didn't work as expected, and I traced this down to the $service_name and $service_port variables not being filled as expected:

> kubectl ingress-nginx conf | grep -C4 service_name
--
        location / {

            set $namespace      "emojivoto";
            set $ingress_name   "emojivoto";
            set $service_name   "";
            set $service_port   "";
            set $location_path  "/";

            rewrite_by_lua_block {
--

What you expected to happen:

The $service_name variable being set to web-svc and the $service_port variable being set to 80.

How to reproduce it:

Install the emojivoto sample application and apply the following Ingress object:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: emojivoto
  namespace: emojivoto
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: web-svc
          servicePort: 80
        path: /

Then use the ingress-nginx kubectl plugin to check the nginx config (kubectl ingress-nginx conf | grep -C4 service_name).

Anything else we need to know:

/kind bug

[aledbf]

servicePort: 8080

@praseodym please check the ingress controller pod log.
This should not be 80? (from the web.yaml emojivoto yaml)

[praseodym]

@praseodym please check the ingress controller pod log.
This should not be 80? (from the web.yaml emojivoto yaml)

You're right, it should be servicePort: 80. Unfortunately that doesn't make a difference: $service_name and $service_port are still empty.

[aledbf]

@praseodym please post the pod logs. If that is empty, something should appear there.

[praseodym]

Please see the ingress-nginx pod logs with --v=2.

[aledbf]

@praseodym are you using the sample ingress definition? (or did you create an ingress without host field?)

[praseodym]

I created an ingress object without a host field (the definition in the first post is the full object I applied).

[praseodym]

Adding a host field to the ingress fixes the problem:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: emojivoto
  namespace: emojivoto
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
spec:
  rules:
  - host: localhost
    http:
      paths:
      - backend:
          serviceName: web-svc
          servicePort: 80
        path: /

gives:

        location / {

            set $namespace      "emojivoto";
            set $ingress_name   "emojivoto";
            set $service_name   "web-svc";
            set $service_port   "80";
            set $location_path  "/";

            rewrite_by_lua_block {

While it isn't too common to not have a host field defined, it'd still expect that to work or log a warning message if it is not a supported configuration.

[aledbf]

While it isn't too common to not have a host field defined, it'd still expect that to work or log a warning message if it is not a supported configuration.

This is a bug, I am just trying to narrow where is located.

[aledbf]

@praseodym please test quay.io/kubernetes-ingress-controller/nginx-ingress-controller-amd64:dev

[praseodym]

@praseodym please test quay.io/kubernetes-ingress-controller/nginx-ingress-controller-amd64:dev

This fixes the problem, thank you!

[wmorgan]

Thanks from the Linkerd team, @aledbf!

[zaharidichev]

@aledbf We are still running into this problem. Namely, I have the following resource:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
      grpc_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local:$service_port;
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: backend-one-svc-west
          servicePort: 8888
        path: /test

I find the l5d-dst-override header to be set to ".default.svc.cluster.local:" We are using 0.33.0

[aledbf]

@zaharidichev thank you for the report. I will open a PR to fix this issue.

That said, what's the use case for an ingress without a host field? I asked because, without that field, paths cannot be repeated, and accessing an ingress controller by an IP address is not the "correct" approach.

[gauravkohli]

@aledbf

We are having a similar problem even if we have specified the host variable like below. So while using below ingress we get "set $service_name ""; " in the nginx.conf ( so a empty service_anme) and if I remove annotation "nginx.ingress.kubernetes.io/rewrite-target: /" then it works fine and I get "set $service_name "API";"

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: "ingress-api"
  namespace: test
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header l5d-dst-override $service_name.$namespace.svc.cluster.local;
spec:
  tls:
  - hosts:
    - test.io
    secretName: tls-secret
  rules:
    - host: test.io
      http:
        paths:
        - path: /api/v1/
          backend:
            serviceName: api
            servicePort: 80

Is this something you could have a look at?

[mbelang]

I stumble upon this while debugging the linkerd annotations for ingress. It seems not that service_port variable is always 80 by default.

I'm using controller version v0.34.1