Ingress-nginx: Preserve client IP address

[mihaimyh]

I am using iptables NAT rules to forward incoming traffic to the ingress-nginx service on my bare-metal Kubernetes cluster but I am unable to forward client's IP address to.

The ingress controller has an external IP allocated by MetalLb, I've tried different solutions I've came across, with no real success:

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
    service.beta.kubernetes.io/do-loadbalancer-hostname: "do-k8s.example.com"
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx"},"name":"nginx-configuration","namespace":"ingress-nginx"}}
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  name: nginx-configuration
  namespace: ingress-nginx
data:
  set-real-ip-from: "0.0.0.0/0"
  use-forwarded-headers: "true"
  compute-full-forwarded-for: "true"
  force-ssl-redirect: "true"
  proxy-buffer-size: "128k"
  proxy-buffers: "4 256k"
  proxy-busy-buffers-size: "256k"
  client-header-buffer-size: "64k"
  http2-max-field-size: "16k"
  http2-max-header-size: "128k"
  large-client-header-buffers: "8 64k"
  fastcgi-buffers: "16 16k"
  fastcgi-buffer-size: "32k"
#  use-proxy-protocol: "true"



md5-c2e923e049cd6e3982df4a7d92e20e76



apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  namespace: example
  name: example-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/session-cookie-hash: sha1
    nginx.ingress.kubernetes.io/session-cookie-name: REALTIMESERVERID
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-body-size: 50m
spec:
  tls:
  - hosts:
    - example.com
    - '*.example.com'
    secretName: wildcard-example-com
  rules:
  - host: docker-hub.example.com
    http:
      paths:
      - backend:
          serviceName: hooktrigger-api
          servicePort: 443

Anyone know if this is possible on bare-metal with NAT & MetalLb?

[aledbf]

Closing. Please use the #ingress-nginx Kubernetes slack channel or #metallb to ask this kind of question. Thanks

[aledbf]

As a side note:

service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
service.beta.kubernetes.io/do-loadbalancer-hostname: "do-k8s.example.com"

These annotations are valid only in Digital Ocean, not bare-metal

set-real-ip-from: "0.0.0.0/0"
use-forwarded-headers: "true"
compute-full-forwarded-for: "true"

This makes sense only if NGINX receives the real source IP address in a header, something is not present with just NAT/Metallb

[mihaimyh]

@aledbf Thank you for the notes, I will try to find answers on Slack, hopefully someone will be kind enough to help.
I thought opening an issue here will help others in the same situation.