Kubernetes-ingress: Port forwarding for mail

[RazaGR]

I am trying to setup postfix while using kubernetess-ingress with a load balancer, I can't figure out how can I forward smtp ports i.e 25 from load balancer to my postfix service, I am using ingress-template: in my configMap and have ingress setup for domain names like this:

spec:
  tls:
  - hosts:
    - mydomain.com
    - "*.mydomain.com"
    secretName: certificate-tls
  rules:
  - host: mydomain.com
    http:
      paths:
      - path: /
        backend:
          serviceName: appsvc1
          servicePort: 9000

I couldn't find any information about it, any help please?

$ kubectl get services --namespace=nginx-ingress
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                                                               AGE
nginx-ingress   LoadBalancer   10.245.163.159   MY-EXTERNAL-IP   80:30589/TCP,443:31197/TCP,25:32309/TCP,465:32029/TCP,587:32147/TCP   26h
postfixsvc1     ClusterIP      10.245.72.240    <none>           25/TCP,465/TCP,587/TCP                                                8m17s

my load balancer:

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
  - port: 25
    targetPort: 25
    name: smtpd
    protocol: TCP
  - port: 465
    targetPort: 465
    name: smtpssl
    protocol: TCP
  - port: 587
    targetPort: 587
    name: smtp
    protocol: TCP
  selector:
    app: nginx-ingress

postfix service and deployment:

apiVersion: v1
kind: Service
metadata:
  name: postfixsvc1
  namespace: nginx-ingress
spec:
  ports:
  - port: 25
    targetPort: 25
    name: smtpd
    protocol: TCP
  - port: 465
    targetPort: 465
    name: smtpssl
    protocol: TCP
  - port: 587
    targetPort: 587
    name: smtp
    protocol: TCP
  selector:
    app: postfix1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postfix1
  namespace: nginx-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postfix1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: postfix1
    spec:
      containers:
      - image: myimage-of-/postfix
        name: postfix1
        ports:
         - containerPort: 25
           name: smtpd
           protocol: TCP
         - containerPort: 465
           name: smtpssl
           protocol: TCP
         - containerPort: 587
           name: smtp
           protocol: TCP
        volumeMounts:
          - name: certificate-config-volume
            mountPath: /secret
            readOnly: true
      volumes:
        - name: certificate-config-volume
          secret:
            secretName: certificate-tls

[RazaGR]

while exploring docs I noticed a new "TransportServer" kind, I even upgraded and tried with below, but still can't reach port 25, what I am doing wrong?

apiVersion: v1

kind: Service
metadata:
  namespace: nginx-ingress
  name: postfix-app
  labels:
    app: postfix-app
spec:
  ports:
  - port: 55
    targetPort: smtp
    name: smtp-svc
    protocol: TCP
  - port: 465
    targetPort: smtpssl
    name: smtpssl-svc
    protocol: TCP
  - port: 587
    targetPort: smtpauth
    name: smtpauth-svc
    protocol: TCP
  selector:
    app: postfix-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postfix-app
  namespace: nginx-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postfix-app
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: postfix-app
    spec:
      containers:
      - image: razalabs/postfix
        name: postfix-app
        ports:
         - containerPort: 25
           name: smtp
           protocol: TCP
         - containerPort: 465
           name: smtpssl
           protocol: TCP
         - containerPort: 587
           name: smtpauth
           protocol: TCP
        volumeMounts:
          - name: certificate-config-volume
            mountPath: /secret
            readOnly: true
      volumes:
        - name: certificate-config-volume
          secret:
            secretName: giglancer-tls
---
apiVersion: k8s.nginx.org/v1alpha1
kind: GlobalConfiguration
metadata:
  name: nginx-configuration
  namespace: nginx-ingress
spec:
  listeners:
  - name: postfix-tcp
    port: 25
    protocol: TCP
---
apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
  name: postfix-tcp
  namespace: nginx-ingress
spec:
  listener:
    name: postfix-tcp
    protocol: TCP
  upstreams:
  - name: postfix-app
    service: postfix-app
    port: 25
  action:
    pass: postfix-app

[Rulox]

Hi @RazaGR

You are right, you need to use TransportServer for TCP LoadBalancing. Make sure you are referencing the correct GlobalConfiguration with the -global-configuration cli argument in your pod.

Anyways, your manifest is almost right, notice how you're using port 55 instead of 25 in your service. if you change that to 25 it should work:

apiVersion: v1
kind: Service
metadata:
  namespace: nginx-ingress
  name: postfix-app
  labels:
    app: postfix-app
spec:
  ports:
  - port: 25 # This was 55 in your example
    targetPort: smtp
    name: smtp-svc
    protocol: TCP
  - port: 465
    targetPort: smtpssl
    name: smtpssl-svc
    protocol: TCP
  - port: 587
    targetPort: smtpauth
    name: smtpauth-svc
    protocol: TCP
  selector:
    app: postfix-app

Apart from that, make sure you are exposing port 25 both in the NGINX Ingress Controller pod and service:

. . . 
    - name: postfix
      port: 25
      protocol: TCP
      targetPort: 25

Let me know if this works for you. I've tested with a postfix docker image and it should be working:

 telnet $IC_IP 25
Trying 192.168.99.248...
Connected to 192.168.99.248.
Escape character is '^]'.
220 postfix-app-6ff46b8bbc-wxqxh.localdomain ESMTP Postfix

[RazaGR]

Hi @Rulox

I was just performing a test on 55 and forgot to update when I posted here.

The real issue was that I forgot to add cli argument for GlobalConfiguration -global-configuration, Now it is accepting connections.

Thank you

[RazaGR]

Hi @Rulox , I am stuck on this for couple of days so opening again.
apart from port 25 I am not able to connect other ports to outside of cluster, I am trying to open 30100 port to accept socket connection, I have setup a bash script to perform some certain actions when visit that port. ie.

nc -lv -s 0.0.0.0 -p30100 | ./run/my/bashscript.sh

I am able to call the service within cluster from other pods and it works perfectly:
curl --output /dev/null --silent --head --fail postfix-app:30100

but if send a socket request on My-External-IP:30100, it doesn't reach to postfix-app:30100 pod. what I am doing wrong?

kubectl get svc

.......
service/nginx-ingress   LoadBalancer   10.245.6.131     My-External-IP 80:31017/TCP,443:31127/TCP,25:30737/TCP,465:32080/TCP,587:30368/TCP,30100:30022/TCP   4h42m
service/postfix-app     ClusterIP      10.245.74.164    <none>        25/TCP,465/TCP,587/TCP,30100/TCP
.......

I have setup in ingress

.......
      containers:
      - image: nginx/nginx-ingress:1.7.0
        name: nginx-ingress
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: smtp
          containerPort: 25
        - name: emailserver
          containerPort: 30100
.......

in load balancer

  type: LoadBalancer
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  - port: 443
    targetPort: 443
    protocol: TCP
    name: https
  - port: 25
    targetPort: 25
    name: smtp
    protocol: TCP
  - port: 30100
    targetPort: 30100
    name: emailserver
    protocol: TCP

and service/deployment etc

apiVersion: v1
kind: Service
metadata:
  namespace: nginx-ingress
  name: postfix-app
  labels:
    app: postfix-app
spec:
  ports:
  - port: 25
    targetPort: smtp
    name: smtp-svc
    protocol: TCP
  - port: 30100
    targetPort: emailserver
    name: emailserver-svc
    protocol: TCP
  selector:
    app: postfix-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postfix-app
  namespace: nginx-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postfix-app
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: postfix-app
    spec:
      containers:
      - image: razalabs/postfix:latest
        imagePullPolicy: Always
        name: postfix-app
        ports:
         - containerPort: 25
           name: smtp
           protocol: TCP
         - containerPort: 30100
           name: emailserver
           protocol: TCP
        volumeMounts:
          - name: certificate-config-volume
            mountPath: /secret
            readOnly: true
      volumes:
        - name: certificate-config-volume
          secret:
            secretName: giglancer-tls
---
apiVersion: k8s.nginx.org/v1alpha1
kind: GlobalConfiguration
metadata:
  name: nginx-configuration
  namespace: nginx-ingress
spec:
  listeners:
  - name: postfix-tcp
    port: 25
    protocol: TCP
  - name: emailserver-tcp
    port: 30100
    protocol: TCP
---
apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
  name: postfix-tcp
  namespace: nginx-ingress
spec:
  listener:
    name: postfix-tcp
    protocol: TCP
  upstreams:
  - name: postfix-app
    service: postfix-app
    port: 25
  - name: emailserver-app
    service: postfix-app
    port: 30100
  action:
    pass: postfix-app

p.s I see below in Nginx-ingress logs, don't know if its related with it

W0528 16:11:30.080768       1 listers.go:79] can not retrieve list of objects using index : Index with name namespace does not exist
W0528 16:11:30.080817       1 listers.go:79] can not retrieve list of objects using index : Index with name namespace does not exist

[RazaGR]

Hi @Rulox I was able to make it work by using two TransportServer, Is it really like this or am I missing something to make it work with single TransportServer?

apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
  name: postfix-tcp
  namespace: nginx-ingress
spec:
  listener:
    name: postfix-tcp
    protocol: TCP
  upstreams:
  - name: postfix-app
    service: postfix-app
    port: 25
  action:
    pass: postfix-app
---
apiVersion: k8s.nginx.org/v1alpha1
kind: TransportServer
metadata:
  name: emailserver-tcp
  namespace: nginx-ingress
spec:
  listener:
    name: emailserver-tcp
    protocol: TCP
  upstreams:
  - name: emailserver-app
    service: postfix-app
    port: 30100
  action:
    pass: emailserver-app

[RazaGR]

ok I think I finally found the main issue, I was sending socket request and it was closing immediately , I put a delay of 2 seconds before closing socket request and it fixed the issue with only using single TransportServer configuration.