kube-router: Direct Server Return (tunneling mode) can't forward packet (DF flag, mtu > 1480)

[napat1412]

What happened:
kube-router: Direct Server Return (tunneling mode) can't forward packet (DF flag, mtu > 1480).
I measure TCP bandwidth with iperf3.
iperf3 -C <$public-ip> //fail
iperf3 -C <$public-ip> -M 1441 //fail
iperf3 -C <$public-ip> -M 1440 //success

---

What you expected to happen:
kube-router: Direct Server Return can reply normal packet (default mtu 1500).
iperf3 -C <$public-ip> //should be success

---

Question:
Q1) Could kube-router clear DF flag before send packet to tunneling?

Q2) how to increase MTU size for tunneling interface and related interface (e.g. bridge interface or physical interface)?

How to reproduce it (as minimally and precisely as possible):
I deploy kube-router following kube-router: DSR
I deploy iperf3 service on kubernetes

kind: Deployment
metadata:
  name: iperf
  namespace: default
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: iperf
    spec:
      containers:
      - name: iperf
        image: "networkstatic/iperf3"
        args: ["-s"]
        ports:
          - name: iperf
            containerPort: 5201
            protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  #annotations:
  #  kube-router.io/service.dsr: tunnel
  name: iperf
  namespace: default
  labels:
    app: iperf
spec:
  selector:
    app: iperf
  externalIPs: [$public-ip]
  ports:
  - name: iperf
    port: 5201
    targetPort: 5201
    protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kube-router.io/service.dsr: tunnel
  name: iperf-dsr
  namespace: default
  labels:
    app: iperf
spec:
  selector:
    app: iperf
  externalIPs: [$public-ip]
  ports:
  - name: iperf
    port: 5201
    targetPort: 5201
    protocol: TCP

[icefed]

my fork fixed this problem by set mss with iptables.

[murali-reddy]

@icefed would you mind raising a PR?

[icefed]

@murali-reddy PR opened.
https://github.com/cloudnativelabs/kube-router/pull/769

[murali-reddy]

thanks @icefed will take a look

[rearden-steel]

@murali-reddy this problem still exists and made us a lot of pain.
Please consider merging the PR.

[aauren]

Adding to 1.2 release as this seems like a reasonable request, the work has already been submitted, and it's still able to be merged.

[qingkunl]

Hit this issue in our production as well

[qingkunl]

BTW, looks like this PR https://github.com/cloudnativelabs/kube-router/pull/769 only covers the TCP case. What about the UDP case?

[aauren]

Created a new PR with a fix in #1063

@qingkunl UDP does not have a concept of MSS so there is no way to adapt this fix to work for UDP. Instead, we probably need to fix PMTU for DSR (https://github.com/cloudnativelabs/kube-router/issues/685#issuecomment-487611186) which would allow the client to figure out the maximum size for a UDP datagram based upon ICMP queries.

At least now that #733 has been merged PMTU works for most kube-router services (excluding DSR services).