Kube-router: Kube-router should auto-configure sensible MTU for kube-bridge instead of default one

[murali-reddy]

CNI bridge plug-in can be configured explictly to set MTU for the bridge and pod interfaces connected to the bridge

https://github.com/containernetworking/plugins/tree/master/plugins/main/bridge

Kube-router does not set MTU value by defaults, so it get sets to default value of 1500.

In cases where physical interface has large MTU (for e.g 9001) then kube-bridge can become bottleneck as path MTU effectively reduces to 1500

[dimm0]

@murali-reddy any expected date for this? We really need it.

Also I was able to set manually the veth devices MTU, then kube-bridge MTU, then added the proper MTU to /etc/docker/daemon.json, but still having 1500 on my container's eth0

[murali-reddy]

@dimm0 i am not sure how to go about existing clusters, but in new cluster certainly MTU can be set. I will see what other CNI providers that use bridge is inferring a sensible MTU.

[jpds]

In addition to autoconfiguring this, there should be a environment variable or option that sets the MTU for the CNI. With Calico IP-in-IP, they recommend setting this to 8950 on hosts with MTU of 9000.

[murali-reddy]

I have marked this issue for next release. Will try to get resolution for this issue.

[jjo]

For other ones facing this issue, a very nasty _dont-show-your-mom_-like workaround
I'm using on deployed nodes (via terraform, in my case), as changing /etc/docker/daemon.json
doesn't apply to devices other than docker0 (kube-bridge an others attached to it in this case):

sudo tee /etc/udev/rules.d/71-docker-mtu.rules << EOF
## Force MTU down for devices w/1500 and addr_assign_type != permanent address (pre-set)
## ^ doesn't work as eth0 inside container is not touched :(
# SUBSYSTEM=="net", ACTION=="add", ATTR{mtu}=="1500", ATTR{addr_assign_type}!="0", RUN+="/sbin/ip link set mtu 1450 dev '%k'"
# SUBSYSTEM=="net", ACTION=="add", KERNEL=="kube-bridge", RUN+="/sbin/ip link set mtu 1450 dev '%k'"
## Hack any e(thernet) with mtu!=1500 for MSS clamp
SUBSYSTEM=="net", ACTION=="add", KERNEL=="e*", ATTR{mtu}!="1500", RUN+="/sbin/iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS  --clamp-mss-to-pmtu -o %k"
SUBSYSTEM=="net", ACTION=="add", KERNEL=="e*", ATTR{mtu}!="1500", RUN+="/sbin/iptables -I OUTPUT  -p tcp --tcp-flags SYN,RST SYN -j TCPMSS  --clamp-mss-to-pmtu -o %k"
EOF
sudo udevadm control -R
sudo udevadm trigger --attr-match=subsystem=net -c add

[carnivorelogic]

I have marked this issue for next release. Will try to get resolution for this issue.

Hey --- any news about resolution of putting mtu as variable? Bridge plugin has already interface for it: _mtu (integer, optional): explicitly set MTU to the specified value. Defaults to the value chosen by the kernel_.

Thanks

[michaelajr]

Any update?

[murali-reddy]

@carnivorelogic @michaelajr I will work on fix for this. Should be part of next release.

[rearden-steel]

@murali-reddy is there any chance to get this fix implemented?