Linux: Jumbo ping kills ethertnet interface in 4.14 and 4.18 - RPI3

[mistepien]

Pi 3 Model B Rev 1.2
Raspbian Stretch

Connected to 1000M switch.
If you execute
ping -s 4000 some_ip_eg_raspberry_pi
to or from raspberry pi then Ethernet interface is dead.

This is problem because huge part of my LAN is 9000 mtu (eg. very fast NAS). There is S912 media player (libreelec) which is 100M/1500 mtu capable and it works fine. But any contact between NAS and raspberry pi causes that pi is not responsive any more.
But "ping -s 4000" kills pi even if it is executed on router which is only 1500 mtu capable.

[lategoodbye]

Could you please explain what do you mean with kills ethernet (driver crash)?
Can you provide a dmesg from the Raspberry pi via console after the Pi is unreachable?
Did you change the MTU on the Raspberry Pi 3?

[mistepien]

I mean that raspbian becomes unresponsive via eth0. No ssh, no ping (to or from raspbian). I have not checked wifi. No informations in dmesg. MTU is not changed. Ifconfig says it is 1500mtu. I have even connected raspberry pi to lan via nexx3020 in bridge mode. nexx3020 is 100mbit/1500 mtu capable only. But ping -s 4000 was still effective against raspbian.

[lategoodbye]

So it's unresponsive even if you make ping without the parameter "-s 4000"?

[mistepien]

No, it is unresponsive if ping -s 4000 is used (or if machine with 9000 mtu contacts). Can you reproduce the problem?

[lategoodbye]

No, currently i don't have much time. But i will try to reproduce it.

[lategoodbye]

First of all, according to the LAN95xx databook the maximum allowed packet length (without VLAN tag) is 1518 bytes, so there is no official support for Jumbo frames.

I also tried "ping -s 4000" in my non-jumbo LAN and it works as expected. So the only issue i can see is that the driver accept unsupported MTU values.

So my recommendation is to decrease the MTU to a value which all your devices support.

[mistepien]

But the problem is not my mtu configuration. Problem is that raspberry pi eth interface goes south. I do notthink that databook says something about interface freeze as a result of icmp packet of any type.

Tomorrow I will test it with two computers set, without switch to make more easy to reproduce.

PS. BTW, if problem is with raspberry pi and not all other devices this is quite weird to adjust whole LAN just for one malfunction device. This way you will never get all these patches in mainline kernel. Ask yourself what would be feature of linux in server services if the answer is "adjust all other devices". This is even better than DOS attack because one ping makes server disfunctional.

[lategoodbye]

Okay, ssh worked before ping -s 4000 and after that it runs into a timeout?

[mistepien]

Just like someone took off the cable.

I do not have access now to my hardware. Tomorrow I install fresh stretch (I will write which version). I have netbook with 100Mbit interface. I will try with with crosslink, ip will be set static, without interference with mtu (thus 1500). And I will paste eth statistics.

[maxnet]

This is problem because huge part of my LAN is 9000 mtu (eg. very fast NAS). There is S912 media
player (libreelec) which is 100M/1500 mtu capable and it works fine.

Strange.
That is not supposed to work fine. (ping and ssh may work as that only sends small packets, but not file transfers from/to NAS)

Everything inside same VLAN must have same MTU, as it has no way of detecting what the others support.
Only when communicating outside the subnet, you can get notification from your router, that the packet is being dropped because it is too large, and system can then retry sending smaller packets in response to that.

But the problem is not my mtu configuration. Problem is that raspberry pi eth interface goes south. I do
think that databook says something about interface freeze.

Simply drops too large packets.

ethtool -S is not supported on lan95xx, but you can still see that happening by looking at the stats in /sys
This is what it looks likes after sending 25 jumbo packets to it:

$ grep "" /sys/class/net/eth0/statistics/*error*
/sys/class/net/eth0/statistics/rx_crc_errors:0
/sys/class/net/eth0/statistics/rx_errors:50
/sys/class/net/eth0/statistics/rx_fifo_errors:0
/sys/class/net/eth0/statistics/rx_frame_errors:25
/sys/class/net/eth0/statistics/rx_length_errors:25
/sys/class/net/eth0/statistics/rx_missed_errors:0
/sys/class/net/eth0/statistics/rx_over_errors:0
/sys/class/net/eth0/statistics/tx_aborted_errors:0
/sys/class/net/eth0/statistics/tx_carrier_errors:0
/sys/class/net/eth0/statistics/tx_errors:0
/sys/class/net/eth0/statistics/tx_fifo_errors:0
/sys/class/net/eth0/statistics/tx_heartbeat_errors:0
/sys/class/net/eth0/statistics/tx_window_errors:0

Can still communicate fine with other devices after that.

[lategoodbye]

I did the following test with my notebook (e1000e nic) and mainline 4.19rc1:

sudo ifconfig eno1 mtu 4074
ping <rpi>
# receive replies as expected

ping -s 4000 <rpi>
# doesn't receive replies (as expected)

ping <rpi>
# receive replies as expected

[mistepien]

And the puzzle is solved. The problem is dhclient interfering with mtu.
In my lan dhpc server in dnsmasq on openwrt with options
list dhcp_option 'option:mtu, 9000'
It informs dhpc clients to set mtu to 9000. As I said I have few devices capable 100Mbit/1500mtu and they simply ignore this request. But in raspbian something goes wrong.

But if I comment line about mtu in /etc/dhcpcd.conf, like this
#option interface_mtu
then everything is just fine. Raspbian stretch with this "amendment" works in my lan.

[maxnet]

But if I comment line about mtu in /etc/dhcpcd.conf, like this

option interface_mtu

then everything is just fine. Raspbian stretch with this "amendment" works in my lan.

Have you also tested letting the Pi download something large from your NAS that has mtu 9000?

[mistepien]

Via rsync (from rsyncd running on NAS with 9000 mtu)
received 7,038,955,954 bytes 8,915,713.75 bytes/sec

it was one huge file

[lategoodbye]

Here is my proposal to fix the kernel issue:
https://gist.github.com/lategoodbye/bb93f1efceb87b91cebd80f3ce34cba3

[lategoodbye]

@mistepien Are you able to test this patch? In case of success (works even with option interface_mtu), i would make a pull request.

[mistepien]

But how can I generate raspberrypi-kernel deb package?

[lategoodbye]

This isn't necessary for testing:
https://www.raspberrypi.org/documentation/linux/kernel/building.md

[mistepien]

Dmesg says:
eth0: Invalid MTU 9000 requested, hw max 1500

Thus it works like eg. my laptop's wifi. This is how it should work. Push it into code. I think issue can be closed.

[lategoodbye]

A fix has been accepted mainline and will be queued for stable:
https://marc.info/?l=linux-netdev&m=154173574216784&w=2

[mistepien]

so we can close it :D

[nanjirparmar]

Hi All,
Is jumbo MTU supported in RPI image with RPI 4 ?
I have tried latest raspberry pi image having Linux version 4.19.73-v7l+. But still I am not able to set MTU > 1500 !

Any help please ?