Zerotierone: TCP as first class P2P transport

[kulheim]

Some providers significantly reduce UDP speed, for example OVH on dedicated servers allows only 1.05 Mb/s UDP whilst 1Gb/s for TCP.

I wanted to replace Tinc by Zerotier, but Zerotier 1.28 on OVH servers performs worse than Tinc (if Tinc is configured to use 'TCPOnly=yes').
I must disabled Tinc device in /var/lib/zerotier/local.conf, because Zerotier initially routed traffic through Tinc. After that 'zerotier-cli listpeers' shows correct external addresses and port 9993, but I suppose traffic goes through some zerotier planet server.

Zerotier should be clever and fallback to TCP when available on port 9993.

[coretemp]

Not a bad idea, but I think working with OVH support to remove this limitation seems more useful for everyone. For me, it would completely kill OVH as a place to host anything. If it's implemented I would also like to see it as an option, not a default (or if it's a default, it should output a warning that this option is being applied and what the implications are).

I.e., them not having full speed UDP networking support is costing them money.

Perhaps their target market is just the low end of the market, however and I suppose their typical problem is that their users cause more support costs when UDP traffic isn't limited.

[adamierymenko]

This has been planned for some time, so marking as backlog.

BTW didn't know about OVH. Is that actually true? We host some things there so I am planning to contact them on this. This is a silly limitation.

[kulheim]

My mistake - limit 1.05Mb/s is a default value in Iperf3 for UDP bandwidth.
At OVH there's permanent AntiDDOS which lowers UDP trafic slightly more than TCP.
Real TCP transfer speed through Zerotier (UDP) is at 80-90% of Tinc with TCPOnly='yes'.

[ivan4th]

Currently, Yota cellular operator in Russia applies some strong traffic shaping rules to all of UDP traffic with an exception of UDP/1194 which is to be used by VPNs (the exception works for both OpenVPN and WireGuard). ZeroTier traffic gets very slow. I've contacted their support, and they said they've starting working on making an exception for ZT too (making an exception for port 9993 wouldn't be enough) but not sure if their DPI architecture can support this. So, TCP fallback would help a lot.

[adamierymenko]

Closing since this one's been on the table for a long time but is pending a few things including more research into whether we can use HTTP2 or WebSockets encapsulation and how to prevent head-of-line blocking.