Zerotierone: Managed routes overwrite existing routes on OPNsense

[darkain]

I have two OPNsense routers each with their own WAN connection and connected to the same shared LAN. Both are connecting to the same ZT network. Within that ZT network, one of the routers is set as a Managed Route gateway. The router that isnt the gateway has its LAN subnet pointing to the wrong interface and becomes entirely inaccessible from the LAN.

Local LAN: 192.168.100.0/24
ZT LAN: 192.168.200.0/24
Managed Route: 192.168.100.0/24 > 192.168.200.1

Router 1:
LAN: 192.168.100.1
ZT: 192.168.200.1

Router 2:
LAN: 192.168.100.5
ZT: 192.168.200.5

With this configuration, Router 2's LAN subnet (192.168.100.0/24) is now pointing to the ZT virtual address of Router 1 (192.168.200.1) instead of its own local interface, even though it is physically directly connected to that LAN.

Before adding a managed route locally, I think ZT should make the effort to see if that subnet already exists. If it does, no action should be taken.

[glimberg]

Sounds like in this case you want to set allowManaged=0 on router 2 and manage the IP and route manually. zerotier-cli set <network_id> allowManaged=0. Once set, ZeroTier won't automatically push the IP addresses or routes to that router.

[darkain]

If that is set, then ALL other managed routes disappear too. Also, using the CLI tool, will that setting persist between reboots, or is it session only?

[glimberg]

Yes, the allowManaged setting will persist between reboots, assuming you don't delete ZeroTier's data dir on reboot.

[laduke]

It'll stay, but you can also add it to a file in /var/lib/zerotier-one/networks.d/${networkdId}.local.conf, if that's easier to manage.

$ cat 1234565963d77bcd.local.conf 
allowManaged=1
allowGlobal=0
allowDefault=0

Other ideas:

• blacklist the address or interface in local.conf

• Change the ZT managed route to /23 - It'll be less specific than the LAN, so the os will use the LAN

[darkain]

I have several routes being managed, not just one. Using ZT as basically the central router between several building's LANs. Luckily, none of the LAN subnets are next to each other, so the /23 trick is working. But I still think that ZT shouldn't override/replace any routes that already exist on physical interfaces.

[jdrews]

Referring to https://github.com/zerotier/ZeroTierOne/issues/354
I think it would be better if allowManaged=0 would not set any managed routes, but still set the IP of the zt interface.

[darkain]

Curious if there has been any work on this bug at all? ZT should not be overwritting LAN interface routes/subnets. I was just bitten by this again on one of my networks. Luckily the other network is on OSPF now, and that is smart enough to know local vs remote routes and manage it properly.

[ihipop]

https://github.com/zerotier/ZeroTierOne/issues/750#issuecomment-570445251
I think a router metric to static routers on my.zerotier.com is a better Solution

[reedog117]

I just got bitten by this on a complex CARP redundant gateway installation with OPNsense. The allowManaged setting doesn’t seem to be exposed in the OPNsense UI so I’m wondering if an update is needed to better manage this.

[ropeguru]

I am running into this same issue.

Two interfaces on my router, em0 and em1. Em0 is LAN and em1 is WAN.

LAN is 192.168.1.204
WAN is 10.1.10.15
ZT is 192.168.196.195
All subnets are /24

In ZeroTier I have a managed route for the 192.168.1.0/24 subnet so other location know how to get it from remote locations. When OpnSense boots, ZeroTier overwrites the connected interface entry in the routing table and at that point, neither the 192.168.1.204 nor the 192.168.196.195 addresses are reachable. As soon as the ZeroTier route is removed and the correct connected route entry is applied, both the LAN and ZT inferaces become available.

A connected interface's entry in a route table should never be overwritten.

Is this a ZeroTier issue or a OpnSense issue?