Zerotierone: Feature Request - How to set Bridge mode, with one NIC?

Created on 22 Mar 2017 · 12Comments · Source: zerotier/ZeroTierOne

I would like to use it like similar with Hamachi in "bridge" mode.

I try it yesterday (Ubuntu Server, CLI), but no success (every time i kill my full network connection...)

Please add a option to easy to use one NIC bridge mode/configuration.

backlog

Source

vampywiz17

Most helpful comment

Tried with the below config (/etc/network/interfaces) and it works perfectly. The only downside is you would need a static IP configuration. If you want DHCP, you might have to add another veth interface to the bridge which can be configured to do DHCP. I tested all of this in a new LXC container, but it works exactly the same for any Ubuntu host with single NIC.

# Your main physical interface / internet connection
auto eth0
iface eth0 inet manual

# Bridge which combines eth0 and zt0 networks
auto br0
iface br0 inet static
        address 192.168.1.100
        netmask 255.255.255.0
        gateway 192.168.1.1
        dns-nameservers 8.8.8.8 8.8.4.4
        bridge_ports eth0 zt0
        bridge_fd 0
        bridge_maxage 0

Some notes / configuration steps :

Enable Allow Ethernet Bridging and Do Not Auto-Assign IPs within the web-interface for your bridge device.
Install bridge-utils and you can use brctl show to know if both the interfaces are actually linked.
Set allowManaged=0 in /var/lib/zerotier-one/networks.d/XXXX.local.conf as zt client automatically set a route through the zt0 interface, which is useless as it's now bridged.
Run ip route add 10.0.1.0/24 dev br0 and replace the subnet with your zt subnet.

Run ip route to know the active route. It should be something like this:

default via 192.168.1.1 dev br0 onlink
192.168.1.1/24 dev br0 proto kernel scope link src 192.168.1.100
10.0.1.0/24 dev br0

Restart networking and zerotier-one after config. I have this config all setup to run a few seconds post boot as the bridge_ports eth0 zt0 doesn't work at startup as the zt0 interface wouldn't be up yet. You can do ifdown and ifup on eth0 and br0 to get it working again. I have the bash script set to run once it finds zt0 interface is up. Wish there was a way to "prepoulate" the zt0 interface so we wouldn't have to resort to bash scripts. Oh, and the ip route too.
Ping 8.8.8.8, followed by some device in your local network, followed by any device within the zt subnet to test! Maybe I will post a little detailed guide/tutorial after I play with it for a bit. Let me know if any part doesn't work.

Final ifconfig (not much to see here really):

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.1  netmask 255.255.255.0  broadcast 192.168.1.254
        inet6 fe80::216:3eff:fe7f:3bac  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:7f:3b:ac  txqueuelen 1000  (Ethernet)
        RX packets 3673  bytes 616716 (616.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2207  bytes 397794 (397.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:16:3e:7f:3b:ac  txqueuelen 1000  (Ethernet)
        RX packets 4607  bytes 2228777 (2.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2721  bytes 455359 (455.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 64  bytes 6669 (6.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 64  bytes 6669 (6.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

zt0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 2800
        inet6 fe80::1018:f1ff:fe86:1f06  prefixlen 64  scopeid 0x20<link>
        ether 12:18:f1:86:1f:06  txqueuelen 1000  (Ethernet)
        RX packets 703  bytes 169240 (169.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 30  bytes 2677 (2.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@xxxxx:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.00163e7f3bac       no              eth0
                                                        zt0

More ref:

https://support.zerotier.com/knowledgebase.php?entry=show&search-for=&article=N2ViNmQxZmZiMDRkM2UzODZlOWJiOWFiODFkOGRlZDE_ (wasn't really detailed though, more of a general overview)

small-mallet on 19 Sep 2017

❤2

All 12 comments

Hmm... how does Hamachi do it at the OS level? Does it create a bridge device or does it just do its own bridging by attaching to the Ethernet device?

(You can see with ifconfig or ip addr list)

If so I'm guessing they attach to the network device with a raw socket or pcap and implement their own bridge.

adamierymenko on 12 Apr 2017

enp0s25: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::201:80ff:fe76:9844  prefixlen 64  scopeid 0x20<link>
        ether 00:01:80:76:98:44  txqueuelen 1000  (Ethernet)
        RX packets 115597  bytes 76685845 (76.6 MB)
        RX errors 0  dropped 7  overruns 0  frame 0
        TX packets 79141  bytes 19492846 (19.4 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xfdfc0000-fdfe0000

ham0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1404
        inet 25.49.110.199  netmask 255.0.0.0  broadcast 25.255.255.255
        inet6 fe80::7879:19ff:fe31:6ec7  prefixlen 64  scopeid 0x20<link>
        inet6 2620:9b::1931:6ec7  prefixlen 96  scopeid 0x0<global>
        ether 7a:79:19:31:6e:c7  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1313  bytes 101986 (101.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ham1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1404
        inet6 fe80::223:c3ff:fe31:6ec7  prefixlen 64  scopeid 0x20<link>
        ether 00:23:c3:31:6e:c7  txqueuelen 1000  (Ethernet)
        RX packets 52  bytes 3120 (3.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6738  bytes 1244079 (1.2 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ham-br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1404
        inet 192.168.31.105  netmask 255.255.255.0  broadcast 192.168.31.255
        inet6 fe80::201:80ff:fe76:9844  prefixlen 64  scopeid 0x20<link>
        ether 00:01:80:76:98:44  txqueuelen 1000  (Ethernet)
        RX packets 94786  bytes 73642119 (73.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 74812  bytes 18672595 (18.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 6652  bytes 1937359 (1.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 6652  bytes 1937359 (1.9 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

vampywiz17 on 12 Apr 2017

Interesting. They are creating a bridge. We'll have to investigate this.

adamierymenko on 19 Apr 2017

Hello there, any info that it is possible to do this?

vampywiz17 on 31 May 2017

I have been using it with a simple linux bridge on Ubuntu (bridge-utils) and assigning them to LXD/LXC containers. Was able to bridge the virtual port and zt0 too without any issue. Will post my config later maybe.

small-mallet on 18 Sep 2017

Thanks a lot!

vampywiz17 on 18 Sep 2017

# Your main physical interface / internet connection
auto eth0
iface eth0 inet manual

# Bridge which combines eth0 and zt0 networks
auto br0
iface br0 inet static
        address 192.168.1.100
        netmask 255.255.255.0
        gateway 192.168.1.1
        dns-nameservers 8.8.8.8 8.8.4.4
        bridge_ports eth0 zt0
        bridge_fd 0
        bridge_maxage 0

Some notes / configuration steps :

Enable Allow Ethernet Bridging and Do Not Auto-Assign IPs within the web-interface for your bridge device.
Install bridge-utils and you can use brctl show to know if both the interfaces are actually linked.
Set allowManaged=0 in /var/lib/zerotier-one/networks.d/XXXX.local.conf as zt client automatically set a route through the zt0 interface, which is useless as it's now bridged.
Run ip route add 10.0.1.0/24 dev br0 and replace the subnet with your zt subnet.

Run ip route to know the active route. It should be something like this:

default via 192.168.1.1 dev br0 onlink
192.168.1.1/24 dev br0 proto kernel scope link src 192.168.1.100
10.0.1.0/24 dev br0

Restart networking and zerotier-one after config. I have this config all setup to run a few seconds post boot as the bridge_ports eth0 zt0 doesn't work at startup as the zt0 interface wouldn't be up yet. You can do ifdown and ifup on eth0 and br0 to get it working again. I have the bash script set to run once it finds zt0 interface is up. Wish there was a way to "prepoulate" the zt0 interface so we wouldn't have to resort to bash scripts. Oh, and the ip route too.
Ping 8.8.8.8, followed by some device in your local network, followed by any device within the zt subnet to test! Maybe I will post a little detailed guide/tutorial after I play with it for a bit. Let me know if any part doesn't work.

Final ifconfig (not much to see here really):

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.1  netmask 255.255.255.0  broadcast 192.168.1.254
        inet6 fe80::216:3eff:fe7f:3bac  prefixlen 64  scopeid 0x20<link>
        ether 00:16:3e:7f:3b:ac  txqueuelen 1000  (Ethernet)
        RX packets 3673  bytes 616716 (616.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2207  bytes 397794 (397.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 00:16:3e:7f:3b:ac  txqueuelen 1000  (Ethernet)
        RX packets 4607  bytes 2228777 (2.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2721  bytes 455359 (455.3 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 64  bytes 6669 (6.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 64  bytes 6669 (6.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

zt0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 2800
        inet6 fe80::1018:f1ff:fe86:1f06  prefixlen 64  scopeid 0x20<link>
        ether 12:18:f1:86:1f:06  txqueuelen 1000  (Ethernet)
        RX packets 703  bytes 169240 (169.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 30  bytes 2677 (2.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

root@xxxxx:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.00163e7f3bac       no              eth0
                                                        zt0

More ref:

https://support.zerotier.com/knowledgebase.php?entry=show&search-for=&article=N2ViNmQxZmZiMDRkM2UzODZlOWJiOWFiODFkOGRlZDE_ (wasn't really detailed though, more of a general overview)

small-mallet on 19 Sep 2017

❤2

@small-mallet Thanks! I had been trying this off and on, but hadn't thought to allowManaged=0. Seems to be working on a raspberry-pi.

laduke on 19 Sep 2017

@laduke Yeah, got it from here: https://github.com/zerotier/ZeroTierOne/issues/509

small-mallet on 19 Sep 2017

@small-mallet I tried to follow your instructions and think I did everything but it does not work.

My local network is 172.30.0.0/24 with the Ubuntu VM i'm using being 172.30.0.3. The ZeroTier Network is 172.30.1.0/24 with the IP 172.30.1.3 assigned to the VM for ZT0.

br0       Link encap:Ethernet  HWaddr 00:0c:29:cd:38:2c  
          inet addr:172.30.0.3  Bcast:172.30.0.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fecd:382c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2829 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4083 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:360512 (360.5 KB)  TX bytes:984719 (984.7 KB)

ens160    Link encap:Ethernet  HWaddr 00:0c:29:cd:38:2c  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2828 errors:0 dropped:5 overruns:0 frame:0
          TX packets:4075 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:400202 (400.2 KB)  TX bytes:983759 (983.7 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:83 errors:0 dropped:0 overruns:0 frame:0
          TX packets:83 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:6256 (6.2 KB)  TX bytes:6256 (6.2 KB)

zt0       Link encap:Ethernet  HWaddr ea:58:ff:b4:d5:a1  
          inet6 addr: fe80::e858:ffff:feb4:d5a1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:2800  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1158 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:630 (630.0 B)  TX bytes:219385 (219.3 KB)

default via 172.30.0.1 dev br0 onlink 
172.30.0.0/24 dev br0  proto kernel  scope link  src 172.30.0.3 
172.30.1.0/24 dev br0  scope link 

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto ens160
iface ens160 inet manual

# Bridge which combines ens160 and zt0 networks
auto br0
iface br0 inet static
        address 172.30.0.3
        netmask 255.255.255.0
        gateway 172.30.0.1
        dns-nameservers 8.8.8.8 172.30.0.1
        bridge_ports ens160 zt0
        bridge_fd 0
        bridge_maxage 0

bridge name bridge id       STP enabled interfaces
br0     8000.000c29cd382c   no      ens160
                            zt0

I also set allowManaged=0

I restart the interfaces and add the IP route after I do that and it can't reach the ZeroTier network at all. The Zerotier-cli status is up and listing the network shows connected with the correct IP. I have other computers with the client installed and it can communicate just fine.

mbond@ZTBridge:~$ ping 172.30.1.55
PING 172.30.1.55 (172.30.1.55) 56(84) bytes of data.
From 172.30.0.3 icmp_seq=1 Destination Host Unreachable
From 172.30.0.3 icmp_seq=2 Destination Host Unreachable
From 172.30.0.3 icmp_seq=3 Destination Host Unreachable
From 172.30.0.3 icmp_seq=4 Destination Host Unreachable
From 172.30.0.3 icmp_seq=5 Destination Host Unreachable
From 172.30.0.3 icmp_seq=6 Destination Host Unreachable
From 172.30.0.3 icmp_seq=7 Destination Host Unreachable
From 172.30.0.3 icmp_seq=8 Destination Host Unreachable
From 172.30.0.3 icmp_seq=9 Destination Host Unreachable

Anything you can think of that I should try?

mbond-vs on 17 Nov 2017

😕1

@bmullan That's correct. An interface that is a member of a bridge should not have IPs assigned as it cannot be used directly. Addresses are set on the bridge instead.