Flannel: Can flannel connect to more than one ethernet interface on the same server?

[vijaykumark]

We have two ethernet interfaces (eth0, eth1) on one host server which are connected to two different networks. The idea is to connect both networks to the docker containers over flannel.
Does flannel support connecting to more than one ethernet interface using multiple --iface options?
The documentation doesn't mention about multiple --iface options. Is there any way to connect to more than one interface simultaneously?

Edit: Added a diagram which explains the required feature clearly.
Currently Flannel runs as shown in Figure 1.
We are looking for a way to run flanneld on top of two ethernet interfaces as shown in Figure 2. The traffic from two different ethernet interfaces should be listened by flanneld instead of one ethernet interface currently.

[philips]

Is there a reason you don't just bond the two interfaces? I am confused why you want to have two interfaces in use.

[vijaykumark]

AFAIK, bonding is used when you need redundant links, fault tolerance or load balancing networks. Bonding allows you to aggregate multiple ports into a single group, effectively combining the bandwidth into a single connection.

In our scenario, the two ethernet interfaces (eth0 and eth1) are connected to two different networks and both interfaces have IP address from different subnets (eg. 10.0.0.1/20, 192.168.0.0/20).
Not sure if we could use bonding in our scenario.

[yogeshmsharma]

to add on this, for traffic management incoming traffic can be pushed over eth0 and outgoing on eth1. or eth0 talking to external world and eth1 to internal world.

How flannel can be configured to do so?

[NikolayMurha]

Hello.
We have the same issue.
We have three dedicated servers in one datacenter (DC1). This servers have public interface eth0 with public IP. One server is kubernetes master+worker, another two - workers. All worked fine.
But we have a issue to add workers from another DC with private subnet 172.25.0.0/16.
I've openvpn tunnel from servers (in DC1) to DC2.
So all servers in DC1 have two interfaces (eth0 - public IP, and tun0 - IP in private subnet)
All nodes see each other, traffic for nodes in private DC routed to the tun0 and another traffic routed to eth0.

Unfortunately, in this case, PODs started on servers in the DC1 cannot see the PODs on servers in the DC2 because traffic for DC1 should be passed to another interface (tun0).
I can change --iface to the tun0, and reconfigure all nodes, to communicate through private IPS, but then all the traffic will goes through the VPN tunnel, even the between nodes in the same DC.
I think it is better use the routing table and not to send everything through one interface.
Routing table contains everything you need for this:

 default         x.x.x.x   0.0.0.0         UG    0      0        0 eth0
 10.10.0.0       *               255.255.0.0     U     0      0        0 flannel0
 10.10.38.0      *               255.255.255.0   U     0      0        0 docker0
 172.16.0.0       172.16.5.1      255.255.0.0     UG    0      0        0 tun0

[celevra]

we have also 2 interfaces on each node in a 3 node cluster
we want the "real" traffic through eth0 and the "Cluster interconnect" traffic through eth1

[vijaykumark]

@philips : After your question, added more information in the comments sections. Also updated the issue with a diagram, which should explain the issue more clearly. Hope this helps.

[qrpike]

Did anyone solve this issue or know the configuration needed to do so? I also have a private and public network and would like to do the same.

Thanks,

[AnberLu]

@vijaykumark hi，have you slove it ?

[vijaykumark]

There is no response from @philips or from flannel team since last year.

[tomdee]

I'm still not sure I understand what's being asked for here. The way the diagram is written above implies that the UDP backend is being used, but I think it would be better to just center the discussion around the vxlan backend. This means that the flanneld isn't in the packet path.

[joshrosso]

@tomdee @philips

I setup a cluster routing to 2 nics (not bonded) yesterday using the vxlan backend as described by @tomdee. I believe this issue can be closed?

[vijaykumark]

@tomdee @philips @joshrosso
We are not using vxlan backend and we are not using UDP.
We use two physical ethernet/fiber networks attached over two ethernet interfaces to the same server. All the networking is managed externally by different switches.
So, the server will not know how the networks are managed once the traffic goes out of the ethernet interfaces.
Is it possible to attach flanneld to such a network? If yes, do you have documentation which I can refer to?

[tomdee]

@vijaykumark Sorry, flannel vxlan only supports binding to single ethernet device. I'm still not sure what flannel backend you're using - can you share that information?

[vijaykumark]

@tomdee We are running flannel on baremetal servers (CentOS 7) which have two ethernet ports connecting to two different physical ethernet networks. There is no other SDN solution involved other than flannel.

[warmchang]

@vijaykumark , flannel used to support multi networks (as EXPERIMENTAL), but removed later.
Multi-network mode (EXPERIMENTAL) & https://github.com/coreos/flannel/pull/633.

[wzrdtales]

That is very unfortunate @warmchang :/

[vijaykumark]

Closing the issue as it's stale.