Re: how to handle bonding failover when using a bridge over the bond?

[Jay Vosburgh <fubar@us.ibm.com>]

Chris Friesen <chris.friesen@genband.com> wrote:

>On 02/14/2013 02:01 AM, Cong Wang wrote:
>> On Wed, 13 Feb 2013 at 00:30 GMT, Chris Friesen<chris.friesen@genband.com>  wrote:
>>> On 02/12/2013 06:02 PM, Jay Vosburgh wrote:
>>>> 	The bond doesn't track all of the MACs that go through it, but
>>>> the bridge presumably does, and could respond to the FAILOVER notifier
>>>> with something to notify the switch that the port assignments for the
>>>> various MACs have changed.
>>>
>>> That would probably make sense.  I've added the bridging folks, maybe
>>> they'll have a suggestion how this sort of thing should be handled.
>>>
>>
>> It is already handled. When BONDING_FAILOVER is triggered and the MAC has
>> been changed, NETDEV_CHANGEADDR is issued too, then bridge will capture
>> it and update its fdb:
>>
>>          case NETDEV_CHANGEADDR:
>>                  spin_lock_bh(&br->lock);
>>                  br_fdb_changeaddr(p, dev->dev_addr);
>>                  changed_addr = br_stp_recalculate_bridge_id(br);
>>                  spin_unlock_bh(&br->lock);
>>
>>                  if (changed_addr)
>>                          call_netdevice_notifiers(NETDEV_CHANGEADDR, br->dev);
>>
>>                  break;
>
>I'm not familiar with the bridge code, can you elaborate on how this helps?

	I'm not sure that it does, even if you're using STP (although
I'd want to try it with STP to make sure).  This only updates the fdb's
MAC for the bond's port.  It won't affect the VM's MACs (which it
shouldn't, because they don't change), and won't send any gratuitous
updates through the bond's port to the switch that would notify the
second switch ("B" in Chris's description, below) that the switch port
for the VM's MAC(s) has changed.

	Also, if the bond has fail_over_mac=follow, then no CHANGEADDR
is issued, because the MAC address does not change.  This is not common
(and not the case in the configuration described below), but does occur.

>The problem scenario is this:
>
>I have a host with eth0/eth1 bonded together as bond0.  eth0/eth1 are
>connected to separate L2 switches, which are interconnected.
>
>On the host there are a number of virtual machines, each with a virtual
>interface.
>
>All the virtual interfaces as well as bond0 are bridged together to allow
>the VMs, the host, and the outside world to talk to each other.
>
>Currently the host does NOT participate in STP because it is considered an
>edge node.
>
>Suppose eth0 is the active link and we pull it.  The bond will make eth1
>active and emit gratuitous arp packets for itself, so the external L2
>switches will update the location of the MAC address belonging to the
>bond.  On loss of carrier for the link to eth0 L2 switch "A" will drop the
>entries for the MAC addresses, including the ones for the virtual
>machines.
>
>The problem is that L2 switch "B" still thinks that all the virtual
>machines are accessible via L2 switch "A".  Thus any incoming packets
>destined for a virtual machine will get dropped.

	I'm trying to track down the system I tested previously to see
exactly how it is set up and why it works when yours does not.  It's
possible that it doesn't work, and the testing we did simply missed this
case.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com