Bonding in active-backup mode with arp monitoring on Xen

[Tsutomu Fujii <t-fujii@nb.jp.nec.com>]

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Hi.

I was trying to use bonding from a guest OS on xen in active-backup
mode with arp monitoring, but the virtual server can't communicate to
the peer server via bonding interface.

The network structure is the following.

+-------------------------------+
|        virtual I/F(eth0)      | <- have IP address.
+-------------------------------+
|      virtual bridge(xenbr0)   |
+-------------------------------+
|         bonding(bond0)        | <- don't have IP address
+--------------+-+--------------+
|phy I/F(peth0)| |phy I/F(peth1)|
+--------------+ +--------------+

The attached file describes how I configured the network.

In this case, when I run 'cat /proc/net/bonding/bond0', 'MII Status'
is shown as down.

This behavior only occurs in active-backup mode with arp monitoring.
It does not happen in active-backup mode with mii monitoring nor other
mode that can use arp monitoring.

>From the output of tcpdump, it seems that arp packet isn't transmitted.
This problem is caused because bond_activebackup_arp_mon(), which is the
function that sends arp packet in active-backup mode, checks whether
bonding master interface has an IP address(bond_has_ip()), before arp
packets are transmitted using bond_arp_send_all().
In the above setting, bonding master interface don't have IP address.

How about removing bond_has_ip() from the condition for calling
bond_arp_send_all() to use bonding with xen?
When I removed bond_has_ip(), the resulting kernel worked fine and
active-backup mode works with arp monitoring.

I confirmed this behavior in RHEL5(2.6.18-44.el5xen). And I confirmed
that the upstream kernel(2.6.23) also has the same condition before
calling bond_arp_send_all().

Thanks.

---
Tsutomu Fujii



[-- Attachment #2: set_bonding.txt --]
[-- Type: text/plain, Size: 1444 bytes --]

Setting procedure of bonding.

1) Make or modify the following config file on dom0.
   - /etc/sysconfig/network-scripts/ifcfg-bond0
     TYPE=Bonding
     DEVICE=bond0
     BOOTPROTO=none
     ONBOOT=no

   - /etc/sysconfig/network-scripts/ifcfg-eth1
     DEVICE=eth1
     BOOTPROTO=none
     ONBOOT=no

   - /etc/sysconfig/network-scripts/ifcfg-eth2
     DEVICE=eth2
     BOOTPROTO=none
     ONBOOT=no

   - /etc/modprobe.conf (Add to following two lines.)
     alias bond0 bonding
     options bonding mode=1 arp_interval=100 arp_ip_target=192.168.0.254

2) Run the following command on dom0.
   # brctl addbr xenbr0
   # ip link set xenbr0 arp off
   # ip link set xenbr0 multicast off

   # ip link set eth0 name peth0
   # ip link set eth1 name peth1
   # ip link set veth0 name eth0
   # ip link set veth1 name eth1

   # ifup bond0
   # ifenslave bond0 peth0 peth1
   # ifdown bond0

   # ip link set bond0 down
   # ip link set bond0 arp off
   # ip link set bond0 multicast off
   # ip link set bond0 addr fe:ff:ff:ff:ff:ff

   # ip link set vif0.0 down
   # ip link set vif0.0 arp off
   # ip link set vif0.0 multicast off
   # ip link set vif0.0 addr fe:ff:ff:ff:ff:ff

   # ip link set xenbr0 up
   # brctl addif xenbr0 bond0
   # ip link set bond0 up
   # brctl addif xenbr0 vif0.0
   # ip link set vif0.0 up

   # ip link set eth0 addr 00:16:3e:70:00:03
   # ip link set eth0 arp on
   # ifup eth0
   # ifconfig eth0 192.168.0.1