stable bonding config

stable bonding config

[Simon Chen]

Hi all,

Not sure if this is the right place to ask, but I got very good
answers about this topic in the past...

I am mostly using Ubuntu 10.10. Each server has dual intel 10G NICs,
which I want to bond. I configured the switches to support 802.3ad.
Based on this post http://ubuntuforums.org/showthread.php?t=1595177, I
configure the server as:

/home/simon# cat /etc/modprobe.d/aliases.conf
alias bond0 bonding
options bonding mode=4 miimon=100 lacp_rate=1 downdelay=200
updelay=200 ad_select=1 xmit_hash_policy=1

/home/simon# cat /etc/network/interfaces
...
auto bond0
iface bond0 inet manual
        bond-slaves none
        bond-mode 4
        bond-miimon 100
        up /sbin/ifenslave bond0 eth2 eth3
        down /sbin/ifenslave bond0 eth2 eth3

auto eth2
iface eth2 inet manual
bond-master bond0
bond-primary eth2 eth3

auto eth3
iface eth3 inet manual
bond-master bond0
bond-primary eth2 eth3

auto bond0.200
iface bond0.200 inet static
  address ...
  vlan_raw_device bond0
==========================

However, after a reboot the bonding doesn't work immediately. In particular:

/home/simon# cat /proc/net/bonding/bond0
...
802.3ad info
LACP rate: fast
Aggregator selection policy (ad_select): bandwidth
Active Aggregator Info:
        Aggregator ID: 3
        Number of ports: 2
        Actor Key: 33
        Partner Key: 19
        Partner Mac Address: 00:00:00:00:00:00 <------bonding not working
======================================

The only magic to get it work that I know of, is to perform "rmmod
ixgbe" "modprobe ixgbe" and then "ifconfig eth2 0.0.0.0 up" "ifconfig
eth3 0.0.0.0 up" "ifdown bond0" "ifup bond0".

Is there a hassle-free bonding configuration that I can use? Or some
trick that I am not aware of?

Thanks.
-Simon

Re: stable bonding config

[Simon Chen]

I am using the following:

Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
ixgbe 3.7.17 driver (latest from intel)

Thanks.
-Simon

On Thu, Jan 12, 2012 at 9:31 PM, Simon Chen <simonchennj@gmail.com> wrote:
> Hi all,
>
> Not sure if this is the right place to ask, but I got very good
> answers about this topic in the past...
>
> I am mostly using Ubuntu 10.10. Each server has dual intel 10G NICs,
> which I want to bond. I configured the switches to support 802.3ad.
> Based on this post http://ubuntuforums.org/showthread.php?t=1595177, I
> configure the server as:
>
> /home/simon# cat /etc/modprobe.d/aliases.conf
> alias bond0 bonding
> options bonding mode=4 miimon=100 lacp_rate=1 downdelay=200
> updelay=200 ad_select=1 xmit_hash_policy=1
>
> /home/simon# cat /etc/network/interfaces
> ...
> auto bond0
> iface bond0 inet manual
>        bond-slaves none
>        bond-mode 4
>        bond-miimon 100
>        up /sbin/ifenslave bond0 eth2 eth3
>        down /sbin/ifenslave bond0 eth2 eth3
>
> auto eth2
> iface eth2 inet manual
> bond-master bond0
> bond-primary eth2 eth3
>
> auto eth3
> iface eth3 inet manual
> bond-master bond0
> bond-primary eth2 eth3
>
> auto bond0.200
> iface bond0.200 inet static
>  address ...
>  vlan_raw_device bond0
> ==========================
>
> However, after a reboot the bonding doesn't work immediately. In particular:
>
> /home/simon# cat /proc/net/bonding/bond0
> ...
> 802.3ad info
> LACP rate: fast
> Aggregator selection policy (ad_select): bandwidth
> Active Aggregator Info:
>        Aggregator ID: 3
>        Number of ports: 2
>        Actor Key: 33
>        Partner Key: 19
>        Partner Mac Address: 00:00:00:00:00:00 <------bonding not working
> ======================================
>
> The only magic to get it work that I know of, is to perform "rmmod
> ixgbe" "modprobe ixgbe" and then "ifconfig eth2 0.0.0.0 up" "ifconfig
> eth3 0.0.0.0 up" "ifdown bond0" "ifup bond0".
>
> Is there a hassle-free bonding configuration that I can use? Or some
> trick that I am not aware of?
>
> Thanks.
> -Simon

Re: stable bonding config

[Andy Gospodarek]

On Thu, Jan 12, 2012 at 09:52:25PM -0500, Simon Chen wrote:
> I am using the following:
> 
> Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
> ixgbe 3.7.17 driver (latest from intel)
> 
> Thanks.
> -Simon
> 

Use the module option to disable RSC and I think it will work.

> On Thu, Jan 12, 2012 at 9:31 PM, Simon Chen <simonchennj@gmail.com> wrote:
> > Hi all,
> >
> > Not sure if this is the right place to ask, but I got very good
> > answers about this topic in the past...
> >
> > I am mostly using Ubuntu 10.10. Each server has dual intel 10G NICs,
> > which I want to bond. I configured the switches to support 802.3ad.
> > Based on this post http://ubuntuforums.org/showthread.php?t=1595177, I
> > configure the server as:
> >
> > /home/simon# cat /etc/modprobe.d/aliases.conf
> > alias bond0 bonding
> > options bonding mode=4 miimon=100 lacp_rate=1 downdelay=200
> > updelay=200 ad_select=1 xmit_hash_policy=1
> >
> > /home/simon# cat /etc/network/interfaces
> > ...
> > auto bond0
> > iface bond0 inet manual
> >        bond-slaves none
> >        bond-mode 4
> >        bond-miimon 100
> >        up /sbin/ifenslave bond0 eth2 eth3
> >        down /sbin/ifenslave bond0 eth2 eth3
> >
> > auto eth2
> > iface eth2 inet manual
> > bond-master bond0
> > bond-primary eth2 eth3
> >
> > auto eth3
> > iface eth3 inet manual
> > bond-master bond0
> > bond-primary eth2 eth3
> >
> > auto bond0.200
> > iface bond0.200 inet static
> >  address ...
> >  vlan_raw_device bond0
> > ==========================
> >
> > However, after a reboot the bonding doesn't work immediately. In particular:
> >
> > /home/simon# cat /proc/net/bonding/bond0
> > ...
> > 802.3ad info
> > LACP rate: fast
> > Aggregator selection policy (ad_select): bandwidth
> > Active Aggregator Info:
> >        Aggregator ID: 3
> >        Number of ports: 2
> >        Actor Key: 33
> >        Partner Key: 19
> >        Partner Mac Address: 00:00:00:00:00:00 <------bonding not working
> > ======================================
> >
> > The only magic to get it work that I know of, is to perform "rmmod
> > ixgbe" "modprobe ixgbe" and then "ifconfig eth2 0.0.0.0 up" "ifconfig
> > eth3 0.0.0.0 up" "ifdown bond0" "ifup bond0".
> >
> > Is there a hassle-free bonding configuration that I can use? Or some
> > trick that I am not aware of?
> >
> > Thanks.
> > -Simon
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stable bonding config

[Nicolas de Pesloüan]

Le 13/01/2012 03:31, Simon Chen a écrit :
> Hi all,
>
> Not sure if this is the right place to ask, but I got very good
> answers about this topic in the past...
>
> I am mostly using Ubuntu 10.10. Each server has dual intel 10G NICs,
> which I want to bond. I configured the switches to support 802.3ad.
> Based on this post http://ubuntuforums.org/showthread.php?t=1595177, I
> configure the server as:

[snip]

Consider reading the documentations in /usr/share/doc/ifenslave-2.6/ and the example files in the 
examples subdirectory.

	Nicolas.