Cascading Bond devices

Cascading Bond devices

[Thomas Glanzmann]

Hello,
I wonder if it is possible to cascade bond devices. I have currently a
box with four network cards. Two are going to switch a and two are going
to switchb. And what I would like to do is the following:

switch-01 port 1 - eth0 \
switch-01 port 2 - eth1 - bond0 (802.3ad Layer 2 hash) \
switch-02 port 1 - eth2 - bond1 (802.3ad Layer 2 hash) - bond2 (active-passive)
switch-02 port 2 - eth3 /

If it is possible, how would I configure such a scenario? bond0 and
bond1 is easy and in fact I already have them. But I don't get howto to
use adifferent hashing algorithm for bond2.

I also wanted to use tlb/alb in the past but experienced broadcast
storms when using IPv6 on such devices. Also I was not able to set an
IPv6 address because of duplicated address detection told me that it was
already online on the subnet which it was not. This was with kernel 3.12
and 3.13.

Cheers,
        Thomas

Re: Cascading Bond devices

[Jay Vosburgh]

Thomas Glanzmann <thomas@glanzmann.de> wrote:

>I wonder if it is possible to cascade bond devices. I have currently a
>box with four network cards. Two are going to switch a and two are going
>to switchb. And what I would like to do is the following:
>
>switch-01 port 1 - eth0 \
>switch-01 port 2 - eth1 - bond0 (802.3ad Layer 2 hash) \
>switch-02 port 1 - eth2 - bond1 (802.3ad Layer 2 hash) - bond2 (active-passive)
>switch-02 port 2 - eth3 /

	For this specific case, it is unnecessary to nest the bonds, as
one feature of 802.3ad is that it will group the ports correctly into
multiple aggregators, and fail over from one to the other under certain
circumstances selected by the ad_select parameter.  It is not possible
to select different hash algorithms on a per-aggregator basis, though.

>If it is possible, how would I configure such a scenario? bond0 and
>bond1 is easy and in fact I already have them. But I don't get howto to
>use adifferent hashing algorithm for bond2.

	Generally speaking, nesting of bonds does not function
correctly.  It is possible to configure, but various parts do not
function as expected from the nesting (when last I checked, transmit
generally functioned, but receive generally did not).

>I also wanted to use tlb/alb in the past but experienced broadcast
>storms when using IPv6 on such devices. Also I was not able to set an
>IPv6 address because of duplicated address detection told me that it was
>already online on the subnet which it was not. This was with kernel 3.12
>and 3.13.

	With a nesting, or just in general?  I have not tested alb/tlb
modes much with IPv6, but the remote peer load balancing scheme for alb
mode in particular is only implemented for IPv4.  I do recall that both
of them should balance outgoing traffic for IPv6.

	As far as storms go, normally for tlb/alb, one slave is the
"active" slave, and for both alb and tlb should be the only slave that
receives broadcasts or multicasts.  For tlb mode, the active slave is
the only slave that receives anything at all, the others are
transmit-only.  For alb mode, the other slaves receive unicast traffic
from network peers according to the balance algorithm issuing special
ARP frames to those peers to direct their traffic (this is the IPv4-only
part).

	I have not tested tlb/alb with very recent kernels, so it's
possible that something has been broken in some of the substantial
changes over the last few months.

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: Cascading Bond devices

[Thomas Glanzmann]

Hello Jay,

> For this specific case, it is unnecessary to nest the bonds, as one
> feature of 802.3ad is that it will group the ports correctly into
> multiple aggregators, and fail over from one to the other under
> certain circumstances selected by the ad_select parameter.  It is not
> possible to select different hash algorithms on a per-aggregator
> basis, though.

perfect. Thank you a lot for clarifying. I'll reconfigure my system
tongiht accordingly.

> Generally speaking, nesting of bonds does not function correctly.  It
> is possible to configure, but various parts do not function as
> expected from the nesting (when last I checked, transmit generally
> functioned, but receive generally did not).

I see.

> > Trouble with tlb with IPv4 and IPv6.

> With a nesting, or just in general?

In general. To be precise I had the following scenario:

switch-1 port-1 \
switch-1 port-2 \
switch-2 port-1 - bond0 (tlb)
switch-2 port-2 /

But at the same time I had tagged and untagged vlans on the bond
Interface. What always for me happened is that I was unable to set an
IPv6 address on the bond. This was on 3.2.0-4 which ships with Debian
Wheezy (7). So what I did is I started the bond with one interface, set
the IPv6 interface and enslaved another interface.

This worked perfectly fine for weeks until one morning at 05:00 am my
provider took my interface down because of broadcast storms. There were
two systems on that particular broadcast domain. The otherone had just
one network card and nothing was going on there, so I assume it was the
system with the bonding, but I never actually verified it.

The second problem I had is in order to avoid not be able to set the
IPv6 address on the bond I tried to disable 'dad', so I did that and it
worked perfectly fine, until I rebooted. And immediately my provider
took down the interface again because of broadcast storms. So I disabled
it and never looked back.

The Juniper router of my provider was complaining about:

storm control in effect on the port

So what I will do is in a lab system which has not access to the
upstream switch, I'll configure tlb with IPv6 as described as above and
let you know the results and provide you with pcaps if necessary.
Hopefully we can figure out what is going wrong and fix it.

> I have not tested alb/tlb modes much with IPv6, but the remote peer
> load balancing scheme for alb mode in particular is only implemented
> for IPv4.  I do recall that both of them should balance outgoing
> traffic for IPv6.

I see. TLB would fit my scenario perfectly: The system in question is an
install server for 60 servers which are installed concurrently on a
weekly basis over Gbit. So having four 1 GBIT links help especially as
the system in question is the default gateway so that it actually can
use the bandwidth and is not limited by router in between.

> As far as storms go, normally for tlb/alb, one slave is the "active"
> slave, and for both alb and tlb should be the only slave that receives
> broadcasts or multicasts. For tlb mode, the active slave is the only
> slave that receives anything at all, the others are transmit-only. For
> alb mode, the other slaves receive unicast traffic from network peers
> according to the balance algorithm issuing special ARP frames to those
> peers to direct their traffic (this is the IPv4-only part).

I see.

> I have not tested tlb/alb with very recent kernels, so it's possible
> that something has been broken in some of the substantial changes over
> the last few months.

I'll do some more testing and as soon as I have something substantial,
I'll report back with a problem or with a good to go. :-)

Thanks again for the clarification.

Cheers,
        Thomas

Re: Cascading Bond devices

[Ding Tianhong]

On 2014/2/7 15:41, Thomas Glanzmann wrote:
> Hello,
> I wonder if it is possible to cascade bond devices. I have currently a
> box with four network cards. Two are going to switch a and two are going
> to switchb. And what I would like to do is the following:
> 
> switch-01 port 1 - eth0 \
> switch-01 port 2 - eth1 - bond0 (802.3ad Layer 2 hash) \
> switch-02 port 1 - eth2 - bond1 (802.3ad Layer 2 hash) - bond2 (active-passive)
> switch-02 port 2 - eth3 /
> 
> If it is possible, how would I configure such a scenario? bond0 and
> bond1 is easy and in fact I already have them. But I don't get howto to
> use adifferent hashing algorithm for bond2.
> 

Hi Thomas:

bond2 (active-passive)？ if you mean active-backup,
I think this situation will work well, and I test it by two card:

 switch-01 port 2 - eth1 - bond0 (802.3ad Layer 2 hash) \
 switch-02 port 1 - eth2 - bond1 (802.3ad Layer 2 hash) - bond2 (active-backup) 

the bond2 could work.

> I also wanted to use tlb/alb in the past but experienced broadcast
> storms when using IPv6 on such devices. Also I was not able to set an
> IPv6 address because of duplicated address detection told me that it was
> already online on the subnet which it was not. This was with kernel 3.12
> and 3.13.
> 

I think you have to reset ipv6 address yourself.

Ding

> Cheers,
>         Thomas
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
>