Bonding/LACP on RTL8169sc/8110sc (R1869)

Bonding/LACP on RTL8169sc/8110sc (R1869)

[Jonathan Thibault]

Greetings all,

I have a a pair of Jetway motherboards with an add-on 3Gbit LAN modules
and have been doing some testing for a linux router project.  I found
that I while I can get LACP working properly using eth0 and eth1, using
the exact same setup with any of the other three lan fails.  Is this a
known issue?

This is on Linux 2.6.32.10.  Here is a blurb of dmesg showing the NICs
detected.

eth0: RTL8168b/8111b at 0xf8adc000, 00:30:18:ac:a6:80, XID 0c200000 IRQ 24
eth1: RTL8168b/8111b at 0xf8ae0000, 00:30:18:ac:a6:81, XID 0c200000 IRQ 25
eth2: RTL8169sc/8110sc at 0xf8ae4c00, 00:30:18:ae:34:3a, XID 18000000 IRQ 18
eth3: RTL8169sc/8110sc at 0xf8ae8800, 00:30:18:ae:34:3b, XID 18000000 IRQ 19
eth4: RTL8169sc/8110sc at 0xf8aec400, 00:30:18:ae:34:3c, XID 18000000 IRQ 16

I can probably manage this project using only eth0 and eth1 in LACP
configuration but I figured I'd give a heads up.

The interfaces seem able to detect when the link is up or down, bonding
removes and adds them to the LACP trunk but I can't get traffic through
them.

Kind regards,

Jonathan

Re: Bonding/LACP on RTL8169sc/8110sc (R1869)

[Jay Vosburgh]

Jonathan Thibault <jonathan@navigue.com> wrote:
>I have a a pair of Jetway motherboards with an add-on 3Gbit LAN modules
>and have been doing some testing for a linux router project.  I found
>that I while I can get LACP working properly using eth0 and eth1, using
>the exact same setup with any of the other three lan fails.  Is this a
>known issue?

	Fails how, exactly?  If eth2/3/4 is part of the bond, then no
aggregation forms at all, or those devices don't join, or what?

>This is on Linux 2.6.32.10.  Here is a blurb of dmesg showing the NICs
>detected.

	That's a fairly old kernel; one relatively recent fix that comes
to mind is:

commit ab12811c89e88f2e66746790b1fe4469ccb7bdd9
Author: Andy Gospodarek <andy@greyhouse.net>
Date:   Fri Sep 10 11:43:20 2010 +0000

    bonding: correctly process non-linear skbs
    
    It was recently brought to my attention that 802.3ad mode bonds would no
    longer form when using some network hardware after a driver update.
    After snooping around I realized that the particular hardware was using
    page-based skbs and found that skb->data did not contain a valid LACPDU
    as it was not stored there.  That explained the inability to form an
    802.3ad-based bond.  For balance-alb mode bonds this was also an issue
    as ARPs would not be properly processed.

>eth0: RTL8168b/8111b at 0xf8adc000, 00:30:18:ac:a6:80, XID 0c200000 IRQ 24
>eth1: RTL8168b/8111b at 0xf8ae0000, 00:30:18:ac:a6:81, XID 0c200000 IRQ 25
>eth2: RTL8169sc/8110sc at 0xf8ae4c00, 00:30:18:ae:34:3a, XID 18000000 IRQ 18
>eth3: RTL8169sc/8110sc at 0xf8ae8800, 00:30:18:ae:34:3b, XID 18000000 IRQ 19
>eth4: RTL8169sc/8110sc at 0xf8aec400, 00:30:18:ae:34:3c, XID 18000000 IRQ 16

	The fix might apply if your eth2/3/4 hardware uses page based
skbs as mentioned in this commit.

	Also, what is your network topology?  Are all the devices
connected to the same switch?

>I can probably manage this project using only eth0 and eth1 in LACP
>configuration but I figured I'd give a heads up.
>
>The interfaces seem able to detect when the link is up or down, bonding
>removes and adds them to the LACP trunk but I can't get traffic through
>them.

	If the above commit doesn't resolve the problem, can you post
some sample output of /proc/net/bonding/bondX (with the appropriate
value for "X") when you've got the "bad" devices in the bond, along with
a description of exactly what doesn't work?

	-J

---
	-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com

Re: Bonding/LACP on RTL8169sc/8110sc (R1869)

[François Romieu]

On Tue, Apr 12, 2011 at 03:47:45PM -0700, Jay Vosburgh wrote:
> Jonathan Thibault <jonathan@navigue.com> wrote:
[2.6.32.old]
> 	That's a fairly old kernel; one relatively recent fix that comes
> to mind is:

The r8169 driver has undergone some multicast fixes since 2.6.32 as well.

If I remember correctly, the old 8169 - not the 8168 - was hit.

Upgrading is really suggested.

-- 
Ueimor

Re: Bonding/LACP on RTL8169sc/8110sc (R1869)

[Ben Hutchings]

On Wed, 2011-04-13 at 19:49 +0200, François Romieu wrote:
> On Tue, Apr 12, 2011 at 03:47:45PM -0700, Jay Vosburgh wrote:
> > Jonathan Thibault <jonathan@navigue.com> wrote:
> [2.6.32.old]
> > 	That's a fairly old kernel; one relatively recent fix that comes
> > to mind is:
> 
> The r8169 driver has undergone some multicast fixes since 2.6.32 as well.
> 
> If I remember correctly, the old 8169 - not the 8168 - was hit.
> 
> Upgrading is really suggested.

If you mean the changes to register writes (commits
78f1cd02457252e1ffbc6caa44a17424a45286b8 and
908ba2bfd22253f26fa910cd855e4ccffb1467d0), those went into 2.6.32.y.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.