Re: igb + balance-rr + bridge + IPv6 = no go without promiscuous mode

["Nicolas de Pesloüan" <nicolas.2p.debian@gmail.com>]

[ Forwarded to netdev, because two previous e-mail erroneously sent in HTML ]

Le 23/12/2011 11:15, Chris Boot a écrit :
> On 23/12/2011 09:52, Nicolas de Pesloüan wrote:
>>
>>
>> Le 23 déc. 2011 10:42, "Chris Boot" <bootc@bootc.net <mailto:bootc@bootc.net>> a écrit :
>> >
>> > Hi folks,
>> >
>> > As per Eric Dumazet and Dave Miller, I'm opening up a separate thread on this issue.
>> >
>> > I have two identical servers in a cluster for running KVM virtual machines. They each have a
>> single connection to the Internet (irrelevant for this) and two gigabit connections between each
>> other for cluster replication, etc... These two connections are in a balance-rr bonded connection,
>> which is itself member of a bridge that the VMs attach to. I'm running v3.2-rc6-140-gb9e26df on
>> Debian Wheezy.
>> >
>> > When the bridge is brought up, IPv4 works fine but IPv6 does not. I can use neither the
>> automatic link-local on the brid ge nor the static global address I assign. Neither machine can
>> perform neighbour discovery over the link until I put the bond members (eth0 and eth1) into
>> promiscuous mode.  I can do this either with tcpdump or 'ip link set dev ethX promisc on' and this
>> is enough to make the link spring to life.
>>
>> For as far as I remember, setting bond0 to promisc should set the bonding member to promisc too.
>> And inserting bond0 into br0 should set bond0 to promisc... So everything should be in promisc
>> mode anyway... but you shoudn't have to do it by hand.
>>
>
> Sorry, I should have added that I tried this. Setting bond0 or br0 to promisc has no effect. I
> discovered this by running tcpdump on br0 first, then bond0, then eventually each bond member in
> turn. Only at the last stage did things jump to life.
>
>> >
>> > This cluster is not currently live so I can easily test patches and various configurations.
>>
>> Can you try to remove the bonding part, connecting eth0 and eth1 directly to br0 and see if it
>> works better? (This is a test ony. I perfectly understand that you would loose balance-rr in this
>> setup.)
>>
>
> Good call. Let's see.
>
> I took br0 and bond0 apart, took eth0 and eth1 out of enforced promisc mode, then manually built a
> br0 with eth0 in only so I didn't cause a network loop. Adding eth0 to br0 did not make it go into
> promisc mode, but IPv6 does work over this setup. I also made sure ip -6 neigh was empty on both
> machines before I started.
>
> I then decided to try the test with just the bond0 in balance-rr mode. Once again I took everything
> down and ensured no promisc mode and no ip -6 neigh. I noticed bond0 wasn't getting a link-local and
> I found out for some reason /proc/sys/net/ipv6/conf/bond0/disable_ipv6 was set on both servers so I
> set it to 0. That brought things to life.
>
> So then I put it all back together again and it didn't work. I once again noticed disable_ipv6 was
> set on the bond0 interfaces, now part of the bridge. Toggling this on the _bond_ interface made
> things work again.
>
> What's setting disable_ipv6? Should this be having an impact if the port is part of a bridge?
>
> Chris
>
> --
> Chris Boot
> bootc@bootc.net
>