[question] bonding: should assert dormant for active protocols like LACP?

[question] bonding: should assert dormant for active protocols like LACP?

[Jonathan Toppins]

So running the following script:

--%<-----
  ip link add name link-bond0 type veth peer name link-end0
  ip link add bond0 type bond mode 4 miimon 100
  ip link set link-bond0 master bond0 down
  ip netns add n1
  ip link set link-end0 netns n1 up
  ip link set bond0 up
  cat /sys/class/net/bond0/bonding/ad_partner_mac
  cat /sys/class/net/bond0/operstate
--%<-----

The bond reports its operstate to be "up" even though the bond will 
never be able to establish an LACP partner. Should bonding for active 
protocols, LACP, assert dormant[0] until the protocol has established 
and frames actually are passed?

Having a predictable operstate where up actually means frames will 
attempt to be delivered would make management applications, f.e. Network 
Manager, easier to write. I have developers asking me what detailed 
states for LACP should they be looking for to determine when an LACP 
bond is "up". This seems like an incorrect implementation of operstate 
and RFC2863 3.1.12.

Does anyone see why this would be a bad idea?

-Jon

[0] Documentation/networking/operstates.rst

Re: [question] bonding: should assert dormant for active protocols like LACP?

[Jay Vosburgh]

Jonathan Toppins <jtoppins@redhat.com> wrote:

>So running the following script:
>
>--%<-----
> ip link add name link-bond0 type veth peer name link-end0
> ip link add bond0 type bond mode 4 miimon 100
> ip link set link-bond0 master bond0 down
> ip netns add n1
> ip link set link-end0 netns n1 up
> ip link set bond0 up
> cat /sys/class/net/bond0/bonding/ad_partner_mac
> cat /sys/class/net/bond0/operstate
>--%<-----
>
>The bond reports its operstate to be "up" even though the bond will never
>be able to establish an LACP partner. Should bonding for active protocols,
>LACP, assert dormant[0] until the protocol has established and frames
>actually are passed?
>
>Having a predictable operstate where up actually means frames will attempt
>to be delivered would make management applications, f.e. Network Manager,
>easier to write. I have developers asking me what detailed states for LACP
>should they be looking for to determine when an LACP bond is "up". This
>seems like an incorrect implementation of operstate and RFC2863 3.1.12.
>
>Does anyone see why this would be a bad idea?

	The catch with LACP is that it has a fallback, in that ports
that don't complete LACP negotiation go to "Solitary" state (I believe
this was called "Individual" in older versions of the 802.1AX / 802.3ad
standard; bonding calls this "is_individual" internally).

	If there is no suitable partnered port, then a Solitary port is
made active.  This permits connectivity if one end is set for LACP but
the other end is not (e.g., PXE boot to a switch port set for LACP).
For reference, I'm looking at 6.3.5 and 6.3.6 of IEEE 802.1AX-2020.

	So, how should operstate be set if "has LACP partner" isn't
really the test for whether or not the interface is (to use RCC 2863
language) "in a condition to pass packets"?  In your example above, I
believe the bond should be able to pass packets just fine, the packets
just won't go anywhere after they leave the bond.

	-J

>-Jon
>
>[0] Documentation/networking/operstates.rst
>

---
	-Jay Vosburgh, jay.vosburgh@canonical.com

Re: [question] bonding: should assert dormant for active protocols like LACP?

[Jonathan Toppins]

On 5/17/22 17:49, Jay Vosburgh wrote:
> Jonathan Toppins <jtoppins@redhat.com> wrote:
> 
>> So running the following script:
>>
>> --%<-----
>> ip link add name link-bond0 type veth peer name link-end0
>> ip link add bond0 type bond mode 4 miimon 100
>> ip link set link-bond0 master bond0 down
>> ip netns add n1
>> ip link set link-end0 netns n1 up
>> ip link set bond0 up
>> cat /sys/class/net/bond0/bonding/ad_partner_mac
>> cat /sys/class/net/bond0/operstate
>> --%<-----
>>
>> The bond reports its operstate to be "up" even though the bond will never
>> be able to establish an LACP partner. Should bonding for active protocols,
>> LACP, assert dormant[0] until the protocol has established and frames
>> actually are passed?
>>
>> Having a predictable operstate where up actually means frames will attempt
>> to be delivered would make management applications, f.e. Network Manager,
>> easier to write. I have developers asking me what detailed states for LACP
>> should they be looking for to determine when an LACP bond is "up". This
>> seems like an incorrect implementation of operstate and RFC2863 3.1.12.
>>
>> Does anyone see why this would be a bad idea?
> 
> 	The catch with LACP is that it has a fallback, in that ports
> that don't complete LACP negotiation go to "Solitary" state (I believe
> this was called "Individual" in older versions of the 802.1AX / 802.3ad
> standard; bonding calls this "is_individual" internally).
> 
> 	If there is no suitable partnered port, then a Solitary port is
> made active.  This permits connectivity if one end is set for LACP but
> the other end is not (e.g., PXE boot to a switch port set for LACP).
> For reference, I'm looking at 6.3.5 and 6.3.6 of IEEE 802.1AX-2020.
> 
> 	So, how should operstate be set if "has LACP partner" isn't
> really the test for whether or not the interface is (to use RCC 2863
> language) "in a condition to pass packets"?  In your example above, I
> believe the bond should be able to pass packets just fine, the packets
> just won't go anywhere after they leave the bond.

That makes sense and appears to work:

--%<------
sh -c 'ip link add name link-bond0 type veth peer name link-end0
ip link add bond0 type bond mode 4 miimon 100
ip link set link-bond0 master bond0 down
ip netns add n1
ip link set link-end0 netns n1 up
ip netns exec n1 ip link add name br0 type bridge
ip netns exec n1 ip link set br0 up
ip netns exec n1 ip link set link-end0 master br0
ip addr add 192.168.180.2/24 dev bond0
ip netns exec n1 ip addr add 192.168.180.3/24 dev br0
ip link set bond0 up
cat /sys/class/net/bond0/bonding/ad_partner_mac
cat /sys/class/net/bond0/operstate
sleep 20
ping -I bond0 -c8 192.168.180.3'
--%<------
00:00:00:00:00:00
up
PING 192.168.180.3 (192.168.180.3) from 192.168.180.2 bond0: 56(84) 
bytes of data.
64 bytes from 192.168.180.3: icmp_seq=1 ttl=64 time=0.065 ms
64 bytes from 192.168.180.3: icmp_seq=2 ttl=64 time=0.059 ms
64 bytes from 192.168.180.3: icmp_seq=3 ttl=64 time=0.046 ms
64 bytes from 192.168.180.3: icmp_seq=4 ttl=64 time=0.046 ms
64 bytes from 192.168.180.3: icmp_seq=5 ttl=64 time=0.045 ms
64 bytes from 192.168.180.3: icmp_seq=6 ttl=64 time=0.045 ms
64 bytes from 192.168.180.3: icmp_seq=7 ttl=64 time=0.044 ms
64 bytes from 192.168.180.3: icmp_seq=8 ttl=64 time=0.042 ms

--- 192.168.180.3 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7191ms
rtt min/avg/max/mdev = 0.042/0.049/0.065/0.007 ms
[root@fedora ~]# uname -a
Linux fedora 5.17.5-200.fc35.x86_64 #1 SMP PREEMPT Thu Apr 28 15:41:41 
UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Time to go ask more questions of these developers.

Thanks,
-Jon