Re: 802.3ad bonding brain damaged?

[Rick Jones <rick.jones2@hp.com>]

On 08/08/2011 01:06 PM, Phillip Susi wrote:
> On 8/8/2011 3:57 AM, David Lamparter wrote:
>> No, it isn't. 802.3ad/.1AX explicitly requires that no packet
>> re-ordering may ever occur, which can only be guaranteed by enqueueing
>> packets for one host on one TX interface. This behaviour is mandated by
>> 802.1AX-2008 page 15 which reads:
>
> Outch, that does cause a big problem for store-and-forward switching.
> You basically can't split up packets from a single stream without very
> careful cut-through switching, which we obviously can't do in Linux.
> That seems a rather silly requirement given that higher level protocols
> already deal with packet reordering. Why not an option to say stuff the
> standard?


At even in the case of protocols that deal with packet reordering, it is 
still quite possible to be sub-optimal.  Try running a TCP_STREAM test 
through a mode-rr bond with 4 or more links in it.  I suspect that even 
without injecting the occasional "other" packet there can be enough 
re-ordering to trigger spurious fast retransmissions.  At the very least 
it will trigger lots of immediate ACKnowledgements, which will drive-up 
the CPU utilization per KB transferred.  And if these spread packets 
arrive still spread at the receiver, round-robin will probably preclude 
effective GRO and certainly preclude LRO.

Apart from some very carefully controlled conditions, if one needs a 
single flow to go faster than a single link, it is probably time to move 
up to the next higher link speed.

rick jones