Re: bridging: flow control regression

[Eric Dumazet <eric.dumazet@gmail.com>]

Le lundi 01 novembre 2010 à 21:29 +0900, Simon Horman a écrit :
> Hi,
> 
> I have observed what appears to be a regression between 2.6.34 and
> 2.6.35-rc1. The behaviour described below is still present in Linus's
> current tree (2.6.36+).
> 
> On 2.6.34 and earlier when sending a UDP stream to a bonded interface
> the throughput is approximately equal to the available physical bandwidth.
> 
> # netperf -c -4 -t UDP_STREAM -H 172.17.50.253 -l 30 -- -m 1472
> UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 172.17.50.253 (172.17.50.253) port 0 AF_INET
> Socket  Message  Elapsed      Messages                   CPU      Service
> Size    Size     Time         Okay Errors   Throughput   Util     Demand
> bytes   bytes    secs            #      #   10^6bits/sec % SU     us/KB
> 
> 114688    1472   30.00     2438265      0      957.1     18.09    3.159 
> 109568           30.00     2389980             938.1     -1.00    -1.000
> 
> On 2.6.35-rc1 netpref sends~7Gbits/s.
> Curiously it only consumes 50% CPU, I would expect this to be CPU bound.
> 
> # netperf -c -4 -t UDP_STREAM -H 172.17.50.253 -l 30 -- -m 1472
> UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 172.17.50.253 (172.17.50.253) port 0 AF_INET
> Socket  Message  Elapsed      Messages                   CPU      Service
> Size    Size     Time         Okay Errors   Throughput   Util     Demand
> bytes   bytes    secs            #      #   10^6bits/sec % SU     us/KB
> 
> 116736    1472   30.00     18064360      0     7090.8     50.62    8.665 
> 109568           30.00     2438090             957.0     -1.00    -1.000
> 
> In this case the bonding device has a single gitabit slave device
> and is running in balance-rr mode. I have observed similar results
> with two and three slave devices.
> 
> I have bisected the problem and the offending commit appears to be
> "net: Introduce skb_orphan_try()". My tired eyes tell me that change
> frees skb's earlier than they otherwise would be unless tx timestamping
> is in effect. That does seem to make sense in relation to this problem,
> though I am yet to dig into specifically why bonding is adversely affected.
> 

I assume you meant "bonding: flow control regression", ie this is not
related to bridging ?

One problem on bonding is that the xmit() method always returns
NETDEV_TX_OK.

So a flooder cannot know some of its frames were lost.

So yes, the patch you mention has the effect of allowing UDP to flood
bonding device, since we orphan skb before giving it to device (bond or
ethX)

With a normal device (with a qdisc), we queue skb, and orphan it only
when leaving queue. With a not too big socket send buffer, it slows down
the sender enough to "send UDP frames at line rate only"