Re: bonding: flow control regression [was Re: bridging: flow control regression]

[Eric Dumazet <eric.dumazet@gmail.com>]

Le mercredi 08 décembre 2010 à 22:22 +0900, Simon Horman a écrit :

> Hi Eric,
> 
> do you have any thoughts on this?
> 
> I measured the performance impact of your patch on 2.6.37-rc1
> and I can see why early orphaning is a win.
> 
> The tests are run over a bond with 3 slaves.
> The bond is in rr-balance mode. Other parameters of interest are:
> 	MTU=1500
> 	client,server:tcp_reordering=3(default)
> 	client:GSO=off,
> 	client:TSO=off
> 	server:GRO=off
> 	server:rx-usecs=3(default)
> 
> Without your no early-orphan patch
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 	172.17.60.216 (172.17.60.216) port 0 AF_INET
> Recv   Send    Send                          Utilization       Service Demand
> Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
> Size   Size    Size     Time     Throughput  local    remote   local   remote
> bytes  bytes   bytes    secs.    10^6bits/s  % S      % U      us/KB   us/KB
> 
>  87380  16384  16384    10.00      1621.03   16.31    6.48     1.648   2.621
> 
> With your no early-orphan patch
> # netperf -C -c -4 -t TCP_STREAM -H 172.17.60.216
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to
> 	172.17.60.216 (172.17.60.216) port 0 AF_INET
> Recv   Send    Send                          Utilization       Service Demand
> Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
> Size   Size    Size     Time     Throughput  local    remote   local   remote
> bytes  bytes   bytes    secs.    10^6bits/s  % S      % U      us/KB   us/KB
> 
>  87380  16384  16384    10.00      1433.48   9.60     5.45     1.098   2.490
> 

It seems strange this makes such big difference with one flow


> 
> However in the case of virtualisation I think it is a win to be able to do
> flow control on UDP traffic from guests (using vitio). Am I missing
> something and flow control can be bypassed anyway? If not perhaps making
> the change that your patch makes configurable through proc or ethtool is an
> option?
> 

virtio_net start_xmit() does one skb_orphan() anyway, so not doing it
some nano seconds before wont change anything.

Real perf problem is when skb are queued (for example on eth driver TX
ring or qdisc queue), then freed some micro (or milli) seconds later.

Maybe your ethtool suggestion is the way to go, so that we can remove
special "skb_orphans()" that can be done in some drivers : Let core
network stack decide to skb_orphan() itself, not the driver.