Re: limited network bandwidth with 3.2.x kernels

[Ben Hutchings <bhutchings@solarflare.com>]

On Thu, 2012-02-16 at 14:40 +0100, Eric Dumazet wrote:
> Le jeudi 16 février 2012 à 08:29 +0100, Eric Dumazet a écrit :
> > So I took the time to setup a netem in my lab (and had to fix netem by
> > the way). Gigabit link.
> > 
> > On sender :
> > tc qdisc add dev vlan.103 root netem delay 50ms
> > 
> > And netperf session is a bit strange, since receiver window is about
> > 1Mbytes (17097*64) but sender never uses half of it.
> > 
> > cwnd is ~408
> > 
> > $ ss -emoi  dst 192.168.20.108
> > State      Recv-Q Send-Q                                 Local
> > Address:Port                                     Peer Address:Port   
> > ESTAB      0      450976
> > 192.168.20.110:52017
> > 192.168.20.108:44169    timer:(on,235ms,0) ino:21711 sk:f24d2d00
> > 	 mem:(r0,w698880,f206336,t0) ts sack ecn bic wscale:6,8 rto:252
> > rtt:52/0.75 cwnd:408 send 90.9Mbps rcv_space:14600
> > 
> > $ tc -s -d qdisc show dev vlan.103
> > qdisc netem 8009: root refcnt 2 limit 1000 delay 50.0ms
> >  Sent 15149167430 bytes 10017438 pkt (dropped 0, overlimits 0 requeues
> > 0) 
> >  rate 74878Kbit 6193pps backlog 476760b 316p requeues 0 
> 
> Its seems a problem with GRO.
> 
> Receiver opens its window each time it sends an ACK, without taking care
> how many segments were ACKed.
> 
> In these traces, I give the ACKS sent by receiver, first trace GRO on,
> second with GRO off
> 
> We can see that with GRO off, receiver opens its window way faster
> (against number of received bytes)
> 
> (REC & SENDER are both 3.0 kernels, so problem is quite old)
[...]

I'm aware of this problem and I believe it exists with most
implementations of LRO.

The out-of-tree version of the sfc driver has its own soft-LRO
implementation (SSR) which does some limited connection tracking to
detect slow start and disable aggregation temporarily.  I've been
meaning to look into enhancing GRO to match SSR, but haven't got round
to it yet.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.