Re: limited network bandwidth with 3.2.x kernels

[David Miller <davem@davemloft.net>]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 21 Feb 2012 19:41:23 +0100

> Le vendredi 17 février 2012 à 11:41 -0500, Neal Cardwell a écrit :
>> On Thu, Feb 16, 2012 at 5:35 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > An incoming skb is considered as "good citizen" in term of memory usage
>> > if its truesize is no more than
>> > len + len/4   (if adv_win_scale == 2)
>> >
>> > That was true when truesize was 1500+NET_SKB_PAD+sizeof(sk_buff),
>> > but not true anymore when truesize is 2048+sizeof(sk_buff), or even more
>> > when its 4096+sizeof(sk_buff)
>> >
>> > So receiver doesnt increase rcv_ssthresh and cannot open its window.
>> >
>> > tcp_grow_window() should be tweaked to :
>> >
>> > 1) Relax the requirements
>> > 2) Allow bigger increase in case of super packets (LRO/GRO)
>> 
>> That sounds great. Is this something you're planning on tackling?
>> 
> 
> Hmm, I thought about following (untested) patch :
> 
> Idea is to increase rcv_sshthresh by 2*len * (len/truesize), instead of
> 2*mss.

This looks like a good start.

I think the core task is to seperate two things:

1) The test for "full size SKB" which erroneously is done using that
   "X + (X/N)" formula.

   To be quite honest, I think we don't care about this condition at
   all.  What we really care about is whether the frame was a full MSS
   (or at least, close to it).

2) The growth calculation which does in fact need to be based upon
   truesize because this determines what we'll need to do to make
   more frames actually fit in the receive window given the witnessed
   truesize ratio.

And it seems your patch takes care of #2 but not #1.

The error in the current code is that it tries to estimate what a
"good" truesize might look like with a very ad-hoc sysctl that is, at
best, magic.  We should kill this sysctl and everything using it, and
therefore make our decisions on window growth purely based upon
skb->len and MSS.