Re: [PATCH net] tcp: fix cwnd-limited bug for TSO deferral where we send nothing

[Eric Dumazet <eric.dumazet@gmail.com>]

On 12/10/20 10:50 AM, Ingemar Johansson S wrote:
> Hi
> Slighty off topic 
> It is a smaller mystery why I am listed as having reported this artifact ?. 
> I don't have any memory that I did so.. strange 😐. 
> 

I think this was your report :

https://mailarchive.ietf.org/arch/msg/tcpm/3U--r1vC81blOfZ5JwAYWIbm4vE/

Have fun !

> Regards
> Ingemar
> 
>> -----Original Message-----
>> From: Jakub Kicinski <kuba@kernel.org>
>> Sent: den 10 december 2020 01:14
>> To: Neal Cardwell <ncardwell.kernel@gmail.com>
>> Cc: David Miller <davem@davemloft.net>; netdev@vger.kernel.org; Neal
>> Cardwell <ncardwell@google.com>; Ingemar Johansson S
>> <ingemar.s.johansson@ericsson.com>; Yuchung Cheng
>> <ycheng@google.com>; Soheil Hassas Yeganeh <soheil@google.com>; Eric
>> Dumazet <edumazet@google.com>
>> Subject: Re: [PATCH net] tcp: fix cwnd-limited bug for TSO deferral where we
>> send nothing
>>
>> On Tue,  8 Dec 2020 22:57:59 -0500 Neal Cardwell wrote:
>>> From: Neal Cardwell <ncardwell@google.com>
>>>
>>> When cwnd is not a multiple of the TSO skb size of N*MSS, we can get
>>> into persistent scenarios where we have the following sequence:
>>>
>>> (1) ACK for full-sized skb of N*MSS arrives
>>>   -> tcp_write_xmit() transmit full-sized skb with N*MSS
>>>   -> move pacing release time forward
>>>   -> exit tcp_write_xmit() because pacing time is in the future
>>>
>>> (2) TSQ callback or TCP internal pacing timer fires
>>>   -> try to transmit next skb, but TSO deferral finds remainder of
>>>      available cwnd is not big enough to trigger an immediate send
>>>      now, so we defer sending until the next ACK.
>>>
>>> (3) repeat...
>>>
>>> So we can get into a case where we never mark ourselves as
>>> cwnd-limited for many seconds at a time, even with
>>> bulk/infinite-backlog senders, because:
>>>
>>> o In case (1) above, every time in tcp_write_xmit() we have enough
>>> cwnd to send a full-sized skb, we are not fully using the cwnd
>>> (because cwnd is not a multiple of the TSO skb size). So every time we
>>> send data, we are not cwnd limited, and so in the cwnd-limited
>>> tracking code in tcp_cwnd_validate() we mark ourselves as not
>>> cwnd-limited.
>>>
>>> o In case (2) above, every time in tcp_write_xmit() that we try to
>>> transmit the "remainder" of the cwnd but defer, we set the local
>>> variable is_cwnd_limited to true, but we do not send any packets, so
>>> sent_pkts is zero, so we don't call the cwnd-limited logic to update
>>> tp->is_cwnd_limited.
>>>
>>> Fixes: ca8a22634381 ("tcp: make cwnd-limited checks measurement-based,
>>> and gentler")
>>> Reported-by: Ingemar Johansson <ingemar.s.johansson@ericsson.com>
>>> Signed-off-by: Neal Cardwell <ncardwell@google.com>
>>> Signed-off-by: Yuchung Cheng <ycheng@google.com>
>>> Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
>>> Signed-off-by: Eric Dumazet <edumazet@google.com>
>>
>> Applied, thank you!