[PATCH net] net: fix the RTO timer retransmitting skb every 1ms if linear option is enabled

[PATCH net] net: fix the RTO timer retransmitting skb every 1ms if linear option is enabled

[Jason Xing]

From: Jason Xing <kernelxing@tencent.com>

In the real workload, I encountered an issue which could cause the RTO
timer to retransmit the skb per 1ms with linear option enabled. The amount
of lost-retransmitted skbs can go up to 1000+ instantly.

The root cause is that if the icsk_rto happens to be zero in the 6th round
(which is the TCP_THIN_LINEAR_RETRIES value), then it will alway be zero
due to the changed calculation method in tcp_retransmit_timer() as follows:

icsk->icsk_rto = min(icsk->icsk_rto << 1, TCP_RTO_MAX);

Above line could be converted to
icsk->icsk_rto = min(0 << 1, TCP_RTO_MAX) = 0

Therefore, the timer expires so quickly without any doubt.

I read through the RFC 6298 and found that the RTO value can be rounded
up to a certain value, in Linux, say TCP_RTO_MIN as default, which is
regarded as the lower bound in this patch as suggested by Eric.

Fixes: 36e31b0af587 ("net: TCP thin linear timeouts")
Suggested-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Jason Xing <kernelxing@tencent.com>
---
 net/ipv4/tcp_timer.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
index d45c96c7f5a4..b2b25861355c 100644
--- a/net/ipv4/tcp_timer.c
+++ b/net/ipv4/tcp_timer.c
@@ -599,7 +599,9 @@ void tcp_retransmit_timer(struct sock *sk)
 	    tcp_stream_is_thin(tp) &&
 	    icsk->icsk_retransmits <= TCP_THIN_LINEAR_RETRIES) {
 		icsk->icsk_backoff = 0;
-		icsk->icsk_rto = min(__tcp_set_rto(tp), TCP_RTO_MAX);
+		icsk->icsk_rto = clamp(__tcp_set_rto(tp),
+					    tcp_rto_min(sk),
+					    TCP_RTO_MAX);
 	} else if (sk->sk_state != TCP_SYN_SENT ||
 		   icsk->icsk_backoff >
 		   READ_ONCE(net->ipv4.sysctl_tcp_syn_linear_timeouts)) {
-- 
2.37.3

Re: [PATCH net] net: fix the RTO timer retransmitting skb every 1ms if linear option is enabled

[Simon Horman]

On Thu, Aug 10, 2023 at 07:21:48PM +0800, Jason Xing wrote:
> From: Jason Xing <kernelxing@tencent.com>
> 
> In the real workload, I encountered an issue which could cause the RTO
> timer to retransmit the skb per 1ms with linear option enabled. The amount
> of lost-retransmitted skbs can go up to 1000+ instantly.
> 
> The root cause is that if the icsk_rto happens to be zero in the 6th round
> (which is the TCP_THIN_LINEAR_RETRIES value), then it will alway be zero

nit: alway -> always

     checkpatch.pl --codespell is your friend

> due to the changed calculation method in tcp_retransmit_timer() as follows:
> 
> icsk->icsk_rto = min(icsk->icsk_rto << 1, TCP_RTO_MAX);
> 
> Above line could be converted to
> icsk->icsk_rto = min(0 << 1, TCP_RTO_MAX) = 0
> 
> Therefore, the timer expires so quickly without any doubt.
> 
> I read through the RFC 6298 and found that the RTO value can be rounded
> up to a certain value, in Linux, say TCP_RTO_MIN as default, which is
> regarded as the lower bound in this patch as suggested by Eric.
> 
> Fixes: 36e31b0af587 ("net: TCP thin linear timeouts")
> Suggested-by: Eric Dumazet <edumazet@google.com>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>
> ---
>  net/ipv4/tcp_timer.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
> index d45c96c7f5a4..b2b25861355c 100644
> --- a/net/ipv4/tcp_timer.c
> +++ b/net/ipv4/tcp_timer.c
> @@ -599,7 +599,9 @@ void tcp_retransmit_timer(struct sock *sk)
>  	    tcp_stream_is_thin(tp) &&
>  	    icsk->icsk_retransmits <= TCP_THIN_LINEAR_RETRIES) {
>  		icsk->icsk_backoff = 0;
> -		icsk->icsk_rto = min(__tcp_set_rto(tp), TCP_RTO_MAX);
> +		icsk->icsk_rto = clamp(__tcp_set_rto(tp),
> +					    tcp_rto_min(sk),
> +					    TCP_RTO_MAX);

nit: this indentation looks a bit odd.

		icsk->icsk_rto = clamp(__tcp_set_rto(tp),
				       tcp_rto_min(sk),
				       TCP_RTO_MAX);

>  	} else if (sk->sk_state != TCP_SYN_SENT ||
>  		   icsk->icsk_backoff >
>  		   READ_ONCE(net->ipv4.sysctl_tcp_syn_linear_timeouts)) {
> -- 
> 2.37.3
> 
> 

Re: [PATCH net] net: fix the RTO timer retransmitting skb every 1ms if linear option is enabled

[Jason Xing]

On Fri, Aug 11, 2023 at 4:32 AM Simon Horman <horms@kernel.org> wrote:
>
> On Thu, Aug 10, 2023 at 07:21:48PM +0800, Jason Xing wrote:
> > From: Jason Xing <kernelxing@tencent.com>
> >
> > In the real workload, I encountered an issue which could cause the RTO
> > timer to retransmit the skb per 1ms with linear option enabled. The amount
> > of lost-retransmitted skbs can go up to 1000+ instantly.
> >
> > The root cause is that if the icsk_rto happens to be zero in the 6th round
> > (which is the TCP_THIN_LINEAR_RETRIES value), then it will alway be zero
>
> nit: alway -> always
>
>      checkpatch.pl --codespell is your friend

Thanks for your reminder.

>
> > due to the changed calculation method in tcp_retransmit_timer() as follows:
> >
> > icsk->icsk_rto = min(icsk->icsk_rto << 1, TCP_RTO_MAX);
> >
> > Above line could be converted to
> > icsk->icsk_rto = min(0 << 1, TCP_RTO_MAX) = 0
> >
> > Therefore, the timer expires so quickly without any doubt.
> >
> > I read through the RFC 6298 and found that the RTO value can be rounded
> > up to a certain value, in Linux, say TCP_RTO_MIN as default, which is
> > regarded as the lower bound in this patch as suggested by Eric.
> >
> > Fixes: 36e31b0af587 ("net: TCP thin linear timeouts")
> > Suggested-by: Eric Dumazet <edumazet@google.com>
> > Signed-off-by: Jason Xing <kernelxing@tencent.com>
> > ---
> >  net/ipv4/tcp_timer.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
> > index d45c96c7f5a4..b2b25861355c 100644
> > --- a/net/ipv4/tcp_timer.c
> > +++ b/net/ipv4/tcp_timer.c
> > @@ -599,7 +599,9 @@ void tcp_retransmit_timer(struct sock *sk)
> >           tcp_stream_is_thin(tp) &&
> >           icsk->icsk_retransmits <= TCP_THIN_LINEAR_RETRIES) {
> >               icsk->icsk_backoff = 0;
> > -             icsk->icsk_rto = min(__tcp_set_rto(tp), TCP_RTO_MAX);
> > +             icsk->icsk_rto = clamp(__tcp_set_rto(tp),
> > +                                         tcp_rto_min(sk),
> > +                                         TCP_RTO_MAX);
>
> nit: this indentation looks a bit odd.

Yeah, I'm always stuck with this kind of indentation issue. I'll fix
it in v2 patch.

Thanks,
Jason

>
>                 icsk->icsk_rto = clamp(__tcp_set_rto(tp),
>                                        tcp_rto_min(sk),
>                                        TCP_RTO_MAX);
>
> >       } else if (sk->sk_state != TCP_SYN_SENT ||
> >                  icsk->icsk_backoff >
> >                  READ_ONCE(net->ipv4.sysctl_tcp_syn_linear_timeouts)) {
> > --
> > 2.37.3
> >
> >