Re: iperf performance regression since Linux 5.18

[Stefan Wahren <wahrenst@gmx.net>]

Hi Eric,

Am 16.10.23 um 12:35 schrieb Eric Dumazet:
> On Mon, Oct 16, 2023 at 11:49 AM Eric Dumazet <edumazet@google.com> wrote:
> Speaking of TSQ, it seems an old change (commit 75eefc6c59fd "tcp:
> tsq: add a shortcut in tcp_small_queue_check()")
> has been accidentally removed in 2017 (75c119afe14f "tcp: implement
> rb-tree based retransmit queue")
>
> Could you try this fix:
>
> diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
> index 9c8c42c280b7638f0f4d94d68cd2c73e3c6c2bcc..e61a3a381d51b554ec8440928e22a290712f0b6b
> 100644
> --- a/net/ipv4/tcp_output.c
> +++ b/net/ipv4/tcp_output.c
> @@ -2542,6 +2542,18 @@ static bool tcp_pacing_check(struct sock *sk)
>          return true;
>   }
>
> +static bool tcp_rtx_queue_empty_or_single_skb(const struct sock *sk)
> +{
> +       const struct rb_node *node = sk->tcp_rtx_queue.rb_node;
> +
> +       /* No skb in the rtx queue. */
> +       if (!node)
> +               return true;
> +
> +       /* Only one skb in rtx queue. */
> +       return !node->rb_left && !node->rb_right;
> +}
> +
>   /* TCP Small Queues :
>    * Control number of packets in qdisc/devices to two packets / or ~1 ms.
>    * (These limits are doubled for retransmits)
> @@ -2579,12 +2591,12 @@ static bool tcp_small_queue_check(struct sock
> *sk, const struct sk_buff *skb,
>                  limit += extra_bytes;
>          }
>          if (refcount_read(&sk->sk_wmem_alloc) > limit) {
> -               /* Always send skb if rtx queue is empty.
> +               /* Always send skb if rtx queue is empty or has one skb.
>                   * No need to wait for TX completion to call us back,
>                   * after softirq/tasklet schedule.
>                   * This helps when TX completions are delayed too much.
>                   */
> -               if (tcp_rtx_queue_empty(sk))
> +               if (tcp_rtx_queue_empty_or_single_skb(sk))
>                          return false;
>
>                  set_bit(TSQ_THROTTLED, &sk->sk_tsq_flags);

This patch applied on top of Linux 6.1.49, TSO on, gso_max_size 65535,
CONFIG_HZ_100=y

root@tarragon:/boot# iperf -t 10 -i 1 -c 192.168.1.129
------------------------------------------------------------
Client connecting to 192.168.1.129, TCP port 5001
TCP window size:  192 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.12 port 59714 connected with 192.168.1.129 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  11.5 MBytes  96.5 Mbits/sec
[  3]  1.0- 2.0 sec  11.4 MBytes  95.4 Mbits/sec
[  3]  2.0- 3.0 sec  11.1 MBytes  93.3 Mbits/sec
[  3]  3.0- 4.0 sec  11.2 MBytes  94.4 Mbits/sec
[  3]  4.0- 5.0 sec  11.1 MBytes  93.3 Mbits/sec
[  3]  5.0- 6.0 sec  11.2 MBytes  94.4 Mbits/sec
[  3]  6.0- 7.0 sec  11.2 MBytes  94.4 Mbits/sec
[  3]  7.0- 8.0 sec  11.1 MBytes  93.3 Mbits/sec
[  3]  8.0- 9.0 sec  11.4 MBytes  95.4 Mbits/sec
[  3]  9.0-10.0 sec  11.2 MBytes  94.4 Mbits/sec
[  3]  0.0-10.0 sec   113 MBytes  94.4 Mbits/sec

The figures are comparable to disabling TSO -> Good

Thanks