From mboxrd@z Thu Jan  1 00:00:00 1970
From: Oleksandr Natalenko <oleksandr@natalenko.name>
Subject: Re: TCP and BBR: reproducibly low cwnd and bandwidth
Date: Sat, 17 Feb 2018 11:01:19 +0100
Message-ID: <5668348.WVIY7FqTii@natalenko.name>
References: <1697118.nv5eASg0nx@natalenko.name> <7409814.oObJlsYiIU@natalenko.name> <CANn89iL+ykS1d10wPLHJiYsgZj0u1q8EqEOkbBuFtBkONT8GSg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Cc: Neal Cardwell <ncardwell@google.com>,
        Eric Dumazet <eric.dumazet@gmail.com>,
        "David S. Miller" <davem@davemloft.net>,
        Netdev <netdev@vger.kernel.org>,
        Yuchung Cheng <ycheng@google.com>,
        Soheil Hassas Yeganeh <soheil@google.com>,
        Jerry Chu <hkchu@google.com>, Dave Taht <dave.taht@gmail.com>
To: Eric Dumazet <edumazet@google.com>
Return-path: <netdev-owner@vger.kernel.org>
Received: from vulcan.natalenko.name ([104.207.131.136]:33640 "EHLO
        vulcan.natalenko.name" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1751077AbeBQKBW (ORCPT
        <rfc822;netdev@vger.kernel.org>); Sat, 17 Feb 2018 05:01:22 -0500
In-Reply-To: <CANn89iL+ykS1d10wPLHJiYsgZj0u1q8EqEOkbBuFtBkONT8GSg@mail.gmail.com>
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>

Hi.

On p=E1tek 16. =FAnora 2018 23:59:52 CET Eric Dumazet wrote:
> Well, no effect  here on e1000e (1 Gbit) at least
>=20
> # ethtool -K eth3 sg off
> Actual changes:
> scatter-gather: off
> tx-scatter-gather: off
> tcp-segmentation-offload: off
> tx-tcp-segmentation: off [requested on]
> tx-tcp6-segmentation: off [requested on]
> generic-segmentation-offload: off [requested on]
>=20
> # tc qd replace dev eth3 root pfifo_fast
> # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
>     941
> # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
>     941
> # tc qd replace dev eth3 root fq
> # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
>     941
> # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
>     941
> # tc qd replace dev eth3 root fq_codel
> # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
>     941
> # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
>     941
> #

That really looks strange to me. I'm able to reproduce the effect caused by=
=20
disabling scatter-gather even on the VM (using iperf3, as usual):

BBR+fq_codel:
sg on:  4.23 Gbits/sec
sg off: 121 Mbits/sec

BBR+fq:
sg on:  6.38 Gbits/sec
sg off: 437 Mbits/sec

Reno+fq_codel:
sg on:  6.74 Gbits/sec
sg off: 1.37 Gbits/sec

Reno+fq:
sg on:  6.53 Gbits/sec
sg off: 1.19 Gbits/sec

Regardless of which congestion algorithm and qdisc is in use, the throughpu=
t=20
drops, but when BBR is in use, especially with something non-fq, it drops t=
he=20
most.

Oleksandr