From mboxrd@z Thu Jan  1 00:00:00 1970
From: Eric Dumazet <eric.dumazet@gmail.com>
Subject: Re: TCP and BBR: reproducibly low cwnd and bandwidth
Date: Sat, 17 Feb 2018 10:52:51 -0800
Message-ID: <1518893571.55655.12.camel@gmail.com>
References: <1697118.nv5eASg0nx@natalenko.name>
         <7409814.oObJlsYiIU@natalenko.name>
         <CANn89iL+ykS1d10wPLHJiYsgZj0u1q8EqEOkbBuFtBkONT8GSg@mail.gmail.com>
         <5668348.WVIY7FqTii@natalenko.name>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Cc: Neal Cardwell <ncardwell@google.com>,
        "David S. Miller" <davem@davemloft.net>,
        Netdev <netdev@vger.kernel.org>,
        Yuchung Cheng <ycheng@google.com>,
        Soheil Hassas Yeganeh <soheil@google.com>,
        Jerry Chu <hkchu@google.com>, Dave Taht <dave.taht@gmail.com>
To: Oleksandr Natalenko <oleksandr@natalenko.name>,
        Eric Dumazet <edumazet@google.com>
Return-path: <netdev-owner@vger.kernel.org>
Received: from mail-pl0-f50.google.com ([209.85.160.50]:35920 "EHLO
        mail-pl0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1751049AbeBQSwz (ORCPT
        <rfc822;netdev@vger.kernel.org>); Sat, 17 Feb 2018 13:52:55 -0500
Received: by mail-pl0-f50.google.com with SMTP id v3so3466231plg.3
        for <netdev@vger.kernel.org>; Sat, 17 Feb 2018 10:52:55 -0800 (PST)
In-Reply-To: <5668348.WVIY7FqTii@natalenko.name>
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>

On Sat, 2018-02-17 at 11:01 +0100, Oleksandr Natalenko wrote:
> Hi.
> 
> On pátek 16. února 2018 23:59:52 CET Eric Dumazet wrote:
> > Well, no effect  here on e1000e (1 Gbit) at least
> > 
> > # ethtool -K eth3 sg off
> > Actual changes:
> > scatter-gather: off
> > tx-scatter-gather: off
> > tcp-segmentation-offload: off
> > tx-tcp-segmentation: off [requested on]
> > tx-tcp6-segmentation: off [requested on]
> > generic-segmentation-offload: off [requested on]
> > 
> > # tc qd replace dev eth3 root pfifo_fast
> > # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
> >     941
> > # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
> >     941
> > # tc qd replace dev eth3 root fq
> > # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
> >     941
> > # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
> >     941
> > # tc qd replace dev eth3 root fq_codel
> > # ./super_netperf 1 -H 7.7.7.84 -- -K cubic
> >     941
> > # ./super_netperf 1 -H 7.7.7.84 -- -K bbr
> >     941
> > #
> 
> That really looks strange to me. I'm able to reproduce the effect caused by 
> disabling scatter-gather even on the VM (using iperf3, as usual):

This must be some race condition in the code I added in TCP for self-
pacing, when a sort timeout is programmed.

Disabling SG means TCP cooks 1-MSS packets.

I will take a look, probably after the (long) week-end : Tuesday.

Thanks !