From mboxrd@z Thu Jan 1 00:00:00 1970 From: Oleksandr Natalenko Subject: Re: TCP and BBR: reproducibly low cwnd and bandwidth Date: Fri, 16 Feb 2018 18:35:03 +0100 Message-ID: <2938690.9Cepv1nWrF@natalenko.name> References: <1697118.nv5eASg0nx@natalenko.name> <2189487.nPhU5NAnbi@natalenko.name> <061740d0-9876-c905-7466-ef225ec3cdc5@applied-asynchrony.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Cc: "David S. Miller" , Alexey Kuznetsov , Hideaki YOSHIFUJI , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Eric Dumazet , Soheil Hassas Yeganeh , Neal Cardwell , Yuchung Cheng , Van Jacobson , Jerry Chu To: Holger =?ISO-8859-1?Q?Hoffst=E4tte?= Return-path: Received: from vulcan.natalenko.name ([104.207.131.136]:43324 "EHLO vulcan.natalenko.name" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1161197AbeBPRfF (ORCPT ); Fri, 16 Feb 2018 12:35:05 -0500 In-Reply-To: <061740d0-9876-c905-7466-ef225ec3cdc5@applied-asynchrony.com> Sender: netdev-owner@vger.kernel.org List-ID: Hi. On p=E1tek 16. =FAnora 2018 17:26:11 CET Holger Hoffst=E4tte wrote: > These are very odd configurations. :) > Non-preempt/100 might well be too slow, whereas PREEMPT/1000 might simply > have too much overhead. Since the pacing is based on hrtimers, should HZ matter at all? Even if so,= =20 poor 1 Gbps link shouldn't drop to below 100 Mbps, for sure. > BBR in general will run with lower cwnd than e.g. Cubic or others. > That's a feature and necessary for WAN transfers. Okay, got it. > Something seems really wrong with your setup. I get completely > expected throughput on wired 1Gb between two hosts: > /* snip */ Yes, and that's strange :/. And that's why I'm wondering what I am missing= =20 since things cannot be *that* bad. > /* snip */ > Please note that BBR was developed to address the case of WAN transfers > (or more precisely high BDP paths) which often suffer from TCP throughput > collapse due to single packet loss events. While it might "work" in other > scenarios as well, strictly speaking delay-based anything is increasingly > less likely to work when there is no meaningful notion of delay - such > as on a LAN. (yes, this is very simplified..) >=20 > The BBR mailing list has several nice reports why the current BBR > implementation (dubbed v1) has a few - sometimes severe - problems. > These are being addressed as we speak. >=20 > (let me know if you want some of those tech reports by email. :) Well, yes, please, why not :). > /* snip */ > I'm not sure testing the old version without builtin pacing is going to h= elp > matters in finding the actual problem. :) > Several people have reported severe performance regressions with 4.15.x, > maybe that's related. Can you test latest 4.14.x? Observed this on v4.14 too but didn't pay much attention until realised tha= t=20 things look definitely wrong. > Out of curiosity, what is the expected use case for BBR here? Nothing special, just assumed it could be set as a default for both WAN and= =20 LAN usage. Regards, Oleksandr