From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C146C4361B for ; Thu, 10 Dec 2020 10:44:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 181F223D69 for ; Thu, 10 Dec 2020 10:44:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388794AbgLJKok (ORCPT ); Thu, 10 Dec 2020 05:44:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728800AbgLJKo3 (ORCPT ); Thu, 10 Dec 2020 05:44:29 -0500 Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C0FBC061793 for ; Thu, 10 Dec 2020 02:43:49 -0800 (PST) Received: by mail-wm1-x344.google.com with SMTP id k10so4211393wmi.3 for ; Thu, 10 Dec 2020 02:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=6c9t4cnf3DvAPdhFWGATkIFEAIlNdLosBaBfkPc2QEA=; b=V6znspd0dc3v5ui9xolGs19QGbmOwM4RPyYmEVbvbxpHq3wxrgvC0Gvv0JR9ax+bGz P4mqsSbHwfEVSR7EZntJYe2hpS7k8Xs6NrYwpwrnrabDe5A8RHq4KrqHCQci60X3Cri2 wWMHBzmtsGaMAPPW5z6p8J/mM/fxDMj+/AkF0eQ7hwYKJERrYBITit1+e47y/G5n2PgR dK1EPJpn5xNq/cxZw3ueANjW8EOtTkcmBNLAiSR3b3QO4jyly3BhhvrxAAwWaJH6bOGb wN73ilhNxx5N5bjHzWycTCYU1hARjokkf/ZJcmb7HHyjq70G6rM8ZYSu5D4h2HVXRjN1 djNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=6c9t4cnf3DvAPdhFWGATkIFEAIlNdLosBaBfkPc2QEA=; b=Uk3jOOlu7+7IzrkVAlZBRyIKjoRn/Pu8poXefJTcs81ZuDoUeK9JTPgzdyc6oCqNt/ 1tI8ICt+6tkWQUhF21Za/2eTQvu9oMWdmR1pkYtnZEBpn9JVt79Src5zH9zwl2aJmW2f U8glSLryL5wZL3QcKaYE0wCtxdS1gaU7fBikEI0SXduFk2zGA5C6BJUmf3GRqOUrs4Ay +eH7cqvEKl1f5d9MBGPOF+gWLojCGrIaowZpVjifO+6EDicXsYWRknzEldgDzAmtioWc gvtMcr0LWcBQH3X2XwZ49XInX0J7kLYvbsGIrT5PKUXF+BsXPkz6KQbxYdMvRlxq4P8A 8Dlg== X-Gm-Message-State: AOAM531TDQ85PCHGb98YjwjtDKKZQlc4j8tpR2wRDZoYO97jk1Wbw1p8 W0k1XHX9hgNBnitD4C43mZY= X-Google-Smtp-Source: ABdhPJyl7IpbgI685L+4f1rfoLN7ylhfUT1gytIpX8+HlFxnnE2s5yE4EWgsFjJLLMgYUFFRlHH9fw== X-Received: by 2002:a7b:c751:: with SMTP id w17mr7287865wmk.121.1607597027913; Thu, 10 Dec 2020 02:43:47 -0800 (PST) Received: from [192.168.8.116] ([37.171.242.50]) by smtp.gmail.com with ESMTPSA id v20sm9571385wra.19.2020.12.10.02.43.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 10 Dec 2020 02:43:47 -0800 (PST) Subject: Re: [PATCH net] tcp: fix cwnd-limited bug for TSO deferral where we send nothing To: Ingemar Johansson S , Jakub Kicinski , Neal Cardwell Cc: David Miller , "netdev@vger.kernel.org" , Neal Cardwell , Yuchung Cheng , Soheil Hassas Yeganeh , Eric Dumazet References: <20201209035759.1225145-1-ncardwell.kernel@gmail.com> <20201209161403.47177093@kicinski-fedora-pc1c0hjn.DHCP.thefacebook.com> From: Eric Dumazet Message-ID: <944d0a94-1ec1-539b-a463-b762ebf5ed8f@gmail.com> Date: Thu, 10 Dec 2020 11:43:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 12/10/20 10:50 AM, Ingemar Johansson S wrote: > Hi > Slighty off topic > It is a smaller mystery why I am listed as having reported this artifact ?. > I don't have any memory that I did so.. strange 😐. > I think this was your report : https://mailarchive.ietf.org/arch/msg/tcpm/3U--r1vC81blOfZ5JwAYWIbm4vE/ Have fun ! > Regards > Ingemar > >> -----Original Message----- >> From: Jakub Kicinski >> Sent: den 10 december 2020 01:14 >> To: Neal Cardwell >> Cc: David Miller ; netdev@vger.kernel.org; Neal >> Cardwell ; Ingemar Johansson S >> ; Yuchung Cheng >> ; Soheil Hassas Yeganeh ; Eric >> Dumazet >> Subject: Re: [PATCH net] tcp: fix cwnd-limited bug for TSO deferral where we >> send nothing >> >> On Tue, 8 Dec 2020 22:57:59 -0500 Neal Cardwell wrote: >>> From: Neal Cardwell >>> >>> When cwnd is not a multiple of the TSO skb size of N*MSS, we can get >>> into persistent scenarios where we have the following sequence: >>> >>> (1) ACK for full-sized skb of N*MSS arrives >>> -> tcp_write_xmit() transmit full-sized skb with N*MSS >>> -> move pacing release time forward >>> -> exit tcp_write_xmit() because pacing time is in the future >>> >>> (2) TSQ callback or TCP internal pacing timer fires >>> -> try to transmit next skb, but TSO deferral finds remainder of >>> available cwnd is not big enough to trigger an immediate send >>> now, so we defer sending until the next ACK. >>> >>> (3) repeat... >>> >>> So we can get into a case where we never mark ourselves as >>> cwnd-limited for many seconds at a time, even with >>> bulk/infinite-backlog senders, because: >>> >>> o In case (1) above, every time in tcp_write_xmit() we have enough >>> cwnd to send a full-sized skb, we are not fully using the cwnd >>> (because cwnd is not a multiple of the TSO skb size). So every time we >>> send data, we are not cwnd limited, and so in the cwnd-limited >>> tracking code in tcp_cwnd_validate() we mark ourselves as not >>> cwnd-limited. >>> >>> o In case (2) above, every time in tcp_write_xmit() that we try to >>> transmit the "remainder" of the cwnd but defer, we set the local >>> variable is_cwnd_limited to true, but we do not send any packets, so >>> sent_pkts is zero, so we don't call the cwnd-limited logic to update >>> tp->is_cwnd_limited. >>> >>> Fixes: ca8a22634381 ("tcp: make cwnd-limited checks measurement-based, >>> and gentler") >>> Reported-by: Ingemar Johansson >>> Signed-off-by: Neal Cardwell >>> Signed-off-by: Yuchung Cheng >>> Acked-by: Soheil Hassas Yeganeh >>> Signed-off-by: Eric Dumazet >> >> Applied, thank you!