From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: BBR and TCP internal pacing causing interrupt storm with pfifo_fast Date: Mon, 15 Oct 2018 09:23:03 -0700 Message-ID: <0b3bb4c5-49bb-11a7-eac8-5515ef72851e@gmail.com> References: <183b7fe0-0757-cf63-555c-925ea840c67f@gmail.com> <1988670d-0d87-2b9e-24cb-a9a610ea33fa@gmail.com> <252ea882-bcd2-b205-7d68-541e88b5d617@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: Eric Dumazet , Kevin Yang , netdev To: Eric Dumazet , Gasper Zejn Return-path: Received: from mail-pl1-f195.google.com ([209.85.214.195]:44862 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726663AbeJPAJB (ORCPT ); Mon, 15 Oct 2018 20:09:01 -0400 Received: by mail-pl1-f195.google.com with SMTP id p25-v6so9540469pli.11 for ; Mon, 15 Oct 2018 09:23:06 -0700 (PDT) In-Reply-To: Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 10/15/2018 07:50 AM, Eric Dumazet wrote: > On Mon, Oct 15, 2018 at 3:26 AM Gasper Zejn wrote: >> >> >> I've tried to isolate the issue as best I could. There seems to be an >> issue if the TCP socket has keepalive set and send queue is not empty >> and the route goes away. >> >> https://github.com/zejn/bbr_pfifo_interrupts_issue >> >> Hope this helps, >> Gasper > > This is awesome Gasper, I will take a look thanks. > > Note that we are about to send a patch series (targeting net-next) to > polish the EDT patch series that was merged last month for linux-4.20. > TCP internal pacing is going to be much better performance-wise. > Yeah, I believe that : Commit c092dd5f4a7f4e4dbbcc8cf2e50b516bf07e432f ("tcp: switch tcp_internal_pacing() to tcp_wstamp_ns") has incidentally fixed the issue. That is because it calls tcp_internal_pacing() from tcp_update_skb_after_send() which is called only if the packet was correctly sent by IP layer. Before this patch, tcp_internal_pacing() was called from __tcp_transmit_skb() before we attempted to send the clone and the clone could be dropped in IP layer (lack of route for example) right away. So in case the packet was not sent because of a route problem, the high resolution timer would kick soon after and TCP xmit path would be entered again, triggering this loop problem. I am going to send the 2nd round of EDT patches, so that you can try David Miller net-next tree with all the patches we believe are needed for 4.20. Once proven to work, we might have to backport the series to 4.18 and 4.19 Thanks !