From mboxrd@z Thu Jan  1 00:00:00 1970
From: Gasper Zejn <zelo.zejn@gmail.com>
Subject: Re: BBR and TCP internal pacing causing interrupt storm with
 pfifo_fast
Date: Mon, 15 Oct 2018 12:26:06 +0200
Message-ID: <252ea882-bcd2-b205-7d68-541e88b5d617@gmail.com>
References: <183b7fe0-0757-cf63-555c-925ea840c67f@gmail.com>
 <cb3f5cd7-dc71-6004-9e68-748616ad7871@gmail.com>
 <1988670d-0d87-2b9e-24cb-a9a610ea33fa@gmail.com>
 <CANn89iLOHxOOuGzqNODk0hNWJ7an0bZXs4G-8mQjq3WWhZPo9g@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Cc: Eric Dumazet <eric.dumazet@gmail.com>, Kevin Yang <yyd@google.com>,
        netdev <netdev@vger.kernel.org>
To: Eric Dumazet <edumazet@google.com>
Return-path: <netdev-owner@vger.kernel.org>
Received: from mail-wr1-f46.google.com ([209.85.221.46]:36197 "EHLO
        mail-wr1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1726273AbeJOSKs (ORCPT
        <rfc822;netdev@vger.kernel.org>); Mon, 15 Oct 2018 14:10:48 -0400
Received: by mail-wr1-f46.google.com with SMTP id y16so20664616wrw.3
        for <netdev@vger.kernel.org>; Mon, 15 Oct 2018 03:26:09 -0700 (PDT)
In-Reply-To: <CANn89iLOHxOOuGzqNODk0hNWJ7an0bZXs4G-8mQjq3WWhZPo9g@mail.gmail.com>
Content-Language: en-US
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>


On 09. 10. 2018 19:26, Eric Dumazet wrote:
> On Tue, Oct 9, 2018 at 10:22 AM Gasper Zejn <zelo.zejn@gmail.com> wrote:
>> On 09. 10. 2018 19:00, Eric Dumazet wrote:
>>> On 10/09/2018 09:38 AM, Gasper Zejn wrote:
>>>> Hello,
>>>>
>>>> I am seeing interrupt storms of over 100k-900k local timer interrupts
>>>> when changing between network devices or networks with open TCP
>>>> connections when not using sch_fq (I was using pfifo_fast). Using sch_fq
>>>> makes the bug with interrupt storm go away.
>>>>
>>> That is for what kind of traffic ?
>>>
>>> If your TCP flows send 100k-3M packets per second, then yes, the pacing timers
>>> could be setup in the 100k-900k range.
>>>
>> Traffic is nowhere in that range, think of having a few browser tabs of
>> javascript rich
>> web pages open, mostly idle, for example slack, gmail or tweetdeck. No
>> significant
>> packet rate is needed, just open connections.
> No idea of what is going on really. A repro would be nice.

I've tried to isolate the issue as best I could. There seems to be an
issue if the TCP socket has keepalive set and send queue is not empty
and the route goes away.

https://github.com/zejn/bbr_pfifo_interrupts_issue

Hope this helps,
Gasper