From mboxrd@z Thu Jan  1 00:00:00 1970
From: Eric Dumazet <eric.dumazet@gmail.com>
Subject: Re: pegged softirq and NAPI race (?)
Date: Tue, 18 Sep 2018 14:21:34 -0700
Message-ID: <324c4a85-1749-9129-bf03-73a49d9a4059@gmail.com>
References: <A782704A-DF97-4E85-B10A-D2268A67DFD7@fb.com>
 <CANn89iLAPvPn2C9J+sSHsQ5EeF0hq47iMiiAfwoQkdZurQiyzA@mail.gmail.com>
 <a946c4d2-5e29-b1d4-3af6-8657caeebc11@fb.com>
 <CANn89iJTEMjYepKbr-8pmk0i03d9D+CfDFLPx+J=fqZivDJ9zQ@mail.gmail.com>
 <0FD562CC-CDE9-43C8-9623-B42AC7A208C8@fb.com>
 <CANn89i+kg=tE9aRYoFb+9ws35CCJJQz9Yc9i=9w=vDj=Epvj=A@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Cc: Alexei Starovoitov <ast@fb.com>, netdev <netdev@vger.kernel.org>,
        Jeff Kirsher <jeffrey.t.kirsher@intel.com>,
        Alexander Duyck <alexander.h.duyck@intel.com>,
        michael.chan@broadcom.com, kernel-team <Kernel-team@fb.com>
To: Eric Dumazet <edumazet@google.com>, songliubraving@fb.com
Return-path: <netdev-owner@vger.kernel.org>
Received: from mail-pl1-f194.google.com ([209.85.214.194]:39069 "EHLO
        mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1729902AbeISC4E (ORCPT
        <rfc822;netdev@vger.kernel.org>); Tue, 18 Sep 2018 22:56:04 -0400
Received: by mail-pl1-f194.google.com with SMTP id w14-v6so1585193plp.6
        for <netdev@vger.kernel.org>; Tue, 18 Sep 2018 14:21:37 -0700 (PDT)
In-Reply-To: <CANn89i+kg=tE9aRYoFb+9ws35CCJJQz9Yc9i=9w=vDj=Epvj=A@mail.gmail.com>
Content-Language: en-US
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>



On 09/18/2018 02:13 PM, Eric Dumazet wrote:
> On Tue, Sep 18, 2018 at 1:37 PM Song Liu <songliubraving@fb.com> wrote:
>>
> 
>> Looks like a patch like the following fixes the issue for ixgbe. But I
>> cannot explain it yet.
>>
>> Does this ring a bell?
> 
> I dunno, it looks like the NIC is  generating an interrupt while it should not,
> and constantly sets NAPI_STATE_MISSED.
> 
> Or maybe we need to properly check napi_complete_done() return value.
> 
> diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_main.c
> b/drivers/net/ethernet/intel/ixgb/ixgb_main.c
> index d3e72d0f66ef428b08e4bd88508e05b734bc43a4..c4c565c982a98a5891603cedcdcb72dc1c401813
> 100644
> --- a/drivers/net/ethernet/intel/ixgb/ixgb_main.c
> +++ b/drivers/net/ethernet/intel/ixgb/ixgb_main.c
> @@ -1773,8 +1773,8 @@ ixgb_clean(struct napi_struct *napi, int budget)
>         ixgb_clean_rx_irq(adapter, &work_done, budget);
> 
>         /* If budget not fully consumed, exit the polling mode */
> -       if (work_done < budget) {
> -               napi_complete_done(napi, work_done);
> +       if (work_done < budget &&
> +           napi_complete_done(napi, work_done)) {
>                 if (!test_bit(__IXGB_DOWN, &adapter->flags))
>                         ixgb_irq_enable(adapter);
>         }
> 


ixgbe patch would be :

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 604282f03d236e4358fc91e64d8ba00a9b36cb8c..80d00aecb6e3e3e950ce6309bfe3639953dd73d9 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -3196,12 +3196,12 @@ int ixgbe_poll(struct napi_struct *napi, int budget)
                return budget;
 
        /* all work done, exit the polling mode */
-       napi_complete_done(napi, work_done);
-       if (adapter->rx_itr_setting & 1)
-               ixgbe_set_itr(q_vector);
-       if (!test_bit(__IXGBE_DOWN, &adapter->state))
-               ixgbe_irq_enable_queues(adapter, BIT_ULL(q_vector->v_idx));
-
+       if (napi_complete_done(napi, work_done)) {
+               if (adapter->rx_itr_setting & 1)
+                       ixgbe_set_itr(q_vector);
+               if (!test_bit(__IXGBE_DOWN, &adapter->state))
+                       ixgbe_irq_enable_queues(adapter, BIT_ULL(q_vector->v_idx));
+       }
        return min(work_done, budget - 1);
 }