Netdev List
 help / color / mirror / Atom feed
From: Joe Damato <joe@dama.to>
To: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com>
Cc: Michael Chan <michael.chan@broadcom.com>,
	netdev@vger.kernel.org, andrew+netdev@lunn.ch,
	davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
	pabeni@redhat.com, horms@kernel.org, pavan.chebbi@broadcom.com,
	linux-kernel@vger.kernel.org,
	Selvin Xavier <selvin.xavier@broadcom.com>
Subject: Re: [PATCH net v5 0/4] bnxt_en: Make RING FREE more robust
Date: Tue, 17 Nov 2026 08:10:29 -0800	[thread overview]
Message-ID: <avx8da+htUim/x9i@devvm20253.cco0.facebook.com> (raw)
In-Reply-To: <CAH-L+nOyNgAUMCF74OE+oU2gSCaV_tVcgm2QQFvjge10A+qAYg@mail.gmail.com>

On Thu, Nov 12, 2026 at 09:37:55PM +0530, Kalesh Anakkur Purayil wrote:
> Hi Michael/ Joe,
> 
> On Wed, Nov 11, 2026 at 4:34 AM Michael Chan <michael.chan@broadcom.com>
> wrote:
> 
> > On Tue, Nov 10, 2026 at 2:13 PM Joe Damato <joe@dama.to> wrote:
> > >
> > > On Tue, Nov 10, 2026 at 10:14:50AM -0800, Michael Chan wrote:
> > > > I'm copying our RoCE developers Selvin and Kalesh.  I think that if
> > > > you call bnxt_re_suspend() before you disable DMA, nothing special
> > > > needs to be done because the chip is still functioning.  It will go
> > > > through the normal removal process.
> > >
> > > I'm probably missing something but I think that the ring frees can only
> > fail
> > > in __bnxt_close_nic, where rtnl and the netdev instance lock are held.
> > DMA has
> > > to be stopped before bnxt_free_mem. bnxt_re_suspend() takes both locks,
> > so it
> > > can't be called.
> > >
> > > I think there's two options:
> > >
> > > 1. Stop DMA as now, but queue a work item to run bnxt_ulp_stop without
> > the
> > > locks. bnxt_re_suspend would need to treat BNXT_STATE_DMA_STOPPED like
> > > FW_FATAL_COND and take the detach path.
> > >
> > > or
> > >
> > > 2. Call bnxt_ulp_stop() in bnxt_reset() before taking the locks, the way
> > > bnxt_fw_reset() does, and bnxt_ulp_start() after a successful reopen.
> > That
> > > covers the TX timeout reset, which is where all the ring free failves
> > I've
> > > seen come from and it doesn't need any bnxt_re change, but it restarts
> > RoCE on
> > > every TX timeout and it wouldn't cover a case where the user triggers
> > > close/open.
> > >
> >
> > I have a slight preference for option 2 because TX timeout ->
> > bnxt_reset() should be rare and stopping RoCE briefly even if all the
> > HWRM_RING_FREE calls succeed should be ok.  We don't want to stop RoCE
> > unnecessarily during a user-initiated close/open though (failure
> > should be very rare).  So we may still need option 1 to cover the
> > other rare failure cases.
> >
> > Let's wait for Selvin's and Kalesh's feedback.
> >
> 
> Yes, we are okay with option 2.

One more question before I put together v6.

bnxt_reset() is also called for the silent reset on a VF: the
VF_CFG_CHANGE async event sets BNXT_RESET_TASK_SILENT_SP_EVENT, and
bnxt_sp_task() calls bnxt_reset(bp, true). With the ULP patch as planned,
RoCE on the VF would be stopped and restarted every time the PF changes
that VF's config.

Not sure how often config changes happen with RoCE in use on VFs? If not
often, then treating silent and non-silent resets the same should be OK.
The cost is a brief RoCE restart on the VF for every config change.

Let me know what you think.

      parent reply	other threads:[~2026-11-17 16:10 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-06 19:14 [PATCH net v5 0/4] bnxt_en: Make RING FREE more robust Joe Damato
2026-11-06 19:14 ` [PATCH net v5 1/4] bnxt_en: return the RING_FREE status to callers Joe Damato
2026-11-07 19:58   ` netdev-bot+sashiko
2026-11-09 17:45     ` Joe Damato
2026-11-09 18:26       ` Michael Chan
2026-11-06 19:14 ` [PATCH net v5 2/4] bnxt_en: check HWRM response if completion never arrives Joe Damato
2026-11-07 19:58   ` netdev-bot+sashiko
2026-11-09 18:05     ` Joe Damato
2026-11-06 19:14 ` [PATCH net v5 3/4] bnxt_en: stop DMA before releasing rings the firmware did not free Joe Damato
2026-11-07 19:58   ` netdev-bot+sashiko
2026-11-09 19:52     ` Joe Damato
2026-11-06 19:14 ` [PATCH net v5 4/4] bnxt_en: refuse to open a device with stopped DMA Joe Damato
2026-11-07 19:58   ` netdev-bot+sashiko
2026-11-09 20:01     ` Joe Damato
2026-11-09 20:12 ` [PATCH net v5 0/4] bnxt_en: Make RING FREE more robust Joe Damato
2026-11-10  0:13   ` Michael Chan
2026-11-10 17:39     ` Joe Damato
2026-11-10 18:14       ` Michael Chan
2026-11-10 22:13         ` Joe Damato
2026-11-10 23:04           ` Michael Chan
     [not found]             ` <CAH-L+nOyNgAUMCF74OE+oU2gSCaV_tVcgm2QQFvjge10A+qAYg@mail.gmail.com>
2026-11-17 16:10               ` Joe Damato [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avx8da+htUim/x9i@devvm20253.cco0.facebook.com \
    --to=joe@dama.to \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=horms@kernel.org \
    --cc=kalesh-anakkur.purayil@broadcom.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=michael.chan@broadcom.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=pavan.chebbi@broadcom.com \
    --cc=selvin.xavier@broadcom.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox