Netdev List
 help / color / mirror / Atom feed
From: Dimitri Daskalakis <dimitri.daskalakis1@gmail.com>
To: "David S. Miller" <davem@davemloft.net>
Cc: "Alexander Duyck" <alexanderduyck@fb.com>,
	"Jakub Kicinski" <kuba@kernel.org>,
	kernel-team@meta.com, "Andrew Lunn" <andrew+netdev@lunn.ch>,
	"Eric Dumazet" <edumazet@kernel.org>,
	"Paolo Abeni" <pabeni@redhat.com>,
	"Willem de Bruijn" <willemb@google.com>,
	"Simon Horman" <horms@kernel.org>,
	"Björn Töpel" <bjorn@kernel.org>,
	"Bobby Eshleman" <bobbyeshleman@meta.com>,
	"Mina Almasry" <almasrymina@google.com>,
	"Dimitri Daskalakis" <daskald@meta.com>,
	netdev@vger.kernel.org,
	"Dimitri Daskalakis" <dimitri.daskalakis1@gmail.com>
Subject: [PATCH net 1/4] eth: fbnic: restart queues in place when fbnic_queue_stop() fails
Date: Fri, 20 Nov 2026 19:32:12 -0800	[thread overview]
Message-ID: <20261121033215.1020506-2-dimitri.daskalakis1@gmail.com> (raw)
In-Reply-To: <20261121033215.1020506-1-dimitri.daskalakis1@gmail.com>

From: Dimitri Daskalakis <daskald@meta.com>

If the idle wait in fbnic_queue_stop() times out, the vector is
restarted, which resets head/tail to 0. NAPI then re-walks stale
completions and dereferences Tx buffers that were already freed.

We could flush the rings before restarting, but that risks in-flight DMA
and completions landing in rings if the HW truly does not stop.

Instead, set the enable bits back without resetting anything, so the HW
resumes from where it stopped.

This bug was found by AI and reproduced on HW.

This is not something we expect to hit in production, but we injected
idle failures with a test patch to verify the behavior if it ever
happened. Devmem was bound to rxq 5 under traffic, with a strict IOMMU:
  HW is idled, but waiting for queues to idle fails:
    before: BUG: kernel NULL pointer dereference, address: 000000000000002e
            RIP: 0010:fbnic_clean_twq0+0xa8/0x290
    after:  bind fails with ETIMEDOUT, rxq 5 keeps receiving (5/5)
  HW never stops (rings left enabled, and wait skipped):
    flush:  AMD-Vi: Event logged [IO_PAGE_FAULT ... flags=0x0020] (4/4)
    after:  no IOMMU faults, rxq 5 keeps receiving (10/10)
25/25 injected runs clean after, across Rx and Tx traffic and
non-combined channels.

Fixes: da43127a8edc ("eth: fbnic: support queue ops / zero-copy Rx")
Signed-off-by: Dimitri Daskalakis <daskald@meta.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 50 ++++++++++++++++++--
 1 file changed, 47 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 381831b7e108..c0fe89425430 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -2326,6 +2326,50 @@ fbnic_nv_disable(struct fbnic_net *fbn, struct fbnic_napi_vector *nv)
 	fbnic_wrfl(fbn->fbd);
 }
 
+static void fbnic_reenable_twq(struct fbnic_ring *twq, u32 csr)
+{
+	u32 twq_ctl;
+
+	if (!twq->size_mask)
+		return;
+
+	twq_ctl = fbnic_ring_rd32(twq, csr);
+	fbnic_ring_wr32(twq, csr, twq_ctl | FBNIC_QUEUE_TWQ_CTL_ENABLE);
+}
+
+static void
+fbnic_nv_reenable(struct fbnic_net *fbn, struct fbnic_napi_vector *nv)
+{
+	int j, t;
+
+	for (t = 0; t < nv->txt_count; t++) {
+		struct fbnic_q_triad *qt = &nv->qt[t];
+
+		if (qt->cmpl.size_mask) {
+			fbnic_ring_wr32(&qt->cmpl, FBNIC_QUEUE_TIM_MASK, 0);
+			fbnic_ring_wr32(&qt->cmpl, FBNIC_QUEUE_TCQ_CTL,
+					FBNIC_QUEUE_TCQ_CTL_ENABLE);
+		}
+		fbnic_reenable_twq(&qt->sub1, FBNIC_QUEUE_TWQ1_CTL);
+		fbnic_reenable_twq(&qt->sub0, FBNIC_QUEUE_TWQ0_CTL);
+	}
+
+	for (j = 0; j < nv->rxt_count; j++, t++) {
+		struct fbnic_q_triad *qt = &nv->qt[t];
+		u32 bdq_ctl;
+
+		fbnic_ring_wr32(&qt->cmpl, FBNIC_QUEUE_RIM_MASK, 0);
+		fbnic_ring_wr32(&qt->cmpl, FBNIC_QUEUE_RCQ_CTL,
+				FBNIC_QUEUE_RCQ_CTL_ENABLE);
+
+		bdq_ctl = fbnic_ring_rd32(&qt->sub0, FBNIC_QUEUE_BDQ_CTL);
+		fbnic_ring_wr32(&qt->sub0, FBNIC_QUEUE_BDQ_CTL,
+				bdq_ctl | FBNIC_QUEUE_BDQ_CTL_ENABLE);
+	}
+
+	fbnic_wrfl(fbn->fbd);
+}
+
 void fbnic_dbg_down(struct fbnic_net *fbn)
 {
 	int i;
@@ -3092,9 +3136,6 @@ static void __fbnic_nv_restart(struct fbnic_net *fbn,
 	struct fbnic_dev *fbd = fbn->fbd;
 	int i;
 
-	fbnic_nv_enable(fbn, nv);
-	fbnic_nv_fill(nv);
-
 	napi_enable_locked(&nv->napi);
 	fbnic_nv_irq_enable(nv);
 	fbnic_wr32(fbd, FBNIC_INTR_SET(nv->v_idx / 32), BIT(nv->v_idx % 32));
@@ -3125,6 +3166,8 @@ static int fbnic_queue_start(struct net_device *dev,
 
 	memcpy(real, qmem, sizeof(*real));
 
+	fbnic_nv_enable(fbn, nv);
+	fbnic_nv_fill(nv);
 	__fbnic_nv_restart(fbn, nv);
 
 	return 0;
@@ -3170,6 +3213,7 @@ static int fbnic_queue_stop(struct net_device *dev, void *qmem, int idx)
 	return 0;
 
 err_restart:
+	fbnic_nv_reenable(fbn, nv);
 	__fbnic_nv_restart(fbn, nv);
 	return err;
 }
-- 
2.52.1


  reply	other threads:[~2026-11-21  3:32 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-21  3:32 [PATCH net 0/4] eth: fbnic: queue stop fixes Dimitri Daskalakis
2026-11-21  3:32 ` Dimitri Daskalakis [this message]
2026-11-21  3:32 ` [PATCH net 2/4] net: add netif_disable_subqueue() Dimitri Daskalakis
2026-11-21  3:32 ` [PATCH net 3/4] eth: fbnic: wait for in-flight xmit in fbnic_queue_stop() Dimitri Daskalakis
2026-11-21  3:32 ` [PATCH net 4/4] eth: fbnic: wait for both TWQs of a Tx queue to go idle Dimitri Daskalakis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121033215.1020506-2-dimitri.daskalakis1@gmail.com \
    --to=dimitri.daskalakis1@gmail.com \
    --cc=alexanderduyck@fb.com \
    --cc=almasrymina@google.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=bjorn@kernel.org \
    --cc=bobbyeshleman@meta.com \
    --cc=daskald@meta.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=horms@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox