Netdev List
 help / color / mirror / Atom feed
From: Dimitri Daskalakis <dimitri.daskalakis1@gmail.com>
To: "David S. Miller" <davem@davemloft.net>
Cc: "Alexander Duyck" <alexanderduyck@fb.com>,
	"Jakub Kicinski" <kuba@kernel.org>,
	kernel-team@meta.com, "Andrew Lunn" <andrew+netdev@lunn.ch>,
	"Eric Dumazet" <edumazet@kernel.org>,
	"Paolo Abeni" <pabeni@redhat.com>,
	"Willem de Bruijn" <willemb@google.com>,
	"Simon Horman" <horms@kernel.org>,
	"Björn Töpel" <bjorn@kernel.org>,
	"Bobby Eshleman" <bobbyeshleman@meta.com>,
	"Mina Almasry" <almasrymina@google.com>,
	"Dimitri Daskalakis" <daskald@meta.com>,
	netdev@vger.kernel.org,
	"Dimitri Daskalakis" <dimitri.daskalakis1@gmail.com>
Subject: [PATCH net 4/4] eth: fbnic: wait for both TWQs of a Tx queue to go idle
Date: Fri, 20 Nov 2026 19:32:15 -0800	[thread overview]
Message-ID: <20261121033215.1020506-5-dimitri.daskalakis1@gmail.com> (raw)
In-Reply-To: <20261121033215.1020506-1-dimitri.daskalakis1@gmail.com>

From: Dimitri Daskalakis <daskald@meta.com>

TWQ_IDLE, TQS_IDLE and TDE_IDLE have a bit per Tx work queue. Bit 2q
is for TWQ0, and bit 2q+1 is for TWQ1. fbnic_wait_queue_idle() only
tests bit q, which belongs to queue q/2. Thus, the wait covers a TWQ
of the wrong queue (except when q = 0).

Test both of the queue's TWQ bits in those registers.

This bug was found by AI and verified in QEMU.

Forced TWQ1 of Tx queue 3 to never read idle in QEMU, with rx 2 tx 2
(queue 3 is the XDP Tx queue of rxq 1's vector), and bound devmem to
rxq 1:
  before: bind succeeds
  after:  bind fails with ETIMEDOUT:
    wait for queue Tx3 idle failed 0x0800(0): ffffff7f (mask: 000000c0)

Fixes: da43127a8edc ("eth: fbnic: support queue ops / zero-copy Rx")
Signed-off-by: Dimitri Daskalakis <daskald@meta.com>
---
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 32 ++++++++++++--------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 62b67940c13e..fe769c33b117 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -2532,36 +2532,44 @@ int fbnic_wait_all_queues_idle(struct fbnic_dev *fbd, bool may_fail)
 	return err;
 }
 
+struct fbnic_queue_idle_reg {
+	u32 reg;
+	u8 bits_per_q;
+};
+
 static int
 fbnic_wait_queue_idle(struct fbnic_net *fbn, bool rx, unsigned int idx)
 {
-	static const unsigned int tx_regs[] = {
-		FBNIC_QM_TWQ_IDLE(0), FBNIC_QM_TQS_IDLE(0),
-		FBNIC_QM_TDE_IDLE(0), FBNIC_QM_TCQ_IDLE(0),
+	static const struct fbnic_queue_idle_reg tx_regs[] = {
+		{ FBNIC_QM_TWQ_IDLE(0), 2 }, { FBNIC_QM_TQS_IDLE(0), 2 },
+		{ FBNIC_QM_TDE_IDLE(0), 2 }, { FBNIC_QM_TCQ_IDLE(0), 1 },
 	}, rx_regs[] = {
-		FBNIC_QM_HPQ_IDLE(0), FBNIC_QM_PPQ_IDLE(0),
-		FBNIC_QM_RCQ_IDLE(0),
+		{ FBNIC_QM_HPQ_IDLE(0), 1 }, { FBNIC_QM_PPQ_IDLE(0), 1 },
+		{ FBNIC_QM_RCQ_IDLE(0), 1 },
 	};
+	const struct fbnic_queue_idle_reg *regs;
 	struct fbnic_dev *fbd = fbn->fbd;
 	unsigned int val, mask, off;
-	const unsigned int *regs;
 	unsigned int reg_cnt;
 	int i, err;
 
 	regs = rx ? rx_regs : tx_regs;
 	reg_cnt = rx ? ARRAY_SIZE(rx_regs) : ARRAY_SIZE(tx_regs);
 
-	off = idx / 32;
-	mask = BIT(idx % 32);
-
 	for (i = 0; i < reg_cnt; i++) {
-		err = read_poll_timeout_atomic(fbnic_rd32, val, val & mask,
+		unsigned int bit = idx * regs[i].bits_per_q;
+
+		off = bit / 32;
+		mask = GENMASK(regs[i].bits_per_q - 1, 0) << (bit % 32);
+
+		err = read_poll_timeout_atomic(fbnic_rd32, val,
+					       (val & mask) == mask,
 					       2, 500000, false,
-					       fbd, regs[i] + off);
+					       fbd, regs[i].reg + off);
 		if (err) {
 			netdev_err(fbd->netdev,
 				   "wait for queue %s%d idle failed 0x%04x(%d): %08x (mask: %08x)\n",
-				   rx ? "Rx" : "Tx", idx, regs[i] + off, i,
+				   rx ? "Rx" : "Tx", idx, regs[i].reg + off, i,
 				   val, mask);
 			return err;
 		}
-- 
2.52.1


      parent reply	other threads:[~2026-11-21  3:32 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-21  3:32 [PATCH net 0/4] eth: fbnic: queue stop fixes Dimitri Daskalakis
2026-11-21  3:32 ` [PATCH net 1/4] eth: fbnic: restart queues in place when fbnic_queue_stop() fails Dimitri Daskalakis
2026-11-21  3:32 ` [PATCH net 2/4] net: add netif_disable_subqueue() Dimitri Daskalakis
2026-11-21  3:32 ` [PATCH net 3/4] eth: fbnic: wait for in-flight xmit in fbnic_queue_stop() Dimitri Daskalakis
2026-11-21  3:32 ` Dimitri Daskalakis [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121033215.1020506-5-dimitri.daskalakis1@gmail.com \
    --to=dimitri.daskalakis1@gmail.com \
    --cc=alexanderduyck@fb.com \
    --cc=almasrymina@google.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=bjorn@kernel.org \
    --cc=bobbyeshleman@meta.com \
    --cc=daskald@meta.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=horms@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox