[PATCH net 0/2] Fix page fragment handling when PAGE_SIZE > 4K

[PATCH net 0/2] Fix page fragment handling when PAGE_SIZE > 4K

[Dimitri Daskalakis]

From: Dimitri Daskalakis <daskald@meta.com>

FBNIC operates on fixed size descriptors (4K). When the OS supports pages
larger than 4K, we fragment the page across multiple descriptors.

While performance testing, I found several issues with our page fragment
handling, resulting in low throughput and potential RX stalls.

Dimitri Daskalakis (2):
  eth: fbnic: Account for page fragments when updating BDQ tail
  eth: fbnic: Fix debugfs output for BDQ's with page frags

 drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c | 2 +-
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c    | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

-- 
2.52.0

[PATCH net 1/2] eth: fbnic: Account for page fragments when updating BDQ tail

[Dimitri Daskalakis]

From: Dimitri Daskalakis <daskald@meta.com>

FBNIC supports fixed size buffers of 4K. When PAGE_SIZE > 4K, we
fragment the page across multiple descriptors (FBNIC_BD_FRAG_COUNT).
When refilling the BDQ, the correct number of entries are populated,
but tail was only incremented by one. So on a system with 64K pages,
HW would get one descriptor refilled for every 16 we populate.

Additionally, we program the ring size in the HW when enabling the BDQ.
This was not accounting for page fragments, so on systems with 64K pages,
the HW used 1/16th of the ring.

Fixes: 0cb4c0a13723 ("eth: fbnic: Implement Rx queue alloc/start/stop/free")
Signed-off-by: Dimitri Daskalakis <daskald@meta.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 9fb91d4f3971..9cd85a0d0c3a 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -927,7 +927,7 @@ static void fbnic_fill_bdq(struct fbnic_ring *bdq)
 		/* Force DMA writes to flush before writing to tail */
 		dma_wmb();
 
-		writel(i, bdq->doorbell);
+		writel(i * FBNIC_BD_FRAG_COUNT, bdq->doorbell);
 	}
 }
 
@@ -2564,7 +2564,7 @@ static void fbnic_enable_bdq(struct fbnic_ring *hpq, struct fbnic_ring *ppq)
 	hpq->tail = 0;
 	hpq->head = 0;
 
-	log_size = fls(hpq->size_mask);
+	log_size = fls(hpq->size_mask) + ilog2(FBNIC_BD_FRAG_COUNT);
 
 	/* Store descriptor ring address and size */
 	fbnic_ring_wr32(hpq, FBNIC_QUEUE_BDQ_HPQ_BAL, lower_32_bits(hpq->dma));
@@ -2576,7 +2576,7 @@ static void fbnic_enable_bdq(struct fbnic_ring *hpq, struct fbnic_ring *ppq)
 	if (!ppq->size_mask)
 		goto write_ctl;
 
-	log_size = fls(ppq->size_mask);
+	log_size = fls(ppq->size_mask) + ilog2(FBNIC_BD_FRAG_COUNT);
 
 	/* Add enabling of PPQ to BDQ control */
 	bdq_ctl |= FBNIC_QUEUE_BDQ_CTL_PPQ_ENABLE;
-- 
2.52.0

[PATCH net 2/2] eth: fbnic: Fix debugfs output for BDQ's with page frags

[Dimitri Daskalakis]

From: Dimitri Daskalakis <daskald@meta.com>

The rings size_mask represents the number of pages, so we need to
determine the number of page frags when dumping the descriptors.

Fixes: df04373b0dab ("eth fbnic: Add debugfs hooks for tx/rx rings")
Signed-off-by: Dimitri Daskalakis <daskald@meta.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
 drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
index 08270db2dee8..3c4563c8f403 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_debugfs.c
@@ -197,7 +197,7 @@ static int fbnic_dbg_bdq_desc_seq_show(struct seq_file *s, void *v)
 		return 0;
 	}
 
-	for (i = 0; i <= ring->size_mask; i++) {
+	for (i = 0; i < (ring->size_mask + 1) * FBNIC_BD_FRAG_COUNT; i++) {
 		u64 bd = le64_to_cpu(ring->desc[i]);
 
 		seq_printf(s, "%04x %#04llx %#014llx\n", i,
-- 
2.52.0

Re: [PATCH net 0/2] Fix page fragment handling when PAGE_SIZE > 4K

[Jakub Kicinski]

On Tue, 24 Mar 2026 12:51:21 -0700 Dimitri Daskalakis wrote:
> FBNIC operates on fixed size descriptors (4K). When the OS supports pages
> larger than 4K, we fragment the page across multiple descriptors.
> 
> While performance testing, I found several issues with our page fragment
> handling, resulting in low throughput and potential RX stalls.

Google AI reviewer points out that we will also be toast if someone
configures the ring size to just 16 descriptors. Can you check what
is the min size that doesn't wedge w/ 64kB pages and let's just set
the min to that instead of the current 16?

Re: [PATCH net 0/2] Fix page fragment handling when PAGE_SIZE > 4K

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Tue, 24 Mar 2026 12:51:21 -0700 you wrote:
> From: Dimitri Daskalakis <daskald@meta.com>
> 
> FBNIC operates on fixed size descriptors (4K). When the OS supports pages
> larger than 4K, we fragment the page across multiple descriptors.
> 
> While performance testing, I found several issues with our page fragment
> handling, resulting in low throughput and potential RX stalls.
> 
> [...]

Here is the summary with links:
  - [net,1/2] eth: fbnic: Account for page fragments when updating BDQ tail
    https://git.kernel.org/netdev/net/c/b38c55320bf8
  - [net,2/2] eth: fbnic: Fix debugfs output for BDQ's with page frags
    https://git.kernel.org/netdev/net/c/f3567dd428b2

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html