* [PATCH net-next v6 0/2] net: macb: implement ethtool set channels count operation
@ 2026-11-13 17:32 Théo Lebrun
2026-11-13 17:32 ` [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
0 siblings, 2 replies; 10+ messages in thread
From: Théo Lebrun @ 2026-11-13 17:32 UTC (permalink / raw)
To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun
Add support for changing the active number of queues.
Tested on Mobileye EyeQ5.
Now that context swapping landed, update this series and use the feature
on ethtool_ops.set_channels() requests.
- We move num_queues from bp to ctx.
- All readers of the bp->num_queues value now need to pick between:
bp->max_num_queues / bp->configured_num_queues / bp->ctx->num_queues.
- The context swapping in set_channels can fail because
netif_set_real_num_queues can fail. Update macb_context_swap_end()
to allow taking NULL as new context argument. If that is received,
it restarts the interface with the same context as before.
This highlights a few callbacks that are pretty broken if accessed while
interface is down, because they access hardware registers while runtime
suspended. Most importantly stats and taprio. We don't attempt to fix
them in this series! I am mentioning this here to prompt Sashiko. :)
Have a nice day,
Thanks,
Théo
[0]: https://lore.kernel.org/netdev/20261113-macb-caps-v1-0-42e0984b2f09@bootlin.com/
---
Changes in v6:
- Rebase onto latest net-next/main (573af8015529). This means rebasing
onto context swapping, which is now used on set_channels requests.
- Move the QUEUE_DISABLE caps patch out into a separate series [0].
- Drop kunit tests. Thanks Jakub.
- Fix the TAPRIO gate test if qdisc is destroyed while the interface is
down (because bp->taprio_gated_queues was stall in that case).
Thanks Nicolai.
- Link to v5: https://patch.msgid.link/20261016-macb-set-channels-v5-0-90ce0c0b8e75@bootlin.com
Changes in v5:
- New patch: add missing MACB_CAPS_QUEUE_DISABLE to 3 compatibles based
on publicly available datasheets. That means they can benefit from
the set_channels implementation.
- set_channels patch:
- Refuse set_channels if it disables a queue with active TAPRIO.
- Use TBQP.QUEUE_DISABLE bit to also disable transmit; previously we
only disabled the receive engine.
- Migrate code to newly introduced macb_queue_isr_clear() helper.
- Adapt to newly introduced macb_quiesce_start(), where we want to
synchronize_irq() on all queues, even the disabled ones.
- IRQ handler now self-disarms on disabled queues.
- Drop `old_count` unused stack variable.
- Only allow TAPRIO rules on active queues.
- Fix comments style and text in macb_set_channels(); in particular
mention we'll use context-swapping once it is available.
- Distribute SRAM segments patch:
- Introduce GEM_TXQSEGALLOC_MASK(q) and use field_prep|get() macros.
- Introduce capability for this feature, as many publicly available
GEM datasheets don't document the registers.
- Remove upper register support. Lower register is enough to handle 8
queues, which is the max we support.
- Add edge case kunit for num_segments=128.
- Introduce static_assert(MACB_MAX_QUEUES == 8) in
gem_sram_distribute_segments() to signal that code will need
updating if/when the constant changes.
- Update To/Cc list: drop Nicolas and Claudiu, add Conor and Nicolai.
- Rebase onto latest net-next/main (6b3e73774815).
- Link to v4: https://patch.msgid.link/20260317-macb-set-channels-v4-0-1bd4f4ffcfca@bootlin.com
Changes in v4:
- macb_set_channels(): drop `count == old_count` check, it is redundant
with both netlink and ioctl codepaths. See early returns in
ethnl_set_channels() and ethtool_set_channels(). Discussion @ V2.
- Rebase to latest net-next (8737d7194d6d); nothing to report.
- Link to v3: https://patch.msgid.link/20260313-macb-set-channels-v3-0-c66b7781ddb8@bootlin.com
Changes in v3:
- Makefile: fix compilation of kunit tests. This bug was hidden before
because everything was builtin so it ended up linked together anyway.
Now macb.o is the amalgation of macb_{main,ptp,utils}.o and
macb_test.o is the amalgation of macb_{kunit,utils}.o.
- Kconfig CONFIG_MACB_KUNIT_TEST entry:
- Turn into tristate; no good reason to restrict to bool.
- Drop "depends on MACB". We do not need to compile the full MACB
driver to test gem_sram_distribute_segments().
- Add usual MODULE_*() macros to macb_kunit.c.
- Drop unused init value of `ret` in macb_set_channels().
- Rebase to latest net-next; nothing to report.
- Link to v2: https://patch.msgid.link/20260311-macb-set-channels-v2-0-982693a1f5fc@bootlin.com
Changes in v2:
- Move gem_sram_distribute_segments() from macb_main.c to macb_utils.c
and kunit tests from macb_main.c to macb_kunit.c.
- macb_set_channels():
- Add comment about MACB_CAPS_QUEUE_DISABLE.
- Refuse operation if netif_running(), with code comment.
- Drop useless sanity checks on ch->{combined,rx,tx}_count.
- Add help text to CONFIG_MACB_KUNIT_TEST.
- Fix `checkpatch --max-line-length=80` warnings.
- Rebase onto latest net-next; nothing to report.
- Link to v1: https://lore.kernel.org/r/20260305-macb-set-channels-v1-0-28e3a96a3dc3@bootlin.com
To: Conor Dooley <conor.dooley@microchip.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>
To: David S. Miller <davem@davemloft.net>
To: Eric Dumazet <edumazet@kernel.org>
To: Jakub Kicinski <kuba@kernel.org>
To: Paolo Abeni <pabeni@redhat.com>
To: Russell King <linux@armlinux.org.uk>
Cc: netdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Nicolai Buchwitz <nb@tipi-net.de>
Cc: Paolo Valerio <pvalerio@redhat.com>
Cc: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
Cc: Gregory CLEMENT <gregory.clement@bootlin.com>
Cc: Benoît Monin <benoit.monin@bootlin.com>
Cc: Tawfik Bayouk <tawfik.bayouk@mobileye.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
---
Théo Lebrun (2):
net: macb: implement ethtool_ops.get|set_channels()
net: macb: distribute evenly Tx SRAM segments
drivers/net/ethernet/cadence/macb.h | 21 +-
drivers/net/ethernet/cadence/macb_main.c | 355 ++++++++++++++++++++++++-------
2 files changed, 294 insertions(+), 82 deletions(-)
---
base-commit: 573af801552999954caa8347e4490b04cf4763f0
change-id: 20260305-macb-set-channels-5bf6e07f3270
Best regards,
--
Théo Lebrun <theo.lebrun@bootlin.com>
^ permalink raw reply [flat|nested] 10+ messages in thread* [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() 2026-11-13 17:32 [PATCH net-next v6 0/2] net: macb: implement ethtool set channels count operation Théo Lebrun @ 2026-11-13 17:32 ` Théo Lebrun 2026-11-15 19:39 ` Nicolai Buchwitz 2026-11-17 19:43 ` netdev-bot+sashiko 2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun 1 sibling, 2 replies; 10+ messages in thread From: Théo Lebrun @ 2026-11-13 17:32 UTC (permalink / raw) To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun bp->num_queues is the total number of queues and is constant from probe. MACB/GEM does not know about Rx/Tx specific queues; it only has combined queues. Move bp->num_queues to ctx->num_queues as it now belongs to the currently active context (bp->ctx). Also introduce bp->max_num_queues which takes the current role of bp->num_queues and allow `0 < ctx->num_queues <= bp->max_num_queues`. Lastly introduce bp->configured_num_queues to remember queue count across interface down+up cycles. With all that, implement the .set_channels() operation. It ends up only being an assignment to bp->configured_num_queues plus a call into netif_set_real_num_queues(). However if the interface is running, we use a context swap window to safely realloc on the fly. Note we refuse the operation if any TAPRIO rules are in place on queues that would get disabled. If we allow the operation to take place, then a transmission window would be reserved but unusable. Nit: fix an alignment issue inside gem_ethtool_ops which does not deserve its own patch. Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com> --- drivers/net/ethernet/cadence/macb.h | 7 +- drivers/net/ethernet/cadence/macb_main.c | 279 ++++++++++++++++++++++--------- 2 files changed, 206 insertions(+), 80 deletions(-) diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h index b7b4b6a38298..0879fb2b198b 100644 --- a/drivers/net/ethernet/cadence/macb.h +++ b/drivers/net/ethernet/cadence/macb.h @@ -1292,7 +1292,6 @@ struct macb_info { struct platform_device *pdev; struct net_device *netdev; struct macb_or_gem_ops macbgem_ops; - unsigned int num_queues; u32 caps; int rx_bd_rd_prefetch; int tx_bd_rd_prefetch; @@ -1323,6 +1322,7 @@ struct macb_context { unsigned int rx_buffer_size; unsigned int rx_ring_size; unsigned int tx_ring_size; + unsigned int num_queues; struct macb_rxq rxq[MACB_MAX_QUEUES]; struct macb_txq txq[MACB_MAX_QUEUES]; }; @@ -1352,6 +1352,11 @@ struct macb { unsigned int configured_tx_ring_size; struct macb_context *ctx; + unsigned int configured_num_queues; + unsigned int max_num_queues; + /* Queue mask gated by the current TAPRIO schedule (RTNL-protected). */ + u32 taprio_gated_queues; + struct macb_dma_desc *rx_ring_tieoff; dma_addr_t rx_ring_tieoff_dma; diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index 5bd299fc85a9..e7325af4b399 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -504,11 +504,25 @@ static void macb_init_buffers(struct macb *bp) macb_writel(bp, TBQPH, upper_32_bits(txq->ring_dma)); } - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { - rxq = &bp->ctx->rxq[q]; - txq = &bp->ctx->txq[q]; - queue_writel(queue, RBQP, lower_32_bits(rxq->ring_dma)); - queue_writel(queue, TBQP, lower_32_bits(txq->ring_dma)); + for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue) { + if (q < bp->ctx->num_queues) { + rxq = &bp->ctx->rxq[q]; + txq = &bp->ctx->txq[q]; + queue_writel(queue, RBQP, lower_32_bits(rxq->ring_dma)); + queue_writel(queue, TBQP, lower_32_bits(txq->ring_dma)); + } else { + /* macb_set_channels(), which is the only way of writing + * to bp->configured_num_queues, is only allowed if + * MACB_CAPS_QUEUE_DISABLE. + */ + queue_writel(queue, RBQP, MACB_BIT(QUEUE_DISABLE)); + queue_writel(queue, TBQP, MACB_BIT(QUEUE_DISABLE)); + + /* Disable all interrupts */ + queue_writel(queue, IDR, -1); + queue_readl(queue, ISR); + macb_queue_isr_clear(bp, queue, -1); + } } } @@ -695,7 +709,7 @@ static bool macb_tx_all_queues_idle(struct macb *bp) struct macb_txq *txq; unsigned int q; - for (q = 0; q < bp->num_queues; ++q) { + for (q = 0; q < bp->ctx->num_queues; ++q) { txq = &bp->ctx->txq[q]; if (READ_ONCE(txq->head) != READ_ONCE(txq->tail)) return false; @@ -849,7 +863,8 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode, bp->link_up = false; if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; + ++q, ++queue) queue_writel(queue, IDR, bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); @@ -951,7 +966,7 @@ static void gem_shuffle_tx_rings(struct macb *bp) struct macb_queue *queue; unsigned int q; - for (q = 0, queue = bp->queues; q < bp->num_queues; q++, queue++) + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; q++, queue++) gem_shuffle_tx_one_ring(queue); } @@ -995,10 +1010,10 @@ static void macb_mac_link_up(struct phylink_config *config, if (rx_pause) ctrl |= MACB_BIT(PAE); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; + ++q, ++queue) queue_writel(queue, IER, bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); - } } macb_or_gem_writel(bp, NCFGR, ctrl); @@ -2150,12 +2165,12 @@ static void macb_quiesce_start(struct macb *bp) bp->irq_quiesced = true; spin_unlock_irqrestore(&bp->lock, flags); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) + for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue) synchronize_irq(queue->irq); cancel_work_sync(&bp->hresp_err_bh_work); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) { /* Must be done before NAPI is disabled: the task ends with a * napi_enable() call. */ @@ -2183,7 +2198,7 @@ static void macb_quiesce_end(struct macb *bp) unsigned long flags; unsigned int q; - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) { napi_enable(&queue->napi_rx); napi_enable(&queue->napi_tx); } @@ -2202,7 +2217,7 @@ static void macb_hresp_error_task(struct work_struct *work) unsigned int q; u32 ctrl; - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) { queue_writel(queue, IDR, bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP)); @@ -2220,7 +2235,7 @@ static void macb_hresp_error_task(struct work_struct *work) macb_init_buffers(bp); /* Enable interrupts */ - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) + for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) queue_writel(queue, IER, bp->rx_intr_mask | MACB_TX_INT_FLAGS | @@ -2321,7 +2336,9 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id) { struct macb_queue *queue = dev_id; struct macb *bp = queue->bp; + unsigned int num_queues = 0; struct net_device *netdev = bp->netdev; + unsigned int q = queue - bp->queues; bool hresp_err = false; u32 status; @@ -2332,19 +2349,22 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id) spin_lock(&bp->lock); + if (bp->ctx) + num_queues = READ_ONCE(bp->ctx->num_queues); + while (status) { - /* self-disarm while the netdev is closed - * or during context swap + /* self-disarm while the netdev is closed, + * during context swap, or when the current queue is disabled */ - if (unlikely(bp->irq_quiesced)) { + if (unlikely(bp->irq_quiesced || + q >= num_queues)) { queue_writel(queue, IDR, -1); macb_queue_isr_clear(bp, queue, -1); break; } netdev_vdbg(netdev, "queue = %u, isr = 0x%08lx\n", - (unsigned int)(queue - bp->queues), - (unsigned long)status); + q, (unsigned long)status); if (status & bp->rx_intr_mask) { /* There's no point taking any more interrupts @@ -2402,7 +2422,7 @@ static void macb_poll_controller(struct net_device *netdev) unsigned int q; local_irq_save(flags); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) macb_interrupt(netdev->irq, queue); local_irq_restore(flags); } @@ -2845,7 +2865,7 @@ static void gem_free_rx_buffers(struct macb_context *ctx) unsigned int q; int i; - for (q = 0; q < ctx->info->num_queues; ++q) { + for (q = 0; q < ctx->num_queues; ++q) { rxq = &ctx->rxq[q]; if (!rxq->skbuff) @@ -2907,14 +2927,14 @@ static void macb_free(struct macb_context *ctx) ctx->info->macbgem_ops.mog_free_rx_buffers(ctx); txq = &ctx->txq[0]; - size = ctx->info->num_queues * macb_tx_ring_size_per_queue(ctx); + size = ctx->num_queues * macb_tx_ring_size_per_queue(ctx); dma_free_coherent(dev, size, txq->ring, txq->ring_dma); rxq = &ctx->rxq[0]; - size = ctx->info->num_queues * macb_rx_ring_size_per_queue(ctx); + size = ctx->num_queues * macb_rx_ring_size_per_queue(ctx); dma_free_coherent(dev, size, rxq->ring, rxq->ring_dma); - for (q = 0; q < ctx->info->num_queues; ++q) { + for (q = 0; q < ctx->num_queues; ++q) { txq = &ctx->txq[q]; rxq = &ctx->rxq[q]; @@ -2947,7 +2967,7 @@ static int gem_alloc_rx_buffers(struct macb_context *ctx) unsigned int q; int size; - for (q = 0; q < ctx->info->num_queues; ++q) { + for (q = 0; q < ctx->num_queues; ++q) { rxq = &ctx->rxq[q]; size = ctx->rx_ring_size * sizeof(struct sk_buff *); rxq->skbuff = kzalloc(size, GFP_KERNEL); @@ -2981,8 +3001,8 @@ static int macb_alloc_rx_buffers(struct macb_context *ctx) static int macb_alloc(struct macb_context *ctx) { - unsigned int num_queues = ctx->info->num_queues; struct device *dev = &ctx->info->pdev->dev; + unsigned int num_queues = ctx->num_queues; dma_addr_t tx_dma, rx_dma; struct macb_txq *txq; struct macb_rxq *rxq; @@ -3051,6 +3071,7 @@ static int macb_alloc(struct macb_context *ctx) } static struct macb_context *macb_context_alloc(struct macb *bp, + unsigned int num_queues, unsigned int mtu, unsigned int rx_ring_size, unsigned int tx_ring_size) @@ -3063,6 +3084,7 @@ static struct macb_context *macb_context_alloc(struct macb *bp, return ERR_PTR(-ENOMEM); ctx->info = &bp->info; + ctx->num_queues = num_queues; ctx->rx_buffer_size = macb_rx_buffer_size(bp, mtu); ctx->rx_ring_size = rx_ring_size; ctx->tx_ring_size = tx_ring_size; @@ -3110,7 +3132,7 @@ static int gem_init_rings(struct macb_context *ctx) int err; int i; - for (q = 0; q < ctx->info->num_queues; ++q) { + for (q = 0; q < ctx->num_queues; ++q) { txq = &ctx->txq[q]; for (i = 0; i < ctx->tx_ring_size; i++) { desc = macb_tx_desc(ctx, q, i); @@ -3173,7 +3195,7 @@ static void macb_reset_hw(struct macb *bp) gem_writel(bp, PBUFRXCUT, 0); /* Disable all interrupts */ - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) { queue_writel(queue, IDR, -1); queue_readl(queue, ISR); macb_queue_isr_clear(bp, queue, -1); @@ -3263,7 +3285,8 @@ static void macb_configure_dma(struct macb *bp) buffer_size = bp->ctx->rx_buffer_size / RX_BUFFER_MULTIPLE; if (macb_is_gem(&bp->info)) { dmacfg = gem_readl(bp, DMACFG) & ~GEM_BF(RXBS, -1L); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; + ++q, ++queue) { if (q) queue_writel(queue, RBQS, buffer_size); else @@ -3312,7 +3335,7 @@ static void macb_context_swap_start(struct macb *bp) netif_tx_disable(bp->netdev); /* Now that everything is stopped, clear DQL. */ - for (q = 0; q < bp->num_queues; ++q) + for (q = 0; q < bp->ctx->num_queues; ++q) netdev_tx_reset_queue(netdev_get_tx_queue(bp->netdev, q)); /* Safe to call outside bp->lock because bp->irq_quiesced ensures the IRQ @@ -3328,7 +3351,7 @@ static void macb_context_swap_start(struct macb *bp) ctrl = macb_readl(bp, NCR); macb_writel(bp, NCR, ctrl & ~(MACB_BIT(RE) | MACB_BIT(TE))); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) { queue_writel(queue, IDR, -1); queue_readl(queue, ISR); macb_queue_isr_clear(bp, queue, -1); @@ -3351,16 +3374,21 @@ static void macb_context_swap_end(struct macb *bp, lockdep_assert_held(&bp->mac_cfg_lock); - /* Swap contexts & give buffer pointers to HW. */ + /* Swap contexts & give buffer pointers to HW. + * !new_ctx means that the swap failed and we keep the old context. + */ + + if (new_ctx) { + old_ctx = bp->ctx; + WRITE_ONCE(bp->ctx, new_ctx); + wmb(); /* ensure IRQ enabled below read the new context */ + } - old_ctx = bp->ctx; - WRITE_ONCE(bp->ctx, new_ctx); - wmb(); /* ensure IRQ enabled below read the new context */ macb_init_buffers(bp); /* Start NAPI, HW Tx/Rx and software Tx. */ - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) { napi_enable(&queue->napi_rx); napi_enable(&queue->napi_tx); } @@ -3373,7 +3401,8 @@ static void macb_context_swap_end(struct macb *bp, macb_configure_dma(bp); if (bp->link_up) { - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; + ++q, ++queue) { queue_writel(queue, IER, bp->rx_intr_mask | MACB_TX_INT_FLAGS | @@ -3397,8 +3426,10 @@ static void macb_context_swap_end(struct macb *bp, /* Free old context. */ - macb_free(old_ctx); - kfree(old_ctx); + if (new_ctx) { + macb_free(old_ctx); + kfree(old_ctx); + } } static void macb_init_hw(struct macb *bp) @@ -3572,7 +3603,7 @@ static int macb_open(struct net_device *netdev) if (err < 0) return err; - ctx = macb_context_alloc(bp, netdev->mtu, + ctx = macb_context_alloc(bp, bp->configured_num_queues, netdev->mtu, bp->configured_rx_ring_size, bp->configured_tx_ring_size); if (IS_ERR(ctx)) { @@ -3637,7 +3668,7 @@ static int macb_close(struct net_device *netdev) * and tx_error_task wake them up. */ netif_tx_stop_all_queues(netdev); - for (q = 0; q < bp->num_queues; ++q) + for (q = 0; q < bp->ctx->num_queues; ++q) netdev_tx_reset_queue(netdev_get_tx_queue(netdev, q)); phylink_stop(bp->phylink); @@ -3673,7 +3704,8 @@ static int macb_change_mtu(struct net_device *netdev, int new_mtu) if (bp->caps & MACB_CAPS_MACB_IS_EMAC) return -EBUSY; - new_ctx = macb_context_alloc(bp, new_mtu, + new_ctx = macb_context_alloc(bp, bp->configured_num_queues, + new_mtu, bp->configured_rx_ring_size, bp->configured_tx_ring_size); if (IS_ERR(new_ctx)) @@ -3726,7 +3758,7 @@ static void gem_update_stats(struct macb *bp) } idx = GEM_STATS_LEN; - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) + for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue) for (i = 0, stat = &queue->stats.first; i < QUEUE_STATS_LEN; ++i, ++stat) bp->ethtool_stats[idx++] = *stat; } @@ -3777,7 +3809,8 @@ static void gem_get_ethtool_stats(struct net_device *netdev, spin_lock_irq(&bp->stats_lock); gem_update_stats(bp); memcpy(data, &bp->ethtool_stats, sizeof(u64) - * (GEM_STATS_LEN + QUEUE_STATS_LEN * bp->num_queues)); + * (GEM_STATS_LEN + QUEUE_STATS_LEN * + bp->max_num_queues)); spin_unlock_irq(&bp->stats_lock); } @@ -3787,7 +3820,7 @@ static int gem_get_sset_count(struct net_device *netdev, int sset) switch (sset) { case ETH_SS_STATS: - return GEM_STATS_LEN + bp->num_queues * QUEUE_STATS_LEN; + return GEM_STATS_LEN + bp->max_num_queues * QUEUE_STATS_LEN; default: return -EOPNOTSUPP; } @@ -3806,7 +3839,8 @@ static void gem_get_ethtool_strings(struct net_device *netdev, u32 sset, u8 *p) memcpy(p, gem_statistics[i].stat_string, ETH_GSTRING_LEN); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->max_num_queues; + ++q, ++queue) { for (i = 0; i < QUEUE_STATS_LEN; i++) ethtool_sprintf(&p, "q%u_%s", q, queue_statistics[i].stat_string); } @@ -4165,8 +4199,9 @@ static int macb_set_ringparam(struct net_device *netdev, } if (running) { - new_ctx = macb_context_alloc(bp, netdev->mtu, - new_rx_size, new_tx_size); + new_ctx = macb_context_alloc(bp, bp->configured_num_queues, + netdev->mtu, new_rx_size, + new_tx_size); if (IS_ERR(new_ctx)) return PTR_ERR(new_ctx); @@ -4513,7 +4548,7 @@ static u32 gem_get_rx_ring_count(struct net_device *netdev) { struct macb *bp = netdev_priv(netdev); - return bp->num_queues; + return bp->max_num_queues; } static int gem_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd, @@ -4544,6 +4579,7 @@ static int gem_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd, static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd) { struct macb *bp = netdev_priv(netdev); + unsigned int num_queues = bp->configured_num_queues; int ret; if (!(netdev->hw_features & NETIF_F_NTUPLE)) @@ -4551,8 +4587,8 @@ static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd) switch (cmd->cmd) { case ETHTOOL_SRXCLSRLINS: - if ((cmd->fs.location >= bp->max_tuples) - || (cmd->fs.ring_cookie >= bp->num_queues)) { + if (cmd->fs.location >= bp->max_tuples || + cmd->fs.ring_cookie >= num_queues) { ret = -EINVAL; break; } @@ -4570,6 +4606,75 @@ static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd) return ret; } +static void macb_get_channels(struct net_device *netdev, + struct ethtool_channels *ch) +{ + struct macb *bp = netdev_priv(netdev); + + ch->max_combined = bp->max_num_queues; + ch->combined_count = bp->configured_num_queues; +} + +static int macb_set_channels(struct net_device *netdev, + struct ethtool_channels *ch) +{ + struct macb *bp = netdev_priv(netdev); + unsigned int num_queues = ch->combined_count; + bool running = netif_running(bp->netdev); + struct macb_context *new_ctx; + int ret; + + /* MACB_CAPS_QUEUE_DISABLE means that the field QUEUE_DISABLE/BIT0 in + * the per-queue RBQP/TBQP registers disable Tx/Rx queues. If we don't + * have that capability we can have multiple queues but we must always + * run with all enabled. + */ + if (!(bp->caps & MACB_CAPS_QUEUE_DISABLE)) + return -EOPNOTSUPP; + + if (num_queues > bp->max_num_queues) + return -EINVAL; + + if (num_queues == bp->configured_num_queues) + return 0; + + /* If we allow disable a queue which has TAPRIO active, we'll keep a + * transmission window which will be unusable. + */ + if (bp->taprio_gated_queues >> num_queues) { + netdev_err(bp->netdev, "TAPRIO schedule gates queues >= %u; destroy the qdisc before reducing channels\n", + num_queues); + return -EBUSY; + } + + if (running) { + new_ctx = macb_context_alloc(bp, num_queues, netdev->mtu, + bp->configured_rx_ring_size, + bp->configured_tx_ring_size); + if (IS_ERR(new_ctx)) + return PTR_ERR(new_ctx); + + macb_context_swap_start(bp); + } + + ret = netif_set_real_num_queues(bp->netdev, num_queues, num_queues); + if (ret) { + if (running) { + macb_free(new_ctx); + kfree(new_ctx); + macb_context_swap_end(bp, NULL); + } + return ret; + } + + bp->configured_num_queues = num_queues; + + if (running) + macb_context_swap_end(bp, new_ctx); + + return 0; +} + static const struct ethtool_ops macb_ethtool_ops = { .get_regs_len = macb_get_regs_len, .get_regs = macb_get_regs, @@ -4585,6 +4690,8 @@ static const struct ethtool_ops macb_ethtool_ops = { .set_link_ksettings = macb_set_link_ksettings, .get_ringparam = macb_get_ringparam, .set_ringparam = macb_set_ringparam, + .get_channels = macb_get_channels, + .set_channels = macb_set_channels, }; static int macb_get_eee(struct net_device *netdev, struct ethtool_keee *eee) @@ -4619,12 +4726,14 @@ static const struct ethtool_ops gem_ethtool_ops = { .set_link_ksettings = macb_set_link_ksettings, .get_ringparam = macb_get_ringparam, .set_ringparam = macb_set_ringparam, - .get_rxnfc = gem_get_rxnfc, - .set_rxnfc = gem_set_rxnfc, - .get_rx_ring_count = gem_get_rx_ring_count, - .nway_reset = phy_ethtool_nway_reset, + .get_rxnfc = gem_get_rxnfc, + .set_rxnfc = gem_set_rxnfc, + .get_rx_ring_count = gem_get_rx_ring_count, + .nway_reset = phy_ethtool_nway_reset, .get_eee = macb_get_eee, .set_eee = macb_set_eee, + .get_channels = macb_get_channels, + .set_channels = macb_set_channels, }; static int macb_ioctl(struct net_device *netdev, struct ifreq *rq, int cmd) @@ -4765,9 +4874,9 @@ static int macb_taprio_setup_replace(struct net_device *netdev, size_t i, q; int err; - if (conf->num_entries > bp->num_queues) { - netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n", - conf->num_entries, bp->num_queues); + if (conf->num_entries > bp->configured_num_queues) { + netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n", + conf->num_entries, bp->configured_num_queues); return -EINVAL; } @@ -4790,7 +4899,7 @@ static int macb_taprio_setup_replace(struct net_device *netdev, return -EINVAL; } - enst_queue = kzalloc_objs(*enst_queue, bp->num_queues); + enst_queue = kzalloc_objs(*enst_queue, bp->configured_num_queues); if (unlikely(!enst_queue)) return -ENOMEM; @@ -4815,9 +4924,10 @@ static int macb_taprio_setup_replace(struct net_device *netdev, /* gate_mask must not select queues outside the valid queues */ queue_id = order_base_2(entry->gate_mask); - if (queue_id >= bp->num_queues) { - netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n", - i, entry->gate_mask, bp->num_queues); + if (queue_id >= bp->configured_num_queues) { + netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n", + i, entry->gate_mask, + bp->configured_num_queues); err = -EINVAL; goto cleanup; } @@ -4876,11 +4986,12 @@ static int macb_taprio_setup_replace(struct net_device *netdev, /* All validations passed - proceed with hardware configuration */ scoped_guard(spinlock_irqsave, &bp->lock) { /* Disable ENST queues if running before configuring */ - queue_mask = BIT_U32(bp->num_queues) - 1; + queue_mask = BIT_U32(bp->max_num_queues) - 1; gem_writel(bp, ENST_CONTROL, queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET); - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->configured_num_queues; + ++q, ++queue) { /* Configure queue timing registers */ queue_writel(queue, ENST_START_TIME, enst_queue[q].start_time_mask); @@ -4894,6 +5005,9 @@ static int macb_taprio_setup_replace(struct net_device *netdev, gem_writel(bp, ENST_CONTROL, configured_queues); } + /* Remember which queues are gated, to validate .set_channels(). */ + bp->taprio_gated_queues = configured_queues; + netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n", conf->num_entries, hweight32(configured_queues)); @@ -4910,7 +5024,12 @@ static void macb_taprio_destroy(struct net_device *netdev) unsigned int q; netdev_reset_tc(netdev); - queue_mask = BIT_U32(bp->num_queues) - 1; + bp->taprio_gated_queues = 0; + + if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) + return; + + queue_mask = BIT_U32(bp->max_num_queues) - 1; scoped_guard(spinlock_irqsave, &bp->lock) { /* Single disable command for all queues */ @@ -4918,7 +5037,8 @@ static void macb_taprio_destroy(struct net_device *netdev) queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET); /* Clear all queue ENST registers in batch */ - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { + for (q = 0, queue = bp->queues; q < bp->max_num_queues; + ++q, ++queue) { queue_writel(queue, ENST_START_TIME, 0); queue_writel(queue, ENST_ON_TIME, 0); queue_writel(queue, ENST_OFF_TIME, 0); @@ -4936,15 +5056,14 @@ static int macb_setup_taprio(struct net_device *netdev, if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC))) return -EOPNOTSUPP; - /* Check if Device is in runtime suspend */ - if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) { - netdev_err(netdev, "Device is in runtime suspend\n"); - return -EOPNOTSUPP; - } - switch (taprio->cmd) { case TAPRIO_CMD_REPLACE: - err = macb_taprio_setup_replace(netdev, taprio); + if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) { + netdev_err(netdev, "Device is in runtime suspend\n"); + err = -EOPNOTSUPP; + } else { + err = macb_taprio_setup_replace(netdev, taprio); + } break; case TAPRIO_CMD_DESTROY: macb_taprio_destroy(netdev); @@ -5196,7 +5315,7 @@ static int macb_init_dflt(struct platform_device *pdev) * register mapping but we don't want to test the queue index then * compute the corresponding register offset at run time. */ - for (hw_q = 0, q = 0; hw_q < bp->num_queues; ++hw_q) { + for (hw_q = 0, q = 0; hw_q < bp->max_num_queues; ++hw_q) { queue = &bp->queues[q]; queue->bp = bp; spin_lock_init(&queue->tx_ptr_lock); @@ -6308,7 +6427,8 @@ static int macb_probe(struct platform_device *pdev) bp->macb_reg_readl = hw_readl; bp->macb_reg_writel = hw_writel; } - bp->num_queues = num_queues; + bp->max_num_queues = num_queues; + bp->configured_num_queues = num_queues; bp->dma_burst_length = macb_config->dma_burst_length; bp->pclk = pclk; bp->hclk = hclk; @@ -6517,6 +6637,7 @@ static int __maybe_unused macb_suspend(struct device *dev) { struct net_device *netdev = dev_get_drvdata(dev); struct macb *bp = netdev_priv(netdev); + struct macb_context *ctx = bp->ctx; struct in_ifaddr *ifa = NULL; struct macb_queue *queue; struct in_device *idev; @@ -6558,7 +6679,7 @@ static int __maybe_unused macb_suspend(struct device *dev) macb_writel(bp, RBQPH, upper_32_bits(bp->rx_ring_tieoff_dma)); #endif - for (q = 0, queue = bp->queues; q < bp->num_queues; + for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) { /* Disable RX queues */ if (bp->caps & MACB_CAPS_QUEUE_DISABLE) { @@ -6599,7 +6720,7 @@ static int __maybe_unused macb_suspend(struct device *dev) } netif_device_detach(netdev); - for (q = 0, queue = bp->queues; q < bp->num_queues; + for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) { napi_disable(&queue->napi_rx); napi_disable(&queue->napi_tx); @@ -6674,7 +6795,7 @@ static int __maybe_unused macb_resume(struct device *dev) if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) macb_init_buffers(bp); - for (q = 0, queue = bp->queues; q < bp->num_queues; + for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) { if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) { if (macb_is_gem(&bp->info)) -- 2.56.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() 2026-11-13 17:32 ` [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun @ 2026-11-15 19:39 ` Nicolai Buchwitz 2026-11-17 10:20 ` Théo Lebrun 2026-11-17 19:43 ` netdev-bot+sashiko 1 sibling, 1 reply; 10+ messages in thread From: Nicolai Buchwitz @ 2026-11-15 19:39 UTC (permalink / raw) To: Théo Lebrun Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni Hi Théo On 13.11.2026 18:32, Théo Lebrun wrote: > [...] > @@ -3351,16 +3374,21 @@ static void macb_context_swap_end(struct macb > *bp, > > lockdep_assert_held(&bp->mac_cfg_lock); > > - /* Swap contexts & give buffer pointers to HW. */ > + /* Swap contexts & give buffer pointers to HW. > + * !new_ctx means that the swap failed and we keep the old context. > + */ > + > + if (new_ctx) { > + old_ctx = bp->ctx; > + WRITE_ONCE(bp->ctx, new_ctx); > + wmb(); /* ensure IRQ enabled below read the new context */ > + } > > - old_ctx = bp->ctx; > - WRITE_ONCE(bp->ctx, new_ctx); > - wmb(); /* ensure IRQ enabled below read the new context */ > macb_init_buffers(bp); The NULL case reuses the old rings without mog_init_rings(). The HW restarts at descriptor 0 but head/tail don't, so Tx stalls after a failed ethtool -L? macb_hresp_error_task() reinits the rings first. > [...] Regards, Nicolai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() 2026-11-15 19:39 ` Nicolai Buchwitz @ 2026-11-17 10:20 ` Théo Lebrun 0 siblings, 0 replies; 10+ messages in thread From: Théo Lebrun @ 2026-11-17 10:20 UTC (permalink / raw) To: Nicolai Buchwitz Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni Hello Nicolai, On Sun Nov 15, 2026 at 8:39 PM CET, Nicolai Buchwitz wrote: > On 13.11.2026 18:32, Théo Lebrun wrote: >> @@ -3351,16 +3374,21 @@ static void macb_context_swap_end(struct macb >> *bp, >> >> lockdep_assert_held(&bp->mac_cfg_lock); >> >> - /* Swap contexts & give buffer pointers to HW. */ >> + /* Swap contexts & give buffer pointers to HW. >> + * !new_ctx means that the swap failed and we keep the old context. >> + */ >> + >> + if (new_ctx) { >> + old_ctx = bp->ctx; >> + WRITE_ONCE(bp->ctx, new_ctx); >> + wmb(); /* ensure IRQ enabled below read the new context */ >> + } >> >> - old_ctx = bp->ctx; >> - WRITE_ONCE(bp->ctx, new_ctx); >> - wmb(); /* ensure IRQ enabled below read the new context */ >> macb_init_buffers(bp); > > The NULL case reuses the old rings without mog_init_rings(). The HW > restarts at descriptor 0 but head/tail don't, so Tx stalls after a > failed ethtool -L? macb_hresp_error_task() reinits the rings first. Hum, so you are totally right on that. Both Tx (txq->head + txq->tail) and Rx (rxq->tail + rxq->prepared_head) cursors are wrong by being out of sync with the hardware cursors. But the approach taken by macb_hresp_error_task() is wrong: it resets the Tx cursors and Tx descriptors in the existing context but by doing so leak all Tx buffers. Rx buffers aren't leaking because gem_init_rx_ring() sets RX_USED on all slots then refills. Sashiko [0] points out the same issue; I'll wait for the end of the Clashiko [1] embargo period before re-sending, to see what it has to say. --- There can be two solutions to this: - We don't touch the HW cursors RBQPH/TBQPH/RBQP/TBQP by not calling macb_init_buffers() at swap_end. So here the swap operation will have been as close as possible to a no-op where we just stopped the HW for a small window. - We do reset the HW cursors. But that means we need to make sure both rings are in a good shape (which they are not with v6). It isn't a massive amount of work, but it still is more code (that might be buggy). We need to free pending Tx slots or bring them to the front of the ring. For Rx we should consume RX_USED slots. I prefer the first approach. It is simpler and alloc-free (ENOMEM is probably the reason why the set_channels context swap failed when calling netif_set_real_num_queues). [0]: https://sashiko.dev/#/patchset/20261113-macb-set-channels-v6-0-5de59d9e7f02%40bootlin.com?part=1 [1]: https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261113-macb-set-channels-v6-0-5de59d9e7f02%40bootlin.com Thanks, -- Théo Lebrun, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() 2026-11-13 17:32 ` [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun 2026-11-15 19:39 ` Nicolai Buchwitz @ 2026-11-17 19:43 ` netdev-bot+sashiko 1 sibling, 0 replies; 10+ messages in thread From: netdev-bot+sashiko @ 2026-11-17 19:43 UTC (permalink / raw) To: theo.lebrun Cc: conor.dooley, andrew+netdev, davem, edumazet, kuba, pabeni, linux, netdev, linux-kernel, nb, pvalerio, vladimir.kondratiev, gregory.clement, benoit.monin, tawfik.bayouk, thomas.petazzoni Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider. Critical: 0 · High: 0 · Medium: 7 · Low: 0 If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough). - [Medium] macb_taprio_destroy() (drivers/net/ethernet/cadence/macb_main.c) now clears the software… - [Medium] gem_get_rx_ring_count() now returns bp->max_num_queues. - [Medium] When the interface is running, macb_set_channels() calls macb_context_swap_start(bp) before netif_set_real_num_queues(). - [Medium] macb_interrupt() now runs `if (bp->ctx) num_queues = READ_ONCE(bp->ctx->num_queues);` (lines 2352-2353) under bp->lock,… - [Medium] macb_poll_controller() now evaluates `q < bp->ctx->num_queues` on every loop iteration with only local_irq_save(), without bp->lock,… - [Medium] net: macb: at91ether_open() leaves bp->ctx->num_queues uninitialized as 0 - [Medium] net: macb: macb_taprio_setup_replace() skips ENST init on disabled queues -- net: macb: implement ethtool_ops.get|set_channels() This commit moves the queue count into the active context (ctx->num_queues) and adds bp->max_num_queues and bp->configured_num_queues. It implements .get_channels() and .set_channels(), using the context swap window when the interface is running. It also records the TAPRIO-gated queue mask so that .set_channels() can refuse to disable gated queues. > diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c > index 5bd299fc85a9c..e7325af4b399c 100644 > --- a/drivers/net/ethernet/cadence/macb_main.c > +++ b/drivers/net/ethernet/cadence/macb_main.c [ ... ] > @@ -2332,19 +2349,22 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id) > > spin_lock(&bp->lock); > > + if (bp->ctx) > + num_queues = READ_ONCE(bp->ctx->num_queues); > + > while (status) { > - /* self-disarm while the netdev is closed > - * or during context swap > + /* self-disarm while the netdev is closed, > + * during context swap, or when the current queue is disabled > */ > - if (unlikely(bp->irq_quiesced)) { > + if (unlikely(bp->irq_quiesced || > + q >= num_queues)) { [Severity: Medium] Can this read a freed context? bp->ctx is now dereferenced before bp->irq_quiesced is checked. The previous handler never touched bp->ctx while quiesced. macb_close() frees and clears the context outside bp->lock, and the reset_hw error path in macb_open() does the same: macb_close() { ... macb_free(bp->ctx); kfree(bp->ctx); bp->ctx = NULL; ... } The queue IRQs are requested with IRQF_SHARED in macb_init_dflt() and stay registered while the interface is down. The handler can still be entered after the synchronize_irq() in macb_quiesce_start(), for example when another device on the shared line fires and ISR reads non-zero. If that happens between kfree() and the NULL store, ->num_queues is read from freed memory. bp->ctx is also loaded twice with plain loads. If the compiler re-reads it after the closing CPU has stored NULL, would the handler dereference NULL? > queue_writel(queue, IDR, -1); > macb_queue_isr_clear(bp, queue, -1); > break; > } [ ... ] > @@ -2402,7 +2422,7 @@ static void macb_poll_controller(struct net_device *netdev) > unsigned int q; > > local_irq_save(flags); > - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) > + for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) > macb_interrupt(netdev->irq, queue); > local_irq_restore(flags); > } [Severity: Medium] Is it safe to dereference bp->ctx here? The loop bound used to be the constant bp->num_queues. It is now re-read from bp->ctx on every iteration, protected only by local_irq_save(): no bp->lock, no mac_cfg_lock and no RCU. netpoll is not serialized against .set_channels(), .set_ringparam() or ndo_change_mtu. Each of these can swap bp->ctx, and macb_context_swap_end() frees the old context right after publishing the new one: macb_context_swap_end() { ... WRITE_ONCE(bp->ctx, new_ctx); ... mutex_unlock(&bp->mac_cfg_lock); ... if (new_ctx) { macb_free(old_ctx); kfree(old_ctx); } } A CPU in macb_poll_controller() that has already loaded the old bp->ctx can then read ->num_queues from freed memory. If that memory has been reused, could the garbage bound walk bp->queues[] past MACB_MAX_QUEUES and call macb_interrupt() on an out-of-bounds queue? The new .set_channels() path adds another runtime trigger for this swap. [ ... ] > @@ -4513,7 +4548,7 @@ static u32 gem_get_rx_ring_count(struct net_device *netdev) > { > struct macb *bp = netdev_priv(netdev); > > - return bp->num_queues; > + return bp->max_num_queues; > } [Severity: Medium] Should this return bp->configured_num_queues? In the same gem_ethtool_ops table, macb_get_channels() reports combined_count as bp->configured_num_queues. gem_set_rxnfc() also rejects any ring_cookie >= bp->configured_num_queues. Take 4-queue hardware after "ethtool -L ethX combined 2". ETHTOOL_GRXRINGS (ethtool_get_rxrings() in net/ethtool/ioctl.c) still reports 4 RX rings. Rings 2 and 3 have QUEUE_DISABLE written by macb_init_buffers(), and the driver's own rxnfc insert path rejects steering to them with -EINVAL. include/linux/ethtool.h documents this callback as returning the number of RX rings. [ ... ] > +static int macb_set_channels(struct net_device *netdev, > + struct ethtool_channels *ch) > +{ [ ... ] > + if (running) { > + new_ctx = macb_context_alloc(bp, num_queues, netdev->mtu, > + bp->configured_rx_ring_size, > + bp->configured_tx_ring_size); > + if (IS_ERR(new_ctx)) > + return PTR_ERR(new_ctx); > + > + macb_context_swap_start(bp); > + } > + > + ret = netif_set_real_num_queues(bp->netdev, num_queues, num_queues); > + if (ret) { > + if (running) { > + macb_free(new_ctx); > + kfree(new_ctx); > + macb_context_swap_end(bp, NULL); > + } > + return ret; > + } [Severity: Medium] What state is the old context in if netif_set_real_num_queues() fails here? When the queue count grows on a registered netdev, it can return -ENOMEM from netdev_queue_update_kobjects() or net_rx_queue_update_kobjects(). By that point macb_context_swap_start() has done the following, which only makes sense if the old context is about to be discarded: - quiesced NAPI and IRQs - halted Tx and cleared TE/RE - called netdev_tx_reset_queue() on every old queue The old rings can still hold in-flight or completed-but-unreaped Tx descriptors between txq->tail and txq->head. macb_context_swap_end(bp, NULL) then re-activates the old context: macb_context_swap_end(bp, NULL) macb_init_buffers(bp) <- TBQP/RBQP rewound to ring index 0 napi_enable() IER writes, TE/RE re-enabled if link is up Neither mog_init_rings() nor gem_shuffle_tx_rings() is called. The software txq->head/tail and rxq->tail stay mid-ring while the hardware restarts from the ring base. Later, macb_tx_complete() or macb_tx_error_task() reaps the old frames and netdev_tx_completed_queue() reports bytes that DQL no longer counts. Could that trip this check? lib/dynamic_queue_limits.c:dql_completed() { ... BUG_ON(count > num_queued - dql->num_completed); ... } Even without the BUG_ON, wouldn't the mismatched hardware and software ring indices stall Tx/Rx? macb_change_mtu() and macb_set_ringparam() do all of their fallible work before macb_context_swap_start(). This rollback-after-swap path is new in this patch. > + > + bp->configured_num_queues = num_queues; > + > + if (running) > + macb_context_swap_end(bp, new_ctx); > + > + return 0; > +} [ ... ] > @@ -4910,7 +5024,12 @@ static void macb_taprio_destroy(struct net_device *netdev) > unsigned int q; > > netdev_reset_tc(netdev); > - queue_mask = BIT_U32(bp->num_queues) - 1; > + bp->taprio_gated_queues = 0; > + > + if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) > + return; [Severity: Medium] Can this leave ENST gating enabled in hardware while bp->taprio_gated_queues says that no queue is gated? When the device is runtime-suspended, the function returns before writing ENST_CONTROL or the per-queue ENST_START_TIME, ENST_ON_TIME and ENST_OFF_TIME registers. macb_setup_taprio() used to reject every command while runtime-suspended. With this patch only TAPRIO_CMD_REPLACE is rejected: case TAPRIO_CMD_REPLACE: if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) { So DESTROY now succeeds silently while suspended. The commit message does not mention this change. macb_runtime_suspend() only gates clocks, so the ENST registers keep their contents. Only macb_taprio_setup_replace() and macb_taprio_destroy() write ENST_CONTROL. macb_open(), macb_init_hw() and macb_reset_hw() never clear it. The new guard in macb_set_channels() checks only the software mask: if (bp->taprio_gated_queues >> num_queues) { That would let this sequence through: 1. Interface up, with a taprio offload gating queue 3. 2. ip link set down: macb_close() calls pm_runtime_put() and the device autosuspends. 3. tc qdisc del: the mask is cleared and the hardware is left untouched. 4. ethtool -L ethX combined 2: accepted. 5. ip link set up. On open, macb_init_buffers() writes QUEUE_DISABLE to RBQP/TBQP for queue 3 while its ENST window is still enabled. Isn't this the case the commit message describes as refused: "a transmission window would be reserved but unusable"? Separately, this part is a pre-existing issue and not introduced by this patch: the pm_runtime_suspended() check takes no PM reference. The device can therefore suspend between the check and the MMIO writes that follow. > + > + queue_mask = BIT_U32(bp->max_num_queues) - 1; > > scoped_guard(spinlock_irqsave, &bp->lock) { [ ... ] > @@ -3063,6 +3084,7 @@ static struct macb_context *macb_context_alloc(struct macb *bp, > return ERR_PTR(-ENOMEM); > > ctx->info = &bp->info; > + ctx->num_queues = num_queues; > ctx->rx_buffer_size = macb_rx_buffer_size(bp, mtu); > ctx->rx_ring_size = rx_ring_size; > ctx->tx_ring_size = tx_ring_size; [Severity: Medium] Does this leave ctx->num_queues at zero on EMAC? With this patch, macb_context_alloc() is the only place that initializes the field. at91ether_open() does not call macb_context_alloc(); it allocates the context itself: at91ether_open() { ... WRITE_ONCE(bp->ctx, kzalloc_obj(*bp->ctx)); ... bp->ctx->info = &bp->info; ... } As a result, bp->ctx->num_queues stays 0 for as long as an AT91RM9200 EMAC interface is up. That breaks the invariant 0 < ctx->num_queues <= bp->max_num_queues stated in the commit message. Before this patch, the loops below were bounded by bp->num_queues, which is 1 on EMAC. They now run zero times: - macb_suspend(): the WoL loop that writes QUEUE_DISABLE or the tie-off address to RBQP and masks the queue interrupts, and the napi_disable() loop. - macb_resume(): the matching per-queue loop. - macb_reset_hw(): the IDR/ISR clear loop, which macb_suspend() calls when WoL is disabled. On system suspend, queue 0 interrupts are therefore no longer masked by macb_reset_hw(). With WoL enabled, the RX queue is not tied off before RE is turned back on. Should at91ether_open() set bp->ctx->num_queues to 1 (or to bp->configured_num_queues)? Or is there a reason not to use macb_context_alloc() there? [ ... ] > @@ -4876,11 +4986,12 @@ static int macb_taprio_setup_replace(struct net_device *netdev, > /* All validations passed - proceed with hardware configuration */ > scoped_guard(spinlock_irqsave, &bp->lock) { > /* Disable ENST queues if running before configuring */ > - queue_mask = BIT_U32(bp->num_queues) - 1; > + queue_mask = BIT_U32(bp->max_num_queues) - 1; > gem_writel(bp, ENST_CONTROL, > queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET); > > - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) { > + for (q = 0, queue = bp->queues; q < bp->configured_num_queues; > + ++q, ++queue) { > /* Configure queue timing registers */ > queue_writel(queue, ENST_START_TIME, > enst_queue[q].start_time_mask); [Severity: Medium] Should this loop still cover bp->max_num_queues? Other code already does: - The ENST_CONTROL write just above disables all bp->max_num_queues queues. - macb_taprio_destroy() clears ENST_START_TIME, ENST_ON_TIME and ENST_OFF_TIME for all bp->max_num_queues queues. Here, however, enst_queue is sized for bp->configured_num_queues, and only those queues get their timing registers written. Take 4-queue hardware after "ethtool -L ethX combined 2". A later taprio replace leaves queues 2 and 3 with whatever their ENST registers already hold. On a device that never had a schedule, that is the reset value (0x0001FFFF in ENST_ON_TIME). commit 04a943067baa ("net: macb: configure ENST registers for all queues") fixed exactly this: leaving the reset values on unused queues disrupts ENST scheduling of the configured queues, even when ENST_CONTROL enables only the configured ones. Before this patch, every queue below bp->num_queues was written, with zeroes for the unused ones. Would it work to allocate enst_queue with bp->max_num_queues and run this loop up to bp->max_num_queues? The num_entries and queue_id checks would still compare against bp->configured_num_queues. -- Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261113-macb-set-channels-v6-0-5de59d9e7f02%40bootlin.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments 2026-11-13 17:32 [PATCH net-next v6 0/2] net: macb: implement ethtool set channels count operation Théo Lebrun 2026-11-13 17:32 ` [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun @ 2026-11-13 17:32 ` Théo Lebrun 2026-11-13 18:18 ` Théo Lebrun ` (2 more replies) 1 sibling, 3 replies; 10+ messages in thread From: Théo Lebrun @ 2026-11-13 17:32 UTC (permalink / raw) To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun GEM has registers to configure the Tx SRAM segments distribution across queues. The reset value is apprioriate (even spread) but we need to care if/when number of active queues is modified (or if we inherited unevenly initialised hardware from bootloader). To distribute segments, we take as input the number of queues (ctx->num_queues) and the number of segments (found inside DCFG6). Its output is a number of segments for each queue, formatted as powers-of-two (eg 2 for queue 0 means it has 2^2=4 segments). Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com> --- drivers/net/ethernet/cadence/macb.h | 14 ++++++ drivers/net/ethernet/cadence/macb_main.c | 76 +++++++++++++++++++++++++++++++- 2 files changed, 88 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h index 0879fb2b198b..cc57071c869a 100644 --- a/drivers/net/ethernet/cadence/macb.h +++ b/drivers/net/ethernet/cadence/macb.h @@ -194,6 +194,10 @@ /* Screener Type 2 match registers */ #define GEM_SCRT2 0x540 +/* Tx queue segment allocation */ +#define GEM_TXQSEGALLOC_LOWER 0x05A0 +#define GEM_TXQSEGALLOC_UPPER 0x05A4 + /* EtherType registers */ #define GEM_ETHT 0x06E0 @@ -545,6 +549,8 @@ #define GEM_PBUF_CUTTHRU_SIZE 1 #define GEM_DAW64_OFFSET 23 #define GEM_DAW64_SIZE 1 +#define GEM_SEGMENTS_BIT_SIZE_OFFSET 16 +#define GEM_SEGMENTS_BIT_SIZE_SIZE 3 /* Bitfields in DCFG8. */ #define GEM_T1SCR_OFFSET 24 @@ -657,6 +663,13 @@ #define GEM_CMPCEN_OFFSET 30 /* Compare C Enable */ #define GEM_CMPCEN_SIZE 1 +/* Bitfields in TXQSEGALLOC (Q0 at 2:0, Q1 at 6:4, etc.) */ +#define GEM_TXQSEGALLOC_MASK(q_) \ + ({ \ + typeof(q_) q__ = (q_); \ + GENMASK(4 * (q__) + 2, 4 * (q__)); \ + }) + /* Bitfields in ETHT */ #define GEM_ETHTCMP_OFFSET 0 /* EtherType compare value */ #define GEM_ETHTCMP_SIZE 16 @@ -785,6 +798,7 @@ #define MACB_CAPS_USRIO_HAS_MII BIT(26) #define MACB_CAPS_USRIO_HAS_REFCLK_SOURCE BIT(27) #define MACB_CAPS_USRIO_HAS_TSUCLK_SOURCE BIT(28) +#define MACB_CAPS_TXQSEGALLOC BIT(29) /* LSO settings */ #define MACB_LSO_UFO_ENABLE 0x01 diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index e7325af4b399..dc1624815f71 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -3432,6 +3432,63 @@ static void macb_context_swap_end(struct macb *bp, } } +/* + * Distribute evenly available segments across queues. The computation is + * complex because (1) segments are counted in powers of two and (2) a queue + * can only use up to 8 segments. There are four types of cases: + * - Sharing all segments equally is doable. Take num_queues=4 and + * num_segments=16. Each queue will get 2^2=4 segments. + * - Sharing all segments is doable. Take num_queues=5 and num_segments=16. + * Three queues will get 2^2=4 segments and two will get 2^1=2 segments. + * - Sharing all segments is not doable because not enough queues are + * available. Take num_queues=1 and num_segments=16; queue 0 can only have 8 + * segments. + * - Sharing all segments is not doable because not enough segments are + * available. Take num_queues=4 and num_segments=2. + * + * We start by computing the power each queue will have. For num_queues=5 and + * num_segments=16, each queue will have at least 2^1 segments. That leaves us + * with remaining_segments=6. If we increase the power for a queue, we get a + * delta of 2 (2^2-2^1). The first three queues will therefore be advantaged + * and each have 2^2 segments. The remaining 2 queues will only have 2^1 + * segments. + */ +static u32 gem_sram_distribute_segments(unsigned int num_queues, + unsigned int num_segments) +{ + unsigned int pow, remaining_segments, i; + unsigned int num_advantaged_queues = 0; + u32 val = 0; + + /* When growing the max # of queues, make sure to return a u64 here and + * to write into GEM_TXQSEGALLOC_UPPER. + */ + static_assert(MACB_MAX_QUEUES == 8); + + /* pow=0 for all queues. ilog2(0) is dangerous. */ + if (num_queues >= num_segments) + return 0; + + pow = min_t(unsigned int, ilog2(num_segments / num_queues), 3); + remaining_segments = num_segments - num_queues * (1U << pow); + + /* We can only distribute remaining segments if (1) there are remaining + * segments and (2) we did not reach the max segments per queue (2^3). + */ + if (remaining_segments != 0 && pow != 3) { + unsigned int delta = (1U << (pow + 1)) - (1U << pow); + + num_advantaged_queues = remaining_segments / delta; + } + + for (i = 0; i < num_advantaged_queues; i++) + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow + 1); + for (i = num_advantaged_queues; i < num_queues; i++) + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow); + + return val; +} + static void macb_init_hw(struct macb *bp) { u32 config; @@ -3465,6 +3522,20 @@ static void macb_init_hw(struct macb *bp) if (bp->caps & MACB_CAPS_JUMBO) bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK; + /* Distribute Tx SRAM segments evenly based on active number of queues. + * TXQSEGALLOC is only documented for some GEM integrations. + */ + if (bp->caps & MACB_CAPS_TXQSEGALLOC) { + unsigned int num_segments; + u32 val; + + num_segments = 1U << GEM_BFEXT(SEGMENTS_BIT_SIZE, + gem_readl(bp, DCFG6)); + val = gem_sram_distribute_segments(bp->ctx->num_queues, + num_segments); + gem_writel(bp, TXQSEGALLOC_LOWER, val); + } + macb_configure_dma(bp); /* Enable RX partial store and forward and set watermark */ @@ -6234,6 +6305,7 @@ static const struct macb_config zynqmp_config = { .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO | MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH | + MACB_CAPS_TXQSEGALLOC | MACB_CAPS_USRIO_HAS_MII, .dma_burst_length = 16, .init = init_reset_optional, @@ -6285,7 +6357,7 @@ static const struct macb_config versal_config = { .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO | MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH | MACB_CAPS_NEED_TSUCLK | MACB_CAPS_QUEUE_DISABLE | - MACB_CAPS_QBV | + MACB_CAPS_QBV | MACB_CAPS_TXQSEGALLOC | MACB_CAPS_USRIO_HAS_MII, .dma_burst_length = 16, .init = init_reset_optional, @@ -6296,7 +6368,7 @@ static const struct macb_config versal_config = { static const struct macb_config eyeq5_config = { .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO | MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_QUEUE_DISABLE | - MACB_CAPS_NO_LSO | MACB_CAPS_EEE, + MACB_CAPS_NO_LSO | MACB_CAPS_EEE | MACB_CAPS_TXQSEGALLOC, .dma_burst_length = 16, .init = eyeq5_init, .jumbo_max_len = 10240, -- 2.56.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments 2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun @ 2026-11-13 18:18 ` Théo Lebrun 2026-11-15 19:40 ` Nicolai Buchwitz 2026-11-17 19:43 ` netdev-bot+sashiko 2 siblings, 0 replies; 10+ messages in thread From: Théo Lebrun @ 2026-11-13 18:18 UTC (permalink / raw) To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun On Fri Nov 13, 2026 at 6:32 PM CET, Théo Lebrun wrote: > +/* > + * Distribute evenly available segments across queues. The computation is > + * complex because (1) segments are counted in powers of two and (2) a queue > + * can only use up to 8 segments. There are four types of cases: > + * - Sharing all segments equally is doable. Take num_queues=4 and > + * num_segments=16. Each queue will get 2^2=4 segments. > + * - Sharing all segments is doable. Take num_queues=5 and num_segments=16. > + * Three queues will get 2^2=4 segments and two will get 2^1=2 segments. > + * - Sharing all segments is not doable because not enough queues are > + * available. Take num_queues=1 and num_segments=16; queue 0 can only have 8 > + * segments. > + * - Sharing all segments is not doable because not enough segments are > + * available. Take num_queues=4 and num_segments=2. > + * > + * We start by computing the power each queue will have. For num_queues=5 and > + * num_segments=16, each queue will have at least 2^1 segments. That leaves us > + * with remaining_segments=6. If we increase the power for a queue, we get a > + * delta of 2 (2^2-2^1). The first three queues will therefore be advantaged > + * and each have 2^2 segments. The remaining 2 queues will only have 2^1 > + * segments. > + */ > +static u32 gem_sram_distribute_segments(unsigned int num_queues, > + unsigned int num_segments) > +{ > + unsigned int pow, remaining_segments, i; > + unsigned int num_advantaged_queues = 0; > + u32 val = 0; > + > + /* When growing the max # of queues, make sure to return a u64 here and > + * to write into GEM_TXQSEGALLOC_UPPER. > + */ > + static_assert(MACB_MAX_QUEUES == 8); > + > + /* pow=0 for all queues. ilog2(0) is dangerous. */ > + if (num_queues >= num_segments) > + return 0; > + > + pow = min_t(unsigned int, ilog2(num_segments / num_queues), 3); > + remaining_segments = num_segments - num_queues * (1U << pow); > + > + /* We can only distribute remaining segments if (1) there are remaining > + * segments and (2) we did not reach the max segments per queue (2^3). > + */ > + if (remaining_segments != 0 && pow != 3) { > + unsigned int delta = (1U << (pow + 1)) - (1U << pow); > + > + num_advantaged_queues = remaining_segments / delta; > + } > + > + for (i = 0; i < num_advantaged_queues; i++) > + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow + 1); > + for (i = num_advantaged_queues; i < num_queues; i++) > + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow); > + > + return val; > +} This implementation doesn't use all segments if num_segments > 8*num_queues. Reported by Nicolai and discussed here [0]. I don't think any hardware lands in that case, but let's fix the bug. I will let some time pass before iterating. [0]: https://lore.kernel.org/all/1113dac19771ff7881b60bfdd14b0af6@tipi-net.de/ Thanks, -- Théo Lebrun, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments 2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun 2026-11-13 18:18 ` Théo Lebrun @ 2026-11-15 19:40 ` Nicolai Buchwitz 2026-11-17 10:23 ` Théo Lebrun 2026-11-17 19:43 ` netdev-bot+sashiko 2 siblings, 1 reply; 10+ messages in thread From: Nicolai Buchwitz @ 2026-11-15 19:40 UTC (permalink / raw) To: Théo Lebrun Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni On 13.11.2026 18:32, Théo Lebrun wrote: > GEM has registers to configure the Tx SRAM segments distribution across > queues. The reset value is apprioriate (even spread) but we need to > care if/when number of active queues is modified (or if we inherited > unevenly initialised hardware from bootloader). > > To distribute segments, we take as input the number of queues > (ctx->num_queues) and the number of segments (found inside DCFG6). > Its output is a number of segments for each queue, formatted as > powers-of-two (eg 2 for queue 0 means it has 2^2=4 segments). > > Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com> > --- > drivers/net/ethernet/cadence/macb.h | 14 ++++++ > drivers/net/ethernet/cadence/macb_main.c | 76 > +++++++++++++++++++++++++++++++- > [...] > static void macb_init_hw(struct macb *bp) > { > u32 config; > @@ -3465,6 +3522,20 @@ static void macb_init_hw(struct macb *bp) > if (bp->caps & MACB_CAPS_JUMBO) > bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK; > > + /* Distribute Tx SRAM segments evenly based on active number of > queues. > + * TXQSEGALLOC is only documented for some GEM integrations. > + */ > + if (bp->caps & MACB_CAPS_TXQSEGALLOC) { > + unsigned int num_segments; > + u32 val; > + > + num_segments = 1U << GEM_BFEXT(SEGMENTS_BIT_SIZE, > + gem_readl(bp, DCFG6)); > + val = gem_sram_distribute_segments(bp->ctx->num_queues, > + num_segments); > + gem_writel(bp, TXQSEGALLOC_LOWER, val); This only runs on open, not on a context swap. So ethtool -L on a running interface keeps the old split until the next down/up? > [...] Regards, Nicolai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments 2026-11-15 19:40 ` Nicolai Buchwitz @ 2026-11-17 10:23 ` Théo Lebrun 0 siblings, 0 replies; 10+ messages in thread From: Théo Lebrun @ 2026-11-17 10:23 UTC (permalink / raw) To: Nicolai Buchwitz Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel, Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin, Tawfik Bayouk, Thomas Petazzoni Hello Nicolai, On Sun Nov 15, 2026 at 8:40 PM CET, Nicolai Buchwitz wrote: > On 13.11.2026 18:32, Théo Lebrun wrote: >> GEM has registers to configure the Tx SRAM segments distribution across >> queues. The reset value is apprioriate (even spread) but we need to >> care if/when number of active queues is modified (or if we inherited >> unevenly initialised hardware from bootloader). >> >> To distribute segments, we take as input the number of queues >> (ctx->num_queues) and the number of segments (found inside DCFG6). >> Its output is a number of segments for each queue, formatted as >> powers-of-two (eg 2 for queue 0 means it has 2^2=4 segments). >> >> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com> >> --- >> drivers/net/ethernet/cadence/macb.h | 14 ++++++ >> drivers/net/ethernet/cadence/macb_main.c | 76 >> +++++++++++++++++++++++++++++++- > >> [...] > >> static void macb_init_hw(struct macb *bp) >> { >> u32 config; >> @@ -3465,6 +3522,20 @@ static void macb_init_hw(struct macb *bp) >> if (bp->caps & MACB_CAPS_JUMBO) >> bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK; >> >> + /* Distribute Tx SRAM segments evenly based on active number of >> queues. >> + * TXQSEGALLOC is only documented for some GEM integrations. >> + */ >> + if (bp->caps & MACB_CAPS_TXQSEGALLOC) { >> + unsigned int num_segments; >> + u32 val; >> + >> + num_segments = 1U << GEM_BFEXT(SEGMENTS_BIT_SIZE, >> + gem_readl(bp, DCFG6)); >> + val = gem_sram_distribute_segments(bp->ctx->num_queues, >> + num_segments); >> + gem_writel(bp, TXQSEGALLOC_LOWER, val); > > This only runs on open, not on a context swap. So ethtool -L on a > running interface keeps the old split until the next down/up? Agreed! How I'll solve this: - Make gem_sram_distribute_segments() self contained: it does the bp->caps check, figures out num_queues+num_segments on its own and does the writel(val, TXQSEGALLOC_LOWER) at the end. - With that, we can call gem_sram_distribute_segments(bp) once at macb_init_hw() and once at swap_end. Thanks, -- Théo Lebrun, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments 2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun 2026-11-13 18:18 ` Théo Lebrun 2026-11-15 19:40 ` Nicolai Buchwitz @ 2026-11-17 19:43 ` netdev-bot+sashiko 2 siblings, 0 replies; 10+ messages in thread From: netdev-bot+sashiko @ 2026-11-17 19:43 UTC (permalink / raw) To: theo.lebrun Cc: conor.dooley, andrew+netdev, davem, edumazet, kuba, pabeni, linux, netdev, linux-kernel, nb, pvalerio, vladimir.kondratiev, gregory.clement, benoit.monin, tawfik.bayouk, thomas.petazzoni Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider. Critical: 0 · High: 1 · Medium: 1 · Low: 0 If you believe none of the issues are legitimate, please reply to each issue of Medium severity or higher and explain why the AI is wrong. If any of them are legitimate and you plan to respin, please let patchwork know by including "pw-bot: cr" as a separate line at the end of your reply (one such reply per series is enough). - [High] The commit message says TXQSEGALLOC must be re-applied "if/when number of active queues is modified", but the patch only programs it… - [Medium] gem_sram_distribute_segments() can build a TXQSEGALLOC value whose total segment count is larger than the num_segments that… -- net: macb: distribute evenly Tx SRAM segments This patch programs the GEM TXQSEGALLOC register from macb_init_hw(). It spreads the Tx SRAM segments advertised in DCFG6 across the active queues, with each queue's share encoded as a power of two. The new MACB_CAPS_TXQSEGALLOC capability is enabled for ZynqMP, Versal and EyeQ5. > diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c > index e7325af4b399c..dc1624815f71e 100644 > --- a/drivers/net/ethernet/cadence/macb_main.c > +++ b/drivers/net/ethernet/cadence/macb_main.c > @@ -3432,6 +3432,63 @@ static void macb_context_swap_end(struct macb *bp, [ ... ] > +static u32 gem_sram_distribute_segments(unsigned int num_queues, > + unsigned int num_segments) > +{ [ ... ] > + /* pow=0 for all queues. ilog2(0) is dangerous. */ > + if (num_queues >= num_segments) > + return 0; [ ... ] > + for (i = 0; i < num_advantaged_queues; i++) > + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow + 1); > + for (i = num_advantaged_queues; i < num_queues; i++) > + val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow); > + > + return val; > +} [Severity: Medium] Can the value built by gem_sram_distribute_segments() add up to more segments than the num_segments read from DCFG6? A field value of 0 still means 2^0 = 1 segment. There seem to be two ways to over-allocate. First, when num_queues > num_segments, the early return gives every active queue 1 segment. The comment above uses num_queues=4 and num_segments=2 as an example, which claims 4 segments out of 2. Second, the loops only fill fields for i < num_queues. macb_init_hw() passes bp->ctx->num_queues, not the hardware queue count. The fields for hardware queues from ctx->num_queues up to bp->max_num_queues stay at 0. Each of those still claims 1 segment, on top of an active split that already uses every segment. With 4 hardware queues and 16 segments: num_queues=2: 8 + 8 + 1 + 1 = 18 num_queues=3: 8 + 4 + 4 + 1 = 17 On Versal and EyeQ5 (MACB_CAPS_QUEUE_DISABLE), this happens after ethtool -L while the interface is down, followed by an open. The reset value is an even split over all hardware queues, so it adds up to the available segments. If the GEM requires the total to stay within the DCFG6 count, could this give queues overlapping or out-of-range Tx SRAM regions? > @@ -3465,6 +3522,20 @@ static void macb_init_hw(struct macb *bp) > if (bp->caps & MACB_CAPS_JUMBO) > bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK; > > + /* Distribute Tx SRAM segments evenly based on active number of queues. > + * TXQSEGALLOC is only documented for some GEM integrations. > + */ > + if (bp->caps & MACB_CAPS_TXQSEGALLOC) { > + unsigned int num_segments; > + u32 val; > + > + num_segments = 1U << GEM_BFEXT(SEGMENTS_BIT_SIZE, > + gem_readl(bp, DCFG6)); > + val = gem_sram_distribute_segments(bp->ctx->num_queues, > + num_segments); > + gem_writel(bp, TXQSEGALLOC_LOWER, val); > + } [Severity: High] The commit message says the distribution needs care "if/when number of active queues is modified". Is that case covered when the queue count changes on a running interface? Only macb_open() and the resume path call macb_init_hw(). Running ethtool -L ethX combined N while the interface is up goes through: macb_set_channels() macb_context_swap_start() macb_context_swap_end(bp, new_ctx) macb_init_buffers() macb_configure_dma() macb_context_swap_end() installs a context with the new num_queues and turns RE/TE back on. Nothing on this path rewrites GEM_TXQSEGALLOC_LOWER. The Tx SRAM split stays sized for the old queue count until the next down/up or suspend/resume. Two examples: Open with 1 queue. q0 gets 8 segments and the other fields stay at 0 (1 segment each). If the interface then grows to N queues while up, each new queue runs with a single segment while TXPBMS is set in DMACFG. Going from 2 queues to 1 while up leaves half of the SRAM assigned to the disabled queue. Versal and EyeQ5 set both MACB_CAPS_QUEUE_DISABLE and MACB_CAPS_TXQSEGALLOC, so this path is reachable there. Could the write be moved into a helper and also called from macb_context_swap_end() under bp->lock, next to macb_configure_dma()? [ ... ] -- Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/20261113-macb-set-channels-v6-0-5de59d9e7f02%40bootlin.com ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2026-11-17 19:43 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-13 17:32 [PATCH net-next v6 0/2] net: macb: implement ethtool set channels count operation Théo Lebrun 2026-11-13 17:32 ` [PATCH net-next v6 1/2] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun 2026-11-15 19:39 ` Nicolai Buchwitz 2026-11-17 10:20 ` Théo Lebrun 2026-11-17 19:43 ` netdev-bot+sashiko 2026-11-13 17:32 ` [PATCH net-next v6 2/2] net: macb: distribute evenly Tx SRAM segments Théo Lebrun 2026-11-13 18:18 ` Théo Lebrun 2026-11-15 19:40 ` Nicolai Buchwitz 2026-11-17 10:23 ` Théo Lebrun 2026-11-17 19:43 ` netdev-bot+sashiko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox