* [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation
@ 2026-11-18 20:53 Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Théo Lebrun @ 2026-11-18 20:53 UTC (permalink / raw)
To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun
Add support for changing the active number of queues.
Tested on Mobileye EyeQ5.
If the interface is running, use context swapping to realloc rings on
the fly. To avoid introducing bugs with stale ENST config, we start by
fixing how ENST config gets applied in the first patch.
One pretty significant change to our context swapping implementation:
the set_channels callback needs to be able to fail in the context
swapping window. If it failed, then we keep using the old context and
don't reconfigure HW. From its standpoint, it got disabled then
re-enabled with no change to its rings.
I tried *not fixing* TAPRIO/ENST but that was hard, so instead I preface
this series with a commit reworking how ENST works. We now handle
close/open and refuse set_channels if a queue-to-be-disabled had an
ENST schedule attached.
Have a nice day,
Thanks,
Théo
---
Changes in v7:
- Patch 1 (new ENST patch):
- Preface the series by a patch fixing TAPRIO ENST config across
interface flap and context swap operations.
- It does so by storing a cached ENST config in bp and restoring it
on context swap and link-up. Previously we could only write the
ENST registers on TAPRIO callbacks.
- Patch 2 (ethtool set_channels implementation):
- Fix context swap rollback: if the window failed (set_channels is the
first context swap that can fail), then we need to *not* write HW
ring cursors and *not* reset DQL.
- macb_interrupt(): avoid NULL ptr deref when reading bp->ctx fields.
- macb_poll_controller(): avoid bp->ctx NULL ptr deref by using
bp->configured_num_queues rather than bp->ctx->num_queues.
- gem_get_rx_ring_count(): use bp->configured_num_queues rather than
bp->max_num_queues.
- at91ether_open(): assign `bp->ctx->num_queues = 1` to avoid breaking
shared code that iterates on queues (suspend PM ops mostly).
- Patch 3 (Tx SRAM segment distribution):
- DCFG6 register has 4 bits to tell us the # of segments, not 3.
- The config of the # of segments per queue is 3 bits (not 2): fix
distribution algorithm to allow going up to the limit. I don't
know of any hardware on which this could have an incidence.
- SRAM segment distribution is also done at context swap end, for
this new set_channels callback.
- Rebase onto latest net-next/main (d6816d9a93b6); tiny conflict inside
macb_config caps.
- Link to v6: https://patch.msgid.link/20261113-macb-set-channels-v6-0-5de59d9e7f02@bootlin.com
Changes in v6:
- Rebase onto latest net-next/main (573af8015529). This means rebasing
onto context swapping, which is now used on set_channels requests.
- Move the QUEUE_DISABLE caps patch out into a separate series.
https://lore.kernel.org/netdev/20261113-macb-caps-v1-0-42e0984b2f09@bootlin.com/
- Drop kunit tests. Thanks Jakub.
- Fix the TAPRIO gate test if qdisc is destroyed while the interface is
down (because bp->taprio_gated_queues was stall in that case).
Thanks Nicolai.
- Link to v5: https://patch.msgid.link/20261016-macb-set-channels-v5-0-90ce0c0b8e75@bootlin.com
Changes in v5:
- New patch: add missing MACB_CAPS_QUEUE_DISABLE to 3 compatibles based
on publicly available datasheets. That means they can benefit from
the set_channels implementation.
- set_channels patch:
- Refuse set_channels if it disables a queue with active TAPRIO.
- Use TBQP.QUEUE_DISABLE bit to also disable transmit; previously we
only disabled the receive engine.
- Migrate code to newly introduced macb_queue_isr_clear() helper.
- Adapt to newly introduced macb_quiesce_start(), where we want to
synchronize_irq() on all queues, even the disabled ones.
- IRQ handler now self-disarms on disabled queues.
- Drop `old_count` unused stack variable.
- Only allow TAPRIO rules on active queues.
- Fix comments style and text in macb_set_channels(); in particular
mention we'll use context-swapping once it is available.
- Distribute SRAM segments patch:
- Introduce GEM_TXQSEGALLOC_MASK(q) and use field_prep|get() macros.
- Introduce capability for this feature, as many publicly available
GEM datasheets don't document the registers.
- Remove upper register support. Lower register is enough to handle 8
queues, which is the max we support.
- Add edge case kunit for num_segments=128.
- Introduce static_assert(MACB_MAX_QUEUES == 8) in
gem_sram_distribute_segments() to signal that code will need
updating if/when the constant changes.
- Update To/Cc list: drop Nicolas and Claudiu, add Conor and Nicolai.
- Rebase onto latest net-next/main (6b3e73774815).
- Link to v4: https://patch.msgid.link/20260317-macb-set-channels-v4-0-1bd4f4ffcfca@bootlin.com
Changes in v4:
- macb_set_channels(): drop `count == old_count` check, it is redundant
with both netlink and ioctl codepaths. See early returns in
ethnl_set_channels() and ethtool_set_channels(). Discussion @ V2.
- Rebase to latest net-next (8737d7194d6d); nothing to report.
- Link to v3: https://patch.msgid.link/20260313-macb-set-channels-v3-0-c66b7781ddb8@bootlin.com
Changes in v3:
- Makefile: fix compilation of kunit tests. This bug was hidden before
because everything was builtin so it ended up linked together anyway.
Now macb.o is the amalgation of macb_{main,ptp,utils}.o and
macb_test.o is the amalgation of macb_{kunit,utils}.o.
- Kconfig CONFIG_MACB_KUNIT_TEST entry:
- Turn into tristate; no good reason to restrict to bool.
- Drop "depends on MACB". We do not need to compile the full MACB
driver to test gem_sram_distribute_segments().
- Add usual MODULE_*() macros to macb_kunit.c.
- Drop unused init value of `ret` in macb_set_channels().
- Rebase to latest net-next; nothing to report.
- Link to v2: https://patch.msgid.link/20260311-macb-set-channels-v2-0-982693a1f5fc@bootlin.com
Changes in v2:
- Move gem_sram_distribute_segments() from macb_main.c to macb_utils.c
and kunit tests from macb_main.c to macb_kunit.c.
- macb_set_channels():
- Add comment about MACB_CAPS_QUEUE_DISABLE.
- Refuse operation if netif_running(), with code comment.
- Drop useless sanity checks on ch->{combined,rx,tx}_count.
- Add help text to CONFIG_MACB_KUNIT_TEST.
- Fix `checkpatch --max-line-length=80` warnings.
- Rebase onto latest net-next; nothing to report.
- Link to v1: https://lore.kernel.org/r/20260305-macb-set-channels-v1-0-28e3a96a3dc3@bootlin.com
To: Conor Dooley <conor.dooley@microchip.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>
To: David S. Miller <davem@davemloft.net>
To: Eric Dumazet <edumazet@kernel.org>
To: Jakub Kicinski <kuba@kernel.org>
To: Paolo Abeni <pabeni@redhat.com>
To: Russell King <linux@armlinux.org.uk>
Cc: netdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Nicolai Buchwitz <nb@tipi-net.de>
Cc: Paolo Valerio <pvalerio@redhat.com>
Cc: Vladimir Kondratiev <vladimir.kondratiev@mobileye.com>
Cc: Gregory CLEMENT <gregory.clement@bootlin.com>
Cc: Benoît Monin <benoit.monin@bootlin.com>
Cc: Tawfik Bayouk <tawfik.bayouk@mobileye.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
---
Théo Lebrun (3):
net: macb: cache ENST config across link down/up
net: macb: implement ethtool_ops.get|set_channels()
net: macb: distribute evenly Tx SRAM segments
drivers/net/ethernet/cadence/macb.h | 66 +++-
drivers/net/ethernet/cadence/macb_main.c | 512 +++++++++++++++++++++----------
2 files changed, 397 insertions(+), 181 deletions(-)
---
base-commit: d6816d9a93b66a12bf01e10037e377f3333ad6cc
change-id: 20260305-macb-set-channels-5bf6e07f3270
Best regards,
--
Théo Lebrun <theo.lebrun@bootlin.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
@ 2026-11-18 20:53 ` Théo Lebrun
2026-11-19 11:49 ` Nicolai Buchwitz
2026-11-19 12:36 ` Vineeth Karumanchi
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
2 siblings, 2 replies; 9+ messages in thread
From: Théo Lebrun @ 2026-11-18 20:53 UTC (permalink / raw)
To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun
Currently, ENST is not explicitly restored across interface close/open.
If HW keeps its register state then ENST is kept, otherwise it is lost;
but we do not write the ENST schedule again on open.
This causes trouble if the config is not compatible with the new open
and link-up. The two issues that can occur are a link speed change (as
all timings are configured in a unit relative to the link speed) or a
change in the # of queues.
The latter is the main reason we do this change before implementing
ethtool -L support.
If on link-up, the cached ENST config is invalid (link speed changed),
then we drop the cached config, log a warning and explicitly override
all ENST timings previously configured in the registers. If it is
valid, we explicitly re-apply it.
Take this opportunity to replace a kmalloc in
macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
Drop the old hack of checking for pm_runtime_suspended(), which has
little value because we could race with the runtime_suspend() call.
Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
phylink_ethtool_ksettings_get(), cache the link speed as bp->speed_mbps.
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
---
drivers/net/ethernet/cadence/macb.h | 47 +++++---
drivers/net/ethernet/cadence/macb_main.c | 187 +++++++++++++++----------------
2 files changed, 124 insertions(+), 110 deletions(-)
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index b7b4b6a38298..00258866623f 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -1327,6 +1327,35 @@ struct macb_context {
struct macb_txq txq[MACB_MAX_QUEUES];
};
+/**
+ * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
+ * @start_time_mask: Bitmask representing the start time for the queue
+ * @on_time_bytes: "on" time nsec expressed in bytes
+ * @off_time_bytes: "off" time nsec expressed in bytes
+ *
+ * This structure holds the configuration parameters for an ENST queue,
+ * used to control time-based transmission scheduling in the MACB driver.
+ */
+struct macb_queue_enst_config {
+ u32 start_time_mask;
+ u32 on_time_bytes;
+ u32 off_time_bytes;
+};
+
+/**
+ * struct macb_enst - Cached Enhanced Scheduled Traffic configuration
+ * @queues: per-queue timing (zeroed for queues without schedule)
+ * @speed_mbps: link speed the timings were computed for, 0 if no schedule
+ * @queue_mask: queues enabled by the current schedule, 0 if no schedule
+ *
+ * The timings are expressed in bytes, so they depend on the link speed.
+ */
+struct macb_enst {
+ struct macb_queue_enst_config queues[MACB_MAX_QUEUES];
+ u32 speed_mbps;
+ u32 queue_mask;
+};
+
struct macb {
void __iomem *regs;
bool native_io;
@@ -1427,11 +1456,14 @@ struct macb {
/* Redundant to netif_carrier_ok(), but set under bp->mac_cfg_lock. */
bool link_up;
+ u32 speed_mbps;
u32 rx_intr_mask;
struct macb_pm_data pm_data;
const struct macb_usrio_config *usrio;
+
+ struct macb_enst enst;
};
#ifdef CONFIG_MACB_USE_HWSTAMP
@@ -1532,19 +1564,4 @@ struct macb_platform_data {
struct clk *hclk;
};
-/**
- * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
- * @start_time_mask: Bitmask representing the start time for the queue
- * @on_time_bytes: "on" time nsec expressed in bytes
- * @off_time_bytes: "off" time nsec expressed in bytes
- *
- * This structure holds the configuration parameters for an ENST queue,
- * used to control time-based transmission scheduling in the MACB driver.
- */
-struct macb_queue_enst_config {
- u32 start_time_mask;
- u32 on_time_bytes;
- u32 off_time_bytes;
-};
-
#endif /* _MACB_H */
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 9f6111661b40..1e0666692116 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -847,6 +847,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
mutex_lock(&bp->mac_cfg_lock);
bp->link_up = false;
+ bp->speed_mbps = 0;
if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
@@ -955,6 +956,43 @@ static void gem_shuffle_tx_rings(struct macb *bp)
gem_shuffle_tx_one_ring(queue);
}
+static void macb_apply_enst_config(struct macb *bp)
+{
+ struct macb_queue_enst_config *config;
+ struct macb_queue *queue;
+ u32 queue_mask;
+ size_t q;
+
+ /* Short-circuit to avoid writing to invalid register offsets */
+ if (!(bp->caps & MACB_CAPS_QBV))
+ return;
+
+ guard(spinlock_irqsave)(&bp->lock);
+
+ /* Disable ENST for all queues.
+ * Reconfiguring timings must be done with ENST disabled.
+ */
+ queue_mask = BIT_U32(bp->num_queues) - 1;
+ gem_writel(bp, ENST_CONTROL,
+ queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
+
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ /* Configure queue timing registers.
+ *
+ * Those fields must be zero if the queue has ENST disabled:
+ * notice we don't bother reading bp->enst.queue_mask & BIT(q).
+ */
+ config = &bp->enst.queues[q];
+ queue_writel(queue, ENST_START_TIME, config->start_time_mask);
+ queue_writel(queue, ENST_ON_TIME, config->on_time_bytes);
+ queue_writel(queue, ENST_OFF_TIME, config->off_time_bytes);
+ }
+
+ /* Enable ENST for all configured queues in one write */
+ if (bp->enst.queue_mask)
+ gem_writel(bp, ENST_CONTROL, bp->enst.queue_mask);
+}
+
static void macb_mac_link_up(struct phylink_config *config,
struct phy_device *phydev,
unsigned int mode, phy_interface_t interface,
@@ -972,6 +1010,7 @@ static void macb_mac_link_up(struct phylink_config *config,
spin_lock_irqsave(&bp->lock, flags);
bp->link_up = true;
+ bp->speed_mbps = speed > 0 ? speed : 0;
ctrl = macb_or_gem_readl(bp, NCFGR);
@@ -1009,6 +1048,15 @@ static void macb_mac_link_up(struct phylink_config *config,
spin_unlock_irqrestore(&bp->lock, flags);
+ if (bp->enst.speed_mbps && speed != bp->enst.speed_mbps) {
+ netdev_warn(bp->netdev,
+ "TAPRIO dropped because link speed changed\n");
+
+ memset(&bp->enst, 0, sizeof(bp->enst));
+ }
+
+ macb_apply_enst_config(bp);
+
if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) {
macb_set_tx_clk(bp, speed);
gem_shuffle_tx_rings(bp);
@@ -4754,19 +4802,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
struct tc_taprio_qopt_offload *conf)
{
u64 total_on_time = 0, start_time_sec = 0, start_time = conf->base_time;
- u32 configured_queues = 0, speed = 0, start_time_nsec;
- struct macb_queue_enst_config *enst_queue;
- struct ethtool_link_ksettings kset = {};
+ u32 configured_queues = 0, start_time_nsec;
struct macb *bp = netdev_priv(netdev);
struct tc_taprio_sched_entry *entry;
- struct macb_queue *queue;
- u32 queue_mask;
+ struct macb_enst config = {};
u8 queue_id;
- size_t i, q;
- int err;
+ u64 off;
+ size_t i;
+
+ guard(mutex)(&bp->mac_cfg_lock);
+
+ if (unlikely(!bp->speed_mbps)) {
+ netdev_err(netdev, "Invalid speed, link-down?\n");
+ return -EINVAL;
+ }
if (conf->num_entries > bp->num_queues) {
- netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n",
+ netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n",
conf->num_entries, bp->num_queues);
return -EINVAL;
}
@@ -4777,23 +4829,6 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
return -ERANGE;
}
- /* Get the current link speed */
- err = phylink_ethtool_ksettings_get(bp->phylink, &kset);
- if (unlikely(err)) {
- netdev_err(netdev, "Failed to get link settings: %d\n", err);
- return err;
- }
-
- speed = kset.base.speed;
- if (unlikely(speed == SPEED_UNKNOWN || !speed)) {
- netdev_err(netdev, "Invalid speed %d, link-down?\n", speed);
- return -EINVAL;
- }
-
- enst_queue = kzalloc_objs(*enst_queue, bp->num_queues);
- if (unlikely(!enst_queue))
- return -ENOMEM;
-
/* Pre-validate all entries before making any hardware changes */
for (i = 0; i < conf->num_entries; i++) {
entry = &conf->entries[i];
@@ -4801,25 +4836,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
if (entry->command != TC_TAPRIO_CMD_SET_GATES) {
netdev_err(netdev, "Entry %zu: unsupported command %d\n",
i, entry->command);
- err = -EOPNOTSUPP;
- goto cleanup;
+ return -EOPNOTSUPP;
}
/* Validate gate_mask: must be nonzero, single queue, and within range */
if (!is_power_of_2(entry->gate_mask)) {
netdev_err(netdev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
i, entry->gate_mask);
- err = -EINVAL;
- goto cleanup;
+ return -EINVAL;
}
/* gate_mask must not select queues outside the valid queues */
queue_id = order_base_2(entry->gate_mask);
if (queue_id >= bp->num_queues) {
- netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
- i, entry->gate_mask, bp->num_queues);
- err = -EINVAL;
- goto cleanup;
+ netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n",
+ i, entry->gate_mask,
+ bp->num_queues);
+ return -EINVAL;
}
/* Check for start time limits */
@@ -4828,34 +4861,33 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
if (start_time_sec > GENMASK(GEM_START_TIME_SEC_SIZE - 1, 0)) {
netdev_err(netdev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
i, start_time_sec);
- err = -ERANGE;
- goto cleanup;
+ return -ERANGE;
}
/* Check for on time limit */
- if (entry->interval > enst_max_hw_interval(speed)) {
+ if (entry->interval > enst_max_hw_interval(bp->speed_mbps)) {
netdev_err(netdev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
- i, entry->interval, enst_max_hw_interval(speed));
- err = -ERANGE;
- goto cleanup;
+ i, entry->interval,
+ enst_max_hw_interval(bp->speed_mbps));
+ return -ERANGE;
}
/* Check for off time limit*/
- if ((conf->cycle_time - entry->interval) > enst_max_hw_interval(speed)) {
+ off = conf->cycle_time - entry->interval;
+ if (off > enst_max_hw_interval(bp->speed_mbps)) {
netdev_err(netdev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
- i, conf->cycle_time - entry->interval,
- enst_max_hw_interval(speed));
- err = -ERANGE;
- goto cleanup;
+ i, off,
+ enst_max_hw_interval(bp->speed_mbps));
+ return -ERANGE;
}
- enst_queue[queue_id].start_time_mask =
+ config.queues[queue_id].start_time_mask =
(start_time_sec << GEM_START_TIME_SEC_OFFSET) |
start_time_nsec;
- enst_queue[queue_id].on_time_bytes =
- enst_ns_to_hw_units(entry->interval, speed);
- enst_queue[queue_id].off_time_bytes =
- enst_ns_to_hw_units(conf->cycle_time - entry->interval, speed);
+ config.queues[queue_id].on_time_bytes =
+ enst_ns_to_hw_units(entry->interval, bp->speed_mbps);
+ config.queues[queue_id].off_time_bytes =
+ enst_ns_to_hw_units(off, bp->speed_mbps);
configured_queues |= entry->gate_mask;
total_on_time += entry->interval;
@@ -4866,82 +4898,47 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
if (total_on_time > conf->cycle_time) {
netdev_err(netdev, "Total ON %llu ns exceeds cycle time %llu ns\n",
total_on_time, conf->cycle_time);
- err = -EINVAL;
- goto cleanup;
+ return -EINVAL;
}
netdev_dbg(netdev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
conf->num_entries, conf->base_time, conf->cycle_time);
- /* All validations passed - proceed with hardware configuration */
- scoped_guard(spinlock_irqsave, &bp->lock) {
- /* Disable ENST queues if running before configuring */
- queue_mask = BIT_U32(bp->num_queues) - 1;
- gem_writel(bp, ENST_CONTROL,
- queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
+ config.queue_mask = configured_queues;
+ config.speed_mbps = bp->speed_mbps;
+ bp->enst = config;
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- /* Configure queue timing registers */
- queue_writel(queue, ENST_START_TIME,
- enst_queue[q].start_time_mask);
- queue_writel(queue, ENST_ON_TIME,
- enst_queue[q].on_time_bytes);
- queue_writel(queue, ENST_OFF_TIME,
- enst_queue[q].off_time_bytes);
- }
-
- /* Enable ENST for all configured queues in one write */
- gem_writel(bp, ENST_CONTROL, configured_queues);
- }
+ macb_apply_enst_config(bp);
netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
conf->num_entries, hweight32(configured_queues));
-cleanup:
- kfree(enst_queue);
- return err;
+ return 0;
}
static void macb_taprio_destroy(struct net_device *netdev)
{
struct macb *bp = netdev_priv(netdev);
- struct macb_queue *queue;
- u32 queue_mask;
- unsigned int q;
+
+ guard(mutex)(&bp->mac_cfg_lock);
netdev_reset_tc(netdev);
- queue_mask = BIT_U32(bp->num_queues) - 1;
+ memset(&bp->enst, 0, sizeof(bp->enst));
- scoped_guard(spinlock_irqsave, &bp->lock) {
- /* Single disable command for all queues */
- gem_writel(bp, ENST_CONTROL,
- queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
+ if (bp->link_up)
+ macb_apply_enst_config(bp);
- /* Clear all queue ENST registers in batch */
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- queue_writel(queue, ENST_START_TIME, 0);
- queue_writel(queue, ENST_ON_TIME, 0);
- queue_writel(queue, ENST_OFF_TIME, 0);
- }
- }
netdev_info(netdev, "TAPRIO destroy: All gates disabled\n");
}
static int macb_setup_taprio(struct net_device *netdev,
struct tc_taprio_qopt_offload *taprio)
{
- struct macb *bp = netdev_priv(netdev);
int err = 0;
if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC)))
return -EOPNOTSUPP;
- /* Check if Device is in runtime suspend */
- if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) {
- netdev_err(netdev, "Device is in runtime suspend\n");
- return -EOPNOTSUPP;
- }
-
switch (taprio->cmd) {
case TAPRIO_CMD_REPLACE:
err = macb_taprio_setup_replace(netdev, taprio);
--
2.56.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels()
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
@ 2026-11-18 20:53 ` Théo Lebrun
2026-11-19 11:54 ` Nicolai Buchwitz
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
2 siblings, 1 reply; 9+ messages in thread
From: Théo Lebrun @ 2026-11-18 20:53 UTC (permalink / raw)
To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun
bp->num_queues is the total number of queues and is constant from probe.
MACB/GEM does not know about Rx/Tx specific queues; it only has
combined queues. Move bp->num_queues to ctx->num_queues as it now
belongs to the currently active context (bp->ctx).
Also introduce bp->max_num_queues which takes the current role of
bp->num_queues and allow `0 < ctx->num_queues <= bp->max_num_queues`.
Lastly introduce bp->configured_num_queues to remember queue count
across interface down+up cycles.
With all that, implement the .set_channels() operation. It ends up only
being an assignment to bp->configured_num_queues plus a call into
netif_set_real_num_queues(). However if the interface is running, we
use a context swap window to safely realloc on the fly.
Note we refuse the operation if any ENST schedule is in place on a
queue we would disable. If we allow the operation to take place, then
a transmission window would be reserved but unusable.
Nit: fix an alignment issue inside gem_ethtool_ops which does not
deserve its own patch.
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
---
drivers/net/ethernet/cadence/macb.h | 5 +-
drivers/net/ethernet/cadence/macb_main.c | 258 ++++++++++++++++++++++---------
2 files changed, 191 insertions(+), 72 deletions(-)
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index 00258866623f..d60b06e41b83 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -1292,7 +1292,6 @@ struct macb_info {
struct platform_device *pdev;
struct net_device *netdev;
struct macb_or_gem_ops macbgem_ops;
- unsigned int num_queues;
u32 caps;
int rx_bd_rd_prefetch;
int tx_bd_rd_prefetch;
@@ -1323,6 +1322,7 @@ struct macb_context {
unsigned int rx_buffer_size;
unsigned int rx_ring_size;
unsigned int tx_ring_size;
+ unsigned int num_queues;
struct macb_rxq rxq[MACB_MAX_QUEUES];
struct macb_txq txq[MACB_MAX_QUEUES];
};
@@ -1381,6 +1381,9 @@ struct macb {
unsigned int configured_tx_ring_size;
struct macb_context *ctx;
+ unsigned int configured_num_queues;
+ unsigned int max_num_queues;
+
struct macb_dma_desc *rx_ring_tieoff;
dma_addr_t rx_ring_tieoff_dma;
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 1e0666692116..180da5d62f09 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -504,11 +504,25 @@ static void macb_init_buffers(struct macb *bp)
macb_writel(bp, TBQPH, upper_32_bits(txq->ring_dma));
}
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- rxq = &bp->ctx->rxq[q];
- txq = &bp->ctx->txq[q];
- queue_writel(queue, RBQP, lower_32_bits(rxq->ring_dma));
- queue_writel(queue, TBQP, lower_32_bits(txq->ring_dma));
+ for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue) {
+ if (q < bp->ctx->num_queues) {
+ rxq = &bp->ctx->rxq[q];
+ txq = &bp->ctx->txq[q];
+ queue_writel(queue, RBQP, lower_32_bits(rxq->ring_dma));
+ queue_writel(queue, TBQP, lower_32_bits(txq->ring_dma));
+ } else {
+ /* macb_set_channels(), which is the only way of writing
+ * to bp->configured_num_queues, is only allowed if
+ * MACB_CAPS_QUEUE_DISABLE.
+ */
+ queue_writel(queue, RBQP, MACB_BIT(QUEUE_DISABLE));
+ queue_writel(queue, TBQP, MACB_BIT(QUEUE_DISABLE));
+
+ /* Disable all interrupts */
+ queue_writel(queue, IDR, -1);
+ queue_readl(queue, ISR);
+ macb_queue_isr_clear(bp, queue, -1);
+ }
}
}
@@ -695,7 +709,7 @@ static bool macb_tx_all_queues_idle(struct macb *bp)
struct macb_txq *txq;
unsigned int q;
- for (q = 0; q < bp->num_queues; ++q) {
+ for (q = 0; q < bp->ctx->num_queues; ++q) {
txq = &bp->ctx->txq[q];
if (READ_ONCE(txq->head) != READ_ONCE(txq->tail))
return false;
@@ -850,7 +864,8 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
bp->speed_mbps = 0;
if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues;
+ ++q, ++queue)
queue_writel(queue, IDR,
bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP));
@@ -952,7 +967,7 @@ static void gem_shuffle_tx_rings(struct macb *bp)
struct macb_queue *queue;
unsigned int q;
- for (q = 0, queue = bp->queues; q < bp->num_queues; q++, queue++)
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; q++, queue++)
gem_shuffle_tx_one_ring(queue);
}
@@ -972,11 +987,11 @@ static void macb_apply_enst_config(struct macb *bp)
/* Disable ENST for all queues.
* Reconfiguring timings must be done with ENST disabled.
*/
- queue_mask = BIT_U32(bp->num_queues) - 1;
+ queue_mask = BIT_U32(bp->max_num_queues) - 1;
gem_writel(bp, ENST_CONTROL,
queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue) {
/* Configure queue timing registers.
*
* Those fields must be zero if the queue has ENST disabled:
@@ -1034,10 +1049,10 @@ static void macb_mac_link_up(struct phylink_config *config,
if (rx_pause)
ctrl |= MACB_BIT(PAE);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues;
+ ++q, ++queue)
queue_writel(queue, IER,
bp->rx_intr_mask | MACB_TX_INT_FLAGS | MACB_BIT(HRESP));
- }
}
macb_or_gem_writel(bp, NCFGR, ctrl);
@@ -2198,12 +2213,12 @@ static void macb_quiesce_start(struct macb *bp)
bp->irq_quiesced = true;
spin_unlock_irqrestore(&bp->lock, flags);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue)
synchronize_irq(queue->irq);
cancel_work_sync(&bp->hresp_err_bh_work);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) {
/* Must be done before NAPI is disabled: the task ends with a
* napi_enable() call.
*/
@@ -2231,7 +2246,7 @@ static void macb_quiesce_end(struct macb *bp)
unsigned long flags;
unsigned int q;
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) {
napi_enable(&queue->napi_rx);
napi_enable(&queue->napi_tx);
}
@@ -2250,7 +2265,7 @@ static void macb_hresp_error_task(struct work_struct *work)
unsigned int q;
u32 ctrl;
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue) {
queue_writel(queue, IDR, bp->rx_intr_mask |
MACB_TX_INT_FLAGS |
MACB_BIT(HRESP));
@@ -2268,7 +2283,7 @@ static void macb_hresp_error_task(struct work_struct *work)
macb_init_buffers(bp);
/* Enable interrupts */
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ for (q = 0, queue = bp->queues; q < ctx->num_queues; ++q, ++queue)
queue_writel(queue, IER,
bp->rx_intr_mask |
MACB_TX_INT_FLAGS |
@@ -2369,7 +2384,9 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
{
struct macb_queue *queue = dev_id;
struct macb *bp = queue->bp;
+ unsigned int num_queues = 0;
struct net_device *netdev = bp->netdev;
+ unsigned int q = queue - bp->queues;
bool hresp_err = false;
u32 status;
@@ -2380,19 +2397,24 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
spin_lock(&bp->lock);
+ /* Only dereference bp->ctx when not quiesced.
+ * Quiesced state cannot change as we are in a bp->lock section.
+ */
+ if (!bp->irq_quiesced && bp->ctx)
+ num_queues = READ_ONCE(bp->ctx->num_queues);
+
while (status) {
- /* self-disarm while the netdev is closed
- * or during context swap
+ /* self-disarm while the netdev is closed,
+ * during context swap, or when the current queue is disabled
*/
- if (unlikely(bp->irq_quiesced)) {
+ if (unlikely(bp->irq_quiesced || q >= num_queues)) {
queue_writel(queue, IDR, -1);
macb_queue_isr_clear(bp, queue, -1);
break;
}
netdev_vdbg(netdev, "queue = %u, isr = 0x%08lx\n",
- (unsigned int)(queue - bp->queues),
- (unsigned long)status);
+ q, (unsigned long)status);
if (status & bp->rx_intr_mask) {
/* There's no point taking any more interrupts
@@ -2450,7 +2472,8 @@ static void macb_poll_controller(struct net_device *netdev)
unsigned int q;
local_irq_save(flags);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ for (q = 0, queue = bp->queues; q < bp->configured_num_queues;
+ ++q, ++queue)
macb_interrupt(netdev->irq, queue);
local_irq_restore(flags);
}
@@ -2893,7 +2916,7 @@ static void gem_free_rx_buffers(struct macb_context *ctx)
unsigned int q;
int i;
- for (q = 0; q < ctx->info->num_queues; ++q) {
+ for (q = 0; q < ctx->num_queues; ++q) {
rxq = &ctx->rxq[q];
if (!rxq->skbuff)
@@ -2955,14 +2978,14 @@ static void macb_free(struct macb_context *ctx)
ctx->info->macbgem_ops.mog_free_rx_buffers(ctx);
txq = &ctx->txq[0];
- size = ctx->info->num_queues * macb_tx_ring_size_per_queue(ctx);
+ size = ctx->num_queues * macb_tx_ring_size_per_queue(ctx);
dma_free_coherent(dev, size, txq->ring, txq->ring_dma);
rxq = &ctx->rxq[0];
- size = ctx->info->num_queues * macb_rx_ring_size_per_queue(ctx);
+ size = ctx->num_queues * macb_rx_ring_size_per_queue(ctx);
dma_free_coherent(dev, size, rxq->ring, rxq->ring_dma);
- for (q = 0; q < ctx->info->num_queues; ++q) {
+ for (q = 0; q < ctx->num_queues; ++q) {
txq = &ctx->txq[q];
rxq = &ctx->rxq[q];
@@ -2995,7 +3018,7 @@ static int gem_alloc_rx_buffers(struct macb_context *ctx)
unsigned int q;
int size;
- for (q = 0; q < ctx->info->num_queues; ++q) {
+ for (q = 0; q < ctx->num_queues; ++q) {
rxq = &ctx->rxq[q];
size = ctx->rx_ring_size * sizeof(struct sk_buff *);
rxq->skbuff = kzalloc(size, GFP_KERNEL);
@@ -3029,8 +3052,8 @@ static int macb_alloc_rx_buffers(struct macb_context *ctx)
static int macb_alloc(struct macb_context *ctx)
{
- unsigned int num_queues = ctx->info->num_queues;
struct device *dev = &ctx->info->pdev->dev;
+ unsigned int num_queues = ctx->num_queues;
dma_addr_t tx_dma, rx_dma;
struct macb_txq *txq;
struct macb_rxq *rxq;
@@ -3099,6 +3122,7 @@ static int macb_alloc(struct macb_context *ctx)
}
static struct macb_context *macb_context_alloc(struct macb *bp,
+ unsigned int num_queues,
unsigned int mtu,
unsigned int rx_ring_size,
unsigned int tx_ring_size)
@@ -3111,6 +3135,7 @@ static struct macb_context *macb_context_alloc(struct macb *bp,
return ERR_PTR(-ENOMEM);
ctx->info = &bp->info;
+ ctx->num_queues = num_queues;
ctx->rx_buffer_size = macb_rx_buffer_size(bp, mtu);
ctx->rx_ring_size = rx_ring_size;
ctx->tx_ring_size = tx_ring_size;
@@ -3158,7 +3183,7 @@ static int gem_init_rings(struct macb_context *ctx)
int err;
int i;
- for (q = 0; q < ctx->info->num_queues; ++q) {
+ for (q = 0; q < ctx->num_queues; ++q) {
txq = &ctx->txq[q];
for (i = 0; i < ctx->tx_ring_size; i++) {
desc = macb_tx_desc(ctx, q, i);
@@ -3221,7 +3246,7 @@ static void macb_reset_hw(struct macb *bp)
gem_writel(bp, PBUFRXCUT, 0);
/* Disable all interrupts */
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) {
queue_writel(queue, IDR, -1);
queue_readl(queue, ISR);
macb_queue_isr_clear(bp, queue, -1);
@@ -3311,7 +3336,8 @@ static void macb_configure_dma(struct macb *bp)
buffer_size = bp->ctx->rx_buffer_size / RX_BUFFER_MULTIPLE;
if (macb_is_gem(&bp->info)) {
dmacfg = gem_readl(bp, DMACFG) & ~GEM_BF(RXBS, -1L);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues;
+ ++q, ++queue) {
if (q)
queue_writel(queue, RBQS, buffer_size);
else
@@ -3359,10 +3385,6 @@ static void macb_context_swap_start(struct macb *bp)
*/
netif_tx_disable(bp->netdev);
- /* Now that everything is stopped, clear DQL. */
- for (q = 0; q < bp->num_queues; ++q)
- netdev_tx_reset_queue(netdev_get_tx_queue(bp->netdev, q));
-
/* Safe to call outside bp->lock because bp->irq_quiesced ensures the IRQ
* handling is a no-op and all BH features are disabled.
*
@@ -3376,7 +3398,7 @@ static void macb_context_swap_start(struct macb *bp)
ctrl = macb_readl(bp, NCR);
macb_writel(bp, NCR, ctrl & ~(MACB_BIT(RE) | MACB_BIT(TE)));
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) {
queue_writel(queue, IDR, -1);
queue_readl(queue, ISR);
macb_queue_isr_clear(bp, queue, -1);
@@ -3399,16 +3421,26 @@ static void macb_context_swap_end(struct macb *bp,
lockdep_assert_held(&bp->mac_cfg_lock);
- /* Swap contexts & give buffer pointers to HW. */
+ /* Swap contexts & give buffer pointers to HW.
+ * !new_ctx means that the swap failed and we keep the old context.
+ */
- old_ctx = bp->ctx;
- WRITE_ONCE(bp->ctx, new_ctx);
- wmb(); /* ensure IRQ enabled below read the new context */
- macb_init_buffers(bp);
+ if (new_ctx) {
+ old_ctx = bp->ctx;
+ WRITE_ONCE(bp->ctx, new_ctx);
+ wmb(); /* ensure IRQ enabled below read the new context */
+
+ macb_init_buffers(bp);
+
+ /* Clear DQL (only on successful context swap). */
+ for (q = 0; q < bp->ctx->num_queues; ++q)
+ netdev_tx_reset_queue(netdev_get_tx_queue(bp->netdev,
+ q));
+ }
/* Start NAPI, HW Tx/Rx and software Tx. */
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues; ++q, ++queue) {
napi_enable(&queue->napi_rx);
napi_enable(&queue->napi_tx);
}
@@ -3421,7 +3453,8 @@ static void macb_context_swap_end(struct macb *bp,
macb_configure_dma(bp);
if (bp->link_up) {
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->ctx->num_queues;
+ ++q, ++queue) {
queue_writel(queue, IER,
bp->rx_intr_mask |
MACB_TX_INT_FLAGS |
@@ -3445,8 +3478,10 @@ static void macb_context_swap_end(struct macb *bp,
/* Free old context. */
- macb_free(old_ctx);
- kfree(old_ctx);
+ if (new_ctx) {
+ macb_free(old_ctx);
+ kfree(old_ctx);
+ }
}
static void macb_init_hw(struct macb *bp)
@@ -3620,7 +3655,7 @@ static int macb_open(struct net_device *netdev)
if (err < 0)
return err;
- ctx = macb_context_alloc(bp, netdev->mtu,
+ ctx = macb_context_alloc(bp, bp->configured_num_queues, netdev->mtu,
bp->configured_rx_ring_size,
bp->configured_tx_ring_size);
if (IS_ERR(ctx)) {
@@ -3685,7 +3720,7 @@ static int macb_close(struct net_device *netdev)
* and tx_error_task wake them up.
*/
netif_tx_stop_all_queues(netdev);
- for (q = 0; q < bp->num_queues; ++q)
+ for (q = 0; q < bp->ctx->num_queues; ++q)
netdev_tx_reset_queue(netdev_get_tx_queue(netdev, q));
phylink_stop(bp->phylink);
@@ -3721,7 +3756,8 @@ static int macb_change_mtu(struct net_device *netdev, int new_mtu)
if (bp->caps & MACB_CAPS_MACB_IS_EMAC)
return -EBUSY;
- new_ctx = macb_context_alloc(bp, new_mtu,
+ new_ctx = macb_context_alloc(bp, bp->configured_num_queues,
+ new_mtu,
bp->configured_rx_ring_size,
bp->configured_tx_ring_size);
if (IS_ERR(new_ctx))
@@ -3774,7 +3810,7 @@ static void gem_update_stats(struct macb *bp)
}
idx = GEM_STATS_LEN;
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ for (q = 0, queue = bp->queues; q < bp->max_num_queues; ++q, ++queue)
for (i = 0, stat = &queue->stats.first; i < QUEUE_STATS_LEN; ++i, ++stat)
bp->ethtool_stats[idx++] = *stat;
}
@@ -3825,7 +3861,8 @@ static void gem_get_ethtool_stats(struct net_device *netdev,
spin_lock_irq(&bp->stats_lock);
gem_update_stats(bp);
memcpy(data, &bp->ethtool_stats, sizeof(u64)
- * (GEM_STATS_LEN + QUEUE_STATS_LEN * bp->num_queues));
+ * (GEM_STATS_LEN + QUEUE_STATS_LEN *
+ bp->max_num_queues));
spin_unlock_irq(&bp->stats_lock);
}
@@ -3835,7 +3872,7 @@ static int gem_get_sset_count(struct net_device *netdev, int sset)
switch (sset) {
case ETH_SS_STATS:
- return GEM_STATS_LEN + bp->num_queues * QUEUE_STATS_LEN;
+ return GEM_STATS_LEN + bp->max_num_queues * QUEUE_STATS_LEN;
default:
return -EOPNOTSUPP;
}
@@ -3854,7 +3891,8 @@ static void gem_get_ethtool_strings(struct net_device *netdev, u32 sset, u8 *p)
memcpy(p, gem_statistics[i].stat_string,
ETH_GSTRING_LEN);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (q = 0, queue = bp->queues; q < bp->max_num_queues;
+ ++q, ++queue) {
for (i = 0; i < QUEUE_STATS_LEN; i++)
ethtool_sprintf(&p, "q%u_%s", q, queue_statistics[i].stat_string);
}
@@ -4213,8 +4251,9 @@ static int macb_set_ringparam(struct net_device *netdev,
}
if (running) {
- new_ctx = macb_context_alloc(bp, netdev->mtu,
- new_rx_size, new_tx_size);
+ new_ctx = macb_context_alloc(bp, bp->configured_num_queues,
+ netdev->mtu, new_rx_size,
+ new_tx_size);
if (IS_ERR(new_ctx))
return PTR_ERR(new_ctx);
@@ -4561,7 +4600,7 @@ static u32 gem_get_rx_ring_count(struct net_device *netdev)
{
struct macb *bp = netdev_priv(netdev);
- return bp->num_queues;
+ return bp->configured_num_queues;
}
static int gem_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
@@ -4592,6 +4631,7 @@ static int gem_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd)
{
struct macb *bp = netdev_priv(netdev);
+ unsigned int num_queues = bp->configured_num_queues;
int ret;
if (!(netdev->hw_features & NETIF_F_NTUPLE))
@@ -4599,8 +4639,8 @@ static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd)
switch (cmd->cmd) {
case ETHTOOL_SRXCLSRLINS:
- if ((cmd->fs.location >= bp->max_tuples)
- || (cmd->fs.ring_cookie >= bp->num_queues)) {
+ if (cmd->fs.location >= bp->max_tuples ||
+ cmd->fs.ring_cookie >= num_queues) {
ret = -EINVAL;
break;
}
@@ -4618,6 +4658,75 @@ static int gem_set_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd)
return ret;
}
+static void macb_get_channels(struct net_device *netdev,
+ struct ethtool_channels *ch)
+{
+ struct macb *bp = netdev_priv(netdev);
+
+ ch->max_combined = bp->max_num_queues;
+ ch->combined_count = bp->configured_num_queues;
+}
+
+static int macb_set_channels(struct net_device *netdev,
+ struct ethtool_channels *ch)
+{
+ struct macb *bp = netdev_priv(netdev);
+ unsigned int num_queues = ch->combined_count;
+ bool running = netif_running(bp->netdev);
+ struct macb_context *new_ctx;
+ int ret;
+
+ /* MACB_CAPS_QUEUE_DISABLE means that the field QUEUE_DISABLE/BIT0 in
+ * the per-queue RBQP/TBQP registers disable Tx/Rx queues. If we don't
+ * have that capability we can have multiple queues but we must always
+ * run with all enabled.
+ */
+ if (!(bp->caps & MACB_CAPS_QUEUE_DISABLE))
+ return -EOPNOTSUPP;
+
+ if (num_queues > bp->max_num_queues)
+ return -EINVAL;
+
+ if (num_queues == bp->configured_num_queues)
+ return 0;
+
+ /* If we allow disabling a queue which has TAPRIO active, we'll keep a
+ * transmission window which would be unusable.
+ */
+ if (bp->enst.queue_mask >> num_queues) {
+ netdev_err(bp->netdev, "TAPRIO schedule gates queues >= %u; destroy the qdisc before reducing channels\n",
+ num_queues);
+ return -EBUSY;
+ }
+
+ if (running) {
+ new_ctx = macb_context_alloc(bp, num_queues, netdev->mtu,
+ bp->configured_rx_ring_size,
+ bp->configured_tx_ring_size);
+ if (IS_ERR(new_ctx))
+ return PTR_ERR(new_ctx);
+
+ macb_context_swap_start(bp);
+ }
+
+ ret = netif_set_real_num_queues(bp->netdev, num_queues, num_queues);
+ if (ret) {
+ if (running) {
+ macb_free(new_ctx);
+ kfree(new_ctx);
+ macb_context_swap_end(bp, NULL);
+ }
+ return ret;
+ }
+
+ bp->configured_num_queues = num_queues;
+
+ if (running)
+ macb_context_swap_end(bp, new_ctx);
+
+ return 0;
+}
+
static const struct ethtool_ops macb_ethtool_ops = {
.get_regs_len = macb_get_regs_len,
.get_regs = macb_get_regs,
@@ -4633,6 +4742,8 @@ static const struct ethtool_ops macb_ethtool_ops = {
.set_link_ksettings = macb_set_link_ksettings,
.get_ringparam = macb_get_ringparam,
.set_ringparam = macb_set_ringparam,
+ .get_channels = macb_get_channels,
+ .set_channels = macb_set_channels,
};
static int macb_get_eee(struct net_device *netdev, struct ethtool_keee *eee)
@@ -4667,12 +4778,14 @@ static const struct ethtool_ops gem_ethtool_ops = {
.set_link_ksettings = macb_set_link_ksettings,
.get_ringparam = macb_get_ringparam,
.set_ringparam = macb_set_ringparam,
- .get_rxnfc = gem_get_rxnfc,
- .set_rxnfc = gem_set_rxnfc,
- .get_rx_ring_count = gem_get_rx_ring_count,
- .nway_reset = phy_ethtool_nway_reset,
+ .get_rxnfc = gem_get_rxnfc,
+ .set_rxnfc = gem_set_rxnfc,
+ .get_rx_ring_count = gem_get_rx_ring_count,
+ .nway_reset = phy_ethtool_nway_reset,
.get_eee = macb_get_eee,
.set_eee = macb_set_eee,
+ .get_channels = macb_get_channels,
+ .set_channels = macb_set_channels,
};
static int macb_ioctl(struct net_device *netdev, struct ifreq *rq, int cmd)
@@ -4817,9 +4930,9 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
return -EINVAL;
}
- if (conf->num_entries > bp->num_queues) {
+ if (conf->num_entries > bp->configured_num_queues) {
netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n",
- conf->num_entries, bp->num_queues);
+ conf->num_entries, bp->configured_num_queues);
return -EINVAL;
}
@@ -4848,10 +4961,10 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
/* gate_mask must not select queues outside the valid queues */
queue_id = order_base_2(entry->gate_mask);
- if (queue_id >= bp->num_queues) {
+ if (queue_id >= bp->configured_num_queues) {
netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n",
i, entry->gate_mask,
- bp->num_queues);
+ bp->configured_num_queues);
return -EINVAL;
}
@@ -5193,7 +5306,7 @@ static int macb_init_dflt(struct platform_device *pdev)
* register mapping but we don't want to test the queue index then
* compute the corresponding register offset at run time.
*/
- for (hw_q = 0, q = 0; hw_q < bp->num_queues; ++hw_q) {
+ for (hw_q = 0, q = 0; hw_q < bp->max_num_queues; ++hw_q) {
queue = &bp->queues[q];
queue->bp = bp;
spin_lock_init(&queue->tx_ptr_lock);
@@ -5528,6 +5641,7 @@ static int at91ether_open(struct net_device *netdev)
ret = -ENOMEM;
goto pm_exit;
}
+ bp->ctx->num_queues = 1;
bp->ctx->info = &bp->info;
/* Clear internal statistics */
@@ -6323,7 +6437,8 @@ static int macb_probe(struct platform_device *pdev)
bp->macb_reg_readl = hw_readl;
bp->macb_reg_writel = hw_writel;
}
- bp->num_queues = num_queues;
+ bp->max_num_queues = num_queues;
+ bp->configured_num_queues = num_queues;
bp->dma_burst_length = macb_config->dma_burst_length;
bp->pclk = pclk;
bp->hclk = hclk;
@@ -6532,6 +6647,7 @@ static int __maybe_unused macb_suspend(struct device *dev)
{
struct net_device *netdev = dev_get_drvdata(dev);
struct macb *bp = netdev_priv(netdev);
+ struct macb_context *ctx = bp->ctx;
struct in_ifaddr *ifa = NULL;
struct macb_queue *queue;
struct in_device *idev;
@@ -6573,7 +6689,7 @@ static int __maybe_unused macb_suspend(struct device *dev)
macb_writel(bp, RBQPH,
upper_32_bits(bp->rx_ring_tieoff_dma));
#endif
- for (q = 0, queue = bp->queues; q < bp->num_queues;
+ for (q = 0, queue = bp->queues; q < ctx->num_queues;
++q, ++queue) {
/* Disable RX queues */
if (bp->caps & MACB_CAPS_QUEUE_DISABLE) {
@@ -6614,7 +6730,7 @@ static int __maybe_unused macb_suspend(struct device *dev)
}
netif_device_detach(netdev);
- for (q = 0, queue = bp->queues; q < bp->num_queues;
+ for (q = 0, queue = bp->queues; q < ctx->num_queues;
++q, ++queue) {
napi_disable(&queue->napi_rx);
napi_disable(&queue->napi_tx);
@@ -6689,7 +6805,7 @@ static int __maybe_unused macb_resume(struct device *dev)
if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
macb_init_buffers(bp);
- for (q = 0, queue = bp->queues; q < bp->num_queues;
+ for (q = 0, queue = bp->queues; q < ctx->num_queues;
++q, ++queue) {
if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) {
if (macb_is_gem(&bp->info))
--
2.56.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
@ 2026-11-18 20:53 ` Théo Lebrun
2026-11-19 11:45 ` Nicolai Buchwitz
2 siblings, 1 reply; 9+ messages in thread
From: Théo Lebrun @ 2026-11-18 20:53 UTC (permalink / raw)
To: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni, Théo Lebrun
GEM has registers to configure the Tx SRAM segments distribution across
active queues. The reset value is appropriate (even spread) but we need
to care if/when number of active queues is modified (or if we inherited
unevenly initialised hardware from bootloader).
To distribute segments, we take as input the number of queues (current
context num_queues) and the number of segments (found inside DCFG6).
Its output is a number of segments for each queue, formatted as
powers-of-two. For example, 2 for queue 0 means it has 2^2=4 segments.
We define register offset GEM_TXQSEGALLOC_UPPER which is never used.
It shall be used when an IP supports >8 number of queues.
Note that the manual mentions explicitly that only active queues are
taken into account. So when we write 0x1 in TXQSEGALLOC and a single
queue is active, this means we assign (1<<1) == 2 segments to the
first (active) queue and none on the others.
Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
---
drivers/net/ethernet/cadence/macb.h | 14 ++++++
drivers/net/ethernet/cadence/macb_main.c | 73 +++++++++++++++++++++++++++++++-
2 files changed, 85 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index d60b06e41b83..f8335b9fd129 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -194,6 +194,10 @@
/* Screener Type 2 match registers */
#define GEM_SCRT2 0x540
+/* Tx queue segment allocation */
+#define GEM_TXQSEGALLOC_LOWER 0x05A0
+#define GEM_TXQSEGALLOC_UPPER 0x05A4
+
/* EtherType registers */
#define GEM_ETHT 0x06E0
@@ -545,6 +549,8 @@
#define GEM_PBUF_CUTTHRU_SIZE 1
#define GEM_DAW64_OFFSET 23
#define GEM_DAW64_SIZE 1
+#define GEM_SEGMENTS_BIT_SIZE_OFFSET 16
+#define GEM_SEGMENTS_BIT_SIZE_SIZE 4
/* Bitfields in DCFG8. */
#define GEM_T1SCR_OFFSET 24
@@ -657,6 +663,13 @@
#define GEM_CMPCEN_OFFSET 30 /* Compare C Enable */
#define GEM_CMPCEN_SIZE 1
+/* Bitfields in TXQSEGALLOC (Q0 at 2:0, Q1 at 6:4, etc.) */
+#define GEM_TXQSEGALLOC_MASK(q_) \
+ ({ \
+ typeof(q_) q__ = (q_); \
+ GENMASK(4 * (q__) + 2, 4 * (q__)); \
+ })
+
/* Bitfields in ETHT */
#define GEM_ETHTCMP_OFFSET 0 /* EtherType compare value */
#define GEM_ETHTCMP_SIZE 16
@@ -785,6 +798,7 @@
#define MACB_CAPS_USRIO_HAS_MII BIT(26)
#define MACB_CAPS_USRIO_HAS_REFCLK_SOURCE BIT(27)
#define MACB_CAPS_USRIO_HAS_TSUCLK_SOURCE BIT(28)
+#define MACB_CAPS_TXQSEGALLOC BIT(29)
/* LSO settings */
#define MACB_LSO_UFO_ENABLE 0x01
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 180da5d62f09..5d382f0e4aec 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -3319,6 +3319,69 @@ static u32 macb_dbw(struct macb *bp)
}
}
+/*
+ * Distribute evenly available segments across active queues. The computation
+ * is complex because (1) segments are counted in powers of two and (2) a queue
+ * can only use up to 128 segments. There are four types of cases:
+ * - Sharing all segments equally is doable. Take num_queues=4 and
+ * num_segments=16. Each queue will get 2^2=4 segments.
+ * - Sharing all segments is doable. Take num_queues=5 and num_segments=16.
+ * Three queues will get 2^2=4 segments and two will get 2^1=2 segments.
+ * - Sharing all segments is not doable because not enough queues are
+ * available. Take num_queues=1 and num_segments=256; queue 0 can only
+ * have 128 segments.
+ * - Sharing all segments is not doable because not enough segments are
+ * available. Take num_queues=4 and num_segments=2.
+ *
+ * We start by computing the power each queue will have. For num_queues=5 and
+ * num_segments=16, each queue will have at least 2^1 segments. That leaves us
+ * with remaining_segments=6. If we increase the power for a queue, we get a
+ * delta of 2 (2^2-2^1). The first three queues will therefore be advantaged
+ * and each have 2^2 segments. The remaining 2 queues will only have 2^1
+ * segments.
+ */
+static void gem_sram_distribute_segments(struct macb *bp)
+{
+ unsigned int num_segments, pow, remaining_segments, i;
+ unsigned int num_queues = bp->ctx->num_queues;
+ unsigned int num_advantaged_queues = 0;
+ u32 val = 0;
+
+ if (!(bp->caps & MACB_CAPS_TXQSEGALLOC))
+ return;
+
+ num_segments = 1U << GEM_BFEXT(SEGMENTS_BIT_SIZE, gem_readl(bp, DCFG6));
+
+ /* When growing the max # of queues, make sure to return a u64 here and
+ * to write into GEM_TXQSEGALLOC_UPPER.
+ */
+ static_assert(MACB_MAX_QUEUES == 8);
+
+ /* pow=0 for all queues. ilog2(0) is dangerous. */
+ if (num_queues >= num_segments)
+ goto writel;
+
+ pow = min_t(unsigned int, ilog2(num_segments / num_queues), 7);
+ remaining_segments = num_segments - num_queues * (1U << pow);
+
+ /* We can only distribute remaining segments if (1) there are remaining
+ * segments and (2) we did not reach the max segments per queue (2^7).
+ */
+ if (remaining_segments != 0 && pow != 7) {
+ unsigned int delta = (1U << (pow + 1)) - (1U << pow);
+
+ num_advantaged_queues = remaining_segments / delta;
+ }
+
+ for (i = 0; i < num_advantaged_queues; i++)
+ val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow + 1);
+ for (i = num_advantaged_queues; i < num_queues; i++)
+ val |= field_prep(GEM_TXQSEGALLOC_MASK(i), pow);
+
+writel:
+ gem_writel(bp, TXQSEGALLOC_LOWER, val);
+}
+
/* Configure the receive DMA engine
* - use the correct receive buffer size
* - set best burst length for DMA operations
@@ -3450,6 +3513,8 @@ static void macb_context_swap_end(struct macb *bp,
/* Re-arm normal interrupt processing before enabling IRQs. */
bp->irq_quiesced = false;
+ gem_sram_distribute_segments(bp);
+
macb_configure_dma(bp);
if (bp->link_up) {
@@ -3517,6 +3582,8 @@ static void macb_init_hw(struct macb *bp)
if (bp->caps & MACB_CAPS_JUMBO)
bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK;
+ gem_sram_distribute_segments(bp);
+
macb_configure_dma(bp);
/* Enable RX partial store and forward and set watermark */
@@ -6226,6 +6293,7 @@ static const struct macb_config zynqmp_config = {
.caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE |
MACB_CAPS_JUMBO |
MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH |
+ MACB_CAPS_TXQSEGALLOC |
MACB_CAPS_USRIO_HAS_MII,
.dma_burst_length = 16,
.init = init_reset_optional,
@@ -6293,7 +6361,7 @@ static const struct macb_config versal_config = {
.caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO |
MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH |
MACB_CAPS_NEED_TSUCLK | MACB_CAPS_QUEUE_DISABLE |
- MACB_CAPS_QBV |
+ MACB_CAPS_QBV | MACB_CAPS_TXQSEGALLOC |
MACB_CAPS_USRIO_HAS_MII,
.dma_burst_length = 16,
.init = init_reset_optional,
@@ -6304,7 +6372,8 @@ static const struct macb_config versal_config = {
static const struct macb_config eyeq5_config = {
.caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO |
MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_QUEUE_DISABLE |
- MACB_CAPS_NO_LSO | MACB_CAPS_EEE | MACB_CAPS_QBV,
+ MACB_CAPS_NO_LSO | MACB_CAPS_EEE | MACB_CAPS_QBV |
+ MACB_CAPS_TXQSEGALLOC,
.dma_burst_length = 16,
.init = eyeq5_init,
.jumbo_max_len = 10240,
--
2.56.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
@ 2026-11-19 11:45 ` Nicolai Buchwitz
0 siblings, 0 replies; 9+ messages in thread
From: Nicolai Buchwitz @ 2026-11-19 11:45 UTC (permalink / raw)
To: Théo Lebrun
Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel,
Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT,
Benoît Monin, Tawfik Bayouk, Thomas Petazzoni
Hi Théo
On 18.11.2026 21:53, Théo Lebrun wrote:
> GEM has registers to configure the Tx SRAM segments distribution across
> active queues. The reset value is appropriate (even spread) but we need
> to care if/when number of active queues is modified (or if we inherited
> unevenly initialised hardware from bootloader).
>
> To distribute segments, we take as input the number of queues (current
> context num_queues) and the number of segments (found inside DCFG6).
> Its output is a number of segments for each queue, formatted as
> powers-of-two. For example, 2 for queue 0 means it has 2^2=4 segments.
>
> We define register offset GEM_TXQSEGALLOC_UPPER which is never used.
> It shall be used when an IP supports >8 number of queues.
>
> Note that the manual mentions explicitly that only active queues are
> taken into account. So when we write 0x1 in TXQSEGALLOC and a single
> queue is active, this means we assign (1<<1) == 2 segments to the
> first (active) queue and none on the others.
>
> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
> [...]
Reviewed-by: Nicolai Buchwitz <nb@tipi-net.de>
Thanks,
Nicolai
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
@ 2026-11-19 11:49 ` Nicolai Buchwitz
2026-11-19 12:36 ` Vineeth Karumanchi
1 sibling, 0 replies; 9+ messages in thread
From: Nicolai Buchwitz @ 2026-11-19 11:49 UTC (permalink / raw)
To: Théo Lebrun
Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel,
Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT,
Benoît Monin, Tawfik Bayouk, Thomas Petazzoni
On 18.11.2026 21:53, Théo Lebrun wrote:
> Currently, ENST is not explicitly restored across interface close/open.
> If HW keeps its register state then ENST is kept, otherwise it is lost;
> but we do not write the ENST schedule again on open.
>
> This causes trouble if the config is not compatible with the new open
> and link-up. The two issues that can occur are a link speed change (as
> all timings are configured in a unit relative to the link speed) or a
> change in the # of queues.
>
> The latter is the main reason we do this change before implementing
> ethtool -L support.
>
> If on link-up, the cached ENST config is invalid (link speed changed),
> then we drop the cached config, log a warning and explicitly override
> all ENST timings previously configured in the registers. If it is
> valid, we explicitly re-apply it.
>
> Take this opportunity to replace a kmalloc in
> macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
>
> Drop the old hack of checking for pm_runtime_suspended(), which has
> little value because we could race with the runtime_suspend() call.
>
> Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
> by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
> ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
> phylink_ethtool_ksettings_get(), cache the link speed as
> bp->speed_mbps.
>
> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
> [...]
Reviewed-by: Nicolai Buchwitz <nb@tipi-net.de>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels()
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
@ 2026-11-19 11:54 ` Nicolai Buchwitz
0 siblings, 0 replies; 9+ messages in thread
From: Nicolai Buchwitz @ 2026-11-19 11:54 UTC (permalink / raw)
To: Théo Lebrun
Cc: Conor Dooley, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Russell King, netdev, linux-kernel,
Paolo Valerio, Vladimir Kondratiev, Gregory CLEMENT,
Benoît Monin, Tawfik Bayouk, Thomas Petazzoni
Hi Théo
On 18.11.2026 21:53, Théo Lebrun wrote:
> [...]
> @@ -3376,7 +3398,7 @@ static void macb_context_swap_start(struct macb
> *bp)
> [...]
> + if (new_ctx) {
> + old_ctx = bp->ctx;
> + WRITE_ONCE(bp->ctx, new_ctx);
> + wmb(); /* ensure IRQ enabled below read the new context */
> +
> + macb_init_buffers(bp);
> +
> + /* Clear DQL (only on successful context swap). */
> + for (q = 0; q < bp->ctx->num_queues; ++q)
> + netdev_tx_reset_queue(netdev_get_tx_queue(bp->netdev,
> + q));
> + }
The queues we just dropped never get their BQL reset, not on close or
open either. So after shrinking and growing again they can stay stuck.
Maybe just reset all max_num_queues here and in macb_close()?
> [...]
> +static int macb_set_channels(struct net_device *netdev,
> [...]
> + ret = netif_set_real_num_queues(bp->netdev, num_queues, num_queues);
> + if (ret) {
> + if (running) {
> + macb_free(new_ctx);
> + kfree(new_ctx);
> + macb_context_swap_end(bp, NULL);
swap_start() cleared TE, so HW is back at the ring start. Doesn't this
needs a gem_shuffle_tx_rings() like link_up does, otherwise Tx stalls?
> [...]
Regards,
Nicolai
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
2026-11-19 11:49 ` Nicolai Buchwitz
@ 2026-11-19 12:36 ` Vineeth Karumanchi
2026-11-19 15:40 ` Théo Lebrun
1 sibling, 1 reply; 9+ messages in thread
From: Vineeth Karumanchi @ 2026-11-19 12:36 UTC (permalink / raw)
To: Théo Lebrun, Conor Dooley, Andrew Lunn, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni
Hi Théo,
On 11/19/2026 2:23 AM, Théo Lebrun wrote:
> Currently, ENST is not explicitly restored across interface close/open.
> If HW keeps its register state then ENST is kept, otherwise it is lost;
> but we do not write the ENST schedule again on open.
>
> This causes trouble if the config is not compatible with the new open
> and link-up. The two issues that can occur are a link speed change (as
> all timings are configured in a unit relative to the link speed) or a
> change in the # of queues.
>
Thanks for the patch.
We are working on the same problem for CBS, covering the four scenarios
listed in:
https://lore.kernel.org/netdev/a04aa4d5-335b-47c1-b969-e89b18c9b142@amd.com/
Since this patch reworks how ENST state is kept and restored, it would
be good to introduce a generic top-level tc structure here, which ENST,
CBS and any later TSN offloads can share, instead of an ENST-specific one.
> The latter is the main reason we do this change before implementing
> ethtool -L support.
>
> If on link-up, the cached ENST config is invalid (link speed changed),
> then we drop the cached config, log a warning and explicitly override
> all ENST timings previously configured in the registers. If it is
> valid, we explicitly re-apply it.
>
> Take this opportunity to replace a kmalloc in
> macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
>
> Drop the old hack of checking for pm_runtime_suspended(), which has
> little value because we could race with the runtime_suspend() call.
>
> Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
> by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
> ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
> phylink_ethtool_ksettings_get(), cache the link speed as bp->speed_mbps.
>
> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
> ---
> drivers/net/ethernet/cadence/macb.h | 47 +++++---
> drivers/net/ethernet/cadence/macb_main.c | 187 +++++++++++++++----------------
> 2 files changed, 124 insertions(+), 110 deletions(-)
>
> diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
> index b7b4b6a38298..00258866623f 100644
> --- a/drivers/net/ethernet/cadence/macb.h
> +++ b/drivers/net/ethernet/cadence/macb.h
> @@ -1327,6 +1327,35 @@ struct macb_context {
> struct macb_txq txq[MACB_MAX_QUEUES];
> };
>
> +/**
> + * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> + * @start_time_mask: Bitmask representing the start time for the queue
> + * @on_time_bytes: "on" time nsec expressed in bytes
> + * @off_time_bytes: "off" time nsec expressed in bytes
> + *
> + * This structure holds the configuration parameters for an ENST queue,
> + * used to control time-based transmission scheduling in the MACB driver.
> + */
> +struct macb_queue_enst_config {
> + u32 start_time_mask;
> + u32 on_time_bytes;
> + u32 off_time_bytes;
> +};
> +
> +/**
> + * struct macb_enst - Cached Enhanced Scheduled Traffic configuration
> + * @queues: per-queue timing (zeroed for queues without schedule)
> + * @speed_mbps: link speed the timings were computed for, 0 if no schedule
> + * @queue_mask: queues enabled by the current schedule, 0 if no schedule
> + *
> + * The timings are expressed in bytes, so they depend on the link speed.
> + */
> +struct macb_enst {
> + struct macb_queue_enst_config queues[MACB_MAX_QUEUES];
> + u32 speed_mbps;
> + u32 queue_mask;
> +};
> +
Could this be a per-queue structure shared by all tc offloads, so that
new features only add their own part?
For example:
struct macb_tc_queue {
struct {
u32 idleslope; /* kbps */
} cbs;
struct {
u32 start_time; /* ENST_START_TIME register value */
u32 on_time_ns; /* gate open interval */
u64 off_time_ns; /* gate closed interval */
} enst;
};
> struct macb {
> void __iomem *regs;
> bool native_io;
> @@ -1427,11 +1456,14 @@ struct macb {
>
> /* Redundant to netif_carrier_ok(), but set under bp->mac_cfg_lock. */
> bool link_up;
> + u32 speed_mbps;
>
> u32 rx_intr_mask;
>
> struct macb_pm_data pm_data;
> const struct macb_usrio_config *usrio;
> +
> + struct macb_enst enst;
> };
>
> #ifdef CONFIG_MACB_USE_HWSTAMP
> @@ -1532,19 +1564,4 @@ struct macb_platform_data {
> struct clk *hclk;
> };
>
> -/**
> - * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> - * @start_time_mask: Bitmask representing the start time for the queue
> - * @on_time_bytes: "on" time nsec expressed in bytes
> - * @off_time_bytes: "off" time nsec expressed in bytes
> - *
> - * This structure holds the configuration parameters for an ENST queue,
> - * used to control time-based transmission scheduling in the MACB driver.
> - */
> -struct macb_queue_enst_config {
> - u32 start_time_mask;
> - u32 on_time_bytes;
> - u32 off_time_bytes;
> -};
> -
> #endif /* _MACB_H */
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index 9f6111661b40..1e0666692116 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -847,6 +847,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
> mutex_lock(&bp->mac_cfg_lock);
>
> bp->link_up = false;
> + bp->speed_mbps = 0;
>
> if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
> for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
> @@ -955,6 +956,43 @@ static void gem_shuffle_tx_rings(struct macb *bp)
> gem_shuffle_tx_one_ring(queue);
> }
>
> +static void macb_apply_enst_config(struct macb *bp)
> +{
> + struct macb_queue_enst_config *config;
> + struct macb_queue *queue;
> + u32 queue_mask;
> + size_t q;
> +
> + /* Short-circuit to avoid writing to invalid register offsets */
> + if (!(bp->caps & MACB_CAPS_QBV))
> + return;
> +
> + guard(spinlock_irqsave)(&bp->lock);
> +
> + /* Disable ENST for all queues.
> + * Reconfiguring timings must be done with ENST disabled.
> + */
> + queue_mask = BIT_U32(bp->num_queues) - 1;
> + gem_writel(bp, ENST_CONTROL,
> + queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> +
> + for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> + /* Configure queue timing registers.
> + *
> + * Those fields must be zero if the queue has ENST disabled:
> + * notice we don't bother reading bp->enst.queue_mask & BIT(q).
> + */
> + config = &bp->enst.queues[q];
> + queue_writel(queue, ENST_START_TIME, config->start_time_mask);
> + queue_writel(queue, ENST_ON_TIME, config->on_time_bytes);
> + queue_writel(queue, ENST_OFF_TIME, config->off_time_bytes);
> + }
> +
> + /* Enable ENST for all configured queues in one write */
> + if (bp->enst.queue_mask)
> + gem_writel(bp, ENST_CONTROL, bp->enst.queue_mask);
> +}
> +
> static void macb_mac_link_up(struct phylink_config *config,
> struct phy_device *phydev,
> unsigned int mode, phy_interface_t interface,
> @@ -972,6 +1010,7 @@ static void macb_mac_link_up(struct phylink_config *config,
> spin_lock_irqsave(&bp->lock, flags);
>
> bp->link_up = true;
> + bp->speed_mbps = speed > 0 ? speed : 0;
>
> ctrl = macb_or_gem_readl(bp, NCFGR);
>
> @@ -1009,6 +1048,15 @@ static void macb_mac_link_up(struct phylink_config *config,
>
> spin_unlock_irqrestore(&bp->lock, flags);
>
> + if (bp->enst.speed_mbps && speed != bp->enst.speed_mbps) {
Why drop the schedule on a speed change? The taprio qdisc stays
installed, so the user's configuration is still valid;
Could we instead recompute the ON/OFF times for the new speed on
link-up, and only disable (with a warning) the queues whose intervals no
longer fit the hardware range at that speed?
> + netdev_warn(bp->netdev,
> + "TAPRIO dropped because link speed changed\n");
> +
> + memset(&bp->enst, 0, sizeof(bp->enst));
> + }
> +
> + macb_apply_enst_config(bp);
> +
Could this be a single macb_apply_tc_config() (or similar) that
reprograms every enabled tc offload? CBS needs exactly the same
handling, since idleslope is also relative to the link rate.
Thanks,
> if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) {
> macb_set_tx_clk(bp, speed);
> gem_shuffle_tx_rings(bp);
> @@ -4754,19 +4802,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> struct tc_taprio_qopt_offload *conf)
> {
> u64 total_on_time = 0, start_time_sec = 0, start_time = conf->base_time;
> - u32 configured_queues = 0, speed = 0, start_time_nsec;
> - struct macb_queue_enst_config *enst_queue;
> - struct ethtool_link_ksettings kset = {};
> + u32 configured_queues = 0, start_time_nsec;
> struct macb *bp = netdev_priv(netdev);
> struct tc_taprio_sched_entry *entry;
> - struct macb_queue *queue;
> - u32 queue_mask;
> + struct macb_enst config = {};
> u8 queue_id;
> - size_t i, q;
> - int err;
> + u64 off;
> + size_t i;
> +
> + guard(mutex)(&bp->mac_cfg_lock);
> +
> + if (unlikely(!bp->speed_mbps)) {
> + netdev_err(netdev, "Invalid speed, link-down?\n");
> + return -EINVAL;
> + }
>
> if (conf->num_entries > bp->num_queues) {
> - netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n",
> + netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n",
> conf->num_entries, bp->num_queues);
> return -EINVAL;
> }
> @@ -4777,23 +4829,6 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> return -ERANGE;
> }
>
> - /* Get the current link speed */
> - err = phylink_ethtool_ksettings_get(bp->phylink, &kset);
> - if (unlikely(err)) {
> - netdev_err(netdev, "Failed to get link settings: %d\n", err);
> - return err;
> - }
> -
> - speed = kset.base.speed;
> - if (unlikely(speed == SPEED_UNKNOWN || !speed)) {
> - netdev_err(netdev, "Invalid speed %d, link-down?\n", speed);
> - return -EINVAL;
> - }
> -
> - enst_queue = kzalloc_objs(*enst_queue, bp->num_queues);
> - if (unlikely(!enst_queue))
> - return -ENOMEM;
> -
> /* Pre-validate all entries before making any hardware changes */
> for (i = 0; i < conf->num_entries; i++) {
> entry = &conf->entries[i];
> @@ -4801,25 +4836,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (entry->command != TC_TAPRIO_CMD_SET_GATES) {
> netdev_err(netdev, "Entry %zu: unsupported command %d\n",
> i, entry->command);
> - err = -EOPNOTSUPP;
> - goto cleanup;
> + return -EOPNOTSUPP;
> }
>
> /* Validate gate_mask: must be nonzero, single queue, and within range */
> if (!is_power_of_2(entry->gate_mask)) {
> netdev_err(netdev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
> i, entry->gate_mask);
> - err = -EINVAL;
> - goto cleanup;
> + return -EINVAL;
> }
>
> /* gate_mask must not select queues outside the valid queues */
> queue_id = order_base_2(entry->gate_mask);
> if (queue_id >= bp->num_queues) {
> - netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
> - i, entry->gate_mask, bp->num_queues);
> - err = -EINVAL;
> - goto cleanup;
> + netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n",
> + i, entry->gate_mask,
> + bp->num_queues);
> + return -EINVAL;
> }
>
> /* Check for start time limits */
> @@ -4828,34 +4861,33 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (start_time_sec > GENMASK(GEM_START_TIME_SEC_SIZE - 1, 0)) {
> netdev_err(netdev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
> i, start_time_sec);
> - err = -ERANGE;
> - goto cleanup;
> + return -ERANGE;
> }
>
> /* Check for on time limit */
> - if (entry->interval > enst_max_hw_interval(speed)) {
> + if (entry->interval > enst_max_hw_interval(bp->speed_mbps)) {
> netdev_err(netdev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
> - i, entry->interval, enst_max_hw_interval(speed));
> - err = -ERANGE;
> - goto cleanup;
> + i, entry->interval,
> + enst_max_hw_interval(bp->speed_mbps));
> + return -ERANGE;
> }
>
> /* Check for off time limit*/
> - if ((conf->cycle_time - entry->interval) > enst_max_hw_interval(speed)) {
> + off = conf->cycle_time - entry->interval;
> + if (off > enst_max_hw_interval(bp->speed_mbps)) {
> netdev_err(netdev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
> - i, conf->cycle_time - entry->interval,
> - enst_max_hw_interval(speed));
> - err = -ERANGE;
> - goto cleanup;
> + i, off,
> + enst_max_hw_interval(bp->speed_mbps));
> + return -ERANGE;
> }
>
> - enst_queue[queue_id].start_time_mask =
> + config.queues[queue_id].start_time_mask =
> (start_time_sec << GEM_START_TIME_SEC_OFFSET) |
> start_time_nsec;
> - enst_queue[queue_id].on_time_bytes =
> - enst_ns_to_hw_units(entry->interval, speed);
> - enst_queue[queue_id].off_time_bytes =
> - enst_ns_to_hw_units(conf->cycle_time - entry->interval, speed);
> + config.queues[queue_id].on_time_bytes =
> + enst_ns_to_hw_units(entry->interval, bp->speed_mbps);
> + config.queues[queue_id].off_time_bytes =
> + enst_ns_to_hw_units(off, bp->speed_mbps);
>
> configured_queues |= entry->gate_mask;
> total_on_time += entry->interval;
> @@ -4866,82 +4898,47 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (total_on_time > conf->cycle_time) {
> netdev_err(netdev, "Total ON %llu ns exceeds cycle time %llu ns\n",
> total_on_time, conf->cycle_time);
> - err = -EINVAL;
> - goto cleanup;
> + return -EINVAL;
> }
>
> netdev_dbg(netdev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
> conf->num_entries, conf->base_time, conf->cycle_time);
>
> - /* All validations passed - proceed with hardware configuration */
> - scoped_guard(spinlock_irqsave, &bp->lock) {
> - /* Disable ENST queues if running before configuring */
> - queue_mask = BIT_U32(bp->num_queues) - 1;
> - gem_writel(bp, ENST_CONTROL,
> - queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> + config.queue_mask = configured_queues;
> + config.speed_mbps = bp->speed_mbps;
> + bp->enst = config;
>
> - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> - /* Configure queue timing registers */
> - queue_writel(queue, ENST_START_TIME,
> - enst_queue[q].start_time_mask);
> - queue_writel(queue, ENST_ON_TIME,
> - enst_queue[q].on_time_bytes);
> - queue_writel(queue, ENST_OFF_TIME,
> - enst_queue[q].off_time_bytes);
> - }
> -
> - /* Enable ENST for all configured queues in one write */
> - gem_writel(bp, ENST_CONTROL, configured_queues);
> - }
> + macb_apply_enst_config(bp);
>
> netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
> conf->num_entries, hweight32(configured_queues));
>
> -cleanup:
> - kfree(enst_queue);
> - return err;
> + return 0;
> }
>
> static void macb_taprio_destroy(struct net_device *netdev)
> {
> struct macb *bp = netdev_priv(netdev);
> - struct macb_queue *queue;
> - u32 queue_mask;
> - unsigned int q;
> +
> + guard(mutex)(&bp->mac_cfg_lock);
>
> netdev_reset_tc(netdev);
> - queue_mask = BIT_U32(bp->num_queues) - 1;
> + memset(&bp->enst, 0, sizeof(bp->enst));
>
> - scoped_guard(spinlock_irqsave, &bp->lock) {
> - /* Single disable command for all queues */
> - gem_writel(bp, ENST_CONTROL,
> - queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> + if (bp->link_up)
> + macb_apply_enst_config(bp);
>
> - /* Clear all queue ENST registers in batch */
> - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> - queue_writel(queue, ENST_START_TIME, 0);
> - queue_writel(queue, ENST_ON_TIME, 0);
> - queue_writel(queue, ENST_OFF_TIME, 0);
> - }
> - }
> netdev_info(netdev, "TAPRIO destroy: All gates disabled\n");
> }
>
> static int macb_setup_taprio(struct net_device *netdev,
> struct tc_taprio_qopt_offload *taprio)
> {
> - struct macb *bp = netdev_priv(netdev);
> int err = 0;
>
> if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC)))
> return -EOPNOTSUPP;
>
> - /* Check if Device is in runtime suspend */
> - if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) {
> - netdev_err(netdev, "Device is in runtime suspend\n");
> - return -EOPNOTSUPP;
> - }
> -
> switch (taprio->cmd) {
> case TAPRIO_CMD_REPLACE:
> err = macb_taprio_setup_replace(netdev, taprio);
>
--
🙏 Vineeth
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
2026-11-19 12:36 ` Vineeth Karumanchi
@ 2026-11-19 15:40 ` Théo Lebrun
0 siblings, 0 replies; 9+ messages in thread
From: Théo Lebrun @ 2026-11-19 15:40 UTC (permalink / raw)
To: Vineeth Karumanchi, Conor Dooley, Andrew Lunn, David S. Miller,
Eric Dumazet, Jakub Kicinski, Paolo Abeni, Russell King
Cc: netdev, linux-kernel, Nicolai Buchwitz, Paolo Valerio,
Vladimir Kondratiev, Gregory CLEMENT, Benoît Monin,
Tawfik Bayouk, Thomas Petazzoni
Hello Vineeth,
On Thu Nov 19, 2026 at 1:36 PM CET, Vineeth Karumanchi wrote:
> Hi Théo,
>
> On 11/19/2026 2:23 AM, Théo Lebrun wrote:
>> Currently, ENST is not explicitly restored across interface close/open.
>> If HW keeps its register state then ENST is kept, otherwise it is lost;
>> but we do not write the ENST schedule again on open.
>>
>> This causes trouble if the config is not compatible with the new open
>> and link-up. The two issues that can occur are a link speed change (as
>> all timings are configured in a unit relative to the link speed) or a
>> change in the # of queues.
>>
>
> Thanks for the patch.
>
> We are working on the same problem for CBS, covering the four scenarios
> listed in:
> https://lore.kernel.org/netdev/a04aa4d5-335b-47c1-b969-e89b18c9b142@amd.com/
>
> Since this patch reworks how ENST state is kept and restored, it would
> be good to introduce a generic top-level tc structure here, which ENST,
> CBS and any later TSN offloads can share, instead of an ENST-specific one.
Sorry I forgot replying on that thread. It is still marked as unread in
my mailbox!
Let me quote it here and reply:
On Thu Oct 8, 2026 at 5:27 PM CEST, Karumanchi, Vineeth wrote:
> On 9/17/2026 3:02 PM, Théo Lebrun wrote:
>> So we write cbs_control/0x04BC only on ndo_setup_tc(TC_SETUP_QDISC_CBS)
>> callback. As Sashiko pointed how I think that causes an issue with HW
>> resets & suspend:
>> - what if configured then link down then link up?
>> - what if configured before link up?
>>
>> Also when not using CBS we hope tx_sched_ctrl/0x0580 is at its reset
>> value (0b00, fixed priority) for all queues. I would expect an
>> unconditional writel tx_sched_ctrl/0x0580 in macb_init_hw(). This
>> solves the down/up issue, the speed change and also ensures we don't
>> inherit CBS or DWRR from the boot stages (super unlikely though).
>>
>> Also, it might be simpler to always use tx_sched_ctrl/0x0580 rather than
>> sometimes tx_sched_ctrl/0x0580 and sometimes cbs_control/0x04BC, which
>> are aliases.
>>
>> What do you think?
>
> Sorry for the delayed response.
>
> I tested the current implementation in the following four scenarios:
>
> 1. Configuring CBS while the link is down: CBS cannot be programmed
> because the link speed is unknown.
>
> 2. Link down followed by link up: The hardware retains the CBS register
> values, and CBS is effective again after the link comes up at the same
> speed.
Here you talk about link-down followed by link-up.
But about a close then open: this behavior might be board specific.
Runtime PM does suspend and this might shut off the IP on some
platforms and not others. That would in turn mean the IP loses its
state.
There is no easy way of knowing if that occurs on any platform that uses
MACB. It sounds probable though.
> 3. Suspend/resume: The CBS register contents are lost after resume.
> However, the CBS child qdisc remains installed in the kernel and is
> still shown by tc as offloaded, leaving the software configuration
> inconsistent with the hardware state.
Again, this is platform specific. Some boards + suspend combo keep the
SoC and its peripherals alive enough for them to keep their state.
A board with little runtime PM doing suspend-to-idle probably won't lose
CBS register state. A board doing proper suspend-to-RAM will surely
lose its state.
> 4. Link-speed change: The CBS register values are retained and shaping
> remains enabled. However, those values were calculated for the previous
> speed, so the resulting shaping bandwidth is incorrect.
>
> I see two possible approaches:
>
> - Retain the requested CBS parameters in the driver and restore the
> hardware configuration, following the saved-state approach used by IGB.
> For MACB, we would also recalculate the hardware idleslope on link-up
> using the negotiated speed.
> If the saved request cannot be supported at the new speed, retain the
> request, leave CBS disabled for that queue, and warn the user that the
> configuration is invalid for the new link speed.
>
> - Explicitly reset the scheduling configuration in macb_init_hw() and
> require users to remove and recreate the CBS qdisc. This establishes a
> known hardware state, but leaves the installed qdisc inconsistent until
> users reconfigure it. It would also need handling for speed changes that
> do not invoke macb_init_hw().
>
> Please let me know your thoughts on these approaches.
Option 2 is not super doable. There is no API for the driver to
say to the network stack and userspace "I'm dropping your qdisc, please
re-enable one if you care".
So ideally we would do option 1, but dealing with link speed changes
sounds complex. I'm super afraid of introducing bugs with code that
recomputes the timings on the fly. I'm sure I will create rounding bugs
in the edge cases.
That's why I made the cached state be in number of words and store the
link speed alongside. This means we cannot introduce such bugs AND is
less code overall AND is the common case.
It feels like easier and smaller incremental changes.
Now continuing onto the original quote:
>> The latter is the main reason we do this change before implementing
>> ethtool -L support.
>>
>> If on link-up, the cached ENST config is invalid (link speed changed),
>> then we drop the cached config, log a warning and explicitly override
>> all ENST timings previously configured in the registers. If it is
>> valid, we explicitly re-apply it.
>>
>> Take this opportunity to replace a kmalloc in
>> macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
>>
>> Drop the old hack of checking for pm_runtime_suspended(), which has
>> little value because we could race with the runtime_suspend() call.
>>
>> Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
>> by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
>> ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
>> phylink_ethtool_ksettings_get(), cache the link speed as bp->speed_mbps.
>>
>> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
>> ---
>> drivers/net/ethernet/cadence/macb.h | 47 +++++---
>> drivers/net/ethernet/cadence/macb_main.c | 187 +++++++++++++++----------------
>> 2 files changed, 124 insertions(+), 110 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
>> index b7b4b6a38298..00258866623f 100644
>> --- a/drivers/net/ethernet/cadence/macb.h
>> +++ b/drivers/net/ethernet/cadence/macb.h
>> @@ -1327,6 +1327,35 @@ struct macb_context {
>> struct macb_txq txq[MACB_MAX_QUEUES];
>> };
>>
>> +/**
>> + * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
>> + * @start_time_mask: Bitmask representing the start time for the queue
>> + * @on_time_bytes: "on" time nsec expressed in bytes
>> + * @off_time_bytes: "off" time nsec expressed in bytes
>> + *
>> + * This structure holds the configuration parameters for an ENST queue,
>> + * used to control time-based transmission scheduling in the MACB driver.
>> + */
>> +struct macb_queue_enst_config {
>> + u32 start_time_mask;
>> + u32 on_time_bytes;
>> + u32 off_time_bytes;
>> +};
>> +
>> +/**
>> + * struct macb_enst - Cached Enhanced Scheduled Traffic configuration
>> + * @queues: per-queue timing (zeroed for queues without schedule)
>> + * @speed_mbps: link speed the timings were computed for, 0 if no schedule
>> + * @queue_mask: queues enabled by the current schedule, 0 if no schedule
>> + *
>> + * The timings are expressed in bytes, so they depend on the link speed.
>> + */
>> +struct macb_enst {
>> + struct macb_queue_enst_config queues[MACB_MAX_QUEUES];
>> + u32 speed_mbps;
>> + u32 queue_mask;
>> +};
>> +
>
> Could this be a per-queue structure shared by all tc offloads, so that
> new features only add their own part?
> For example:
>
> struct macb_tc_queue {
> struct {
> u32 idleslope; /* kbps */
> } cbs;
> struct {
> u32 start_time; /* ENST_START_TIME register value */
> u32 on_time_ns; /* gate open interval */
> u64 off_time_ns; /* gate closed interval */
> } enst;
> };
Yes sure!
Either
// as you suggest
struct macb_tc_queue tc_queues[MACB_MAX_QUEUES];
Or:
struct macb_enst_queue enst_queues[MACB_MAX_QUEUES];
struct macb_cbs_queue cbs_queues[MACB_MAX_QUEUES];
// ... etc ...
Both would work.
>
>> struct macb {
>> void __iomem *regs;
>> bool native_io;
>> @@ -1427,11 +1456,14 @@ struct macb {
>>
>> /* Redundant to netif_carrier_ok(), but set under bp->mac_cfg_lock. */
>> bool link_up;
>> + u32 speed_mbps;
>>
>> u32 rx_intr_mask;
>>
>> struct macb_pm_data pm_data;
>> const struct macb_usrio_config *usrio;
>> +
>> + struct macb_enst enst;
>> };
>>
>> #ifdef CONFIG_MACB_USE_HWSTAMP
>> @@ -1532,19 +1564,4 @@ struct macb_platform_data {
>> struct clk *hclk;
>> };
>>
>> -/**
>> - * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
>> - * @start_time_mask: Bitmask representing the start time for the queue
>> - * @on_time_bytes: "on" time nsec expressed in bytes
>> - * @off_time_bytes: "off" time nsec expressed in bytes
>> - *
>> - * This structure holds the configuration parameters for an ENST queue,
>> - * used to control time-based transmission scheduling in the MACB driver.
>> - */
>> -struct macb_queue_enst_config {
>> - u32 start_time_mask;
>> - u32 on_time_bytes;
>> - u32 off_time_bytes;
>> -};
>> -
>> #endif /* _MACB_H */
>> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
>> index 9f6111661b40..1e0666692116 100644
>> --- a/drivers/net/ethernet/cadence/macb_main.c
>> +++ b/drivers/net/ethernet/cadence/macb_main.c
>> @@ -847,6 +847,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
>> mutex_lock(&bp->mac_cfg_lock);
>>
>> bp->link_up = false;
>> + bp->speed_mbps = 0;
>>
>> if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
>> for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
>> @@ -955,6 +956,43 @@ static void gem_shuffle_tx_rings(struct macb *bp)
>> gem_shuffle_tx_one_ring(queue);
>> }
>>
>> +static void macb_apply_enst_config(struct macb *bp)
>> +{
>> + struct macb_queue_enst_config *config;
>> + struct macb_queue *queue;
>> + u32 queue_mask;
>> + size_t q;
>> +
>> + /* Short-circuit to avoid writing to invalid register offsets */
>> + if (!(bp->caps & MACB_CAPS_QBV))
>> + return;
>> +
>> + guard(spinlock_irqsave)(&bp->lock);
>> +
>> + /* Disable ENST for all queues.
>> + * Reconfiguring timings must be done with ENST disabled.
>> + */
>> + queue_mask = BIT_U32(bp->num_queues) - 1;
>> + gem_writel(bp, ENST_CONTROL,
>> + queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
>> +
>> + for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
>> + /* Configure queue timing registers.
>> + *
>> + * Those fields must be zero if the queue has ENST disabled:
>> + * notice we don't bother reading bp->enst.queue_mask & BIT(q).
>> + */
>> + config = &bp->enst.queues[q];
>> + queue_writel(queue, ENST_START_TIME, config->start_time_mask);
>> + queue_writel(queue, ENST_ON_TIME, config->on_time_bytes);
>> + queue_writel(queue, ENST_OFF_TIME, config->off_time_bytes);
>> + }
>> +
>> + /* Enable ENST for all configured queues in one write */
>> + if (bp->enst.queue_mask)
>> + gem_writel(bp, ENST_CONTROL, bp->enst.queue_mask);
>> +}
>> +
>> static void macb_mac_link_up(struct phylink_config *config,
>> struct phy_device *phydev,
>> unsigned int mode, phy_interface_t interface,
>> @@ -972,6 +1010,7 @@ static void macb_mac_link_up(struct phylink_config *config,
>> spin_lock_irqsave(&bp->lock, flags);
>>
>> bp->link_up = true;
>> + bp->speed_mbps = speed > 0 ? speed : 0;
>>
>> ctrl = macb_or_gem_readl(bp, NCFGR);
>>
>> @@ -1009,6 +1048,15 @@ static void macb_mac_link_up(struct phylink_config *config,
>>
>> spin_unlock_irqrestore(&bp->lock, flags);
>>
>> + if (bp->enst.speed_mbps && speed != bp->enst.speed_mbps) {
>
> Why drop the schedule on a speed change? The taprio qdisc stays
> installed, so the user's configuration is still valid;
> Could we instead recompute the ON/OFF times for the new speed on
> link-up, and only disable (with a warning) the queues whose intervals no
> longer fit the hardware range at that speed?
See explanation above. It felt like a good local maximum: simple code
that probably doesn't introduce rounding bugs and solves it for most
usecases.
In any case, we will sometimes have to drop the qdisc and warn about it.
>
>> + netdev_warn(bp->netdev,
>> + "TAPRIO dropped because link speed changed\n");
>> +
>> + memset(&bp->enst, 0, sizeof(bp->enst));
>> + }
>> +
>> + macb_apply_enst_config(bp);
>> +
>
> Could this be a single macb_apply_tc_config() (or similar) that
> reprograms every enabled tc offload? CBS needs exactly the same
> handling, since idleslope is also relative to the link rate.
Yes sure!
Do you want to take over and handle this? You seem to be interested in
those TC features. You can either copy the patch as-is or modify it
under your own authorship, I don't mind.
Or if you prefer, I keep it here and iterate. But if I keep it, I'll
continue with ENST and then at a later stage we'll add CBS & co onto
it. This patch looks like a good starting point to add more onto it.
I'm mostly focused on getting a set_channels implementation (to then
have XDP/XSK) and I fixed ENST along the way.
Thanks,
--
Théo Lebrun, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-11-19 15:40 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
2026-11-19 11:49 ` Nicolai Buchwitz
2026-11-19 12:36 ` Vineeth Karumanchi
2026-11-19 15:40 ` Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
2026-11-19 11:54 ` Nicolai Buchwitz
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
2026-11-19 11:45 ` Nicolai Buchwitz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox