Netdev List
 help / color / mirror / Atom feed
From: Vineeth Karumanchi <vineeth.karumanchi@amd.com>
To: "Théo Lebrun" <theo.lebrun@bootlin.com>,
	"Conor Dooley" <conor.dooley@microchip.com>,
	"Andrew Lunn" <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	"Eric Dumazet" <edumazet@kernel.org>,
	"Jakub Kicinski" <kuba@kernel.org>,
	"Paolo Abeni" <pabeni@redhat.com>,
	"Russell King" <linux@armlinux.org.uk>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
	"Nicolai Buchwitz" <nb@tipi-net.de>,
	"Paolo Valerio" <pvalerio@redhat.com>,
	"Vladimir Kondratiev" <vladimir.kondratiev@mobileye.com>,
	"Gregory CLEMENT" <gregory.clement@bootlin.com>,
	"Benoît Monin" <benoit.monin@bootlin.com>,
	"Tawfik Bayouk" <tawfik.bayouk@mobileye.com>,
	"Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
Subject: Re: [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
Date: Thu, 19 Nov 2026 18:06:33 +0530	[thread overview]
Message-ID: <fe08db37-4415-4b4b-b226-cc1a0ec4f029@amd.com> (raw)
In-Reply-To: <20261118-macb-set-channels-v7-1-2dbc00f686af@bootlin.com>

Hi Théo,

On 11/19/2026 2:23 AM, Théo Lebrun wrote:
> Currently, ENST is not explicitly restored across interface close/open.
> If HW keeps its register state then ENST is kept, otherwise it is lost;
> but we do not write the ENST schedule again on open.
> 
> This causes trouble if the config is not compatible with the new open
> and link-up. The two issues that can occur are a link speed change (as
> all timings are configured in a unit relative to the link speed) or a
> change in the # of queues.
> 

Thanks for the patch.

We are working on the same problem for CBS, covering the four scenarios
listed in:
https://lore.kernel.org/netdev/a04aa4d5-335b-47c1-b969-e89b18c9b142@amd.com/

Since this patch reworks how ENST state is kept and restored, it would
be good to introduce a generic top-level tc structure here, which ENST,
CBS and any later TSN offloads can share, instead of an ENST-specific one.

> The latter is the main reason we do this change before implementing
> ethtool -L support.
> 
> If on link-up, the cached ENST config is invalid (link speed changed),
> then we drop the cached config, log a warning and explicitly override
> all ENST timings previously configured in the registers. If it is
> valid, we explicitly re-apply it.
> 
> Take this opportunity to replace a kmalloc in
> macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
> 
> Drop the old hack of checking for pm_runtime_suspended(), which has
> little value because we could race with the runtime_suspend() call.
> 
> Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
> by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
> ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
> phylink_ethtool_ksettings_get(), cache the link speed as bp->speed_mbps.
> 
> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
> ---
>  drivers/net/ethernet/cadence/macb.h      |  47 +++++---
>  drivers/net/ethernet/cadence/macb_main.c | 187 +++++++++++++++----------------
>  2 files changed, 124 insertions(+), 110 deletions(-)
> 
> diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
> index b7b4b6a38298..00258866623f 100644
> --- a/drivers/net/ethernet/cadence/macb.h
> +++ b/drivers/net/ethernet/cadence/macb.h
> @@ -1327,6 +1327,35 @@ struct macb_context {
>  	struct macb_txq		txq[MACB_MAX_QUEUES];
>  };
>  
> +/**
> + * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> + * @start_time_mask:  Bitmask representing the start time for the queue
> + * @on_time_bytes:    "on" time nsec expressed in bytes
> + * @off_time_bytes:   "off" time nsec expressed in bytes
> + *
> + * This structure holds the configuration parameters for an ENST queue,
> + * used to control time-based transmission scheduling in the MACB driver.
> + */
> +struct macb_queue_enst_config {
> +	u32 start_time_mask;
> +	u32 on_time_bytes;
> +	u32 off_time_bytes;
> +};
> +
> +/**
> + * struct macb_enst - Cached Enhanced Scheduled Traffic configuration
> + * @queues:     per-queue timing (zeroed for queues without schedule)
> + * @speed_mbps: link speed the timings were computed for, 0 if no schedule
> + * @queue_mask: queues enabled by the current schedule, 0 if no schedule
> + *
> + * The timings are expressed in bytes, so they depend on the link speed.
> + */
> +struct macb_enst {
> +	struct macb_queue_enst_config	queues[MACB_MAX_QUEUES];
> +	u32				speed_mbps;
> +	u32				queue_mask;
> +};
> +

Could this be a per-queue structure shared by all tc offloads, so that
new features only add their own part?
For example:

struct macb_tc_queue {
        struct {
                u32 idleslope;          /* kbps */
        } cbs;
        struct {
                u32 start_time;         /* ENST_START_TIME register value */
                u32 on_time_ns;         /* gate open interval */
                u64 off_time_ns;        /* gate closed interval */
        } enst;
};

>  struct macb {
>  	void __iomem		*regs;
>  	bool			native_io;
> @@ -1427,11 +1456,14 @@ struct macb {
>  
>  	/* Redundant to netif_carrier_ok(), but set under bp->mac_cfg_lock. */
>  	bool			link_up;
> +	u32			speed_mbps;
>  
>  	u32	rx_intr_mask;
>  
>  	struct macb_pm_data pm_data;
>  	const struct macb_usrio_config *usrio;
> +
> +	struct macb_enst enst;
>  };
>  
>  #ifdef CONFIG_MACB_USE_HWSTAMP
> @@ -1532,19 +1564,4 @@ struct macb_platform_data {
>  	struct clk	*hclk;
>  };
>  
> -/**
> - * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> - * @start_time_mask:  Bitmask representing the start time for the queue
> - * @on_time_bytes:    "on" time nsec expressed in bytes
> - * @off_time_bytes:   "off" time nsec expressed in bytes
> - *
> - * This structure holds the configuration parameters for an ENST queue,
> - * used to control time-based transmission scheduling in the MACB driver.
> - */
> -struct macb_queue_enst_config {
> -	u32 start_time_mask;
> -	u32 on_time_bytes;
> -	u32 off_time_bytes;
> -};
> -
>  #endif /* _MACB_H */
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index 9f6111661b40..1e0666692116 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -847,6 +847,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
>  	mutex_lock(&bp->mac_cfg_lock);
>  
>  	bp->link_up = false;
> +	bp->speed_mbps = 0;
>  
>  	if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
>  		for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
> @@ -955,6 +956,43 @@ static void gem_shuffle_tx_rings(struct macb *bp)
>  		gem_shuffle_tx_one_ring(queue);
>  }
>  
> +static void macb_apply_enst_config(struct macb *bp)
> +{
> +	struct macb_queue_enst_config *config;
> +	struct macb_queue *queue;
> +	u32 queue_mask;
> +	size_t q;
> +
> +	/* Short-circuit to avoid writing to invalid register offsets */
> +	if (!(bp->caps & MACB_CAPS_QBV))
> +		return;
> +
> +	guard(spinlock_irqsave)(&bp->lock);
> +
> +	/* Disable ENST for all queues.
> +	 * Reconfiguring timings must be done with ENST disabled.
> +	 */
> +	queue_mask = BIT_U32(bp->num_queues) - 1;
> +	gem_writel(bp, ENST_CONTROL,
> +		   queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> +
> +	for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> +		/* Configure queue timing registers.
> +		 *
> +		 * Those fields must be zero if the queue has ENST disabled:
> +		 * notice we don't bother reading bp->enst.queue_mask & BIT(q).
> +		 */
> +		config = &bp->enst.queues[q];
> +		queue_writel(queue, ENST_START_TIME, config->start_time_mask);
> +		queue_writel(queue, ENST_ON_TIME, config->on_time_bytes);
> +		queue_writel(queue, ENST_OFF_TIME, config->off_time_bytes);
> +	}
> +
> +	/* Enable ENST for all configured queues in one write */
> +	if (bp->enst.queue_mask)
> +		gem_writel(bp, ENST_CONTROL, bp->enst.queue_mask);
> +}
> +
>  static void macb_mac_link_up(struct phylink_config *config,
>  			     struct phy_device *phydev,
>  			     unsigned int mode, phy_interface_t interface,
> @@ -972,6 +1010,7 @@ static void macb_mac_link_up(struct phylink_config *config,
>  	spin_lock_irqsave(&bp->lock, flags);
>  
>  	bp->link_up = true;
> +	bp->speed_mbps = speed > 0 ? speed : 0;
>  
>  	ctrl = macb_or_gem_readl(bp, NCFGR);
>  
> @@ -1009,6 +1048,15 @@ static void macb_mac_link_up(struct phylink_config *config,
>  
>  	spin_unlock_irqrestore(&bp->lock, flags);
>  
> +	if (bp->enst.speed_mbps && speed != bp->enst.speed_mbps) {

Why drop the schedule on a speed change? The taprio qdisc stays
installed, so the user's configuration is still valid;
Could we instead recompute the ON/OFF times for the new speed on
link-up, and only disable (with a warning) the queues whose intervals no
longer fit the hardware range at that speed?

> +		netdev_warn(bp->netdev,
> +			    "TAPRIO dropped because link speed changed\n");
> +
> +		memset(&bp->enst, 0, sizeof(bp->enst));
> +	}
> +
> +	macb_apply_enst_config(bp);
> +

Could this be a single macb_apply_tc_config() (or similar) that
reprograms every enabled tc offload? CBS needs exactly the same
handling, since idleslope is also relative to the link rate.

Thanks,

>  	if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) {
>  		macb_set_tx_clk(bp, speed);
>  		gem_shuffle_tx_rings(bp);
> @@ -4754,19 +4802,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
>  				     struct tc_taprio_qopt_offload *conf)
>  {
>  	u64 total_on_time = 0, start_time_sec = 0, start_time = conf->base_time;
> -	u32 configured_queues = 0, speed = 0, start_time_nsec;
> -	struct macb_queue_enst_config *enst_queue;
> -	struct ethtool_link_ksettings kset = {};
> +	u32 configured_queues = 0, start_time_nsec;
>  	struct macb *bp = netdev_priv(netdev);
>  	struct tc_taprio_sched_entry *entry;
> -	struct macb_queue *queue;
> -	u32 queue_mask;
> +	struct macb_enst config = {};
>  	u8 queue_id;
> -	size_t i, q;
> -	int err;
> +	u64 off;
> +	size_t i;
> +
> +	guard(mutex)(&bp->mac_cfg_lock);
> +
> +	if (unlikely(!bp->speed_mbps)) {
> +		netdev_err(netdev, "Invalid speed, link-down?\n");
> +		return -EINVAL;
> +	}
>  
>  	if (conf->num_entries > bp->num_queues) {
> -		netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n",
> +		netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n",
>  			   conf->num_entries, bp->num_queues);
>  		return -EINVAL;
>  	}
> @@ -4777,23 +4829,6 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
>  		return -ERANGE;
>  	}
>  
> -	/* Get the current link speed */
> -	err = phylink_ethtool_ksettings_get(bp->phylink, &kset);
> -	if (unlikely(err)) {
> -		netdev_err(netdev, "Failed to get link settings: %d\n", err);
> -		return err;
> -	}
> -
> -	speed = kset.base.speed;
> -	if (unlikely(speed == SPEED_UNKNOWN || !speed)) {
> -		netdev_err(netdev, "Invalid speed %d, link-down?\n", speed);
> -		return -EINVAL;
> -	}
> -
> -	enst_queue = kzalloc_objs(*enst_queue, bp->num_queues);
> -	if (unlikely(!enst_queue))
> -		return -ENOMEM;
> -
>  	/* Pre-validate all entries before making any hardware changes */
>  	for (i = 0; i < conf->num_entries; i++) {
>  		entry = &conf->entries[i];
> @@ -4801,25 +4836,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
>  		if (entry->command != TC_TAPRIO_CMD_SET_GATES) {
>  			netdev_err(netdev, "Entry %zu: unsupported command %d\n",
>  				   i, entry->command);
> -			err = -EOPNOTSUPP;
> -			goto cleanup;
> +			return -EOPNOTSUPP;
>  		}
>  
>  		/* Validate gate_mask: must be nonzero, single queue, and within range */
>  		if (!is_power_of_2(entry->gate_mask)) {
>  			netdev_err(netdev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
>  				   i, entry->gate_mask);
> -			err = -EINVAL;
> -			goto cleanup;
> +			return -EINVAL;
>  		}
>  
>  		/* gate_mask must not select queues outside the valid queues */
>  		queue_id = order_base_2(entry->gate_mask);
>  		if (queue_id >= bp->num_queues) {
> -			netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
> -				   i, entry->gate_mask, bp->num_queues);
> -			err = -EINVAL;
> -			goto cleanup;
> +			netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n",
> +				   i, entry->gate_mask,
> +				   bp->num_queues);
> +			return -EINVAL;
>  		}
>  
>  		/* Check for start time limits */
> @@ -4828,34 +4861,33 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
>  		if (start_time_sec > GENMASK(GEM_START_TIME_SEC_SIZE - 1, 0)) {
>  			netdev_err(netdev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
>  				   i, start_time_sec);
> -			err = -ERANGE;
> -			goto cleanup;
> +			return -ERANGE;
>  		}
>  
>  		/* Check for on time limit */
> -		if (entry->interval > enst_max_hw_interval(speed)) {
> +		if (entry->interval > enst_max_hw_interval(bp->speed_mbps)) {
>  			netdev_err(netdev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
> -				   i, entry->interval, enst_max_hw_interval(speed));
> -			err = -ERANGE;
> -			goto cleanup;
> +				   i, entry->interval,
> +				   enst_max_hw_interval(bp->speed_mbps));
> +			return -ERANGE;
>  		}
>  
>  		/* Check for off time limit*/
> -		if ((conf->cycle_time - entry->interval) > enst_max_hw_interval(speed)) {
> +		off = conf->cycle_time - entry->interval;
> +		if (off > enst_max_hw_interval(bp->speed_mbps)) {
>  			netdev_err(netdev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
> -				   i, conf->cycle_time - entry->interval,
> -				   enst_max_hw_interval(speed));
> -			err = -ERANGE;
> -			goto cleanup;
> +				   i, off,
> +				   enst_max_hw_interval(bp->speed_mbps));
> +			return -ERANGE;
>  		}
>  
> -		enst_queue[queue_id].start_time_mask =
> +		config.queues[queue_id].start_time_mask =
>  			(start_time_sec << GEM_START_TIME_SEC_OFFSET) |
>  			start_time_nsec;
> -		enst_queue[queue_id].on_time_bytes =
> -			enst_ns_to_hw_units(entry->interval, speed);
> -		enst_queue[queue_id].off_time_bytes =
> -			enst_ns_to_hw_units(conf->cycle_time - entry->interval, speed);
> +		config.queues[queue_id].on_time_bytes =
> +			enst_ns_to_hw_units(entry->interval, bp->speed_mbps);
> +		config.queues[queue_id].off_time_bytes =
> +			enst_ns_to_hw_units(off, bp->speed_mbps);
>  
>  		configured_queues |= entry->gate_mask;
>  		total_on_time += entry->interval;
> @@ -4866,82 +4898,47 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
>  	if (total_on_time > conf->cycle_time) {
>  		netdev_err(netdev, "Total ON %llu ns exceeds cycle time %llu ns\n",
>  			   total_on_time, conf->cycle_time);
> -		err = -EINVAL;
> -		goto cleanup;
> +		return -EINVAL;
>  	}
>  
>  	netdev_dbg(netdev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
>  		   conf->num_entries, conf->base_time, conf->cycle_time);
>  
> -	/* All validations passed - proceed with hardware configuration */
> -	scoped_guard(spinlock_irqsave, &bp->lock) {
> -		/* Disable ENST queues if running before configuring */
> -		queue_mask = BIT_U32(bp->num_queues) - 1;
> -		gem_writel(bp, ENST_CONTROL,
> -			   queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> +	config.queue_mask = configured_queues;
> +	config.speed_mbps = bp->speed_mbps;
> +	bp->enst = config;
>  
> -		for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> -			/* Configure queue timing registers */
> -			queue_writel(queue, ENST_START_TIME,
> -				     enst_queue[q].start_time_mask);
> -			queue_writel(queue, ENST_ON_TIME,
> -				     enst_queue[q].on_time_bytes);
> -			queue_writel(queue, ENST_OFF_TIME,
> -				     enst_queue[q].off_time_bytes);
> -		}
> -
> -		/* Enable ENST for all configured queues in one write */
> -		gem_writel(bp, ENST_CONTROL, configured_queues);
> -	}
> +	macb_apply_enst_config(bp);
>  
>  	netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
>  		    conf->num_entries, hweight32(configured_queues));
>  
> -cleanup:
> -	kfree(enst_queue);
> -	return err;
> +	return 0;
>  }
>  
>  static void macb_taprio_destroy(struct net_device *netdev)
>  {
>  	struct macb *bp = netdev_priv(netdev);
> -	struct macb_queue *queue;
> -	u32 queue_mask;
> -	unsigned int q;
> +
> +	guard(mutex)(&bp->mac_cfg_lock);
>  
>  	netdev_reset_tc(netdev);
> -	queue_mask = BIT_U32(bp->num_queues) - 1;
> +	memset(&bp->enst, 0, sizeof(bp->enst));
>  
> -	scoped_guard(spinlock_irqsave, &bp->lock) {
> -		/* Single disable command for all queues */
> -		gem_writel(bp, ENST_CONTROL,
> -			   queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> +	if (bp->link_up)
> +		macb_apply_enst_config(bp);
>  
> -		/* Clear all queue ENST registers in batch */
> -		for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> -			queue_writel(queue, ENST_START_TIME, 0);
> -			queue_writel(queue, ENST_ON_TIME, 0);
> -			queue_writel(queue, ENST_OFF_TIME, 0);
> -		}
> -	}
>  	netdev_info(netdev, "TAPRIO destroy: All gates disabled\n");
>  }
>  
>  static int macb_setup_taprio(struct net_device *netdev,
>  			     struct tc_taprio_qopt_offload *taprio)
>  {
> -	struct macb *bp = netdev_priv(netdev);
>  	int err = 0;
>  
>  	if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC)))
>  		return -EOPNOTSUPP;
>  
> -	/* Check if Device is in runtime suspend */
> -	if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) {
> -		netdev_err(netdev, "Device is in runtime suspend\n");
> -		return -EOPNOTSUPP;
> -	}
> -
>  	switch (taprio->cmd) {
>  	case TAPRIO_CMD_REPLACE:
>  		err = macb_taprio_setup_replace(netdev, taprio);
> 

-- 
🙏 Vineeth


  parent reply	other threads:[~2026-11-19 12:36 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
2026-11-19 11:49   ` Nicolai Buchwitz
2026-11-19 12:36   ` Vineeth Karumanchi [this message]
2026-11-19 15:40     ` Théo Lebrun
2026-11-19 16:10       ` Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
2026-11-19 11:54   ` Nicolai Buchwitz
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
2026-11-19 11:45   ` Nicolai Buchwitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fe08db37-4415-4b4b-b226-cc1a0ec4f029@amd.com \
    --to=vineeth.karumanchi@amd.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=benoit.monin@bootlin.com \
    --cc=conor.dooley@microchip.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@kernel.org \
    --cc=gregory.clement@bootlin.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux@armlinux.org.uk \
    --cc=nb@tipi-net.de \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=pvalerio@redhat.com \
    --cc=tawfik.bayouk@mobileye.com \
    --cc=theo.lebrun@bootlin.com \
    --cc=thomas.petazzoni@bootlin.com \
    --cc=vladimir.kondratiev@mobileye.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox