[PATCH] net/mlx5: move to polling for LSC interrupts

[PATCH] net/mlx5: move to polling for LSC interrupts

[Dariusz Sosnowski]

mlx5 networking driver on Linux implemented LSC interrupts
by listening on netlink events.
This method is unfortunately not reliable enough,
because of linkwatch mechanism. It can delay link up/down events
for up to ~1 second. This introduces unnecessary delay
when mlx5 ports are used with DPDK bonding driver for examples.

This patch addresses that by migrating LSC checks
to polling on link status using ETHTOOL_GLINK command.
This ethtool command, implemented by mlx5 since kernel 4.14,
will always get the current link status.
As a side effect it is also flushes the pending link watch events.

Fixes: 17f95513adca ("net/mlx5: fix link status change detection")
Cc: dmitry.kozliuk@gmail.com
Cc: stable@dpdk.org

Signed-off-by: Cosmin Ratiu <cratiu@nvidia.com>
Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
---
 .mailmap                                 |   1 +
 drivers/common/mlx5/linux/mlx5_nl.c      |  29 ---
 drivers/common/mlx5/linux/mlx5_nl.h      |   2 -
 drivers/net/mlx5/linux/meson.build       |   1 +
 drivers/net/mlx5/linux/mlx5_ethdev_os.c  | 147 ++++-------
 drivers/net/mlx5/linux/mlx5_lsc_intr.c   | 302 +++++++++++++++++++++++
 drivers/net/mlx5/linux/mlx5_os.c         |  22 --
 drivers/net/mlx5/mlx5.c                  |   7 -
 drivers/net/mlx5/mlx5.h                  |   4 +-
 drivers/net/mlx5/mlx5_lsc_intr.h         |  15 ++
 drivers/net/mlx5/mlx5_trigger.c          |  19 +-
 drivers/net/mlx5/windows/meson.build     |   1 +
 drivers/net/mlx5/windows/mlx5_lsc_intr.c |  24 ++
 13 files changed, 398 insertions(+), 176 deletions(-)
 create mode 100644 drivers/net/mlx5/linux/mlx5_lsc_intr.c
 create mode 100644 drivers/net/mlx5/mlx5_lsc_intr.h
 create mode 100644 drivers/net/mlx5/windows/mlx5_lsc_intr.c

diff --git a/.mailmap b/.mailmap
index 6d9e262cfb..ba580bdd1c 100644
--- a/.mailmap
+++ b/.mailmap
@@ -315,6 +315,7 @@ Congjie Zhou <zcjie0802@qq.com>
 Congwen Zhang <zhang.congwen@zte.com.cn>
 Conor Fogarty <conor.fogarty@intel.com>
 Conor Walsh <conor.walsh@intel.com>
+Cosmin Ratiu <cratiu@nvidia.com>
 Cristian Bidea <cristian.bidea@keysight.com>
 Cristian Dumitrescu <cristian.dumitrescu@intel.com>
 Cristian Sovaiala <cristian.sovaiala@nxp.com>
diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c
index 3fabcfaee0..cc33a9231f 100644
--- a/drivers/common/mlx5/linux/mlx5_nl.c
+++ b/drivers/common/mlx5/linux/mlx5_nl.c
@@ -1994,35 +1994,6 @@ mlx5_nl_enable_roce_set(int nlsk_fd, int family_id, const char *pci_addr,
 	return mlx5_nl_driver_reload(nlsk_fd, family_id, pci_addr);
 }
 
-/**
- * Try to parse a Netlink message as a link status update.
- *
- * @param hdr
- *  Netlink message header.
- * @param[out] ifindex
- *  Index of the updated interface.
- *
- * @return
- *  0 on success, negative on failure.
- */
-RTE_EXPORT_INTERNAL_SYMBOL(mlx5_nl_parse_link_status_update)
-int
-mlx5_nl_parse_link_status_update(struct nlmsghdr *hdr, uint32_t *ifindex)
-{
-	struct ifinfomsg *info;
-
-	switch (hdr->nlmsg_type) {
-	case RTM_NEWLINK:
-	case RTM_DELLINK:
-	case RTM_GETLINK:
-	case RTM_SETLINK:
-		info = NLMSG_DATA(hdr);
-		*ifindex = info->ifi_index;
-		return 0;
-	}
-	return -1;
-}
-
 /**
  * Read pending events from a Netlink socket.
  *
diff --git a/drivers/common/mlx5/linux/mlx5_nl.h b/drivers/common/mlx5/linux/mlx5_nl.h
index 07a3b531b5..d1c0c92e1e 100644
--- a/drivers/common/mlx5/linux/mlx5_nl.h
+++ b/drivers/common/mlx5/linux/mlx5_nl.h
@@ -102,8 +102,6 @@ int mlx5_nl_enable_roce_set(int nlsk_fd, int family_id, const char *pci_addr,
 
 __rte_internal
 int mlx5_nl_read_events(int nlsk_fd, mlx5_nl_event_cb *cb, void *cb_arg);
-__rte_internal
-int mlx5_nl_parse_link_status_update(struct nlmsghdr *hdr, uint32_t *ifindex);
 
 __rte_internal
 int mlx5_nl_devlink_esw_multiport_get(int nlsk_fd, int family_id,
diff --git a/drivers/net/mlx5/linux/meson.build b/drivers/net/mlx5/linux/meson.build
index e45ea988db..9fdb1555a6 100644
--- a/drivers/net/mlx5/linux/meson.build
+++ b/drivers/net/mlx5/linux/meson.build
@@ -10,6 +10,7 @@ sources += files(
         'mlx5_mp_os.c',
         'mlx5_vlan_os.c',
         'mlx5_flow_os.c',
+        'mlx5_lsc_intr.c',
 )
 
 cflags += no_wvla_cflag
diff --git a/drivers/net/mlx5/linux/mlx5_ethdev_os.c b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
index f6394ebd19..d6103896f5 100644
--- a/drivers/net/mlx5/linux/mlx5_ethdev_os.c
+++ b/drivers/net/mlx5/linux/mlx5_ethdev_os.c
@@ -317,6 +317,40 @@ mlx5_find_master_dev(struct rte_eth_dev *dev)
 	return NULL;
 }
 
+/**
+ * Retrieve the carrier state of a device.
+ *
+ * The IFF_RUNNING flag reported by SIOCGIFFLAGS reflects the RFC 2863
+ * operational state, which the kernel only refreshes from its link watch
+ * work queue and which may therefore lag the actual carrier by up to a
+ * second. ETHTOOL_GLINK reports the carrier directly, and additionally
+ * flushes any link watch event still pending for the device.
+ *
+ * @param dev
+ *   Pointer to Ethernet device structure.
+ *
+ * @return
+ *   1 if the carrier is up, 0 otherwise.
+ *   Negative errno value is returned on error.
+ */
+int
+mlx5_link_carrier_get(struct rte_eth_dev *dev)
+{
+	struct ethtool_value ecmd = {
+		.cmd = ETHTOOL_GLINK,
+	};
+	struct ifreq ifr;
+	int ret;
+
+	memset(&ifr, 0, sizeof(ifr));
+	ifr.ifr_data = (void *)&ecmd;
+	ret = mlx5_ifreq(dev, SIOCETHTOOL, &ifr);
+	if (ret != 0)
+		return ret;
+
+	return !!ecmd.data;
+}
+
 /**
  * DPDK callback to retrieve physical link information.
  *
@@ -347,9 +381,11 @@ mlx5_link_update_unlocked_gset(struct rte_eth_dev *dev,
 			dev->data->port_id, strerror(rte_errno));
 		return ret;
 	}
+	ret = mlx5_link_carrier_get(dev);
+	if (ret < 0)
+		return ret;
 	dev_link = (struct rte_eth_link) {
-		.link_status = ((ifr.ifr_flags & IFF_UP) &&
-				(ifr.ifr_flags & IFF_RUNNING)),
+		.link_status = ((ifr.ifr_flags & IFF_UP) && ret),
 	};
 	ifr = (struct ifreq) {
 		.ifr_data = (void *)&edata,
@@ -428,9 +464,11 @@ mlx5_link_update_unlocked_gs(struct rte_eth_dev *dev,
 			dev->data->port_id, strerror(rte_errno));
 		return ret;
 	}
+	ret = mlx5_link_carrier_get(dev);
+	if (ret < 0)
+		return ret;
 	dev_link = (struct rte_eth_link) {
-		.link_status = ((ifr.ifr_flags & IFF_UP) &&
-				(ifr.ifr_flags & IFF_RUNNING)),
+		.link_status = ((ifr.ifr_flags & IFF_UP) && ret),
 	};
 	ifr = (struct ifreq) {
 		.ifr_data = (void *)&gcmd,
@@ -537,10 +575,7 @@ mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete)
 			return ret;
 		}
 	} while (wait_to_complete || retry-- > 0);
-	ret = !!memcmp(&dev->data->dev_link, &dev_link,
-		       sizeof(struct rte_eth_link));
-	dev->data->dev_link = dev_link;
-	return ret;
+	return rte_eth_linkstatus_set(dev, &dev_link) == 0;
 }
 
 /**
@@ -663,102 +698,6 @@ mlx5_dev_interrupt_device_fatal(struct mlx5_dev_ctx_shared *sh)
 	}
 }
 
-static bool
-mlx5_dev_nl_ifindex_verify(uint32_t if_index, struct mlx5_priv *priv)
-{
-	struct mlx5_bond_info *bond = &priv->sh->bond;
-	int i;
-
-	if (bond->n_port == 0)
-		return (if_index == priv->if_index);
-
-	if (if_index == bond->ifindex)
-		return true;
-	for (i = 0; i < MLX5_BOND_MAX_PORTS; i++) {
-		if (!strlen(priv->sh->bond.ports[i].ifname))
-			continue;
-		if (if_index == bond->ports[i].ifindex)
-			return true;
-	}
-
-	return false;
-}
-
-static void
-mlx5_link_update_bond(struct rte_eth_dev *dev)
-{
-	struct mlx5_priv *priv = dev->data->dev_private;
-	struct mlx5_bond_info *bond = &priv->sh->bond;
-	struct ifreq ifr = (struct ifreq) {
-		.ifr_flags = 0,
-	};
-	int ret;
-
-	ret = mlx5_ifreq_by_ifname(bond->ifname, SIOCGIFFLAGS, &ifr);
-	if (ret) {
-		DRV_LOG(WARNING, "ifname %s ioctl(SIOCGIFFLAGS) failed: %s",
-			bond->ifname, strerror(rte_errno));
-		return;
-	}
-	dev->data->dev_link.link_status =
-		((ifr.ifr_flags & IFF_UP) && (ifr.ifr_flags & IFF_RUNNING));
-}
-
-static void
-mlx5_dev_interrupt_nl_cb(struct nlmsghdr *hdr, void *cb_arg)
-{
-	struct mlx5_dev_ctx_shared *sh = cb_arg;
-	uint32_t i;
-	uint32_t if_index;
-
-	if (mlx5_nl_parse_link_status_update(hdr, &if_index) < 0)
-		return;
-
-	for (i = 0; i < sh->max_port; i++) {
-		struct mlx5_dev_shared_port *port = &sh->port[i];
-		struct rte_eth_dev *dev;
-		struct mlx5_priv *priv;
-
-		if (port->nl_ih_port_id >= RTE_MAX_ETHPORTS)
-			continue;
-		dev = &rte_eth_devices[port->nl_ih_port_id];
-		/* Probing may initiate an LSC before configuration is done. */
-		if (dev->data->dev_configured &&
-		    !dev->data->dev_conf.intr_conf.lsc)
-			break;
-		priv = dev->data->dev_private;
-		if (mlx5_dev_nl_ifindex_verify(if_index, priv)) {
-			/* Block logical LSC events. */
-			uint16_t prev_status = dev->data->dev_link.link_status;
-
-			if (mlx5_link_update(dev, 0) < 0) {
-				DRV_LOG(ERR, "Failed to update link status: %s",
-					rte_strerror(rte_errno));
-			} else {
-				if (priv->sh->bond.n_port)
-					mlx5_link_update_bond(dev);
-				if (prev_status != dev->data->dev_link.link_status)
-					rte_eth_dev_callback_process
-						(dev, RTE_ETH_EVENT_INTR_LSC, NULL);
-			}
-			break;
-		}
-	}
-}
-
-void
-mlx5_dev_interrupt_handler_nl(void *arg)
-{
-	struct mlx5_dev_ctx_shared *sh = arg;
-	int nlsk_fd = rte_intr_fd_get(sh->intr_handle_nl);
-
-	if (nlsk_fd < 0)
-		return;
-	if (mlx5_nl_read_events(nlsk_fd, mlx5_dev_interrupt_nl_cb, sh) < 0)
-		DRV_LOG(ERR, "Failed to process Netlink events: %s",
-			rte_strerror(rte_errno));
-}
-
 /**
  * Handle shared asynchronous events the NIC (removal event
  * and link status change). Supports multiport IB device.
diff --git a/drivers/net/mlx5/linux/mlx5_lsc_intr.c b/drivers/net/mlx5/linux/mlx5_lsc_intr.c
new file mode 100644
index 0000000000..2fb3b9adfc
--- /dev/null
+++ b/drivers/net/mlx5/linux/mlx5_lsc_intr.c
@@ -0,0 +1,302 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2026 NVIDIA Corporation & Affiliates
+ */
+
+#include <stdint.h>
+#include <time.h>
+
+#include <ethdev_driver.h>
+#include <rte_alarm.h>
+#include <rte_bitset.h>
+#include <rte_interrupts.h>
+#include <rte_pause.h>
+#include <rte_spinlock.h>
+#include <rte_stdatomic.h>
+
+#include "mlx5.h"
+#include "mlx5_lsc_intr.h"
+#include "mlx5_utils.h"
+
+#define LINK_POLL_DEFAULT_PERIOD_US (UINT64_C(100) * 1000)
+#define LINK_POLL_MINIMUM_PERIOD_US LINK_POLL_DEFAULT_PERIOD_US
+#define LINK_POLL_MAXIMUM_PERIOD_US (UINT64_C(800) * 1000)
+
+#define POLLED_PORTS_MAX (RTE_MAX_ETHPORTS)
+
+struct mlx5_lsc_intr_ctx {
+	RTE_BITSET_DECLARE(polled_ports, POLLED_PORTS_MAX);
+	uint64_t period_us;
+	bool armed;
+	rte_spinlock_t lock;
+	RTE_ATOMIC(bool) cb_in_progress;
+};
+
+static struct mlx5_lsc_intr_ctx lsc_intr_ctx = {
+	.period_us = LINK_POLL_DEFAULT_PERIOD_US,
+	.lock = RTE_SPINLOCK_INITIALIZER,
+};
+
+static inline bool
+lsc_intr_cb_in_progress(struct mlx5_lsc_intr_ctx *ctx)
+{
+	return rte_atomic_load_explicit(&ctx->cb_in_progress, rte_memory_order_seq_cst);
+}
+
+static inline void
+lsc_intr_cb_set_in_progress(struct mlx5_lsc_intr_ctx *ctx, bool val)
+{
+	rte_atomic_store_explicit(&ctx->cb_in_progress, val, rte_memory_order_seq_cst);
+}
+
+static struct timespec
+timespec_sub(const struct timespec *t0, const struct timespec *t1)
+{
+	struct timespec ts;
+
+	ts.tv_sec = t0->tv_sec - t1->tv_sec;
+	ts.tv_nsec = t0->tv_nsec - t1->tv_nsec;
+	if (ts.tv_nsec < 0) {
+		ts.tv_sec--;
+		ts.tv_nsec += 1000000000L;
+	}
+	return ts;
+}
+
+static uint64_t
+adjust_period_and_calc_delay(struct mlx5_lsc_intr_ctx *ctx,
+			     struct timespec *after,
+			     struct timespec *before)
+{
+	struct timespec delta = timespec_sub(after, before);
+	uint64_t new_period;
+	uint64_t delta_us;
+
+	delta_us = delta.tv_sec * 1000000;
+	delta_us += delta.tv_nsec / 1000;
+
+	new_period = ctx->period_us;
+
+	if (delta_us > new_period / 2) {
+		/* If takes too long, adjust period up. */
+		while (new_period < LINK_POLL_MAXIMUM_PERIOD_US && delta_us > new_period / 2)
+			new_period = RTE_MIN(new_period * 2, LINK_POLL_MAXIMUM_PERIOD_US);
+	} else if (delta_us < new_period / 4) {
+		/* If polling is fast, adjust period down. */
+		while (new_period > LINK_POLL_MINIMUM_PERIOD_US && delta_us < new_period / 4)
+			new_period = RTE_MAX(new_period / 2, LINK_POLL_MINIMUM_PERIOD_US);
+	}
+
+	if (new_period != ctx->period_us) {
+		DRV_LOG(DEBUG, "adjusting link polling period: %" PRIu64 " us to %" PRIu64 " us",
+			ctx->period_us, new_period);
+		ctx->period_us = new_period;
+	}
+
+	if (ctx->period_us == LINK_POLL_MAXIMUM_PERIOD_US) {
+		/*
+		 * If at maximum period and querying link still takes too long, then default to
+		 * half of the maximum period so other interrupt thread users are not starved.
+		 */
+		if (delta_us > ctx->period_us / 2)
+			return LINK_POLL_MAXIMUM_PERIOD_US / 2;
+	}
+
+	MLX5_ASSERT(delta_us <= ctx->period_us / 2);
+	return ctx->period_us - delta_us;
+}
+
+static void
+lsc_intr_disarm(struct mlx5_lsc_intr_ctx *ctx)
+{
+	rte_spinlock_lock(&ctx->lock);
+	ctx->armed = false;
+	rte_spinlock_unlock(&ctx->lock);
+}
+
+static bool
+lsc_intr_dev_add(struct mlx5_lsc_intr_ctx *ctx, struct rte_eth_dev *dev)
+{
+	uint16_t port_id = dev->data->port_id;
+	bool schedule = false;
+
+	rte_spinlock_lock(&ctx->lock);
+	rte_bitset_set(ctx->polled_ports, port_id);
+	if (!ctx->armed) {
+		ctx->armed = true;
+		schedule = true;
+	}
+	rte_spinlock_unlock(&ctx->lock);
+
+	return schedule;
+}
+
+static void
+lsc_intr_dev_remove(struct mlx5_lsc_intr_ctx *ctx, struct rte_eth_dev *dev, bool disarm)
+{
+	rte_spinlock_lock(&ctx->lock);
+	rte_bitset_clear(ctx->polled_ports, dev->data->port_id);
+	if (disarm)
+		ctx->armed = false;
+	rte_spinlock_unlock(&ctx->lock);
+}
+
+static int
+lsc_intr_gather_devs(struct mlx5_lsc_intr_ctx *ctx,
+		     struct rte_eth_dev *out[POLLED_PORTS_MAX])
+{
+	ssize_t port_id;
+	int ret = 0;
+
+	rte_spinlock_lock(&ctx->lock);
+	RTE_BITSET_FOREACH_SET(port_id, ctx->polled_ports, POLLED_PORTS_MAX) {
+		out[ret++] = &rte_eth_devices[port_id];
+	}
+	rte_spinlock_unlock(&ctx->lock);
+
+	return ret;
+}
+
+static void
+lsc_intr_check_links(struct rte_eth_dev **devs, int devs_nb)
+{
+	for (int i = 0; i < devs_nb; ++i) {
+		struct rte_eth_dev *dev = devs[i];
+		struct rte_eth_link dev_link;
+		int ret;
+
+		ret = mlx5_link_carrier_get(dev);
+		if (ret < 0) {
+			devs[i] = NULL;
+			continue;
+		}
+
+		dev_link = dev->data->dev_link;
+		if (dev_link.link_status == ret) {
+			devs[i] = NULL;
+			continue;
+		}
+		dev_link.link_status = ret;
+		dev->data->dev_link = dev_link;
+	}
+}
+
+static void
+lsc_intr_notify(struct rte_eth_dev **devs, int devs_nb)
+{
+	for (int i = 0; i < devs_nb; ++i) {
+		if (devs[i] == NULL)
+			continue;
+
+		rte_eth_dev_callback_process(devs[i], RTE_ETH_EVENT_INTR_LSC, NULL);
+	}
+}
+
+static bool
+lsc_intr_should_reschedule(struct mlx5_lsc_intr_ctx *ctx)
+{
+	bool should;
+
+	rte_spinlock_lock(&ctx->lock);
+	should = rte_bitset_count_set(ctx->polled_ports, POLLED_PORTS_MAX) > 0;
+	if (!should)
+		ctx->armed = false;
+	rte_spinlock_unlock(&ctx->lock);
+
+	return should;
+}
+
+static bool
+lsc_intr_cb(struct mlx5_lsc_intr_ctx *ctx, uint64_t *delay_us)
+{
+	static struct rte_eth_dev *to_process[POLLED_PORTS_MAX];
+	struct timespec before;
+	struct timespec after;
+	int to_process_nb = 0;
+	bool have_time = true;
+
+	if (clock_gettime(CLOCK_MONOTONIC, &before) < 0)
+		have_time = false;
+
+	to_process_nb = lsc_intr_gather_devs(ctx, to_process);
+	if (to_process_nb == 0)
+		return lsc_intr_should_reschedule(ctx);
+
+	lsc_intr_check_links(to_process, to_process_nb);
+	lsc_intr_notify(to_process, to_process_nb);
+
+	if (clock_gettime(CLOCK_MONOTONIC, &after) < 0)
+		have_time = false;
+
+	if (have_time) {
+		*delay_us = adjust_period_and_calc_delay(ctx, &after, &before);
+	} else {
+		DRV_LOG(WARNING, "link polling callback failed to calculate polling time");
+		*delay_us = ctx->period_us;
+	}
+
+	return lsc_intr_should_reschedule(ctx);
+}
+
+static void
+mlx5_lsc_intr_cb(void *arg)
+{
+	struct mlx5_lsc_intr_ctx *ctx = arg;
+	uint64_t delay_us = ctx->period_us;
+	bool reschedule;
+	int ret;
+
+	lsc_intr_cb_set_in_progress(ctx, true);
+	reschedule = lsc_intr_cb(ctx, &delay_us);
+	lsc_intr_cb_set_in_progress(ctx, false);
+
+	/* Reschedule the polling */
+	if (reschedule) {
+		ret = rte_eal_alarm_set(delay_us, mlx5_lsc_intr_cb, ctx);
+		if (ret < 0) {
+			DRV_LOG(ERR, "rescheduling link polling callback failed (%s)",
+				rte_strerror(-ret));
+			lsc_intr_disarm(ctx);
+		}
+	}
+}
+
+int
+mlx5_lsc_intr_setup(struct rte_eth_dev *dev)
+{
+	struct mlx5_lsc_intr_ctx *ctx = &lsc_intr_ctx;
+	bool schedule;
+	int ret;
+
+	schedule = lsc_intr_dev_add(ctx, dev);
+	if (!schedule)
+		return 0;
+
+	ret = rte_eal_alarm_set(ctx->period_us, mlx5_lsc_intr_cb, ctx);
+	if (ret < 0) {
+		DRV_LOG(ERR, "port %u registering link polling callback failed (%s)",
+			dev->data->port_id, rte_strerror(-ret));
+		lsc_intr_dev_remove(ctx, dev, true);
+		return ret;
+	}
+	DRV_LOG(DEBUG, "registered link polling callback");
+
+	return 0;
+}
+
+void
+mlx5_lsc_intr_cleanup(struct rte_eth_dev *dev)
+{
+	struct mlx5_lsc_intr_ctx *ctx = &lsc_intr_ctx;
+
+	lsc_intr_dev_remove(ctx, dev, false);
+
+	/*
+	 * If callback is running, then wait until it finishes to prevent
+	 * freeing dev while it is running.
+	 *
+	 * mlx5_lsc_intr_cb will be called only from interrupt thread.
+	 * If cleanup was called from the callback, then no need to wait.
+	 */
+	while (lsc_intr_cb_in_progress(ctx) && !rte_thread_is_intr())
+		rte_pause();
+}
diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c
index 7c2c35c945..08dd2b46d9 100644
--- a/drivers/net/mlx5/linux/mlx5_os.c
+++ b/drivers/net/mlx5/linux/mlx5_os.c
@@ -1803,8 +1803,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 		eth_dev->data->port_id);
 	/* Read link status in case it is up and there will be no event. */
 	mlx5_link_update(eth_dev, 0);
-	/* Watch LSC interrupts between port probe and port start. */
-	priv->sh->port[priv->dev_port - 1].nl_ih_port_id = eth_dev->data->port_id;
 	mlx5_set_link_up(eth_dev);
 	for (i = 0; i < MLX5_FLOW_TYPE_MAXI; i++) {
 		icfg[i].release_mem_en = !!sh->config.reclaim_mode;
@@ -1956,8 +1954,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev,
 	return eth_dev;
 error:
 	if (priv) {
-		priv->sh->port[priv->dev_port - 1].nl_ih_port_id =
-							       RTE_MAX_ETHPORTS;
 		rte_io_wmb();
 #ifdef HAVE_MLX5_HWS_SUPPORT
 		if (eth_dev &&
@@ -3262,19 +3258,6 @@ mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
 			}
 		}
 	}
-	nlsk_fd = mlx5_nl_init(NETLINK_ROUTE, RTMGRP_LINK);
-	if (nlsk_fd < 0) {
-		DRV_LOG(ERR, "Failed to create a socket for Netlink events: %s",
-			rte_strerror(rte_errno));
-		return;
-	}
-	sh->intr_handle_nl = mlx5_os_interrupt_handler_create
-		(RTE_INTR_INSTANCE_F_SHARED, true,
-		 nlsk_fd, mlx5_dev_interrupt_handler_nl, sh);
-	if (sh->intr_handle_nl == NULL) {
-		DRV_LOG(ERR, "Fail to allocate intr_handle");
-		return;
-	}
 	if (sh->cdev->config.devx) {
 #ifdef HAVE_IBV_DEVX_ASYNC
 		struct mlx5dv_devx_cmd_comp *devx_comp;
@@ -3312,11 +3295,6 @@ mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
 
 	mlx5_os_interrupt_handler_destroy(sh->intr_handle,
 					  mlx5_dev_interrupt_handler, sh);
-	fd = rte_intr_fd_get(sh->intr_handle_nl);
-	mlx5_os_interrupt_handler_destroy(sh->intr_handle_nl,
-					  mlx5_dev_interrupt_handler_nl, sh);
-	if (fd >= 0)
-		close(fd);
 #ifdef HAVE_IBV_DEVX_ASYNC
 	mlx5_os_interrupt_handler_destroy(sh->intr_handle_devx,
 					  mlx5_dev_interrupt_handler_devx, sh);
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index a04884fb4b..2fdd17f08e 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1930,7 +1930,6 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn,
 	for (i = 0; i < sh->max_port; i++) {
 		sh->port[i].ih_port_id = RTE_MAX_ETHPORTS;
 		sh->port[i].devx_ih_port_id = RTE_MAX_ETHPORTS;
-		sh->port[i].nl_ih_port_id = RTE_MAX_ETHPORTS;
 	}
 	if (sh->cdev->config.devx) {
 		sh->td = mlx5_devx_cmd_create_td(sh->cdev->ctx);
@@ -2559,12 +2558,6 @@ mlx5_dev_close(struct rte_eth_dev *dev)
 	mlx5_free(priv->mac);
 	dev->data->mac_addrs = NULL;
 	mlx5_free(priv->mac_own);
-	sh->port[priv->dev_port - 1].nl_ih_port_id = RTE_MAX_ETHPORTS;
-	/*
-	 * The interrupt handler port id must be reset before priv is reset
-	 * since 'mlx5_dev_interrupt_nl_cb' uses priv.
-	 */
-	rte_io_wmb();
 	/*
 	 * Free the shared context in last turn, because the cleanup
 	 * routines above may use some shared fields, like
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index e3df458915..51c789bf45 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -838,7 +838,6 @@ struct mlx5_age_info {
 struct mlx5_dev_shared_port {
 	uint32_t ih_port_id;
 	uint32_t devx_ih_port_id;
-	uint32_t nl_ih_port_id;
 	/*
 	 * Interrupt handler port_id. Used by shared interrupt
 	 * handler to find the corresponding rte_eth device
@@ -1649,7 +1648,6 @@ struct mlx5_dev_ctx_shared {
 	/* Shared interrupt handler section. */
 	struct rte_intr_handle *intr_handle; /* Interrupt handler for device. */
 	struct rte_intr_handle *intr_handle_devx; /* DEVX interrupt handler. */
-	struct rte_intr_handle *intr_handle_nl; /* Netlink interrupt handler. */
 	struct rte_intr_handle *intr_handle_ib; /* Interrupt handler for IB device. */
 	void *devx_comp; /* DEVX async comp obj. */
 	struct mlx5_devx_obj *tis[16]; /* TIS object. */
@@ -2382,13 +2380,13 @@ int mlx5_get_mtu(struct rte_eth_dev *dev, uint16_t *mtu);
 int mlx5_set_mtu(struct rte_eth_dev *dev, uint16_t mtu);
 int mlx5_read_clock(struct rte_eth_dev *dev, uint64_t *clock);
 int mlx5_link_update(struct rte_eth_dev *dev, int wait_to_complete);
+int mlx5_link_carrier_get(struct rte_eth_dev *dev);
 int mlx5_dev_get_flow_ctrl(struct rte_eth_dev *dev,
 			   struct rte_eth_fc_conf *fc_conf);
 int mlx5_dev_set_flow_ctrl(struct rte_eth_dev *dev,
 			   struct rte_eth_fc_conf *fc_conf);
 void mlx5_dev_interrupt_handler(void *arg);
 void mlx5_dev_interrupt_handler_devx(void *arg);
-void mlx5_dev_interrupt_handler_nl(void *arg);
 void mlx5_dev_interrupt_handler_ib(void *arg);
 int mlx5_set_link_down(struct rte_eth_dev *dev);
 int mlx5_set_link_up(struct rte_eth_dev *dev);
diff --git a/drivers/net/mlx5/mlx5_lsc_intr.h b/drivers/net/mlx5/mlx5_lsc_intr.h
new file mode 100644
index 0000000000..4e4a1eef5b
--- /dev/null
+++ b/drivers/net/mlx5/mlx5_lsc_intr.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2026 NVIDIA Corporation & Affiliates
+ */
+
+#ifndef RTE_PMD_MLX5_LSC_INTR_H_
+#define RTE_PMD_MLX5_LSC_INTR_H_
+
+#include <stdbool.h>
+
+struct rte_eth_dev;
+
+int mlx5_lsc_intr_setup(struct rte_eth_dev *dev);
+void mlx5_lsc_intr_cleanup(struct rte_eth_dev *dev);
+
+#endif /* RTE_PMD_MLX5_LSC_INTR_H_ */
diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c
index c5f493117b..7c166fa687 100644
--- a/drivers/net/mlx5/mlx5_trigger.c
+++ b/drivers/net/mlx5/mlx5_trigger.c
@@ -18,6 +18,7 @@
 #include "mlx5_rx.h"
 #include "mlx5_tx.h"
 #include "mlx5_utils.h"
+#include "mlx5_lsc_intr.h"
 #include "rte_pmd_mlx5.h"
 
 static void mlx5_traffic_disable_legacy(struct rte_eth_dev *dev);
@@ -1507,6 +1508,14 @@ mlx5_dev_start(struct rte_eth_dev *dev)
 	dev->rx_pkt_burst = mlx5_select_rx_function(dev);
 	/* Enable datapath on secondary process. */
 	mlx5_mp_os_req_start_rxtx(dev);
+	if (dev->data->dev_conf.intr_conf.lsc) {
+		ret = mlx5_lsc_intr_setup(dev);
+		if (ret < 0) {
+			DRV_LOG(INFO, "port %u failed configure LSC interrupts, starting without",
+				dev->data->port_id);
+			dev->data->dev_conf.intr_conf.lsc = 0;
+		}
+	}
 	if (rte_intr_fd_get(priv->sh->intr_handle) >= 0) {
 		priv->sh->port[priv->dev_port - 1].ih_port_id =
 					(uint32_t)dev->data->port_id;
@@ -1515,14 +1524,6 @@ mlx5_dev_start(struct rte_eth_dev *dev)
 			dev->data->port_id);
 		dev->data->dev_conf.intr_conf.rmv = 0;
 	}
-	if (rte_intr_fd_get(priv->sh->intr_handle_nl) >= 0) {
-		priv->sh->port[priv->dev_port - 1].nl_ih_port_id =
-					(uint32_t)dev->data->port_id;
-	} else {
-		DRV_LOG(INFO, "port %u starts without LSC interrupts.",
-			dev->data->port_id);
-		dev->data->dev_conf.intr_conf.lsc = 0;
-	}
 	if (rte_intr_fd_get(priv->sh->intr_handle_devx) >= 0)
 		priv->sh->port[priv->dev_port - 1].devx_ih_port_id =
 					(uint32_t)dev->data->port_id;
@@ -1664,7 +1665,7 @@ mlx5_dev_stop(struct rte_eth_dev *dev)
 	mlx5_rx_intr_vec_disable(dev);
 	priv->sh->port[priv->dev_port - 1].ih_port_id = RTE_MAX_ETHPORTS;
 	priv->sh->port[priv->dev_port - 1].devx_ih_port_id = RTE_MAX_ETHPORTS;
-	priv->sh->port[priv->dev_port - 1].nl_ih_port_id = RTE_MAX_ETHPORTS;
+	mlx5_lsc_intr_cleanup(dev);
 	mlx5_txq_stop(dev);
 	mlx5_rxq_stop(dev);
 	mlx5_dev_free_consec_tx_mem(dev, true);
diff --git a/drivers/net/mlx5/windows/meson.build b/drivers/net/mlx5/windows/meson.build
index c7d6617275..73cdcc47ae 100644
--- a/drivers/net/mlx5/windows/meson.build
+++ b/drivers/net/mlx5/windows/meson.build
@@ -8,4 +8,5 @@ sources += files(
         'mlx5_ethdev_os.c',
         'mlx5_vlan_os.c',
         'mlx5_flow_os.c',
+        'mlx5_lsc_intr.c',
 )
diff --git a/drivers/net/mlx5/windows/mlx5_lsc_intr.c b/drivers/net/mlx5/windows/mlx5_lsc_intr.c
new file mode 100644
index 0000000000..eac5bfdac2
--- /dev/null
+++ b/drivers/net/mlx5/windows/mlx5_lsc_intr.c
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (c) 2026 NVIDIA Corporation & Affiliates
+ */
+
+#include <errno.h>
+#include <stdbool.h>
+
+#include <ethdev_driver.h>
+#include <rte_common.h>
+
+#include "mlx5_utils.h"
+#include "mlx5_lsc_intr.h"
+
+int
+mlx5_lsc_intr_setup(struct rte_eth_dev *dev)
+{
+	DRV_LOG(ERR, "port %u does not support LSC interrupts", dev->data->port_id);
+	return -ENOTSUP;
+}
+
+void
+mlx5_lsc_intr_cleanup(struct rte_eth_dev *dev __rte_unused)
+{
+}
-- 
2.43.0

Re: [PATCH] net/mlx5: move to polling for LSC interrupts

[Stephen Hemminger]

On Mon, 26 Oct 2026 17:04:00 +0100
Dariusz Sosnowski <dsosnowski@nvidia.com> wrote:

> +/**
> + * Retrieve the carrier state of a device.
> + *
> + * The IFF_RUNNING flag reported by SIOCGIFFLAGS reflects the RFC 2863
> + * operational state, which the kernel only refreshes from its link watch
> + * work queue and which may therefore lag the actual carrier by up to a
> + * second. ETHTOOL_GLINK reports the carrier directly, and additionally
> + * flushes any link watch event still pending for the device.
> + *
> + * @param dev
> + *   Pointer to Ethernet device structure.
> + *
> + * @return
> + *   1 if the carrier is up, 0 otherwise.
> + *   Negative errno value is returned on error.
> + */
> +int
> +mlx5_link_carrier_get(struct rte_eth_dev *dev)
> +{
> +	struct ethtool_value ecmd = {
> +		.cmd = ETHTOOL_GLINK,
> +	};
> +	struct ifreq ifr;
> +	int ret;
> +
> +	memset(&ifr, 0, sizeof(ifr));

Odd, why not use structure initialization for both?

Re: [PATCH] net/mlx5: move to polling for LSC interrupts

[Stephen Hemminger]

On Mon, 26 Oct 2026 17:04:00 +0100
Dariusz Sosnowski <dsosnowski@nvidia.com> wrote:

> mlx5 networking driver on Linux implemented LSC interrupts
> by listening on netlink events.
> This method is unfortunately not reliable enough,
> because of linkwatch mechanism. It can delay link up/down events
> for up to ~1 second. This introduces unnecessary delay
> when mlx5 ports are used with DPDK bonding driver for examples.
> 
> This patch addresses that by migrating LSC checks
> to polling on link status using ETHTOOL_GLINK command.
> This ethtool command, implemented by mlx5 since kernel 4.14,
> will always get the current link status.
> As a side effect it is also flushes the pending link watch events.
> 
> Fixes: 17f95513adca ("net/mlx5: fix link status change detection")
> Cc: dmitry.kozliuk@gmail.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Cosmin Ratiu <cratiu@nvidia.com>
> Signed-off-by: Dariusz Sosnowski <dsosnowski@nvidia.com>
> ---

Agree that 1s delay is bad for bonding, there are historical reasons for
link watch. With some old NIC's the PHY would stream up/down on loose
cables causing DoS on the machine. If you are using LSC for bonding
and implementing LSC in driver, you might want to consider some rate
limiting.

AI review showed some issues.

[PATCH] net/mlx5: move to polling for LSC interrupts

Applies cleanly to main (a635140). Moves Linux LSC detection from
netlink RTM_NEWLINK to a 100ms EAL alarm polling ETHTOOL_GLINK.

Warning

1. Poller stores a partial, non-atomic dev_link.

lsc_intr_check_links() copies dev->data->dev_link, flips only
link_status, and writes the struct back with a plain assignment.
mlx5_link_update() (now using rte_eth_linkstatus_set(), an atomic
exchange on val64) can run concurrently from an application thread,
so the read-modify-write can lose that update. On link up, speed and
duplex are left at whatever the last link-down stored. The poller
also ignores IFF_UP while mlx5_link_update() reports
IFF_UP && carrier, so the two paths can disagree.

Keep the cheap carrier probe, and on change refresh the full record
through the existing path:

	ret = mlx5_link_carrier_get(dev);
	if (ret < 0 || ret == dev->data->dev_link.link_status ||
	    mlx5_link_update(dev, 0) <= 0) {
		devs[i] = NULL;
		continue;
	}

2. Polling cost on the host.

ETHTOOL_GLINK runs under rtnl_lock in the kernel. This is 10 Hz per
started port with lsc=1, including representors in switchdev setups,
per DPDK process. The adaptive period only reacts to latency seen by
the DPDK interrupt thread, not to rtnl contention imposed on the rest
of the system. Was this measured with a large number of representors?

The delay 17f95513adca worked around was IFF_RUNNING lagging the IB
port event. With the query now done via ETHTOOL_GLINK (reads carrier
and syncs linkwatch), going back to IBV_EVENT_PORT_ACTIVE/ERR on the
existing shared async handler and querying carrier there would give
immediate events without polling. Please explain why that was not
chosen, or use it.

3. Not stable material as is.

A new 300 line polling framework using rte_bitset (24.11 only) is
not a backport candidate for 23.11, and reducing a ~1s notification
delay is not a regression fix. Suggest splitting:
  - small fix: use ETHTOOL_GLINK in mlx5_link_update_unlocked_gs/gset
    (Fixes: 17f95513adca, Cc: stable)
  - the LSC mechanism change, without Fixes/Cc stable

Info

4. Windows stub logs ERR on every dev_start with lsc=1, while the PCI
driver still advertises RTE_PCI_DRV_INTR_LSC. The old path logged
INFO. Make the stub silent; mlx5_dev_start() already logs. Also
"failed configure" -> "failed to configure" in that message.

5. Leftovers: the rte_io_wmb() in the mlx5_dev_spawn() error path
ordered the removed nl_ih_port_id store and can go; <stdbool.h> in
mlx5_lsc_intr.h is unused.

6. mlx5_link_update() now reports only link_status changes instead
of a memcmp of the whole struct. Fine, but mention it in the commit
message.

7. Commit message: "for examples" -> "for example",
"it is also flushes" -> "it also flushes".