From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F0BD735F17B for ; Wed, 4 Mar 2026 16:00:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.46 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772640055; cv=none; b=ZUTh22R9X8u2GWnKJ9rScJiox+KFZrg3gJdwdts5S2DixEwfVD+i6yN8g6B3qz583QDSJ6OFX7W6GpkvU6F9ZCR2GJiyaxZym/0QpzwqO9HtLFPrkN48AQTqr8bdtle0/VjjEP67x5d38G2HUikO/IA6+NYfQxbo9q5ZzAu/WKY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772640055; c=relaxed/simple; bh=nAUS3G3ePSgJTCurwLZklAEdq9FSoOxVlxZL3wKzHh4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tM6JfYY55py2UeOlI6nw4WZ0oP9R+fD3viTmPuKd3Gbhmexl0E8/AigmNY95QwZU1k81CiamvDGCsYzSq2PMja85XF42R2Bdkn1ps1xAhkxR+kAgm9PPQbPecxF3Bqh9iq4vJXYDGrA6xJxFksNZiU4aNW8KHrj8mmoRUh8L9lQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=resnulli.us; spf=none smtp.mailfrom=resnulli.us; dkim=pass (2048-bit key) header.d=resnulli-us.20230601.gappssmtp.com header.i=@resnulli-us.20230601.gappssmtp.com header.b=zshcCcTM; arc=none smtp.client-ip=209.85.221.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=resnulli.us Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=resnulli.us Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20230601.gappssmtp.com header.i=@resnulli-us.20230601.gappssmtp.com header.b="zshcCcTM" Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-439c9eb5d36so716201f8f.2 for ; Wed, 04 Mar 2026 08:00:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20230601.gappssmtp.com; s=20230601; t=1772640052; x=1773244852; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sq+BNITtc6p+HH6jh9NI7lkPW1WCKPvQbV7cZpOCkHI=; b=zshcCcTM5vzhC3nX68hniKrLraivQekWsnTGJLY5YKSMG1x7POpaW1KgWQoziwYpqZ DASnhyT2QYMl630e9eql3oegGnmh8ei58/+mkkBMQELoFe56ciUH4I02cxtl4v4nIFku CVQ4OE4/RCDbWW5128LlZTSh3MaJhlhVOXOyIaC5vLWF/qx2/w3IacaYk1XgT0lGiwrX dyK2lS9q30KmG1utBq+z2D6qEYNf4ij1oLsjX3lM+TafqAE2ef47HDYZbTEUR8worxYU tqUd0g+ZfsHXs5aWcvW71VMYWXhwn0e1KbSYkg/gZW9qqv71Y3hQ8TvJUnsL+nrDHqAc 2EjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772640052; x=1773244852; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=sq+BNITtc6p+HH6jh9NI7lkPW1WCKPvQbV7cZpOCkHI=; b=uhlUIItKR03gcBVxYGio2fsRopXDZN9SHMYypGuUwPW0Aimtedat7BOy9t/UJt4pR4 BmARf0KOpKwIpGOJaaRky517m0QSZzNs3OYgb+6VZq6wzcUyepBrRjfKqhpmtJ9Hx1uL TCWhxk4tX6pHmhWHayBn7prj3zFisyruorNiBwW3fqyG7VShcJVOacdqbhZAdyRQWD/0 WnxYD+pZCtcrt5EnMEzzIiXvtt+PAH+wuElS9FATY5OaIaSDXM8/ek6IlgjBgZKEz/sO 2WRlhjkOTTNNJ5IL3gWyX07NKIbBkrg8O8cDbUvL7/DSm+z4U2FAAO3yBzPYsJtFo7gn JBzA== X-Gm-Message-State: AOJu0Yx80WAPlLuFSxuXfofEbV1l0eRz9F5LOZ6nUWwbYCW8vF2UUMCw ydJmwYo6R+yxvWq0kR1w1O+VdMNKuUFRsh9pIfR1BxcBzYkn0OZrihW+kCiNGkUJWKFV56Fa/Ru De7A4Sk4= X-Gm-Gg: ATEYQzymUHDKGv4VAOjfG8sd+YyijpggRqGyJFyBKTQzAjybdoXs/U7cvSng6Nzt+uF WPaQbdup2vOeDsrPvZsUq5XmxePa2wzSK/prOPNIYhrDgg1PM4DMFXknlEMADaSdXfKANDKerll 9LPE97O3qrpZw7REZNK0Mkbm4jM0QLCv+sRjIvil2scPbGaYcSlfrKLzeWYFN7lOnGoeFWLVmXc sIzPzGkDeR+sYZIGbPdHSsJlOo4n680T6uOR+IADNfFEGBvDk1Fj85BWGSS6URijo8flP+PwFgr GvTvV1d/YP9qLIgw2Iu8XERpv1aK1WR2OsEe+7vD5Brf/D/VLa020u5u+tEWKbMIlRhkuteS7tY t6r8/GhL3VDNqtdK0cPPQqLx0q13fJ9yf9vvK8vcC/d2SIEeOYKXT5oyUebX4vksxti7uTfhQYL PQrwfZMHfy/KGBMSHs/pXJu+B6JBfmYWK2N58voVK/840dCQ== X-Received: by 2002:a05:6000:230c:b0:439:b619:33b2 with SMTP id ffacd0b85a97d-439c7fe64b4mr4723346f8f.43.1772640052085; Wed, 04 Mar 2026 08:00:52 -0800 (PST) Received: from localhost (46-13-72-179.customers.tmcz.cz. [46.13.72.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-439c65e0b23sm7919924f8f.32.2026.03.04.08.00.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Mar 2026 08:00:51 -0800 (PST) From: Jiri Pirko To: netdev@vger.kernel.org Cc: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, horms@kernel.org, donald.hunter@gmail.com, corbet@lwn.net, skhan@linuxfoundation.org, saeedm@nvidia.com, leon@kernel.org, tariqt@nvidia.com, mbloch@nvidia.com, przemyslaw.kitszel@intel.com, mschmidt@redhat.com, andrew+netdev@lunn.ch, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, chuck.lever@oracle.com, matttbe@kernel.org, cjubran@nvidia.com, daniel.zahka@gmail.com, linux-doc@vger.kernel.org, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH net-next v3 13/13] net/mlx5: Add a shared devlink instance for PFs on same chip Date: Wed, 4 Mar 2026 17:00:22 +0100 Message-ID: <20260304160022.6114-14-jiri@resnulli.us> X-Mailer: git-send-email 2.51.1 In-Reply-To: <20260304160022.6114-1-jiri@resnulli.us> References: <20260304160022.6114-1-jiri@resnulli.us> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Jiri Pirko Use the previously introduced shared devlink infrastructure to create a shared devlink instance for mlx5 PFs that reside on the same physical chip. The shared instance is identified by the chip's serial number extracted from PCI VPD (V3 keyword, with fallback to serial number for older devices). Each PF that probes calls mlx5_shd_init() which extracts the chip serial number and uses devlink_shd_get() to get or create the shared instance. When a PF is removed, mlx5_shd_uninit() calls devlink_shd_put() to release the reference. The shared instance is automatically destroyed when the last PF is removed. Make the PF devlink instances nested in this shared devlink instance, allowing userspace to identify which PFs belong to the same physical chip. Example: pci/0000:08:00.0: index 0 nested_devlink: auxiliary/mlx5_core.eth.0 devlink_index/1: index 1 nested_devlink: pci/0000:08:00.0 pci/0000:08:00.1 auxiliary/mlx5_core.eth.0: index 2 pci/0000:08:00.1: index 3 nested_devlink: auxiliary/mlx5_core.eth.1 auxiliary/mlx5_core.eth.1: index 4 Signed-off-by: Jiri Pirko --- v2->v3: - removed "const" from "sn" - passing driver pointer to devlink_shd_get() --- .../net/ethernet/mellanox/mlx5/core/Makefile | 5 +- .../net/ethernet/mellanox/mlx5/core/main.c | 17 ++++++ .../ethernet/mellanox/mlx5/core/sh_devlink.c | 61 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/sh_devlink.h | 12 ++++ include/linux/mlx5/driver.h | 1 + 5 files changed, 94 insertions(+), 2 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 8ffa286a18f5..d39fe9c4a87c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -16,8 +16,9 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \ transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \ fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \ lib/devcom.o lib/pci_vsc.o lib/dm.o lib/fs_ttc.o diag/fs_tracepoint.o \ - diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o diag/reporter_vnic.o \ - fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o lib/nv_param.o + diag/fw_tracer.o diag/crdump.o devlink.o sh_devlink.o diag/rsc_dump.o \ + diag/reporter_vnic.o fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o \ + lib/nv_param.o # # Netdev basic diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index fdc3ba20912e..1c35c3fc3bb3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -74,6 +74,7 @@ #include "mlx5_irq.h" #include "hwmon.h" #include "lag/lag.h" +#include "sh_devlink.h" MODULE_AUTHOR("Eli Cohen "); MODULE_DESCRIPTION("Mellanox 5th generation network adapters (ConnectX series) core driver"); @@ -1520,10 +1521,16 @@ int mlx5_init_one(struct mlx5_core_dev *dev) int err; devl_lock(devlink); + if (dev->shd) { + err = devl_nested_devlink_set(dev->shd, devlink); + if (err) + goto unlock; + } devl_register(devlink); err = mlx5_init_one_devl_locked(dev); if (err) devl_unregister(devlink); +unlock: devl_unlock(devlink); return err; } @@ -2005,6 +2012,13 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id) goto pci_init_err; } + err = mlx5_shd_init(dev); + if (err) { + mlx5_core_err(dev, "mlx5_shd_init failed with error code %d\n", + err); + goto shd_init_err; + } + err = mlx5_init_one(dev); if (err) { mlx5_core_err(dev, "mlx5_init_one failed with error code %d\n", @@ -2018,6 +2032,8 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id) return 0; err_init_one: + mlx5_shd_uninit(dev); +shd_init_err: mlx5_pci_close(dev); pci_init_err: mlx5_mdev_uninit(dev); @@ -2039,6 +2055,7 @@ static void remove_one(struct pci_dev *pdev) mlx5_drain_health_wq(dev); mlx5_sriov_disable(pdev, false); mlx5_uninit_one(dev); + mlx5_shd_uninit(dev); mlx5_pci_close(dev); mlx5_mdev_uninit(dev); mlx5_adev_idx_free(dev->priv.adev_idx); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c new file mode 100644 index 000000000000..bc33f95302df --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include +#include + +#include "sh_devlink.h" + +static const struct devlink_ops mlx5_shd_ops = { +}; + +int mlx5_shd_init(struct mlx5_core_dev *dev) +{ + u8 *vpd_data __free(kfree) = NULL; + struct pci_dev *pdev = dev->pdev; + unsigned int vpd_size, kw_len; + struct devlink *devlink; + char *sn, *end; + int start; + int err; + + if (!mlx5_core_is_pf(dev)) + return 0; + + vpd_data = pci_vpd_alloc(pdev, &vpd_size); + if (IS_ERR(vpd_data)) { + err = PTR_ERR(vpd_data); + return err == -ENODEV ? 0 : err; + } + start = pci_vpd_find_ro_info_keyword(vpd_data, vpd_size, "V3", &kw_len); + if (start < 0) { + /* Fall-back to SN for older devices. */ + start = pci_vpd_find_ro_info_keyword(vpd_data, vpd_size, + PCI_VPD_RO_KEYWORD_SERIALNO, &kw_len); + if (start < 0) + return -ENOENT; + } + sn = kstrndup(vpd_data + start, kw_len, GFP_KERNEL); + if (!sn) + return -ENOMEM; + /* Firmware may return spaces at the end of the string, strip it. */ + end = strchrnul(sn, ' '); + *end = '\0'; + + /* Get or create shared devlink instance */ + devlink = devlink_shd_get(sn, &mlx5_shd_ops, 0, pdev->dev.driver); + kfree(sn); + if (!devlink) + return -ENOMEM; + + dev->shd = devlink; + return 0; +} + +void mlx5_shd_uninit(struct mlx5_core_dev *dev) +{ + if (!dev->shd) + return; + + devlink_shd_put(dev->shd); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h new file mode 100644 index 000000000000..8ab8d6940227 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/sh_devlink.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_SH_DEVLINK_H__ +#define __MLX5_SH_DEVLINK_H__ + +#include + +int mlx5_shd_init(struct mlx5_core_dev *dev); +void mlx5_shd_uninit(struct mlx5_core_dev *dev); + +#endif /* __MLX5_SH_DEVLINK_H__ */ diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 04dcd09f7517..1268fcf35ec7 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -798,6 +798,7 @@ struct mlx5_core_dev { enum mlx5_wc_state wc_state; /* sync write combining state */ struct mutex wc_state_lock; + struct devlink *shd; }; struct mlx5_db { -- 2.51.1