netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Tariq Toukan <tariqt@nvidia.com>
Cc: Eric Dumazet <edumazet@google.com>,
	Paolo Abeni <pabeni@redhat.com>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Donald Hunter <donald.hunter@gmail.com>,
	Jiri Pirko <jiri@resnulli.us>, Jonathan Corbet <corbet@lwn.net>,
	Saeed Mahameed <saeedm@nvidia.com>,
	"Leon Romanovsky" <leon@kernel.org>,
	Mark Bloch <mbloch@nvidia.com>, <netdev@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-doc@vger.kernel.org>,
	<linux-rdma@vger.kernel.org>, Gal Pressman <gal@nvidia.com>,
	Moshe Shemesh <moshe@nvidia.com>,
	Carolina Jubran <cjubran@nvidia.com>,
	Cosmin Ratiu <cratiu@nvidia.com>, Jiri Pirko <jiri@nvidia.com>,
	Randy Dunlap <rdunlap@infradead.org>
Subject: Re: [PATCH net-next V4 02/14] documentation: networking: add shared devlink documentation
Date: Thu, 27 Nov 2025 20:16:45 -0800	[thread overview]
Message-ID: <20251127201645.3d7a10f6@kernel.org> (raw)
In-Reply-To: <1764101173-1312171-3-git-send-email-tariqt@nvidia.com>

On Tue, 25 Nov 2025 22:06:01 +0200 Tariq Toukan wrote:
> From: Jiri Pirko <jiri@nvidia.com>
> 
> Document shared devlink instances for multiple PFs on the same chip.
> 
> Signed-off-by: Jiri Pirko <jiri@nvidia.com>
> Reviewed-by: Cosmin Ratiu <cratiu@nvidia.com>
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  .../networking/devlink/devlink-shared.rst     | 66 +++++++++++++++++++
>  Documentation/networking/devlink/index.rst    |  1 +
>  2 files changed, 67 insertions(+)
>  create mode 100644 Documentation/networking/devlink/devlink-shared.rst
> 
> diff --git a/Documentation/networking/devlink/devlink-shared.rst b/Documentation/networking/devlink/devlink-shared.rst
> new file mode 100644
> index 000000000000..be9dd6f295df
> --- /dev/null
> +++ b/Documentation/networking/devlink/devlink-shared.rst
> @@ -0,0 +1,66 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +============================
> +Devlink Shared Instances
> +============================
> +
> +Overview
> +========
> +
> +Shared devlink instances allow multiple physical functions (PFs) on the same
> +chip to share an additional devlink instance for chip-wide operations. This
> +should be implemented within individual drivers alongside the individual PF
> +devlink instances, not replacing them.
> +
> +The shared devlink instance should be backed by a faux device and should
> +provide a common interface for operations that affect the entire chip
> +rather than individual PFs.

If we go with this we must state very clearly that this is a crutch and
_not_ the recommended configuration...

> +Implementation
> +==============
> +
> +Architecture
> +------------
> +
> +The implementation should use:
> +
> +* **Faux device**: Virtual device backing the shared devlink instance
> +* **Chip identification**: PFs are grouped by chip using a driver-specific identifier
> +* **Shared instance management**: Global list of shared instances with reference counting
> +
> +Initialization Flow
> +-------------------
> +
> +1. **PF calls shared devlink init** during driver probe
> +2. **Chip identification** using driver-specific method to determine device identity
> +3. **Lookup existing shared instance** for this chip identifier
> +4. **Create new shared instance** if none exists:
> +
> +   * Create faux device with chip identifier as name
> +   * Allocate and register devlink instance
> +   * Add to global shared instances list
> +
> +5. **Add PF to shared instance** PF list
> +6. **Set nested devlink instance** for the PF devlink instance

... because presumably we could use this infra to manage a single
devlink instance? Which is what I asked for initially.

> +Cleanup Flow
> +------------
> +
> +1. **Cleanup** when PF is removed; destroy shared instance when last PF is removed
> +
> +Chip Identification
> +-------------------
> +
> +PFs belonging to the same chip are identified using a driver-specific method.
> +The driver is free to choose any identifier that is suitable for determining
> +whether two PFs are part of the same device. Examples include VPD serial numbers,
> +device tree properties, or other hardware-specific identifiers.
> +
> +Locking
> +-------
> +
> +A global per-driver mutex protects the shared instances list and individual shared
> +instance PF lists during registration/deregistration.

Why can't this mutex live in the core?

> +Similarly to other nested devlink instance relationships, devlink lock of
> +the shared instance should be always taken after the devlink lock of PF.
> diff --git a/Documentation/networking/devlink/index.rst b/Documentation/networking/devlink/index.rst
> index 35b12a2bfeba..f7ba7dcf477d 100644
> --- a/Documentation/networking/devlink/index.rst
> +++ b/Documentation/networking/devlink/index.rst
> @@ -68,6 +68,7 @@ general.
>     devlink-resource
>     devlink-selftests
>     devlink-trap
> +   devlink-shared
>  
>  Driver-specific documentation
>  -----------------------------


  reply	other threads:[~2025-11-28  4:16 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-25 20:05 [PATCH net-next V4 00/14] devlink and mlx5: Support cross-function rate scheduling Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 01/14] devlink: Reverse locking order for nested instances Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 02/14] documentation: networking: add shared devlink documentation Tariq Toukan
2025-11-28  4:16   ` Jakub Kicinski [this message]
2025-11-28 11:00     ` Jiri Pirko
2025-11-29  3:19       ` Jakub Kicinski
2025-12-01 10:50         ` Jiri Pirko
2025-12-01 21:49           ` Jakub Kicinski
2025-12-02  7:43             ` Jiri Pirko
2025-12-02 18:14               ` Jakub Kicinski
2025-12-03 10:36                 ` Jiri Pirko
2025-12-04 18:57                   ` Jakub Kicinski
2025-12-05  9:39                     ` Jiri Pirko
2025-11-25 20:06 ` [PATCH net-next V4 03/14] devlink: Add helpers to lock nested-in instances Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 04/14] devlink: Refactor devlink_rate_nodes_check Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 05/14] devlink: Decouple rate storage from associated devlink object Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 06/14] devlink: Add parent dev to devlink API Tariq Toukan
2025-11-27 15:28   ` Simon Horman
2025-11-27 19:18     ` Cosmin Ratiu
2025-11-25 20:06 ` [PATCH net-next V4 07/14] devlink: Allow parent dev for rate-set and rate-new Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 08/14] devlink: Allow rate node parents from other devlinks Tariq Toukan
2025-11-28  4:09   ` Jakub Kicinski
2025-11-28  9:57     ` Cosmin Ratiu
2025-11-25 20:06 ` [PATCH net-next V4 09/14] net/mlx5: Introduce shared devlink instance for PFs on same chip Tariq Toukan
2025-11-29 14:08   ` Krzysztof Kozlowski
2025-11-25 20:06 ` [PATCH net-next V4 10/14] net/mlx5: Expose a function to clear a vport's parent Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 11/14] net/mlx5: Store QoS sched nodes in the sh_devlink Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 12/14] net/mlx5: qos: Support cross-device tx scheduling Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 13/14] net/mlx5: qos: Enable cross-device scheduling Tariq Toukan
2025-11-25 20:06 ` [PATCH net-next V4 14/14] net/mlx5: Document devlink rates Tariq Toukan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251127201645.3d7a10f6@kernel.org \
    --to=kuba@kernel.org \
    --cc=andrew+netdev@lunn.ch \
    --cc=cjubran@nvidia.com \
    --cc=corbet@lwn.net \
    --cc=cratiu@nvidia.com \
    --cc=davem@davemloft.net \
    --cc=donald.hunter@gmail.com \
    --cc=edumazet@google.com \
    --cc=gal@nvidia.com \
    --cc=jiri@nvidia.com \
    --cc=jiri@resnulli.us \
    --cc=leon@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mbloch@nvidia.com \
    --cc=moshe@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=rdunlap@infradead.org \
    --cc=saeedm@nvidia.com \
    --cc=tariqt@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).