[PATCH net-next 00/12] net/mlx5: Add satellite PF support

[PATCH net-next 00/12] net/mlx5: Add satellite PF support

[Tariq Toukan]

Hi,

See detailed feature description by Moshe below.

Regards,
Tariq

A satellite PF is a new SmartNIC configuration that adds another
physical function on the DPU that is not an eswitch manager and not a
page manager. The satellite PF can have its own SFs and can be passed
through to a VM on the DPU, providing an isolated function for users who
should not have access to the privileged ECPF. The ECPF handles the
satellite PF and the host PF in a similar way, using the same management
framework.

This series adds support for satellite PFs (SPFs) in the mlx5 eswitch.
SPFs are discovered through the v1 response layout of the
query_esw_functions command, introduced in the previous infrastructure
preparation series.

The first four patches discover satellite PFs, allocate eswitch vports
for them and their SFs, and extend the SF hardware table to manage SPF
SF entries.

The next five patches expose PF numbers from firmware, map SF
controllers to their pfnum, register devlink ports with proper
attributes, and register SF resource on satellite PF ports.

The final four patches add devlink port state management, FDB peer miss
rules, dedicated page accounting, and SF resource registration for
satellite PF vports.

This series builds on the eswitch infrastructure preparation series
previously submitted.


Moshe Shemesh (11):
  net/mlx5: Add satellite PF vport support
  net/mlx5: Introduce generic helper for PF SFs info
  net/mlx5: Initialize host PF host number earlier
  net/mlx5: Initialize satellite PF SF vports
  net/mlx5: Support SPF SFs in SF hardware table
  net/mlx5: Expose PF number from query_esw_functions
  net/mlx5: Map SF controller to pfnum for satellite PFs
  net/mlx5: Register devlink ports for satellite PFs
  net/mlx5: Support state get/set for satellite PF ports
  net/mlx5: Add FDB peer miss rules for satellite PFs
  net/mlx5: Add SPF function type for page management

Or Har-Toov (1):
  net/mlx5: Register SF resource on satellite PF ports

 .../net/ethernet/mellanox/mlx5/core/debugfs.c |   3 +
 .../net/ethernet/mellanox/mlx5/core/ecpf.c    |   5 +
 .../mellanox/mlx5/core/esw/adj_vport.c        |   6 +-
 .../mellanox/mlx5/core/esw/devlink_port.c     |  59 ++-
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 398 ++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  43 +-
 .../mellanox/mlx5/core/eswitch_offloads.c     |  86 ++--
 .../ethernet/mellanox/mlx5/core/pagealloc.c   |   3 +
 .../ethernet/mellanox/mlx5/core/sf/devlink.c  |  14 +-
 .../ethernet/mellanox/mlx5/core/sf/hw_table.c |  89 +++-
 include/linux/mlx5/driver.h                   |   1 +
 11 files changed, 595 insertions(+), 112 deletions(-)


base-commit: 33fb2e2bc7a43c79f02dad79c39ff04ae6dc224f
-- 
2.44.0

[PATCH net-next 01/12] net/mlx5: Add satellite PF vport support

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Discover satellite PFs from query_esw_functions output and allocate
eswitch vports for them. For each satellite PF, create a vport via the
CREATE_ESW_VPORT command using its vhca_id and allocate it in the
eswitch vport table.

When enabling switchdev mode, the ECPF acting as the eswitch manager
activates each satellite PF with enable_hca, loads its vport and adds
a representor. Since satellite PF devlink ports are registered in a
later patch, guard mlx5_esw_offloads_devlink_port() against vports
with no devlink port to avoid NULL dereference during representor
attach.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/adj_vport.c        |   6 +-
 .../mellanox/mlx5/core/esw/devlink_port.c     |   7 +-
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 159 +++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  10 ++
 4 files changed, 171 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c
index 250af09b5af2..ca249b50f830 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/adj_vport.c
@@ -23,7 +23,7 @@ int mlx5_esw_adj_vport_modify(struct mlx5_core_dev *dev, u16 vport,
 	return mlx5_cmd_exec_in(dev, modify_vport_state, in);
 }
 
-static void mlx5_esw_destroy_esw_vport(struct mlx5_core_dev *dev, u16 vport)
+void mlx5_esw_destroy_esw_vport(struct mlx5_core_dev *dev, u16 vport)
 {
 	u32 in[MLX5_ST_SZ_DW(destroy_esw_vport_in)] = {};
 
@@ -34,8 +34,8 @@ static void mlx5_esw_destroy_esw_vport(struct mlx5_core_dev *dev, u16 vport)
 	mlx5_cmd_exec_in(dev, destroy_esw_vport, in);
 }
 
-static int mlx5_esw_create_esw_vport(struct mlx5_core_dev *dev, u16 vhca_id,
-				     u16 *vport_num)
+int mlx5_esw_create_esw_vport(struct mlx5_core_dev *dev, u16 vhca_id,
+			      u16 *vport_num)
 {
 	u32 out[MLX5_ST_SZ_DW(create_esw_vport_out)] = {};
 	u32 in[MLX5_ST_SZ_DW(create_esw_vport_in)] = {};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index 8a79764345e7..0730f0c883fe 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -253,5 +253,10 @@ struct devlink_port *mlx5_esw_offloads_devlink_port(struct mlx5_eswitch *esw, u1
 	struct mlx5_vport *vport;
 
 	vport = mlx5_eswitch_get_vport(esw, vport_num);
-	return IS_ERR(vport) ? ERR_CAST(vport) : &vport->dl_port->dl_port;
+	if (IS_ERR(vport))
+		return ERR_CAST(vport);
+	if (!vport->dl_port)
+		return ERR_PTR(-ENODEV);
+
+	return &vport->dl_port->dl_port;
 }
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 206911817a04..e75925a99852 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1517,8 +1517,11 @@ int
 mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 				 enum mlx5_eswitch_vport_event enabled_events)
 {
+	struct mlx5_esw_functions *esw_funcs = &esw->esw_funcs;
 	bool pf_needed;
+	u16 vport_num;
 	int ret;
+	int i;
 
 	pf_needed = mlx5_core_is_ecpf_esw_manager(esw->dev) ||
 		    esw->mode == MLX5_ESWITCH_LEGACY;
@@ -1548,14 +1551,14 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 	/* Enable ECVF vports */
 	if (mlx5_core_ec_sriov_enabled(esw->dev)) {
 		ret = mlx5_eswitch_load_ec_vf_vports(esw,
-						     esw->esw_funcs.num_ec_vfs,
+						     esw_funcs->num_ec_vfs,
 						     enabled_events);
 		if (ret)
 			goto ec_vf_err;
 	}
 
 	/* Enable VF vports */
-	ret = mlx5_eswitch_load_vf_vports(esw, esw->esw_funcs.num_vfs,
+	ret = mlx5_eswitch_load_vf_vports(esw, esw_funcs->num_vfs,
 					  enabled_events);
 	if (ret)
 		goto vf_err;
@@ -1565,13 +1568,36 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
 	if (ret)
 		goto unload_vf_vports;
 
+	/* Enable satellite PF vports */
+	for (i = 0; i < esw_funcs->num_spfs; i++) {
+		vport_num = esw_funcs->spfs[i].vport_num;
+
+		ret = mlx5_eswitch_load_pf_vf_vport(esw, vport_num,
+						    enabled_events);
+		if (ret)
+			goto spf_err;
+
+		ret = mlx5_esw_pf_enable_hca(esw->dev, vport_num);
+		if (ret) {
+			mlx5_eswitch_unload_pf_vf_vport(esw, vport_num);
+			goto spf_err;
+		}
+	}
+
 	return 0;
 
+spf_err:
+	while (i-- > 0) {
+		vport_num = esw_funcs->spfs[i].vport_num;
+		mlx5_esw_pf_disable_hca(esw->dev, vport_num);
+		mlx5_eswitch_unload_pf_vf_vport(esw, vport_num);
+	}
+	mlx5_eswitch_unload_adj_vf_vports(esw);
 unload_vf_vports:
-	mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs);
+	mlx5_eswitch_unload_vf_vports(esw, esw_funcs->num_vfs);
 vf_err:
 	if (mlx5_core_ec_sriov_enabled(esw->dev))
-		mlx5_eswitch_unload_ec_vf_vports(esw, esw->esw_funcs.num_ec_vfs);
+		mlx5_eswitch_unload_ec_vf_vports(esw, esw_funcs->num_ec_vfs);
 ec_vf_err:
 	if (mlx5_ecpf_vport_exists(esw->dev))
 		mlx5_eswitch_unload_pf_vf_vport(esw, MLX5_VPORT_ECPF);
@@ -1589,13 +1615,22 @@ mlx5_eswitch_enable_pf_vf_vports(struct mlx5_eswitch *esw,
  */
 void mlx5_eswitch_disable_pf_vf_vports(struct mlx5_eswitch *esw)
 {
+	struct mlx5_esw_functions *esw_funcs = &esw->esw_funcs;
+	u16 vport_num;
+	int i;
+
+	for (i = 0; i < esw_funcs->num_spfs; i++) {
+		vport_num = esw_funcs->spfs[i].vport_num;
+		mlx5_esw_pf_disable_hca(esw->dev, vport_num);
+		mlx5_eswitch_unload_pf_vf_vport(esw, vport_num);
+	}
+
 	mlx5_eswitch_unload_adj_vf_vports(esw);
 
-	mlx5_eswitch_unload_vf_vports(esw, esw->esw_funcs.num_vfs);
+	mlx5_eswitch_unload_vf_vports(esw, esw_funcs->num_vfs);
 
 	if (mlx5_core_ec_sriov_enabled(esw->dev))
-		mlx5_eswitch_unload_ec_vf_vports(esw,
-						 esw->esw_funcs.num_ec_vfs);
+		mlx5_eswitch_unload_ec_vf_vports(esw, esw_funcs->num_ec_vfs);
 
 	if (mlx5_ecpf_vport_exists(esw->dev)) {
 		mlx5_eswitch_unload_pf_vf_vport(esw, MLX5_VPORT_ECPF);
@@ -2065,11 +2100,105 @@ void mlx5_esw_vport_free(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
 	kfree(vport);
 }
 
+static void mlx5_esw_spfs_cleanup(struct mlx5_eswitch *esw)
+{
+	struct mlx5_esw_functions *esw_funcs = &esw->esw_funcs;
+	int i;
+
+	for (i = 0; i < esw_funcs->num_spfs; i++)
+		mlx5_esw_destroy_esw_vport(esw->dev,
+					   esw_funcs->spfs[i].vport_num);
+
+	kfree(esw_funcs->spfs);
+	esw_funcs->spfs = NULL;
+	esw_funcs->num_spfs = 0;
+}
+
+static int mlx5_esw_spfs_init(struct mlx5_eswitch *esw)
+{
+	struct mlx5_esw_functions *esw_funcs = &esw->esw_funcs;
+	struct mlx5_core_dev *dev = esw->dev;
+	int num_entries;
+	const u8 *entry;
+	const u32 *out;
+	int err = 0;
+	int pf_type;
+	u16 vhca_id;
+	int i;
+
+	if (!MLX5_CAP_GEN(dev, query_host_net_function_v1))
+		return 0;
+
+	out = mlx5_esw_query_functions(dev);
+	if (IS_ERR(out))
+		return PTR_ERR(out);
+
+	num_entries = MLX5_GET(query_esw_functions_out, out, net_function_num);
+	if (!num_entries)
+		goto out_free;
+
+	esw_funcs->spfs = kcalloc(num_entries, sizeof(*esw_funcs->spfs),
+				  GFP_KERNEL);
+	if (!esw_funcs->spfs) {
+		err = -ENOMEM;
+		goto out_free;
+	}
+
+	entry = MLX5_ADDR_OF(query_esw_functions_out, out, net_function_params);
+
+	for (i = 0; i < num_entries; i++) {
+		u16 vport_num;
+
+		pf_type = MLX5_GET(network_function_params, entry, pci_pf_type);
+		if (pf_type != MLX5_PCI_PF_TYPE_SATELLITE_PF) {
+			entry += MLX5_UN_SZ_BYTES(net_function_params);
+			continue;
+		}
+
+		if (!MLX5_GET(network_function_params, entry,
+			      esw_vport_manual)) {
+			esw_warn(dev, "Satellite PF without esw_vport_manual is not supported\n");
+			entry += MLX5_UN_SZ_BYTES(net_function_params);
+			continue;
+		}
+
+		vhca_id = MLX5_GET(network_function_params, entry, vhca_id);
+
+		err = mlx5_esw_create_esw_vport(dev, vhca_id, &vport_num);
+		if (err) {
+			esw_warn(dev, "Failed to create satellite PF vport for vhca_id 0x%x, err %d\n",
+				 vhca_id, err);
+			goto spfs_cleanup;
+		}
+
+		esw_funcs->spfs[esw_funcs->num_spfs].vport_num = vport_num;
+		esw_funcs->spfs[esw_funcs->num_spfs].vhca_id = vhca_id;
+		esw_funcs->num_spfs++;
+
+		entry += MLX5_UN_SZ_BYTES(net_function_params);
+	}
+
+	if (!esw_funcs->num_spfs) {
+		kfree(esw_funcs->spfs);
+		esw_funcs->spfs = NULL;
+	}
+
+	kvfree(out);
+	return 0;
+
+spfs_cleanup:
+	mlx5_esw_spfs_cleanup(esw);
+out_free:
+	kvfree(out);
+	return err;
+}
+
 static void mlx5_esw_vports_cleanup(struct mlx5_eswitch *esw)
 {
 	struct mlx5_vport *vport;
 	unsigned long i;
 
+	mlx5_esw_spfs_cleanup(esw);
 	mlx5_esw_for_each_vport(esw, i, vport)
 		mlx5_esw_vport_free(esw, vport);
 	xa_destroy(&esw->vports);
@@ -2123,6 +2252,22 @@ static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 		idx++;
 	}
 
+	err = mlx5_esw_spfs_init(esw);
+	if (err)
+		goto err;
+
+	for (i = 0; i < esw->esw_funcs.num_spfs; i++) {
+		struct mlx5_vport *vport;
+		u16 vport_num;
+
+		vport_num = esw->esw_funcs.spfs[i].vport_num;
+		err = mlx5_esw_vport_alloc(esw, idx++, vport_num);
+		if (err)
+			goto err;
+		vport = mlx5_eswitch_get_vport(esw, vport_num);
+		vport->vhca_id = esw->esw_funcs.spfs[i].vhca_id;
+	}
+
 	if (mlx5_core_ec_sriov_enabled(esw->dev)) {
 		int ec_vf_base_num = mlx5_core_ec_vf_vport_base(dev);
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index a5f832ed2251..19419799a26d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -349,11 +349,18 @@ struct mlx5_host_work {
 	void (*func)(struct mlx5_eswitch *esw);
 };
 
+struct mlx5_esw_spf {
+	u16 vport_num;
+	u16 vhca_id;
+};
+
 struct mlx5_esw_functions {
 	struct mlx5_nb		nb;
 	bool			host_funcs_disabled;
 	u16			num_vfs;
 	u16			num_ec_vfs;
+	struct mlx5_esw_spf	*spfs;
+	int			num_spfs;
 };
 
 enum {
@@ -666,6 +673,9 @@ void mlx5_esw_adjacent_vhcas_setup(struct mlx5_eswitch *esw);
 void mlx5_esw_adjacent_vhcas_cleanup(struct mlx5_eswitch *esw);
 int mlx5_esw_adj_vport_modify(struct mlx5_core_dev *dev, u16 vport,
 			      bool connect);
+int mlx5_esw_create_esw_vport(struct mlx5_core_dev *dev, u16 vhca_id,
+			      u16 *vport_num);
+void mlx5_esw_destroy_esw_vport(struct mlx5_core_dev *dev, u16 vport);
 
 #define MLX5_DEBUG_ESWITCH_MASK BIT(3)
 
-- 
2.44.0

[PATCH net-next 02/12] net/mlx5: Introduce generic helper for PF SFs info

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Introduce mlx5_esw_sf_max_pf_functions() that queries a PF's max_num_sf
and sf_base_id using mlx5_vport_get_other_func_general_cap(), which
supports both function_id and vhca_id based addressing.

Refactor mlx5_esw_sf_max_hpf_functions() into a thin wrapper that adds
the host PF precondition checks and calls the new generic helper. Remove
mlx5_query_hca_cap_host_pf() as it is not used anymore.

This prepares for querying SFs info of Satellite PFs.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 38 +++++++++----------
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index e75925a99852..815538ba754f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -2023,37 +2023,20 @@ void mlx5_eswitch_disable(struct mlx5_eswitch *esw)
 	mlx5_lag_enable_change(esw->dev);
 }
 
-static int mlx5_query_hca_cap_host_pf(struct mlx5_core_dev *dev, void *out)
-{
-	u16 opmod = (MLX5_CAP_GENERAL << 1) | (HCA_CAP_OPMOD_GET_MAX & 0x01);
-	u8 in[MLX5_ST_SZ_BYTES(query_hca_cap_in)] = {};
-
-	MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
-	MLX5_SET(query_hca_cap_in, in, op_mod, opmod);
-	MLX5_SET(query_hca_cap_in, in, function_id, MLX5_VPORT_HOST_PF);
-	MLX5_SET(query_hca_cap_in, in, other_function, true);
-	return mlx5_cmd_exec_inout(dev, query_hca_cap, in, out);
-}
-
-int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *sf_base_id)
-
+static int mlx5_esw_sf_max_pf_functions(struct mlx5_core_dev *dev,
+					u16 vport_num, u16 *max_sfs,
+					u16 *sf_base_id)
 {
 	int query_out_sz = MLX5_ST_SZ_BYTES(query_hca_cap_out);
 	void *query_ctx;
 	void *hca_caps;
 	int err;
 
-	if (!mlx5_core_is_ecpf(dev) ||
-	    !mlx5_esw_host_functions_enabled(dev)) {
-		*max_sfs = 0;
-		return 0;
-	}
-
 	query_ctx = kzalloc(query_out_sz, GFP_KERNEL);
 	if (!query_ctx)
 		return -ENOMEM;
 
-	err = mlx5_query_hca_cap_host_pf(dev, query_ctx);
+	err = mlx5_vport_get_other_func_general_cap(dev, vport_num, query_ctx);
 	if (err)
 		goto out_free;
 
@@ -2066,6 +2049,19 @@ int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *
 	return err;
 }
 
+int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs,
+				  u16 *sf_base_id)
+{
+	if (!mlx5_core_is_ecpf(dev) ||
+	    !mlx5_esw_host_functions_enabled(dev)) {
+		*max_sfs = 0;
+		return 0;
+	}
+
+	return mlx5_esw_sf_max_pf_functions(dev, MLX5_VPORT_HOST_PF, max_sfs,
+					    sf_base_id);
+}
+
 int mlx5_esw_vport_alloc(struct mlx5_eswitch *esw, int index, u16 vport_num)
 {
 	struct mlx5_vport *vport;
-- 
2.44.0

[PATCH net-next 03/12] net/mlx5: Initialize host PF host number earlier

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Move host_number from esw->offloads to esw->esw_funcs as hpf_host_number
and initialize it during vports_init instead of offloads_enable. This
makes the host PF host number available earlier in the initialization
sequence, which is required for upcoming SF hardware table support for
satellite PFs.

Add a mlx5_esw_get_hpf_host_number() accessor to retrieve the stored
host number.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/devlink_port.c     |  2 +-
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 33 +++++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  4 ++-
 .../mellanox/mlx5/core/eswitch_offloads.c     | 25 +-------------
 4 files changed, 38 insertions(+), 26 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index 0730f0c883fe..e723f05cd4d3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -34,7 +34,7 @@ static void mlx5_esw_offloads_pf_vf_devlink_port_attrs_set(struct mlx5_eswitch *
 	pfnum = PCI_FUNC(dev->pdev->devfn);
 	external = mlx5_core_is_ecpf_esw_manager(dev);
 	if (external)
-		controller_num = dev->priv.eswitch->offloads.host_number + 1;
+		controller_num = mlx5_esw_get_hpf_host_number(dev) + 1;
 
 	if (vport_num == MLX5_VPORT_HOST_PF) {
 		memcpy(dl_port->attrs.switch_id.id, ppid.id, ppid.id_len);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 815538ba754f..f9085b8dc20b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -2062,6 +2062,35 @@ int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs,
 					    sf_base_id);
 }
 
+u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+
+	if (!mlx5_esw_allowed(esw))
+		return 0;
+
+	return esw->esw_funcs.hpf_host_number;
+}
+
+static int mlx5_esw_hpf_host_number_init(struct mlx5_eswitch *esw)
+{
+	struct mlx5_esw_pf_info host_pf_info;
+	const u32 *query_host_out;
+
+	if (!mlx5_core_is_ecpf_esw_manager(esw->dev))
+		return 0;
+
+	query_host_out = mlx5_esw_query_functions(esw->dev);
+	if (IS_ERR(query_host_out))
+		return PTR_ERR(query_host_out);
+
+	/* Mark non local controller with non zero controller number. */
+	host_pf_info = mlx5_esw_get_host_pf_info(esw->dev, query_host_out);
+	esw->esw_funcs.hpf_host_number = host_pf_info.host_number;
+	kvfree(query_host_out);
+	return 0;
+}
+
 int mlx5_esw_vport_alloc(struct mlx5_eswitch *esw, int index, u16 vport_num)
 {
 	struct mlx5_vport *vport;
@@ -2211,6 +2240,10 @@ static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 
 	xa_init(&esw->vports);
 
+	err = mlx5_esw_hpf_host_number_init(esw);
+	if (err)
+		goto err;
+
 	if (mlx5_esw_host_functions_enabled(dev)) {
 		err = mlx5_esw_vport_alloc(esw, idx, MLX5_VPORT_HOST_PF);
 		if (err)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 19419799a26d..abdb4c460b06 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -332,7 +332,6 @@ struct mlx5_esw_offload {
 	u64 num_block_mode;
 	enum devlink_eswitch_encap_mode encap;
 	struct ida vport_metadata_ida;
-	unsigned int host_number; /* ECPF supports one external host */
 };
 
 /* E-Switch MC FDB table hash node */
@@ -359,6 +358,7 @@ struct mlx5_esw_functions {
 	bool			host_funcs_disabled;
 	u16			num_vfs;
 	u16			num_ec_vfs;
+	u16			hpf_host_number;
 	struct mlx5_esw_spf	*spfs;
 	int			num_spfs;
 };
@@ -879,6 +879,8 @@ struct devlink_port *mlx5_esw_offloads_devlink_port(struct mlx5_eswitch *esw, u1
 
 int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *sf_base_id);
 
+u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev);
+
 int mlx5_esw_vport_vhca_id_map(struct mlx5_eswitch *esw,
 			       struct mlx5_vport *vport);
 void mlx5_esw_vport_vhca_id_unmap(struct mlx5_eswitch *esw,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index b06b10d443bd..f17db51abe2d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3819,25 +3819,6 @@ int mlx5_esw_funcs_changed_handler(struct notifier_block *nb,
 	return NOTIFY_OK;
 }
 
-static int mlx5_esw_host_number_init(struct mlx5_eswitch *esw)
-{
-	struct mlx5_esw_pf_info host_pf_info;
-	const u32 *query_host_out;
-
-	if (!mlx5_core_is_ecpf_esw_manager(esw->dev))
-		return 0;
-
-	query_host_out = mlx5_esw_query_functions(esw->dev);
-	if (IS_ERR(query_host_out))
-		return PTR_ERR(query_host_out);
-
-	/* Mark non local controller with non zero controller number. */
-	host_pf_info = mlx5_esw_get_host_pf_info(esw->dev, query_host_out);
-	esw->offloads.host_number = host_pf_info.host_number;
-	kvfree(query_host_out);
-	return 0;
-}
-
 bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 controller)
 {
 	/* Local controller is always valid */
@@ -3848,7 +3829,7 @@ bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 cont
 		return false;
 
 	/* External host number starts with zero in device */
-	return (controller == esw->offloads.host_number + 1);
+	return (controller == mlx5_esw_get_hpf_host_number(esw->dev) + 1);
 }
 
 int esw_offloads_enable(struct mlx5_eswitch *esw)
@@ -3867,10 +3848,6 @@ int esw_offloads_enable(struct mlx5_eswitch *esw)
 	if (err)
 		goto err_roce;
 
-	err = mlx5_esw_host_number_init(esw);
-	if (err)
-		goto err_metadata;
-
 	err = esw_offloads_metadata_init(esw);
 	if (err)
 		goto err_metadata;
-- 
2.44.0

[PATCH net-next 04/12] net/mlx5: Initialize satellite PF SF vports

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Extend satellite PF (SPF) initialization to allocate SF vports for each
SPF. For each discovered SPF, query its SF capabilities, allocate SF
vports, and store the host_number for controller identification.

Add accessor APIs mlx5_esw_get_num_spfs(),
mlx5_esw_spf_get_host_number(), mlx5_esw_sf_max_spf_functions(), and
mlx5_esw_has_spf_sfs() for use by the SF hardware table in a subsequent
patch. Also extend mlx5_esw_offloads_controller_valid() to accept SPF
controllers in addition to the host PF controller.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 81 ++++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  8 ++
 .../mellanox/mlx5/core/eswitch_offloads.c     | 13 ++-
 3 files changed, 97 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index f9085b8dc20b..42cdb4309258 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -2062,6 +2062,51 @@ int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs,
 					    sf_base_id);
 }
 
+int mlx5_esw_sf_max_spf_functions(struct mlx5_core_dev *dev, int spf_idx,
+				  u16 *max_sfs, u16 *sf_base_id)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+	u16 vport_num;
+
+	if (!mlx5_esw_allowed(esw)) {
+		*max_sfs = 0;
+		return 0;
+	}
+
+	if (spf_idx >= esw->esw_funcs.num_spfs)
+		return -EINVAL;
+
+	vport_num = esw->esw_funcs.spfs[spf_idx].vport_num;
+	return mlx5_esw_sf_max_pf_functions(dev, vport_num, max_sfs,
+					    sf_base_id);
+}
+
+int mlx5_esw_get_num_spfs(struct mlx5_core_dev *dev)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+
+	if (!mlx5_esw_allowed(esw))
+		return 0;
+
+	return esw->esw_funcs.num_spfs;
+}
+
+int mlx5_esw_spf_get_host_number(struct mlx5_core_dev *dev, int spf_idx,
+				 u16 *host_number)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+
+	if (!mlx5_esw_allowed(esw))
+		return -EPERM;
+
+	if (spf_idx >= esw->esw_funcs.num_spfs)
+		return -EINVAL;
+
+	*host_number = esw->esw_funcs.spfs[spf_idx].host_number;
+
+	return 0;
+}
+
 u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev)
 {
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
@@ -2072,6 +2117,16 @@ u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev)
 	return esw->esw_funcs.hpf_host_number;
 }
 
+bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+
+	if (!mlx5_esw_allowed(esw))
+		return false;
+
+	return esw->esw_funcs.has_spf_sfs;
+}
+
 static int mlx5_esw_hpf_host_number_init(struct mlx5_eswitch *esw)
 {
 	struct mlx5_esw_pf_info host_pf_info;
@@ -2198,6 +2253,8 @@ static int mlx5_esw_spfs_init(struct mlx5_eswitch *esw)
 
 		esw_funcs->spfs[esw_funcs->num_spfs].vport_num = vport_num;
 		esw_funcs->spfs[esw_funcs->num_spfs].vhca_id = vhca_id;
+		esw_funcs->spfs[esw_funcs->num_spfs].host_number =
+			MLX5_GET(network_function_params, entry, host_number);
 		esw_funcs->num_spfs++;
 
 		entry += MLX5_UN_SZ_BYTES(net_function_params);
@@ -2224,6 +2281,7 @@ static void mlx5_esw_vports_cleanup(struct mlx5_eswitch *esw)
 	unsigned long i;
 
 	mlx5_esw_spfs_cleanup(esw);
+	esw->esw_funcs.has_spf_sfs = false;
 	mlx5_esw_for_each_vport(esw, i, vport)
 		mlx5_esw_vport_free(esw, vport);
 	xa_destroy(&esw->vports);
@@ -2232,8 +2290,7 @@ static void mlx5_esw_vports_cleanup(struct mlx5_eswitch *esw)
 static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 {
 	struct mlx5_core_dev *dev = esw->dev;
-	u16 max_host_pf_sfs;
-	u16 base_sf_num;
+	u16 max_sfs, base_sf_num;
 	int idx = 0;
 	int err;
 	int i;
@@ -2270,10 +2327,10 @@ static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 		idx++;
 	}
 
-	err = mlx5_esw_sf_max_hpf_functions(dev, &max_host_pf_sfs, &base_sf_num);
+	err = mlx5_esw_sf_max_hpf_functions(dev, &max_sfs, &base_sf_num);
 	if (err)
 		goto err;
-	for (i = 0; i < max_host_pf_sfs; i++) {
+	for (i = 0; i < max_sfs; i++) {
 		err = mlx5_esw_vport_alloc(esw, idx, base_sf_num + i);
 		if (err)
 			goto err;
@@ -2295,6 +2352,22 @@ static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 			goto err;
 		vport = mlx5_eswitch_get_vport(esw, vport_num);
 		vport->vhca_id = esw->esw_funcs.spfs[i].vhca_id;
+
+		err = mlx5_esw_sf_max_spf_functions(dev, i,
+						    &max_sfs, &base_sf_num);
+		if (err)
+			goto err;
+		if (max_sfs)
+			esw->esw_funcs.has_spf_sfs = true;
+		for (int j = 0; j < max_sfs; j++) {
+			err = mlx5_esw_vport_alloc(esw, idx,
+						   base_sf_num + j);
+			if (err)
+				goto err;
+			xa_set_mark(&esw->vports, base_sf_num + j,
+				    MLX5_ESW_VPT_SF);
+			idx++;
+		}
 	}
 
 	if (mlx5_core_ec_sriov_enabled(esw->dev)) {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index abdb4c460b06..88041dd8a39d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -351,6 +351,7 @@ struct mlx5_host_work {
 struct mlx5_esw_spf {
 	u16 vport_num;
 	u16 vhca_id;
+	u16 host_number;
 };
 
 struct mlx5_esw_functions {
@@ -359,6 +360,7 @@ struct mlx5_esw_functions {
 	u16			num_vfs;
 	u16			num_ec_vfs;
 	u16			hpf_host_number;
+	bool			has_spf_sfs;
 	struct mlx5_esw_spf	*spfs;
 	int			num_spfs;
 };
@@ -878,8 +880,14 @@ void mlx5_esw_offloads_devlink_port_unregister(struct mlx5_vport *vport);
 struct devlink_port *mlx5_esw_offloads_devlink_port(struct mlx5_eswitch *esw, u16 vport_num);
 
 int mlx5_esw_sf_max_hpf_functions(struct mlx5_core_dev *dev, u16 *max_sfs, u16 *sf_base_id);
+int mlx5_esw_sf_max_spf_functions(struct mlx5_core_dev *dev, int spf_idx,
+				  u16 *max_sfs, u16 *sf_base_id);
 
+int mlx5_esw_get_num_spfs(struct mlx5_core_dev *dev);
+int mlx5_esw_spf_get_host_number(struct mlx5_core_dev *dev, int spf_idx,
+				 u16 *host_number);
 u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev);
+bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev);
 
 int mlx5_esw_vport_vhca_id_map(struct mlx5_eswitch *esw,
 			       struct mlx5_vport *vport);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index f17db51abe2d..c229a96a111f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -3821,6 +3821,9 @@ int mlx5_esw_funcs_changed_handler(struct notifier_block *nb,
 
 bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 controller)
 {
+	const struct mlx5_esw_functions *esw_funcs;
+	int i;
+
 	/* Local controller is always valid */
 	if (controller == 0)
 		return true;
@@ -3829,7 +3832,15 @@ bool mlx5_esw_offloads_controller_valid(const struct mlx5_eswitch *esw, u32 cont
 		return false;
 
 	/* External host number starts with zero in device */
-	return (controller == mlx5_esw_get_hpf_host_number(esw->dev) + 1);
+	if (controller == mlx5_esw_get_hpf_host_number(esw->dev) + 1)
+		return true;
+
+	esw_funcs = &esw->esw_funcs;
+	for (i = 0; i < esw_funcs->num_spfs; i++) {
+		if (controller == esw_funcs->spfs[i].host_number + 1)
+			return true;
+	}
+	return false;
 }
 
 int esw_offloads_enable(struct mlx5_eswitch *esw)
-- 
2.44.0

[PATCH net-next 05/12] net/mlx5: Support SPF SFs in SF hardware table

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Convert the SF hardware table from a fixed-size hwc array to a
dynamically allocated one, supporting satellite PF (SPF) SFs alongside
local and external host SFs. Initialize hwc entries for each SPF using
its host_number as controller. Rename MLX5_SF_HWC_EXTERNAL to
MLX5_SF_HWC_EXT_HOST and add MLX5_SF_HWC_FIRST_SPF for clarity.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../ethernet/mellanox/mlx5/core/sf/hw_table.c | 89 ++++++++++++++-----
 1 file changed, 69 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
index 049dfd431618..0bc9146a3598 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c
@@ -21,25 +21,33 @@ struct mlx5_sf_hwc_table {
 	struct mlx5_sf_hw *sfs;
 	int max_fn;
 	u16 start_fn_id;
+	u32 controller;
 };
 
-enum mlx5_sf_hwc_index {
+enum {
 	MLX5_SF_HWC_LOCAL,
-	MLX5_SF_HWC_EXTERNAL,
-	MLX5_SF_HWC_MAX,
+	MLX5_SF_HWC_EXT_HOST,
+	MLX5_SF_HWC_FIRST_SPF,
 };
 
 struct mlx5_sf_hw_table {
 	struct mutex table_lock; /* Serializes sf deletion and vhca state change handler. */
-	struct mlx5_sf_hwc_table hwc[MLX5_SF_HWC_MAX];
+	struct mlx5_sf_hwc_table *hwc;
+	int num_hwc;
 };
 
 static struct mlx5_sf_hwc_table *
 mlx5_sf_controller_to_hwc(struct mlx5_core_dev *dev, u32 controller)
 {
-	int idx = !!controller;
+	struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table;
+	int i;
+
+	for (i = MLX5_SF_HWC_FIRST_SPF; i < table->num_hwc; i++) {
+		if (table->hwc[i].controller == controller)
+			return &table->hwc[i];
+	}
 
-	return &dev->priv.sf_hw_table->hwc[idx];
+	return &table->hwc[!!controller];
 }
 
 u16 mlx5_sf_sw_to_hw_id(struct mlx5_core_dev *dev, u32 controller, u16 sw_id)
@@ -60,7 +68,7 @@ mlx5_sf_table_fn_to_hwc(struct mlx5_sf_hw_table *table, u16 fn_id)
 {
 	int i;
 
-	for (i = 0; i < ARRAY_SIZE(table->hwc); i++) {
+	for (i = 0; i < table->num_hwc; i++) {
 		if (table->hwc[i].max_fn &&
 		    fn_id >= table->hwc[i].start_fn_id &&
 		    fn_id < (table->hwc[i].start_fn_id + table->hwc[i].max_fn))
@@ -221,9 +229,10 @@ static void mlx5_sf_hw_table_hwc_dealloc_all(struct mlx5_core_dev *dev,
 static void mlx5_sf_hw_table_dealloc_all(struct mlx5_core_dev *dev,
 					 struct mlx5_sf_hw_table *table)
 {
-	mlx5_sf_hw_table_hwc_dealloc_all(dev,
-					 &table->hwc[MLX5_SF_HWC_EXTERNAL]);
-	mlx5_sf_hw_table_hwc_dealloc_all(dev, &table->hwc[MLX5_SF_HWC_LOCAL]);
+	int i;
+
+	for (i = 0; i < table->num_hwc; i++)
+		mlx5_sf_hw_table_hwc_dealloc_all(dev, &table->hwc[i]);
 }
 
 static int mlx5_sf_hw_table_hwc_init(struct mlx5_sf_hwc_table *hwc, u16 max_fn, u16 base_id)
@@ -277,11 +286,13 @@ static int mlx5_sf_hw_table_res_register(struct mlx5_core_dev *dev, u16 max_fn,
 int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev)
 {
 	struct mlx5_sf_hw_table *table;
+	int num_spfs, num_hwc;
 	u16 max_ext_fn = 0;
 	u16 ext_base_id = 0;
 	u16 base_id;
 	u16 max_fn;
 	int err;
+	int i;
 
 	if (!mlx5_vhca_event_supported(dev))
 		return 0;
@@ -295,7 +306,7 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev)
 	if (mlx5_sf_hw_table_res_register(dev, max_fn, max_ext_fn))
 		mlx5_core_dbg(dev, "failed to register max SFs resources");
 
-	if (!max_fn && !max_ext_fn)
+	if (!max_fn && !max_ext_fn && !mlx5_esw_has_spf_sfs(dev))
 		return 0;
 
 	table = kzalloc_obj(*table);
@@ -304,26 +315,62 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev)
 		goto alloc_err;
 	}
 
+	num_spfs = mlx5_esw_get_num_spfs(dev);
+	num_hwc = MLX5_SF_HWC_FIRST_SPF + num_spfs;
+	table->hwc = kcalloc(num_hwc, sizeof(*table->hwc), GFP_KERNEL);
+	if (!table->hwc) {
+		err = -ENOMEM;
+		goto hwc_alloc_err;
+	}
+	table->num_hwc = num_hwc;
+
 	mutex_init(&table->table_lock);
 	dev->priv.sf_hw_table = table;
 
+	table->hwc[MLX5_SF_HWC_LOCAL].controller = 0;
 	base_id = mlx5_sf_start_function_id(dev);
 	err = mlx5_sf_hw_table_hwc_init(&table->hwc[MLX5_SF_HWC_LOCAL], max_fn, base_id);
 	if (err)
-		goto table_err;
+		goto hwc_init_err;
 
-	err = mlx5_sf_hw_table_hwc_init(&table->hwc[MLX5_SF_HWC_EXTERNAL],
+	table->hwc[MLX5_SF_HWC_EXT_HOST].controller =
+		mlx5_esw_get_hpf_host_number(dev) + 1;
+	err = mlx5_sf_hw_table_hwc_init(&table->hwc[MLX5_SF_HWC_EXT_HOST],
 					max_ext_fn, ext_base_id);
 	if (err)
-		goto ext_err;
+		goto hwc_init_err;
+
+	for (i = 0; i < num_spfs; i++) {
+		u16 spf_max_sfs, spf_base_id, host_number;
+		int hwc_idx = MLX5_SF_HWC_FIRST_SPF + i;
+
+		err = mlx5_esw_spf_get_host_number(dev, i, &host_number);
+		if (err)
+			goto hwc_init_err;
+
+		err = mlx5_esw_sf_max_spf_functions(dev, i, &spf_max_sfs,
+						    &spf_base_id);
+		if (err)
+			goto hwc_init_err;
 
-	mlx5_core_dbg(dev, "SF HW table: max sfs = %d, ext sfs = %d\n", max_fn, max_ext_fn);
+		table->hwc[hwc_idx].controller = host_number + 1;
+		err = mlx5_sf_hw_table_hwc_init(&table->hwc[hwc_idx],
+						spf_max_sfs, spf_base_id);
+		if (err)
+			goto hwc_init_err;
+	}
+
+	mlx5_core_dbg(dev, "SF HW table: max sfs = %d, ext sfs = %d, num spfs = %d\n",
+		      max_fn, max_ext_fn, num_spfs);
 	return 0;
 
-ext_err:
-	mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]);
-table_err:
+hwc_init_err:
+	dev->priv.sf_hw_table = NULL;
+	for (i = 0; i < num_hwc; i++)
+		mlx5_sf_hw_table_hwc_cleanup(&table->hwc[i]);
 	mutex_destroy(&table->table_lock);
+	kfree(table->hwc);
+hwc_alloc_err:
 	kfree(table);
 alloc_err:
 	mlx5_sf_hw_table_res_unregister(dev);
@@ -333,13 +380,15 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev)
 void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev)
 {
 	struct mlx5_sf_hw_table *table = dev->priv.sf_hw_table;
+	int i;
 
 	if (!table)
 		goto res_unregister;
 
-	mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_EXTERNAL]);
-	mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]);
+	for (i = 0; i < table->num_hwc; i++)
+		mlx5_sf_hw_table_hwc_cleanup(&table->hwc[i]);
 	mutex_destroy(&table->table_lock);
+	kfree(table->hwc);
 	kfree(table);
 	dev->priv.sf_hw_table = NULL;
 res_unregister:
-- 
2.44.0

[PATCH net-next 06/12] net/mlx5: Expose PF number from query_esw_functions

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Extract pci_device_function from the query_esw_functions output for both
the host PF and satellite PFs, storing it alongside the existing
host_number field.

Add mlx5_esw_get_hpf_pf_num() helper that returns the host PF's actual
PCI device function when the new query format is supported, falling back
to PCI_FUNC(dev->pdev->devfn) for older firmware. Use it in devlink port
attribute setup so that host PF and VF devlink ports report the correct
PF number rather than the ECPF's own PCI function number.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/devlink_port.c     |  4 ++++
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 22 ++++++++++++++++---
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  4 ++++
 3 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index e723f05cd4d3..d5f0101aa966 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -37,6 +37,8 @@ static void mlx5_esw_offloads_pf_vf_devlink_port_attrs_set(struct mlx5_eswitch *
 		controller_num = mlx5_esw_get_hpf_host_number(dev) + 1;
 
 	if (vport_num == MLX5_VPORT_HOST_PF) {
+		if (external)
+			pfnum = mlx5_esw_get_hpf_pf_num(dev);
 		memcpy(dl_port->attrs.switch_id.id, ppid.id, ppid.id_len);
 		dl_port->attrs.switch_id.id_len = ppid.id_len;
 		devlink_port_attrs_pci_pf_set(dl_port, controller_num, pfnum, external);
@@ -49,6 +51,8 @@ static void mlx5_esw_offloads_pf_vf_devlink_port_attrs_set(struct mlx5_eswitch *
 		if (vport->adjacent) {
 			func_id = vport->adj_info.function_id;
 			pfnum = vport->adj_info.parent_pci_devfn;
+		} else if (external) {
+			pfnum = mlx5_esw_get_hpf_pf_num(dev);
 		}
 
 		devlink_port_attrs_pci_vf_set(dl_port, controller_num, pfnum,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 42cdb4309258..8e2ac759d1f3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1157,6 +1157,8 @@ mlx5_esw_host_pf_from_net_func_params(const u8 *entry, int num_entries)
 					      entry, pci_total_vfs),
 			.host_number = MLX5_GET(network_function_params,
 						entry, host_number),
+			.pf_num = MLX5_GET(network_function_params, entry,
+					   pci_device_function),
 		};
 	}
 
@@ -2103,7 +2105,6 @@ int mlx5_esw_spf_get_host_number(struct mlx5_core_dev *dev, int spf_idx,
 		return -EINVAL;
 
 	*host_number = esw->esw_funcs.spfs[spf_idx].host_number;
-
 	return 0;
 }
 
@@ -2117,6 +2118,17 @@ u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev)
 	return esw->esw_funcs.hpf_host_number;
 }
 
+u16 mlx5_esw_get_hpf_pf_num(struct mlx5_core_dev *dev)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+
+	if (mlx5_core_is_ecpf_esw_manager(dev) &&
+	    MLX5_CAP_GEN(dev, query_host_net_function_v1))
+		return esw->esw_funcs.hpf_pf_num;
+
+	return PCI_FUNC(dev->pdev->devfn);
+}
+
 bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev)
 {
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
@@ -2127,7 +2139,7 @@ bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev)
 	return esw->esw_funcs.has_spf_sfs;
 }
 
-static int mlx5_esw_hpf_host_number_init(struct mlx5_eswitch *esw)
+static int mlx5_esw_hpf_info_init(struct mlx5_eswitch *esw)
 {
 	struct mlx5_esw_pf_info host_pf_info;
 	const u32 *query_host_out;
@@ -2142,6 +2154,7 @@ static int mlx5_esw_hpf_host_number_init(struct mlx5_eswitch *esw)
 	/* Mark non local controller with non zero controller number. */
 	host_pf_info = mlx5_esw_get_host_pf_info(esw->dev, query_host_out);
 	esw->esw_funcs.hpf_host_number = host_pf_info.host_number;
+	esw->esw_funcs.hpf_pf_num = host_pf_info.pf_num;
 	kvfree(query_host_out);
 	return 0;
 }
@@ -2255,6 +2268,9 @@ static int mlx5_esw_spfs_init(struct mlx5_eswitch *esw)
 		esw_funcs->spfs[esw_funcs->num_spfs].vhca_id = vhca_id;
 		esw_funcs->spfs[esw_funcs->num_spfs].host_number =
 			MLX5_GET(network_function_params, entry, host_number);
+		esw_funcs->spfs[esw_funcs->num_spfs].pf_num =
+			MLX5_GET(network_function_params, entry,
+				 pci_device_function);
 		esw_funcs->num_spfs++;
 
 		entry += MLX5_UN_SZ_BYTES(net_function_params);
@@ -2297,7 +2313,7 @@ static int mlx5_esw_vports_init(struct mlx5_eswitch *esw)
 
 	xa_init(&esw->vports);
 
-	err = mlx5_esw_hpf_host_number_init(esw);
+	err = mlx5_esw_hpf_info_init(esw);
 	if (err)
 		goto err;
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 88041dd8a39d..03c7582d7b95 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -77,6 +77,7 @@ struct mlx5_esw_pf_info {
 	u16 num_of_vfs;
 	u16 total_vfs;
 	u16 host_number;
+	u16 pf_num;
 };
 
 #ifdef CONFIG_MLX5_ESWITCH
@@ -352,6 +353,7 @@ struct mlx5_esw_spf {
 	u16 vport_num;
 	u16 vhca_id;
 	u16 host_number;
+	u16 pf_num;
 };
 
 struct mlx5_esw_functions {
@@ -360,6 +362,7 @@ struct mlx5_esw_functions {
 	u16			num_vfs;
 	u16			num_ec_vfs;
 	u16			hpf_host_number;
+	u16			hpf_pf_num;
 	bool			has_spf_sfs;
 	struct mlx5_esw_spf	*spfs;
 	int			num_spfs;
@@ -887,6 +890,7 @@ int mlx5_esw_get_num_spfs(struct mlx5_core_dev *dev);
 int mlx5_esw_spf_get_host_number(struct mlx5_core_dev *dev, int spf_idx,
 				 u16 *host_number);
 u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev);
+u16 mlx5_esw_get_hpf_pf_num(struct mlx5_core_dev *dev);
 bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev);
 
 int mlx5_esw_vport_vhca_id_map(struct mlx5_eswitch *esw,
-- 
2.44.0

[PATCH net-next 07/12] net/mlx5: Map SF controller to pfnum for satellite PFs

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

SF devlink port creation and registration used the ECPF's PCI function
as pfnum. Extend this to support satellite PF controllers by introducing
mlx5_esw_sf_controller_to_pfnum() that maps a controller number to the
corresponding PF number, and use it in SF port attribute setup and SF
creation validation.

Reorder the checks in mlx5_devlink_sf_port_new() so that
mlx5_sf_table_supported() runs before attribute validation, since the
new helper requires the eswitch to be initialized.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/devlink_port.c       |  2 +-
 .../net/ethernet/mellanox/mlx5/core/eswitch.c   | 17 +++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/eswitch.h   |  1 +
 .../ethernet/mellanox/mlx5/core/sf/devlink.c    | 14 +++++++++-----
 4 files changed, 28 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index d5f0101aa966..fddb108bcbff 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -125,7 +125,7 @@ static void mlx5_esw_offloads_sf_devlink_port_attrs_set(struct mlx5_eswitch *esw
 	struct netdev_phys_item_id ppid = {};
 	u16 pfnum;
 
-	pfnum = PCI_FUNC(dev->pdev->devfn);
+	pfnum = mlx5_esw_sf_controller_to_pfnum(dev, controller);
 	mlx5_esw_get_port_parent_id(dev, &ppid);
 	memcpy(dl_port->attrs.switch_id.id, &ppid.id[0], ppid.id_len);
 	dl_port->attrs.switch_id.id_len = ppid.id_len;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 8e2ac759d1f3..f734f9364b2c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -2129,6 +2129,23 @@ u16 mlx5_esw_get_hpf_pf_num(struct mlx5_core_dev *dev)
 	return PCI_FUNC(dev->pdev->devfn);
 }
 
+u16 mlx5_esw_sf_controller_to_pfnum(struct mlx5_core_dev *dev, u32 controller)
+{
+	struct mlx5_eswitch *esw = dev->priv.eswitch;
+	struct mlx5_esw_functions *esw_funcs;
+	int i;
+
+	if (!controller)
+		return PCI_FUNC(dev->pdev->devfn);
+
+	esw_funcs = &esw->esw_funcs;
+	for (i = 0; i < esw_funcs->num_spfs; i++)
+		if (controller == esw_funcs->spfs[i].host_number + 1)
+			return esw_funcs->spfs[i].pf_num;
+
+	return mlx5_esw_get_hpf_pf_num(dev);
+}
+
 bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev)
 {
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 03c7582d7b95..f85be8e39953 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -891,6 +891,7 @@ int mlx5_esw_spf_get_host_number(struct mlx5_core_dev *dev, int spf_idx,
 				 u16 *host_number);
 u16 mlx5_esw_get_hpf_host_number(struct mlx5_core_dev *dev);
 u16 mlx5_esw_get_hpf_pf_num(struct mlx5_core_dev *dev);
+u16 mlx5_esw_sf_controller_to_pfnum(struct mlx5_core_dev *dev, u32 controller);
 bool mlx5_esw_has_spf_sfs(struct mlx5_core_dev *dev);
 
 int mlx5_esw_vport_vhca_id_map(struct mlx5_eswitch *esw,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
index 2fc69897e35b..b6cecbcc392d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c
@@ -265,6 +265,8 @@ static int
 mlx5_sf_new_check_attr(struct mlx5_core_dev *dev, const struct devlink_port_new_attrs *new_attr,
 		       struct netlink_ext_ack *extack)
 {
+	u32 controller;
+
 	if (new_attr->flavour != DEVLINK_PORT_FLAVOUR_PCI_SF) {
 		NL_SET_ERR_MSG_MOD(extack, "Driver supports only SF port addition");
 		return -EOPNOTSUPP;
@@ -284,7 +286,9 @@ mlx5_sf_new_check_attr(struct mlx5_core_dev *dev, const struct devlink_port_new_
 		NL_SET_ERR_MSG_MOD(extack, "External controller is unsupported");
 		return -EOPNOTSUPP;
 	}
-	if (new_attr->pfnum != PCI_FUNC(dev->pdev->devfn)) {
+	controller = new_attr->controller_valid ? new_attr->controller : 0;
+	if (new_attr->pfnum !=
+	    mlx5_esw_sf_controller_to_pfnum(dev, controller)) {
 		NL_SET_ERR_MSG_MOD(extack, "Invalid pfnum supplied");
 		return -EOPNOTSUPP;
 	}
@@ -306,10 +310,6 @@ int mlx5_devlink_sf_port_new(struct devlink *devlink,
 	struct mlx5_sf_table *table = dev->priv.sf_table;
 	int err;
 
-	err = mlx5_sf_new_check_attr(dev, new_attr, extack);
-	if (err)
-		return err;
-
 	if (!mlx5_sf_table_supported(dev)) {
 		NL_SET_ERR_MSG_MOD(extack, "SF ports are not supported.");
 		return -EOPNOTSUPP;
@@ -321,6 +321,10 @@ int mlx5_devlink_sf_port_new(struct devlink *devlink,
 		return -EOPNOTSUPP;
 	}
 
+	err = mlx5_sf_new_check_attr(dev, new_attr, extack);
+	if (err)
+		return err;
+
 	return mlx5_sf_add(dev, table, new_attr, extack, dl_port);
 }
 
-- 
2.44.0

[PATCH net-next 08/12] net/mlx5: Register devlink ports for satellite PFs

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Include satellite PFs in mlx5_eswitch_is_pf_vf_vport() so they receive
the standard PF/VF devlink port operations. Update
mlx5_esw_devlink_port_supported() and devlink port attribute setup to
register SPF devlink ports with controller number and PF number.

Add mlx5_esw_spf_vport_to_idx() to look up the SPF array index by vport
number, and mlx5_esw_is_spf_vport() boolean wrapper to identify
satellite PF vports.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/devlink_port.c     | 13 +++++++++++-
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 21 ++++++++++++++++++-
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  2 ++
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index fddb108bcbff..05d89769b917 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -16,7 +16,8 @@ static bool mlx5_esw_devlink_port_supported(struct mlx5_eswitch *esw, u16 vport_
 	return (mlx5_core_is_ecpf(esw->dev) &&
 		vport_num == MLX5_VPORT_HOST_PF) ||
 	       mlx5_eswitch_is_vf_vport(esw, vport_num) ||
-	       mlx5_core_is_ec_vf_vport(esw->dev, vport_num);
+	       mlx5_core_is_ec_vf_vport(esw->dev, vport_num) ||
+	       mlx5_esw_is_spf_vport(esw, vport_num);
 }
 
 static void mlx5_esw_offloads_pf_vf_devlink_port_attrs_set(struct mlx5_eswitch *esw,
@@ -64,6 +65,16 @@ static void mlx5_esw_offloads_pf_vf_devlink_port_attrs_set(struct mlx5_eswitch *
 		dl_port->attrs.switch_id.id_len = ppid.id_len;
 		devlink_port_attrs_pci_vf_set(dl_port, 0, pfnum,
 					      vport_num - base_vport, false);
+	} else if (mlx5_esw_is_spf_vport(esw, vport_num)) {
+		int spf_idx = mlx5_esw_spf_vport_to_idx(esw, vport_num);
+
+		controller_num = esw->esw_funcs.spfs[spf_idx].host_number + 1;
+		pfnum = esw->esw_funcs.spfs[spf_idx].pf_num;
+
+		memcpy(dl_port->attrs.switch_id.id, ppid.id, ppid.id_len);
+		dl_port->attrs.switch_id.id_len = ppid.id_len;
+		devlink_port_attrs_pci_pf_set(dl_port, controller_num, pfnum,
+					      true);
 	}
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index f734f9364b2c..8bee014140b8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -2651,10 +2651,29 @@ bool mlx5_eswitch_is_vf_vport(struct mlx5_eswitch *esw, u16 vport_num)
 	return mlx5_esw_check_port_type(esw, vport_num, MLX5_ESW_VPT_VF);
 }
 
+int mlx5_esw_spf_vport_to_idx(struct mlx5_eswitch *esw, u16 vport_num)
+{
+	struct mlx5_esw_functions *esw_funcs = &esw->esw_funcs;
+	int i;
+
+	for (i = 0; i < esw_funcs->num_spfs; i++) {
+		if (esw_funcs->spfs[i].vport_num == vport_num)
+			return i;
+	}
+
+	return -ENOENT;
+}
+
+bool mlx5_esw_is_spf_vport(struct mlx5_eswitch *esw, u16 vport_num)
+{
+	return mlx5_esw_spf_vport_to_idx(esw, vport_num) >= 0;
+}
+
 bool mlx5_eswitch_is_pf_vf_vport(struct mlx5_eswitch *esw, u16 vport_num)
 {
 	return vport_num == MLX5_VPORT_HOST_PF ||
-		mlx5_eswitch_is_vf_vport(esw, vport_num);
+		mlx5_eswitch_is_vf_vport(esw, vport_num) ||
+		mlx5_esw_is_spf_vport(esw, vport_num);
 }
 
 bool mlx5_esw_is_sf_vport(struct mlx5_eswitch *esw, u16 vport_num)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index f85be8e39953..7da1a888aa7c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -798,6 +798,8 @@ struct mlx5_vport *__must_check
 mlx5_eswitch_get_vport(struct mlx5_eswitch *esw, u16 vport_num);
 
 bool mlx5_eswitch_is_vf_vport(struct mlx5_eswitch *esw, u16 vport_num);
+int mlx5_esw_spf_vport_to_idx(struct mlx5_eswitch *esw, u16 vport_num);
+bool mlx5_esw_is_spf_vport(struct mlx5_eswitch *esw, u16 vport_num);
 bool mlx5_eswitch_is_pf_vf_vport(struct mlx5_eswitch *esw, u16 vport_num);
 bool mlx5_esw_is_sf_vport(struct mlx5_eswitch *esw, u16 vport_num);
 
-- 
2.44.0

[PATCH net-next 09/12] net/mlx5: Register SF resource on satellite PF ports

[Tariq Toukan]

From: Or Har-Toov <ohartoov@nvidia.com>

Extend port-level resource registration to satellite PF vports.

Signed-off-by: Or Har-Toov <ohartoov@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../mellanox/mlx5/core/esw/devlink_port.c     | 31 +++++++++++++------
 1 file changed, 21 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
index 05d89769b917..6e50311faa27 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c
@@ -176,14 +176,28 @@ static const struct devlink_port_ops mlx5_esw_dl_sf_port_ops = {
 };
 
 static int mlx5_esw_devlink_port_res_register(struct mlx5_eswitch *esw,
-					      struct devlink_port *dl_port)
+					      struct devlink_port *dl_port,
+					      u16 vport_num)
 {
 	struct devlink_resource_size_params size_params;
 	struct mlx5_core_dev *dev = esw->dev;
 	u16 max_sfs, sf_base_id;
 	int err;
 
-	err = mlx5_esw_sf_max_hpf_functions(dev, &max_sfs, &sf_base_id);
+	if (vport_num != MLX5_VPORT_HOST_PF &&
+	    !mlx5_esw_is_spf_vport(esw, vport_num))
+		return 0;
+
+	if (vport_num == MLX5_VPORT_HOST_PF) {
+		err = mlx5_esw_sf_max_hpf_functions(dev, &max_sfs,
+						    &sf_base_id);
+	} else {
+		int spf_idx = mlx5_esw_spf_vport_to_idx(esw, vport_num);
+
+		err = mlx5_esw_sf_max_spf_functions(dev, spf_idx, &max_sfs,
+						    &sf_base_id);
+	}
+
 	if (err)
 		return err;
 
@@ -232,14 +246,11 @@ int mlx5_esw_offloads_devlink_port_register(struct mlx5_eswitch *esw, struct mlx
 	if (err)
 		goto rate_err;
 
-	if (vport_num == MLX5_VPORT_HOST_PF) {
-		err = mlx5_esw_devlink_port_res_register(esw,
-							 &dl_port->dl_port);
-		if (err)
-			mlx5_core_dbg(dev,
-				      "Failed to register port resources: %d\n",
-				       err);
-	}
+	err = mlx5_esw_devlink_port_res_register(esw, &dl_port->dl_port,
+						 vport_num);
+	if (err)
+		mlx5_core_dbg(dev, "Failed to register port resources: %d\n",
+			      err);
 
 	return 0;
 
-- 
2.44.0

[PATCH net-next 10/12] net/mlx5: Support state get/set for satellite PF ports

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Extend mlx5_devlink_pf_port_fn_state_get() to support satellite PF
vports by querying their vhca_state from the query_esw_functions output
using the vport's vhca_id.

Extend mlx5_devlink_pf_port_fn_state_set() to support satellite PFs by
using the generic mlx5_esw_pf_enable/disable_hca() functions.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch.c | 31 +++++++++++++++++--
 .../net/ethernet/mellanox/mlx5/core/eswitch.h |  4 +++
 .../mellanox/mlx5/core/eswitch_offloads.c     | 28 +++++++++++------
 3 files changed, 52 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 8bee014140b8..9f82fc4dbf43 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -1187,6 +1187,33 @@ mlx5_esw_get_host_pf_info(struct mlx5_core_dev *dev, const u32 *out)
 	return mlx5_esw_host_pf_from_host_params(entry);
 }
 
+bool mlx5_esw_get_spf_disabled(struct mlx5_core_dev *dev, const u32 *out,
+			       u16 vhca_id)
+{
+	int num_entries;
+	const u8 *entry;
+	int i;
+
+	num_entries = MLX5_GET(query_esw_functions_out, out, net_function_num);
+	entry = MLX5_ADDR_OF(query_esw_functions_out, out, net_function_params);
+
+	for (i = 0; i < num_entries; i++) {
+		u16 entry_vhca_id = MLX5_GET(network_function_params,
+					     entry, vhca_id);
+
+		if (entry_vhca_id == vhca_id) {
+			int state;
+
+			state = MLX5_GET(network_function_params, entry,
+					 vhca_state);
+			return state != MLX5_VHCA_STATE_IN_USE;
+		}
+		entry += MLX5_UN_SZ_BYTES(net_function_params);
+	}
+
+	return true;
+}
+
 static int mlx5_esw_host_functions_enabled_query(struct mlx5_eswitch *esw)
 {
 	struct mlx5_esw_pf_info host_pf_info;
@@ -1454,7 +1481,7 @@ static int mlx5_eswitch_load_ec_vf_vports(struct mlx5_eswitch *esw, u16 num_ec_v
 	return err;
 }
 
-static int mlx5_esw_pf_enable_hca(struct mlx5_core_dev *dev, u16 vport_num)
+int mlx5_esw_pf_enable_hca(struct mlx5_core_dev *dev, u16 vport_num)
 {
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
 	struct mlx5_vport *vport;
@@ -1480,7 +1507,7 @@ static int mlx5_esw_pf_enable_hca(struct mlx5_core_dev *dev, u16 vport_num)
 	return 0;
 }
 
-static int mlx5_esw_pf_disable_hca(struct mlx5_core_dev *dev, u16 vport_num)
+int mlx5_esw_pf_disable_hca(struct mlx5_core_dev *dev, u16 vport_num)
 {
 	struct mlx5_eswitch *esw = dev->priv.eswitch;
 	struct mlx5_vport *vport;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 7da1a888aa7c..1d8e2486d518 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -671,6 +671,10 @@ bool mlx5_esw_multipath_prereq(struct mlx5_core_dev *dev0,
 const u32 *mlx5_esw_query_functions(struct mlx5_core_dev *dev);
 struct mlx5_esw_pf_info mlx5_esw_get_host_pf_info(struct mlx5_core_dev *dev,
 						  const u32 *out);
+bool mlx5_esw_get_spf_disabled(struct mlx5_core_dev *dev, const u32 *out,
+			       u16 vhca_id);
+int mlx5_esw_pf_enable_hca(struct mlx5_core_dev *dev, u16 vport_num);
+int mlx5_esw_pf_disable_hca(struct mlx5_core_dev *dev, u16 vport_num);
 int mlx5_esw_host_pf_enable_hca(struct mlx5_core_dev *dev);
 int mlx5_esw_host_pf_disable_hca(struct mlx5_core_dev *dev);
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index c229a96a111f..59446c444570 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -4961,10 +4961,11 @@ int mlx5_devlink_pf_port_fn_state_get(struct devlink_port *port,
 				      struct netlink_ext_ack *extack)
 {
 	struct mlx5_vport *vport = mlx5_devlink_port_vport_get(port);
-	struct mlx5_esw_pf_info host_pf_info;
+	struct mlx5_eswitch *esw = vport->dev->priv.eswitch;
 	const u32 *query_out;
+	bool pf_disabled;
 
-	if (vport->vport != MLX5_VPORT_HOST_PF) {
+	if (mlx5_eswitch_is_vf_vport(esw, vport->vport)) {
 		NL_SET_ERR_MSG_MOD(extack, "State get is not supported for VF");
 		return -EOPNOTSUPP;
 	}
@@ -4976,11 +4977,19 @@ int mlx5_devlink_pf_port_fn_state_get(struct devlink_port *port,
 	if (IS_ERR(query_out))
 		return PTR_ERR(query_out);
 
-	host_pf_info = mlx5_esw_get_host_pf_info(vport->dev, query_out);
+	if (vport->vport == MLX5_VPORT_HOST_PF) {
+		struct mlx5_esw_pf_info host_pf_info;
+
+		host_pf_info = mlx5_esw_get_host_pf_info(vport->dev,
+							 query_out);
+		pf_disabled = host_pf_info.pf_disabled;
+	} else {
+		pf_disabled = mlx5_esw_get_spf_disabled(vport->dev, query_out,
+							vport->vhca_id);
+	}
 
-	*opstate = host_pf_info.pf_disabled ?
-			DEVLINK_PORT_FN_OPSTATE_DETACHED :
-			DEVLINK_PORT_FN_OPSTATE_ATTACHED;
+	*opstate = pf_disabled ? DEVLINK_PORT_FN_OPSTATE_DETACHED :
+				 DEVLINK_PORT_FN_OPSTATE_ATTACHED;
 
 	kvfree(query_out);
 	return 0;
@@ -4991,9 +5000,10 @@ int mlx5_devlink_pf_port_fn_state_set(struct devlink_port *port,
 				      struct netlink_ext_ack *extack)
 {
 	struct mlx5_vport *vport = mlx5_devlink_port_vport_get(port);
+	struct mlx5_eswitch *esw = vport->dev->priv.eswitch;
 	struct mlx5_core_dev *dev;
 
-	if (vport->vport != MLX5_VPORT_HOST_PF) {
+	if (mlx5_eswitch_is_vf_vport(esw, vport->vport)) {
 		NL_SET_ERR_MSG_MOD(extack, "State set is not supported for VF");
 		return -EOPNOTSUPP;
 	}
@@ -5002,9 +5012,9 @@ int mlx5_devlink_pf_port_fn_state_set(struct devlink_port *port,
 
 	switch (state) {
 	case DEVLINK_PORT_FN_STATE_ACTIVE:
-		return mlx5_esw_host_pf_enable_hca(dev);
+		return mlx5_esw_pf_enable_hca(dev, vport->vport);
 	case DEVLINK_PORT_FN_STATE_INACTIVE:
-		return mlx5_esw_host_pf_disable_hca(dev);
+		return mlx5_esw_pf_disable_hca(dev, vport->vport);
 	default:
 		return -EOPNOTSUPP;
 	}
-- 
2.44.0

[PATCH net-next 11/12] net/mlx5: Add FDB peer miss rules for satellite PFs

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Add satellite PF (SPF) vports to the FDB peer miss rules flow.
Introduce mlx5_esw_for_each_spf_vport() macro to iterate SPF vports.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/eswitch.h | 10 +++++++++
 .../mellanox/mlx5/core/eswitch_offloads.c     | 22 ++++++++++++++++++-
 2 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
index 1d8e2486d518..c8d6c94a4475 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.h
@@ -790,6 +790,16 @@ void mlx5e_tc_clean_fdb_peer_flows(struct mlx5_eswitch *esw);
 			  MLX5_CAP_GEN_2((esw->dev), ec_vf_vport_base) +\
 			  (last) - 1)
 
+/* SPF vport numbers are not contiguous, iterate via the spfs array
+ * and look up each vport in the xarray.
+ */
+#define mlx5_esw_for_each_spf_vport(esw, index, vport)			\
+	for ((index) = 0;						\
+	     (index) < (esw)->esw_funcs.num_spfs &&			\
+	     ((vport) = xa_load(&(esw)->vports,				\
+		(esw)->esw_funcs.spfs[(index)].vport_num));		\
+	     (index)++)
+
 #define mlx5_esw_for_each_rep(esw, i, rep) \
 	xa_for_each(&((esw)->offloads.vport_reps), i, rep)
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
index 59446c444570..355d27934fb4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c
@@ -1231,6 +1231,19 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
 		flows[peer_vport->index] = flow;
 	}
 
+	mlx5_esw_for_each_spf_vport(peer_esw, i, peer_vport) {
+		esw_set_peer_miss_rule_source_port(esw, peer_esw, spec,
+						   peer_vport->vport);
+
+		flow = mlx5_add_flow_rules(mlx5_eswitch_get_slow_fdb(esw),
+					   spec, &flow_act, &dest, 1);
+		if (IS_ERR(flow)) {
+			err = PTR_ERR(flow);
+			goto add_ecpf_flow_err;
+		}
+		flows[peer_vport->index] = flow;
+	}
+
 	if (mlx5_ecpf_vport_exists(peer_dev)) {
 		peer_vport = mlx5_eswitch_get_vport(peer_esw, MLX5_VPORT_ECPF);
 		MLX5_SET(fte_match_set_misc, misc, source_port, MLX5_VPORT_ECPF);
@@ -1299,7 +1312,11 @@ static int esw_add_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
 		mlx5_del_flow_rules(flows[peer_vport->index]);
 	}
 add_ecpf_flow_err:
-
+	mlx5_esw_for_each_spf_vport(peer_esw, i, peer_vport) {
+		if (!flows[peer_vport->index])
+			continue;
+		mlx5_del_flow_rules(flows[peer_vport->index]);
+	}
 	if (mlx5_core_is_ecpf_esw_manager(peer_dev) &&
 	    mlx5_esw_host_functions_enabled(peer_dev)) {
 		peer_vport = mlx5_eswitch_get_vport(peer_esw,
@@ -1343,6 +1360,9 @@ static void esw_del_fdb_peer_miss_rules(struct mlx5_eswitch *esw,
 		mlx5_del_flow_rules(flows[peer_vport->index]);
 	}
 
+	mlx5_esw_for_each_spf_vport(peer_esw, i, peer_vport)
+		mlx5_del_flow_rules(flows[peer_vport->index]);
+
 	if (mlx5_core_is_ecpf_esw_manager(peer_dev) &&
 	    mlx5_esw_host_functions_enabled(peer_dev)) {
 		peer_vport = mlx5_eswitch_get_vport(peer_esw,
-- 
2.44.0

[PATCH net-next 12/12] net/mlx5: Add SPF function type for page management

[Tariq Toukan]

From: Moshe Shemesh <moshe@nvidia.com>

Add MLX5_SPF to enum mlx5_func_type so SPFs get their own page counter,
and add the corresponding WARN check at page cleanup. Wait for SPF pages
to be reclaimed during ECPF teardown, alongside the existing host PF and
VF page waits.

SPF page requests are always identified by vhca_id, so the legacy
func_id_to_type() path is not reached for satellite PFs.

Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/debugfs.c   | 3 +++
 drivers/net/ethernet/mellanox/mlx5/core/ecpf.c      | 5 +++++
 drivers/net/ethernet/mellanox/mlx5/core/eswitch.c   | 2 ++
 drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c | 3 +++
 include/linux/mlx5/driver.h                         | 1 +
 5 files changed, 14 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/debugfs.c b/drivers/net/ethernet/mellanox/mlx5/core/debugfs.c
index 6347957fefcb..30be2b631e7c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/debugfs.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/debugfs.c
@@ -316,6 +316,8 @@ void mlx5_pages_by_func_type_debugfs_init(struct mlx5_core_dev *dev)
 			   &dev->priv.page_counters[MLX5_SF]);
 	debugfs_create_u32("fw_pages_host_pf", 0400, pages,
 			   &dev->priv.page_counters[MLX5_HOST_PF]);
+	debugfs_create_u32("fw_pages_spfs", 0400, pages,
+			   &dev->priv.page_counters[MLX5_SPF]);
 }
 
 void mlx5_pages_by_func_type_debugfs_cleanup(struct mlx5_core_dev *dev)
@@ -329,6 +331,7 @@ void mlx5_pages_by_func_type_debugfs_cleanup(struct mlx5_core_dev *dev)
 	debugfs_lookup_and_remove("fw_pages_ec_vfs", pages);
 	debugfs_lookup_and_remove("fw_pages_sfs", pages);
 	debugfs_lookup_and_remove("fw_pages_host_pf", pages);
+	debugfs_lookup_and_remove("fw_pages_spfs", pages);
 }
 
 static u64 qp_read_field(struct mlx5_core_dev *dev, struct mlx5_core_qp *qp,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
index 350c47d3643b..9839f1a58640 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ecpf.c
@@ -102,6 +102,11 @@ void mlx5_ec_cleanup(struct mlx5_core_dev *dev)
 	if (err)
 		mlx5_core_warn(dev, "Timeout reclaiming external host PF pages err(%d)\n", err);
 
+	err = mlx5_wait_for_pages(dev, &dev->priv.page_counters[MLX5_SPF]);
+	if (err)
+		mlx5_core_warn(dev, "Timeout reclaiming SPF pages err(%d)\n",
+			       err);
+
 	err = mlx5_wait_for_pages(dev, &dev->priv.page_counters[MLX5_VF]);
 	if (err)
 		mlx5_core_warn(dev, "Timeout reclaiming external host VFs pages err(%d)\n", err);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
index 9f82fc4dbf43..5df3ec641ae3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c
@@ -863,6 +863,8 @@ esw_vport_to_func_type(struct mlx5_eswitch *esw, struct mlx5_vport *vport)
 		return MLX5_SF;
 	if (xa_get_mark(&esw->vports, vport_num, MLX5_ESW_VPT_VF))
 		return MLX5_VF;
+	if (mlx5_esw_is_spf_vport(esw, vport_num))
+		return MLX5_SPF;
 	return MLX5_EC_VF;
 }
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
index ce2f7fa9bd48..7fef3a7fee6e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c
@@ -885,6 +885,9 @@ int mlx5_reclaim_startup_pages(struct mlx5_core_dev *dev)
 	WARN(dev->priv.page_counters[MLX5_HOST_PF],
 	     "External host PF FW pages counter is %d after reclaiming all pages\n",
 	     dev->priv.page_counters[MLX5_HOST_PF]);
+	WARN(dev->priv.page_counters[MLX5_SPF],
+	     "SPFs FW pages counter is %d after reclaiming all pages\n",
+	     dev->priv.page_counters[MLX5_SPF]);
 	WARN(dev->priv.page_counters[MLX5_EC_VF],
 	     "EC VFs FW pages counter is %d after reclaiming all pages\n",
 	     dev->priv.page_counters[MLX5_EC_VF]);
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 9a4bb25d8e0a..b1871c0821d0 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -557,6 +557,7 @@ enum mlx5_func_type {
 	MLX5_VF,
 	MLX5_SF,
 	MLX5_HOST_PF,
+	MLX5_SPF,
 	MLX5_EC_VF,
 	MLX5_FUNC_TYPE_NUM,
 	MLX5_FUNC_TYPE_NONE = MLX5_FUNC_TYPE_NUM,
-- 
2.44.0

Re: [PATCH net-next 00/12] net/mlx5: Add satellite PF support

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Thu, 21 May 2026 14:08:31 +0300 you wrote:
> Hi,
> 
> See detailed feature description by Moshe below.
> 
> Regards,
> Tariq
> 
> [...]

Here is the summary with links:
  - [net-next,01/12] net/mlx5: Add satellite PF vport support
    https://git.kernel.org/netdev/net-next/c/ed9671b8bd4f
  - [net-next,02/12] net/mlx5: Introduce generic helper for PF SFs info
    https://git.kernel.org/netdev/net-next/c/69978da9bb71
  - [net-next,03/12] net/mlx5: Initialize host PF host number earlier
    https://git.kernel.org/netdev/net-next/c/171b7fb59f8d
  - [net-next,04/12] net/mlx5: Initialize satellite PF SF vports
    https://git.kernel.org/netdev/net-next/c/7aed78522df2
  - [net-next,05/12] net/mlx5: Support SPF SFs in SF hardware table
    https://git.kernel.org/netdev/net-next/c/beca1cd919e0
  - [net-next,06/12] net/mlx5: Expose PF number from query_esw_functions
    https://git.kernel.org/netdev/net-next/c/0b43d2b76cc2
  - [net-next,07/12] net/mlx5: Map SF controller to pfnum for satellite PFs
    https://git.kernel.org/netdev/net-next/c/90a6aabb74a4
  - [net-next,08/12] net/mlx5: Register devlink ports for satellite PFs
    https://git.kernel.org/netdev/net-next/c/e020a7067295
  - [net-next,09/12] net/mlx5: Register SF resource on satellite PF ports
    https://git.kernel.org/netdev/net-next/c/ac338d8011c0
  - [net-next,10/12] net/mlx5: Support state get/set for satellite PF ports
    https://git.kernel.org/netdev/net-next/c/425ac0e7a6a0
  - [net-next,11/12] net/mlx5: Add FDB peer miss rules for satellite PFs
    https://git.kernel.org/netdev/net-next/c/652be53b37d8
  - [net-next,12/12] net/mlx5: Add SPF function type for page management
    https://git.kernel.org/netdev/net-next/c/ea0dada7194e

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html