[PATCH rdma-next 00/23] mlx5: Adversarial mode for confidential computing guests

[PATCH rdma-next 00/23] mlx5: Adversarial mode for confidential computing guests

[Yishai Hadas]

In a confidential computing (CC) guest, the hypervisor and any device
assigned to the guest are outside its trust boundary. The mlx5 drivers,
like most drivers, trust the NIC: they use the capabilities, events and
completions it reports to size buffers, index arrays and pick the memory
they work on. A malicious or compromised NIC can turn that trust into
memory corruption or information leaks inside the guest.

This series adds an adversarial mode, in which the mlx5 drivers treat
the NIC as untrusted. It is enabled automatically in CC guests, based on
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT), with no administrator
configuration. Outside a CC guest the behaviour is unchanged.

The approach is to keep the attack surface small rather than harden
every path. In adversarial mode only the uverbs path is exposed: kernel
ULPs don't bind to the device and unneeded features are not offered.
The data path is owned by rdma-core, which must be aware of the mode.
When it opens a context, rdma-core declares that it supports adversarial
mode, and the kernel reports back that the device is in it, so rdma-core
validates what the NIC writes to the user queues. An rdma-core that
doesn't declare support is refused a context, so it can't use the device
while trusting the NIC. What remains in the kernel validates what it
gets from the NIC. A NIC can always stop working, so denial of service
is out of scope.

We believe that this series provides comprehensive security against an
adversarial device within rdma. Many teams at NVIDIA have been involved
in evaluating the various attack vectors and developing a comprehensive
mitigation, including AI analysis of the driver, fuzzing, and first
principles architecture review. It chooses to block and disable
functions that may open risk, and results in a reduction in driver
functionality rather than leave open potential security holes.

The series is organized as follows:

mlx5_core (patches 1-4)
- Derive adversarial mode from CC guest memory encryption.
- Filter the FW capabilities: start from none, pass through an allow
  list, and validate or clamp the numeric ones, failing the load on
  unexpected values.
- Add helpers to read log-encoded capabilities without an undefined
  shift.
- Add a forced health recovery, for faults the health sensors can't see,
  such as a NIC that keeps answering while forging completions.

Attack surface reduction (patches 5-8)
- Check the FW capability for every QP transport type, and accept the DC
  vendor flags only on IB_QPT_DRIVER.
- Keep kernel ULPs off the device by clearing kverbs_provider, so only
  uverbs binds.

UMR completion verification (patches 9-14)
- Verify each UMR CQE against a shadow FIFO of the posted requests,
  instead of trusting its wqe_counter. The UMR CQ is the only CQ the
  kernel polls.
- On a forged CQE, keep the UMR path failed and force the health
  recovery.
- Refuse the generic CQ poll path, which trusts the CQE.

Validation of FW reported values (patches 15-21)
- Set the CQ completion handler only once its poll context is set up,
  and ignore completion events for a CQ without one.
- Bound port numbers from EQEs and caps.
- Bound RQ/SQ states from QUERY_RQ and QUERY_SQ.
- Clamp the log_max_* shifts used for the CQ, QP and device limits.

Userspace interface (patches 22-23)
The GET_CONTEXT handshake: the new
UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED input flag and the
IB_UVERBS_CORE_SUPPORT_ADVERSARIAL core_support bit, also reported by
QUERY_CONTEXT.

The matching rdma-core series [1] declares the support and validates
what it reads from the NIC.

[1] https://github.com/linux-rdma/rdma-core/pull/1844

Yishai

Dragos Tatulea (2):
  net/mlx5: Add capability filtering in adversarial mode
  net/mlx5: Add macros for reading log-based caps

Yishai Hadas (21):
  net/mlx5: Add mlx5_trigger_health_work_force()
  net/mlx5: Derive adversarial mode from CC guest memory encryption
  RDMA/mlx5: Reject QP transport types not enabled by FW capability
  RDMA/mlx5: Require IB_QPT_DRIVER for the DC vendor QP types
  RDMA/mlx5: Derive mlx5_ib_adversarial_mode global from mlx5_core
    adversarial API
  RDMA/core: Suppress kverbs_provider for adversarial_mode devices
  RDMA/mlx5: Remove duplicate mlx5_ib_umr_context struct
  RDMA/mlx5: Introduce per-CQ poll_one hook in mlx5_ib_cq
  RDMA/mlx5: Add shadow FIFO infrastructure for UMR completions
  RDMA/mlx5: Verify UMR CQ polling against a hostile NIC
  RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected
  RDMA/mlx5: Block generic CQ poll path in adversarial mode
  RDMA/core: Set the CQ comp_handler after its poll context is set up
  RDMA/mlx5: Ignore completion events for a CQ without a handler
  RDMA/mlx5: Validate EQE port and GSI port before indexing devr.ports[]
  RDMA/mlx5: Validate firmware RQ/SQ state before sqrq_trans[] lookup
  RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks
  RDMA/mlx5: Clamp log_max_* shifts in QP and WQ limit checks
  RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths
  RDMA/core: Report and enforce adversarial mode via GET_CONTEXT
  RDMA/mlx5: Advertise adversarial mode via ib_device::adversarial_mode

 drivers/infiniband/core/cq.c                  |  19 +-
 drivers/infiniband/core/device.c              |   8 +
 drivers/infiniband/core/uverbs_cmd.c          |   5 +
 .../infiniband/core/uverbs_std_types_device.c |  16 +
 drivers/infiniband/hw/mlx5/cq.c               | 220 +++++++++-
 drivers/infiniband/hw/mlx5/gsi.c              |   4 +
 drivers/infiniband/hw/mlx5/mad.c              |   2 +-
 drivers/infiniband/hw/mlx5/main.c             |  71 ++--
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  24 +-
 drivers/infiniband/hw/mlx5/qp.c               |  68 ++--
 drivers/infiniband/hw/mlx5/srq.c              |   2 +-
 drivers/infiniband/hw/mlx5/umr.c              | 111 ++++-
 drivers/infiniband/hw/mlx5/umr.h              |   6 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 .../ethernet/mellanox/mlx5/core/caps_filter.c | 381 ++++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/health.c  |  82 +++-
 .../net/ethernet/mellanox/mlx5/core/main.c    |  15 +-
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   2 +
 include/linux/mlx5/device.h                   |  20 +
 include/linux/mlx5/driver.h                   |   3 +
 include/rdma/ib_verbs.h                       |   2 +
 include/uapi/rdma/ib_user_ioctl_cmds.h        |   5 +
 include/uapi/rdma/ib_user_ioctl_verbs.h       |   1 +
 23 files changed, 957 insertions(+), 112 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c

-- 
2.18.1

[PATCH rdma-next 01/23] net/mlx5: Add mlx5_trigger_health_work_force()

[Yishai Hadas]

mlx5_trigger_health_work() queues the health recovery, which enters
MLX5_DEVICE_STATE_INTERNAL_ERROR only if
mlx5_health_check_fatal_sensors() reports a fault.

A consumer can detect a fault the sensors cannot see, such as a device
that keeps answering while misbehaving. It then needs the recovery to
run with the error state forced, whatever the sensors say.

Add mlx5_trigger_health_work_force(), which queues a dedicated work that
runs the recovery with the error state forced. It only queues the work,
so it can be called from atomic context.

Before entering the error state the forced work stops the health poller
and asks the firmware to stop with mlx5_cmd_fast_teardown_hca(), as the
device might still be running. The error state is forced whether or not
the teardown succeeds. The teardown runs under intf_state_mutex and only
while the interface is up, so it cannot race an unload.

Export it for mlx5_ib, which will use it in a later patch of this series
when it detects a NIC forging completion queue entries, a fault the
health sensors cannot see.

Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/health.c  | 82 ++++++++++++++++---
 include/linux/mlx5/driver.h                   |  2 +
 2 files changed, 73 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index aeeb136f5ebc..d000ac335fa7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -627,18 +627,11 @@ mlx5_fw_fatal_reporter_dump(struct devlink_health_reporter *reporter,
 	return err;
 }
 
-static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
+static void mlx5_fw_fatal_reporter_err(struct mlx5_core_dev *dev, bool forced)
 {
 	struct mlx5_fw_reporter_ctx fw_reporter_ctx;
-	struct mlx5_core_health *health;
-	struct mlx5_core_dev *dev;
-	struct devlink *devlink;
-	struct mlx5_priv *priv;
-
-	health = container_of(work, struct mlx5_core_health, fatal_report_work);
-	priv = container_of(health, struct mlx5_priv, health);
-	dev = container_of(priv, struct mlx5_core_dev, priv);
-	devlink = priv_to_devlink(dev);
+	struct mlx5_core_health *health = &dev->priv.health;
+	struct devlink *devlink = priv_to_devlink(dev);
 
 	mutex_lock(&dev->intf_state_mutex);
 	if (test_bit(MLX5_DROP_HEALTH_WORK, &health->flags)) {
@@ -646,8 +639,26 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 		mutex_unlock(&dev->intf_state_mutex);
 		return;
 	}
+	/*
+	 * The firmware might still be running, so stop it first. Do it under
+	 * intf_state_mutex and only while UP, so the cmd interface is valid.
+	 */
+	if (forced && test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state)) {
+		int err;
+
+		mlx5_stop_health_poll(dev, false);
+		err = mlx5_cmd_fast_teardown_hca(dev);
+		if (err)
+			mlx5_core_err(dev, "Failed to stop the device before forcing the error state, err %d\n",
+				      err);
+	}
+
+	/*
+	 * Enter the error state before unlocking, or an unload could send
+	 * TEARDOWN_HCA to the stopped NIC, fail and skip function disable.
+	 */
+	enter_error_state(dev, forced);
 	mutex_unlock(&dev->intf_state_mutex);
-	enter_error_state(dev, false);
 	if (IS_ERR_OR_NULL(health->fw_fatal_reporter)) {
 		devl_lock(devlink);
 		if (mlx5_health_try_recover(dev))
@@ -669,6 +680,33 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 	}
 }
 
+static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
+{
+	struct mlx5_core_health *health;
+	struct mlx5_core_dev *dev;
+	struct mlx5_priv *priv;
+
+	health = container_of(work, struct mlx5_core_health, fatal_report_work);
+	priv = container_of(health, struct mlx5_priv, health);
+	dev = container_of(priv, struct mlx5_core_dev, priv);
+
+	mlx5_fw_fatal_reporter_err(dev, false);
+}
+
+static void mlx5_fw_fatal_reporter_force_err_work(struct work_struct *work)
+{
+	struct mlx5_core_health *health;
+	struct mlx5_core_dev *dev;
+	struct mlx5_priv *priv;
+
+	health = container_of(work, struct mlx5_core_health,
+			      force_fatal_report_work);
+	priv = container_of(health, struct mlx5_priv, health);
+	dev = container_of(priv, struct mlx5_core_dev, priv);
+
+	mlx5_fw_fatal_reporter_err(dev, true);
+}
+
 #define MLX5_FW_REPORTER_ECPF_GRACEFUL_PERIOD 180000
 #define MLX5_FW_REPORTER_PF_GRACEFUL_PERIOD 60000
 #define MLX5_FW_REPORTER_VF_GRACEFUL_PERIOD 30000
@@ -759,6 +797,25 @@ void mlx5_trigger_health_work(struct mlx5_core_dev *dev)
 		queue_work(health->wq, &health->fatal_report_work);
 }
 
+/**
+ * mlx5_trigger_health_work_force - enter the error state and recover, even
+ * if the health sensors report the device as healthy
+ * @dev: mlx5 core device
+ *
+ * For faults only a consumer can detect, where the firmware keeps answering
+ * and mlx5_health_check_fatal_sensors() sees nothing wrong. Without forcing,
+ * the device would stay in MLX5_DEVICE_STATE_UP and consumers waiting on
+ * outstanding work would never be released.
+ */
+void mlx5_trigger_health_work_force(struct mlx5_core_dev *dev)
+{
+	struct mlx5_core_health *health = &dev->priv.health;
+
+	if (!mlx5_dev_is_lightweight(dev))
+		queue_work(health->wq, &health->force_fatal_report_work);
+}
+EXPORT_SYMBOL_GPL(mlx5_trigger_health_work_force);
+
 #define MLX5_MSEC_PER_HOUR (MSEC_PER_SEC * 60 * 60)
 static void mlx5_health_log_ts_update(struct work_struct *work)
 {
@@ -874,6 +931,7 @@ void mlx5_drain_health_wq(struct mlx5_core_dev *dev)
 	cancel_delayed_work_sync(&health->update_fw_log_ts_work);
 	cancel_work_sync(&health->report_work);
 	cancel_work_sync(&health->fatal_report_work);
+	cancel_work_sync(&health->force_fatal_report_work);
 }
 
 void mlx5_health_cleanup(struct mlx5_core_dev *dev)
@@ -911,6 +969,8 @@ int mlx5_health_init(struct mlx5_core_dev *dev)
 	if (!health->wq)
 		goto out_err;
 	INIT_WORK(&health->fatal_report_work, mlx5_fw_fatal_reporter_err_work);
+	INIT_WORK(&health->force_fatal_report_work,
+		  mlx5_fw_fatal_reporter_force_err_work);
 	INIT_WORK(&health->report_work, mlx5_fw_reporter_err_work);
 	INIT_DELAYED_WORK(&health->update_fw_log_ts_work, mlx5_health_log_ts_update);
 
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index abb99371ff13..a1275e6aa6c1 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -450,6 +450,7 @@ struct mlx5_core_health {
 	struct workqueue_struct	       *wq;
 	unsigned long			flags;
 	struct work_struct		fatal_report_work;
+	struct work_struct		force_fatal_report_work;
 	struct work_struct		report_work;
 	struct devlink_health_reporter *fw_reporter;
 	struct devlink_health_reporter *fw_fatal_reporter;
@@ -1034,6 +1035,7 @@ void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health);
 void mlx5_start_health_fw_log_up(struct mlx5_core_dev *dev);
 void mlx5_drain_health_wq(struct mlx5_core_dev *dev);
 void mlx5_trigger_health_work(struct mlx5_core_dev *dev);
+void mlx5_trigger_health_work_force(struct mlx5_core_dev *dev);
 int mlx5_frag_buf_alloc_node(struct mlx5_core_dev *dev, int size,
 			     struct mlx5_frag_buf *buf, int node);
 void mlx5_frag_buf_free(struct mlx5_core_dev *dev, struct mlx5_frag_buf *buf);
-- 
2.18.1

[PATCH rdma-next 02/23] net/mlx5: Derive adversarial mode from CC guest memory encryption

[Yishai Hadas]

Add mlx5_is_adversarial() to report whether the driver treats the NIC
as untrusted. In a confidential computing guest, the hypervisor and any
passthrough device are outside the guest's trust boundary, so the mode
is derived from cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT).

This turns adversarial mode on automatically in such guests, without
any administrator configuration.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/main.c | 7 +++++++
 include/linux/mlx5/driver.h                    | 1 +
 2 files changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 1a45474ce03d..f9e10464a71b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -31,6 +31,7 @@
  */
 
 #include <linux/highmem.h>
+#include <linux/cc_platform.h>
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/errno.h>
@@ -484,6 +485,12 @@ bool mlx5_is_roce_on(struct mlx5_core_dev *dev)
 }
 EXPORT_SYMBOL(mlx5_is_roce_on);
 
+bool mlx5_is_adversarial(void)
+{
+	return cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT);
+}
+EXPORT_SYMBOL_GPL(mlx5_is_adversarial);
+
 static int handle_hca_cap_2(struct mlx5_core_dev *dev, void *set_ctx)
 {
 	bool do_set = false;
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index a1275e6aa6c1..3d3bc16e581d 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -1328,6 +1328,7 @@ enum {
 };
 
 bool mlx5_is_roce_on(struct mlx5_core_dev *dev);
+bool mlx5_is_adversarial(void);
 
 static inline bool mlx5_get_roce_state(struct mlx5_core_dev *dev)
 {
-- 
2.18.1

[PATCH rdma-next 03/23] net/mlx5: Add capability filtering in adversarial mode

[Yishai Hadas]

From: Dragos Tatulea <dtatulea@nvidia.com>

When in adversarial mode, trust only a subset of capabilities
and do some additional validations.

Start off by clearing all firmware capabilities and enabling
them one by one. Some numeric capabilities are validated.
Unexpected values will trigger a driver load failure flow.

Where it applies, the filtering is done on both current and max caps.
Allowed capabilities are simply passed through. Clamped and checked
capabilities also apply to max values.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 .../ethernet/mellanox/mlx5/core/caps_filter.c | 381 ++++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/main.c    |   8 +-
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   2 +
 4 files changed, 388 insertions(+), 5 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index a1871259210d..dc8eabee631f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -11,7 +11,7 @@ obj-$(CONFIG_MLX5_CORE) += mlx5_core.o
 #
 # mlx5 core basic
 #
-mlx5_core-y :=	main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
+mlx5_core-y :=	main.o caps_filter.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
 		health.o mcg.o cq.o alloc.o port.o mr.o pd.o \
 		transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \
 		fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
new file mode 100644
index 000000000000..67b260339745
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
@@ -0,0 +1,381 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include "linux/mlx5/device.h"
+#include <linux/mlx5/driver.h>
+#include <linux/mlx5/qp.h>
+#include "mlx5_core.h"
+
+#define MLX5_HCA_CAP_ALLOW(cap, fld) \
+	MLX5_SET(cap, dst, fld, MLX5_GET(cap, src, fld))
+#define MLX5_HCA_CAP_ALLOW64(cap, fld) \
+	MLX5_SET64(cap, dst, fld, MLX5_GET64(cap, src, fld))
+
+/* Allow a numeric capability range restricted to [lo, hi]. */
+#define MLX5_HCA_CAP_CLAMP(cap, fld, lo, hi) \
+	MLX5_SET(cap, dst, fld, mlx5_cap_clamp(dev, #fld, \
+					       MLX5_GET(cap, src, fld), \
+					       (lo), (hi)))
+
+/* Check cap range and return error otherwise. */
+#define MLX5_HCA_CAP_CHECK(cap, fld, lo, hi) \
+	mlx5_cap_check(dev, #fld, MLX5_GET(cap, dst, fld), (lo), (hi), false)
+/* Same but zero is also a valid value. */
+#define MLX5_HCA_CAP_CHECK_Z(cap, fld, lo, hi) \
+	mlx5_cap_check(dev, #fld, MLX5_GET(cap, dst, fld), (lo), (hi), true)
+
+static u32 mlx5_cap_clamp(struct mlx5_core_dev *dev, const char *name,
+			  u32 val, u32 lo, u32 hi)
+{
+	u32 clamped = clamp_t(u32, val, lo, hi);
+
+	if (clamped != val)
+		mlx5_core_warn(dev, "cap filter: FW reported %s=%u, outside [%u, %u], using %u\n",
+			       name, val, lo, hi, clamped);
+
+	return clamped;
+}
+
+static int mlx5_cap_check(struct mlx5_core_dev *dev, const char *name,
+			  u32 val, u32 lo, u32 hi, bool zero)
+{
+	if (val >= lo && val <= hi)
+		return 0;
+
+	if (zero && !val)
+		return 0;
+
+	mlx5_core_err(dev, "cap filter: FW reported %s=%u, outside [%u, %u] %s\n",
+		      name, val, lo, hi,
+		      zero ? "or 0" : "");
+
+	return -EINVAL;
+}
+
+static int mlx5_adversarial_filter_general(struct mlx5_core_dev *dev, void *dst,
+					   void *src)
+{
+	int err;
+
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, page_request_disable);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_resource_manager);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, hca_cap_2);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_teardown_request);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_in_use);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_active);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_allocated);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_invalid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_srq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_qp_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_cap);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_srq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_sgl_for_optimized_performance);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_cq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_write_umr);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read_umr);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_cq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_write);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read_pci_enabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, terminate_scatter_list_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fast_teardown);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_eq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_indirection);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fixed_buffer_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, force_teardown);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_extended_translation_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_klm_list_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qpc_extension);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_req_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cq_overrun);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_res_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, release_all_pages);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce_accl);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_req_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_res_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, end_pad);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cc_query_allowed);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cc_modify_allowed);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cache_line_128byte);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rts2rts_qp_counters_set_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_int_rq_oob);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sbcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, gid_table_size);
+
+	/* Values outside of [0, 4] range are unexpected. */
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, gid_table_size, 0, 4);
+	if (err)
+		return err;
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, out_of_seq_cnt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, retransmission_q_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, debug);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, modify_rq_counter_set_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rq_delay_drop);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_qp_cnt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pkey_table_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_group_manager);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_queue_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, ets);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, nic_flow_table);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, mcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, local_ca_ack_delay);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, enhanced_error_q_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, beacon_led);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, port_type);
+	/* 0 is invalid, 8 is for MLX5_MAX_PORTS */
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_ports, 1, 8);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_msg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_tc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dcbx);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, general_notification_event);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, stat_rate_support);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pci_sync_for_fw_update_event);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cqe_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_modify_entity_size_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_modify_atomic_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_indirect_mkey_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_fence);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cmdif_checksum);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sho);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qos);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, eth_net_offloads);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, atomic);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_moderation);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_period_mode_modify);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_period_start_from_cqe);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, apm);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_ptr_rlky);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qp_packet_based);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pkv);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, uar_4k);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fl_rc_qp_when_roce_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_pg_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, bf);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, driver_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, mkey_by_name);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_bf_reg_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, disciplined_fr_counter);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qp_error_syndrome);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_lag_ports, 0, 1);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_wqe_sz_sq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, icm_mng_function_id_mode);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, tis_tir_td_order);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_wqe_sz_rq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, query_host_net_function_num_max);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_qp_mcg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_mcg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_transport_domain);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_pd);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_ud);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_uc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_xrc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_rc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pcie_reset_using_hotreset_method);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap,
+			   pci_sync_for_fw_update_with_driver_unload);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_steering_fail);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_counter_local_loopback);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, q_counter_aggregation);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, q_counter_other_vport);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, nic_receive_steering_discard);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, receive_discard_vport_down);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, transmit_discard_vport_down);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, eq_overrun_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, invalid_command_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, quota_exceeded_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_flow_counter_bulk);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_sq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_tir);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, basic_cyclic_rcv_wqe);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rqt);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, log_max_rqt_size, 0, 13);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_bar_uar_access);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_odp_page_fault);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce_rw_supported);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_wq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_state);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_vlan_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_mc_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_uc_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sq_ts_format);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rq_ts_format);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, steering_format_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, create_qp_start_hint);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, ats);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cross_vhca_rqt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_uctx);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_num_eqs);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_uar_page_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_geneve_tlv_options);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_geneve_tlv_option_data_len);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_dci_stream_channels);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_dci_errored_streams);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, crossing_vhca_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, native_port_num);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_vhca_ports, 0, 1);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sw_owner_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_num_of_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_ppcnt_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_q_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, apu);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, uctx_cap);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_total_dynamic_vf_msix);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, min_dynamic_vf_msix_table_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, data_direct);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_dynamic_vf_msix_table_size);
+	MLX5_HCA_CAP_ALLOW64(cmd_hca_cap, vhca_tunnel_commands);
+
+	MLX5_SET(cmd_hca_cap, dst, num_of_uars_per_page, MLX5_UARS_IN_PAGE);
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, flow_counter_bulk_alloc);
+
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, log_pg_sz,
+				 MLX5_ADAPTER_PAGE_SHIFT,
+				 MLX5_ADAPTER_PAGE_SHIFT);
+	if (err)
+		return err;
+
+#define LOG_BF_REG_SIZE_MIN ilog2(2 * MLX5_SEND_WQE_BB)
+#define LOG_BF_REG_SIZE_MAX \
+	ilog2((MLX5_ADAPTER_PAGE_SIZE - MLX5_BF_OFFSET) / MLX5_BFREGS_PER_UAR)
+
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, log_bf_reg_size,
+				 LOG_BF_REG_SIZE_MIN,
+				 LOG_BF_REG_SIZE_MAX);
+	if (err)
+		return err;
+
+#undef LOG_BF_REG_SIZE_MAX
+#undef LOG_BF_REG_SIZE_MIN
+
+	return MLX5_HCA_CAP_CHECK(cmd_hca_cap, min_dynamic_vf_msix_table_size, 0,
+				  MLX5_GET(cmd_hca_cap, dst,
+					   max_dynamic_vf_msix_table_size));
+}
+
+static int mlx5_adversarial_filter_general_2(struct mlx5_core_dev *dev, void *dst,
+					     void *src)
+{
+	int err;
+	u32 val;
+
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, dp_ordering_force);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, query_vuid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, umr_log_entity_size_5);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_insert_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_insert_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_remove_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_remove_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, sf_eq_usage);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_rqt_vhca_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   min_mkey_log_entity_size_fixed_buffer_valid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, log_min_mkey_entity_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   min_mkey_log_entity_size_fixed_buffer);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   max_mkey_log_entity_size_fixed_buffer);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_mkey_log_entity_size_mtt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_num_eqs_24b);
+
+	err = MLX5_HCA_CAP_CHECK_Z(cmd_hca_cap_2, max_mkey_log_entity_size_mtt,
+				   MLX5_ADAPTER_PAGE_SHIFT, 63);
+	if (err)
+		return err;
+
+	err = MLX5_HCA_CAP_CHECK_Z(cmd_hca_cap_2,
+				   max_mkey_log_entity_size_fixed_buffer,
+				   MLX5_ADAPTER_PAGE_SHIFT, 63);
+	if (err)
+		return err;
+
+	val = MLX5_GET(cmd_hca_cap_2, dst, max_mkey_log_entity_size_mtt) ?: 31;
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap_2, log_min_mkey_entity_size, 0,
+				 val);
+	if (err)
+		return err;
+
+	/* The fixed buffer min is don't-care when its valid bit is clear. */
+	if (!MLX5_GET(cmd_hca_cap_2, dst,
+		      min_mkey_log_entity_size_fixed_buffer_valid))
+		return 0;
+
+	val = MLX5_GET(cmd_hca_cap_2, dst,
+		       max_mkey_log_entity_size_fixed_buffer) ?: 31;
+	return MLX5_HCA_CAP_CHECK(cmd_hca_cap_2,
+				  min_mkey_log_entity_size_fixed_buffer, 0, val);
+}
+
+static int mlx5_adversarial_filter_per_protocol(struct mlx5_core_dev *dev,
+						 void *dst, void *src)
+{
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, csum_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, vlan_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   self_lb_en_modifiable);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, max_lso_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   wqe_inline_mode);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   rss_ind_tbl_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   cqe_checksum_full);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   max_vxlan_udp_ports);
+
+	return 0;
+}
+
+#undef MLX5_HCA_CAP_ALLOW
+#undef MLX5_HCA_CAP_ALLOW64
+#undef MLX5_HCA_CAP_CLAMP
+#undef MLX5_HCA_CAP_CHECK
+#undef MLX5_HCA_CAP_CHECK_Z
+
+int mlx5_store_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
+		    void *dst, void *src)
+{
+	if (!mlx5_is_adversarial()) {
+		memcpy(dst, src, MLX5_UN_SZ_BYTES(hca_cap_union));
+		return 0;
+	}
+
+	switch (cap_type) {
+	case MLX5_CAP_GENERAL:
+		return mlx5_adversarial_filter_general(dev, dst, src);
+	case MLX5_CAP_GENERAL_2:
+		return mlx5_adversarial_filter_general_2(dev, dst, src);
+	case MLX5_CAP_ETHERNET_OFFLOADS:
+		return mlx5_adversarial_filter_per_protocol(dev, dst, src);
+
+	case MLX5_CAP_ATOMIC:
+	case MLX5_CAP_ROCE:
+	case MLX5_CAP_FLOW_TABLE:
+	case MLX5_CAP_DEV_EVENT:
+		memcpy(dst, src, MLX5_UN_SZ_BYTES(hca_cap_union));
+		break;
+
+	default:
+		memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+		break;
+	}
+
+	return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index f9e10464a71b..e76a4486b5a3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -325,12 +325,12 @@ int mlx5_core_get_caps_mode(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_ty
 
 	switch (cap_mode) {
 	case HCA_CAP_OPMOD_GET_MAX:
-		memcpy(dev->caps.hca[cap_type]->max, hca_caps,
-		       MLX5_UN_SZ_BYTES(hca_cap_union));
+		err = mlx5_store_caps(dev, cap_type,
+				      dev->caps.hca[cap_type]->max, hca_caps);
 		break;
 	case HCA_CAP_OPMOD_GET_CUR:
-		memcpy(dev->caps.hca[cap_type]->cur, hca_caps,
-		       MLX5_UN_SZ_BYTES(hca_cap_union));
+		err = mlx5_store_caps(dev, cap_type,
+				      dev->caps.hca[cap_type]->cur, hca_caps);
 		break;
 	default:
 		mlx5_core_warn(dev,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index d84d0424d04e..0084bed669f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -215,6 +215,8 @@ static inline int mlx5_flexible_inlen(struct mlx5_core_dev *dev, size_t fixed,
 	mlx5_flexible_inlen(dev, fixed, item_size, num_items, __func__, __LINE__)
 
 int mlx5_core_get_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type);
+int mlx5_store_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
+		    void *dst, void *src);
 int mlx5_core_get_caps_mode(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
 			    enum mlx5_cap_mode cap_mode);
 int mlx5_query_hca_caps(struct mlx5_core_dev *dev);
-- 
2.18.1

[PATCH rdma-next 04/23] net/mlx5: Add macros for reading log-based caps

[Yishai Hadas]

From: Dragos Tatulea <dtatulea@nvidia.com>

Add macros that can be used to exponentiate log based capabilities.

When a capability can represent more than the used version (u32 , u64),
value will be clamped to the largest representable power of two.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
 include/linux/mlx5/device.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index 37727b5ab78f..f45454f65d5c 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -38,6 +38,7 @@
 #include <linux/mlx5/mlx5_ifc.h>
 #include <linux/mlx5/mlx5_ifc_macros.h>
 #include <linux/bitfield.h>
+#include <linux/minmax.h>
 
 #if defined(__LITTLE_ENDIAN)
 #define MLX5_SET_HOST_ENDIANNESS	0
@@ -47,6 +48,16 @@
 #error Host endianness not defined
 #endif
 
+/* Largest shift count that is defined behaviour for a value of type @T. */
+#define __mlx5_max_shift(T) (BITS_PER_TYPE(T) - 1 - is_signed_type(T))
+
+/* Calculate 2^@fld from a log2 cap field. Exponents that don't fit in @T
+ * saturate to the largest representable power of two (not to the type
+ * maximum), avoiding undefined shifts.
+ */
+#define MLX5_GET_LOGSZ_CLAMP(typ, p, fld, T) \
+	((T)((T)1 << min_t(u32, MLX5_GET(typ, p, fld), __mlx5_max_shift(T))))
+
 enum mlx5_inline_modes {
 	MLX5_INLINE_MODE_NONE,
 	MLX5_INLINE_MODE_L2,
@@ -1096,6 +1107,15 @@ enum mlx5_qcam_feature_groups {
 #define MLX5_CAP_GEN_MAX(mdev, cap) \
 	MLX5_GET(cmd_hca_cap, mdev->caps.hca[MLX5_CAP_GENERAL]->max, cap)
 
+/* 2^cap for log2-encoded general caps */
+#define MLX5_CAP_GEN_LOGSZ_U32(mdev, cap) \
+	MLX5_GET_LOGSZ_CLAMP(cmd_hca_cap, \
+			     mdev->caps.hca[MLX5_CAP_GENERAL]->cur, cap, u32)
+
+#define MLX5_CAP_GEN_LOGSZ_U64(mdev, cap) \
+	MLX5_GET_LOGSZ_CLAMP(cmd_hca_cap, \
+			     mdev->caps.hca[MLX5_CAP_GENERAL]->cur, cap, u64)
+
 #define MLX5_CAP_GEN_2(mdev, cap) \
 	MLX5_GET(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
 
-- 
2.18.1

[PATCH rdma-next 05/23] RDMA/mlx5: Reject QP transport types not enabled by FW capability

[Yishai Hadas]

check_qp_type() checks the FW capability only for XRC and DC. RC, UC
and UD QPs are created even if the FW does not report the matching
capability bit.

Check the rc, uc and ud capabilities as well, so that QP creation is
refused for a transport the FW does not support.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 627dd17804c9..943ee5c55538 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2824,15 +2824,24 @@ static int check_qp_type(struct mlx5_ib_dev *dev, struct ib_qp_init_attr *attr,
 	case IB_QPT_XRC_INI:
 		if (!MLX5_CAP_GEN(dev->mdev, xrc))
 			goto out;
-		fallthrough;
+		break;
 	case IB_QPT_RC:
+		if (!MLX5_CAP_GEN(dev->mdev, rc))
+			goto out;
+		break;
 	case IB_QPT_UC:
+		if (!MLX5_CAP_GEN(dev->mdev, uc))
+			goto out;
+		break;
+	case IB_QPT_UD:
+		if (!MLX5_CAP_GEN(dev->mdev, ud))
+			goto out;
+		break;
 	case IB_QPT_SMI:
 	case MLX5_IB_QPT_HW_GSI:
 	case IB_QPT_DRIVER:
 	case IB_QPT_GSI:
 	case IB_QPT_RAW_PACKET:
-	case IB_QPT_UD:
 	case MLX5_IB_QPT_REG_UMR:
 		break;
 	default:
-- 
2.18.1

[PATCH rdma-next 06/23] RDMA/mlx5: Require IB_QPT_DRIVER for the DC vendor QP types

[Yishai Hadas]

check_qp_type() checks the dct capability only for IB_QPT_DRIVER, but
process_vendor_flags() turns any QP that carries a DC type flag into a
DCI or DCT QP. A QP created as, for example, IB_QPT_RC with a DC flag
thus becomes a DC QP without the capability check.

Accept the DC type flags only on IB_QPT_DRIVER, the type userspace
uses to create DC QPs.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 943ee5c55538..19cbf6b2b979 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2965,18 +2965,22 @@ static int process_vendor_flags(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,
 
 	switch (flags & (MLX5_QP_FLAG_TYPE_DCT | MLX5_QP_FLAG_TYPE_DCI)) {
 	case MLX5_QP_FLAG_TYPE_DCI:
+		if (qp->type != IB_QPT_DRIVER)
+			return -EINVAL;
 		qp->type = MLX5_IB_QPT_DCI;
 		break;
 	case MLX5_QP_FLAG_TYPE_DCT:
+		if (qp->type != IB_QPT_DRIVER)
+			return -EINVAL;
 		qp->type = MLX5_IB_QPT_DCT;
 		break;
+	case 0:
+		/* IB_QPT_DRIVER needs a subtype */
+		if (qp->type == IB_QPT_DRIVER)
+			return -EINVAL;
+		break;
 	default:
-		if (qp->type != IB_QPT_DRIVER)
-			break;
-		/*
-		 * It is IB_QPT_DRIVER and or no subtype or
-		 * wrong subtype were provided.
-		 */
+		/* Both subtypes */
 		return -EINVAL;
 	}
 
-- 
2.18.1

[PATCH rdma-next 07/23] RDMA/mlx5: Derive mlx5_ib_adversarial_mode global from mlx5_core adversarial API

[Yishai Hadas]

Use the mlx5_core mlx5_is_adversarial() exported API to set a
module-level static bool mlx5_ib_adversarial_mode at init time,
instead of a per-device bit field. This avoids propagating the flag
through every mlx5_ib_dev pointer and makes the module-wide policy
clear at the call sites.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c    | 3 +++
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
 2 files changed, 4 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 65455aedd559..f4f2ad80e76c 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -76,6 +76,7 @@ enum {
 	MLX5_ATOMIC_SIZE_QP_8BYTES = 1 << 3,
 };
 
+bool mlx5_ib_adversarial_mode;
 static struct workqueue_struct *mlx5_ib_event_wq;
 static LIST_HEAD(mlx5_ib_unaffiliated_port_list);
 static LIST_HEAD(mlx5_ib_dev_list);
@@ -5431,6 +5432,8 @@ static int __init mlx5_ib_init(void)
 		return -ENOMEM;
 	}
 
+	mlx5_ib_adversarial_mode = mlx5_is_adversarial();
+
 	ret = mlx5_ib_qp_event_init();
 	if (ret)
 		goto qp_event_err;
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a6ee14ffd7d..0f81b8aa487e 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1582,6 +1582,7 @@ static inline int get_uars_per_sys_page(struct mlx5_ib_dev *dev, bool lib_suppor
 }
 
 extern void *xlt_emergency_page;
+extern bool mlx5_ib_adversarial_mode;
 
 int bfregn_to_uar_index(struct mlx5_ib_dev *dev,
 			struct mlx5_bfreg_info *bfregi, u32 bfregn,
-- 
2.18.1

[PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Yishai Hadas]

In adversarial mode the NIC is not trusted. Kernel ULPs such as MAD,
IB CM, rdma_cm, IPoIB, SRP, iSER and NVMe-oF trust what the device
reports, its completions and the data it writes, so they must not use
such a device.

Add an adversarial_mode field to struct ib_device and clear
kverbs_provider when it is set. add_client_context() then skips every
client but uverbs, the only one that sets no_kverbs_req, so the kernel
ULPs never bind to the device.

The driver still implements the kverbs ops, as the uverbs path shares
them, so kverbs_provider can't be cleared by the existing check of the
mandatory ops.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/device.c | 8 ++++++++
 include/rdma/ib_verbs.h          | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 29af9fe860e2..eb6aa567d365 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -312,6 +312,14 @@ static void ib_device_check_mandatory(struct ib_device *device)
 			break;
 		}
 	}
+
+	/*
+	 * Kernel ULPs must not bind to an adversarial_mode device. Its kverbs
+	 * ops are still present, as uverbs uses them, so clear kverbs_provider
+	 * explicitly. uverbs sets no_kverbs_req, so it still binds.
+	 */
+	if (device->adversarial_mode)
+		device->kverbs_provider = false;
 }
 
 /*
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d6eaa3fee66e..6209a5166f0d 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -3008,6 +3008,8 @@ struct ib_device {
 	u16                          use_cq_dim:1;
 	/* CoCo guest with DMA bounce buffering required */
 	u16                          cc_dma_bounce:1;
+	/* Untrusted-NIC, adversarial mode is required */
+	u8			     adversarial_mode;
 	u8                           node_type;
 	u32			     phys_port_cnt;
 	struct ib_device_attr        attrs;
-- 
2.18.1

[PATCH rdma-next 09/23] RDMA/mlx5: Remove duplicate mlx5_ib_umr_context struct

[Yishai Hadas]

mlx5_ib_umr_context in mlx5_ib.h and mlx5r_umr_context in umr.h have
identical layouts: an ib_cqe, a status, and a completion.  The done
callback used mlx5_ib_umr_context via container_of, while every other
site in umr.c used mlx5r_umr_context.

Remove the duplicate and use mlx5r_umr_context consistently throughout.
No behaviour change.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 6 ------
 drivers/infiniband/hw/mlx5/umr.c     | 4 ++--
 2 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 0f81b8aa487e..980b41800df5 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -749,12 +749,6 @@ struct mlx5_ib_mw {
 	struct mlx5_ib_mkey	mmkey;
 };
 
-struct mlx5_ib_umr_context {
-	struct ib_cqe		cqe;
-	enum ib_wc_status	status;
-	struct completion	done;
-};
-
 enum {
 	MLX5_UMR_STATE_UNINIT,
 	MLX5_UMR_STATE_ACTIVE,
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 711075e3d9ea..4a3fa79eba3a 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -339,8 +339,8 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 
 static void mlx5r_umr_done(struct ib_cq *cq, struct ib_wc *wc)
 {
-	struct mlx5_ib_umr_context *context =
-		container_of(wc->wr_cqe, struct mlx5_ib_umr_context, cqe);
+	struct mlx5r_umr_context *context =
+		container_of(wc->wr_cqe, struct mlx5r_umr_context, cqe);
 
 	context->status = wc->status;
 	complete(&context->done);
-- 
2.18.1

[PATCH rdma-next 10/23] RDMA/mlx5: Introduce per-CQ poll_one hook in mlx5_ib_cq

[Yishai Hadas]

Add a poll_one function pointer to struct mlx5_ib_cq and initialize it
to mlx5_poll_one in both mlx5_ib_create_cq() (kernel-ULP CQs) and
mlx5_ib_create_user_cq() (uverbs CQs, the path used by both the
modern ioctl-based and legacy write-based CQ creation commands).
mlx5_ib_poll_cq() is updated to dispatch through the pointer instead
of calling mlx5_poll_one() directly.

This is a pure refactor with no behaviour change. The indirection
allows a specialised poll_one implementation to be installed on a
specific CQ, which a follow-up patch uses to verify the UMR CQ against
a hostile NIC.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c      | 9 ++++++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 ++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 55ffd7d4de51..11c1153747c2 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -633,7 +633,7 @@ int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 		soft_polled = poll_soft_wc(cq, num_entries, wc, false);
 
 	for (npolled = 0; npolled < num_entries - soft_polled; npolled++) {
-		if (mlx5_poll_one(cq, &cur_qp, wc + soft_polled + npolled))
+		if (cq->poll_one(cq, &cur_qp, wc + soft_polled + npolled))
 			break;
 	}
 
@@ -1010,6 +1010,8 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 
 	cq->mcq.comp = mlx5_add_cq_to_tasklet;
 	cq->mcq.tasklet_ctx.comp = mlx5_ib_cq_comp;
+	/* Set before the CQ is created, see mlx5_ib_create_cq() */
+	cq->poll_one = mlx5_poll_one;
 
 	err = mlx5_core_create_cq(dev->mdev, &cq->mcq, cqb, inlen, out, sizeof(out));
 	if (err)
@@ -1093,6 +1095,11 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	MLX5_SET64(cqc, cqc, dbr_addr, cq->db.dma);
 
 	cq->mcq.comp = mlx5_ib_cq_comp;
+	/*
+	 * Set before mlx5_core_create_cq(). No real CQE can arrive before a
+	 * QP uses the CQ, but a forged one can make mlx5_ib_poll_cq() call it.
+	 */
+	cq->poll_one = mlx5_poll_one;
 
 	err = mlx5_core_create_cq(dev->mdev, &cq->mcq, cqb, inlen, out,
 				  sizeof(out));
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 980b41800df5..4833f42b3fd1 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -592,6 +592,8 @@ struct mlx5_ib_cq {
 	enum ib_cq_notify_flags notify_flags;
 	struct work_struct	notify_work;
 	u16			private_flags; /* Use mlx5_ib_cq_pr_flags */
+	int (*poll_one)(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+			struct ib_wc *wc);
 };
 
 struct mlx5_ib_wc {
-- 
2.18.1

[PATCH rdma-next 11/23] RDMA/mlx5: Add shadow FIFO infrastructure for UMR completions

[Yishai Hadas]

The UMR CQ is polled by the generic path, which finds the completed
request by indexing qp->sq.wrid[] with the CQE wqe_counter. A hostile
NIC can forge it to complete a request other than the one that really
completed, or one that is not pending at all.

The UMR QP is RC and signals every WR, so its completions arrive in
posting order. In adversarial mode, keep a FIFO of the posted UMR
contexts, so a follow-up patch can check each CQE against the oldest
pending request instead of trusting it. Each context also records the
SQ producer counter it was posted at, as its address alone doesn't
identify the post: it is usually on the stack and reused across posts.

Nothing consumes the FIFO yet.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 10 +++++++
 drivers/infiniband/hw/mlx5/umr.c     | 42 +++++++++++++++++++++++-----
 drivers/infiniband/hw/mlx5/umr.h     |  6 ++++
 3 files changed, 51 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4833f42b3fd1..9cc4b2f99b69 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -758,6 +758,12 @@ enum {
 	MLX5_UMR_STATE_ERR,
 };
 
+#define MLX5_UMR_MAX_WR 128
+static_assert((MLX5_UMR_MAX_WR & (MLX5_UMR_MAX_WR - 1)) == 0,
+	      "MLX5_UMR_MAX_WR must be a power of two for FIFO index masking");
+
+struct mlx5r_umr_context;
+
 struct umr_common {
 	struct ib_pd	*pd;
 	struct ib_cq	*cq;
@@ -771,6 +777,10 @@ struct umr_common {
 	unsigned int state;
 	/* Protects from repeat UMR QP creation */
 	struct mutex init_lock;
+	/* Posted UMR contexts in posting order, adversarial_mode only */
+	struct mlx5r_umr_context **pending;
+	u32 pending_head;
+	u32 pending_tail;
 };
 
 struct mlx5_ib_port_resources {
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 4a3fa79eba3a..aba79a19d627 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -98,9 +98,7 @@ static int umr_check_mkey_mask(struct mlx5_ib_dev *dev, u64 mask)
 	return 0;
 }
 
-enum {
-	MAX_UMR_WR = 128,
-};
+#define MAX_UMR_WR MLX5_UMR_MAX_WR
 
 static int mlx5r_umr_qp_rst2rts(struct mlx5_ib_dev *dev, struct ib_qp *qp)
 {
@@ -181,6 +179,16 @@ int mlx5r_umr_resource_init(struct mlx5_ib_dev *dev)
 	if (ret)
 		goto destroy_qp;
 
+	if (mlx5_ib_adversarial_mode) {
+		dev->umrc.pending = kcalloc(MAX_UMR_WR,
+					    sizeof(*dev->umrc.pending),
+					    GFP_KERNEL);
+		if (!dev->umrc.pending) {
+			ret = -ENOMEM;
+			goto destroy_qp;
+		}
+	}
+
 	dev->umrc.cq = cq;
 
 	sema_init(&dev->umrc.sem, MAX_UMR_WR);
@@ -208,6 +216,7 @@ void mlx5r_umr_resource_cleanup(struct mlx5_ib_dev *dev)
 	/* After device init, UMR cp/qp are not unset during the lifetime. */
 	ib_destroy_qp(dev->umrc.qp);
 	ib_free_cq(dev->umrc.cq);
+	kfree(dev->umrc.pending);
 }
 
 int mlx5r_umr_init(struct mlx5_ib_dev *dev)
@@ -236,7 +245,8 @@ void mlx5r_umr_cleanup(struct mlx5_ib_dev *dev)
 }
 
 
-static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
+static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey,
+			       struct mlx5r_umr_context *ctx,
 			       struct mlx5r_umr_wqe *wqe, bool with_data)
 {
 	unsigned int wqe_size =
@@ -245,6 +255,7 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
 				    sizeof(struct mlx5_wqe_data_seg);
 	struct mlx5_ib_dev *dev = to_mdev(ibqp->device);
 	struct mlx5_core_dev *mdev = dev->mdev;
+	struct umr_common *umrc = &dev->umrc;
 	struct mlx5_ib_qp *qp = to_mqp(ibqp);
 	struct mlx5_wqe_ctrl_seg *ctrl;
 	union {
@@ -267,13 +278,30 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
 		goto out;
 
 	qp->sq.wr_data[idx] = MLX5_IB_WR_UMR;
+	if (mlx5_ib_adversarial_mode) {
+		/* mlx5r_finish_wqe() advances cur_post, so record it first */
+		ctx->wqe_ctr = qp->sq.cur_post;
+	}
 
 	mlx5r_memcpy_send_wqe(&qp->sq, &cur_edge, &seg, &size, wqe, wqe_size);
 
-	id.ib_cqe = cqe;
+	id.ib_cqe = &ctx->cqe;
 	mlx5r_finish_wqe(qp, ctrl, seg, size, cur_edge, idx, id.wr_id, 0,
 			 MLX5_FENCE_MODE_INITIATOR_SMALL, MLX5_OPCODE_UMR);
 
+	/*
+	 * Push to the shadow FIFO before the doorbell, so the entry exists
+	 * before a CQE for it can arrive. A hostile NIC can post a CQE before
+	 * the doorbell, so the UMR CQ poller, on another CPU, can't rely on
+	 * the doorbell to see ctx->wqe_ctr and ctx->cqe.done with the entry.
+	 * Publish it with a release, which pairs with the poller's acquire.
+	 */
+	if (mlx5_ib_adversarial_mode) {
+		smp_store_release(&umrc->pending[umrc->pending_tail & (MAX_UMR_WR - 1)],
+				  ctx);
+		umrc->pending_tail++;
+	}
+
 	mlx5r_ring_db(qp, 1, ctrl);
 
 out:
@@ -303,7 +331,7 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	 * This will ensure that all the previous WRs got a completion before
 	 * we set the QP state to RESET.
 	 */
-	err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context->cqe, wqe,
+	err = mlx5r_umr_post_send(umrc->qp, mkey, umr_context, wqe,
 				  with_data);
 	if (err) {
 		mlx5_ib_warn(dev, "UMR recovery post send failed, err %d\n", err);
@@ -380,7 +408,7 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
 			continue;
 		}
 
-		err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context.cqe, wqe,
+		err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context, wqe,
 					  with_data);
 		mutex_unlock(&umrc->lock);
 		if (err) {
diff --git a/drivers/infiniband/hw/mlx5/umr.h b/drivers/infiniband/hw/mlx5/umr.h
index bda7123781a9..0b245eec521f 100644
--- a/drivers/infiniband/hw/mlx5/umr.h
+++ b/drivers/infiniband/hw/mlx5/umr.h
@@ -84,6 +84,12 @@ struct mlx5r_umr_context {
 	struct ib_cqe cqe;
 	enum ib_wc_status status;
 	struct completion done;
+	/*
+	 * SQ producer counter the WQE was posted at, matched against the CQE
+	 * wqe_counter in adversarial_mode. Unlike the context address, which
+	 * is reused across posts, or the ring slot, it identifies the WQE.
+	 */
+	u16 wqe_ctr;
 };
 
 struct mlx5r_umr_wqe {
-- 
2.18.1

[PATCH rdma-next 12/23] RDMA/mlx5: Verify UMR CQ polling against a hostile NIC

[Yishai Hadas]

The generic poll path trusts the CQE content: its opcode, QPN and
wqe_counter decide which request is completed and which waiter is woken.
A forged CQE can therefore complete a request the driver never posted,
or one that already completed. The driver then writes through a context
it no longer owns, which corrupts memory, as the context is usually on
the waiter's stack.

In adversarial mode, install a dedicated poll_one for the UMR CQ that
does not take the CQE's word for it. The driver keeps a shadow FIFO of
its posted requests, and a CQE is accepted only if its opcode and QPN
match the UMR QP and its full wqe_counter matches the oldest pending
request, one already passed to the doorbell. The context is then taken
from the FIFO, never from the CQE. This cannot prove that the NIC
really executed the request, as it can always report the oldest one as
completed; it ensures that the driver completes only a context it owns,
once and in order.

A CQE that fails validation proves the NIC is forging completions, so
nothing it reports on this QP can be trusted any more. Mark the UMR
path permanently failed, so neither a new post nor a QP recovery can
reopen it, stop interpreting this CQ and force the mlx5_core health
recovery. The forced variant is needed because such a NIC still looks
healthy to the fatal sensors. Recovery moves the device to the internal
error state, and the existing flush then completes the pending requests
in software and wakes their waiters, so this path does not complete
them itself.

On teardown, stop polling the CQ before destroying the UMR QP, so a CQE
for the destroyed QP, or for a reused QPN, is never interpreted.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c      | 156 +++++++++++++++++++++++++++
 drivers/infiniband/hw/mlx5/mlx5_ib.h |   5 +
 drivers/infiniband/hw/mlx5/umr.c     |  38 +++++++
 3 files changed, 199 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 11c1153747c2..1a98263ec311 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -37,6 +37,7 @@
 #include "mlx5_ib.h"
 #include "srq.h"
 #include "qp.h"
+#include "umr.h"
 
 #define UVERBS_MODULE_NAME mlx5_ib
 #include <rdma/uverbs_named_ioctl.h>
@@ -608,6 +609,161 @@ static int poll_soft_wc(struct mlx5_ib_cq *cq, int num_entries,
 	return npolled;
 }
 
+static int mlx5_ib_poll_one_stopped(struct mlx5_ib_cq *cq,
+				    struct mlx5_ib_qp **cur_qp, struct ib_wc *wc)
+{
+	return -EAGAIN;
+}
+
+/**
+ * mlx5_ib_cq_stop_polling - stop interpreting CQEs on a CQ
+ * @ibcq: the CQ
+ *
+ * Takes cq->lock, which mlx5_ib_poll_cq() holds while polling, so no poll
+ * is in flight on return and no CQE is looked at again.
+ */
+void mlx5_ib_cq_stop_polling(struct ib_cq *ibcq)
+{
+	struct mlx5_ib_cq *cq = to_mcq(ibcq);
+	unsigned long flags;
+
+	spin_lock_irqsave(&cq->lock, flags);
+	cq->poll_one = mlx5_ib_poll_one_stopped;
+	spin_unlock_irqrestore(&cq->lock, flags);
+}
+
+/*
+ * poll_one of the UMR CQ in adversarial_mode. A CQE is accepted only if it
+ * matches the oldest request in the shadow FIFO, otherwise the device is
+ * treated as compromised.
+ */
+int mlx5_ib_poll_one_umr(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+		      struct ib_wc *wc)
+{
+	struct mlx5_ib_dev *dev = to_mdev(cq->ibcq.device);
+	struct mlx5r_umr_context *expected;
+	struct umr_common *umrc = &dev->umrc;
+	struct mlx5_err_cqe err_cqe;
+	struct mlx5_cqe64 *cqe64;
+	struct mlx5_ib_wq *wq;
+	struct mlx5_ib_qp *qp;
+	struct ib_qp *ibqp;
+	u16 wqe_ctr;
+	u8 opcode;
+	u32 head;
+	u32 sop_qpn;
+	u32 qpn;
+	u8 sop;
+	int idx;
+	void *cqe;
+
+	cqe = next_cqe_sw(cq);
+	if (!cqe)
+		return -EAGAIN;
+
+	cqe64 = (cq->mcq.cqe_sz == 64) ? cqe : cqe + 64;
+	++cq->mcq.cons_index;
+	/*
+	 * Read each field once, so a NIC rewriting the CQE cannot change it
+	 * between validation and use.
+	 */
+	rmb();
+	opcode  = READ_ONCE(cqe64->op_own) >> 4;
+	sop_qpn = ntohl(READ_ONCE(cqe64->sop_drop_qpn));
+	qpn     = sop_qpn & 0xffffff;
+	sop     = sop_qpn >> 24;
+	wqe_ctr = be16_to_cpu(READ_ONCE(cqe64->wqe_counter));
+
+	/*
+	 * This poll_one is installed before the UMR QP is published, so a CQE
+	 * seen while the QP is NULL is forged. Teardown stops polling before
+	 * the QP is destroyed.
+	 */
+	ibqp = smp_load_acquire(&dev->umrc.qp);
+	if (unlikely(!ibqp)) {
+		mlx5_ib_err(dev, "CQE on the UMR CQ before its QP is published\n");
+		goto forged_cqe;
+	}
+	qp = to_mqp(ibqp);
+	wq = &qp->sq;
+	idx = wqe_ctr & (wq->wqe_cnt - 1);
+
+	if (unlikely((opcode != MLX5_CQE_REQ && opcode != MLX5_CQE_REQ_ERR) ||
+		     qpn != umrc->qpn || sop != MLX5_OPCODE_UMR)) {
+		mlx5_ib_err(dev,
+			    "Unexpected CQE on UMR CQ: opcode 0x%x sop 0x%x qpn 0x%x\n",
+			    opcode, sop, qpn);
+		goto forged_cqe;
+	}
+
+	/* Pairs with smp_store_release() in mlx5r_umr_post_send() */
+	head = umrc->pending_head;
+	expected = smp_load_acquire(&umrc->pending[head & (MLX5_UMR_MAX_WR - 1)]);
+
+	if (unlikely(!expected)) {
+		mlx5_ib_err(dev,
+			    "UMR CQE wqe_ctr 0x%x with an empty shadow FIFO, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	if (unlikely(wqe_ctr != expected->wqe_ctr)) {
+		mlx5_ib_err(dev,
+			    "UMR wqe_ctr 0x%x mismatch with expected context, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	/*
+	 * A WQE records wq->head when posted, and mlx5r_ring_db() then advances
+	 * wq->head. If they are still equal, mlx5r_ring_db() was not reached
+	 * yet, so the WQE was never handed to the NIC.
+	 */
+	if (unlikely(wq->wqe_head[idx] == READ_ONCE(wq->head))) {
+		mlx5_ib_err(dev,
+			    "UMR CQE wqe_ctr 0x%x for an uncommitted WQE, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	/* Pop the shadow FIFO head: validated, dispatch to its waiter */
+	WRITE_ONCE(umrc->pending[head & (MLX5_UMR_MAX_WR - 1)], NULL);
+	umrc->pending_head = head + 1;
+	wc->wr_cqe = &expected->cqe;
+
+	wc->qp = &qp->ibqp;
+	wq->tail = wq->wqe_head[idx] + 1;
+	/* Advance wq->last_poll, the internal error flush starts from it */
+	handle_atomics(qp, cqe64, wq->last_poll, idx);
+
+	if (opcode == MLX5_CQE_REQ) {
+		wc->status = IB_WC_SUCCESS;
+	} else {
+		/* Copy the CQE, so its syndrome is read once */
+		err_cqe = *(struct mlx5_err_cqe *)cqe64;
+		mlx5_handle_error_cqe(dev, &err_cqe, wc);
+		/* As in the generic path, a real failure starts the recovery */
+		if (wc->status != IB_WC_WR_FLUSH_ERR)
+			umrc->state = MLX5_UMR_STATE_RECOVER;
+	}
+
+	return 0;
+
+forged_cqe:
+	/* mlx5_ib_poll_cq() updates the CI only for a returned completion */
+	mlx5_cq_set_ci(&cq->mcq);
+	/*
+	 * The NIC forged a CQE. Fail the UMR path for good, ignore any later
+	 * CQE and force the health recovery, as the fatal sensors still read
+	 * clean. The internal error flush then completes the waiters. Return
+	 * an error, not -EAGAIN, as a CQE was consumed.
+	 */
+	umrc->state = MLX5_UMR_STATE_ERR;
+	cq->poll_one = mlx5_ib_poll_one_stopped;
+	mlx5_trigger_health_work_force(dev->mdev);
+	return -EIO;
+}
+
 int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 {
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 9cc4b2f99b69..f7c718e3d974 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -768,6 +768,8 @@ struct umr_common {
 	struct ib_pd	*pd;
 	struct ib_cq	*cq;
 	struct ib_qp	*qp;
+	/* Checked against each CQE in adversarial_mode */
+	u32		qpn;
 	/* Protects from UMR QP overflow
 	 */
 	struct semaphore	sem;
@@ -1311,6 +1313,9 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 			   struct uverbs_attr_bundle *attrs);
 int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
 int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
+int mlx5_ib_poll_one_umr(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+		      struct ib_wc *wc);
+void mlx5_ib_cq_stop_polling(struct ib_cq *ibcq);
 int mlx5_ib_pre_destroy_cq(struct ib_cq *cq);
 void mlx5_ib_post_destroy_cq(struct ib_cq *cq);
 int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index aba79a19d627..b0f00d4bc93c 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -194,6 +194,23 @@ int mlx5r_umr_resource_init(struct mlx5_ib_dev *dev)
 	sema_init(&dev->umrc.sem, MAX_UMR_WR);
 	mutex_init(&dev->umrc.lock);
 	dev->umrc.state = MLX5_UMR_STATE_ACTIVE;
+
+	if (mlx5_ib_adversarial_mode) {
+		struct mlx5_ib_cq *mcq = to_mcq(cq);
+		unsigned long flags;
+
+		dev->umrc.qpn = qp->qp_num;
+		/*
+		 * UMRs can be posted as soon as the QP is published below, by
+		 * this caller or by a concurrent MR registration, so install
+		 * mlx5_ib_poll_one_umr() first. The CQ is already armed, so
+		 * swap it under cq->lock, which the poll loop holds.
+		 */
+		spin_lock_irqsave(&mcq->lock, flags);
+		mcq->poll_one = mlx5_ib_poll_one_umr;
+		spin_unlock_irqrestore(&mcq->lock, flags);
+	}
+
 	smp_store_release(&dev->umrc.qp, qp);
 
 	mutex_unlock(&dev->umrc.init_lock);
@@ -212,6 +229,14 @@ void mlx5r_umr_resource_cleanup(struct mlx5_ib_dev *dev)
 {
 	if (dev->umrc.state == MLX5_UMR_STATE_UNINIT)
 		return;
+
+	/*
+	 * Stop polling before the QP is destroyed, as its QPN may then be
+	 * reused and a forged CQE carrying it must not be interpreted.
+	 */
+	if (mlx5_ib_adversarial_mode)
+		mlx5_ib_cq_stop_polling(dev->umrc.cq);
+
 	mutex_destroy(&dev->umrc.lock);
 	/* After device init, UMR cp/qp are not unset during the lifetime. */
 	ib_destroy_qp(dev->umrc.qp);
@@ -319,6 +344,11 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	int err;
 
 	mutex_lock(&umrc->lock);
+	if (mlx5_ib_adversarial_mode && umrc->state == MLX5_UMR_STATE_ERR) {
+		/* The NIC forged a CQE, don't revive the QP */
+		mutex_unlock(&umrc->lock);
+		return -EFAULT;
+	}
 	/* Preventing any further WRs to be sent now */
 	if (umrc->state != MLX5_UMR_STATE_RECOVER) {
 		mlx5_ib_warn(dev, "UMR recovery encountered an unexpected state=%d\n",
@@ -425,6 +455,14 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
 		if (umr_context.status == IB_WC_WR_FLUSH_ERR)
 			continue;
 
+		if (mlx5_ib_adversarial_mode) {
+			/* A forged CQE was detected, don't recover the QP */
+			if (umrc->state == MLX5_UMR_STATE_ERR) {
+				err = -EFAULT;
+				break;
+			}
+		}
+
 		WARN_ON_ONCE(1);
 		mlx5_ib_warn(dev,
 			"reg umr failed (%u). Trying to recover and resubmit the flushed WQEs, mkey = %u\n",
-- 
2.18.1

[PATCH rdma-next 13/23] RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected

[Yishai Hadas]

In adversarial mode MLX5_UMR_STATE_ERR is a final state: the poller sets
it when it rejects a forged CQE, after which hardware polling is stopped,
and it stops mlx5r_umr_post_send_wait() from posting again.

mlx5r_umr_recover() runs concurrently with that and moves the state with
plain stores: it enters RECOVER from whatever state the path is in and
ends with an unconditional ACTIVE. An ERR set while a recovery is in
flight is therefore overwritten, which opens the UMR path again for a
NIC that was just caught lying, with its poller already stopped.

In adversarial mode move the state with cmpxchg() in recovery, ACTIVE to
RECOVER on entry and RECOVER to ACTIVE at the end. If ERR was set in the
meantime the transition leaves it in place and recovery returns
-EFAULT. Outside adversarial mode the transitions are unchanged.

This keeps new UMRs away from the NIC until the forced health recovery
moves the device to MLX5_DEVICE_STATE_INTERNAL_ERROR, after which posts
fail with -EIO anyway.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/umr.c | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index b0f00d4bc93c..086b397b830c 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -335,6 +335,23 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey,
 	return err;
 }
 
+/*
+ * Move the UMR state from @from to @to. In adversarial_mode ERR is a final
+ * state: it can be set concurrently by the CQ poller, so the move is atomic
+ * and does not replace it. Returns false if ERR was found, and true
+ * otherwise. Outside adversarial_mode this is the plain store it always was.
+ */
+static bool mlx5r_umr_move_state(struct umr_common *umrc, unsigned int from,
+				 unsigned int to)
+{
+	if (!mlx5_ib_adversarial_mode) {
+		umrc->state = to;
+		return true;
+	}
+
+	return cmpxchg(&umrc->state, from, to) != MLX5_UMR_STATE_ERR;
+}
+
 static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 			     struct mlx5r_umr_context *umr_context,
 			     struct mlx5r_umr_wqe *wqe, bool with_data)
@@ -353,7 +370,11 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	if (umrc->state != MLX5_UMR_STATE_RECOVER) {
 		mlx5_ib_warn(dev, "UMR recovery encountered an unexpected state=%d\n",
 			     umrc->state);
-		umrc->state = MLX5_UMR_STATE_RECOVER;
+		if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_ACTIVE,
+					  MLX5_UMR_STATE_RECOVER)) {
+			mutex_unlock(&umrc->lock);
+			return -EFAULT;
+		}
 	}
 	mutex_unlock(&umrc->lock);
 
@@ -387,7 +408,9 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 		goto err;
 	}
 
-	umrc->state = MLX5_UMR_STATE_ACTIVE;
+	if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_RECOVER,
+				  MLX5_UMR_STATE_ACTIVE))
+		return -EFAULT;
 	return 0;
 
 err:
-- 
2.18.1

[PATCH rdma-next 14/23] RDMA/mlx5: Block generic CQ poll path in adversarial mode

[Yishai Hadas]

mlx5_poll_one() trusts the CQE content, so it must not run on a CQE
from an untrusted NIC. In adversarial mode no CQ needs it: the UMR CQ
has its own poll_one that validates the CQE, kernel ULPs don't bind to
the device, and user CQs are polled by userspace. Refuse it there, so
an unexpected caller, such as the uverbs POLL_CQ command, finds the CQ
empty and no CQE is interpreted.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 1a98263ec311..5cc86c3f93f3 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -464,6 +464,15 @@ static int mlx5_poll_one(struct mlx5_ib_cq *cq,
 	void *cqe;
 	int idx;
 
+	/*
+	 * This path trusts the CQE, so refuse it in adversarial_mode. Only an
+	 * unexpected caller gets here, e.g. the uverbs POLL_CQ command or an
+	 * event on the UMR CQ before its own poll_one is installed. Nothing is
+	 * consumed, so such a CQE is left for that poll_one.
+	 */
+	if (unlikely(mlx5_ib_adversarial_mode))
+		return -EOPNOTSUPP;
+
 repoll:
 	cqe = next_cqe_sw(cq);
 	if (!cqe)
-- 
2.18.1

[PATCH rdma-next 15/23] RDMA/core: Set the CQ comp_handler after its poll context is set up

[Yishai Hadas]

__ib_alloc_cq() sets comp_handler before it initializes the poll context
the handler schedules, and for the workqueue contexts it arms the CQ
before setting comp_wq. The CQ is reachable from the completion
interrupt once ops.create_cq() returns, so a completion event raised in
that window would schedule an uninitialized irq_poll or work, or queue
work on a NULL workqueue. A conforming device raises no event for a new,
empty CQ, but nothing in the core relies on that.

Initialize the poll context first, then set comp_handler, and arm the CQ
only after that.

Fixes: 14d3a3b2498e ("IB: add a proper completion queue abstraction")
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/cq.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c
index 12304c9a9403..42e4d272f5ad 100644
--- a/drivers/infiniband/core/cq.c
+++ b/drivers/infiniband/core/cq.c
@@ -217,6 +217,7 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 		.cqe		= nr_cqe,
 		.comp_vector	= comp_vector,
 	};
+	ib_comp_handler comp_handler;
 	struct ib_cq *cq;
 	int ret = -ENOMEM;
 
@@ -248,19 +249,16 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 
 	switch (cq->poll_ctx) {
 	case IB_POLL_DIRECT:
-		cq->comp_handler = ib_cq_completion_direct;
+		comp_handler = ib_cq_completion_direct;
 		break;
 	case IB_POLL_SOFTIRQ:
-		cq->comp_handler = ib_cq_completion_softirq;
-
+		comp_handler = ib_cq_completion_softirq;
 		irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler);
-		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 		break;
 	case IB_POLL_WORKQUEUE:
 	case IB_POLL_UNBOUND_WORKQUEUE:
-		cq->comp_handler = ib_cq_completion_workqueue;
+		comp_handler = ib_cq_completion_workqueue;
 		INIT_WORK(&cq->work, ib_cq_poll_work);
-		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 		cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ?
 				ib_comp_wq : ib_comp_unbound_wq;
 		break;
@@ -269,6 +267,15 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 		goto out_destroy_cq;
 	}
 
+	/*
+	 * The CQ is reachable from the completion interrupt once created. Set
+	 * the handler only after the poll context it schedules is set up, and
+	 * arm the CQ only after that.
+	 */
+	cq->comp_handler = comp_handler;
+	if (cq->poll_ctx != IB_POLL_DIRECT)
+		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
+
 	rdma_restrack_add(&cq->res);
 	trace_cq_alloc(cq, nr_cqe, comp_vector, poll_ctx);
 	return cq;
-- 
2.18.1

[PATCH rdma-next 16/23] RDMA/mlx5: Ignore completion events for a CQ without a handler

[Yishai Hadas]

mlx5_ib_cq_comp() calls the CQ comp_handler unconditionally, but
devr->c0, created by mlx5_ib_dev_res_cq_init() on the first SRQ create,
has none, as it is never armed.

A trusted NIC raises no completion event for a CQ that is not armed, but
a hostile NIC in adversarial mode can do so with any CQ number it
returned, which ends in a NULL call in interrupt context.

Skip the call when no handler is set.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 5cc86c3f93f3..f955eb762051 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -46,7 +46,12 @@ static void mlx5_ib_cq_comp(struct mlx5_core_cq *cq, struct mlx5_eqe *eqe)
 {
 	struct ib_cq *ibcq = &to_mibcq(cq)->ibcq;
 
-	ibcq->comp_handler(ibcq, ibcq->cq_context);
+	/*
+	 * A CQ created without a handler (devr->c0) is never armed, but a
+	 * hostile NIC can still raise a completion event for it.
+	 */
+	if (ibcq->comp_handler)
+		ibcq->comp_handler(ibcq, ibcq->cq_context);
 }
 
 static void mlx5_ib_cq_event(struct mlx5_core_cq *mcq, enum mlx5_event type)
-- 
2.18.1

[PATCH rdma-next 17/23] RDMA/mlx5: Validate EQE port and GSI port before indexing devr.ports[]

[Yishai Hadas]

devr.ports[] has 2 entries, but it is indexed by port numbers taken
from the NIC, which adversarial mode does not trust.

handle_port_change() uses the EQE port before any validation, so port 0
wraps the index to UINT_MAX and a port above 2 overflows the array.
mlx5_ib_create_gsi() gets a port already checked against phys_port_cnt,
but that comes from the num_ports and num_vhca_ports FW caps, which may
report more than 2 ports.

Validate the port and bound it by ARRAY_SIZE() in both paths.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/gsi.c  | 4 ++++
 drivers/infiniband/hw/mlx5/main.c | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/gsi.c b/drivers/infiniband/hw/mlx5/gsi.c
index b2e2a219639d..7ce390a2820f 100644
--- a/drivers/infiniband/hw/mlx5/gsi.c
+++ b/drivers/infiniband/hw/mlx5/gsi.c
@@ -95,6 +95,10 @@ int mlx5_ib_create_gsi(struct ib_pd *pd, struct mlx5_ib_qp *mqp,
 	int num_qps = 0;
 	int ret;
 
+	if (!rdma_is_port_valid(&dev->ib_dev, port_num) ||
+	    port_num > ARRAY_SIZE(dev->devr.ports))
+		return -EINVAL;
+
 	if (mlx5_ib_deth_sqpn_cap(dev)) {
 		if (MLX5_CAP_GEN(dev->mdev,
 				 port_type) == MLX5_CAP_PORT_TYPE_IB)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index f4f2ad80e76c..c29102260989 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3063,6 +3063,10 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
 {
 	u32 port = (eqe->data.port.port >> 4) & 0xf;
 
+	if (!rdma_is_port_valid(&ibdev->ib_dev, port) ||
+	    port > ARRAY_SIZE(ibdev->devr.ports))
+		return -EINVAL;
+
 	ibev->element.port_num = port;
 
 	switch (eqe->sub_type) {
-- 
2.18.1

[PATCH rdma-next 18/23] RDMA/mlx5: Validate firmware RQ/SQ state before sqrq_trans[] lookup

[Yishai Hadas]

sqrq_state_to_qp_state() indexes the 5x5 sqrq_trans[] table with the RQ
and SQ states returned by QUERY_RQ and QUERY_SQ. The state fields are 4
bits wide, so an untrusted NIC can return a state above 4 and the lookup
reads past the table.

Reject out of range states with -EIO before the lookup.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 19cbf6b2b979..05e8e75d734f 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -5081,6 +5081,9 @@ static int sqrq_state_to_qp_state(u8 sq_state, u8 rq_state,
 		},
 	};
 
+	if (sq_state >= MLX5_SQ_NUM_STATE || rq_state >= MLX5_RQ_NUM_STATE)
+		return -EIO;
+
 	*qp_state = sqrq_trans[rq_state][sq_state];
 
 	if (*qp_state == MLX5_QP_STATE_BAD) {
-- 
2.18.1

[PATCH rdma-next 19/23] RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks

[Yishai Hadas]

The CQ create and resize paths check the requested size against
1 << log_max_cq_sz. The cap is 8 bits wide, so an untrusted NIC can
report 31 or more, which makes the shift undefined. A large limit also
lets the rounded up CQ size exceed INT_MAX, and it is then truncated
when passed on as an int.

Take the limit from MLX5_CAP_GEN_LOGSZ_U64(), round up in 64 bits, and
reject sizes above INT_MAX.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 39 +++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index f955eb762051..25344dfba1c1 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -1117,6 +1117,15 @@ static void notify_soft_wc_handler(struct work_struct *work)
 	cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
 }
 
+/*
+ * Next power of 2 above entries. It can be 2^32, which doesn't fit the
+ * unsigned long of roundup_pow_of_two() on 32-bit archs.
+ */
+static u64 cq_roundup_entries(u64 entries)
+{
+	return 1ULL << fls64(entries);
+}
+
 int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 			   const struct ib_cq_init_attr *attr,
 			   struct uverbs_attr_bundle *attrs)
@@ -1124,7 +1133,7 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 	struct ib_udata *udata = &attrs->driver_udata;
 	struct ib_device *ibdev = ibcq->device;
 	struct mlx5_ib_create_cq_resp uresp = {};
-	int entries = attr->cqe;
+	u64 entries = attr->cqe;
 	int vector = attr->comp_vector;
 	struct mlx5_ib_dev *dev = to_mdev(ibdev);
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
@@ -1136,15 +1145,17 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 	int cqe_size;
 	int eqn;
 	int err;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
-	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (attr->cqe > max_cq_entries)
 		return -EINVAL;
 
 	if (check_cq_create_flags(attr->flags))
 		return -EOPNOTSUPP;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	entries = cq_roundup_entries(entries);
+	/* cq->ibcq.cqe and create_cq_user() take an int */
+	if (entries > max_cq_entries || entries > INT_MAX)
 		return -EINVAL;
 
 	cq->ibcq.cqe = entries - 1;
@@ -1214,7 +1225,7 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 		      struct uverbs_attr_bundle *attrs)
 {
 	struct ib_device *ibdev = ibcq->device;
-	int entries = attr->cqe;
+	u64 entries = attr->cqe;
 	int vector = attr->comp_vector;
 	struct mlx5_ib_dev *dev = to_mdev(ibdev);
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
@@ -1226,12 +1237,14 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	int cqe_size;
 	int eqn;
 	int err;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
-	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (attr->cqe > max_cq_entries)
 		return -EINVAL;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	entries = cq_roundup_entries(entries);
+	/* cq->ibcq.cqe and create_cq_kernel() take an int */
+	if (entries > max_cq_entries || entries > INT_MAX)
 		return -EINVAL;
 
 	cq->ibcq.cqe = entries - 1;
@@ -1532,18 +1545,22 @@ int mlx5_ib_resize_cq(struct ib_cq *ibcq, unsigned int entries,
 	int inlen;
 	int cqe_size;
 	unsigned long flags;
+	u64 entries64 = entries;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
 	if (!MLX5_CAP_GEN(dev->mdev, cq_resize)) {
 		pr_info("Firmware does not support resize CQ\n");
 		return -ENOSYS;
 	}
 
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (entries64 > max_cq_entries)
 		return -EINVAL;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)) + 1)
+	entries64 = cq_roundup_entries(entries64);
+	/* resize_user() and resize_kernel() take an int */
+	if (entries64 > max_cq_entries + 1 || entries64 > INT_MAX)
 		return -EINVAL;
+	entries = entries64;
 
 	if (entries == ibcq->cqe + 1)
 		return 0;
-- 
2.18.1

[PATCH rdma-next 20/23] RDMA/mlx5: Clamp log_max_* shifts in QP and WQ limit checks

[Yishai Hadas]

Several QP and WQ size checks in qp.c compute their limit as
1 << cap, where cap is an 8-bit log_max_* FW capability. An untrusted
NIC can report 31 or more, which makes the shift undefined.

Use MLX5_CAP_GEN_LOGSZ_U32() for these limits, as done for
log_max_cq_sz.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 36 +++++++++++++++------------------
 1 file changed, 16 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 05e8e75d734f..d784a35675d5 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -447,7 +447,7 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,
 	int wq_size;
 
 	/* Sanity check RQ size before proceeding */
-	if (cap->max_recv_wr > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz)))
+	if (cap->max_recv_wr > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	if (!has_rq) {
@@ -621,11 +621,11 @@ static int calc_sq_size(struct mlx5_ib_dev *dev, struct ib_qp_init_attr *attr,
 
 	wq_size = roundup_pow_of_two(attr->cap.max_send_wr * wqe_size);
 	qp->sq.wqe_cnt = wq_size / MLX5_SEND_WQE_BB;
-	if (qp->sq.wqe_cnt > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz))) {
-		mlx5_ib_dbg(dev, "send queue size (%d * %d / %d -> %d) exceeds limits(%d)\n",
+	if (qp->sq.wqe_cnt > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz)) {
+		mlx5_ib_dbg(dev, "send queue size (%d * %d / %d -> %d) exceeds limits(%u)\n",
 			    attr->cap.max_send_wr, wqe_size, MLX5_SEND_WQE_BB,
 			    qp->sq.wqe_cnt,
-			    1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz));
+			    MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz));
 		return -ENOMEM;
 	}
 	qp->sq.wqe_shift = ilog2(MLX5_SEND_WQE_BB);
@@ -663,10 +663,10 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 
 	qp->sq.wqe_cnt = ucmd->sq_wqe_count;
 
-	if (qp->sq.wqe_cnt > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz))) {
-		mlx5_ib_warn(dev, "wqe_cnt %d, max_wqes %d\n",
+	if (qp->sq.wqe_cnt > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz)) {
+		mlx5_ib_warn(dev, "wqe_cnt %d, max_wqes %u\n",
 			     qp->sq.wqe_cnt,
-			     1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz));
+			     MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz));
 		return -EINVAL;
 	}
 
@@ -2155,7 +2155,7 @@ static int create_dci(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	    ucmd->rq_wqe_count != qp->rq.wqe_cnt)
 		return -EINVAL;
 
-	if (ucmd->sq_wqe_count > (1 << MLX5_CAP_GEN(mdev, log_max_qp_sz)))
+	if (ucmd->sq_wqe_count > MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	ts_format = get_qp_ts_format(dev, to_mcq(init_attr->send_cq),
@@ -2322,7 +2322,7 @@ static int create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	    ucmd->rq_wqe_count != qp->rq.wqe_cnt)
 		return -EINVAL;
 
-	if (ucmd->sq_wqe_count > (1 << MLX5_CAP_GEN(mdev, log_max_qp_sz)))
+	if (ucmd->sq_wqe_count > MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	if (init_attr->qp_type != IB_QPT_RAW_PACKET) {
@@ -4809,19 +4809,15 @@ static bool mlx5_ib_modify_qp_allowed(struct mlx5_ib_dev *dev,
 static int validate_rd_atomic(struct mlx5_ib_dev *dev, struct ib_qp_attr *attr,
 			      int attr_mask, enum ib_qp_type qp_type)
 {
-	int log_max_ra_res;
-	int log_max_ra_req;
+	u32 log_max_ra_res;
+	u32 log_max_ra_req;
 
 	if (qp_type == MLX5_IB_QPT_DCI) {
-		log_max_ra_res = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_res_dc);
-		log_max_ra_req = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_req_dc);
+		log_max_ra_res = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_res_dc);
+		log_max_ra_req = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_req_dc);
 	} else {
-		log_max_ra_res = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_res_qp);
-		log_max_ra_req = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_req_qp);
+		log_max_ra_res = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_res_qp);
+		log_max_ra_req = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_req_qp);
 	}
 
 	if (attr_mask & IB_QP_MAX_QP_RD_ATOMIC &&
@@ -5511,7 +5507,7 @@ static int set_user_rq_size(struct mlx5_ib_dev *dev,
 			    struct mlx5_ib_rwq *rwq)
 {
 	/* Sanity check RQ size before proceeding */
-	if (wq_init_attr->max_wr > (1 << MLX5_CAP_GEN(dev->mdev, log_max_wq_sz)))
+	if (wq_init_attr->max_wr > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_wq_sz))
 		return -EINVAL;
 
 	if (!ucmd->rq_wqe_count)
-- 
2.18.1

[PATCH rdma-next 21/23] RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths

[Yishai Hadas]

The device, port and ucontext attributes and the SRQ size limit are
computed as 1 << cap from 8-bit log_max_* FW capabilities. An untrusted
NIC can report 31 or more, which makes the shift undefined. The same
applies to 1ull << log_pg_sz with a value of 64 or more.

Use MLX5_CAP_GEN_LOGSZ_U32() and MLX5_CAP_GEN_LOGSZ_U64() for these
values. As the limits can now reach 2^31, compute max_qp_wr in 64 bits
and saturate the max_res_rd_atom and max_total_mcast_qp_attach products
at U32_MAX.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mad.c  |  2 +-
 drivers/infiniband/hw/mlx5/main.c | 63 ++++++++++++++++---------------
 drivers/infiniband/hw/mlx5/srq.c  |  2 +-
 3 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mad.c b/drivers/infiniband/hw/mlx5/mad.c
index 8925331855ff..cf9f6a4d5ccf 100644
--- a/drivers/infiniband/hw/mlx5/mad.c
+++ b/drivers/infiniband/hw/mlx5/mad.c
@@ -629,7 +629,7 @@ int mlx5_query_mad_ifc_port(struct ib_device *ibdev, u32 port,
 	props->phys_state	= out_mad->data[33] >> 4;
 	props->port_cap_flags	= be32_to_cpup((__be32 *)(out_mad->data + 20));
 	props->gid_tbl_len	= out_mad->data[50];
-	props->max_msg_sz	= 1 << MLX5_CAP_GEN(mdev, log_max_msg);
+	props->max_msg_sz	= MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_msg);
 	props->pkey_tbl_len	= dev->pkey_table_len;
 	props->bad_pkey_cntr	= be16_to_cpup((__be16 *)(out_mad->data + 46));
 	props->qkey_viol_cntr	= be16_to_cpup((__be16 *)(out_mad->data + 48));
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index c29102260989..ace4cdc95c9d 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -599,7 +599,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num,
 		props->qkey_viol_cntr = qkey_viol_cntr;
 	}
 	props->max_mtu          = IB_MTU_4096;
-	props->max_msg_sz       = 1 << MLX5_CAP_GEN(dev->mdev, log_max_msg);
+	props->max_msg_sz       = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_msg);
 	props->pkey_tbl_len     = 1;
 	props->state            = IB_PORT_DOWN;
 	props->phys_state       = IB_PORT_PHYS_STATE_DISABLED;
@@ -930,7 +930,7 @@ static u32 mlx5_ib_calc_max_sq_overhead(void)
 static u32 mlx5_ib_calc_max_qp_wr(struct mlx5_ib_dev *dev)
 {
 	struct mlx5_core_dev *mdev = dev->mdev;
-	u32 max_wqe_bb_units = 1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
+	u64 max_wqe_bb_units = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz);
 	u32 max_wqe_size;
 	/* max QP overhead + 1 SGE, no inline, no special features */
 	max_wqe_size = mlx5_ib_calc_max_sq_overhead() +
@@ -940,7 +940,8 @@ static u32 mlx5_ib_calc_max_qp_wr(struct mlx5_ib_dev *dev)
 
 	max_wqe_size = ALIGN(max_wqe_size, MLX5_SEND_WQE_BB);
 
-	return (max_wqe_bb_units * MLX5_SEND_WQE_BB) / max_wqe_size;
+	/* Up to 2^31 units, so the product needs 64 bits */
+	return div_u64(max_wqe_bb_units * MLX5_SEND_WQE_BB, max_wqe_size);
 }
 
 static int mlx5_ib_query_device(struct ib_device *ibdev,
@@ -954,7 +955,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	int max_sq_desc;
 	int max_rq_sg;
 	int max_sq_sg;
-	u64 min_page_size = 1ull << MLX5_CAP_GEN(mdev, log_pg_sz);
+	u64 min_page_size = MLX5_CAP_GEN_LOGSZ_U64(mdev, log_pg_sz);
 	bool raw_support = !mlx5_core_mp_enabled(mdev);
 	struct mlx5_ib_query_device_resp resp = {};
 	size_t resp_len;
@@ -1000,7 +1001,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	if (MLX5_CAP_GEN(mdev, imaicl)) {
 		props->device_cap_flags |= IB_DEVICE_MEM_WINDOW |
 					   IB_DEVICE_MEM_WINDOW_TYPE_2B;
-		props->max_mw = 1 << MLX5_CAP_GEN(mdev, log_max_mkey);
+		props->max_mw = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mkey);
 		/* We support 'Gappy' memory registration too */
 		props->kernel_cap_flags |= IBK_SG_GAPS_REG;
 	}
@@ -1033,7 +1034,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 		if (offsetofend(typeof(resp), tso_caps) <= uhw_outlen) {
 			max_tso = MLX5_CAP_ETH(mdev, max_lso_cap);
 			if (max_tso) {
-				resp.tso_caps.max_tso = 1 << max_tso;
+				resp.tso_caps.max_tso = BIT(max_tso);
 				resp.tso_caps.supported_qpts |=
 					1 << IB_QPT_RAW_PACKET;
 				resp.response_length += sizeof(resp.tso_caps);
@@ -1100,7 +1101,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 
 	props->max_mr_size	   = ~0ull;
 	props->page_size_cap	   = ~(min_page_size - 1);
-	props->max_qp		   = 1 << MLX5_CAP_GEN(mdev, log_max_qp);
+	props->max_qp		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp);
 	props->max_qp_wr = mlx5_ib_calc_max_qp_wr(dev);
 	max_rq_sg =  MLX5_CAP_GEN(mdev, max_wqe_sz_rq) /
 		     sizeof(struct mlx5_wqe_data_seg);
@@ -1111,29 +1112,31 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	props->max_send_sge = max_sq_sg;
 	props->max_recv_sge = max_rq_sg;
 	props->max_sge_rd	   = MLX5_MAX_SGE_RD;
-	props->max_cq		   = 1 << MLX5_CAP_GEN(mdev, log_max_cq);
-	props->max_cqe = (1 << MLX5_CAP_GEN(mdev, log_max_cq_sz)) - 1;
-	props->max_mr		   = 1 << MLX5_CAP_GEN(mdev, log_max_mkey);
-	props->max_pd		   = 1 << MLX5_CAP_GEN(mdev, log_max_pd);
-	props->max_qp_rd_atom	   = 1 << MLX5_CAP_GEN(mdev, log_max_ra_req_qp);
-	props->max_qp_init_rd_atom = 1 << MLX5_CAP_GEN(mdev, log_max_ra_res_qp);
-	props->max_srq		   = 1 << MLX5_CAP_GEN(mdev, log_max_srq);
-	props->max_srq_wr = (1 << MLX5_CAP_GEN(mdev, log_max_srq_sz)) - 1;
+	props->max_cq		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_cq);
+	props->max_cqe = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_cq_sz) - 1;
+	props->max_mr		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mkey);
+	props->max_pd		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_pd);
+	props->max_qp_rd_atom	   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_ra_req_qp);
+	props->max_qp_init_rd_atom = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_ra_res_qp);
+	props->max_srq		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_srq);
+	props->max_srq_wr = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_srq_sz) - 1;
 	props->local_ca_ack_delay  = MLX5_CAP_GEN(mdev, local_ca_ack_delay);
-	props->max_res_rd_atom	   = props->max_qp_rd_atom * props->max_qp;
+	props->max_res_rd_atom	   = min_t(u64, (u64)props->max_qp_rd_atom * props->max_qp,
+					   U32_MAX);
 	props->max_srq_sge	   = max_rq_sg - 1;
 	props->max_fast_reg_page_list_len =
-		1 << MLX5_CAP_GEN(mdev, log_max_klm_list_size);
+		MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_klm_list_size);
 	props->max_pi_fast_reg_page_list_len =
 		props->max_fast_reg_page_list_len / 2;
 	props->max_sgl_rd =
 		MLX5_CAP_GEN(mdev, max_sgl_for_optimized_performance);
 	get_atomic_caps_qp(dev, props);
 	props->masked_atomic_cap   = IB_ATOMIC_NONE;
-	props->max_mcast_grp	   = 1 << MLX5_CAP_GEN(mdev, log_max_mcg);
+	props->max_mcast_grp	   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mcg);
 	props->max_mcast_qp_attach = MLX5_CAP_GEN(mdev, max_qp_mcg);
-	props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
-					   props->max_mcast_grp;
+	props->max_total_mcast_qp_attach =
+		min_t(u64, (u64)props->max_mcast_qp_attach * props->max_mcast_grp,
+		      U32_MAX);
 	props->max_ah = INT_MAX;
 	props->hca_core_clock = MLX5_CAP_GEN(mdev, device_frequency_khz);
 	props->timestamp_mask = 0x7FFFFFFFFFFFFFFFULL;
@@ -1167,19 +1170,19 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	if (mlx5_ib_port_link_layer(ibdev, 1) ==
 	    IB_LINK_LAYER_ETHERNET && raw_support) {
 		props->rss_caps.max_rwq_indirection_tables =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rqt);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rqt);
 		props->rss_caps.max_rwq_indirection_table_size =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rqt_size);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rqt_size);
 		props->rss_caps.supported_qpts = 1 << IB_QPT_RAW_PACKET;
 		props->max_wq_type_rq =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rq);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rq);
 	}
 
 	if (MLX5_CAP_GEN(mdev, tag_matching)) {
 		props->tm_caps.max_num_tags =
-			(1 << MLX5_CAP_GEN(mdev, log_tag_matching_list_sz)) - 1;
+			MLX5_CAP_GEN_LOGSZ_U32(mdev, log_tag_matching_list_sz) - 1;
 		props->tm_caps.max_ops =
-			1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
+			MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz);
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
@@ -1503,7 +1506,7 @@ static int mlx5_query_hca_port(struct ib_device *ibdev, u32 port,
 		props->port_cap_flags &= ~IB_PORT_CM_SUP;
 
 	props->gid_tbl_len	= mlx5_get_gid_table_len(MLX5_CAP_GEN(mdev, gid_table_size));
-	props->max_msg_sz	= 1 << MLX5_CAP_GEN(mdev, log_max_msg);
+	props->max_msg_sz	= MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_msg);
 	props->pkey_tbl_len	= mlx5_to_sw_pkey_sz(MLX5_CAP_GEN(mdev, pkey_table_size));
 	props->bad_pkey_cntr	= rep->pkey_violation_counter;
 	props->qkey_viol_cntr	= rep->qkey_violation_counter;
@@ -2127,16 +2130,16 @@ static int set_ucontext_resp(struct ib_ucontext *uctx,
 			MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DUMP_FILL_MKEY;
 	}
 
-	resp->qp_tab_size = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp);
+	resp->qp_tab_size = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp);
 	if (mlx5_wc_support_get(dev->mdev))
 		resp->bf_reg_size = 1 << MLX5_CAP_GEN(dev->mdev,
 						      log_bf_reg_size);
 	resp->cache_line_size = cache_line_size();
 	resp->max_sq_desc_sz = MLX5_CAP_GEN(dev->mdev, max_wqe_sz_sq);
 	resp->max_rq_desc_sz = MLX5_CAP_GEN(dev->mdev, max_wqe_sz_rq);
-	resp->max_send_wqebb = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
-	resp->max_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
-	resp->max_srq_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
+	resp->max_send_wqebb = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz);
+	resp->max_recv_wr = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz);
+	resp->max_srq_recv_wr = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_srq_sz);
 	resp->cqe_version = context->cqe_version;
 	resp->log_uar_size = MLX5_CAP_GEN(dev->mdev, uar_4k) ?
 				MLX5_ADAPTER_PAGE_SHIFT : PAGE_SHIFT;
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index 6a4a937d97ce..3fbb0eaf1255 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -198,7 +198,7 @@ int mlx5_ib_create_srq(struct ib_srq *ib_srq,
 	size_t buf_size;
 	int err;
 	struct mlx5_srq_attr in = {};
-	__u32 max_srq_wqes = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
+	__u32 max_srq_wqes = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_srq_sz);
 	__u32 max_sge_sz =  MLX5_CAP_GEN(dev->mdev, max_wqe_sz_rq) /
 			    sizeof(struct mlx5_wqe_data_seg);
 
-- 
2.18.1

[PATCH rdma-next 22/23] RDMA/core: Report and enforce adversarial mode via GET_CONTEXT

[Yishai Hadas]

Userspace must know when a device is in adversarial mode, and an
unaware rdma-core must not get a context on such a device, as it
doesn't harden itself against the NIC.

Report the mode with IB_UVERBS_CORE_SUPPORT_ADVERSARIAL in the
core_support of GET_CONTEXT and QUERY_CONTEXT. Add an optional
UVERBS_ATTR_GET_CONTEXT_FLAGS input, where userspace declares
UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED. In adversarial
mode, GET_CONTEXT without that flag fails with -EPERM, and so does the
legacy write() GET_CONTEXT, which has no way to pass it.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/uverbs_cmd.c             |  5 +++++
 .../infiniband/core/uverbs_std_types_device.c    | 16 ++++++++++++++++
 include/uapi/rdma/ib_user_ioctl_cmds.h           |  5 +++++
 include/uapi/rdma/ib_user_ioctl_verbs.h          |  1 +
 4 files changed, 27 insertions(+)

diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index aca7c6ab55cd..b78207a81091 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -292,6 +292,11 @@ static int ib_uverbs_get_context(struct uverbs_attr_bundle *attrs)
 		goto err_ucontext;
 	}
 
+	if (ib_dev->adversarial_mode) {
+		ret = -EPERM;
+		goto err_uobj;
+	}
+
 	resp = (struct ib_uverbs_get_context_resp){
 		.num_comp_vectors = attrs->ufile->device->num_comp_vectors,
 		.async_fd = uobj->id,
diff --git a/drivers/infiniband/core/uverbs_std_types_device.c b/drivers/infiniband/core/uverbs_std_types_device.c
index db65032e80b4..72c67f09533e 100644
--- a/drivers/infiniband/core/uverbs_std_types_device.c
+++ b/drivers/infiniband/core/uverbs_std_types_device.c
@@ -248,6 +248,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 	u32 num_comp = attrs->ufile->device->num_comp_vectors;
 	u64 core_support = IB_UVERBS_CORE_SUPPORT_OPTIONAL_MR_ACCESS;
 	struct ib_device *ib_dev;
+	u32 ctx_flags;
 	int ret;
 
 	ib_dev = srcu_dereference(attrs->ufile->device->ib_dev,
@@ -255,6 +256,15 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 	if (!ib_dev)
 		return -EIO;
 
+	ret = uverbs_get_flags32(&ctx_flags, attrs, UVERBS_ATTR_GET_CONTEXT_FLAGS,
+				 UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED);
+	if (ret)
+		return ret;
+
+	if (ib_dev->adversarial_mode &&
+	    !(ctx_flags & UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED))
+		return -EPERM;
+
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
 			     &num_comp, sizeof(num_comp));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -262,6 +272,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 
 	if (ib_dev->ops.uverbs_robust_udata)
 		core_support |= IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA;
+	if (ib_dev->adversarial_mode)
+		core_support |= IB_UVERBS_CORE_SUPPORT_ADVERSARIAL;
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
 			     &core_support, sizeof(core_support));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -303,6 +315,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_CONTEXT)(
 	if (IS_UVERBS_COPY_ERR(ret))
 		return ret;
 
+	if (ib_dev->adversarial_mode)
+		core_support |= IB_UVERBS_CORE_SUPPORT_ADVERSARIAL;
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_CONTEXT_CORE_SUPPORT,
 			     &core_support, sizeof(core_support));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -519,6 +533,8 @@ DECLARE_UVERBS_NAMED_METHOD(
 			   UVERBS_ATTR_MIN_SIZE(sizeof(int)),
 			   UA_OPTIONAL,
 			   UA_ALLOC_AND_COPY),
+	UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_GET_CONTEXT_FLAGS, u32,
+			     UA_OPTIONAL),
 	UVERBS_ATTR_UHW());
 
 DECLARE_UVERBS_NAMED_METHOD(
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 816b3107464f..3e482a4338f9 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -101,10 +101,15 @@ enum uverbs_attrs_query_comp_cntr_caps_attr_ids {
 	UVERBS_ATTR_QUERY_COMP_CNTR_CAPS_SUPPORTED_QP_ATTACH_OPS,
 };
 
+enum uverbs_attrs_get_context_flags {
+	UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED = 1 << 0,
+};
+
 enum uverbs_attrs_get_context_attr_ids {
 	UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
 	UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
 	UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+	UVERBS_ATTR_GET_CONTEXT_FLAGS,
 };
 
 enum uverbs_attrs_query_context_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f..5861f34f8343 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -47,6 +47,7 @@
 enum ib_uverbs_core_support {
 	IB_UVERBS_CORE_SUPPORT_OPTIONAL_MR_ACCESS = 1 << 0,
 	IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA = 1 << 1,
+	IB_UVERBS_CORE_SUPPORT_ADVERSARIAL = 1 << 2,
 };
 
 enum ib_uverbs_access_flags {
-- 
2.18.1

[PATCH rdma-next 23/23] RDMA/mlx5: Advertise adversarial mode via ib_device::adversarial_mode

[Yishai Hadas]

Set ib_device::adversarial_mode during capability init so that the
uverbs core layer can report adversarial mode to rdma-core and enforce
that only adversarial-mode-aware userspace opens a context on this
device.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index ace4cdc95c9d..f8af90941b53 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -4778,6 +4778,7 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
 	}
 
 	dev->ib_dev.use_cq_dim = true;
+	dev->ib_dev.adversarial_mode = mlx5_ib_adversarial_mode;
 
 	return 0;
 
-- 
2.18.1

Re: [PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Jacob Moroni]

Hi,

Out of curiosity, is the idea that these restrictions will eventually
be lifted once the
ULPs (and I guess the drivers) are hardened? Or is it considered
infeasible? I am
most curious about RDMA CM.

Even if the drivers are hardened (no round-tripping WR IDs thru HW
from WQE->CQE,
sanitizing values from FW, etc.), I think it may be impossible to
protect against certain
bogus payload scenarios such as a NIC reporting a CQE even though it
never wrote the
data (or wrote bogus data), but this sort of risk seems like it would
apply to untrusted
Ethernet devices as well, IIUC. I can imagine it being very difficult
to harden RDMA CM
for this though.

Thanks,
Jake

Re: [PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Jason Gunthorpe]

On Tue, Nov 03, 2026 at 02:03:22PM -0500, Jacob Moroni wrote:
> Hi,
> 
> Out of curiosity, is the idea that these restrictions will
> eventually be lifted once the ULPs (and I guess the drivers) are
> hardened? Or is it considered infeasible? I am most curious about
> RDMA CM.

I think it depends on people's interest. This is not easy work and I
would insist any attempt be "complete" in that it at least
architecturally defends against a remote compromise.

> Even if the drivers are hardened (no round-tripping WR IDs thru HW
> from WQE->CQE, sanitizing values from FW, etc.), I think it may be
> impossible to protect against certain bogus payload scenarios such
> as a NIC reporting a CQE even though it never wrote the data (or
> wrote bogus data), but this sort of risk seems like it would apply
> to untrusted Ethernet devices as well, IIUC. I can imagine it being
> very difficult to harden RDMA CM for this though.

Yes, the verbs user is exposed to some of the threat as the
completions are no longer reliable. The user can assume the wr_id is
correct, and they only get one completion per wr_id, but the rest of
the completion information can't have any effect on integrity.

Jason

Re: [PATCH rdma-next 01/23] net/mlx5: Add mlx5_trigger_health_work_force()

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] net/mlx5: missing internal error check before mlx5_cmd_fast_teardown_hca()
- [High] net/mlx5: forced health work recovers device when interface is already down
--

commit 2777ddd787832a64204ac170752c47964f6bf998
Author: Yishai Hadas <yishaih@nvidia.com>

net/mlx5: Add mlx5_trigger_health_work_force()

This commit adds mlx5_trigger_health_work_force() to queue a dedicated work
item that stops the health poller, runs fast HCA teardown, and forces the
device into MLX5_DEVICE_STATE_INTERNAL_ERROR before running health recovery,
even when the fatal health sensors report no error.

> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
> index aeeb136f5ebc7..d000ac335fa7a 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
> @@ -627,18 +627,11 @@ mlx5_fw_fatal_reporter_dump(struct devlink_health_reporter *reporter,
>  	return err;
>  }
>  
> -static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
> +static void mlx5_fw_fatal_reporter_err(struct mlx5_core_dev *dev, bool forced)
>  {
>  	struct mlx5_fw_reporter_ctx fw_reporter_ctx;
> -	struct mlx5_core_health *health;
> -	struct mlx5_core_dev *dev;
> -	struct devlink *devlink;
> -	struct mlx5_priv *priv;
> -
> -	health = container_of(work, struct mlx5_core_health, fatal_report_work);
> -	priv = container_of(health, struct mlx5_priv, health);
> -	dev = container_of(priv, struct mlx5_core_dev, priv);
> -	devlink = priv_to_devlink(dev);
> +	struct mlx5_core_health *health = &dev->priv.health;
> +	struct devlink *devlink = priv_to_devlink(dev);
>  
>  	mutex_lock(&dev->intf_state_mutex);
>  	if (test_bit(MLX5_DROP_HEALTH_WORK, &health->flags)) {
> @@ -646,8 +639,26 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
>  		mutex_unlock(&dev->intf_state_mutex);
>  		return;
>  	}
> +	/*
> +	 * The firmware might still be running, so stop it first. Do it under
> +	 * intf_state_mutex and only while UP, so the cmd interface is valid.
> +	 */
> +	if (forced && test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state)) {
> +		int err;
> +
> +		mlx5_stop_health_poll(dev, false);
> +		err = mlx5_cmd_fast_teardown_hca(dev);
> +		if (err)
> +			mlx5_core_err(dev, "Failed to stop the device before forcing the error state, err %d\n",
> +				      err);
> +	}

[Severity: High]
Should mlx5_fw_fatal_reporter_err() also verify that dev->state is not
already MLX5_DEVICE_STATE_INTERNAL_ERROR before calling
mlx5_cmd_fast_teardown_hca(dev), similar to mlx5_try_fast_unload()?

drivers/net/ethernet/mellanox/mlx5/core/main.c:mlx5_try_fast_unload() {
    ...
	if (dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR) {
		mlx5_core_dbg(dev, "Device in internal error state, giving up\n");
		return -EAGAIN;
	}
    ...
}

MLX5_INTERFACE_STATE_UP remains set when poll_health() or
mlx5_enter_error_state() transitions dev->state to
MLX5_DEVICE_STATE_INTERNAL_ERROR. When dev->state is already
MLX5_DEVICE_STATE_INTERNAL_ERROR, mlx5_cmd_is_down(dev) is true, so
mlx5_cmd_exec_inout() -> cmd_exec() aborts MLX5_CMD_OP_TEARDOWN_HCA with
-ENXIO without sending it to firmware, and mlx5_cmd_check() ->
mlx5_internal_err_ret_value() emulates MLX5_CMD_STAT_OK (0) with a zeroed
output buffer.

In mlx5_cmd_fast_teardown_hca(), this emulated success causes the driver to
write MLX5_INITIAL_SEG_NIC_INTERFACE_DISABLED to MMIO via
mlx5_set_nic_state() without firmware preparation and busy-wait in a loop
until the teardown timeout expires while holding dev->intf_state_mutex:

drivers/net/ethernet/mellanox/mlx5/core/fw.c:mlx5_cmd_fast_teardown_hca() {
    ...
	ret = mlx5_cmd_exec_inout(dev, teardown_hca, in, out);
	if (ret)
		return ret;

	state = MLX5_GET(teardown_hca_out, out, state);
	if (state == MLX5_TEARDOWN_HCA_OUT_FORCE_STATE_FAIL) {
		mlx5_core_warn(dev, "teardown with fast mode failed\n");
		return -EIO;
	}

	mlx5_set_nic_state(dev, MLX5_INITIAL_SEG_NIC_INTERFACE_DISABLED);
    ...
}

In addition, because mlx5_health_check_fatal_sensors() checks for
MLX5_INITIAL_SEG_NIC_INTERFACE_DISABLED before sensor_fw_synd_rfr(dev),
overwriting the NIC state register masks MLX5_SENSOR_FW_SYND_RFR and
prevents mlx5_error_sw_reset() -> reset_fw_if_needed() from performing a
firmware SW reset.

> +
> +	/*
> +	 * Enter the error state before unlocking, or an unload could send
> +	 * TEARDOWN_HCA to the stopped NIC, fail and skip function disable.
> +	 */
> +	enter_error_state(dev, forced);
>  	mutex_unlock(&dev->intf_state_mutex);

[Severity: High]
What happens in mlx5_fw_fatal_reporter_err() when forced is true, but
MLX5_INTERFACE_STATE_UP is already cleared?

During a non-remove unload such as mlx5_suspend(), mlx5_devlink_reload_down(),
or mlx5_pci_err_detected(), mlx5_unload_one_devl_locked() clears
MLX5_INTERFACE_STATE_UP and calls mlx5_function_teardown(dev, false) ->
mlx5_function_disable(dev, false) -> mlx5_stop_health_poll(dev, false),
which stops the timer without setting MLX5_DROP_HEALTH_WORK:

drivers/net/ethernet/mellanox/mlx5/core/main.c:mlx5_unload_one_devl_locked() {
    ...
	clear_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state);
	mlx5_unload(dev);
	mlx5_function_teardown(dev, false);
out:
	mutex_unlock(&dev->intf_state_mutex);
}

If mlx5_trigger_health_work_force() was queued just before the unload completed,
mlx5_fw_fatal_reporter_err(dev, true) sees MLX5_DROP_HEALTH_WORK == 0 and
MLX5_INTERFACE_STATE_UP == 0. It skips mlx5_cmd_fast_teardown_hca(), but still
calls enter_error_state(dev, true) and proceeds to devlink_health_report() /
mlx5_health_try_recover(), which will reset and reload a device that was
just suspended or unloaded (or tear down a device that just finished
devlink reload).

Should mlx5_fw_fatal_reporter_err() unlock dev->intf_state_mutex and return
early if !test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state) when forced
is true?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=1

Re: [PATCH rdma-next 02/23] net/mlx5: Derive adversarial mode from CC guest memory encryption

[sashiko-bot]

> Add mlx5_is_adversarial() to report whether the driver treats the NIC
> as untrusted. In a confidential computing guest, the hypervisor and any
> passthrough device are outside the guest's trust boundary, so the mode
> is derived from cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT).
> 
> This turns adversarial mode on automatically in such guests, without
> any administrator configuration.
> 
> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=2

Re: [PATCH rdma-next 03/23] net/mlx5: Add capability filtering in adversarial mode

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] net/mlx5: unclamped log_max_flow_counter_bulk causes OOB heap read
- [High] net/mlx5: unchecked minimum EQ caps cause divide-by-zero in alloc_cpu_rmap()
- [Medium] net/mlx5: incomplete log_min_mkey_entity_size check in caps_filter.c
- [Medium] net/mlx5: unclamped log_max_*_list caps overflow signed shift in en_fs.c
--

commit 8f36be5a60064c165f291709b6cbba37c28f527e
Author: Dragos Tatulea <dtatulea@nvidia.com>

net/mlx5: Add capability filtering in adversarial mode

This commit introduces capability filtering in adversarial mode by clearing
firmware capabilities by default and selectively allowing, clamping, or
validating fields in current and max HCA capabilities.

> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
> new file mode 100644
> index 0000000000000..67b2603397450
> --- /dev/null
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
> @@ -0,0 +1,381 @@

[ ... ]

> +static int mlx5_adversarial_filter_general(struct mlx5_core_dev *dev, void *dst,
> +					   void *src)
> +{
> +	int err;
> +
> +	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));

[ ... ]

> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, eq_overrun_count);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, invalid_command_count);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, quota_exceeded_count);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_flow_counter_bulk);

[Severity: High]
Should log_max_flow_counter_bulk be clamped or range-checked here?

During initialization, mlx5_init_once() -> mlx5_fs_core_alloc() ->
mlx5_init_fc_stats() calls get_init_bulk_query_len():

drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c:get_init_bulk_query_len() {
	return min_t(int, MLX5_INIT_COUNTERS_BULK,
		     (1 << MLX5_CAP_GEN(dev, log_max_flow_counter_bulk)));
}

Because log_max_flow_counter_bulk is an 8-bit field, if an untrusted device
reports 31, (1 << 31) overflows a signed 32-bit int to INT_MIN (-2147483648),
and get_init_bulk_query_len() returns -2147483648.

Then mlx5_fc_stats_bulk_query_buf_realloc() passes that negative bulk_len to
mlx5_cmd_fc_get_bulk_query_out_len():

drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c:mlx5_cmd_fc_get_bulk_query_out_len() {
	return MLX5_ST_SZ_BYTES(query_flow_counter_out) +
		MLX5_ST_SZ_BYTES(traffic_counter) * bulk_len;
}

Since MLX5_ST_SZ_BYTES(traffic_counter) is 16UL (size_t), multiplying 16UL by
(size_t)(-2147483648) produces 0xfffffff800000000UL, whose lower 32 bits are
0. Truncating the result to int returns out_len = 16.

kvzalloc(16, GFP_KERNEL) then succeeds while fc_stats->bulk_query_len is set
to -2147483648. When mlx5_fc_stats_work runs
mlx5_fc_stats_query_all_counters() -> update_counter_cache(), could this read
flow_statistics[index] out of bounds past the 16-byte fc_stats->bulk_query_out
buffer?

> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rq);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_sq);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_tir);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, basic_cyclic_rcv_wqe);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rqt);
> +	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, log_max_rqt_size, 0, 13);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_bar_uar_access);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_odp_page_fault);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce_rw_supported);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_wq_sz);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_state);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_vlan_list);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_mc_list);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_uc_list);

[Severity: Medium]
Can unclamped values in log_max_vlan_list, log_max_current_mc_list, and
log_max_current_uc_list overflow the signed shifts in en_fs.c?

In drivers/net/ethernet/mellanox/mlx5/core/en_fs.c,
mlx5e_vport_context_update_vlans() computes:

	max_list_size = 1 << MLX5_CAP_GEN(fs->mdev, log_max_vlan_list);

	if (list_size > max_list_size) {
		fs_warn(fs, "netdev vlans list size (%d) > (%d) max vport list size, some vlans will be dropped\n",
			list_size, max_list_size);
		list_size = max_list_size;
	}

	vlans = kvcalloc(list_size, sizeof(*vlans), GFP_KERNEL);

and mlx5e_vport_context_update_addr_list() similarly computes:

	max_size = is_uc ?
		1 << MLX5_CAP_GEN(fs->mdev, log_max_current_uc_list) :
		1 << MLX5_CAP_GEN(fs->mdev, log_max_current_mc_list);

Since these three capabilities are 5-bit fields (0..31), if an adversarial
device reports 31, (1 << 31) overflows a signed 32-bit int to INT_MIN
(-2147483648). The comparison list_size > max_list_size (or size > max_size)
then evaluates to true, setting the count to -2147483648 and passing
(size_t)(-2147483648) to kvcalloc() and kcalloc() without __GFP_NOWARN.

> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sq_ts_format);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rq_ts_format);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, steering_format_version);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, create_qp_start_hint);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, ats);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cross_vhca_rqt);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_uctx);
> +	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_num_eqs);

[Severity: High]
What happens if the EQ capabilities (log_max_eq, max_num_eqs, or
max_num_eqs_24b in mlx5_adversarial_filter_general_2()) result in a maximum EQ
count less than or equal to MLX5_MAX_ASYNC_EQS?

mlx5_max_eq_cap_get() derives the maximum device EQs from max_num_eqs_24b,
max_num_eqs, or (1 << log_max_eq). During mlx5_load() ->
mlx5_eq_table_create(), get_num_eqs() calculates:

drivers/net/ethernet/mellanox/mlx5/core/eq.c:get_num_eqs() {
	...
	num_eqs = min_t(int, mlx5_irq_table_get_num_comp(eq_table->irq_table),
			max_dev_eqs - MLX5_MAX_ASYNC_EQS);
	...
}

If an untrusted device reports max_dev_eqs == MLX5_MAX_ASYNC_EQS (for example,
max_num_eqs = 4 or log_max_eq = 2 when MLX5_MAX_ASYNC_EQS is 4), get_num_eqs()
sets eq_table->max_comp_eqs = 0.

mlx5_eq_table_create() then calls alloc_rmap() -> alloc_irq_cpu_rmap(0) ->
alloc_cpu_rmap(0, GFP_KERNEL), which divides by size == 0 in cpu % size.
Wouldn't this trigger a divide-by-zero fault during driver load unless the
effective EQ count is checked to be strictly greater than MLX5_MAX_ASYNC_EQS
(and log_max_eq is kept below 31)?

[ ... ]

> +static int mlx5_adversarial_filter_general_2(struct mlx5_core_dev *dev, void *dst,
> +					     void *src)
> +{
> +	int err;
> +	u32 val;
> +
> +	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));

[ ... ]

> +	val = MLX5_GET(cmd_hca_cap_2, dst, max_mkey_log_entity_size_mtt) ?: 31;
> +	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap_2, log_min_mkey_entity_size, 0,
> +				 val);
> +	if (err)
> +		return err;
> +
> +	/* The fixed buffer min is don't-care when its valid bit is clear. */
> +	if (!MLX5_GET(cmd_hca_cap_2, dst,
> +		      min_mkey_log_entity_size_fixed_buffer_valid))
> +		return 0;

[Severity: Medium]
Are there two cases where the validation of log_min_mkey_entity_size in
mlx5_adversarial_filter_general_2() is too permissive?

First, when min_mkey_log_entity_size_fixed_buffer_valid is 0,
mlx5_adversarial_filter_general_2() returns 0 without checking
log_min_mkey_entity_size against max_mkey_log_entity_size_fixed_buffer.
However, in drivers/infiniband/hw/mlx5/mlx5_ib.h, get_min_log_entity_size_cap()
falls back to log_min_mkey_entity_size when that valid bit is clear:

	if (access_mode == MLX5_MKC_ACCESS_MODE_KSM &&
	    MLX5_CAP_GEN_2(dev->mdev,
			   min_mkey_log_entity_size_fixed_buffer_valid))
		min_log_size = MLX5_CAP_GEN_2(
			dev->mdev, min_mkey_log_entity_size_fixed_buffer);
	else
		min_log_size =
			MLX5_CAP_GEN_2(dev->mdev, log_min_mkey_entity_size);

while get_max_log_entity_size_cap() still uses
max_mkey_log_entity_size_fixed_buffer for MLX5_MKC_ACCESS_MODE_KSM. If
min_mkey_log_entity_size_fixed_buffer_valid is 0 and
max_mkey_log_entity_size_fixed_buffer < log_min_mkey_entity_size <=
max_mkey_log_entity_size_mtt, min_log_entity_size_cap will exceed
max_log_entity_size_cap in mlx5_umem_mkc_find_best_pgsz(), producing an
inverted GENMASK_ULL() of 0 and triggering WARN_ON(!page_size) in
alloc_cacheable_mr() and can_use_umr_rereg_pas().

Second, during mlx5e probe, mlx5e_build_rq_params() ->
mlx5e_mpwrq_validate_regular() -> mlx5e_mpwrq_page_shift(mdev, NULL) checks:

drivers/net/ethernet/mellanox/mlx5/core/en/params.c:mlx5e_mpwrq_page_shift() {
	u8 min_page_shift = MLX5_MPWRQ_MIN_LOG_STRIDE_SZ(mdev);
	u8 req_page_shift = xsk ? xsk->log_hrq_ix_sz : PAGE_SHIFT;

	/* Regular RQ uses order-0 pages, the NIC must be able to map them. */
	if (WARN_ON_ONCE(!xsk && req_page_shift < min_page_shift))
		min_page_shift = req_page_shift;
	...
}

where MLX5_MPWRQ_MIN_LOG_STRIDE_SZ(mdev) is log_min_mkey_entity_size ?: 12.
If log_min_mkey_entity_size is allowed to be up to 31 (greater than
PAGE_SHIFT), wouldn't this trigger WARN_ON_ONCE() on probe?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=3

Re: [PATCH rdma-next 04/23] net/mlx5: Add macros for reading log-based caps

[sashiko-bot]

> From: Dragos Tatulea <dtatulea@nvidia.com>
> 
> Add macros that can be used to exponentiate log based capabilities.
> 
> When a capability can represent more than the used version (u32 , u64),
> value will be clamped to the largest representable power of two.
> 
> Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=4

Re: [PATCH rdma-next 05/23] RDMA/mlx5: Reject QP transport types not enabled by FW capability

[sashiko-bot]

> check_qp_type() checks the FW capability only for XRC and DC. RC, UC
> and UD QPs are created even if the FW does not report the matching
> capability bit.
> 
> Check the rc, uc and ud capabilities as well, so that QP creation is
> refused for a transport the FW does not support.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=5

Re: [PATCH rdma-next 06/23] RDMA/mlx5: Require IB_QPT_DRIVER for the DC vendor QP types

[sashiko-bot]

> check_qp_type() checks the dct capability only for IB_QPT_DRIVER, but
> process_vendor_flags() turns any QP that carries a DC type flag into a
> DCI or DCT QP. A QP created as, for example, IB_QPT_RC with a DC flag
> thus becomes a DC QP without the capability check.
> 
> Accept the DC type flags only on IB_QPT_DRIVER, the type userspace
> uses to create DC QPs.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=6

Re: [PATCH rdma-next 07/23] RDMA/mlx5: Derive mlx5_ib_adversarial_mode global from mlx5_core adversarial API

[sashiko-bot]

> Use the mlx5_core mlx5_is_adversarial() exported API to set a
> module-level static bool mlx5_ib_adversarial_mode at init time,
> instead of a per-device bit field. This avoids propagating the flag
> through every mlx5_ib_dev pointer and makes the module-wide policy
> clear at the call sites.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=7

Re: [PATCH rdma-next 09/23] RDMA/mlx5: Remove duplicate mlx5_ib_umr_context struct

[sashiko-bot]

> mlx5_ib_umr_context in mlx5_ib.h and mlx5r_umr_context in umr.h have
> identical layouts: an ib_cqe, a status, and a completion.  The done
> callback used mlx5_ib_umr_context via container_of, while every other
> site in umr.c used mlx5r_umr_context.
> 
> Remove the duplicate and use mlx5r_umr_context consistently throughout.
> No behaviour change.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=9

Re: [PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[sashiko-bot]

> In adversarial mode the NIC is not trusted. Kernel ULPs such as MAD,
> IB CM, rdma_cm, IPoIB, SRP, iSER and NVMe-oF trust what the device
> reports, its completions and the data it writes, so they must not use
> such a device.
> 
> Add an adversarial_mode field to struct ib_device and clear
> kverbs_provider when it is set. add_client_context() then skips every
> client but uverbs, the only one that sets no_kverbs_req, so the kernel
> ULPs never bind to the device.
> 
> The driver still implements the kverbs ops, as the uverbs path shares
> them, so kverbs_provider can't be cleared by the existing check of the
> mandatory ops.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=8

Re: [PATCH rdma-next 10/23] RDMA/mlx5: Introduce per-CQ poll_one hook in mlx5_ib_cq

[sashiko-bot]

> Add a poll_one function pointer to struct mlx5_ib_cq and initialize it
> to mlx5_poll_one in both mlx5_ib_create_cq() (kernel-ULP CQs) and
> mlx5_ib_create_user_cq() (uverbs CQs, the path used by both the
> modern ioctl-based and legacy write-based CQ creation commands).
> mlx5_ib_poll_cq() is updated to dispatch through the pointer instead
> of calling mlx5_poll_one() directly.
> 
> This is a pure refactor with no behaviour change. The indirection
> allows a specialised poll_one implementation to be installed on a
> specific CQ, which a follow-up patch uses to verify the UMR CQ against
> a hostile NIC.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=10

Re: [PATCH rdma-next 11/23] RDMA/mlx5: Add shadow FIFO infrastructure for UMR completions

[sashiko-bot]

> The UMR CQ is polled by the generic path, which finds the completed
> request by indexing qp->sq.wrid[] with the CQE wqe_counter. A hostile
> NIC can forge it to complete a request other than the one that really
> completed, or one that is not pending at all.
> 
> The UMR QP is RC and signals every WR, so its completions arrive in
> posting order. In adversarial mode, keep a FIFO of the posted UMR
> contexts, so a follow-up patch can check each CQE against the oldest
> pending request instead of trusting it. Each context also records the
> SQ producer counter it was posted at, as its address alone doesn't
> identify the post: it is usually on the stack and reused across posts.
> 
> Nothing consumes the FIFO yet.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=11

Re: [PATCH rdma-next 12/23] RDMA/mlx5: Verify UMR CQ polling against a hostile NIC

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] RDMA/mlx5: unvalidated error CQE syndromes in mlx5_ib_poll_one_umr()
- [High] RDMA/mlx5: hung UMR waiters in mlx5_ib_poll_one_umr() on teardown
--

commit da9b5d87a1b071b4bd6eacadaf91a72df89f8c23
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/mlx5: Verify UMR CQ polling against a hostile NIC

This commit adds a dedicated UMR CQ polling handler in adversarial mode
that validates incoming CQEs against a shadow FIFO of posted UMR requests
and triggers forced health recovery if a forged CQE is detected.

> diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
> index 11c1153747c24..1a98263ec3111 100644
> --- a/drivers/infiniband/hw/mlx5/cq.c
> +++ b/drivers/infiniband/hw/mlx5/cq.c

[ ... ]

> @@ -608,6 +609,161 @@ static int poll_soft_wc(struct mlx5_ib_cq *cq, int num_entries,
>  	return npolled;
>  }
>  
> +static int mlx5_ib_poll_one_stopped(struct mlx5_ib_cq *cq,
> +				    struct mlx5_ib_qp **cur_qp, struct ib_wc *wc)
> +{
> +	return -EAGAIN;
> +}

[ ... ]

> +int mlx5_ib_poll_one_umr(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
> +		      struct ib_wc *wc)
> +{

[ ... ]

> +	/* Pop the shadow FIFO head: validated, dispatch to its waiter */
> +	WRITE_ONCE(umrc->pending[head & (MLX5_UMR_MAX_WR - 1)], NULL);
> +	umrc->pending_head = head + 1;
> +	wc->wr_cqe = &expected->cqe;
> +
> +	wc->qp = &qp->ibqp;
> +	wq->tail = wq->wqe_head[idx] + 1;
> +	/* Advance wq->last_poll, the internal error flush starts from it */
> +	handle_atomics(qp, cqe64, wq->last_poll, idx);
> +
> +	if (opcode == MLX5_CQE_REQ) {
> +		wc->status = IB_WC_SUCCESS;
> +	} else {
> +		/* Copy the CQE, so its syndrome is read once */
> +		err_cqe = *(struct mlx5_err_cqe *)cqe64;
> +		mlx5_handle_error_cqe(dev, &err_cqe, wc);
> +		/* As in the generic path, a real failure starts the recovery */
> +		if (wc->status != IB_WC_WR_FLUSH_ERR)
> +			umrc->state = MLX5_UMR_STATE_RECOVER;
> +	}

[Severity: Critical]
Does mlx5_ib_poll_one_umr() need to validate whether the error syndrome in a
MLX5_CQE_REQ_ERR completion is valid for the current umrc->state?

If umrc->state is MLX5_UMR_STATE_ACTIVE and a hostile NIC reports
MLX5_CQE_SYNDROME_WR_FLUSH_ERR for the oldest pending UMR WQE,
mlx5_ib_poll_one_umr() completes the request with IB_WC_WR_FLUSH_ERR while
leaving umrc->state as MLX5_UMR_STATE_ACTIVE. In
drivers/infiniband/hw/mlx5/umr.c:mlx5r_umr_post_send_wait(), the waiter sees
IB_WC_WR_FLUSH_ERR, continues the loop, sees umrc->state is still
MLX5_UMR_STATE_ACTIVE, and reposts the WQE indefinitely.

Conversely, when multiple UMR requests are in flight concurrently and the NIC
returns a non-flush error syndrome for more than one of them, each of those
waiters wakes up in mlx5r_umr_post_send_wait() with a status other than
IB_WC_WR_FLUSH_ERR and calls mlx5r_umr_recover(). Because mlx5r_umr_recover()
drops umrc->lock before posting the barrier WR and resetting the QP:

drivers/infiniband/hw/mlx5/umr.c:mlx5r_umr_recover()
    mutex_unlock(&umrc->lock);
    err = mlx5r_umr_post_send(umrc->qp, mkey, umr_context, wqe, with_data);
    ...
    wait_for_completion(&umr_context->done);
    attr.qp_state = IB_QPS_RESET;
    err = ib_modify_qp(umrc->qp, &attr, IB_QP_STATE);

multiple threads can enter recovery concurrently. When the first thread
resets the QP via __mlx5_ib_modify_qp(), qp->sq.head, qp->sq.tail,
qp->sq.cur_post, and qp->sq.last_poll are zeroed while the second thread's
barrier WR remains in umrc->pending with a non-zero wq->wqe_head[idx].
Completing that second barrier WR then sets wq->tail = wq->wqe_head[idx] + 1
while wq->head is 0. If a subsequent forged CQE triggers internal error
flushing, sw_comp() computes cur = wq->head - wq->tail (which wraps around as
unsigned int) and completes stale wq->wrid[] entries pointing to
stack-allocated mlx5r_umr_context structures of callers that have already
returned.

Should a flush error while active, or a second non-flush error after
entering recovery, be treated as a forged CQE (or should error CQEs on the
UMR QP be disallowed altogether in adversarial mode)?

> +
> +	return 0;
> +
> +forged_cqe:
> +	/* mlx5_ib_poll_cq() updates the CI only for a returned completion */
> +	mlx5_cq_set_ci(&cq->mcq);
> +	/*
> +	 * The NIC forged a CQE. Fail the UMR path for good, ignore any later
> +	 * CQE and force the health recovery, as the fatal sensors still read
> +	 * clean. The internal error flush then completes the waiters. Return
> +	 * an error, not -EAGAIN, as a CQE was consumed.
> +	 */
> +	umrc->state = MLX5_UMR_STATE_ERR;
> +	cq->poll_one = mlx5_ib_poll_one_stopped;
> +	mlx5_trigger_health_work_force(dev->mdev);
> +	return -EIO;
> +}

[Severity: High]
Can relying solely on mlx5_trigger_health_work_force() to flush pending UMR
requests leave waiters in mlx5r_umr_post_send_wait() hung during device
teardown, reload, or removal?

During teardown or reload, mlx5_unload_one_devl_locked() and
mlx5_uninit_one() in drivers/net/ethernet/mellanox/mlx5/core/main.c hold
dev->intf_state_mutex while unregistering the IB device, which revokes open
MRs through:

  mlx5_ib_dereg_mr()
    -> mlx5r_handle_mkey_cleanup()
      -> mlx5_umr_revoke_mr_with_lock()
        -> mlx5r_umr_revoke_mr()
          -> mlx5r_umr_post_send_wait()

In addition, remove_one(), shutdown(), and
mlx5_sync_reset_set_reset_requested() call mlx5_drain_health_wq() or
mlx5_stop_health_poll(), setting MLX5_DROP_HEALTH_WORK before mlx5_ib
teardown finishes.

If a forged CQE is detected in mlx5_ib_poll_one_umr() during these flows,
cq->poll_one is replaced with mlx5_ib_poll_one_stopped without completing
the entries in umrc->pending. The queued force_fatal_report_work calls
mlx5_fw_fatal_reporter_err() in
drivers/net/ethernet/mellanox/mlx5/core/health.c:

    mutex_lock(&dev->intf_state_mutex);
    if (test_bit(MLX5_DROP_HEALTH_WORK, &health->flags)) {
        mlx5_core_err(dev, "health works are not permitted at this stage\n");
        mutex_unlock(&dev->intf_state_mutex);
        return;
    }

Because dev->intf_state_mutex is held by the thread waiting in
wait_for_completion(&umr_context.done), or because MLX5_DROP_HEALTH_WORK is
already set, enter_error_state() and the software CQ flush cannot run,
leaving the UMR waiters blocked forever. Could forged_cqe drain and complete
the remaining contexts in umrc->pending directly?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=12

Re: [PATCH rdma-next 13/23] RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mlx5: unserialized mlx5r_umr_recover() and ERR state overwrite
--

commit a4c9a11e36d34c06e5a6a8bce8ad178d57157c1a
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected

This commit updates mlx5r_umr_recover() to transition umrc->state using
cmpxchg() when mlx5_ib_adversarial_mode is enabled so that a concurrent
transition to MLX5_UMR_STATE_ERR by the CQ poller is not overwritten back to
MLX5_UMR_STATE_RECOVER or MLX5_UMR_STATE_ACTIVE.

> diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
> index b0f00d4bc93ca..086b397b830c6 100644
> --- a/drivers/infiniband/hw/mlx5/umr.c
> +++ b/drivers/infiniband/hw/mlx5/umr.c
> @@ -335,6 +335,23 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey,
>  	return err;
>  }
>  
> +/*
> + * Move the UMR state from @from to @to. In adversarial_mode ERR is a final
> + * state: it can be set concurrently by the CQ poller, so the move is atomic
> + * and does not replace it. Returns false if ERR was found, and true
> + * otherwise. Outside adversarial_mode this is the plain store it always was.
> + */
> +static bool mlx5r_umr_move_state(struct umr_common *umrc, unsigned int from,
> +				 unsigned int to)
> +{
> +	if (!mlx5_ib_adversarial_mode) {
> +		umrc->state = to;
> +		return true;
> +	}
> +
> +	return cmpxchg(&umrc->state, from, to) != MLX5_UMR_STATE_ERR;
> +}
> +
>  static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
>  			     struct mlx5r_umr_context *umr_context,
>  			     struct mlx5r_umr_wqe *wqe, bool with_data)
> @@ -353,7 +370,11 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
>  	if (umrc->state != MLX5_UMR_STATE_RECOVER) {
>  		mlx5_ib_warn(dev, "UMR recovery encountered an unexpected state=%d\n",
>  			     umrc->state);
> -		umrc->state = MLX5_UMR_STATE_RECOVER;
> +		if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_ACTIVE,
> +					  MLX5_UMR_STATE_RECOVER)) {
> +			mutex_unlock(&umrc->lock);
> +			return -EFAULT;
> +		}
>  	}
>  	mutex_unlock(&umrc->lock);
>  
> @@ -387,7 +408,9 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
>  		goto err;
>  	}
>  
> -	umrc->state = MLX5_UMR_STATE_ACTIVE;
> +	if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_RECOVER,
> +				  MLX5_UMR_STATE_ACTIVE))
> +		return -EFAULT;

[Severity: High]
Can a hostile NIC still cause concurrent mlx5r_umr_recover() executions and
overwrite MLX5_UMR_STATE_ERR back to MLX5_UMR_STATE_ACTIVE?

In mlx5_ib_adversarial_mode, up to 64 threads can concurrently post UMR WRs
in mlx5r_umr_post_send_wait() under umrc->sem, placing their contexts into
umrc->pending[]. If a hostile NIC returns valid-header MLX5_CQE_REQ_ERR
completions (matching qpn, sop == MLX5_OPCODE_UMR, and sequential wqe_ctr)
with a non-flush syndrome (such as MLX5_CQE_SYNDROME_LOCAL_QP_OP_ERR, where
wc->status != IB_WC_WR_FLUSH_ERR) for multiple in-flight WRs,
mlx5_ib_poll_one_umr() accepts each CQE and unconditionally writes
MLX5_UMR_STATE_RECOVER:

drivers/infiniband/hw/mlx5/cq.c:mlx5_ib_poll_one_umr() {
	...
	} else {
		/* Copy the CQE, so its syndrome is read once */
		err_cqe = *(struct mlx5_err_cqe *)cqe64;
		mlx5_handle_error_cqe(dev, &err_cqe, wc);
		/* As in the generic path, a real failure starts the recovery */
		if (wc->status != IB_WC_WR_FLUSH_ERR)
			umrc->state = MLX5_UMR_STATE_RECOVER;
	}
}

Multiple waiters in mlx5r_umr_post_send_wait() then wake up with
umr_context.status != IB_WC_WR_FLUSH_ERR and call mlx5r_umr_recover().
Because mlx5r_umr_recover() only holds umrc->lock across the initial state
check and treats umrc->state == MLX5_UMR_STATE_RECOVER as the normal entry
state without serializing recovery callers, multiple threads can run
mlx5r_umr_recover() at the same time.

This appears to allow two problems:

1. If one recovery thread fails at the err label in mlx5r_umr_recover(), it
sets umrc->state = MLX5_UMR_STATE_ERR without stopping cq->poll_one:

drivers/infiniband/hw/mlx5/umr.c:mlx5r_umr_recover() {
	...
err:
	umrc->state = MLX5_UMR_STATE_ERR;
	return err;
}

When mlx5_ib_poll_one_umr() subsequently processes another non-flush error
CQE for an in-flight WR or barrier WR, its plain store overwrites
MLX5_UMR_STATE_ERR back to MLX5_UMR_STATE_RECOVER, allowing a concurrent
recovery thread's mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_RECOVER,
MLX5_UMR_STATE_ACTIVE) call to succeed and reopen the UMR path to
MLX5_UMR_STATE_ACTIVE.

2. When the first recovery thread's barrier WR completes and it calls
ib_modify_qp(umrc->qp, IB_QPS_RESET), __mlx5_ib_modify_qp() resets the QP
and zeroes qp->sq.head, qp->sq.tail, qp->sq.cur_post, and qp->sq.last_poll
while the second recovery thread's barrier WR is still queued in
umrc->pending and sleeping in wait_for_completion(&umr_context->done). Does
that desynchronize umrc->pending from qp->sq and leave the second thread
hung in wait_for_completion() while holding umrc->sem, since even the
internal-error software flush in sw_comp() walks from last_poll to head?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=13

Re: [PATCH rdma-next 14/23] RDMA/mlx5: Block generic CQ poll path in adversarial mode

[sashiko-bot]

> mlx5_poll_one() trusts the CQE content, so it must not run on a CQE
> from an untrusted NIC. In adversarial mode no CQ needs it: the UMR CQ
> has its own poll_one that validates the CQE, kernel ULPs don't bind to
> the device, and user CQs are polled by userspace. Refuse it there, so
> an unexpected caller, such as the uverbs POLL_CQ command, finds the CQ
> empty and no CQE is interpreted.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=14

Re: [PATCH rdma-next 15/23] RDMA/core: Set the CQ comp_handler after its poll context is set up

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/core: comp_handler initialized after create_cq() in __ib_alloc_cq()
--

commit 08ca16c59be95a85d42f773522716fafbd346f19
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/core: Set the CQ comp_handler after its poll context is set up

This commit updates __ib_alloc_cq() to initialize the CQ poll context
(irq_poll, work_struct, and comp_wq) before setting cq->comp_handler and
arming the CQ via ib_req_notify_cq().

> diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c
> index 12304c9a9403e..42e4d272f5ade 100644
> --- a/drivers/infiniband/core/cq.c
> +++ b/drivers/infiniband/core/cq.c
> @@ -217,6 +217,7 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
>  		.cqe		= nr_cqe,
>  		.comp_vector	= comp_vector,
>  	};
> +	ib_comp_handler comp_handler;
>  	struct ib_cq *cq;
>  	int ret = -ENOMEM;
>  
> @@ -248,19 +249,16 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
>  
>  	switch (cq->poll_ctx) {
>  	case IB_POLL_DIRECT:
> -		cq->comp_handler = ib_cq_completion_direct;
> +		comp_handler = ib_cq_completion_direct;
>  		break;
>  	case IB_POLL_SOFTIRQ:
> -		cq->comp_handler = ib_cq_completion_softirq;
> -
> +		comp_handler = ib_cq_completion_softirq;
>  		irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler);
> -		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
>  		break;
>  	case IB_POLL_WORKQUEUE:
>  	case IB_POLL_UNBOUND_WORKQUEUE:
> -		cq->comp_handler = ib_cq_completion_workqueue;
> +		comp_handler = ib_cq_completion_workqueue;
>  		INIT_WORK(&cq->work, ib_cq_poll_work);
> -		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
>  		cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ?
>  				ib_comp_wq : ib_comp_unbound_wq;
>  		break;
> @@ -269,6 +267,15 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
>  		goto out_destroy_cq;
>  	}
>  
> +	/*
> +	 * The CQ is reachable from the completion interrupt once created. Set
> +	 * the handler only after the poll context it schedules is set up, and
> +	 * arm the CQ only after that.
> +	 */
> +	cq->comp_handler = comp_handler;

[Severity: High]
Since __ib_alloc_cq() calls dev->ops.create_cq(cq, &cq_attr, NULL) before
initializing the poll context and setting cq->comp_handler, can a completion
interrupt arriving right after dev->ops.create_cq() registers the CQ in the
device EQ table race with this initialization?

Unlike __ib_create_cq(), ib_uverbs_create_cq(), and
UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE), which set cq->comp_handler before
calling dev->ops.create_cq(), __ib_alloc_cq() calls dev->ops.create_cq() while
cq->comp_handler is still NULL:

drivers/infiniband/core/cq.c:__ib_alloc_cq() {
	...
	ret = dev->ops.create_cq(cq, &cq_attr, NULL);
	if (ret)
		goto out_free_wc;

	rdma_dim_init(cq);

	switch (cq->poll_ctx) {
	...
}

At this commit, mlx5_ib_cq_comp() calls ibcq->comp_handler() without a NULL
check (until the subsequent commit "RDMA/mlx5: Ignore completion events for a
CQ without a handler" adds an if (ibcq->comp_handler) check in mlx5):

drivers/infiniband/hw/mlx5/cq.c:mlx5_ib_cq_comp() {
	struct ib_cq *ibcq = &to_mibcq(cq)->ibcq;

	ibcq->comp_handler(ibcq, ibcq->cq_context);
}

Other RDMA drivers (such as efa_process_comp_eqe() in
drivers/infiniband/hw/efa/efa_main.c, mlx4_ib_cq_comp(),
mthca_cq_completion(), and cxgb4) also unconditionally invoke
cq->ibcq.comp_handler() without a NULL check once the CQ is inserted into the
device CQ table during create_cq(), and cq->comp_handler is never set if the
default branch in switch (cq->poll_ctx) jumps to out_destroy_cq.

In addition, even in drivers that check if (ibcq->comp_handler) in their
completion interrupt handler (such as mlx5_ib_cq_comp() after the follow-up
commit and hns_roce_cq_completion()), the CQ is already reachable on other
CPUs while irq_poll_init(&cq->iop, ...), INIT_WORK(&cq->work, ...), and
cq->comp_wq are being initialized. Because cq->comp_handler = comp_handler is
a plain store without release ordering (and the interrupt handler reads
ibcq->comp_handler without acquire ordering), can compiler or CPU store
reordering on weakly ordered architectures allow an interrupt handler on
another CPU to observe a non-NULL cq->comp_handler before the stores to
cq->iop, cq->work, or cq->comp_wq are visible?

Now that ib_req_notify_cq() is called after the switch (cq->poll_ctx) block,
would it be safer to initialize the poll context (cq->iop, cq->work, and
cq->comp_wq) and set cq->comp_handler before calling dev->ops.create_cq(),
leaving only rdma_dim_init() and ib_req_notify_cq() after create_cq()
succeeds?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=15

Re: [PATCH rdma-next 16/23] RDMA/mlx5: Ignore completion events for a CQ without a handler

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mlx5: missing acquire/release on comp_handler in mlx5_ib_cq_comp()
--

commit 74de7436d2c18504b739bcbd6ccce8e1cfa93dc1
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/mlx5: Ignore completion events for a CQ without a handler

This commit skips calling ibcq->comp_handler in mlx5_ib_cq_comp() when no
completion handler is set, preventing a NULL function pointer call if a
hostile NIC raises an unsolicited completion event on a CQ without a
handler.

> diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
> index 5cc86c3f93f3e..f955eb762051d 100644
> --- a/drivers/infiniband/hw/mlx5/cq.c
> +++ b/drivers/infiniband/hw/mlx5/cq.c
> @@ -46,7 +46,12 @@ static void mlx5_ib_cq_comp(struct mlx5_core_cq *cq, struct mlx5_eqe *eqe)
>  {
>  	struct ib_cq *ibcq = &to_mibcq(cq)->ibcq;
>  
> -	ibcq->comp_handler(ibcq, ibcq->cq_context);
> +	/*
> +	 * A CQ created without a handler (devr->c0) is never armed, but a
> +	 * hostile NIC can still raise a completion event for it.
> +	 */
> +	if (ibcq->comp_handler)
> +		ibcq->comp_handler(ibcq, ibcq->cq_context);

[Severity: High]
When a kernel CQ is allocated via __ib_alloc_cq() (such as dev->umrc.cq in
mlx5r_umr_resource_init()), dev->ops.create_cq() publishes the CQ into the
completion EQ radix tree via mlx5_ib_create_cq() -> mlx5_core_create_cq() ->
mlx5_eq_add_cq() while cq->comp_handler is still NULL.

After create_cq() returns, __ib_alloc_cq() initializes cq->dim and the poll
context before setting cq->comp_handler:

drivers/infiniband/core/cq.c:__ib_alloc_cq() {
	...
	ret = dev->ops.create_cq(cq, &cq_attr, NULL);
	if (ret)
		goto out_free_wc;

	rdma_dim_init(cq);

	switch (cq->poll_ctx) {
	case IB_POLL_DIRECT:
		comp_handler = ib_cq_completion_direct;
		break;
	case IB_POLL_SOFTIRQ:
		comp_handler = ib_cq_completion_softirq;
		irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler);
		break;
	case IB_POLL_WORKQUEUE:
	case IB_POLL_UNBOUND_WORKQUEUE:
		comp_handler = ib_cq_completion_workqueue;
		INIT_WORK(&cq->work, ib_cq_poll_work);
		cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ?
				ib_comp_wq : ib_comp_unbound_wq;
		break;
	...
	}

	cq->comp_handler = comp_handler;
}

If a hostile NIC raises an unsolicited completion event for the newly
created CQN on another CPU while __ib_alloc_cq() is setting up the poll
context, mlx5_eq_comp_int() -> mlx5_ib_cq_comp() reads and invokes
ibcq->comp_handler with plain loads.

Because ibcq in mlx5_ib_cq_comp() is derived from cq (&to_mibcq(cq)->ibcq)
rather than from the loaded ibcq->comp_handler pointer, there is no address
dependency between reading ibcq->comp_handler and accessing fields of ibcq
inside the completion handler, and neither irq_poll_init() nor INIT_WORK()
provides a release barrier before cq->comp_handler = comp_handler.

Could a weakly ordered CPU or the compiler reorder the stores to cq->iop,
cq->work, cq->comp_wq, or cq->dim with the store and load of comp_handler,
causing ib_cq_completion_softirq() or ib_cq_completion_workqueue() to see
uninitialized poll context fields (such as a NULL iop->poll or cq->comp_wq)?
Should __ib_alloc_cq() use smp_store_release(&cq->comp_handler,
comp_handler) paired with smp_load_acquire(&ibcq->comp_handler) in
mlx5_ib_cq_comp()?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=16

Re: [PATCH rdma-next 17/23] RDMA/mlx5: Validate EQE port and GSI port before indexing devr.ports[]

[sashiko-bot]

> devr.ports[] has 2 entries, but it is indexed by port numbers taken
> from the NIC, which adversarial mode does not trust.
> 
> handle_port_change() uses the EQE port before any validation, so port 0
> wraps the index to UINT_MAX and a port above 2 overflows the array.
> mlx5_ib_create_gsi() gets a port already checked against phys_port_cnt,
> but that comes from the num_ports and num_vhca_ports FW caps, which may
> report more than 2 ports.
> 
> Validate the port and bound it by ARRAY_SIZE() in both paths.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=17

Re: [PATCH rdma-next 18/23] RDMA/mlx5: Validate firmware RQ/SQ state before sqrq_trans[] lookup

[sashiko-bot]

> sqrq_state_to_qp_state() indexes the 5x5 sqrq_trans[] table with the RQ
> and SQ states returned by QUERY_RQ and QUERY_SQ. The state fields are 4
> bits wide, so an untrusted NIC can return a state above 4 and the lookup
> reads past the table.
> 
> Reject out of range states with -EIO before the lookup.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=18

Re: [PATCH rdma-next 19/23] RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mlx5: 32-bit entries * cqe_size overflow in CQ create and resize
--

commit 23d4d478ec288a1c091e2edf580c64eccf4e7c49
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks

This commit replaces the 32-bit shift of log_max_cq_sz with
MLX5_CAP_GEN_LOGSZ_U64(), rounds up the requested CQ entry count using 64-bit
arithmetic, and rejects rounded-up entry counts above INT_MAX in the CQ create
and resize paths.

> diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
> index f955eb762051d..25344dfba1c12 100644
> --- a/drivers/infiniband/hw/mlx5/cq.c
> +++ b/drivers/infiniband/hw/mlx5/cq.c
> @@ -1117,6 +1117,15 @@ static void notify_soft_wc_handler(struct work_struct *work)
>  	cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
>  }
>  
> +/*
> + * Next power of 2 above entries. It can be 2^32, which doesn't fit the
> + * unsigned long of roundup_pow_of_two() on 32-bit archs.
> + */
> +static u64 cq_roundup_entries(u64 entries)
> +{
> +	return 1ULL << fls64(entries);
> +}
> +
>  int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
>  			   const struct ib_cq_init_attr *attr,
>  			   struct uverbs_attr_bundle *attrs)
> @@ -1124,7 +1133,7 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
>  	struct ib_udata *udata = &attrs->driver_udata;
>  	struct ib_device *ibdev = ibcq->device;
>  	struct mlx5_ib_create_cq_resp uresp = {};
> -	int entries = attr->cqe;
> +	u64 entries = attr->cqe;
>  	int vector = attr->comp_vector;
>  	struct mlx5_ib_dev *dev = to_mdev(ibdev);
>  	struct mlx5_ib_cq *cq = to_mcq(ibcq);
> @@ -1136,15 +1145,17 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
>  	int cqe_size;
>  	int eqn;
>  	int err;
> +	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
>  
> -	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
> +	if (attr->cqe > max_cq_entries)
>  		return -EINVAL;
>  
>  	if (check_cq_create_flags(attr->flags))
>  		return -EOPNOTSUPP;
>  
> -	entries = roundup_pow_of_two(entries + 1);
> -	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
> +	entries = cq_roundup_entries(entries);
> +	/* cq->ibcq.cqe and create_cq_user() take an int */
> +	if (entries > max_cq_entries || entries > INT_MAX)
>  		return -EINVAL;
>  
>  	cq->ibcq.cqe = entries - 1;
> @@ -1214,7 +1225,7 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
>  		      struct uverbs_attr_bundle *attrs)
>  {
>  	struct ib_device *ibdev = ibcq->device;
> -	int entries = attr->cqe;
> +	u64 entries = attr->cqe;
>  	int vector = attr->comp_vector;
>  	struct mlx5_ib_dev *dev = to_mdev(ibdev);
>  	struct mlx5_ib_cq *cq = to_mcq(ibcq);
> @@ -1226,12 +1237,14 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
>  	int cqe_size;
>  	int eqn;
>  	int err;
> +	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
>  
> -	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
> +	if (attr->cqe > max_cq_entries)
>  		return -EINVAL;
>  
> -	entries = roundup_pow_of_two(entries + 1);
> -	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
> +	entries = cq_roundup_entries(entries);
> +	/* cq->ibcq.cqe and create_cq_kernel() take an int */
> +	if (entries > max_cq_entries || entries > INT_MAX)
>  		return -EINVAL;
>  
>  	cq->ibcq.cqe = entries - 1;
> @@ -1532,18 +1545,22 @@ int mlx5_ib_resize_cq(struct ib_cq *ibcq, unsigned int entries,
>  	int inlen;
>  	int cqe_size;
>  	unsigned long flags;
> +	u64 entries64 = entries;
> +	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
>  
>  	if (!MLX5_CAP_GEN(dev->mdev, cq_resize)) {
>  		pr_info("Firmware does not support resize CQ\n");
>  		return -ENOSYS;
>  	}
>  
> -	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
> +	if (entries64 > max_cq_entries)
>  		return -EINVAL;
>  
> -	entries = roundup_pow_of_two(entries + 1);
> -	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)) + 1)
> +	entries64 = cq_roundup_entries(entries64);
> +	/* resize_user() and resize_kernel() take an int */
> +	if (entries64 > max_cq_entries + 1 || entries64 > INT_MAX)
>  		return -EINVAL;
> +	entries = entries64;

[Severity: High]
When an untrusted NIC reports log_max_cq_sz >= 24, bounding the rounded-up
entry count only by INT_MAX allows power-of-two values of entries from 2^24
up to 2^30 to reach the downstream buffer size calculations. Can those 32-bit
multiplications by cqe_size still overflow?

In mlx5_ib_create_user_cq() -> create_cq_user(), entries (int) is multiplied
by ucmd.cqe_size (__u32, validated as 64 or 128) in 32-bit unsigned
arithmetic before widening to size_t:

drivers/infiniband/hw/mlx5/cq.c:create_cq_user() {
	...
	cq->buf.umem = ib_umem_get_cq_buf_or_va(&dev->ib_dev, attrs,
						ucmd.buf_addr,
						entries * ucmd.cqe_size,
						IB_ACCESS_LOCAL_WRITE);
	...
}

For entries in 2^25..2^30 with ucmd.cqe_size == 128 (or 2^26..2^30 with
ucmd.cqe_size == 64), entries * ucmd.cqe_size wraps to 0, which lets a
1-page user buffer pass the min_size check in ib_umem_get_desc_check() while
programming log_cq_size = 25..30 in the CQ context.

Similarly, in mlx5_ib_create_cq() -> create_cq_kernel() ->
alloc_cq_frag_buf() and mlx5_ib_resize_cq() -> resize_kernel() ->
alloc_cq_frag_buf(), nent * cqe_size is evaluated as a signed 32-bit int and
passed as int size to mlx5_frag_buf_alloc_node():

drivers/infiniband/hw/mlx5/cq.c:alloc_cq_frag_buf() {
	...
	err = mlx5_frag_buf_alloc_node(dev->mdev,
				       nent * cqe_size,
				       frag_buf,
				       dev->mdev->priv.numa_node);
	...
}

For nent == 2^24 (with cqe_size == 128) or nent == 2^25 (with cqe_size == 64),
nent * cqe_size overflows to INT_MIN, and for nent >= 2^25 (cqe_size == 128)
or nent >= 2^26 (cqe_size == 64), it wraps to 0. When size is 0,
mlx5_frag_buf_alloc_node() sets frag_buf->frags = ZERO_SIZE_PTR and returns 0,
and init_cq_frag_buf() then dereferences ZERO_SIZE_PTR when iterating over
buf->nent:

drivers/infiniband/hw/mlx5/cq.c:init_cq_frag_buf() {
	for (i = 0; i < buf->nent; i++) {
		cqe = mlx5_frag_buf_get_wqe(&buf->fbc, i);
		cqe64 = buf->cqe_size == 64 ? cqe : cqe + 64;
		cqe64->op_own = MLX5_CQE_INVALID << 4;
	}
}

Would it make sense to cap entries so that entries * cqe_size cannot overflow
int (or use check_mul_overflow() in create_cq_user() and alloc_cq_frag_buf())?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=19

Re: [PATCH rdma-next 20/23] RDMA/mlx5: Clamp log_max_* shifts in QP and WQ limit checks

[sashiko-bot]

> Several QP and WQ size checks in qp.c compute their limit as
> 1 << cap, where cap is an 8-bit log_max_* FW capability. An untrusted
> NIC can report 31 or more, which makes the shift undefined.
> 
> Use MLX5_CAP_GEN_LOGSZ_U32() for these limits, as done for
> log_max_cq_sz.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=20

Re: [PATCH rdma-next 21/23] RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] RDMA/mlx5: integer overflow and size truncation in mlx5_ib_create_srq()
--

commit 9361613960626bd9707489f3ac7caf042edc2d19
Author: Yishai Hadas <yishaih@nvidia.com>

RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths

This commit replaces open-coded shifts of 8-bit log_max_* and log_pg_sz
firmware capabilities with MLX5_CAP_GEN_LOGSZ_U32() and
MLX5_CAP_GEN_LOGSZ_U64() in device, port, ucontext, and SRQ paths, and
saturates products that can now exceed 32 bits.

> diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
> index 6a4a937d97ce9..3fbb0eaf12558 100644
> --- a/drivers/infiniband/hw/mlx5/srq.c
> +++ b/drivers/infiniband/hw/mlx5/srq.c
> @@ -198,7 +198,7 @@ int mlx5_ib_create_srq(struct ib_srq *ib_srq,
>  	size_t buf_size;
>  	int err;
>  	struct mlx5_srq_attr in = {};
> -	__u32 max_srq_wqes = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
> +	__u32 max_srq_wqes = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_srq_sz);
>  	__u32 max_sge_sz =  MLX5_CAP_GEN(dev->mdev, max_wqe_sz_rq) /
>  			    sizeof(struct mlx5_wqe_data_seg);

[Severity: High]
With MLX5_CAP_GEN_LOGSZ_U32(), max_srq_wqes can now reach 2^31 (0x80000000U)
when the firmware reports log_max_srq_sz >= 31, which allows
init_attr->attr.max_wr values in [0x40000000, 0x7fffffff] to pass the
init_attr->attr.max_wr >= max_srq_wqes check in mlx5_ib_create_srq().

Can this overflow srq->msrq.max and truncate buf_size later in
mlx5_ib_create_srq()?

drivers/infiniband/hw/mlx5/srq.c:mlx5_ib_create_srq() {
	...
	srq->msrq.max    = roundup_pow_of_two(init_attr->attr.max_wr + 1);
	...
	buf_size = srq->msrq.max * desc_size;
	if (buf_size < desc_size)
		return -EINVAL;

	in.type = init_attr->srq_type;

	if (udata)
		err = create_srq_user(ib_srq->pd, srq, &in, udata, buf_size);
	else
		err = create_srq_kernel(dev, srq, &in, buf_size);
	...
}

In struct mlx5_core_srq (drivers/infiniband/hw/mlx5/srq.h), max is a signed
int:

	int max;

When init_attr->attr.max_wr is in [2^30, 2^31 - 1],
roundup_pow_of_two(init_attr->attr.max_wr + 1) evaluates to 2^31, which
overflows srq->msrq.max to INT_MIN. On 64-bit platforms,
srq->msrq.max * desc_size sign-extends srq->msrq.max to size_t before
multiplying by desc_size (a power of two >= 32), producing a large non-zero
size_t (for example, 0xfffffff000000000UL when desc_size == 32) that passes
buf_size < desc_size.

Even when log_max_srq_sz is in [26, 30] and srq->msrq.max stays positive
(e.g. in [2^26, 2^30]), size_t buf_size is >= 2^31 and is passed to
create_srq_user() or create_srq_kernel(), both of which take int buf_size:

	static int create_srq_user(struct ib_pd *pd, struct mlx5_ib_srq *srq,
				   struct mlx5_srq_attr *in,
				   struct ib_udata *udata, int buf_size)

	static int create_srq_kernel(struct mlx5_ib_dev *dev, struct mlx5_ib_srq *srq,
				     struct mlx5_srq_attr *in, int buf_size)

This truncates buf_size to 0 (when buf_size is a multiple of 2^32) or a
negative int.

In mlx5_ib_create_srq() -> create_srq_user(), buf_size == 0 passes
min_size = 0 to ib_umem_get_attr_or_va(), bypassing the minimum buffer size
check on the UVERBS_ATTR_CREATE_SRQ_BUF_UMEM path; then
in.log_size = ilog2(srq->msrq.max) on a negative srq->msrq.max yields 63,
and mlx5_cmd_create_srq() -> create_srq_split() -> create_rmp_cmd() ->
get_pas_size() executes an undefined shift:

	u32 rq_sz = 1 << (log_srq_size + 4 + log_rq_stride);

followed by a WARN_ON in __set_srq_page_size().

In mlx5_ib_create_srq() -> create_srq_kernel(), buf_size == 0 causes
mlx5_frag_buf_alloc_node() to allocate 0 pages (setting
srq->buf.frags = ZERO_SIZE_PTR) and return 0, so the WQE initialization loop
in create_srq_kernel() dereferences ZERO_SIZE_PTR when srq->msrq.max > 0 (or
underflows srq->tail = srq->msrq.max - 1 when srq->msrq.max == INT_MIN).

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=21

Re: [PATCH rdma-next 22/23] RDMA/core: Report and enforce adversarial mode via GET_CONTEXT

[sashiko-bot]

> Userspace must know when a device is in adversarial mode, and an
> unaware rdma-core must not get a context on such a device, as it
> doesn't harden itself against the NIC.
> 
> Report the mode with IB_UVERBS_CORE_SUPPORT_ADVERSARIAL in the
> core_support of GET_CONTEXT and QUERY_CONTEXT. Add an optional
> UVERBS_ATTR_GET_CONTEXT_FLAGS input, where userspace declares
> UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED. In adversarial
> mode, GET_CONTEXT without that flag fails with -EPERM, and so does the
> legacy write() GET_CONTEXT, which has no way to pass it.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=22

Re: [PATCH rdma-next 23/23] RDMA/mlx5: Advertise adversarial mode via ib_device::adversarial_mode

[sashiko-bot]

> Set ib_device::adversarial_mode during capability init so that the
> uverbs core layer can report adversarial mode to rdma-core and enforce
> that only adversarial-mode-aware userspace opens a context on this
> device.
> 
> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103163949.1554-1-yishaih@nvidia.com?part=23