[PATCH rdma-next 00/23] mlx5: Adversarial mode for confidential computing guests

[PATCH rdma-next 00/23] mlx5: Adversarial mode for confidential computing guests

[Yishai Hadas]

In a confidential computing (CC) guest, the hypervisor and any device
assigned to the guest are outside its trust boundary. The mlx5 drivers,
like most drivers, trust the NIC: they use the capabilities, events and
completions it reports to size buffers, index arrays and pick the memory
they work on. A malicious or compromised NIC can turn that trust into
memory corruption or information leaks inside the guest.

This series adds an adversarial mode, in which the mlx5 drivers treat
the NIC as untrusted. It is enabled automatically in CC guests, based on
cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT), with no administrator
configuration. Outside a CC guest the behaviour is unchanged.

The approach is to keep the attack surface small rather than harden
every path. In adversarial mode only the uverbs path is exposed: kernel
ULPs don't bind to the device and unneeded features are not offered.
The data path is owned by rdma-core, which must be aware of the mode.
When it opens a context, rdma-core declares that it supports adversarial
mode, and the kernel reports back that the device is in it, so rdma-core
validates what the NIC writes to the user queues. An rdma-core that
doesn't declare support is refused a context, so it can't use the device
while trusting the NIC. What remains in the kernel validates what it
gets from the NIC. A NIC can always stop working, so denial of service
is out of scope.

We believe that this series provides comprehensive security against an
adversarial device within rdma. Many teams at NVIDIA have been involved
in evaluating the various attack vectors and developing a comprehensive
mitigation, including AI analysis of the driver, fuzzing, and first
principles architecture review. It chooses to block and disable
functions that may open risk, and results in a reduction in driver
functionality rather than leave open potential security holes.

The series is organized as follows:

mlx5_core (patches 1-4)
- Derive adversarial mode from CC guest memory encryption.
- Filter the FW capabilities: start from none, pass through an allow
  list, and validate or clamp the numeric ones, failing the load on
  unexpected values.
- Add helpers to read log-encoded capabilities without an undefined
  shift.
- Add a forced health recovery, for faults the health sensors can't see,
  such as a NIC that keeps answering while forging completions.

Attack surface reduction (patches 5-8)
- Check the FW capability for every QP transport type, and accept the DC
  vendor flags only on IB_QPT_DRIVER.
- Keep kernel ULPs off the device by clearing kverbs_provider, so only
  uverbs binds.

UMR completion verification (patches 9-14)
- Verify each UMR CQE against a shadow FIFO of the posted requests,
  instead of trusting its wqe_counter. The UMR CQ is the only CQ the
  kernel polls.
- On a forged CQE, keep the UMR path failed and force the health
  recovery.
- Refuse the generic CQ poll path, which trusts the CQE.

Validation of FW reported values (patches 15-21)
- Set the CQ completion handler only once its poll context is set up,
  and ignore completion events for a CQ without one.
- Bound port numbers from EQEs and caps.
- Bound RQ/SQ states from QUERY_RQ and QUERY_SQ.
- Clamp the log_max_* shifts used for the CQ, QP and device limits.

Userspace interface (patches 22-23)
The GET_CONTEXT handshake: the new
UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED input flag and the
IB_UVERBS_CORE_SUPPORT_ADVERSARIAL core_support bit, also reported by
QUERY_CONTEXT.

The matching rdma-core series [1] declares the support and validates
what it reads from the NIC.

[1] https://github.com/linux-rdma/rdma-core/pull/1844

Yishai

Dragos Tatulea (2):
  net/mlx5: Add capability filtering in adversarial mode
  net/mlx5: Add macros for reading log-based caps

Yishai Hadas (21):
  net/mlx5: Add mlx5_trigger_health_work_force()
  net/mlx5: Derive adversarial mode from CC guest memory encryption
  RDMA/mlx5: Reject QP transport types not enabled by FW capability
  RDMA/mlx5: Require IB_QPT_DRIVER for the DC vendor QP types
  RDMA/mlx5: Derive mlx5_ib_adversarial_mode global from mlx5_core
    adversarial API
  RDMA/core: Suppress kverbs_provider for adversarial_mode devices
  RDMA/mlx5: Remove duplicate mlx5_ib_umr_context struct
  RDMA/mlx5: Introduce per-CQ poll_one hook in mlx5_ib_cq
  RDMA/mlx5: Add shadow FIFO infrastructure for UMR completions
  RDMA/mlx5: Verify UMR CQ polling against a hostile NIC
  RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected
  RDMA/mlx5: Block generic CQ poll path in adversarial mode
  RDMA/core: Set the CQ comp_handler after its poll context is set up
  RDMA/mlx5: Ignore completion events for a CQ without a handler
  RDMA/mlx5: Validate EQE port and GSI port before indexing devr.ports[]
  RDMA/mlx5: Validate firmware RQ/SQ state before sqrq_trans[] lookup
  RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks
  RDMA/mlx5: Clamp log_max_* shifts in QP and WQ limit checks
  RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths
  RDMA/core: Report and enforce adversarial mode via GET_CONTEXT
  RDMA/mlx5: Advertise adversarial mode via ib_device::adversarial_mode

 drivers/infiniband/core/cq.c                  |  19 +-
 drivers/infiniband/core/device.c              |   8 +
 drivers/infiniband/core/uverbs_cmd.c          |   5 +
 .../infiniband/core/uverbs_std_types_device.c |  16 +
 drivers/infiniband/hw/mlx5/cq.c               | 220 +++++++++-
 drivers/infiniband/hw/mlx5/gsi.c              |   4 +
 drivers/infiniband/hw/mlx5/mad.c              |   2 +-
 drivers/infiniband/hw/mlx5/main.c             |  71 ++--
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  24 +-
 drivers/infiniband/hw/mlx5/qp.c               |  68 ++--
 drivers/infiniband/hw/mlx5/srq.c              |   2 +-
 drivers/infiniband/hw/mlx5/umr.c              | 111 ++++-
 drivers/infiniband/hw/mlx5/umr.h              |   6 +
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 .../ethernet/mellanox/mlx5/core/caps_filter.c | 381 ++++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/health.c  |  82 +++-
 .../net/ethernet/mellanox/mlx5/core/main.c    |  15 +-
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   2 +
 include/linux/mlx5/device.h                   |  20 +
 include/linux/mlx5/driver.h                   |   3 +
 include/rdma/ib_verbs.h                       |   2 +
 include/uapi/rdma/ib_user_ioctl_cmds.h        |   5 +
 include/uapi/rdma/ib_user_ioctl_verbs.h       |   1 +
 23 files changed, 957 insertions(+), 112 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c

-- 
2.18.1

[PATCH rdma-next 01/23] net/mlx5: Add mlx5_trigger_health_work_force()

[Yishai Hadas]

mlx5_trigger_health_work() queues the health recovery, which enters
MLX5_DEVICE_STATE_INTERNAL_ERROR only if
mlx5_health_check_fatal_sensors() reports a fault.

A consumer can detect a fault the sensors cannot see, such as a device
that keeps answering while misbehaving. It then needs the recovery to
run with the error state forced, whatever the sensors say.

Add mlx5_trigger_health_work_force(), which queues a dedicated work that
runs the recovery with the error state forced. It only queues the work,
so it can be called from atomic context.

Before entering the error state the forced work stops the health poller
and asks the firmware to stop with mlx5_cmd_fast_teardown_hca(), as the
device might still be running. The error state is forced whether or not
the teardown succeeds. The teardown runs under intf_state_mutex and only
while the interface is up, so it cannot race an unload.

Export it for mlx5_ib, which will use it in a later patch of this series
when it detects a NIC forging completion queue entries, a fault the
health sensors cannot see.

Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/health.c  | 82 ++++++++++++++++---
 include/linux/mlx5/driver.h                   |  2 +
 2 files changed, 73 insertions(+), 11 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index aeeb136f5ebc..d000ac335fa7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -627,18 +627,11 @@ mlx5_fw_fatal_reporter_dump(struct devlink_health_reporter *reporter,
 	return err;
 }
 
-static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
+static void mlx5_fw_fatal_reporter_err(struct mlx5_core_dev *dev, bool forced)
 {
 	struct mlx5_fw_reporter_ctx fw_reporter_ctx;
-	struct mlx5_core_health *health;
-	struct mlx5_core_dev *dev;
-	struct devlink *devlink;
-	struct mlx5_priv *priv;
-
-	health = container_of(work, struct mlx5_core_health, fatal_report_work);
-	priv = container_of(health, struct mlx5_priv, health);
-	dev = container_of(priv, struct mlx5_core_dev, priv);
-	devlink = priv_to_devlink(dev);
+	struct mlx5_core_health *health = &dev->priv.health;
+	struct devlink *devlink = priv_to_devlink(dev);
 
 	mutex_lock(&dev->intf_state_mutex);
 	if (test_bit(MLX5_DROP_HEALTH_WORK, &health->flags)) {
@@ -646,8 +639,26 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 		mutex_unlock(&dev->intf_state_mutex);
 		return;
 	}
+	/*
+	 * The firmware might still be running, so stop it first. Do it under
+	 * intf_state_mutex and only while UP, so the cmd interface is valid.
+	 */
+	if (forced && test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state)) {
+		int err;
+
+		mlx5_stop_health_poll(dev, false);
+		err = mlx5_cmd_fast_teardown_hca(dev);
+		if (err)
+			mlx5_core_err(dev, "Failed to stop the device before forcing the error state, err %d\n",
+				      err);
+	}
+
+	/*
+	 * Enter the error state before unlocking, or an unload could send
+	 * TEARDOWN_HCA to the stopped NIC, fail and skip function disable.
+	 */
+	enter_error_state(dev, forced);
 	mutex_unlock(&dev->intf_state_mutex);
-	enter_error_state(dev, false);
 	if (IS_ERR_OR_NULL(health->fw_fatal_reporter)) {
 		devl_lock(devlink);
 		if (mlx5_health_try_recover(dev))
@@ -669,6 +680,33 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 	}
 }
 
+static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
+{
+	struct mlx5_core_health *health;
+	struct mlx5_core_dev *dev;
+	struct mlx5_priv *priv;
+
+	health = container_of(work, struct mlx5_core_health, fatal_report_work);
+	priv = container_of(health, struct mlx5_priv, health);
+	dev = container_of(priv, struct mlx5_core_dev, priv);
+
+	mlx5_fw_fatal_reporter_err(dev, false);
+}
+
+static void mlx5_fw_fatal_reporter_force_err_work(struct work_struct *work)
+{
+	struct mlx5_core_health *health;
+	struct mlx5_core_dev *dev;
+	struct mlx5_priv *priv;
+
+	health = container_of(work, struct mlx5_core_health,
+			      force_fatal_report_work);
+	priv = container_of(health, struct mlx5_priv, health);
+	dev = container_of(priv, struct mlx5_core_dev, priv);
+
+	mlx5_fw_fatal_reporter_err(dev, true);
+}
+
 #define MLX5_FW_REPORTER_ECPF_GRACEFUL_PERIOD 180000
 #define MLX5_FW_REPORTER_PF_GRACEFUL_PERIOD 60000
 #define MLX5_FW_REPORTER_VF_GRACEFUL_PERIOD 30000
@@ -759,6 +797,25 @@ void mlx5_trigger_health_work(struct mlx5_core_dev *dev)
 		queue_work(health->wq, &health->fatal_report_work);
 }
 
+/**
+ * mlx5_trigger_health_work_force - enter the error state and recover, even
+ * if the health sensors report the device as healthy
+ * @dev: mlx5 core device
+ *
+ * For faults only a consumer can detect, where the firmware keeps answering
+ * and mlx5_health_check_fatal_sensors() sees nothing wrong. Without forcing,
+ * the device would stay in MLX5_DEVICE_STATE_UP and consumers waiting on
+ * outstanding work would never be released.
+ */
+void mlx5_trigger_health_work_force(struct mlx5_core_dev *dev)
+{
+	struct mlx5_core_health *health = &dev->priv.health;
+
+	if (!mlx5_dev_is_lightweight(dev))
+		queue_work(health->wq, &health->force_fatal_report_work);
+}
+EXPORT_SYMBOL_GPL(mlx5_trigger_health_work_force);
+
 #define MLX5_MSEC_PER_HOUR (MSEC_PER_SEC * 60 * 60)
 static void mlx5_health_log_ts_update(struct work_struct *work)
 {
@@ -874,6 +931,7 @@ void mlx5_drain_health_wq(struct mlx5_core_dev *dev)
 	cancel_delayed_work_sync(&health->update_fw_log_ts_work);
 	cancel_work_sync(&health->report_work);
 	cancel_work_sync(&health->fatal_report_work);
+	cancel_work_sync(&health->force_fatal_report_work);
 }
 
 void mlx5_health_cleanup(struct mlx5_core_dev *dev)
@@ -911,6 +969,8 @@ int mlx5_health_init(struct mlx5_core_dev *dev)
 	if (!health->wq)
 		goto out_err;
 	INIT_WORK(&health->fatal_report_work, mlx5_fw_fatal_reporter_err_work);
+	INIT_WORK(&health->force_fatal_report_work,
+		  mlx5_fw_fatal_reporter_force_err_work);
 	INIT_WORK(&health->report_work, mlx5_fw_reporter_err_work);
 	INIT_DELAYED_WORK(&health->update_fw_log_ts_work, mlx5_health_log_ts_update);
 
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index abb99371ff13..a1275e6aa6c1 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -450,6 +450,7 @@ struct mlx5_core_health {
 	struct workqueue_struct	       *wq;
 	unsigned long			flags;
 	struct work_struct		fatal_report_work;
+	struct work_struct		force_fatal_report_work;
 	struct work_struct		report_work;
 	struct devlink_health_reporter *fw_reporter;
 	struct devlink_health_reporter *fw_fatal_reporter;
@@ -1034,6 +1035,7 @@ void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health);
 void mlx5_start_health_fw_log_up(struct mlx5_core_dev *dev);
 void mlx5_drain_health_wq(struct mlx5_core_dev *dev);
 void mlx5_trigger_health_work(struct mlx5_core_dev *dev);
+void mlx5_trigger_health_work_force(struct mlx5_core_dev *dev);
 int mlx5_frag_buf_alloc_node(struct mlx5_core_dev *dev, int size,
 			     struct mlx5_frag_buf *buf, int node);
 void mlx5_frag_buf_free(struct mlx5_core_dev *dev, struct mlx5_frag_buf *buf);
-- 
2.18.1

[PATCH rdma-next 02/23] net/mlx5: Derive adversarial mode from CC guest memory encryption

[Yishai Hadas]

Add mlx5_is_adversarial() to report whether the driver treats the NIC
as untrusted. In a confidential computing guest, the hypervisor and any
passthrough device are outside the guest's trust boundary, so the mode
is derived from cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT).

This turns adversarial mode on automatically in such guests, without
any administrator configuration.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/main.c | 7 +++++++
 include/linux/mlx5/driver.h                    | 1 +
 2 files changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 1a45474ce03d..f9e10464a71b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -31,6 +31,7 @@
  */
 
 #include <linux/highmem.h>
+#include <linux/cc_platform.h>
 #include <linux/module.h>
 #include <linux/init.h>
 #include <linux/errno.h>
@@ -484,6 +485,12 @@ bool mlx5_is_roce_on(struct mlx5_core_dev *dev)
 }
 EXPORT_SYMBOL(mlx5_is_roce_on);
 
+bool mlx5_is_adversarial(void)
+{
+	return cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT);
+}
+EXPORT_SYMBOL_GPL(mlx5_is_adversarial);
+
 static int handle_hca_cap_2(struct mlx5_core_dev *dev, void *set_ctx)
 {
 	bool do_set = false;
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index a1275e6aa6c1..3d3bc16e581d 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -1328,6 +1328,7 @@ enum {
 };
 
 bool mlx5_is_roce_on(struct mlx5_core_dev *dev);
+bool mlx5_is_adversarial(void);
 
 static inline bool mlx5_get_roce_state(struct mlx5_core_dev *dev)
 {
-- 
2.18.1

[PATCH rdma-next 03/23] net/mlx5: Add capability filtering in adversarial mode

[Yishai Hadas]

From: Dragos Tatulea <dtatulea@nvidia.com>

When in adversarial mode, trust only a subset of capabilities
and do some additional validations.

Start off by clearing all firmware capabilities and enabling
them one by one. Some numeric capabilities are validated.
Unexpected values will trigger a driver load failure flow.

Where it applies, the filtering is done on both current and max caps.
Allowed capabilities are simply passed through. Clamped and checked
capabilities also apply to max values.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/Makefile  |   2 +-
 .../ethernet/mellanox/mlx5/core/caps_filter.c | 381 ++++++++++++++++++
 .../net/ethernet/mellanox/mlx5/core/main.c    |   8 +-
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   2 +
 4 files changed, 388 insertions(+), 5 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index a1871259210d..dc8eabee631f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -11,7 +11,7 @@ obj-$(CONFIG_MLX5_CORE) += mlx5_core.o
 #
 # mlx5 core basic
 #
-mlx5_core-y :=	main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
+mlx5_core-y :=	main.o caps_filter.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
 		health.o mcg.o cq.o alloc.o port.o mr.o pd.o \
 		transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \
 		fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
new file mode 100644
index 000000000000..67b260339745
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/caps_filter.c
@@ -0,0 +1,381 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */
+
+#include "linux/mlx5/device.h"
+#include <linux/mlx5/driver.h>
+#include <linux/mlx5/qp.h>
+#include "mlx5_core.h"
+
+#define MLX5_HCA_CAP_ALLOW(cap, fld) \
+	MLX5_SET(cap, dst, fld, MLX5_GET(cap, src, fld))
+#define MLX5_HCA_CAP_ALLOW64(cap, fld) \
+	MLX5_SET64(cap, dst, fld, MLX5_GET64(cap, src, fld))
+
+/* Allow a numeric capability range restricted to [lo, hi]. */
+#define MLX5_HCA_CAP_CLAMP(cap, fld, lo, hi) \
+	MLX5_SET(cap, dst, fld, mlx5_cap_clamp(dev, #fld, \
+					       MLX5_GET(cap, src, fld), \
+					       (lo), (hi)))
+
+/* Check cap range and return error otherwise. */
+#define MLX5_HCA_CAP_CHECK(cap, fld, lo, hi) \
+	mlx5_cap_check(dev, #fld, MLX5_GET(cap, dst, fld), (lo), (hi), false)
+/* Same but zero is also a valid value. */
+#define MLX5_HCA_CAP_CHECK_Z(cap, fld, lo, hi) \
+	mlx5_cap_check(dev, #fld, MLX5_GET(cap, dst, fld), (lo), (hi), true)
+
+static u32 mlx5_cap_clamp(struct mlx5_core_dev *dev, const char *name,
+			  u32 val, u32 lo, u32 hi)
+{
+	u32 clamped = clamp_t(u32, val, lo, hi);
+
+	if (clamped != val)
+		mlx5_core_warn(dev, "cap filter: FW reported %s=%u, outside [%u, %u], using %u\n",
+			       name, val, lo, hi, clamped);
+
+	return clamped;
+}
+
+static int mlx5_cap_check(struct mlx5_core_dev *dev, const char *name,
+			  u32 val, u32 lo, u32 hi, bool zero)
+{
+	if (val >= lo && val <= hi)
+		return 0;
+
+	if (zero && !val)
+		return 0;
+
+	mlx5_core_err(dev, "cap filter: FW reported %s=%u, outside [%u, %u] %s\n",
+		      name, val, lo, hi,
+		      zero ? "or 0" : "");
+
+	return -EINVAL;
+}
+
+static int mlx5_adversarial_filter_general(struct mlx5_core_dev *dev, void *dst,
+					   void *src)
+{
+	int err;
+
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, page_request_disable);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_resource_manager);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, hca_cap_2);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_teardown_request);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_in_use);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_active);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_allocated);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_on_vhca_state_invalid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_srq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_qp_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, event_cap);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_srq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_sgl_for_optimized_performance);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_cq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_write_umr);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read_umr);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_cq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_write);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read_pci_enabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, terminate_scatter_list_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fast_teardown);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_eq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_indirection);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fixed_buffer_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, force_teardown);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_extended_translation_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_klm_list_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qpc_extension);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_req_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cq_overrun);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_res_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, release_all_pages);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce_accl);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_req_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_ra_res_qp);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, end_pad);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cc_query_allowed);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cc_modify_allowed);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cache_line_128byte);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rts2rts_qp_counters_set_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_int_rq_oob);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sbcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, gid_table_size);
+
+	/* Values outside of [0, 4] range are unexpected. */
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, gid_table_size, 0, 4);
+	if (err)
+		return err;
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, out_of_seq_cnt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, retransmission_q_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, debug);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, modify_rq_counter_set_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rq_delay_drop);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_qp_cnt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pkey_table_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_group_manager);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_queue_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, ets);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, nic_flow_table);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, mcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pcam_reg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, local_ca_ack_delay);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, enhanced_error_q_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, beacon_led);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, port_type);
+	/* 0 is invalid, 8 is for MLX5_MAX_PORTS */
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_ports, 1, 8);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_msg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_tc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dcbx);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, general_notification_event);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, stat_rate_support);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pci_sync_for_fw_update_event);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cqe_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_modify_entity_size_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_modify_atomic_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_indirect_mkey_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_fence);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cmdif_checksum);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sho);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qos);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, eth_net_offloads);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, atomic);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_moderation);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_period_mode_modify);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cq_period_start_from_cqe);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, apm);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, umr_ptr_rlky);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qp_packet_based);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pkv);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, uar_4k);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, fl_rc_qp_when_roce_disabled);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_pg_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, bf);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, driver_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, mkey_by_name);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_bf_reg_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, disciplined_fr_counter);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, qp_error_syndrome);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_lag_ports, 0, 1);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_wqe_sz_sq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, icm_mng_function_id_mode);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, tis_tir_td_order);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_wqe_sz_rq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, query_host_net_function_num_max);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_qp_mcg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_mcg);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_transport_domain);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, relaxed_ordering_read);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_pd);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_ud);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_uc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_xrc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_dc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, dp_ordering_ooo_all_rc);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, pcie_reset_using_hotreset_method);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap,
+			   pci_sync_for_fw_update_with_driver_unload);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_steering_fail);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vport_counter_local_loopback);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, q_counter_aggregation);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, q_counter_other_vport);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, nic_receive_steering_discard);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, receive_discard_vport_down);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, transmit_discard_vport_down);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, eq_overrun_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, invalid_command_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, quota_exceeded_count);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_flow_counter_bulk);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_sq);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_tir);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, basic_cyclic_rcv_wqe);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_rqt);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, log_max_rqt_size, 0, 13);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_bar_uar_access);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vnic_env_cnt_odp_page_fault);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, roce_rw_supported);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_wq_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, vhca_state);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_vlan_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_mc_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_current_uc_list);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sq_ts_format);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, rq_ts_format);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, steering_format_version);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, create_qp_start_hint);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, ats);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, cross_vhca_rqt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_uctx);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_num_eqs);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_uar_page_sz);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_geneve_tlv_options);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_geneve_tlv_option_data_len);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_dci_stream_channels);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, log_max_dci_errored_streams);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, crossing_vhca_mkey);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, native_port_num);
+	MLX5_HCA_CAP_CLAMP(cmd_hca_cap, num_vhca_ports, 0, 1);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, sw_owner_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_num_of_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_ppcnt_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_q_monitor_counters);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, apu);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, uctx_cap);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, num_total_dynamic_vf_msix);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, min_dynamic_vf_msix_table_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, data_direct);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, max_dynamic_vf_msix_table_size);
+	MLX5_HCA_CAP_ALLOW64(cmd_hca_cap, vhca_tunnel_commands);
+
+	MLX5_SET(cmd_hca_cap, dst, num_of_uars_per_page, MLX5_UARS_IN_PAGE);
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap, flow_counter_bulk_alloc);
+
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, log_pg_sz,
+				 MLX5_ADAPTER_PAGE_SHIFT,
+				 MLX5_ADAPTER_PAGE_SHIFT);
+	if (err)
+		return err;
+
+#define LOG_BF_REG_SIZE_MIN ilog2(2 * MLX5_SEND_WQE_BB)
+#define LOG_BF_REG_SIZE_MAX \
+	ilog2((MLX5_ADAPTER_PAGE_SIZE - MLX5_BF_OFFSET) / MLX5_BFREGS_PER_UAR)
+
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap, log_bf_reg_size,
+				 LOG_BF_REG_SIZE_MIN,
+				 LOG_BF_REG_SIZE_MAX);
+	if (err)
+		return err;
+
+#undef LOG_BF_REG_SIZE_MAX
+#undef LOG_BF_REG_SIZE_MIN
+
+	return MLX5_HCA_CAP_CHECK(cmd_hca_cap, min_dynamic_vf_msix_table_size, 0,
+				  MLX5_GET(cmd_hca_cap, dst,
+					   max_dynamic_vf_msix_table_size));
+}
+
+static int mlx5_adversarial_filter_general_2(struct mlx5_core_dev *dev, void *dst,
+					     void *src)
+{
+	int err;
+	u32 val;
+
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, dp_ordering_force);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, query_vuid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, umr_log_entity_size_5);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_insert_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_insert_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_remove_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_reformat_remove_offset);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, sf_eq_usage);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_rqt_vhca_id);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   min_mkey_log_entity_size_fixed_buffer_valid);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, log_min_mkey_entity_size);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   min_mkey_log_entity_size_fixed_buffer);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2,
+			   max_mkey_log_entity_size_fixed_buffer);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_mkey_log_entity_size_mtt);
+	MLX5_HCA_CAP_ALLOW(cmd_hca_cap_2, max_num_eqs_24b);
+
+	err = MLX5_HCA_CAP_CHECK_Z(cmd_hca_cap_2, max_mkey_log_entity_size_mtt,
+				   MLX5_ADAPTER_PAGE_SHIFT, 63);
+	if (err)
+		return err;
+
+	err = MLX5_HCA_CAP_CHECK_Z(cmd_hca_cap_2,
+				   max_mkey_log_entity_size_fixed_buffer,
+				   MLX5_ADAPTER_PAGE_SHIFT, 63);
+	if (err)
+		return err;
+
+	val = MLX5_GET(cmd_hca_cap_2, dst, max_mkey_log_entity_size_mtt) ?: 31;
+	err = MLX5_HCA_CAP_CHECK(cmd_hca_cap_2, log_min_mkey_entity_size, 0,
+				 val);
+	if (err)
+		return err;
+
+	/* The fixed buffer min is don't-care when its valid bit is clear. */
+	if (!MLX5_GET(cmd_hca_cap_2, dst,
+		      min_mkey_log_entity_size_fixed_buffer_valid))
+		return 0;
+
+	val = MLX5_GET(cmd_hca_cap_2, dst,
+		       max_mkey_log_entity_size_fixed_buffer) ?: 31;
+	return MLX5_HCA_CAP_CHECK(cmd_hca_cap_2,
+				  min_mkey_log_entity_size_fixed_buffer, 0, val);
+}
+
+static int mlx5_adversarial_filter_per_protocol(struct mlx5_core_dev *dev,
+						 void *dst, void *src)
+{
+	memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, csum_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, vlan_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   self_lb_en_modifiable);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps, max_lso_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   wqe_inline_mode);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   rss_ind_tbl_cap);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   cqe_checksum_full);
+	MLX5_HCA_CAP_ALLOW(per_protocol_networking_offload_caps,
+			   max_vxlan_udp_ports);
+
+	return 0;
+}
+
+#undef MLX5_HCA_CAP_ALLOW
+#undef MLX5_HCA_CAP_ALLOW64
+#undef MLX5_HCA_CAP_CLAMP
+#undef MLX5_HCA_CAP_CHECK
+#undef MLX5_HCA_CAP_CHECK_Z
+
+int mlx5_store_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
+		    void *dst, void *src)
+{
+	if (!mlx5_is_adversarial()) {
+		memcpy(dst, src, MLX5_UN_SZ_BYTES(hca_cap_union));
+		return 0;
+	}
+
+	switch (cap_type) {
+	case MLX5_CAP_GENERAL:
+		return mlx5_adversarial_filter_general(dev, dst, src);
+	case MLX5_CAP_GENERAL_2:
+		return mlx5_adversarial_filter_general_2(dev, dst, src);
+	case MLX5_CAP_ETHERNET_OFFLOADS:
+		return mlx5_adversarial_filter_per_protocol(dev, dst, src);
+
+	case MLX5_CAP_ATOMIC:
+	case MLX5_CAP_ROCE:
+	case MLX5_CAP_FLOW_TABLE:
+	case MLX5_CAP_DEV_EVENT:
+		memcpy(dst, src, MLX5_UN_SZ_BYTES(hca_cap_union));
+		break;
+
+	default:
+		memset(dst, 0, MLX5_UN_SZ_BYTES(hca_cap_union));
+		break;
+	}
+
+	return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index f9e10464a71b..e76a4486b5a3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -325,12 +325,12 @@ int mlx5_core_get_caps_mode(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_ty
 
 	switch (cap_mode) {
 	case HCA_CAP_OPMOD_GET_MAX:
-		memcpy(dev->caps.hca[cap_type]->max, hca_caps,
-		       MLX5_UN_SZ_BYTES(hca_cap_union));
+		err = mlx5_store_caps(dev, cap_type,
+				      dev->caps.hca[cap_type]->max, hca_caps);
 		break;
 	case HCA_CAP_OPMOD_GET_CUR:
-		memcpy(dev->caps.hca[cap_type]->cur, hca_caps,
-		       MLX5_UN_SZ_BYTES(hca_cap_union));
+		err = mlx5_store_caps(dev, cap_type,
+				      dev->caps.hca[cap_type]->cur, hca_caps);
 		break;
 	default:
 		mlx5_core_warn(dev,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
index d84d0424d04e..0084bed669f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h
@@ -215,6 +215,8 @@ static inline int mlx5_flexible_inlen(struct mlx5_core_dev *dev, size_t fixed,
 	mlx5_flexible_inlen(dev, fixed, item_size, num_items, __func__, __LINE__)
 
 int mlx5_core_get_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type);
+int mlx5_store_caps(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
+		    void *dst, void *src);
 int mlx5_core_get_caps_mode(struct mlx5_core_dev *dev, enum mlx5_cap_type cap_type,
 			    enum mlx5_cap_mode cap_mode);
 int mlx5_query_hca_caps(struct mlx5_core_dev *dev);
-- 
2.18.1

[PATCH rdma-next 04/23] net/mlx5: Add macros for reading log-based caps

[Yishai Hadas]

From: Dragos Tatulea <dtatulea@nvidia.com>

Add macros that can be used to exponentiate log based capabilities.

When a capability can represent more than the used version (u32 , u64),
value will be clamped to the largest representable power of two.

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Dragos Tatulea <dtatulea@nvidia.com>
---
 include/linux/mlx5/device.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index 37727b5ab78f..f45454f65d5c 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -38,6 +38,7 @@
 #include <linux/mlx5/mlx5_ifc.h>
 #include <linux/mlx5/mlx5_ifc_macros.h>
 #include <linux/bitfield.h>
+#include <linux/minmax.h>
 
 #if defined(__LITTLE_ENDIAN)
 #define MLX5_SET_HOST_ENDIANNESS	0
@@ -47,6 +48,16 @@
 #error Host endianness not defined
 #endif
 
+/* Largest shift count that is defined behaviour for a value of type @T. */
+#define __mlx5_max_shift(T) (BITS_PER_TYPE(T) - 1 - is_signed_type(T))
+
+/* Calculate 2^@fld from a log2 cap field. Exponents that don't fit in @T
+ * saturate to the largest representable power of two (not to the type
+ * maximum), avoiding undefined shifts.
+ */
+#define MLX5_GET_LOGSZ_CLAMP(typ, p, fld, T) \
+	((T)((T)1 << min_t(u32, MLX5_GET(typ, p, fld), __mlx5_max_shift(T))))
+
 enum mlx5_inline_modes {
 	MLX5_INLINE_MODE_NONE,
 	MLX5_INLINE_MODE_L2,
@@ -1096,6 +1107,15 @@ enum mlx5_qcam_feature_groups {
 #define MLX5_CAP_GEN_MAX(mdev, cap) \
 	MLX5_GET(cmd_hca_cap, mdev->caps.hca[MLX5_CAP_GENERAL]->max, cap)
 
+/* 2^cap for log2-encoded general caps */
+#define MLX5_CAP_GEN_LOGSZ_U32(mdev, cap) \
+	MLX5_GET_LOGSZ_CLAMP(cmd_hca_cap, \
+			     mdev->caps.hca[MLX5_CAP_GENERAL]->cur, cap, u32)
+
+#define MLX5_CAP_GEN_LOGSZ_U64(mdev, cap) \
+	MLX5_GET_LOGSZ_CLAMP(cmd_hca_cap, \
+			     mdev->caps.hca[MLX5_CAP_GENERAL]->cur, cap, u64)
+
 #define MLX5_CAP_GEN_2(mdev, cap) \
 	MLX5_GET(cmd_hca_cap_2, mdev->caps.hca[MLX5_CAP_GENERAL_2]->cur, cap)
 
-- 
2.18.1

[PATCH rdma-next 05/23] RDMA/mlx5: Reject QP transport types not enabled by FW capability

[Yishai Hadas]

check_qp_type() checks the FW capability only for XRC and DC. RC, UC
and UD QPs are created even if the FW does not report the matching
capability bit.

Check the rc, uc and ud capabilities as well, so that QP creation is
refused for a transport the FW does not support.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 627dd17804c9..943ee5c55538 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2824,15 +2824,24 @@ static int check_qp_type(struct mlx5_ib_dev *dev, struct ib_qp_init_attr *attr,
 	case IB_QPT_XRC_INI:
 		if (!MLX5_CAP_GEN(dev->mdev, xrc))
 			goto out;
-		fallthrough;
+		break;
 	case IB_QPT_RC:
+		if (!MLX5_CAP_GEN(dev->mdev, rc))
+			goto out;
+		break;
 	case IB_QPT_UC:
+		if (!MLX5_CAP_GEN(dev->mdev, uc))
+			goto out;
+		break;
+	case IB_QPT_UD:
+		if (!MLX5_CAP_GEN(dev->mdev, ud))
+			goto out;
+		break;
 	case IB_QPT_SMI:
 	case MLX5_IB_QPT_HW_GSI:
 	case IB_QPT_DRIVER:
 	case IB_QPT_GSI:
 	case IB_QPT_RAW_PACKET:
-	case IB_QPT_UD:
 	case MLX5_IB_QPT_REG_UMR:
 		break;
 	default:
-- 
2.18.1

[PATCH rdma-next 06/23] RDMA/mlx5: Require IB_QPT_DRIVER for the DC vendor QP types

[Yishai Hadas]

check_qp_type() checks the dct capability only for IB_QPT_DRIVER, but
process_vendor_flags() turns any QP that carries a DC type flag into a
DCI or DCT QP. A QP created as, for example, IB_QPT_RC with a DC flag
thus becomes a DC QP without the capability check.

Accept the DC type flags only on IB_QPT_DRIVER, the type userspace
uses to create DC QPs.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 943ee5c55538..19cbf6b2b979 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2965,18 +2965,22 @@ static int process_vendor_flags(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,
 
 	switch (flags & (MLX5_QP_FLAG_TYPE_DCT | MLX5_QP_FLAG_TYPE_DCI)) {
 	case MLX5_QP_FLAG_TYPE_DCI:
+		if (qp->type != IB_QPT_DRIVER)
+			return -EINVAL;
 		qp->type = MLX5_IB_QPT_DCI;
 		break;
 	case MLX5_QP_FLAG_TYPE_DCT:
+		if (qp->type != IB_QPT_DRIVER)
+			return -EINVAL;
 		qp->type = MLX5_IB_QPT_DCT;
 		break;
+	case 0:
+		/* IB_QPT_DRIVER needs a subtype */
+		if (qp->type == IB_QPT_DRIVER)
+			return -EINVAL;
+		break;
 	default:
-		if (qp->type != IB_QPT_DRIVER)
-			break;
-		/*
-		 * It is IB_QPT_DRIVER and or no subtype or
-		 * wrong subtype were provided.
-		 */
+		/* Both subtypes */
 		return -EINVAL;
 	}
 
-- 
2.18.1

[PATCH rdma-next 07/23] RDMA/mlx5: Derive mlx5_ib_adversarial_mode global from mlx5_core adversarial API

[Yishai Hadas]

Use the mlx5_core mlx5_is_adversarial() exported API to set a
module-level static bool mlx5_ib_adversarial_mode at init time,
instead of a per-device bit field. This avoids propagating the flag
through every mlx5_ib_dev pointer and makes the module-wide policy
clear at the call sites.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c    | 3 +++
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
 2 files changed, 4 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 65455aedd559..f4f2ad80e76c 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -76,6 +76,7 @@ enum {
 	MLX5_ATOMIC_SIZE_QP_8BYTES = 1 << 3,
 };
 
+bool mlx5_ib_adversarial_mode;
 static struct workqueue_struct *mlx5_ib_event_wq;
 static LIST_HEAD(mlx5_ib_unaffiliated_port_list);
 static LIST_HEAD(mlx5_ib_dev_list);
@@ -5431,6 +5432,8 @@ static int __init mlx5_ib_init(void)
 		return -ENOMEM;
 	}
 
+	mlx5_ib_adversarial_mode = mlx5_is_adversarial();
+
 	ret = mlx5_ib_qp_event_init();
 	if (ret)
 		goto qp_event_err;
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a6ee14ffd7d..0f81b8aa487e 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1582,6 +1582,7 @@ static inline int get_uars_per_sys_page(struct mlx5_ib_dev *dev, bool lib_suppor
 }
 
 extern void *xlt_emergency_page;
+extern bool mlx5_ib_adversarial_mode;
 
 int bfregn_to_uar_index(struct mlx5_ib_dev *dev,
 			struct mlx5_bfreg_info *bfregi, u32 bfregn,
-- 
2.18.1

[PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Yishai Hadas]

In adversarial mode the NIC is not trusted. Kernel ULPs such as MAD,
IB CM, rdma_cm, IPoIB, SRP, iSER and NVMe-oF trust what the device
reports, its completions and the data it writes, so they must not use
such a device.

Add an adversarial_mode field to struct ib_device and clear
kverbs_provider when it is set. add_client_context() then skips every
client but uverbs, the only one that sets no_kverbs_req, so the kernel
ULPs never bind to the device.

The driver still implements the kverbs ops, as the uverbs path shares
them, so kverbs_provider can't be cleared by the existing check of the
mandatory ops.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/device.c | 8 ++++++++
 include/rdma/ib_verbs.h          | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index 29af9fe860e2..eb6aa567d365 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -312,6 +312,14 @@ static void ib_device_check_mandatory(struct ib_device *device)
 			break;
 		}
 	}
+
+	/*
+	 * Kernel ULPs must not bind to an adversarial_mode device. Its kverbs
+	 * ops are still present, as uverbs uses them, so clear kverbs_provider
+	 * explicitly. uverbs sets no_kverbs_req, so it still binds.
+	 */
+	if (device->adversarial_mode)
+		device->kverbs_provider = false;
 }
 
 /*
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d6eaa3fee66e..6209a5166f0d 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -3008,6 +3008,8 @@ struct ib_device {
 	u16                          use_cq_dim:1;
 	/* CoCo guest with DMA bounce buffering required */
 	u16                          cc_dma_bounce:1;
+	/* Untrusted-NIC, adversarial mode is required */
+	u8			     adversarial_mode;
 	u8                           node_type;
 	u32			     phys_port_cnt;
 	struct ib_device_attr        attrs;
-- 
2.18.1

[PATCH rdma-next 09/23] RDMA/mlx5: Remove duplicate mlx5_ib_umr_context struct

[Yishai Hadas]

mlx5_ib_umr_context in mlx5_ib.h and mlx5r_umr_context in umr.h have
identical layouts: an ib_cqe, a status, and a completion.  The done
callback used mlx5_ib_umr_context via container_of, while every other
site in umr.c used mlx5r_umr_context.

Remove the duplicate and use mlx5r_umr_context consistently throughout.
No behaviour change.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 6 ------
 drivers/infiniband/hw/mlx5/umr.c     | 4 ++--
 2 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 0f81b8aa487e..980b41800df5 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -749,12 +749,6 @@ struct mlx5_ib_mw {
 	struct mlx5_ib_mkey	mmkey;
 };
 
-struct mlx5_ib_umr_context {
-	struct ib_cqe		cqe;
-	enum ib_wc_status	status;
-	struct completion	done;
-};
-
 enum {
 	MLX5_UMR_STATE_UNINIT,
 	MLX5_UMR_STATE_ACTIVE,
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 711075e3d9ea..4a3fa79eba3a 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -339,8 +339,8 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 
 static void mlx5r_umr_done(struct ib_cq *cq, struct ib_wc *wc)
 {
-	struct mlx5_ib_umr_context *context =
-		container_of(wc->wr_cqe, struct mlx5_ib_umr_context, cqe);
+	struct mlx5r_umr_context *context =
+		container_of(wc->wr_cqe, struct mlx5r_umr_context, cqe);
 
 	context->status = wc->status;
 	complete(&context->done);
-- 
2.18.1

[PATCH rdma-next 10/23] RDMA/mlx5: Introduce per-CQ poll_one hook in mlx5_ib_cq

[Yishai Hadas]

Add a poll_one function pointer to struct mlx5_ib_cq and initialize it
to mlx5_poll_one in both mlx5_ib_create_cq() (kernel-ULP CQs) and
mlx5_ib_create_user_cq() (uverbs CQs, the path used by both the
modern ioctl-based and legacy write-based CQ creation commands).
mlx5_ib_poll_cq() is updated to dispatch through the pointer instead
of calling mlx5_poll_one() directly.

This is a pure refactor with no behaviour change. The indirection
allows a specialised poll_one implementation to be installed on a
specific CQ, which a follow-up patch uses to verify the UMR CQ against
a hostile NIC.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c      | 9 ++++++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 ++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 55ffd7d4de51..11c1153747c2 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -633,7 +633,7 @@ int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 		soft_polled = poll_soft_wc(cq, num_entries, wc, false);
 
 	for (npolled = 0; npolled < num_entries - soft_polled; npolled++) {
-		if (mlx5_poll_one(cq, &cur_qp, wc + soft_polled + npolled))
+		if (cq->poll_one(cq, &cur_qp, wc + soft_polled + npolled))
 			break;
 	}
 
@@ -1010,6 +1010,8 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 
 	cq->mcq.comp = mlx5_add_cq_to_tasklet;
 	cq->mcq.tasklet_ctx.comp = mlx5_ib_cq_comp;
+	/* Set before the CQ is created, see mlx5_ib_create_cq() */
+	cq->poll_one = mlx5_poll_one;
 
 	err = mlx5_core_create_cq(dev->mdev, &cq->mcq, cqb, inlen, out, sizeof(out));
 	if (err)
@@ -1093,6 +1095,11 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	MLX5_SET64(cqc, cqc, dbr_addr, cq->db.dma);
 
 	cq->mcq.comp = mlx5_ib_cq_comp;
+	/*
+	 * Set before mlx5_core_create_cq(). No real CQE can arrive before a
+	 * QP uses the CQ, but a forged one can make mlx5_ib_poll_cq() call it.
+	 */
+	cq->poll_one = mlx5_poll_one;
 
 	err = mlx5_core_create_cq(dev->mdev, &cq->mcq, cqb, inlen, out,
 				  sizeof(out));
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 980b41800df5..4833f42b3fd1 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -592,6 +592,8 @@ struct mlx5_ib_cq {
 	enum ib_cq_notify_flags notify_flags;
 	struct work_struct	notify_work;
 	u16			private_flags; /* Use mlx5_ib_cq_pr_flags */
+	int (*poll_one)(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+			struct ib_wc *wc);
 };
 
 struct mlx5_ib_wc {
-- 
2.18.1

[PATCH rdma-next 11/23] RDMA/mlx5: Add shadow FIFO infrastructure for UMR completions

[Yishai Hadas]

The UMR CQ is polled by the generic path, which finds the completed
request by indexing qp->sq.wrid[] with the CQE wqe_counter. A hostile
NIC can forge it to complete a request other than the one that really
completed, or one that is not pending at all.

The UMR QP is RC and signals every WR, so its completions arrive in
posting order. In adversarial mode, keep a FIFO of the posted UMR
contexts, so a follow-up patch can check each CQE against the oldest
pending request instead of trusting it. Each context also records the
SQ producer counter it was posted at, as its address alone doesn't
identify the post: it is usually on the stack and reused across posts.

Nothing consumes the FIFO yet.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 10 +++++++
 drivers/infiniband/hw/mlx5/umr.c     | 42 +++++++++++++++++++++++-----
 drivers/infiniband/hw/mlx5/umr.h     |  6 ++++
 3 files changed, 51 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4833f42b3fd1..9cc4b2f99b69 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -758,6 +758,12 @@ enum {
 	MLX5_UMR_STATE_ERR,
 };
 
+#define MLX5_UMR_MAX_WR 128
+static_assert((MLX5_UMR_MAX_WR & (MLX5_UMR_MAX_WR - 1)) == 0,
+	      "MLX5_UMR_MAX_WR must be a power of two for FIFO index masking");
+
+struct mlx5r_umr_context;
+
 struct umr_common {
 	struct ib_pd	*pd;
 	struct ib_cq	*cq;
@@ -771,6 +777,10 @@ struct umr_common {
 	unsigned int state;
 	/* Protects from repeat UMR QP creation */
 	struct mutex init_lock;
+	/* Posted UMR contexts in posting order, adversarial_mode only */
+	struct mlx5r_umr_context **pending;
+	u32 pending_head;
+	u32 pending_tail;
 };
 
 struct mlx5_ib_port_resources {
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 4a3fa79eba3a..aba79a19d627 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -98,9 +98,7 @@ static int umr_check_mkey_mask(struct mlx5_ib_dev *dev, u64 mask)
 	return 0;
 }
 
-enum {
-	MAX_UMR_WR = 128,
-};
+#define MAX_UMR_WR MLX5_UMR_MAX_WR
 
 static int mlx5r_umr_qp_rst2rts(struct mlx5_ib_dev *dev, struct ib_qp *qp)
 {
@@ -181,6 +179,16 @@ int mlx5r_umr_resource_init(struct mlx5_ib_dev *dev)
 	if (ret)
 		goto destroy_qp;
 
+	if (mlx5_ib_adversarial_mode) {
+		dev->umrc.pending = kcalloc(MAX_UMR_WR,
+					    sizeof(*dev->umrc.pending),
+					    GFP_KERNEL);
+		if (!dev->umrc.pending) {
+			ret = -ENOMEM;
+			goto destroy_qp;
+		}
+	}
+
 	dev->umrc.cq = cq;
 
 	sema_init(&dev->umrc.sem, MAX_UMR_WR);
@@ -208,6 +216,7 @@ void mlx5r_umr_resource_cleanup(struct mlx5_ib_dev *dev)
 	/* After device init, UMR cp/qp are not unset during the lifetime. */
 	ib_destroy_qp(dev->umrc.qp);
 	ib_free_cq(dev->umrc.cq);
+	kfree(dev->umrc.pending);
 }
 
 int mlx5r_umr_init(struct mlx5_ib_dev *dev)
@@ -236,7 +245,8 @@ void mlx5r_umr_cleanup(struct mlx5_ib_dev *dev)
 }
 
 
-static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
+static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey,
+			       struct mlx5r_umr_context *ctx,
 			       struct mlx5r_umr_wqe *wqe, bool with_data)
 {
 	unsigned int wqe_size =
@@ -245,6 +255,7 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
 				    sizeof(struct mlx5_wqe_data_seg);
 	struct mlx5_ib_dev *dev = to_mdev(ibqp->device);
 	struct mlx5_core_dev *mdev = dev->mdev;
+	struct umr_common *umrc = &dev->umrc;
 	struct mlx5_ib_qp *qp = to_mqp(ibqp);
 	struct mlx5_wqe_ctrl_seg *ctrl;
 	union {
@@ -267,13 +278,30 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey, struct ib_cqe *cqe,
 		goto out;
 
 	qp->sq.wr_data[idx] = MLX5_IB_WR_UMR;
+	if (mlx5_ib_adversarial_mode) {
+		/* mlx5r_finish_wqe() advances cur_post, so record it first */
+		ctx->wqe_ctr = qp->sq.cur_post;
+	}
 
 	mlx5r_memcpy_send_wqe(&qp->sq, &cur_edge, &seg, &size, wqe, wqe_size);
 
-	id.ib_cqe = cqe;
+	id.ib_cqe = &ctx->cqe;
 	mlx5r_finish_wqe(qp, ctrl, seg, size, cur_edge, idx, id.wr_id, 0,
 			 MLX5_FENCE_MODE_INITIATOR_SMALL, MLX5_OPCODE_UMR);
 
+	/*
+	 * Push to the shadow FIFO before the doorbell, so the entry exists
+	 * before a CQE for it can arrive. A hostile NIC can post a CQE before
+	 * the doorbell, so the UMR CQ poller, on another CPU, can't rely on
+	 * the doorbell to see ctx->wqe_ctr and ctx->cqe.done with the entry.
+	 * Publish it with a release, which pairs with the poller's acquire.
+	 */
+	if (mlx5_ib_adversarial_mode) {
+		smp_store_release(&umrc->pending[umrc->pending_tail & (MAX_UMR_WR - 1)],
+				  ctx);
+		umrc->pending_tail++;
+	}
+
 	mlx5r_ring_db(qp, 1, ctrl);
 
 out:
@@ -303,7 +331,7 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	 * This will ensure that all the previous WRs got a completion before
 	 * we set the QP state to RESET.
 	 */
-	err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context->cqe, wqe,
+	err = mlx5r_umr_post_send(umrc->qp, mkey, umr_context, wqe,
 				  with_data);
 	if (err) {
 		mlx5_ib_warn(dev, "UMR recovery post send failed, err %d\n", err);
@@ -380,7 +408,7 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
 			continue;
 		}
 
-		err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context.cqe, wqe,
+		err = mlx5r_umr_post_send(umrc->qp, mkey, &umr_context, wqe,
 					  with_data);
 		mutex_unlock(&umrc->lock);
 		if (err) {
diff --git a/drivers/infiniband/hw/mlx5/umr.h b/drivers/infiniband/hw/mlx5/umr.h
index bda7123781a9..0b245eec521f 100644
--- a/drivers/infiniband/hw/mlx5/umr.h
+++ b/drivers/infiniband/hw/mlx5/umr.h
@@ -84,6 +84,12 @@ struct mlx5r_umr_context {
 	struct ib_cqe cqe;
 	enum ib_wc_status status;
 	struct completion done;
+	/*
+	 * SQ producer counter the WQE was posted at, matched against the CQE
+	 * wqe_counter in adversarial_mode. Unlike the context address, which
+	 * is reused across posts, or the ring slot, it identifies the WQE.
+	 */
+	u16 wqe_ctr;
 };
 
 struct mlx5r_umr_wqe {
-- 
2.18.1

[PATCH rdma-next 12/23] RDMA/mlx5: Verify UMR CQ polling against a hostile NIC

[Yishai Hadas]

The generic poll path trusts the CQE content: its opcode, QPN and
wqe_counter decide which request is completed and which waiter is woken.
A forged CQE can therefore complete a request the driver never posted,
or one that already completed. The driver then writes through a context
it no longer owns, which corrupts memory, as the context is usually on
the waiter's stack.

In adversarial mode, install a dedicated poll_one for the UMR CQ that
does not take the CQE's word for it. The driver keeps a shadow FIFO of
its posted requests, and a CQE is accepted only if its opcode and QPN
match the UMR QP and its full wqe_counter matches the oldest pending
request, one already passed to the doorbell. The context is then taken
from the FIFO, never from the CQE. This cannot prove that the NIC
really executed the request, as it can always report the oldest one as
completed; it ensures that the driver completes only a context it owns,
once and in order.

A CQE that fails validation proves the NIC is forging completions, so
nothing it reports on this QP can be trusted any more. Mark the UMR
path permanently failed, so neither a new post nor a QP recovery can
reopen it, stop interpreting this CQ and force the mlx5_core health
recovery. The forced variant is needed because such a NIC still looks
healthy to the fatal sensors. Recovery moves the device to the internal
error state, and the existing flush then completes the pending requests
in software and wakes their waiters, so this path does not complete
them itself.

On teardown, stop polling the CQ before destroying the UMR QP, so a CQE
for the destroyed QP, or for a reused QPN, is never interpreted.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c      | 156 +++++++++++++++++++++++++++
 drivers/infiniband/hw/mlx5/mlx5_ib.h |   5 +
 drivers/infiniband/hw/mlx5/umr.c     |  38 +++++++
 3 files changed, 199 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 11c1153747c2..1a98263ec311 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -37,6 +37,7 @@
 #include "mlx5_ib.h"
 #include "srq.h"
 #include "qp.h"
+#include "umr.h"
 
 #define UVERBS_MODULE_NAME mlx5_ib
 #include <rdma/uverbs_named_ioctl.h>
@@ -608,6 +609,161 @@ static int poll_soft_wc(struct mlx5_ib_cq *cq, int num_entries,
 	return npolled;
 }
 
+static int mlx5_ib_poll_one_stopped(struct mlx5_ib_cq *cq,
+				    struct mlx5_ib_qp **cur_qp, struct ib_wc *wc)
+{
+	return -EAGAIN;
+}
+
+/**
+ * mlx5_ib_cq_stop_polling - stop interpreting CQEs on a CQ
+ * @ibcq: the CQ
+ *
+ * Takes cq->lock, which mlx5_ib_poll_cq() holds while polling, so no poll
+ * is in flight on return and no CQE is looked at again.
+ */
+void mlx5_ib_cq_stop_polling(struct ib_cq *ibcq)
+{
+	struct mlx5_ib_cq *cq = to_mcq(ibcq);
+	unsigned long flags;
+
+	spin_lock_irqsave(&cq->lock, flags);
+	cq->poll_one = mlx5_ib_poll_one_stopped;
+	spin_unlock_irqrestore(&cq->lock, flags);
+}
+
+/*
+ * poll_one of the UMR CQ in adversarial_mode. A CQE is accepted only if it
+ * matches the oldest request in the shadow FIFO, otherwise the device is
+ * treated as compromised.
+ */
+int mlx5_ib_poll_one_umr(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+		      struct ib_wc *wc)
+{
+	struct mlx5_ib_dev *dev = to_mdev(cq->ibcq.device);
+	struct mlx5r_umr_context *expected;
+	struct umr_common *umrc = &dev->umrc;
+	struct mlx5_err_cqe err_cqe;
+	struct mlx5_cqe64 *cqe64;
+	struct mlx5_ib_wq *wq;
+	struct mlx5_ib_qp *qp;
+	struct ib_qp *ibqp;
+	u16 wqe_ctr;
+	u8 opcode;
+	u32 head;
+	u32 sop_qpn;
+	u32 qpn;
+	u8 sop;
+	int idx;
+	void *cqe;
+
+	cqe = next_cqe_sw(cq);
+	if (!cqe)
+		return -EAGAIN;
+
+	cqe64 = (cq->mcq.cqe_sz == 64) ? cqe : cqe + 64;
+	++cq->mcq.cons_index;
+	/*
+	 * Read each field once, so a NIC rewriting the CQE cannot change it
+	 * between validation and use.
+	 */
+	rmb();
+	opcode  = READ_ONCE(cqe64->op_own) >> 4;
+	sop_qpn = ntohl(READ_ONCE(cqe64->sop_drop_qpn));
+	qpn     = sop_qpn & 0xffffff;
+	sop     = sop_qpn >> 24;
+	wqe_ctr = be16_to_cpu(READ_ONCE(cqe64->wqe_counter));
+
+	/*
+	 * This poll_one is installed before the UMR QP is published, so a CQE
+	 * seen while the QP is NULL is forged. Teardown stops polling before
+	 * the QP is destroyed.
+	 */
+	ibqp = smp_load_acquire(&dev->umrc.qp);
+	if (unlikely(!ibqp)) {
+		mlx5_ib_err(dev, "CQE on the UMR CQ before its QP is published\n");
+		goto forged_cqe;
+	}
+	qp = to_mqp(ibqp);
+	wq = &qp->sq;
+	idx = wqe_ctr & (wq->wqe_cnt - 1);
+
+	if (unlikely((opcode != MLX5_CQE_REQ && opcode != MLX5_CQE_REQ_ERR) ||
+		     qpn != umrc->qpn || sop != MLX5_OPCODE_UMR)) {
+		mlx5_ib_err(dev,
+			    "Unexpected CQE on UMR CQ: opcode 0x%x sop 0x%x qpn 0x%x\n",
+			    opcode, sop, qpn);
+		goto forged_cqe;
+	}
+
+	/* Pairs with smp_store_release() in mlx5r_umr_post_send() */
+	head = umrc->pending_head;
+	expected = smp_load_acquire(&umrc->pending[head & (MLX5_UMR_MAX_WR - 1)]);
+
+	if (unlikely(!expected)) {
+		mlx5_ib_err(dev,
+			    "UMR CQE wqe_ctr 0x%x with an empty shadow FIFO, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	if (unlikely(wqe_ctr != expected->wqe_ctr)) {
+		mlx5_ib_err(dev,
+			    "UMR wqe_ctr 0x%x mismatch with expected context, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	/*
+	 * A WQE records wq->head when posted, and mlx5r_ring_db() then advances
+	 * wq->head. If they are still equal, mlx5r_ring_db() was not reached
+	 * yet, so the WQE was never handed to the NIC.
+	 */
+	if (unlikely(wq->wqe_head[idx] == READ_ONCE(wq->head))) {
+		mlx5_ib_err(dev,
+			    "UMR CQE wqe_ctr 0x%x for an uncommitted WQE, possible NIC forgery\n",
+			    wqe_ctr);
+		goto forged_cqe;
+	}
+
+	/* Pop the shadow FIFO head: validated, dispatch to its waiter */
+	WRITE_ONCE(umrc->pending[head & (MLX5_UMR_MAX_WR - 1)], NULL);
+	umrc->pending_head = head + 1;
+	wc->wr_cqe = &expected->cqe;
+
+	wc->qp = &qp->ibqp;
+	wq->tail = wq->wqe_head[idx] + 1;
+	/* Advance wq->last_poll, the internal error flush starts from it */
+	handle_atomics(qp, cqe64, wq->last_poll, idx);
+
+	if (opcode == MLX5_CQE_REQ) {
+		wc->status = IB_WC_SUCCESS;
+	} else {
+		/* Copy the CQE, so its syndrome is read once */
+		err_cqe = *(struct mlx5_err_cqe *)cqe64;
+		mlx5_handle_error_cqe(dev, &err_cqe, wc);
+		/* As in the generic path, a real failure starts the recovery */
+		if (wc->status != IB_WC_WR_FLUSH_ERR)
+			umrc->state = MLX5_UMR_STATE_RECOVER;
+	}
+
+	return 0;
+
+forged_cqe:
+	/* mlx5_ib_poll_cq() updates the CI only for a returned completion */
+	mlx5_cq_set_ci(&cq->mcq);
+	/*
+	 * The NIC forged a CQE. Fail the UMR path for good, ignore any later
+	 * CQE and force the health recovery, as the fatal sensors still read
+	 * clean. The internal error flush then completes the waiters. Return
+	 * an error, not -EAGAIN, as a CQE was consumed.
+	 */
+	umrc->state = MLX5_UMR_STATE_ERR;
+	cq->poll_one = mlx5_ib_poll_one_stopped;
+	mlx5_trigger_health_work_force(dev->mdev);
+	return -EIO;
+}
+
 int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 {
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 9cc4b2f99b69..f7c718e3d974 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -768,6 +768,8 @@ struct umr_common {
 	struct ib_pd	*pd;
 	struct ib_cq	*cq;
 	struct ib_qp	*qp;
+	/* Checked against each CQE in adversarial_mode */
+	u32		qpn;
 	/* Protects from UMR QP overflow
 	 */
 	struct semaphore	sem;
@@ -1311,6 +1313,9 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 			   struct uverbs_attr_bundle *attrs);
 int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
 int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
+int mlx5_ib_poll_one_umr(struct mlx5_ib_cq *cq, struct mlx5_ib_qp **cur_qp,
+		      struct ib_wc *wc);
+void mlx5_ib_cq_stop_polling(struct ib_cq *ibcq);
 int mlx5_ib_pre_destroy_cq(struct ib_cq *cq);
 void mlx5_ib_post_destroy_cq(struct ib_cq *cq);
 int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index aba79a19d627..b0f00d4bc93c 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -194,6 +194,23 @@ int mlx5r_umr_resource_init(struct mlx5_ib_dev *dev)
 	sema_init(&dev->umrc.sem, MAX_UMR_WR);
 	mutex_init(&dev->umrc.lock);
 	dev->umrc.state = MLX5_UMR_STATE_ACTIVE;
+
+	if (mlx5_ib_adversarial_mode) {
+		struct mlx5_ib_cq *mcq = to_mcq(cq);
+		unsigned long flags;
+
+		dev->umrc.qpn = qp->qp_num;
+		/*
+		 * UMRs can be posted as soon as the QP is published below, by
+		 * this caller or by a concurrent MR registration, so install
+		 * mlx5_ib_poll_one_umr() first. The CQ is already armed, so
+		 * swap it under cq->lock, which the poll loop holds.
+		 */
+		spin_lock_irqsave(&mcq->lock, flags);
+		mcq->poll_one = mlx5_ib_poll_one_umr;
+		spin_unlock_irqrestore(&mcq->lock, flags);
+	}
+
 	smp_store_release(&dev->umrc.qp, qp);
 
 	mutex_unlock(&dev->umrc.init_lock);
@@ -212,6 +229,14 @@ void mlx5r_umr_resource_cleanup(struct mlx5_ib_dev *dev)
 {
 	if (dev->umrc.state == MLX5_UMR_STATE_UNINIT)
 		return;
+
+	/*
+	 * Stop polling before the QP is destroyed, as its QPN may then be
+	 * reused and a forged CQE carrying it must not be interpreted.
+	 */
+	if (mlx5_ib_adversarial_mode)
+		mlx5_ib_cq_stop_polling(dev->umrc.cq);
+
 	mutex_destroy(&dev->umrc.lock);
 	/* After device init, UMR cp/qp are not unset during the lifetime. */
 	ib_destroy_qp(dev->umrc.qp);
@@ -319,6 +344,11 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	int err;
 
 	mutex_lock(&umrc->lock);
+	if (mlx5_ib_adversarial_mode && umrc->state == MLX5_UMR_STATE_ERR) {
+		/* The NIC forged a CQE, don't revive the QP */
+		mutex_unlock(&umrc->lock);
+		return -EFAULT;
+	}
 	/* Preventing any further WRs to be sent now */
 	if (umrc->state != MLX5_UMR_STATE_RECOVER) {
 		mlx5_ib_warn(dev, "UMR recovery encountered an unexpected state=%d\n",
@@ -425,6 +455,14 @@ static int mlx5r_umr_post_send_wait(struct mlx5_ib_dev *dev, u32 mkey,
 		if (umr_context.status == IB_WC_WR_FLUSH_ERR)
 			continue;
 
+		if (mlx5_ib_adversarial_mode) {
+			/* A forged CQE was detected, don't recover the QP */
+			if (umrc->state == MLX5_UMR_STATE_ERR) {
+				err = -EFAULT;
+				break;
+			}
+		}
+
 		WARN_ON_ONCE(1);
 		mlx5_ib_warn(dev,
 			"reg umr failed (%u). Trying to recover and resubmit the flushed WQEs, mkey = %u\n",
-- 
2.18.1

[PATCH rdma-next 13/23] RDMA/mlx5: Keep the UMR QP failed once a forged CQE is detected

[Yishai Hadas]

In adversarial mode MLX5_UMR_STATE_ERR is a final state: the poller sets
it when it rejects a forged CQE, after which hardware polling is stopped,
and it stops mlx5r_umr_post_send_wait() from posting again.

mlx5r_umr_recover() runs concurrently with that and moves the state with
plain stores: it enters RECOVER from whatever state the path is in and
ends with an unconditional ACTIVE. An ERR set while a recovery is in
flight is therefore overwritten, which opens the UMR path again for a
NIC that was just caught lying, with its poller already stopped.

In adversarial mode move the state with cmpxchg() in recovery, ACTIVE to
RECOVER on entry and RECOVER to ACTIVE at the end. If ERR was set in the
meantime the transition leaves it in place and recovery returns
-EFAULT. Outside adversarial mode the transitions are unchanged.

This keeps new UMRs away from the NIC until the forced health recovery
moves the device to MLX5_DEVICE_STATE_INTERNAL_ERROR, after which posts
fail with -EIO anyway.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/umr.c | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index b0f00d4bc93c..086b397b830c 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -335,6 +335,23 @@ static int mlx5r_umr_post_send(struct ib_qp *ibqp, u32 mkey,
 	return err;
 }
 
+/*
+ * Move the UMR state from @from to @to. In adversarial_mode ERR is a final
+ * state: it can be set concurrently by the CQ poller, so the move is atomic
+ * and does not replace it. Returns false if ERR was found, and true
+ * otherwise. Outside adversarial_mode this is the plain store it always was.
+ */
+static bool mlx5r_umr_move_state(struct umr_common *umrc, unsigned int from,
+				 unsigned int to)
+{
+	if (!mlx5_ib_adversarial_mode) {
+		umrc->state = to;
+		return true;
+	}
+
+	return cmpxchg(&umrc->state, from, to) != MLX5_UMR_STATE_ERR;
+}
+
 static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 			     struct mlx5r_umr_context *umr_context,
 			     struct mlx5r_umr_wqe *wqe, bool with_data)
@@ -353,7 +370,11 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 	if (umrc->state != MLX5_UMR_STATE_RECOVER) {
 		mlx5_ib_warn(dev, "UMR recovery encountered an unexpected state=%d\n",
 			     umrc->state);
-		umrc->state = MLX5_UMR_STATE_RECOVER;
+		if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_ACTIVE,
+					  MLX5_UMR_STATE_RECOVER)) {
+			mutex_unlock(&umrc->lock);
+			return -EFAULT;
+		}
 	}
 	mutex_unlock(&umrc->lock);
 
@@ -387,7 +408,9 @@ static int mlx5r_umr_recover(struct mlx5_ib_dev *dev, u32 mkey,
 		goto err;
 	}
 
-	umrc->state = MLX5_UMR_STATE_ACTIVE;
+	if (!mlx5r_umr_move_state(umrc, MLX5_UMR_STATE_RECOVER,
+				  MLX5_UMR_STATE_ACTIVE))
+		return -EFAULT;
 	return 0;
 
 err:
-- 
2.18.1

[PATCH rdma-next 14/23] RDMA/mlx5: Block generic CQ poll path in adversarial mode

[Yishai Hadas]

mlx5_poll_one() trusts the CQE content, so it must not run on a CQE
from an untrusted NIC. In adversarial mode no CQ needs it: the UMR CQ
has its own poll_one that validates the CQE, kernel ULPs don't bind to
the device, and user CQs are polled by userspace. Refuse it there, so
an unexpected caller, such as the uverbs POLL_CQ command, finds the CQ
empty and no CQE is interpreted.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 1a98263ec311..5cc86c3f93f3 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -464,6 +464,15 @@ static int mlx5_poll_one(struct mlx5_ib_cq *cq,
 	void *cqe;
 	int idx;
 
+	/*
+	 * This path trusts the CQE, so refuse it in adversarial_mode. Only an
+	 * unexpected caller gets here, e.g. the uverbs POLL_CQ command or an
+	 * event on the UMR CQ before its own poll_one is installed. Nothing is
+	 * consumed, so such a CQE is left for that poll_one.
+	 */
+	if (unlikely(mlx5_ib_adversarial_mode))
+		return -EOPNOTSUPP;
+
 repoll:
 	cqe = next_cqe_sw(cq);
 	if (!cqe)
-- 
2.18.1

[PATCH rdma-next 15/23] RDMA/core: Set the CQ comp_handler after its poll context is set up

[Yishai Hadas]

__ib_alloc_cq() sets comp_handler before it initializes the poll context
the handler schedules, and for the workqueue contexts it arms the CQ
before setting comp_wq. The CQ is reachable from the completion
interrupt once ops.create_cq() returns, so a completion event raised in
that window would schedule an uninitialized irq_poll or work, or queue
work on a NULL workqueue. A conforming device raises no event for a new,
empty CQ, but nothing in the core relies on that.

Initialize the poll context first, then set comp_handler, and arm the CQ
only after that.

Fixes: 14d3a3b2498e ("IB: add a proper completion queue abstraction")
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/cq.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c
index 12304c9a9403..42e4d272f5ad 100644
--- a/drivers/infiniband/core/cq.c
+++ b/drivers/infiniband/core/cq.c
@@ -217,6 +217,7 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 		.cqe		= nr_cqe,
 		.comp_vector	= comp_vector,
 	};
+	ib_comp_handler comp_handler;
 	struct ib_cq *cq;
 	int ret = -ENOMEM;
 
@@ -248,19 +249,16 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 
 	switch (cq->poll_ctx) {
 	case IB_POLL_DIRECT:
-		cq->comp_handler = ib_cq_completion_direct;
+		comp_handler = ib_cq_completion_direct;
 		break;
 	case IB_POLL_SOFTIRQ:
-		cq->comp_handler = ib_cq_completion_softirq;
-
+		comp_handler = ib_cq_completion_softirq;
 		irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler);
-		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 		break;
 	case IB_POLL_WORKQUEUE:
 	case IB_POLL_UNBOUND_WORKQUEUE:
-		cq->comp_handler = ib_cq_completion_workqueue;
+		comp_handler = ib_cq_completion_workqueue;
 		INIT_WORK(&cq->work, ib_cq_poll_work);
-		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 		cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ?
 				ib_comp_wq : ib_comp_unbound_wq;
 		break;
@@ -269,6 +267,15 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private, int nr_cqe,
 		goto out_destroy_cq;
 	}
 
+	/*
+	 * The CQ is reachable from the completion interrupt once created. Set
+	 * the handler only after the poll context it schedules is set up, and
+	 * arm the CQ only after that.
+	 */
+	cq->comp_handler = comp_handler;
+	if (cq->poll_ctx != IB_POLL_DIRECT)
+		ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
+
 	rdma_restrack_add(&cq->res);
 	trace_cq_alloc(cq, nr_cqe, comp_vector, poll_ctx);
 	return cq;
-- 
2.18.1

[PATCH rdma-next 16/23] RDMA/mlx5: Ignore completion events for a CQ without a handler

[Yishai Hadas]

mlx5_ib_cq_comp() calls the CQ comp_handler unconditionally, but
devr->c0, created by mlx5_ib_dev_res_cq_init() on the first SRQ create,
has none, as it is never armed.

A trusted NIC raises no completion event for a CQ that is not armed, but
a hostile NIC in adversarial mode can do so with any CQ number it
returned, which ends in a NULL call in interrupt context.

Skip the call when no handler is set.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 5cc86c3f93f3..f955eb762051 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -46,7 +46,12 @@ static void mlx5_ib_cq_comp(struct mlx5_core_cq *cq, struct mlx5_eqe *eqe)
 {
 	struct ib_cq *ibcq = &to_mibcq(cq)->ibcq;
 
-	ibcq->comp_handler(ibcq, ibcq->cq_context);
+	/*
+	 * A CQ created without a handler (devr->c0) is never armed, but a
+	 * hostile NIC can still raise a completion event for it.
+	 */
+	if (ibcq->comp_handler)
+		ibcq->comp_handler(ibcq, ibcq->cq_context);
 }
 
 static void mlx5_ib_cq_event(struct mlx5_core_cq *mcq, enum mlx5_event type)
-- 
2.18.1

[PATCH rdma-next 17/23] RDMA/mlx5: Validate EQE port and GSI port before indexing devr.ports[]

[Yishai Hadas]

devr.ports[] has 2 entries, but it is indexed by port numbers taken
from the NIC, which adversarial mode does not trust.

handle_port_change() uses the EQE port before any validation, so port 0
wraps the index to UINT_MAX and a port above 2 overflows the array.
mlx5_ib_create_gsi() gets a port already checked against phys_port_cnt,
but that comes from the num_ports and num_vhca_ports FW caps, which may
report more than 2 ports.

Validate the port and bound it by ARRAY_SIZE() in both paths.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/gsi.c  | 4 ++++
 drivers/infiniband/hw/mlx5/main.c | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/gsi.c b/drivers/infiniband/hw/mlx5/gsi.c
index b2e2a219639d..7ce390a2820f 100644
--- a/drivers/infiniband/hw/mlx5/gsi.c
+++ b/drivers/infiniband/hw/mlx5/gsi.c
@@ -95,6 +95,10 @@ int mlx5_ib_create_gsi(struct ib_pd *pd, struct mlx5_ib_qp *mqp,
 	int num_qps = 0;
 	int ret;
 
+	if (!rdma_is_port_valid(&dev->ib_dev, port_num) ||
+	    port_num > ARRAY_SIZE(dev->devr.ports))
+		return -EINVAL;
+
 	if (mlx5_ib_deth_sqpn_cap(dev)) {
 		if (MLX5_CAP_GEN(dev->mdev,
 				 port_type) == MLX5_CAP_PORT_TYPE_IB)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index f4f2ad80e76c..c29102260989 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -3063,6 +3063,10 @@ static int handle_port_change(struct mlx5_ib_dev *ibdev, struct mlx5_eqe *eqe,
 {
 	u32 port = (eqe->data.port.port >> 4) & 0xf;
 
+	if (!rdma_is_port_valid(&ibdev->ib_dev, port) ||
+	    port > ARRAY_SIZE(ibdev->devr.ports))
+		return -EINVAL;
+
 	ibev->element.port_num = port;
 
 	switch (eqe->sub_type) {
-- 
2.18.1

[PATCH rdma-next 18/23] RDMA/mlx5: Validate firmware RQ/SQ state before sqrq_trans[] lookup

[Yishai Hadas]

sqrq_state_to_qp_state() indexes the 5x5 sqrq_trans[] table with the RQ
and SQ states returned by QUERY_RQ and QUERY_SQ. The state fields are 4
bits wide, so an untrusted NIC can return a state above 4 and the lookup
reads past the table.

Reject out of range states with -EIO before the lookup.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 19cbf6b2b979..05e8e75d734f 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -5081,6 +5081,9 @@ static int sqrq_state_to_qp_state(u8 sq_state, u8 rq_state,
 		},
 	};
 
+	if (sq_state >= MLX5_SQ_NUM_STATE || rq_state >= MLX5_RQ_NUM_STATE)
+		return -EIO;
+
 	*qp_state = sqrq_trans[rq_state][sq_state];
 
 	if (*qp_state == MLX5_QP_STATE_BAD) {
-- 
2.18.1

[PATCH rdma-next 19/23] RDMA/mlx5: Clamp log_max_cq_sz shift in CQ size checks

[Yishai Hadas]

The CQ create and resize paths check the requested size against
1 << log_max_cq_sz. The cap is 8 bits wide, so an untrusted NIC can
report 31 or more, which makes the shift undefined. A large limit also
lets the rounded up CQ size exceed INT_MAX, and it is then truncated
when passed on as an int.

Take the limit from MLX5_CAP_GEN_LOGSZ_U64(), round up in 64 bits, and
reject sizes above INT_MAX.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/cq.c | 39 +++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index f955eb762051..25344dfba1c1 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -1117,6 +1117,15 @@ static void notify_soft_wc_handler(struct work_struct *work)
 	cq->ibcq.comp_handler(&cq->ibcq, cq->ibcq.cq_context);
 }
 
+/*
+ * Next power of 2 above entries. It can be 2^32, which doesn't fit the
+ * unsigned long of roundup_pow_of_two() on 32-bit archs.
+ */
+static u64 cq_roundup_entries(u64 entries)
+{
+	return 1ULL << fls64(entries);
+}
+
 int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 			   const struct ib_cq_init_attr *attr,
 			   struct uverbs_attr_bundle *attrs)
@@ -1124,7 +1133,7 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 	struct ib_udata *udata = &attrs->driver_udata;
 	struct ib_device *ibdev = ibcq->device;
 	struct mlx5_ib_create_cq_resp uresp = {};
-	int entries = attr->cqe;
+	u64 entries = attr->cqe;
 	int vector = attr->comp_vector;
 	struct mlx5_ib_dev *dev = to_mdev(ibdev);
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
@@ -1136,15 +1145,17 @@ int mlx5_ib_create_user_cq(struct ib_cq *ibcq,
 	int cqe_size;
 	int eqn;
 	int err;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
-	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (attr->cqe > max_cq_entries)
 		return -EINVAL;
 
 	if (check_cq_create_flags(attr->flags))
 		return -EOPNOTSUPP;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	entries = cq_roundup_entries(entries);
+	/* cq->ibcq.cqe and create_cq_user() take an int */
+	if (entries > max_cq_entries || entries > INT_MAX)
 		return -EINVAL;
 
 	cq->ibcq.cqe = entries - 1;
@@ -1214,7 +1225,7 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 		      struct uverbs_attr_bundle *attrs)
 {
 	struct ib_device *ibdev = ibcq->device;
-	int entries = attr->cqe;
+	u64 entries = attr->cqe;
 	int vector = attr->comp_vector;
 	struct mlx5_ib_dev *dev = to_mdev(ibdev);
 	struct mlx5_ib_cq *cq = to_mcq(ibcq);
@@ -1226,12 +1237,14 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 	int cqe_size;
 	int eqn;
 	int err;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
-	if (attr->cqe > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (attr->cqe > max_cq_entries)
 		return -EINVAL;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	entries = cq_roundup_entries(entries);
+	/* cq->ibcq.cqe and create_cq_kernel() take an int */
+	if (entries > max_cq_entries || entries > INT_MAX)
 		return -EINVAL;
 
 	cq->ibcq.cqe = entries - 1;
@@ -1532,18 +1545,22 @@ int mlx5_ib_resize_cq(struct ib_cq *ibcq, unsigned int entries,
 	int inlen;
 	int cqe_size;
 	unsigned long flags;
+	u64 entries64 = entries;
+	u64 max_cq_entries = MLX5_CAP_GEN_LOGSZ_U64(dev->mdev, log_max_cq_sz);
 
 	if (!MLX5_CAP_GEN(dev->mdev, cq_resize)) {
 		pr_info("Firmware does not support resize CQ\n");
 		return -ENOSYS;
 	}
 
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)))
+	if (entries64 > max_cq_entries)
 		return -EINVAL;
 
-	entries = roundup_pow_of_two(entries + 1);
-	if (entries > (1 << MLX5_CAP_GEN(dev->mdev, log_max_cq_sz)) + 1)
+	entries64 = cq_roundup_entries(entries64);
+	/* resize_user() and resize_kernel() take an int */
+	if (entries64 > max_cq_entries + 1 || entries64 > INT_MAX)
 		return -EINVAL;
+	entries = entries64;
 
 	if (entries == ibcq->cqe + 1)
 		return 0;
-- 
2.18.1

[PATCH rdma-next 20/23] RDMA/mlx5: Clamp log_max_* shifts in QP and WQ limit checks

[Yishai Hadas]

Several QP and WQ size checks in qp.c compute their limit as
1 << cap, where cap is an 8-bit log_max_* FW capability. An untrusted
NIC can report 31 or more, which makes the shift undefined.

Use MLX5_CAP_GEN_LOGSZ_U32() for these limits, as done for
log_max_cq_sz.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/qp.c | 36 +++++++++++++++------------------
 1 file changed, 16 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 05e8e75d734f..d784a35675d5 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -447,7 +447,7 @@ static int set_rq_size(struct mlx5_ib_dev *dev, struct ib_qp_cap *cap,
 	int wq_size;
 
 	/* Sanity check RQ size before proceeding */
-	if (cap->max_recv_wr > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz)))
+	if (cap->max_recv_wr > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	if (!has_rq) {
@@ -621,11 +621,11 @@ static int calc_sq_size(struct mlx5_ib_dev *dev, struct ib_qp_init_attr *attr,
 
 	wq_size = roundup_pow_of_two(attr->cap.max_send_wr * wqe_size);
 	qp->sq.wqe_cnt = wq_size / MLX5_SEND_WQE_BB;
-	if (qp->sq.wqe_cnt > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz))) {
-		mlx5_ib_dbg(dev, "send queue size (%d * %d / %d -> %d) exceeds limits(%d)\n",
+	if (qp->sq.wqe_cnt > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz)) {
+		mlx5_ib_dbg(dev, "send queue size (%d * %d / %d -> %d) exceeds limits(%u)\n",
 			    attr->cap.max_send_wr, wqe_size, MLX5_SEND_WQE_BB,
 			    qp->sq.wqe_cnt,
-			    1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz));
+			    MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz));
 		return -ENOMEM;
 	}
 	qp->sq.wqe_shift = ilog2(MLX5_SEND_WQE_BB);
@@ -663,10 +663,10 @@ static int set_user_buf_size(struct mlx5_ib_dev *dev,
 
 	qp->sq.wqe_cnt = ucmd->sq_wqe_count;
 
-	if (qp->sq.wqe_cnt > (1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz))) {
-		mlx5_ib_warn(dev, "wqe_cnt %d, max_wqes %d\n",
+	if (qp->sq.wqe_cnt > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz)) {
+		mlx5_ib_warn(dev, "wqe_cnt %d, max_wqes %u\n",
 			     qp->sq.wqe_cnt,
-			     1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz));
+			     MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz));
 		return -EINVAL;
 	}
 
@@ -2155,7 +2155,7 @@ static int create_dci(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	    ucmd->rq_wqe_count != qp->rq.wqe_cnt)
 		return -EINVAL;
 
-	if (ucmd->sq_wqe_count > (1 << MLX5_CAP_GEN(mdev, log_max_qp_sz)))
+	if (ucmd->sq_wqe_count > MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	ts_format = get_qp_ts_format(dev, to_mcq(init_attr->send_cq),
@@ -2322,7 +2322,7 @@ static int create_user_qp(struct mlx5_ib_dev *dev, struct ib_pd *pd,
 	    ucmd->rq_wqe_count != qp->rq.wqe_cnt)
 		return -EINVAL;
 
-	if (ucmd->sq_wqe_count > (1 << MLX5_CAP_GEN(mdev, log_max_qp_sz)))
+	if (ucmd->sq_wqe_count > MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz))
 		return -EINVAL;
 
 	if (init_attr->qp_type != IB_QPT_RAW_PACKET) {
@@ -4809,19 +4809,15 @@ static bool mlx5_ib_modify_qp_allowed(struct mlx5_ib_dev *dev,
 static int validate_rd_atomic(struct mlx5_ib_dev *dev, struct ib_qp_attr *attr,
 			      int attr_mask, enum ib_qp_type qp_type)
 {
-	int log_max_ra_res;
-	int log_max_ra_req;
+	u32 log_max_ra_res;
+	u32 log_max_ra_req;
 
 	if (qp_type == MLX5_IB_QPT_DCI) {
-		log_max_ra_res = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_res_dc);
-		log_max_ra_req = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_req_dc);
+		log_max_ra_res = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_res_dc);
+		log_max_ra_req = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_req_dc);
 	} else {
-		log_max_ra_res = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_res_qp);
-		log_max_ra_req = 1 << MLX5_CAP_GEN(dev->mdev,
-						   log_max_ra_req_qp);
+		log_max_ra_res = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_res_qp);
+		log_max_ra_req = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_ra_req_qp);
 	}
 
 	if (attr_mask & IB_QP_MAX_QP_RD_ATOMIC &&
@@ -5511,7 +5507,7 @@ static int set_user_rq_size(struct mlx5_ib_dev *dev,
 			    struct mlx5_ib_rwq *rwq)
 {
 	/* Sanity check RQ size before proceeding */
-	if (wq_init_attr->max_wr > (1 << MLX5_CAP_GEN(dev->mdev, log_max_wq_sz)))
+	if (wq_init_attr->max_wr > MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_wq_sz))
 		return -EINVAL;
 
 	if (!ucmd->rq_wqe_count)
-- 
2.18.1

[PATCH rdma-next 21/23] RDMA/mlx5: Clamp log_max_* shifts in query_device and ucontext paths

[Yishai Hadas]

The device, port and ucontext attributes and the SRQ size limit are
computed as 1 << cap from 8-bit log_max_* FW capabilities. An untrusted
NIC can report 31 or more, which makes the shift undefined. The same
applies to 1ull << log_pg_sz with a value of 64 or more.

Use MLX5_CAP_GEN_LOGSZ_U32() and MLX5_CAP_GEN_LOGSZ_U64() for these
values. As the limits can now reach 2^31, compute max_qp_wr in 64 bits
and saturate the max_res_rd_atom and max_total_mcast_qp_attach products
at U32_MAX.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mad.c  |  2 +-
 drivers/infiniband/hw/mlx5/main.c | 63 ++++++++++++++++---------------
 drivers/infiniband/hw/mlx5/srq.c  |  2 +-
 3 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mad.c b/drivers/infiniband/hw/mlx5/mad.c
index 8925331855ff..cf9f6a4d5ccf 100644
--- a/drivers/infiniband/hw/mlx5/mad.c
+++ b/drivers/infiniband/hw/mlx5/mad.c
@@ -629,7 +629,7 @@ int mlx5_query_mad_ifc_port(struct ib_device *ibdev, u32 port,
 	props->phys_state	= out_mad->data[33] >> 4;
 	props->port_cap_flags	= be32_to_cpup((__be32 *)(out_mad->data + 20));
 	props->gid_tbl_len	= out_mad->data[50];
-	props->max_msg_sz	= 1 << MLX5_CAP_GEN(mdev, log_max_msg);
+	props->max_msg_sz	= MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_msg);
 	props->pkey_tbl_len	= dev->pkey_table_len;
 	props->bad_pkey_cntr	= be16_to_cpup((__be16 *)(out_mad->data + 46));
 	props->qkey_viol_cntr	= be16_to_cpup((__be16 *)(out_mad->data + 48));
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index c29102260989..ace4cdc95c9d 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -599,7 +599,7 @@ static int mlx5_query_port_roce(struct ib_device *device, u32 port_num,
 		props->qkey_viol_cntr = qkey_viol_cntr;
 	}
 	props->max_mtu          = IB_MTU_4096;
-	props->max_msg_sz       = 1 << MLX5_CAP_GEN(dev->mdev, log_max_msg);
+	props->max_msg_sz       = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_msg);
 	props->pkey_tbl_len     = 1;
 	props->state            = IB_PORT_DOWN;
 	props->phys_state       = IB_PORT_PHYS_STATE_DISABLED;
@@ -930,7 +930,7 @@ static u32 mlx5_ib_calc_max_sq_overhead(void)
 static u32 mlx5_ib_calc_max_qp_wr(struct mlx5_ib_dev *dev)
 {
 	struct mlx5_core_dev *mdev = dev->mdev;
-	u32 max_wqe_bb_units = 1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
+	u64 max_wqe_bb_units = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz);
 	u32 max_wqe_size;
 	/* max QP overhead + 1 SGE, no inline, no special features */
 	max_wqe_size = mlx5_ib_calc_max_sq_overhead() +
@@ -940,7 +940,8 @@ static u32 mlx5_ib_calc_max_qp_wr(struct mlx5_ib_dev *dev)
 
 	max_wqe_size = ALIGN(max_wqe_size, MLX5_SEND_WQE_BB);
 
-	return (max_wqe_bb_units * MLX5_SEND_WQE_BB) / max_wqe_size;
+	/* Up to 2^31 units, so the product needs 64 bits */
+	return div_u64(max_wqe_bb_units * MLX5_SEND_WQE_BB, max_wqe_size);
 }
 
 static int mlx5_ib_query_device(struct ib_device *ibdev,
@@ -954,7 +955,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	int max_sq_desc;
 	int max_rq_sg;
 	int max_sq_sg;
-	u64 min_page_size = 1ull << MLX5_CAP_GEN(mdev, log_pg_sz);
+	u64 min_page_size = MLX5_CAP_GEN_LOGSZ_U64(mdev, log_pg_sz);
 	bool raw_support = !mlx5_core_mp_enabled(mdev);
 	struct mlx5_ib_query_device_resp resp = {};
 	size_t resp_len;
@@ -1000,7 +1001,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	if (MLX5_CAP_GEN(mdev, imaicl)) {
 		props->device_cap_flags |= IB_DEVICE_MEM_WINDOW |
 					   IB_DEVICE_MEM_WINDOW_TYPE_2B;
-		props->max_mw = 1 << MLX5_CAP_GEN(mdev, log_max_mkey);
+		props->max_mw = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mkey);
 		/* We support 'Gappy' memory registration too */
 		props->kernel_cap_flags |= IBK_SG_GAPS_REG;
 	}
@@ -1033,7 +1034,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 		if (offsetofend(typeof(resp), tso_caps) <= uhw_outlen) {
 			max_tso = MLX5_CAP_ETH(mdev, max_lso_cap);
 			if (max_tso) {
-				resp.tso_caps.max_tso = 1 << max_tso;
+				resp.tso_caps.max_tso = BIT(max_tso);
 				resp.tso_caps.supported_qpts |=
 					1 << IB_QPT_RAW_PACKET;
 				resp.response_length += sizeof(resp.tso_caps);
@@ -1100,7 +1101,7 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 
 	props->max_mr_size	   = ~0ull;
 	props->page_size_cap	   = ~(min_page_size - 1);
-	props->max_qp		   = 1 << MLX5_CAP_GEN(mdev, log_max_qp);
+	props->max_qp		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp);
 	props->max_qp_wr = mlx5_ib_calc_max_qp_wr(dev);
 	max_rq_sg =  MLX5_CAP_GEN(mdev, max_wqe_sz_rq) /
 		     sizeof(struct mlx5_wqe_data_seg);
@@ -1111,29 +1112,31 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	props->max_send_sge = max_sq_sg;
 	props->max_recv_sge = max_rq_sg;
 	props->max_sge_rd	   = MLX5_MAX_SGE_RD;
-	props->max_cq		   = 1 << MLX5_CAP_GEN(mdev, log_max_cq);
-	props->max_cqe = (1 << MLX5_CAP_GEN(mdev, log_max_cq_sz)) - 1;
-	props->max_mr		   = 1 << MLX5_CAP_GEN(mdev, log_max_mkey);
-	props->max_pd		   = 1 << MLX5_CAP_GEN(mdev, log_max_pd);
-	props->max_qp_rd_atom	   = 1 << MLX5_CAP_GEN(mdev, log_max_ra_req_qp);
-	props->max_qp_init_rd_atom = 1 << MLX5_CAP_GEN(mdev, log_max_ra_res_qp);
-	props->max_srq		   = 1 << MLX5_CAP_GEN(mdev, log_max_srq);
-	props->max_srq_wr = (1 << MLX5_CAP_GEN(mdev, log_max_srq_sz)) - 1;
+	props->max_cq		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_cq);
+	props->max_cqe = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_cq_sz) - 1;
+	props->max_mr		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mkey);
+	props->max_pd		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_pd);
+	props->max_qp_rd_atom	   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_ra_req_qp);
+	props->max_qp_init_rd_atom = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_ra_res_qp);
+	props->max_srq		   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_srq);
+	props->max_srq_wr = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_srq_sz) - 1;
 	props->local_ca_ack_delay  = MLX5_CAP_GEN(mdev, local_ca_ack_delay);
-	props->max_res_rd_atom	   = props->max_qp_rd_atom * props->max_qp;
+	props->max_res_rd_atom	   = min_t(u64, (u64)props->max_qp_rd_atom * props->max_qp,
+					   U32_MAX);
 	props->max_srq_sge	   = max_rq_sg - 1;
 	props->max_fast_reg_page_list_len =
-		1 << MLX5_CAP_GEN(mdev, log_max_klm_list_size);
+		MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_klm_list_size);
 	props->max_pi_fast_reg_page_list_len =
 		props->max_fast_reg_page_list_len / 2;
 	props->max_sgl_rd =
 		MLX5_CAP_GEN(mdev, max_sgl_for_optimized_performance);
 	get_atomic_caps_qp(dev, props);
 	props->masked_atomic_cap   = IB_ATOMIC_NONE;
-	props->max_mcast_grp	   = 1 << MLX5_CAP_GEN(mdev, log_max_mcg);
+	props->max_mcast_grp	   = MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_mcg);
 	props->max_mcast_qp_attach = MLX5_CAP_GEN(mdev, max_qp_mcg);
-	props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
-					   props->max_mcast_grp;
+	props->max_total_mcast_qp_attach =
+		min_t(u64, (u64)props->max_mcast_qp_attach * props->max_mcast_grp,
+		      U32_MAX);
 	props->max_ah = INT_MAX;
 	props->hca_core_clock = MLX5_CAP_GEN(mdev, device_frequency_khz);
 	props->timestamp_mask = 0x7FFFFFFFFFFFFFFFULL;
@@ -1167,19 +1170,19 @@ static int mlx5_ib_query_device(struct ib_device *ibdev,
 	if (mlx5_ib_port_link_layer(ibdev, 1) ==
 	    IB_LINK_LAYER_ETHERNET && raw_support) {
 		props->rss_caps.max_rwq_indirection_tables =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rqt);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rqt);
 		props->rss_caps.max_rwq_indirection_table_size =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rqt_size);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rqt_size);
 		props->rss_caps.supported_qpts = 1 << IB_QPT_RAW_PACKET;
 		props->max_wq_type_rq =
-			1 << MLX5_CAP_GEN(dev->mdev, log_max_rq);
+			MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_rq);
 	}
 
 	if (MLX5_CAP_GEN(mdev, tag_matching)) {
 		props->tm_caps.max_num_tags =
-			(1 << MLX5_CAP_GEN(mdev, log_tag_matching_list_sz)) - 1;
+			MLX5_CAP_GEN_LOGSZ_U32(mdev, log_tag_matching_list_sz) - 1;
 		props->tm_caps.max_ops =
-			1 << MLX5_CAP_GEN(mdev, log_max_qp_sz);
+			MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_qp_sz);
 		props->tm_caps.max_sge = MLX5_TM_MAX_SGE;
 	}
 
@@ -1503,7 +1506,7 @@ static int mlx5_query_hca_port(struct ib_device *ibdev, u32 port,
 		props->port_cap_flags &= ~IB_PORT_CM_SUP;
 
 	props->gid_tbl_len	= mlx5_get_gid_table_len(MLX5_CAP_GEN(mdev, gid_table_size));
-	props->max_msg_sz	= 1 << MLX5_CAP_GEN(mdev, log_max_msg);
+	props->max_msg_sz	= MLX5_CAP_GEN_LOGSZ_U32(mdev, log_max_msg);
 	props->pkey_tbl_len	= mlx5_to_sw_pkey_sz(MLX5_CAP_GEN(mdev, pkey_table_size));
 	props->bad_pkey_cntr	= rep->pkey_violation_counter;
 	props->qkey_viol_cntr	= rep->qkey_violation_counter;
@@ -2127,16 +2130,16 @@ static int set_ucontext_resp(struct ib_ucontext *uctx,
 			MLX5_IB_ALLOC_UCONTEXT_RESP_MASK_DUMP_FILL_MKEY;
 	}
 
-	resp->qp_tab_size = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp);
+	resp->qp_tab_size = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp);
 	if (mlx5_wc_support_get(dev->mdev))
 		resp->bf_reg_size = 1 << MLX5_CAP_GEN(dev->mdev,
 						      log_bf_reg_size);
 	resp->cache_line_size = cache_line_size();
 	resp->max_sq_desc_sz = MLX5_CAP_GEN(dev->mdev, max_wqe_sz_sq);
 	resp->max_rq_desc_sz = MLX5_CAP_GEN(dev->mdev, max_wqe_sz_rq);
-	resp->max_send_wqebb = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
-	resp->max_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_qp_sz);
-	resp->max_srq_recv_wr = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
+	resp->max_send_wqebb = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz);
+	resp->max_recv_wr = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_qp_sz);
+	resp->max_srq_recv_wr = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_srq_sz);
 	resp->cqe_version = context->cqe_version;
 	resp->log_uar_size = MLX5_CAP_GEN(dev->mdev, uar_4k) ?
 				MLX5_ADAPTER_PAGE_SHIFT : PAGE_SHIFT;
diff --git a/drivers/infiniband/hw/mlx5/srq.c b/drivers/infiniband/hw/mlx5/srq.c
index 6a4a937d97ce..3fbb0eaf1255 100644
--- a/drivers/infiniband/hw/mlx5/srq.c
+++ b/drivers/infiniband/hw/mlx5/srq.c
@@ -198,7 +198,7 @@ int mlx5_ib_create_srq(struct ib_srq *ib_srq,
 	size_t buf_size;
 	int err;
 	struct mlx5_srq_attr in = {};
-	__u32 max_srq_wqes = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
+	__u32 max_srq_wqes = MLX5_CAP_GEN_LOGSZ_U32(dev->mdev, log_max_srq_sz);
 	__u32 max_sge_sz =  MLX5_CAP_GEN(dev->mdev, max_wqe_sz_rq) /
 			    sizeof(struct mlx5_wqe_data_seg);
 
-- 
2.18.1

[PATCH rdma-next 22/23] RDMA/core: Report and enforce adversarial mode via GET_CONTEXT

[Yishai Hadas]

Userspace must know when a device is in adversarial mode, and an
unaware rdma-core must not get a context on such a device, as it
doesn't harden itself against the NIC.

Report the mode with IB_UVERBS_CORE_SUPPORT_ADVERSARIAL in the
core_support of GET_CONTEXT and QUERY_CONTEXT. Add an optional
UVERBS_ATTR_GET_CONTEXT_FLAGS input, where userspace declares
UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED. In adversarial
mode, GET_CONTEXT without that flag fails with -EPERM, and so does the
legacy write() GET_CONTEXT, which has no way to pass it.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/core/uverbs_cmd.c             |  5 +++++
 .../infiniband/core/uverbs_std_types_device.c    | 16 ++++++++++++++++
 include/uapi/rdma/ib_user_ioctl_cmds.h           |  5 +++++
 include/uapi/rdma/ib_user_ioctl_verbs.h          |  1 +
 4 files changed, 27 insertions(+)

diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index aca7c6ab55cd..b78207a81091 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -292,6 +292,11 @@ static int ib_uverbs_get_context(struct uverbs_attr_bundle *attrs)
 		goto err_ucontext;
 	}
 
+	if (ib_dev->adversarial_mode) {
+		ret = -EPERM;
+		goto err_uobj;
+	}
+
 	resp = (struct ib_uverbs_get_context_resp){
 		.num_comp_vectors = attrs->ufile->device->num_comp_vectors,
 		.async_fd = uobj->id,
diff --git a/drivers/infiniband/core/uverbs_std_types_device.c b/drivers/infiniband/core/uverbs_std_types_device.c
index db65032e80b4..72c67f09533e 100644
--- a/drivers/infiniband/core/uverbs_std_types_device.c
+++ b/drivers/infiniband/core/uverbs_std_types_device.c
@@ -248,6 +248,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 	u32 num_comp = attrs->ufile->device->num_comp_vectors;
 	u64 core_support = IB_UVERBS_CORE_SUPPORT_OPTIONAL_MR_ACCESS;
 	struct ib_device *ib_dev;
+	u32 ctx_flags;
 	int ret;
 
 	ib_dev = srcu_dereference(attrs->ufile->device->ib_dev,
@@ -255,6 +256,15 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 	if (!ib_dev)
 		return -EIO;
 
+	ret = uverbs_get_flags32(&ctx_flags, attrs, UVERBS_ATTR_GET_CONTEXT_FLAGS,
+				 UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED);
+	if (ret)
+		return ret;
+
+	if (ib_dev->adversarial_mode &&
+	    !(ctx_flags & UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED))
+		return -EPERM;
+
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
 			     &num_comp, sizeof(num_comp));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -262,6 +272,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_GET_CONTEXT)(
 
 	if (ib_dev->ops.uverbs_robust_udata)
 		core_support |= IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA;
+	if (ib_dev->adversarial_mode)
+		core_support |= IB_UVERBS_CORE_SUPPORT_ADVERSARIAL;
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
 			     &core_support, sizeof(core_support));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -303,6 +315,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QUERY_CONTEXT)(
 	if (IS_UVERBS_COPY_ERR(ret))
 		return ret;
 
+	if (ib_dev->adversarial_mode)
+		core_support |= IB_UVERBS_CORE_SUPPORT_ADVERSARIAL;
 	ret = uverbs_copy_to(attrs, UVERBS_ATTR_QUERY_CONTEXT_CORE_SUPPORT,
 			     &core_support, sizeof(core_support));
 	if (IS_UVERBS_COPY_ERR(ret))
@@ -519,6 +533,8 @@ DECLARE_UVERBS_NAMED_METHOD(
 			   UVERBS_ATTR_MIN_SIZE(sizeof(int)),
 			   UA_OPTIONAL,
 			   UA_ALLOC_AND_COPY),
+	UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_GET_CONTEXT_FLAGS, u32,
+			     UA_OPTIONAL),
 	UVERBS_ATTR_UHW());
 
 DECLARE_UVERBS_NAMED_METHOD(
diff --git a/include/uapi/rdma/ib_user_ioctl_cmds.h b/include/uapi/rdma/ib_user_ioctl_cmds.h
index 816b3107464f..3e482a4338f9 100644
--- a/include/uapi/rdma/ib_user_ioctl_cmds.h
+++ b/include/uapi/rdma/ib_user_ioctl_cmds.h
@@ -101,10 +101,15 @@ enum uverbs_attrs_query_comp_cntr_caps_attr_ids {
 	UVERBS_ATTR_QUERY_COMP_CNTR_CAPS_SUPPORTED_QP_ATTACH_OPS,
 };
 
+enum uverbs_attrs_get_context_flags {
+	UVERBS_ATTR_GET_CONTEXT_FLAG_ADVERSARIAL_SUPPORTED = 1 << 0,
+};
+
 enum uverbs_attrs_get_context_attr_ids {
 	UVERBS_ATTR_GET_CONTEXT_NUM_COMP_VECTORS,
 	UVERBS_ATTR_GET_CONTEXT_CORE_SUPPORT,
 	UVERBS_ATTR_GET_CONTEXT_FD_ARR,
+	UVERBS_ATTR_GET_CONTEXT_FLAGS,
 };
 
 enum uverbs_attrs_query_context_attr_ids {
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f..5861f34f8343 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -47,6 +47,7 @@
 enum ib_uverbs_core_support {
 	IB_UVERBS_CORE_SUPPORT_OPTIONAL_MR_ACCESS = 1 << 0,
 	IB_UVERBS_CORE_SUPPORT_ROBUST_UDATA = 1 << 1,
+	IB_UVERBS_CORE_SUPPORT_ADVERSARIAL = 1 << 2,
 };
 
 enum ib_uverbs_access_flags {
-- 
2.18.1

[PATCH rdma-next 23/23] RDMA/mlx5: Advertise adversarial mode via ib_device::adversarial_mode

[Yishai Hadas]

Set ib_device::adversarial_mode during capability init so that the
uverbs core layer can report adversarial mode to rdma-core and enforce
that only adversarial-mode-aware userspace opens a context on this
device.

Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
---
 drivers/infiniband/hw/mlx5/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index ace4cdc95c9d..f8af90941b53 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -4778,6 +4778,7 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev)
 	}
 
 	dev->ib_dev.use_cq_dim = true;
+	dev->ib_dev.adversarial_mode = mlx5_ib_adversarial_mode;
 
 	return 0;
 
-- 
2.18.1

Re: [PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Jacob Moroni]

Hi,

Out of curiosity, is the idea that these restrictions will eventually
be lifted once the
ULPs (and I guess the drivers) are hardened? Or is it considered
infeasible? I am
most curious about RDMA CM.

Even if the drivers are hardened (no round-tripping WR IDs thru HW
from WQE->CQE,
sanitizing values from FW, etc.), I think it may be impossible to
protect against certain
bogus payload scenarios such as a NIC reporting a CQE even though it
never wrote the
data (or wrote bogus data), but this sort of risk seems like it would
apply to untrusted
Ethernet devices as well, IIUC. I can imagine it being very difficult
to harden RDMA CM
for this though.

Thanks,
Jake

Re: [PATCH rdma-next 08/23] RDMA/core: Suppress kverbs_provider for adversarial_mode devices

[Jason Gunthorpe]

On Tue, Nov 03, 2026 at 02:03:22PM -0500, Jacob Moroni wrote:
> Hi,
> 
> Out of curiosity, is the idea that these restrictions will
> eventually be lifted once the ULPs (and I guess the drivers) are
> hardened? Or is it considered infeasible? I am most curious about
> RDMA CM.

I think it depends on people's interest. This is not easy work and I
would insist any attempt be "complete" in that it at least
architecturally defends against a remote compromise.

> Even if the drivers are hardened (no round-tripping WR IDs thru HW
> from WQE->CQE, sanitizing values from FW, etc.), I think it may be
> impossible to protect against certain bogus payload scenarios such
> as a NIC reporting a CQE even though it never wrote the data (or
> wrote bogus data), but this sort of risk seems like it would apply
> to untrusted Ethernet devices as well, IIUC. I can imagine it being
> very difficult to harden RDMA CM for this though.

Yes, the verbs user is exposed to some of the threat as the
completions are no longer reliable. The user can assume the wr_id is
correct, and they only get one completion per wr_id, but the rest of
the completion information can't have any effect on integrity.

Jason