[PATCH rdma-next v2 00/15] RDMA: Support HW requiring relaxed ordering and add Unordered access flag

[PATCH rdma-next v2 00/15] RDMA: Support HW requiring relaxed ordering and add Unordered access flag

[Michael Gur]

This series prepares the mlx5 stack for hardware that restricts strong-
ordered memory accesses, i.e. HW whose fabric doesn't guarantee memory
operation ordering. The HW may choose to reject creation of an mkey that
doesn't request relaxed ordering, so every mkey the driver creates -
internal and user-facing - has to be handled accordingly.

The previously added 'ConnectX-10 NVLink-C2C' PCI device is such a device,
as will be any future PCIe Gen7 UIO devices.

The series does four things:

1. Allow optional access flags on all memory registration
   The uverbs ABI defines the optional access flags range as best-effort:
   drivers must accept any flag in this range and treat support as
   opportunistic, never failing registration. Several registration paths
   (DM, DEVX UMEM, dmabuf ioctl, reg_mr ioctl) did not allow this range
   and would reject such requests. These are fixed to allow newly added
   optional access flag added later in the series and any future optional
   access flag.

2. Groundwork and internal mkeys
   Rename the relaxed-ordering IFC fields to the new naming, add the new
   ordering capability and mkey-context bits, and consolidate RO-setting
   to a single helper in mlx5_core so all consumers share one cap-gated
   implementation. Then enable relaxed ordering on the internal mkeys that
   don't require strong ordering (resource dump, vfio live-migration data,
   ODP null mkey and FW tracer), so they keep working on strict-RO HW.

3. New Unordered access flag and enforcement
   Add IB_ACCESS_UNORDERED, an optional access flag that additionally
   allows read-after-write reordering, and wire up mlx5 support for it.
   The rdma-core series adding the flag, with its full definition in the
   man pages: https://github.com/linux-rdma/rdma-core/pull/1780

4. Enforce no strong-ordering in user MR registration on HW that requires
   it.

Changes in v2:
- Dropped the patch that enabled relaxed ordering on Memory Window mkey on
  strict-RO HW. Following Jason's comment, RO is not enforced over indirect
  mkeys, as it takes no effect.
- Added enabling of RO on FW tracer mkey, setting RO only on HW that
  restricts strong ordering, to work around the mkey creation limitation
  (patch 9).
- Following Jason's comment, extended the new RO setter to support setting
  the new Unordered access as well (patch 7).
- Added RO enforcement on FRMR 'set pinned' commands (patch 15).
- Link to v1: https://lore.kernel.org/r/20260726092943.2880176-1-michaelgur@nvidia.com

Michael Gur (15):
  RDMA/mlx5: Allow optional access flags in DM registration
  RDMA/mlx5: Allow optional access flags in DEVX UMEM registration
  RDMA/core: Allow optional access flags in dmabuf reg ioctl
  RDMA/core: Allow optional access flags in reg mr ioctl
  mlx5: Rename IFC bits of relaxed ordering fields
  net/mlx5: Add IFC bits for new ordering caps
  mlx5: Consolidate mkey ordering setting into a single core helper
  net/mlx5: Enable relaxed ordering on resource dump mkey
  net/mlx5: Enable relaxed ordering on FW tracer mkey
  vfio/mlx5: Enable relaxed ordering on the live migration data mkey
  RDMA/mlx5: Enable relaxed ordering on ODP null mkey
  RDMA/mlx5: Converge UMR access-flag cap checks
  RDMA/uverbs: Add new Unordered MR access flag
  RDMA/mlx5: Support new Unordered access flag
  RDMA/mlx5: Enforce relaxed ordering when HW requires it

 drivers/infiniband/core/umem.c                |  2 +-
 drivers/infiniband/core/uverbs_std_types_mr.c |  4 +-
 drivers/infiniband/hw/mlx5/devx.c             |  2 +-
 drivers/infiniband/hw/mlx5/mlx5_ib.h          | 13 +++-
 drivers/infiniband/hw/mlx5/mr.c               | 76 +++++++++----------
 drivers/infiniband/hw/mlx5/odp.c              |  1 +
 drivers/infiniband/hw/mlx5/umr.c              | 18 ++---
 drivers/infiniband/hw/mlx5/umr.h              | 45 ++++++-----
 .../ethernet/mellanox/mlx5/core/data_direct.c |  8 +-
 .../mellanox/mlx5/core/diag/fw_tracer.c       |  2 +
 .../mellanox/mlx5/core/diag/rsc_dump.c        |  1 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 -
 .../ethernet/mellanox/mlx5/core/en/params.c   |  2 +-
 .../ethernet/mellanox/mlx5/core/en_common.c   | 13 +---
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  4 +-
 drivers/net/ethernet/mellanox/mlx5/core/mr.c  | 32 ++++++++
 drivers/vfio/pci/mlx5/cmd.c                   |  7 +-
 include/linux/mlx5/driver.h                   |  8 ++
 include/linux/mlx5/mlx5_ifc.h                 | 24 ++++--
 include/rdma/ib_verbs.h                       |  1 +
 include/uapi/rdma/ib_user_ioctl_verbs.h       |  1 +
 21 files changed, 160 insertions(+), 105 deletions(-)

-- 
2.52.0

[PATCH rdma-next v2 01/15] RDMA/mlx5: Allow optional access flags in DM registration

[Michael Gur]

The uverbs ABI reserves the optional access flags range as best-effort:
drivers should accept any flag in this range and treat the actual support
as opportunistic, never failing registration when an optional flag is
requested.

Add the optional access flag to the allow-lists of DM registrations.
Uverbs users can now request any optional access flag on DM registrations
and mlx5 will apply the flag's effect on a best-effort basis when
supported.

Fixes: 68d384b906cf ("RDMA/core: Add optional access flags range")
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 4a6ee14ffd7d..86465d9d2b5a 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -661,11 +661,13 @@ struct mlx5_ib_mkey {
 					 IB_ACCESS_REMOTE_WRITE  |\
 					 IB_ACCESS_REMOTE_READ   |\
 					 IB_ACCESS_REMOTE_ATOMIC |\
+					 IB_ACCESS_OPTIONAL      |\
 					 IB_ZERO_BASED)
 
 #define MLX5_IB_DM_SW_ICM_ALLOWED_ACCESS (IB_ACCESS_LOCAL_WRITE   |\
 					  IB_ACCESS_REMOTE_WRITE  |\
 					  IB_ACCESS_REMOTE_READ   |\
+					  IB_ACCESS_OPTIONAL      |\
 					  IB_ZERO_BASED)
 
 #define mlx5_update_odp_stats(mr, counter_name, value)		\
-- 
2.52.0

[PATCH rdma-next v2 02/15] RDMA/mlx5: Allow optional access flags in DEVX UMEM registration

[Michael Gur]

The uverbs ABI reserves the optional access flags range as best-effort:
drivers should accept any flag in this range and treat the actual support
as opportunistic, never failing registration when an optional flag is
requested.

Add the optional access flag to the allow-list of DEVX UMEM registration.
Uverbs users can now request any optional access flag on these
registrations and mlx5 will apply the flag's effect on a best-effort
basis when supported.

Fixes: 72b2f7608a59 ("RDMA/mlx5: Enable ATS support for MRs and umems")
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/devx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index fc6f793dcc65..ec281a4ba42e 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -2410,7 +2410,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_UMEM_REG)(
 				 IB_ACCESS_LOCAL_WRITE |
 				 IB_ACCESS_REMOTE_WRITE |
 				 IB_ACCESS_REMOTE_READ |
-				 IB_ACCESS_RELAXED_ORDERING);
+				 IB_ACCESS_OPTIONAL);
 	if (err)
 		return err;
 
-- 
2.52.0

[PATCH rdma-next v2 03/15] RDMA/core: Allow optional access flags in dmabuf reg ioctl

[Michael Gur]

The uverbs ABI reserves the optional access flags range as best-effort:
drivers should accept any flag in this range and treat the actual support
as opportunistic, never failing registration when an optional flag is
requested.

Add the optional access flag to the allow-list of dmabuf registration
ioctl. Uverbs users can now request any optional access flag on dmabuf
registrations and drivers will apply the flag's effect on a best-effort
basis when supported.

Fixes: bfe0cc6eb249 ("RDMA/uverbs: Add uverbs command for dma-buf based MR registration")
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/core/uverbs_std_types_mr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index 0c72f801e0d3..b6d6cbe448a2 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -229,7 +229,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)(
 				 IB_ACCESS_REMOTE_READ |
 				 IB_ACCESS_REMOTE_WRITE |
 				 IB_ACCESS_REMOTE_ATOMIC |
-				 IB_ACCESS_RELAXED_ORDERING);
+				 IB_ACCESS_OPTIONAL);
 	if (ret)
 		return ret;
 
-- 
2.52.0

[PATCH rdma-next v2 04/15] RDMA/core: Allow optional access flags in reg mr ioctl

[Michael Gur]

The uverbs ABI reserves the optional access flags range as best-effort:
drivers should accept any flag in this range and treat the actual support
as opportunistic, never failing registration when an optional flag is
requested.

Add the optional access flag to the allow-list of dmabuf registration
through the reg_mr ioctl. Uverbs users can now request any optional access
flag on dmabuf registrations and mlx5 will apply the flag's effect on a
best-effort basis when supported.

Fixes: 5b2e45049dc0 ("IB/core: Add UVERBS_METHOD_REG_MR on the MR object")
Cc: stable@vger.kernel.org
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/core/uverbs_std_types_mr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/uverbs_std_types_mr.c b/drivers/infiniband/core/uverbs_std_types_mr.c
index b6d6cbe448a2..ce6c9aaea95d 100644
--- a/drivers/infiniband/core/uverbs_std_types_mr.c
+++ b/drivers/infiniband/core/uverbs_std_types_mr.c
@@ -332,7 +332,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
 				     IB_ACCESS_REMOTE_READ |
 				     IB_ACCESS_REMOTE_WRITE |
 				     IB_ACCESS_REMOTE_ATOMIC |
-				     IB_ACCESS_RELAXED_ORDERING;
+				     IB_ACCESS_OPTIONAL;
 	} else {
 		if (!has_addr || has_fd_offset)
 			return -EINVAL;
-- 
2.52.0

[PATCH rdma-next v2 05/15] mlx5: Rename IFC bits of relaxed ordering fields

[Michael Gur]

Align the IFC field names with the new naming:
- relaxed_ordering_write_umr -> order_write_after_write_umr (cap)
- relaxed_ordering_read_umr -> pci_relaxed_ordered_read_umr (cap)
- relaxed_ordering_write -> mkc_order_write_after_write_ro (cap)
- relaxed_ordering_write -> order_write_after_write (mkc)
- relaxed_ordering_read -> pci_relaxed_ordered_read (cap and mkc)

Update all references across drivers.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mr.c                  | 16 ++++++++--------
 drivers/infiniband/hw/mlx5/umr.c                 | 14 +++++++-------
 drivers/infiniband/hw/mlx5/umr.h                 |  8 ++++----
 .../ethernet/mellanox/mlx5/core/data_direct.c    |  8 ++++----
 .../net/ethernet/mellanox/mlx5/core/en/params.c  |  2 +-
 .../net/ethernet/mellanox/mlx5/core/en_common.c  |  8 ++++----
 include/linux/mlx5/mlx5_ifc.h                    | 12 ++++++------
 7 files changed, 34 insertions(+), 34 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 176f87ba062c..a15610c6211c 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -71,14 +71,14 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
 	MLX5_SET(mkc, mkc, lr, 1);
 
 	if (acc & IB_ACCESS_RELAXED_ORDERING) {
-		if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write))
-			MLX5_SET(mkc, mkc, relaxed_ordering_write, 1);
+		if (MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro))
+			MLX5_SET(mkc, mkc, order_write_after_write, 1);
 
-		if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read) ||
+		if (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
 		    (MLX5_CAP_GEN(dev->mdev,
 				  relaxed_ordering_read_pci_enabled) &&
 		     pcie_relaxed_ordering_enabled(dev->mdev->pdev)))
-			MLX5_SET(mkc, mkc, relaxed_ordering_read, 1);
+			MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, 1);
 	}
 
 	MLX5_SET(mkc, mkc, pd, to_mpd(pd)->pdn);
@@ -146,14 +146,14 @@ static int get_unchangeable_access_flags(struct mlx5_ib_dev *dev,
 		ret |= IB_ACCESS_REMOTE_ATOMIC;
 
 	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-	    MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write) &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write_umr))
+	    MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro) &&
+	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
 		ret |= IB_ACCESS_RELAXED_ORDERING;
 
 	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-	    (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read) ||
+	    (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
 	     MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_pci_enabled)) &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_umr))
+	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
 		ret |= IB_ACCESS_RELAXED_ORDERING;
 
 	return ret;
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 711075e3d9ea..144cf851c41e 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -59,10 +59,10 @@ static __be64 get_umr_update_access_mask(struct mlx5_ib_dev *dev)
 	if (MLX5_CAP_GEN(dev->mdev, atomic))
 		result |= MLX5_MKEY_MASK_A;
 
-	if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write_umr))
+	if (MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
 		result |= MLX5_MKEY_MASK_RELAXED_ORDERING_WRITE;
 
-	if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_umr))
+	if (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
 		result |= MLX5_MKEY_MASK_RELAXED_ORDERING_READ;
 
 	return cpu_to_be64(result);
@@ -88,11 +88,11 @@ static int umr_check_mkey_mask(struct mlx5_ib_dev *dev, u64 mask)
 		return -EPERM;
 
 	if (mask & MLX5_MKEY_MASK_RELAXED_ORDERING_WRITE &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write_umr))
+	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
 		return -EPERM;
 
 	if (mask & MLX5_MKEY_MASK_RELAXED_ORDERING_READ &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_umr))
+	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
 		return -EPERM;
 
 	return 0;
@@ -446,7 +446,7 @@ static void mlx5r_umr_set_access_flags(struct mlx5_ib_dev *dev,
 				       unsigned int access_flags)
 {
 	bool ro_read = (access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-		       (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read) ||
+		       (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
 			pcie_relaxed_ordering_enabled(dev->mdev->pdev));
 
 	MLX5_SET(mkc, seg, a, !!(access_flags & IB_ACCESS_REMOTE_ATOMIC));
@@ -454,9 +454,9 @@ static void mlx5r_umr_set_access_flags(struct mlx5_ib_dev *dev,
 	MLX5_SET(mkc, seg, rr, !!(access_flags & IB_ACCESS_REMOTE_READ));
 	MLX5_SET(mkc, seg, lw, !!(access_flags & IB_ACCESS_LOCAL_WRITE));
 	MLX5_SET(mkc, seg, lr, 1);
-	MLX5_SET(mkc, seg, relaxed_ordering_write,
+	MLX5_SET(mkc, seg, order_write_after_write,
 		 !!(access_flags & IB_ACCESS_RELAXED_ORDERING));
-	MLX5_SET(mkc, seg, relaxed_ordering_read, ro_read);
+	MLX5_SET(mkc, seg, pci_relaxed_ordered_read, ro_read);
 }
 
 int mlx5r_umr_rereg_pd_access(struct mlx5_ib_mr *mr, struct ib_pd *pd,
diff --git a/drivers/infiniband/hw/mlx5/umr.h b/drivers/infiniband/hw/mlx5/umr.h
index bda7123781a9..04dec726b267 100644
--- a/drivers/infiniband/hw/mlx5/umr.h
+++ b/drivers/infiniband/hw/mlx5/umr.h
@@ -61,14 +61,14 @@ static inline bool mlx5r_umr_can_reconfig(struct mlx5_ib_dev *dev,
 		return false;
 
 	if ((diffs & IB_ACCESS_RELAXED_ORDERING) &&
-	    MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write) &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write_umr))
+	    MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro) &&
+	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
 		return false;
 
 	if ((diffs & IB_ACCESS_RELAXED_ORDERING) &&
-	    (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read) ||
+	    (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
 	     MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_pci_enabled)) &&
-	    !MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_umr))
+	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
 		return false;
 
 	return true;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/data_direct.c b/drivers/net/ethernet/mellanox/mlx5/core/data_direct.c
index 5afb11a2a9f0..b2eb2cb0d734 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/data_direct.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/data_direct.c
@@ -145,13 +145,13 @@ static int mlx5_data_direct_create_resources(struct mlx5_core_dev *mdev)
 	mdev->data_direct->pdn = pdn;
 
 	/* create another mkey with RO support */
-	if (MLX5_CAP_GEN(mdev, relaxed_ordering_write)) {
-		MLX5_SET(mkc, mkc, relaxed_ordering_write, 1);
+	if (MLX5_CAP_GEN(mdev, mkc_order_write_after_write_ro)) {
+		MLX5_SET(mkc, mkc, order_write_after_write, 1);
 		ro_supp = true;
 	}
 
-	if (MLX5_CAP_GEN(mdev, relaxed_ordering_read)) {
-		MLX5_SET(mkc, mkc, relaxed_ordering_read, 1);
+	if (MLX5_CAP_GEN(mdev, pci_relaxed_ordered_read)) {
+		MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, 1);
 		ro_supp = true;
 	}
 
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
index 5caf7820a136..b0fc1fdd1339 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c
@@ -912,7 +912,7 @@ static void mlx5e_build_rx_cq_param(struct mlx5_core_dev *mdev,
 static u8 rq_end_pad_mode(struct mlx5_core_dev *mdev, struct mlx5e_params *params)
 {
 	bool lro_en = params->packet_merge.type == MLX5E_PACKET_MERGE_LRO;
-	bool ro = MLX5_CAP_GEN(mdev, relaxed_ordering_write);
+	bool ro = MLX5_CAP_GEN(mdev, mkc_order_write_after_write_ro);
 
 	return ro && lro_en ?
 		MLX5_WQ_END_PAD_MODE_NONE : MLX5_WQ_END_PAD_MODE_ALIGN;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
index c2d4bae533de..2dbcf6673916 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
@@ -40,13 +40,13 @@
 
 void mlx5e_mkey_set_relaxed_ordering(struct mlx5_core_dev *mdev, void *mkc)
 {
-	bool ro_write = MLX5_CAP_GEN(mdev, relaxed_ordering_write);
-	bool ro_read = MLX5_CAP_GEN(mdev, relaxed_ordering_read) ||
+	bool ro_write = MLX5_CAP_GEN(mdev, mkc_order_write_after_write_ro);
+	bool ro_read = MLX5_CAP_GEN(mdev, pci_relaxed_ordered_read) ||
 		       (pcie_relaxed_ordering_enabled(mdev->pdev) &&
 			MLX5_CAP_GEN(mdev, relaxed_ordering_read_pci_enabled));
 
-	MLX5_SET(mkc, mkc, relaxed_ordering_read, ro_read);
-	MLX5_SET(mkc, mkc, relaxed_ordering_write, ro_write);
+	MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, ro_read);
+	MLX5_SET(mkc, mkc, order_write_after_write, ro_write);
 }
 
 int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey)
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 697fc9387b73..8531d611fa9b 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1711,15 +1711,15 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 
 	u8         max_sgl_for_optimized_performance[0x8];
 	u8         log_max_cq_sz[0x8];
-	u8         relaxed_ordering_write_umr[0x1];
-	u8         relaxed_ordering_read_umr[0x1];
+	u8         order_write_after_write_umr[0x1];
+	u8         pci_relaxed_ordered_read_umr[0x1];
 	u8         reserved_at_d2[0x7];
 	u8         virtio_net_device_emualtion_manager[0x1];
 	u8         virtio_blk_device_emualtion_manager[0x1];
 	u8         log_max_cq[0x5];
 
 	u8         log_max_eq_sz[0x8];
-	u8         relaxed_ordering_write[0x1];
+	u8         mkc_order_write_after_write_ro[0x1];
 	u8         relaxed_ordering_read_pci_enabled[0x1];
 	u8         log_max_mkey[0x6];
 	u8         reserved_at_f0[0x6];
@@ -1951,7 +1951,7 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         reserved_at_320[0x3];
 	u8         log_max_transport_domain[0x5];
 	u8         reserved_at_328[0x2];
-	u8	   relaxed_ordering_read[0x1];
+	u8         pci_relaxed_ordered_read[0x1];
 	u8         log_max_pd[0x5];
 	u8         dp_ordering_ooo_all_ud[0x1];
 	u8         dp_ordering_ooo_all_uc[0x1];
@@ -4528,7 +4528,7 @@ struct mlx5_ifc_mkc_bits {
 	u8         reserved_at_2[0x1];
 	u8         access_mode_4_2[0x3];
 	u8         reserved_at_6[0x7];
-	u8         relaxed_ordering_write[0x1];
+	u8         order_write_after_write[0x1];
 	u8         reserved_at_e[0x1];
 	u8         small_fence_on_rdma_read_response[0x1];
 	u8         umr_en[0x1];
@@ -4570,7 +4570,7 @@ struct mlx5_ifc_mkc_bits {
 	u8         translations_octword_size[0x20];
 
 	u8         reserved_at_1c0[0x19];
-	u8         relaxed_ordering_read[0x1];
+	u8         pci_relaxed_ordered_read[0x1];
 	u8         log_page_size[0x6];
 
 	u8         reserved_at_1e0[0x5];
-- 
2.52.0

[PATCH rdma-next v2 06/15] net/mlx5: Add IFC bits for new ordering caps

[Michael Gur]

Add new HCA capability bits:
- mkc_order_read_after_write
- mkc_order_write_after_write_ro_only

Add new mkey context field:
- order_read_after_write (2-bit)

These fields enable a new level of memory access order relaxation,
allowing read-after-write reordering and allow to expose an indication
that devices don't support strict ordering of memory access.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 include/linux/mlx5/mlx5_ifc.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 8531d611fa9b..26801ad51f2c 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -2005,7 +2005,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         roce_rw_supported[0x1];
 	u8         log_max_current_uc_list_wr_supported[0x1];
 	u8         log_max_stride_sz_rq[0x5];
-	u8         reserved_at_3a8[0x3];
+	u8         mkc_order_read_after_write[0x1];
+	u8         mkc_order_write_after_write_ro_only[0x1];
+	u8         reserved_at_3aa[0x1];
 	u8         log_min_stride_sz_rq[0x5];
 	u8         reserved_at_3b0[0x2];
 	u8         qp_latency_sensitive_disable[0x1];
@@ -4522,6 +4524,10 @@ enum {
 	MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX = 0,
 };
 
+enum {
+	MLX5_MKC_ORDER_READ_AFTER_WRITE_RO = 1,
+};
+
 struct mlx5_ifc_mkc_bits {
 	u8         reserved_at_0[0x1];
 	u8         free[0x1];
@@ -4569,7 +4575,9 @@ struct mlx5_ifc_mkc_bits {
 
 	u8         translations_octword_size[0x20];
 
-	u8         reserved_at_1c0[0x19];
+	u8         reserved_at_1c0[0x16];
+	u8         order_read_after_write[0x2];
+	u8         reserved_at_1d8[0x1];
 	u8         pci_relaxed_ordered_read[0x1];
 	u8         log_page_size[0x6];
 
-- 
2.52.0

[PATCH rdma-next v2 07/15] mlx5: Consolidate mkey ordering setting into a single core helper

[Michael Gur]

Consolidate the mkey ordering setting of all mlx5 consumers into a
single exported core helper. The helper takes the requested ordering
level (strict, relaxed or unordered) and sets the mkey ordering fields
according to the device caps, so consumers no longer open-code the cap
checks. The unordered level is used later in this series.

Switch the RDMA MR creation and UMR access-modify paths to the helper.
As before, strict ordering leaves the mkey context untouched.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  7 ++++
 drivers/infiniband/hw/mlx5/mr.c               | 12 ++-----
 drivers/infiniband/hw/mlx5/umr.c              | 10 ++----
 drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 -
 .../ethernet/mellanox/mlx5/core/en_common.c   | 13 +-------
 .../net/ethernet/mellanox/mlx5/core/en_main.c |  4 +--
 drivers/net/ethernet/mellanox/mlx5/core/mr.c  | 32 +++++++++++++++++++
 include/linux/mlx5/driver.h                   |  8 +++++
 8 files changed, 55 insertions(+), 32 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 86465d9d2b5a..dd1fe8fd4220 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1675,6 +1675,13 @@ static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev,
 	return access_flags & IB_ACCESS_RELAXED_ORDERING;
 }
 
+static inline enum mlx5_mkey_order mlx5r_access_to_mkey_order(int access_flags)
+{
+	if (access_flags & IB_ACCESS_RELAXED_ORDERING)
+		return MLX5_MKEY_ORDER_RELAXED;
+	return MLX5_MKEY_ORDER_STRICT;
+}
+
 int set_roce_addr(struct mlx5_ib_dev *dev, u32 port_num,
 		  unsigned int index, const union ib_gid *gid,
 		  const struct ib_gid_attr *attr);
diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index a15610c6211c..6f57dfc7f4cd 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -70,16 +70,8 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
 	MLX5_SET(mkc, mkc, lw, !!(acc & IB_ACCESS_LOCAL_WRITE));
 	MLX5_SET(mkc, mkc, lr, 1);
 
-	if (acc & IB_ACCESS_RELAXED_ORDERING) {
-		if (MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro))
-			MLX5_SET(mkc, mkc, order_write_after_write, 1);
-
-		if (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
-		    (MLX5_CAP_GEN(dev->mdev,
-				  relaxed_ordering_read_pci_enabled) &&
-		     pcie_relaxed_ordering_enabled(dev->mdev->pdev)))
-			MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, 1);
-	}
+	mlx5_core_mkey_set_ordering(dev->mdev, mkc,
+				    mlx5r_access_to_mkey_order(acc));
 
 	MLX5_SET(mkc, mkc, pd, to_mpd(pd)->pdn);
 	MLX5_SET(mkc, mkc, qpn, 0xffffff);
diff --git a/drivers/infiniband/hw/mlx5/umr.c b/drivers/infiniband/hw/mlx5/umr.c
index 144cf851c41e..2f3147c7f78f 100644
--- a/drivers/infiniband/hw/mlx5/umr.c
+++ b/drivers/infiniband/hw/mlx5/umr.c
@@ -445,18 +445,14 @@ static void mlx5r_umr_set_access_flags(struct mlx5_ib_dev *dev,
 				       struct mlx5_mkey_seg *seg,
 				       unsigned int access_flags)
 {
-	bool ro_read = (access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-		       (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
-			pcie_relaxed_ordering_enabled(dev->mdev->pdev));
-
 	MLX5_SET(mkc, seg, a, !!(access_flags & IB_ACCESS_REMOTE_ATOMIC));
 	MLX5_SET(mkc, seg, rw, !!(access_flags & IB_ACCESS_REMOTE_WRITE));
 	MLX5_SET(mkc, seg, rr, !!(access_flags & IB_ACCESS_REMOTE_READ));
 	MLX5_SET(mkc, seg, lw, !!(access_flags & IB_ACCESS_LOCAL_WRITE));
 	MLX5_SET(mkc, seg, lr, 1);
-	MLX5_SET(mkc, seg, order_write_after_write,
-		 !!(access_flags & IB_ACCESS_RELAXED_ORDERING));
-	MLX5_SET(mkc, seg, pci_relaxed_ordered_read, ro_read);
+
+	mlx5_core_mkey_set_ordering(dev->mdev, seg,
+				    mlx5r_access_to_mkey_order(access_flags));
 }
 
 int mlx5r_umr_rereg_pd_access(struct mlx5_ib_mr *mr, struct ib_pd *pd,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index a3945195f454..eea4844d7645 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -1203,7 +1203,6 @@ int mlx5e_modify_tirs_lb(struct mlx5_core_dev *mdev, bool enable_uc_lb,
 			 bool enable_mc_lb);
 int mlx5e_refresh_tirs(struct mlx5_core_dev *mdev, bool enable_uc_lb,
 		       bool enable_mc_lb);
-void mlx5e_mkey_set_relaxed_ordering(struct mlx5_core_dev *mdev, void *mkc);
 
 /* common netdev helpers */
 void mlx5e_create_q_counters(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
index 2dbcf6673916..f01340998f52 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_common.c
@@ -38,17 +38,6 @@
  * Global resources are common to all the netdevices created on the same nic.
  */
 
-void mlx5e_mkey_set_relaxed_ordering(struct mlx5_core_dev *mdev, void *mkc)
-{
-	bool ro_write = MLX5_CAP_GEN(mdev, mkc_order_write_after_write_ro);
-	bool ro_read = MLX5_CAP_GEN(mdev, pci_relaxed_ordered_read) ||
-		       (pcie_relaxed_ordering_enabled(mdev->pdev) &&
-			MLX5_CAP_GEN(mdev, relaxed_ordering_read_pci_enabled));
-
-	MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, ro_read);
-	MLX5_SET(mkc, mkc, order_write_after_write, ro_write);
-}
-
 int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey)
 {
 	int inlen = MLX5_ST_SZ_BYTES(create_mkey_in);
@@ -64,7 +53,7 @@ int mlx5e_create_mkey(struct mlx5_core_dev *mdev, u32 pdn, u32 *mkey)
 	MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_PA);
 	MLX5_SET(mkc, mkc, lw, 1);
 	MLX5_SET(mkc, mkc, lr, 1);
-	mlx5e_mkey_set_relaxed_ordering(mdev, mkc);
+	mlx5_core_mkey_set_ordering(mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 	MLX5_SET(mkc, mkc, pd, pdn);
 	MLX5_SET(mkc, mkc, length64, 1);
 	MLX5_SET(mkc, mkc, qpn, 0xffffff);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index c3c2c5720c6f..efd69b1bfb59 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -462,7 +462,7 @@ static int mlx5e_create_umr_mkey(struct mlx5_core_dev *mdev,
 	MLX5_SET(mkc, mkc, lw, 1);
 	MLX5_SET(mkc, mkc, lr, 1);
 	MLX5_SET(mkc, mkc, access_mode_1_0, mlx5e_mpwrq_access_mode(umr_mode));
-	mlx5e_mkey_set_relaxed_ordering(mdev, mkc);
+	mlx5_core_mkey_set_ordering(mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 	MLX5_SET(mkc, mkc, qpn, 0xffffff);
 	MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.hw_objs.pdn);
 	MLX5_SET64(mkc, mkc, len, npages << page_shift);
@@ -801,7 +801,7 @@ static int mlx5e_create_rq_hd_mkey(struct mlx5_core_dev *mdev,
 	MLX5_SET(mkc, mkc, lw, 1);
 	MLX5_SET(mkc, mkc, lr, 1);
 	MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_MTT);
-	mlx5e_mkey_set_relaxed_ordering(mdev, mkc);
+	mlx5_core_mkey_set_ordering(mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 	MLX5_SET(mkc, mkc, qpn, 0xffffff);
 	MLX5_SET(mkc, mkc, pd, mdev->mlx5e_res.hw_objs.pdn);
 	MLX5_SET64(mkc, mkc, len, shampo->hd_buf_size);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mr.c b/drivers/net/ethernet/mellanox/mlx5/core/mr.c
index 678f0be81375..eced7d24948c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/mr.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/mr.c
@@ -35,6 +35,38 @@
 #include <linux/mlx5/qp.h>
 #include "mlx5_core.h"
 
+/**
+ * mlx5_core_mkey_set_ordering - Set ordering flags in mkey context
+ * @dev: MLX5 core device
+ * @mkc: Memory key context buffer
+ * @order: Requested ordering level
+ *
+ * Sets the ordering fields in @mkc for @order based on device caps. Call when
+ * creating or modifying a memory key. MLX5_MKEY_ORDER_STRICT leaves @mkc as is.
+ */
+void mlx5_core_mkey_set_ordering(struct mlx5_core_dev *dev, void *mkc,
+				 enum mlx5_mkey_order order)
+{
+	bool ro_write, ro_read;
+
+	if (order == MLX5_MKEY_ORDER_STRICT)
+		return;
+
+	ro_write = MLX5_CAP_GEN(dev, mkc_order_write_after_write_ro);
+	ro_read = MLX5_CAP_GEN(dev, pci_relaxed_ordered_read) ||
+		  (MLX5_CAP_GEN(dev, relaxed_ordering_read_pci_enabled) &&
+		   pcie_relaxed_ordering_enabled(dev->pdev));
+
+	MLX5_SET(mkc, mkc, pci_relaxed_ordered_read, ro_read);
+	MLX5_SET(mkc, mkc, order_write_after_write, ro_write);
+
+	if (order == MLX5_MKEY_ORDER_UNORDERED &&
+	    MLX5_CAP_GEN(dev, mkc_order_read_after_write))
+		MLX5_SET(mkc, mkc, order_read_after_write,
+			 MLX5_MKC_ORDER_READ_AFTER_WRITE_RO);
+}
+EXPORT_SYMBOL(mlx5_core_mkey_set_ordering);
+
 int mlx5_core_create_mkey(struct mlx5_core_dev *dev, u32 *mkey, u32 *in,
 			  int inlen)
 {
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index abb99371ff13..901744e3d63a 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -697,6 +697,12 @@ enum mlx5_sw_icm_type {
 	MLX5_SW_ICM_TYPE_SW_ENCAP,
 };
 
+enum mlx5_mkey_order {
+	MLX5_MKEY_ORDER_STRICT,
+	MLX5_MKEY_ORDER_RELAXED,
+	MLX5_MKEY_ORDER_UNORDERED,
+};
+
 #define MLX5_MAX_RESERVED_GIDS 8
 
 struct mlx5_rsvd_gids {
@@ -1037,6 +1043,8 @@ void mlx5_trigger_health_work(struct mlx5_core_dev *dev);
 int mlx5_frag_buf_alloc_node(struct mlx5_core_dev *dev, int size,
 			     struct mlx5_frag_buf *buf, int node);
 void mlx5_frag_buf_free(struct mlx5_core_dev *dev, struct mlx5_frag_buf *buf);
+void mlx5_core_mkey_set_ordering(struct mlx5_core_dev *dev, void *mkc,
+				 enum mlx5_mkey_order order);
 int mlx5_core_create_mkey(struct mlx5_core_dev *dev, u32 *mkey, u32 *in,
 			  int inlen);
 int mlx5_core_destroy_mkey(struct mlx5_core_dev *dev, u32 mkey);
-- 
2.52.0

[PATCH rdma-next v2 08/15] net/mlx5: Enable relaxed ordering on resource dump mkey

[Michael Gur]

As resource dump logic reads data only following an event that is
delivered with strong ordering, there is no need for strong ordering of
its memory accesses.
Add relaxed ordering access flag to its registered memory, thus allowing
better performance and allowing resource dump on HW that restricts
strong ordering.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/diag/rsc_dump.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/rsc_dump.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/rsc_dump.c
index 8044419fb5eb..5b88cd009472 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/diag/rsc_dump.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/rsc_dump.c
@@ -236,6 +236,7 @@ static int mlx5_rsc_dump_create_mkey(struct mlx5_core_dev *mdev, u32 pdn,
 	MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_PA);
 	MLX5_SET(mkc, mkc, lw, 1);
 	MLX5_SET(mkc, mkc, lr, 1);
+	mlx5_core_mkey_set_ordering(mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 
 	MLX5_SET(mkc, mkc, pd, pdn);
 	MLX5_SET(mkc, mkc, length64, 1);
-- 
2.52.0

[PATCH rdma-next v2 09/15] net/mlx5: Enable relaxed ordering on FW tracer mkey

[Michael Gur]

On HW that restricts strong ordering, mkey creation fails unless relaxed
ordering is set, which prevents the FW tracer from being enabled.

Set relaxed ordering on the FW tracer mkey only on such HW, to work
around this limitation. This has no effect on the ordering of the trace
data, as once the mkey is configured as the tracer output buffer, FW and
HW write to it with strong ordering regardless of the mkey ordering
attributes. Strong ordering is required by design, as the driver polls
the buffer and considers a block valid once its last bytes hold a valid
timestamp. Out of order writes could expose a partially written block
to the driver.

Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
index 1493d8106c1a..003ba788dd4f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c
@@ -208,6 +208,8 @@ static int mlx5_fw_tracer_create_mkey(struct mlx5_fw_tracer *tracer)
 		 DIV_ROUND_UP(TRACER_BUFFER_PAGE_NUM, 2));
 	MLX5_SET64(mkc, mkc, start_addr, tracer->buff.dma);
 	MLX5_SET64(mkc, mkc, len, tracer->buff.size);
+	if (MLX5_CAP_GEN(dev, mkc_order_write_after_write_ro_only))
+		mlx5_core_mkey_set_ordering(dev, mkc, MLX5_MKEY_ORDER_RELAXED);
 	err = mlx5_core_create_mkey(dev, &tracer->buff.mkey, in, inlen);
 	if (err)
 		mlx5_core_warn(dev, "FWTracer: Failed to create mkey, %d\n", err);
-- 
2.52.0

[PATCH rdma-next v2 10/15] vfio/mlx5: Enable relaxed ordering on the live migration data mkey

[Michael Gur]

As firmware guarantees that all device-issued data writes complete before
posting of CQE, there is no need for strong ordering of memory accesses
on the live migration data mkey.
Enable relaxed ordering on the mkey via the mlx5_core RO setter.

This allows better performance and supporting vfio live migration on HW
that restricts memory registration with strict ordering.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/vfio/pci/mlx5/cmd.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index 04180035a7fe..335284037c7c 100644
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -320,7 +320,7 @@ static int mlx5vf_cmd_get_vhca_id(struct mlx5_core_dev *mdev, u16 function_id,
 	return ret;
 }
 
-static u32 *alloc_mkey_in(u32 npages, u32 pdn)
+static u32 *alloc_mkey_in(struct mlx5_core_dev *mdev, u32 npages, u32 pdn)
 {
 	int inlen;
 	void *mkc;
@@ -348,6 +348,7 @@ static u32 *alloc_mkey_in(u32 npages, u32 pdn)
 	MLX5_SET(mkc, mkc, log_page_size, PAGE_SHIFT);
 	MLX5_SET(mkc, mkc, translations_octword_size, DIV_ROUND_UP(npages, 2));
 	MLX5_SET64(mkc, mkc, len, npages * PAGE_SIZE);
+	mlx5_core_mkey_set_ordering(mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 
 	return in;
 }
@@ -439,7 +440,7 @@ static int mlx5vf_dma_data_buffer(struct mlx5_vhca_data_buffer *buf)
 	if (buf->mkey_in || !buf->npages)
 		return -EINVAL;
 
-	buf->mkey_in = alloc_mkey_in(buf->npages, buf->migf->pdn);
+	buf->mkey_in = alloc_mkey_in(mdev, buf->npages, buf->migf->pdn);
 	if (!buf->mkey_in)
 		return -ENOMEM;
 
@@ -1439,7 +1440,7 @@ static int mlx5vf_alloc_qp_recv_resources(struct mlx5_core_dev *mdev,
 
 	recv_buf->npages = npages;
 
-	recv_buf->mkey_in = alloc_mkey_in(npages, pdn);
+	recv_buf->mkey_in = alloc_mkey_in(mdev, npages, pdn);
 	if (!recv_buf->mkey_in) {
 		err = -ENOMEM;
 		goto end;
-- 
2.52.0

[PATCH rdma-next v2 11/15] RDMA/mlx5: Enable relaxed ordering on ODP null mkey

[Michael Gur]

Enable relaxed ordering on the ODP implicit-MR null mkey.
This allows supporting implicit ODP on HW that restricts memory
registration with strict ordering.

No functional change as this is a null mkey with no memory mappings.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/odp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 9c91e76029d4..b2643ff871a3 100644
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -579,6 +579,7 @@ static int alloc_implicit_mr_null_mkey(struct mlx5_ib_dev *dev,
 	MLX5_SET(mkc, mkc, rr, 1);
 	MLX5_SET(mkc, mkc, lw, 1);
 	MLX5_SET(mkc, mkc, lr, 1);
+	mlx5_core_mkey_set_ordering(dev->mdev, mkc, MLX5_MKEY_ORDER_RELAXED);
 	MLX5_SET(mkc, mkc, free, 0);
 	MLX5_SET(mkc, mkc, umr_en, 0);
 	MLX5_SET(mkc, mkc, access_mode_1_0, MLX5_MKC_ACCESS_MODE_MTT);
-- 
2.52.0

[PATCH rdma-next v2 12/15] RDMA/mlx5: Converge UMR access-flag cap checks

[Michael Gur]

Move the UMR cap check from both MR registration and UMR paths to a
single helper checking the access flags non-modifiable by UMR.

No behavior change for the current flag set.
Converge to prevent code duplication when adding support to new access
flags that are not modifiable by UMR, like the new Unordered access flag
added in following patches.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mr.c  | 30 +++-----------------------
 drivers/infiniband/hw/mlx5/umr.h | 37 ++++++++++++++++++--------------
 2 files changed, 24 insertions(+), 43 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 6f57dfc7f4cd..6848515e06d7 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -127,30 +127,6 @@ static int get_mkc_octo_size(unsigned int access_mode, unsigned int ndescs)
 	return ret;
 }
 
-static int get_unchangeable_access_flags(struct mlx5_ib_dev *dev,
-					 int access_flags)
-{
-	int ret = 0;
-
-	if ((access_flags & IB_ACCESS_REMOTE_ATOMIC) &&
-	    MLX5_CAP_GEN(dev->mdev, atomic) &&
-	    MLX5_CAP_GEN(dev->mdev, umr_modify_atomic_disabled))
-		ret |= IB_ACCESS_REMOTE_ATOMIC;
-
-	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-	    MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro) &&
-	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
-		ret |= IB_ACCESS_RELAXED_ORDERING;
-
-	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
-	    (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
-	     MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_pci_enabled)) &&
-	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
-		ret |= IB_ACCESS_RELAXED_ORDERING;
-
-	return ret;
-}
-
 #define MLX5_FRMR_POOLS_KEY_ACCESS_MODE_KSM_MASK 1ULL
 #define MLX5_FRMR_POOLS_KEY_VENDOR_KEY_SUPPORTED \
 	MLX5_FRMR_POOLS_KEY_ACCESS_MODE_KSM_MASK
@@ -172,7 +148,7 @@ _mlx5_frmr_pool_alloc(struct mlx5_ib_dev *dev, struct ib_umem *umem,
 
 	mr->ibmr.frmr.key.ats = mlx5_umem_needs_ats(dev, umem, access_flags);
 	mr->ibmr.frmr.key.access_flags =
-		get_unchangeable_access_flags(dev, access_flags);
+		mlx5r_umr_get_unchangeable_access_flags(dev, access_flags);
 	mr->ibmr.frmr.key.num_dma_blocks =
 		ib_umem_num_dma_blocks(umem, page_size);
 	mr->ibmr.frmr.key.vendor_key =
@@ -204,7 +180,7 @@ struct mlx5_ib_mr *mlx5_mr_cache_alloc(struct mlx5_ib_dev *dev,
 {
 	struct ib_frmr_key key = {
 		.access_flags =
-			get_unchangeable_access_flags(dev, access_flags),
+			mlx5r_umr_get_unchangeable_access_flags(dev, access_flags),
 		.vendor_key = access_mode == MLX5_MKC_ACCESS_MODE_MTT ?
 				      0 :
 				      MLX5_FRMR_POOLS_KEY_ACCESS_MODE_KSM_MASK,
@@ -323,7 +299,7 @@ static int mlx5r_build_frmr_key(struct ib_device *device,
 
 	out->ats = in->ats;
 	out->access_flags =
-		get_unchangeable_access_flags(dev, in->access_flags);
+		mlx5r_umr_get_unchangeable_access_flags(dev, in->access_flags);
 	out->vendor_key = in->vendor_key;
 	out->num_dma_blocks = in->num_dma_blocks;
 
diff --git a/drivers/infiniband/hw/mlx5/umr.h b/drivers/infiniband/hw/mlx5/umr.h
index 04dec726b267..a6c4f35b75e6 100644
--- a/drivers/infiniband/hw/mlx5/umr.h
+++ b/drivers/infiniband/hw/mlx5/umr.h
@@ -44,34 +44,39 @@ static inline bool mlx5r_umr_can_load_pas(struct mlx5_ib_dev *dev,
 	return true;
 }
 
-/*
- * true if an existing MR can be reconfigured to new access_flags using UMR.
- * Older HW cannot use UMR to update certain elements of the MKC. See
- * get_umr_update_access_mask() and umr_check_mkey_mask()
- */
-static inline bool mlx5r_umr_can_reconfig(struct mlx5_ib_dev *dev,
-					  unsigned int current_access_flags,
-					  unsigned int target_access_flags)
+/* Return subset of access_flags whose mkey fields can't be updated via UMR. */
+static inline unsigned int
+mlx5r_umr_get_unchangeable_access_flags(struct mlx5_ib_dev *dev,
+					unsigned int access_flags)
 {
-	unsigned int diffs = current_access_flags ^ target_access_flags;
+	unsigned int ret = 0;
 
-	if ((diffs & IB_ACCESS_REMOTE_ATOMIC) &&
+	if ((access_flags & IB_ACCESS_REMOTE_ATOMIC) &&
 	    MLX5_CAP_GEN(dev->mdev, atomic) &&
 	    MLX5_CAP_GEN(dev->mdev, umr_modify_atomic_disabled))
-		return false;
+		ret |= IB_ACCESS_REMOTE_ATOMIC;
 
-	if ((diffs & IB_ACCESS_RELAXED_ORDERING) &&
+	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
 	    MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro) &&
 	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
-		return false;
+		ret |= IB_ACCESS_RELAXED_ORDERING;
 
-	if ((diffs & IB_ACCESS_RELAXED_ORDERING) &&
+	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
 	    (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
 	     MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_pci_enabled)) &&
 	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
-		return false;
+		ret |= IB_ACCESS_RELAXED_ORDERING;
 
-	return true;
+	return ret;
+}
+
+/* true if an existing MR can be reconfigured to new access_flags using UMR. */
+static inline bool mlx5r_umr_can_reconfig(struct mlx5_ib_dev *dev,
+					  unsigned int current_access_flags,
+					  unsigned int target_access_flags)
+{
+	return mlx5r_umr_get_unchangeable_access_flags(
+		dev, current_access_flags ^ target_access_flags) == 0;
 }
 
 static inline u64 mlx5r_umr_get_xlt_octo(u64 bytes)
-- 
2.52.0

[PATCH rdma-next v2 13/15] RDMA/uverbs: Add new Unordered MR access flag

[Michael Gur]

Add a new Unordered access flag for MR registrations.
This access flag allows additional relaxing of the order of memory
accesses to the memory region.
Using this access flag allows for concurrent reads and writes of this MR
to be reordered.

This access flag is part of the optional access flags range, drivers
shouldn't fail registration for any value in this range and are expected
to treat them as best-effort.

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/core/umem.c          | 2 +-
 include/rdma/ib_verbs.h                 | 1 +
 include/uapi/rdma/ib_user_ioctl_verbs.h | 1 +
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index b1ab4f0359b0..e07c5d439fbe 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -211,7 +211,7 @@ static struct ib_umem *__ib_umem_get_va(struct ib_device *device,
 	umem->dma_dir    = dir;
 	umem->owning_mm = mm = current->mm;
 	umem->dma_attrs = DMA_ATTR_REQUIRE_COHERENT;
-	if (access & IB_ACCESS_RELAXED_ORDERING)
+	if (access & (IB_ACCESS_RELAXED_ORDERING | IB_ACCESS_UNORDERED))
 		umem->dma_attrs |= DMA_ATTR_WEAK_ORDERING;
 
 	mmgrab(mm);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d6eaa3fee66e..29228226a236 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1505,6 +1505,7 @@ enum ib_access_flags {
 	IB_ACCESS_ON_DEMAND = IB_UVERBS_ACCESS_ON_DEMAND,
 	IB_ACCESS_HUGETLB = IB_UVERBS_ACCESS_HUGETLB,
 	IB_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_RELAXED_ORDERING,
+	IB_ACCESS_UNORDERED = IB_UVERBS_ACCESS_UNORDERED,
 	IB_ACCESS_FLUSH_GLOBAL = IB_UVERBS_ACCESS_FLUSH_GLOBAL,
 	IB_ACCESS_FLUSH_PERSISTENT = IB_UVERBS_ACCESS_FLUSH_PERSISTENT,
 
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f..ca8b8b8d28f2 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -62,6 +62,7 @@ enum ib_uverbs_access_flags {
 	IB_UVERBS_ACCESS_FLUSH_PERSISTENT = 1 << 9,
 
 	IB_UVERBS_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_OPTIONAL_FIRST,
+	IB_UVERBS_ACCESS_UNORDERED = IB_UVERBS_ACCESS_OPTIONAL_FIRST << 1,
 	IB_UVERBS_ACCESS_OPTIONAL_RANGE =
 		((IB_UVERBS_ACCESS_OPTIONAL_LAST << 1) - 1) &
 		~(IB_UVERBS_ACCESS_OPTIONAL_FIRST - 1)
-- 
2.52.0

[PATCH rdma-next v2 14/15] RDMA/mlx5: Support new Unordered access flag

[Michael Gur]

Add mlx5 driver support for IB_ACCESS_UNORDERED, the new uverbs access
flag that enables relaxing the order of read after write on mkeys.

Map IB_ACCESS_UNORDERED to the Unordered mkey ordering level of the
mlx5 core helper, which sets relaxed ordering and, when supported,
relaxed read-after-write ordering.

Extend the current enablement of ATS over DMABUF with RO access to also
enable for DMABUF with Unordered access.

Mark IB_ACCESS_UNORDERED as non-UMR-modifiable: there is no UMR
mkey-mask bit for order_read_after_write, so UMR cannot drive it.

Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 4 +++-
 drivers/infiniband/hw/mlx5/umr.h     | 2 +-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index dd1fe8fd4220..75ea3f91e931 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -1672,11 +1672,13 @@ static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev,
 {
 	if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf)
 		return false;
-	return access_flags & IB_ACCESS_RELAXED_ORDERING;
+	return access_flags & (IB_ACCESS_RELAXED_ORDERING | IB_ACCESS_UNORDERED);
 }
 
 static inline enum mlx5_mkey_order mlx5r_access_to_mkey_order(int access_flags)
 {
+	if (access_flags & IB_ACCESS_UNORDERED)
+		return MLX5_MKEY_ORDER_UNORDERED;
 	if (access_flags & IB_ACCESS_RELAXED_ORDERING)
 		return MLX5_MKEY_ORDER_RELAXED;
 	return MLX5_MKEY_ORDER_STRICT;
diff --git a/drivers/infiniband/hw/mlx5/umr.h b/drivers/infiniband/hw/mlx5/umr.h
index a6c4f35b75e6..78b91d614a17 100644
--- a/drivers/infiniband/hw/mlx5/umr.h
+++ b/drivers/infiniband/hw/mlx5/umr.h
@@ -49,7 +49,7 @@ static inline unsigned int
 mlx5r_umr_get_unchangeable_access_flags(struct mlx5_ib_dev *dev,
 					unsigned int access_flags)
 {
-	unsigned int ret = 0;
+	unsigned int ret = access_flags & IB_ACCESS_UNORDERED;
 
 	if ((access_flags & IB_ACCESS_REMOTE_ATOMIC) &&
 	    MLX5_CAP_GEN(dev->mdev, atomic) &&
-- 
2.52.0

[PATCH rdma-next v2 15/15] RDMA/mlx5: Enforce relaxed ordering when HW requires it

[Michael Gur]

Add relaxed ordering validation (relaxed or unordered) to enforce no
strong ordered mkeys are created when the HW disallows it.
Reject MR registrations that do not request IB_ACCESS_RELAXED_ORDERING
or IB_ACCESS_UNORDERED.

Invoke the validator from all user-facing MR registration paths,
including the Device Memory path (mlx5_ib_reg_dm_mr), and from the
FRMR pool build_key callback, so that an admin request to pin a pool
with strong ordered access fails up front rather than on mkey creation.

Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 34 ++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 6848515e06d7..7952d80349dc 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -78,6 +78,16 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
 	MLX5_SET64(mkc, mkc, start_addr, start_addr);
 }
 
+static int validate_ordering_access(struct mlx5_ib_dev *dev, int acc)
+{
+	/* If HW disallows Strong Ordered writes, RO/UNORDERED must be set */
+	if (MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro_only)) {
+		if (!(acc & (IB_ACCESS_RELAXED_ORDERING | IB_ACCESS_UNORDERED)))
+			return -EOPNOTSUPP;
+	}
+	return 0;
+}
+
 static void assign_mkey_variant(struct mlx5_ib_dev *dev, u32 *mkey, u32 *in)
 {
 	u8 key = atomic_inc_return(&dev->mkey_var);
@@ -288,6 +298,7 @@ static int mlx5r_build_frmr_key(struct ib_device *device,
 				struct ib_frmr_key *out)
 {
 	struct mlx5_ib_dev *dev = to_mdev(device);
+	int err;
 
 	/* check HW capabilities of users requested frmr key */
 	if ((in->ats && !MLX5_CAP_GEN(dev->mdev, ats)) ||
@@ -297,6 +308,10 @@ static int mlx5r_build_frmr_key(struct ib_device *device,
 	if (in->vendor_key & ~MLX5_FRMR_POOLS_KEY_VENDOR_KEY_SUPPORTED)
 		return -EOPNOTSUPP;
 
+	err = validate_ordering_access(dev, in->access_flags);
+	if (err)
+		return err;
+
 	out->ats = in->ats;
 	out->access_flags =
 		mlx5r_umr_get_unchangeable_access_flags(dev, in->access_flags);
@@ -673,8 +688,13 @@ struct ib_mr *mlx5_ib_reg_dm_mr(struct ib_pd *pd, struct ib_dm *dm,
 {
 	struct mlx5_ib_dm *mdm = to_mdm(dm);
 	struct mlx5_core_dev *dev = to_mdev(dm->device)->mdev;
+	struct mlx5_ib_dev *ib_dev = to_mdev(dm->device);
 	u64 start_addr = mdm->dev_addr + attr->offset;
-	int mode;
+	int mode, err;
+
+	err = validate_ordering_access(ib_dev, attr->access_flags);
+	if (err)
+		return ERR_PTR(err);
 
 	switch (mdm->type) {
 	case MLX5_IB_UAPI_DM_TYPE_MEMIC:
@@ -837,6 +857,10 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
 	mlx5_ib_dbg(dev, "start 0x%llx, iova 0x%llx, length 0x%llx, access_flags 0x%x\n",
 		    start, iova, length, access_flags);
 
+	err = validate_ordering_access(dev, access_flags);
+	if (err)
+		return ERR_PTR(err);
+
 	err = mlx5r_umr_resource_init(dev);
 	if (err)
 		return ERR_PTR(err);
@@ -1028,6 +1052,10 @@ struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
 		    "offset 0x%llx, virt_addr 0x%llx, length 0x%llx, fd %d, access_flags 0x%x, mlx5_access_flags 0x%x\n",
 		    offset, virt_addr, length, fd, access_flags, mlx5_access_flags);
 
+	err = validate_ordering_access(dev, access_flags);
+	if (err)
+		return ERR_PTR(err);
+
 	/* dmabuf requires xlt update via umr to work. */
 	if (!mlx5r_umr_can_load_pas(dev, length))
 		return ERR_PTR(-EINVAL);
@@ -1161,6 +1189,10 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start,
 	if (!(flags & IB_MR_REREG_PD))
 		new_pd = ib_mr->pd;
 
+	err = validate_ordering_access(dev, new_access_flags);
+	if (err)
+		return ERR_PTR(err);
+
 	if (mr->is_odp_implicit && !(flags & IB_MR_REREG_TRANS)) {
 		if (!(new_access_flags & IB_ACCESS_ON_DEMAND))
 			return ERR_PTR(-EOPNOTSUPP);
-- 
2.52.0

Re: [PATCH rdma-next v2 01/15] RDMA/mlx5: Allow optional access flags in DM registration

[sashiko-bot]

> The uverbs ABI reserves the optional access flags range as best-effort:
> drivers should accept any flag in this range and treat the actual support
> as opportunistic, never failing registration when an optional flag is
> requested.
> 
> Add the optional access flag to the allow-lists of DM registrations.
> Uverbs users can now request any optional access flag on DM registrations
> and mlx5 will apply the flag's effect on a best-effort basis when
> supported.
> 
> Fixes: 68d384b906cf ("RDMA/core: Add optional access flags range")
> Cc: stable@vger.kernel.org
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=1

Re: [PATCH rdma-next v2 02/15] RDMA/mlx5: Allow optional access flags in DEVX UMEM registration

[sashiko-bot]

> The uverbs ABI reserves the optional access flags range as best-effort:
> drivers should accept any flag in this range and treat the actual support
> as opportunistic, never failing registration when an optional flag is
> requested.
> 
> Add the optional access flag to the allow-list of DEVX UMEM registration.
> Uverbs users can now request any optional access flag on these
> registrations and mlx5 will apply the flag's effect on a best-effort
> basis when supported.
> 
> Fixes: 72b2f7608a59 ("RDMA/mlx5: Enable ATS support for MRs and umems")
> Cc: stable@vger.kernel.org
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=2

Re: [PATCH rdma-next v2 03/15] RDMA/core: Allow optional access flags in dmabuf reg ioctl

[sashiko-bot]

> The uverbs ABI reserves the optional access flags range as best-effort:
> drivers should accept any flag in this range and treat the actual support
> as opportunistic, never failing registration when an optional flag is
> requested.
> 
> Add the optional access flag to the allow-list of dmabuf registration
> ioctl. Uverbs users can now request any optional access flag on dmabuf
> registrations and drivers will apply the flag's effect on a best-effort
> basis when supported.
> 
> Fixes: bfe0cc6eb249 ("RDMA/uverbs: Add uverbs command for dma-buf based MR registration")
> Cc: stable@vger.kernel.org
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=3

Re: [PATCH rdma-next v2 04/15] RDMA/core: Allow optional access flags in reg mr ioctl

[sashiko-bot]

> The uverbs ABI reserves the optional access flags range as best-effort:
> drivers should accept any flag in this range and treat the actual support
> as opportunistic, never failing registration when an optional flag is
> requested.
> 
> Add the optional access flag to the allow-list of dmabuf registration
> through the reg_mr ioctl. Uverbs users can now request any optional access
> flag on dmabuf registrations and mlx5 will apply the flag's effect on a
> best-effort basis when supported.
> 
> Fixes: 5b2e45049dc0 ("IB/core: Add UVERBS_METHOD_REG_MR on the MR object")
> Cc: stable@vger.kernel.org
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=4

Re: [PATCH rdma-next v2 05/15] mlx5: Rename IFC bits of relaxed ordering fields

[sashiko-bot]

> Align the IFC field names with the new naming:
> - relaxed_ordering_write_umr -> order_write_after_write_umr (cap)
> - relaxed_ordering_read_umr -> pci_relaxed_ordered_read_umr (cap)
> - relaxed_ordering_write -> mkc_order_write_after_write_ro (cap)
> - relaxed_ordering_write -> order_write_after_write (mkc)
> - relaxed_ordering_read -> pci_relaxed_ordered_read (cap and mkc)
> 
> Update all references across drivers.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=5

Re: [PATCH rdma-next v2 06/15] net/mlx5: Add IFC bits for new ordering caps

[sashiko-bot]

> Add new HCA capability bits:
> - mkc_order_read_after_write
> - mkc_order_write_after_write_ro_only
> 
> Add new mkey context field:
> - order_read_after_write (2-bit)
> 
> These fields enable a new level of memory access order relaxation,
> allowing read-after-write reordering and allow to expose an indication
> that devices don't support strict ordering of memory access.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=6

Re: [PATCH rdma-next v2 07/15] mlx5: Consolidate mkey ordering setting into a single core helper

[sashiko-bot]

> Consolidate the mkey ordering setting of all mlx5 consumers into a
> single exported core helper. The helper takes the requested ordering
> level (strict, relaxed or unordered) and sets the mkey ordering fields
> according to the device caps, so consumers no longer open-code the cap
> checks. The unordered level is used later in this series.
> 
> Switch the RDMA MR creation and UMR access-modify paths to the helper.
> As before, strict ordering leaves the mkey context untouched.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=7

Re: [PATCH rdma-next v2 08/15] net/mlx5: Enable relaxed ordering on resource dump mkey

[sashiko-bot]

> As resource dump logic reads data only following an event that is
> delivered with strong ordering, there is no need for strong ordering of
> its memory accesses.
> Add relaxed ordering access flag to its registered memory, thus allowing
> better performance and allowing resource dump on HW that restricts
> strong ordering.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=8

Re: [PATCH rdma-next v2 09/15] net/mlx5: Enable relaxed ordering on FW tracer mkey

[sashiko-bot]

> On HW that restricts strong ordering, mkey creation fails unless relaxed
> ordering is set, which prevents the FW tracer from being enabled.
> 
> Set relaxed ordering on the FW tracer mkey only on such HW, to work
> around this limitation. This has no effect on the ordering of the trace
> data, as once the mkey is configured as the tracer output buffer, FW and
> HW write to it with strong ordering regardless of the mkey ordering
> attributes. Strong ordering is required by design, as the driver polls
> the buffer and considers a block valid once its last bytes hold a valid
> timestamp. Out of order writes could expose a partially written block
> to the driver.
> 
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=9

Re: [PATCH rdma-next v2 10/15] vfio/mlx5: Enable relaxed ordering on the live migration data mkey

[sashiko-bot]

> As firmware guarantees that all device-issued data writes complete before
> posting of CQE, there is no need for strong ordering of memory accesses
> on the live migration data mkey.
> Enable relaxed ordering on the mkey via the mlx5_core RO setter.
> 
> This allows better performance and supporting vfio live migration on HW
> that restricts memory registration with strict ordering.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=10

Re: [PATCH rdma-next v2 11/15] RDMA/mlx5: Enable relaxed ordering on ODP null mkey

[sashiko-bot]

> Enable relaxed ordering on the ODP implicit-MR null mkey.
> This allows supporting implicit ODP on HW that restricts memory
> registration with strict ordering.
> 
> No functional change as this is a null mkey with no memory mappings.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=11

Re: [PATCH rdma-next v2 12/15] RDMA/mlx5: Converge UMR access-flag cap checks

[sashiko-bot]

> Move the UMR cap check from both MR registration and UMR paths to a
> single helper checking the access flags non-modifiable by UMR.
> 
> No behavior change for the current flag set.
> Converge to prevent code duplication when adding support to new access
> flags that are not modifiable by UMR, like the new Unordered access flag
> added in following patches.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=12

Re: [PATCH rdma-next v2 13/15] RDMA/uverbs: Add new Unordered MR access flag

[sashiko-bot]

> Add a new Unordered access flag for MR registrations.
> This access flag allows additional relaxing of the order of memory
> accesses to the memory region.
> Using this access flag allows for concurrent reads and writes of this MR
> to be reordered.
> 
> This access flag is part of the optional access flags range, drivers
> shouldn't fail registration for any value in this range and are expected
> to treat them as best-effort.
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
> Signed-off-by: Michael Gur <michaelgur@nvidia.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=13

Re: [PATCH rdma-next v2 14/15] RDMA/mlx5: Support new Unordered access flag

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] RDMA/mlx5: missing IB_ACCESS_UNORDERED in _mlx5r_umr_update_mr_pas() for DD
--

commit d54e400e3350128fbcd9fb5d5bce4f37dfe241bd
Author: Michael Gur <michaelgur@nvidia.com>

RDMA/mlx5: Support new Unordered access flag

This commit adds mlx5_ib support for the IB_ACCESS_UNORDERED flag by mapping
it to MLX5_MKEY_ORDER_UNORDERED, enabling ATS on DMABUF MRs with unordered
access, and marking the flag as unchangeable via UMR.

> diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
> index dd1fe8fd42206..75ea3f91e9314 100644
> --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
> +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
> @@ -1672,11 +1672,13 @@ static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev,
>  {
>  	if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf)
>  		return false;
> -	return access_flags & IB_ACCESS_RELAXED_ORDERING;
> +	return access_flags & (IB_ACCESS_RELAXED_ORDERING | IB_ACCESS_UNORDERED);
>  }
>  
>  static inline enum mlx5_mkey_order mlx5r_access_to_mkey_order(int access_flags)
>  {
> +	if (access_flags & IB_ACCESS_UNORDERED)
> +		return MLX5_MKEY_ORDER_UNORDERED;
>  	if (access_flags & IB_ACCESS_RELAXED_ORDERING)
>  		return MLX5_MKEY_ORDER_RELAXED;
>  	return MLX5_MKEY_ORDER_STRICT;

[Severity: Medium]
As reported by Sashiko on v1 [1], when a Data-Direct DMABUF MR is registered
with IB_ACCESS_UNORDERED via mlx5_ib_reg_user_mr_dmabuf() ->
reg_user_mr_dmabuf_by_data_direct(), both crossed_mr and crossing_mr are
configured with MLX5_MKEY_ORDER_UNORDERED and ATS via
mlx5r_access_to_mkey_order() and mlx5_umem_needs_ats().

However, when populating the KSM entries in
drivers/infiniband/hw/mlx5/umr.c:_mlx5r_umr_update_mr_pas(), only
IB_ACCESS_RELAXED_ORDERING is checked when selecting between mdd->mkey_ro and
mdd->mkey:

		if (dd) {
			struct mlx5_data_direct *mdd = dev->mdev->data_direct;

			cur_ksm->va = cpu_to_be64(rdma_block_iter_dma_address(&biter));
			if (mr->access_flags & IB_ACCESS_RELAXED_ORDERING &&
			    mdd->mkey_ro_valid)
				cur_ksm->key = cpu_to_be32(mdd->mkey_ro);
			else
				cur_ksm->key = cpu_to_be32(mdd->mkey);

Similarly, in
drivers/infiniband/hw/mlx5/mr.c:reg_user_mr_dmabuf_by_data_direct(), when
mkey_ro_valid is false, only IB_ACCESS_RELAXED_ORDERING is masked out of
access_flags:

	if (!dev->mdev->data_direct->mkey_ro_valid)
		access_flags &= ~IB_ACCESS_RELAXED_ORDERING;

Should _mlx5r_umr_update_mr_pas() and reg_user_mr_dmabuf_by_data_direct()
also handle IB_ACCESS_UNORDERED so that Data-Direct KSM entries and
crossed_mr/crossing_mr ordering match the underlying Data-Direct mkey?

[1] https://lore.kernel.org/r/20260726092943.2880176-15-michaelgur@nvidia.com

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=14

Re: [PATCH rdma-next v2 15/15] RDMA/mlx5: Enforce relaxed ordering when HW requires it

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] RDMA/mlx5: stripped RO flag in FRMR key breaks mlx5r_create_mkeys()
- [High] RDMA/mlx5: ordering check bypass in reg_user_mr_dmabuf_by_data_direct()
--

commit 939d1d09c02a86f15171e7a38c5bf57aaefbee3f
Author: Michael Gur <michaelgur@nvidia.com>

RDMA/mlx5: Enforce relaxed ordering when HW requires it

This commit adds validate_ordering_access() to reject MR registrations and
FRMR pool requests that do not specify IB_ACCESS_RELAXED_ORDERING or
IB_ACCESS_UNORDERED when the hardware sets
mkc_order_write_after_write_ro_only.

> diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
> index 6848515e06d77..7952d80349dc6 100644
> --- a/drivers/infiniband/hw/mlx5/mr.c
> +++ b/drivers/infiniband/hw/mlx5/mr.c
> @@ -78,6 +78,16 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
>  	MLX5_SET64(mkc, mkc, start_addr, start_addr);
>  }
>  
> +static int validate_ordering_access(struct mlx5_ib_dev *dev, int acc)
> +{
> +	/* If HW disallows Strong Ordered writes, RO/UNORDERED must be set */
> +	if (MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro_only)) {
> +		if (!(acc & (IB_ACCESS_RELAXED_ORDERING | IB_ACCESS_UNORDERED)))
> +			return -EOPNOTSUPP;
> +	}
> +	return 0;
> +}
> +
>  static void assign_mkey_variant(struct mlx5_ib_dev *dev, u32 *mkey, u32 *in)
>  {
>  	u8 key = atomic_inc_return(&dev->mkey_var);
> @@ -288,6 +298,7 @@ static int mlx5r_build_frmr_key(struct ib_device *device,
>  				struct ib_frmr_key *out)
>  {
>  	struct mlx5_ib_dev *dev = to_mdev(device);
> +	int err;
>  
>  	/* check HW capabilities of users requested frmr key */
>  	if ((in->ats && !MLX5_CAP_GEN(dev->mdev, ats)) ||
> @@ -297,6 +308,10 @@ static int mlx5r_build_frmr_key(struct ib_device *device,
>  	if (in->vendor_key & ~MLX5_FRMR_POOLS_KEY_VENDOR_KEY_SUPPORTED)
>  		return -EOPNOTSUPP;
>  
> +	err = validate_ordering_access(dev, in->access_flags);
> +	if (err)
> +		return err;
> +
>  	out->ats = in->ats;
>  	out->access_flags =
>  		mlx5r_umr_get_unchangeable_access_flags(dev, in->access_flags);

[Severity: High]
Can this cause mkey creation to fail when mkc_order_write_after_write_ro_only
is set and the device supports modifying relaxed ordering via UMR?

In mlx5r_build_frmr_key() (as well as _mlx5_frmr_pool_alloc() and
mlx5_mr_cache_alloc()), out->access_flags is filtered through
mlx5r_umr_get_unchangeable_access_flags():

drivers/infiniband/hw/mlx5/umr.h:mlx5r_umr_get_unchangeable_access_flags() {
	unsigned int ret = access_flags & IB_ACCESS_UNORDERED;
	...
	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
	    MLX5_CAP_GEN(dev->mdev, mkc_order_write_after_write_ro) &&
	    !MLX5_CAP_GEN(dev->mdev, order_write_after_write_umr))
		ret |= IB_ACCESS_RELAXED_ORDERING;

	if ((access_flags & IB_ACCESS_RELAXED_ORDERING) &&
	    (MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read) ||
	     MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read_pci_enabled)) &&
	    !MLX5_CAP_GEN(dev->mdev, pci_relaxed_ordered_read_umr))
		ret |= IB_ACCESS_RELAXED_ORDERING;

	return ret;
}

When order_write_after_write_umr and pci_relaxed_ordered_read_umr are
supported, IB_ACCESS_RELAXED_ORDERING is stripped from key.access_flags.

When ib_frmr_pools_set_pinned() or ib_frmr_pool_pop() later calls
mlx5r_create_mkeys(), it passes that stripped key->access_flags to
set_mkc_access_pd_addr_fields():

drivers/infiniband/hw/mlx5/mr.c:mlx5r_create_mkeys() {
	...
	set_mkc_access_pd_addr_fields(mkc, key->access_flags, 0, dev->umrc.pd);
	MLX5_SET(mkc, mkc, free, 1);
	MLX5_SET(mkc, mkc, umr_en, 1);
	...
}

Because neither IB_ACCESS_RELAXED_ORDERING nor IB_ACCESS_UNORDERED remains in
key->access_flags, mlx5r_access_to_mkey_order(key->access_flags) returns
MLX5_MKEY_ORDER_STRICT, leaving order_write_after_write as 0 in mkc on
hardware that requires relaxed ordering.

In addition, because the stored pool->key.access_flags has
IB_ACCESS_RELAXED_ORDERING stripped, querying a pool via
RDMA_NLDEV_CMD_FRMR_POOLS_GET and passing that key back to
RDMA_NLDEV_CMD_FRMR_POOLS_SET will fail validate_ordering_access() in
mlx5r_build_frmr_key() with -EOPNOTSUPP.

Should mlx5r_create_mkeys() ensure free mkeys are created with relaxed
ordering (similar to how mlx5_set_umr_free_mkey() passes
IB_ACCESS_RELAXED_ORDERING), or should mlx5r_umr_get_unchangeable_access_flags()
and mlx5r_build_frmr_key() account for mkc_order_write_after_write_ro_only?

[ ... ]

> @@ -1028,6 +1052,10 @@ struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
>  		    "offset 0x%llx, virt_addr 0x%llx, length 0x%llx, fd %d, access_flags 0x%x, mlx5_access_flags 0x%x\n",
>  		    offset, virt_addr, length, fd, access_flags, mlx5_access_flags);
>  
> +	err = validate_ordering_access(dev, access_flags);
> +	if (err)
> +		return ERR_PTR(err);
> +
>  	/* dmabuf requires xlt update via umr to work. */
>  	if (!mlx5r_umr_can_load_pas(dev, length))
>  		return ERR_PTR(-EINVAL);

[Severity: High]
As reported by Sashiko on v1 [1], if MLX5_IB_UAPI_REG_DMABUF_ACCESS_DATA_DIRECT
is set in mlx5_ib_reg_user_mr_dmabuf(), doesn't
reg_user_mr_dmabuf_by_data_direct() bypass this ordering check when
mkey_ro_valid is false?

In mlx5_ib_reg_user_mr_dmabuf(), validate_ordering_access() checks
access_flags before calling reg_user_mr_dmabuf_by_data_direct():

drivers/infiniband/hw/mlx5/mr.c:reg_user_mr_dmabuf_by_data_direct() {
	...
	if (!dev->mdev->data_direct->mkey_ro_valid)
		access_flags &= ~IB_ACCESS_RELAXED_ORDERING;
	crossed_mr = reg_user_mr_dmabuf(pd, &data_direct_dev->pdev->dev,
					offset, length, virt_addr, fd,
					access_flags, MLX5_MKC_ACCESS_MODE_KSM,
					NULL);
	...
	crossing_mr = reg_create_crossing_vhca_mr(pd, virt_addr, length,
						  crossed_mr->ibmr.lkey,
						  access_flags);
	...
}

If the user passes IB_ACCESS_RELAXED_ORDERING and mkey_ro_valid is false,
IB_ACCESS_RELAXED_ORDERING is cleared after validate_ordering_access() has
already succeeded, so reg_user_mr_dmabuf() and reg_create_crossing_vhca_mr()
proceed with strong-ordered access_flags.

Furthermore, if the user passes IB_ACCESS_UNORDERED (without
IB_ACCESS_RELAXED_ORDERING), validate_ordering_access() succeeds, but
_mlx5r_umr_update_mr_pas() only checks IB_ACCESS_RELAXED_ORDERING when
selecting between mdd->mkey_ro and the strong-ordered mdd->mkey:

drivers/infiniband/hw/mlx5/umr.c:_mlx5r_umr_update_mr_pas() {
	...
			if (mr->access_flags & IB_ACCESS_RELAXED_ORDERING &&
			    mdd->mkey_ro_valid)
				cur_ksm->key = cpu_to_be32(mdd->mkey_ro);
			else
				cur_ksm->key = cpu_to_be32(mdd->mkey);
	...
}

Should validate_ordering_access() also be checked after adjusting
access_flags in reg_user_mr_dmabuf_by_data_direct(), and should the Data
Direct path also handle IB_ACCESS_UNORDERED consistently with
IB_ACCESS_RELAXED_ORDERING?

[1] https://lore.kernel.org/r/20260726092943.2880176-16-michaelgur@nvidia.com

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102223325.2115128-1-michaelgur@nvidia.com?part=15