From: Michael Gur <michaelgur@nvidia.com>
To: jgg@ziepe.ca, leon@kernel.org
Cc: linux-rdma@vger.kernel.org, netdev@vger.kernel.org,
Edward Srouji <edwards@nvidia.com>,
Yishai Hadas <yishaih@nvidia.com>,
Michael Gur <michaelgur@nvidia.com>,
Shameer Kolothum <skolothumtho@nvidia.com>,
Kevin Tian <kevin.tian@intel.com>,
Alex Williamson <alex@shazbot.org>,
kvm@vger.kernel.org (open list:VFIO MLX5 PCI DRIVER),
linux-kernel@vger.kernel.org (open list)
Subject: [PATCH rdma-next 09/15] vfio/mlx5: Enable relaxed ordering on the live migration data mkey
Date: Sun, 26 Jul 2026 12:29:23 +0300 [thread overview]
Message-ID: <20260726092943.2880176-10-michaelgur@nvidia.com> (raw)
In-Reply-To: <20260726092943.2880176-1-michaelgur@nvidia.com>
As firmware guarantees that all device-issued data writes complete before
posting of CQE, there is no need for strong ordering of memory accesses
on the live migration data mkey.
Enable relaxed ordering on the mkey via the mlx5_core RO setter.
This allows better performance and supporting vfio live migration on HW
that restricts memory registration with strict ordering.
Reviewed-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
---
drivers/vfio/pci/mlx5/cmd.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/vfio/pci/mlx5/cmd.c b/drivers/vfio/pci/mlx5/cmd.c
index 5fe0621b5fbd..f534c27597bc 100644
--- a/drivers/vfio/pci/mlx5/cmd.c
+++ b/drivers/vfio/pci/mlx5/cmd.c
@@ -320,7 +320,7 @@ static int mlx5vf_cmd_get_vhca_id(struct mlx5_core_dev *mdev, u16 function_id,
return ret;
}
-static u32 *alloc_mkey_in(u32 npages, u32 pdn)
+static u32 *alloc_mkey_in(struct mlx5_core_dev *mdev, u32 npages, u32 pdn)
{
int inlen;
void *mkc;
@@ -348,6 +348,7 @@ static u32 *alloc_mkey_in(u32 npages, u32 pdn)
MLX5_SET(mkc, mkc, log_page_size, PAGE_SHIFT);
MLX5_SET(mkc, mkc, translations_octword_size, DIV_ROUND_UP(npages, 2));
MLX5_SET64(mkc, mkc, len, npages * PAGE_SIZE);
+ mlx5_core_mkey_set_relaxed_ordering(mdev, mkc);
return in;
}
@@ -439,7 +440,7 @@ static int mlx5vf_dma_data_buffer(struct mlx5_vhca_data_buffer *buf)
if (buf->mkey_in || !buf->npages)
return -EINVAL;
- buf->mkey_in = alloc_mkey_in(buf->npages, buf->migf->pdn);
+ buf->mkey_in = alloc_mkey_in(mdev, buf->npages, buf->migf->pdn);
if (!buf->mkey_in)
return -ENOMEM;
@@ -1439,7 +1440,7 @@ static int mlx5vf_alloc_qp_recv_resources(struct mlx5_core_dev *mdev,
recv_buf->npages = npages;
- recv_buf->mkey_in = alloc_mkey_in(npages, pdn);
+ recv_buf->mkey_in = alloc_mkey_in(mdev, npages, pdn);
if (!recv_buf->mkey_in) {
err = -ENOMEM;
goto end;
--
2.52.0
next prev parent reply other threads:[~2026-07-26 9:30 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-07-26 9:29 [PATCH rdma-next 00/15] RDMA: Support HW requiring relaxed ordering and add Unordered access flag Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 01/15] RDMA/mlx5: Allow optional access flags in DM registration Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 02/15] RDMA/mlx5: Allow optional access flags in DEVX UMEM registration Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 03/15] RDMA/core: Allow optional access flags in dmabuf reg ioctl Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 04/15] RDMA/core: Allow optional access flags in reg mr ioctl Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 05/15] mlx5: Rename IFC bits of relaxed ordering fields Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 06/15] net/mlx5: Add IFC bits for new ordering caps Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 07/15] mlx5: Move RO setting helper to core and consolidate mlx5 consumers Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 08/15] net/mlx5: Enable relaxed ordering on resource dump mkey Michael Gur
2026-07-26 9:29 ` Michael Gur [this message]
2026-07-26 9:29 ` [PATCH rdma-next 10/15] RDMA/mlx5: Enable relaxed ordering on ODP null mkey Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 11/15] RDMA/mlx5: Enable relaxed ordering on Memory Window mkey on strict-RO HW Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 12/15] RDMA/mlx5: Converge UMR access-flag cap checks Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 13/15] RDMA/uverbs: Add new Unordered MR access flag Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 14/15] RDMA/mlx5: Support new Unordered " Michael Gur
2026-07-26 9:29 ` [PATCH rdma-next 15/15] RDMA/mlx5: Enforce relaxed ordering when HW requires it Michael Gur
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260726092943.2880176-10-michaelgur@nvidia.com \
--to=michaelgur@nvidia.com \
--cc=alex@shazbot.org \
--cc=edwards@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=skolothumtho@nvidia.com \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox