[PATCH rdma-rc] RDMA/mlx5: Fix DevX umem page size truncation for pages >= 4G

[PATCH rdma-rc] RDMA/mlx5: Fix DevX umem page size truncation for pages >= 4G

[0z5a]

From: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>

devx_umem_find_best_pgsize() computes the page size in an unsigned long
but returns it as unsigned int, and devx_umem_reg_cmd_alloc() stores it
in an unsigned int. When the DMA mapping of the umem is contiguous and
aligned to 4 GiB or more, and the umem length is a multiple of that
size, ib_umem_find_best_pgoff() selects a page size >= 2^32. Since the
value is a power of two, the truncation turns it into 0 and the
registration fails with -EINVAL.

The umem log_page_size field is 5 bits wide on top of
MLX5_ADAPTER_PAGE_SHIFT, so the command can encode these page sizes.

Seen on two GH200 nodes (ConnectX-7, firmware 32.38.3056 and
32.43.1014) with a dma-buf exported from cudaMalloc memory: registering
4 GiB or 8 GiB umems at 4 GiB-aligned offsets of the dma-buf fails with
EINVAL every time, while 2 GiB umems at the same offsets and
ibv_reg_dmabuf_mr() of the same ranges succeed. Restricting the user
pgsz_bitmap to <= 2 GiB makes all of them succeed.

Other large physically contiguous buffers, such as a 16 GiB hugetlb
page on an arm64 kernel with 64K base pages, should hit the same path.

Use unsigned long for the page size, as commit f0c2427412b4
("RDMA/mlx5: Fix page_size variable overflow") already did for the MR
paths.

The bug was found by an AI coding assistant (Claude Opus 5.5, Anthropic)
during an audit of 32-bit truncations of page sizes and offsets in
drivers/infiniband/core and drivers/infiniband/hw/mlx5. The assistant
also wrote the reproducer and this patch. The reproducer was run on the
GH200 nodes described above.

Fixes: 7610ab57de56 ("RDMA/mlx5: Allow larger pages in DevX umem")
Cc: stable@vger.kernel.org
Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
---

Notes:
    Testing notes:
    - The failures above were reproduced with the DOCA-OFED 26.01 mlx5_ib,
      whose devx_umem_find_best_pgsize() is identical to upstream; I do not
      have root on these hosts, so I could not load a patched module.
    - So I have not verified that CREATE_UMEM accepts log_page_size values
      for pages >= 4 GiB. If it does not, capping pgsz_bitmap at 2^31 in
      devx_umem_find_best_pgsize() is the alternative fix.
    - Compile-tested on v7.4-rc1 (arm64 defconfig + MLX5_INFINIBAND, W=1).
    - A standalone reproducer (dma-buf from cuMemGetHandleForAddressRange()
      plus mlx5dv_devx_umem_reg_ex(), sweeping offsets) is available on
      request.

 drivers/infiniband/hw/mlx5/devx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index fc6f793dc..2277160d9 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -2294,8 +2294,8 @@ static int devx_umem_get(struct mlx5_ib_dev *dev, struct ib_ucontext *ucontext,
 	return 0;
 }
 
-static unsigned int devx_umem_find_best_pgsize(struct ib_umem *umem,
-					       unsigned long pgsz_bitmap)
+static unsigned long devx_umem_find_best_pgsize(struct ib_umem *umem,
+						unsigned long pgsz_bitmap)
 {
 	unsigned long page_size;
 
@@ -2333,7 +2333,7 @@ static int devx_umem_reg_cmd_alloc(struct mlx5_ib_dev *dev,
 				   int access)
 {
 	unsigned long pgsz_bitmap;
-	unsigned int page_size;
+	unsigned long page_size;
 	__be64 *mtt;
 	void *umem;
 	int ret;
-- 
2.43.0

[PATCH rdma-rc] RDMA/mlx5: Fix DevX umem page size truncation for pages >= 4G

[0z5a]

From: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>

devx_umem_find_best_pgsize() computes the page size in an unsigned long
but returns it as unsigned int, and devx_umem_reg_cmd_alloc() stores it
in an unsigned int. When the DMA mapping of the umem is contiguous and
aligned to 4 GiB or more, and the umem length is a multiple of that
size, ib_umem_find_best_pgoff() selects a page size >= 2^32. Since the
value is a power of two, the truncation turns it into 0 and the
registration fails with -EINVAL.

The umem log_page_size field is 5 bits wide on top of
MLX5_ADAPTER_PAGE_SHIFT, so the command can encode these page sizes.

Seen on two GH200 nodes (ConnectX-7, firmware 32.38.3056 and
32.43.1014) with a dma-buf exported from cudaMalloc memory: registering
4 GiB or 8 GiB umems at 4 GiB-aligned offsets of the dma-buf fails with
EINVAL every time, while 2 GiB umems at the same offsets and
ibv_reg_dmabuf_mr() of the same ranges succeed. Restricting the user
pgsz_bitmap to <= 2 GiB makes all of them succeed.

Other large physically contiguous buffers, such as a 16 GiB hugetlb
page on an arm64 kernel with 64K base pages, should hit the same path.

Use unsigned long for the page size, as commit f0c2427412b4
("RDMA/mlx5: Fix page_size variable overflow") already did for the MR
paths.

The bug was found by an AI coding assistant (Claude Opus 5.5, Anthropic)
during an audit of 32-bit truncations of page sizes and offsets in
drivers/infiniband/core and drivers/infiniband/hw/mlx5. The assistant
also wrote the reproducer and this patch. The reproducer was run on the
GH200 nodes described above.

Fixes: 7610ab57de56 ("RDMA/mlx5: Allow larger pages in DevX umem")
Cc: stable@vger.kernel.org
Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
---

Notes:
    Testing notes:
    - The failures above were reproduced with the DOCA-OFED 26.01 mlx5_ib,
      whose devx_umem_find_best_pgsize() is identical to upstream; I do not
      have root on these hosts, so I could not load a patched module.
    - So I have not verified that CREATE_UMEM accepts log_page_size values
      for pages >= 4 GiB. If it does not, capping pgsz_bitmap at 2^31 in
      devx_umem_find_best_pgsize() is the alternative fix.
    - Compile-tested on v7.4-rc1 (arm64 defconfig + MLX5_INFINIBAND, W=1).
    - A standalone reproducer (dma-buf from cuMemGetHandleForAddressRange()
      plus mlx5dv_devx_umem_reg_ex(), sweeping offsets) is available on
      request.

 drivers/infiniband/hw/mlx5/devx.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c
index fc6f793dc..2277160d9 100644
--- a/drivers/infiniband/hw/mlx5/devx.c
+++ b/drivers/infiniband/hw/mlx5/devx.c
@@ -2294,8 +2294,8 @@ static int devx_umem_get(struct mlx5_ib_dev *dev, struct ib_ucontext *ucontext,
 	return 0;
 }
 
-static unsigned int devx_umem_find_best_pgsize(struct ib_umem *umem,
-					       unsigned long pgsz_bitmap)
+static unsigned long devx_umem_find_best_pgsize(struct ib_umem *umem,
+						unsigned long pgsz_bitmap)
 {
 	unsigned long page_size;
 
@@ -2333,7 +2333,7 @@ static int devx_umem_reg_cmd_alloc(struct mlx5_ib_dev *dev,
 				   int access)
 {
 	unsigned long pgsz_bitmap;
-	unsigned int page_size;
+	unsigned long page_size;
 	__be64 *mtt;
 	void *umem;
 	int ret;
-- 
2.43.0

Re: [PATCH rdma-rc] RDMA/mlx5: Fix DevX umem page size truncation for pages >= 4G

[sashiko-bot]

> From: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
> 
> devx_umem_find_best_pgsize() computes the page size in an unsigned long
> but returns it as unsigned int, and devx_umem_reg_cmd_alloc() stores it
> in an unsigned int. When the DMA mapping of the umem is contiguous and
> aligned to 4 GiB or more, and the umem length is a multiple of that
> size, ib_umem_find_best_pgoff() selects a page size >= 2^32. Since the
> value is a power of two, the truncation turns it into 0 and the
> registration fails with -EINVAL.
> 
> The umem log_page_size field is 5 bits wide on top of
> MLX5_ADAPTER_PAGE_SHIFT, so the command can encode these page sizes.
> 
> Seen on two GH200 nodes (ConnectX-7, firmware 32.38.3056 and
> 32.43.1014) with a dma-buf exported from cudaMalloc memory: registering
> [ ... ]
> GH200 nodes described above.
> 
> Fixes: 7610ab57de56 ("RDMA/mlx5: Allow larger pages in DevX umem")
> Cc: stable@vger.kernel.org
> Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103142709.66179-1-dezhen.lu@student.uni-tuebingen.de?part=1

Re: [PATCH rdma-rc] RDMA/mlx5: Fix DevX umem page size truncation for pages >= 4G

[sashiko-bot]

> From: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>
> 
> devx_umem_find_best_pgsize() computes the page size in an unsigned long
> but returns it as unsigned int, and devx_umem_reg_cmd_alloc() stores it
> in an unsigned int. When the DMA mapping of the umem is contiguous and
> aligned to 4 GiB or more, and the umem length is a multiple of that
> size, ib_umem_find_best_pgoff() selects a page size >= 2^32. Since the
> value is a power of two, the truncation turns it into 0 and the
> registration fails with -EINVAL.
> 
> The umem log_page_size field is 5 bits wide on top of
> MLX5_ADAPTER_PAGE_SHIFT, so the command can encode these page sizes.
> 
> Seen on two GH200 nodes (ConnectX-7, firmware 32.38.3056 and
> 32.43.1014) with a dma-buf exported from cudaMalloc memory: registering
> [ ... ]
> GH200 nodes described above.
> 
> Fixes: 7610ab57de56 ("RDMA/mlx5: Allow larger pages in DevX umem")
> Cc: stable@vger.kernel.org
> Signed-off-by: Dezhen Lu <dezhen.lu@student.uni-tuebingen.de>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103142804.67210-1-dezhen.lu@student.uni-tuebingen.de?part=1