[PATCH][rdma-next] RDMA/mlx5: Use NUMA-aware allocation for memory region descriptors

[PATCH][rdma-next] RDMA/mlx5: Use NUMA-aware allocation for memory region descriptors

[lirongqing]

From: Li RongQing <lirongqing@baidu.com>

The mlx5_alloc_priv_descs() function currently uses kzalloc(), which
allocates memory on the NUMA node of the calling CPU. On multi-socket
systems, this can lead to cross-node memory access if the HCA is
attached to a different NUMA node than the process allocating the
Memory Region (MR).

Switch to kzalloc_node() and specify the node associated with the
HCA's DMA device. This ensures that the MR descriptors are resident on
the local NUMA node, reducing latency for hardware access.

Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
 drivers/infiniband/hw/mlx5/mr.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
index 665323b..5f4b4da 100644
--- a/drivers/infiniband/hw/mlx5/mr.c
+++ b/drivers/infiniband/hw/mlx5/mr.c
@@ -1990,7 +1990,8 @@ mlx5_alloc_priv_descs(struct ib_device *device,
 		add_size = min_t(int, end - size, add_size);
 	}
 
-	mr->descs_alloc = kzalloc(size + add_size, GFP_KERNEL);
+	mr->descs_alloc = kzalloc_node(size + add_size, GFP_KERNEL,
+			dev_to_node(ddev));
 	if (!mr->descs_alloc)
 		return -ENOMEM;
 
-- 
2.9.4

答复: [PATCH][rdma-next] RDMA/mlx5: Use NUMA-aware allocation for memory region descriptors

[Li,Rongqing(ACG CCN)]

> The mlx5_alloc_priv_descs() function currently uses kzalloc(), which allocates
> memory on the NUMA node of the calling CPU. On multi-socket systems, this
> can lead to cross-node memory access if the HCA is attached to a different
> NUMA node than the process allocating the Memory Region (MR).
> 
> Switch to kzalloc_node() and specify the node associated with the HCA's DMA
> device. This ensures that the MR descriptors are resident on the local NUMA
> node, reducing latency for hardware access.
> 
> Signed-off-by: Li RongQing <lirongqing@baidu.com>

ping

[Li,Rongqing] 



> ---
>  drivers/infiniband/hw/mlx5/mr.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
> index 665323b..5f4b4da 100644
> --- a/drivers/infiniband/hw/mlx5/mr.c
> +++ b/drivers/infiniband/hw/mlx5/mr.c
> @@ -1990,7 +1990,8 @@ mlx5_alloc_priv_descs(struct ib_device *device,
>  		add_size = min_t(int, end - size, add_size);
>  	}
> 
> -	mr->descs_alloc = kzalloc(size + add_size, GFP_KERNEL);
> +	mr->descs_alloc = kzalloc_node(size + add_size, GFP_KERNEL,
> +			dev_to_node(ddev));
>  	if (!mr->descs_alloc)
>  		return -ENOMEM;
> 
> --
> 2.9.4

Re: 答复: [PATCH][rdma-next] RDMA/mlx5: Use NUMA-aware allocation for memory region descriptors

[Leon Romanovsky]

On Fri, Mar 06, 2026 at 05:23:07AM +0000, Li,Rongqing(ACG CCN) wrote:
> 
> > The mlx5_alloc_priv_descs() function currently uses kzalloc(), which allocates
> > memory on the NUMA node of the calling CPU. On multi-socket systems, this
> > can lead to cross-node memory access if the HCA is attached to a different
> > NUMA node than the process allocating the Memory Region (MR).
> > 
> > Switch to kzalloc_node() and specify the node associated with the HCA's DMA
> > device. This ensures that the MR descriptors are resident on the local NUMA
> > node, reducing latency for hardware access.
> > 
> > Signed-off-by: Li RongQing <lirongqing@baidu.com>
> 
> ping
> 
> [Li,Rongqing] 

I have no plans to merge these patches in RDMA subsystem. It is unlikely
can give any benefit to the performance.

Thanks