* Re: [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
2026-10-06 14:16 [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node Seongjun Hong
@ 2026-10-06 14:22 ` sashiko-bot
2026-10-06 21:57 ` Jacob Keller
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: sashiko-bot @ 2026-10-06 14:22 UTC (permalink / raw)
To: Seongjun Hong; +Cc: linux-rdma
> The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> node hint, so they land on the node of whichever CPU happens to create
> or fill the pool, while the DMA pages they describe are allocated on
> the pool's node.
>
> The pool, page and bitmap are dereferenced on every block allocation
> and free. Allocate them on the pool's NUMA node as well, so that all
> of a pool's state lives on one node.
>
> These allocations happen on the control path, so the performance gain
> is expected to be small; the change is mainly for consistency.
>
> Signed-off-by: Seongjun Hong <hsj0512@snu.ac.kr>
Sashiko has reviewed this patch and found no issues. It looks great!
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261006-net-mlx5-numa-allocate-pool-page-v2-1-0dc4c2eb1376@snu.ac.kr?part=1
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
2026-10-06 14:16 [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node Seongjun Hong
2026-10-06 14:22 ` sashiko-bot
@ 2026-10-06 21:57 ` Jacob Keller
2026-10-07 1:56 ` Seongjun Hong
2026-10-11 6:58 ` Tariq Toukan
2026-10-12 14:50 ` patchwork-bot+netdevbpf
3 siblings, 1 reply; 6+ messages in thread
From: Jacob Keller @ 2026-10-06 21:57 UTC (permalink / raw)
To: Seongjun Hong, Saeed Mahameed, Leon Romanovsky, Tariq Toukan,
Mark Bloch, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-rdma, linux-kernel
On 10/6/2026 7:16 AM, Seongjun Hong wrote:
> The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> node hint, so they land on the node of whichever CPU happens to create
> or fill the pool, while the DMA pages they describe are allocated on
> the pool's node.
>
> The pool, page and bitmap are dereferenced on every block allocation
> and free. Allocate them on the pool's NUMA node as well, so that all
> of a pool's state lives on one node.
>
> These allocations happen on the control path, so the performance gain
> is expected to be small; the change is mainly for consistency.
>
> Signed-off-by: Seongjun Hong <hsj0512@snu.ac.kr>
> ---
> Changes in v2:
> - fix typo and specify the structs are allocated on the control path
> - add a prefix net-next to the subject
> - Link to v1: https://lore.kernel.org/r/20261005-net-mlx5-numa-allocate-pool-page-v1-1-01f4283932b4@snu.ac.kr
> ---
> drivers/net/ethernet/mellanox/mlx5/core/alloc.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> index a92cf545bdaf..dcd281c4691d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> @@ -110,7 +110,7 @@ static struct mlx5_dma_pool *mlx5_dma_pool_create(struct mlx5_core_dev *dev,
> {
> struct mlx5_dma_pool *pool;
>
> - pool = kzalloc_obj(*pool);
> + pool = kzalloc_node(sizeof(*pool), GFP_KERNEL, node);
This removes all the benefits of kzalloc_obj() here...
But I guess the __alloc_objs API doesn'thave the ability to add a node?
It seems like we'd benefit from having support for the node parameter..
but it does seem tricky to add. kzalloc_node_obj() could potentially be
added. Staring at the implementation I have no idea how complicated that
would be.
Still, this only really changes two allocations and they're pretty
obviously correct.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> if (!pool)
> return NULL;
>
> @@ -127,19 +127,20 @@ mlx5_dma_pool_page_alloc(struct mlx5_dma_pool *pool)
> {
> int blocks_per_page = BIT(PAGE_SHIFT - pool->block_shift);
> struct mlx5_dma_pool_page *page;
> + int node = pool->node;
>
> - page = kzalloc_obj(*page);
> + page = kzalloc_node(sizeof(*page), GFP_KERNEL, node);
> if (!page)
> goto err_out;
>
> page->pool = pool;
> - page->bitmap = bitmap_zalloc(blocks_per_page, GFP_KERNEL);
> + page->bitmap = bitmap_zalloc_node(blocks_per_page, GFP_KERNEL, node);
> if (!page->bitmap)
> goto err_free_page;
>
> bitmap_fill(page->bitmap, blocks_per_page);
> page->buf = mlx5_dma_zalloc_coherent_node(pool->dev, PAGE_SIZE,
> - &page->dma, pool->node);
> + &page->dma, node);
> if (!page->buf)
> goto err_free_bitmap;
>
> @@ -278,7 +279,7 @@ mlx5_frag_buf_node_pools_create(struct mlx5_core_dev *dev, int node)
> {
> struct mlx5_frag_buf_node_pools *node_pools;
>
> - node_pools = kzalloc_obj(*node_pools);
> + node_pools = kzalloc_node(sizeof(*node_pools), GFP_KERNEL, node);
> if (!node_pools)
> return NULL;
>
>
> ---
> base-commit: 8b4e7209c842d8cb9516f1f5ef0a88aa2d8831a6
> change-id: 20261005-net-mlx5-numa-allocate-pool-page-2ee20a04147e
>
> Best regards,
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
2026-10-06 21:57 ` Jacob Keller
@ 2026-10-07 1:56 ` Seongjun Hong
0 siblings, 0 replies; 6+ messages in thread
From: Seongjun Hong @ 2026-10-07 1:56 UTC (permalink / raw)
To: Jacob Keller
Cc: Saeed Mahameed, Leon Romanovsky, Tariq Toukan, Mark Bloch,
Andrew Lunn, David S. Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, netdev, linux-rdma, linux-kernel
On Tue, Oct 06, 2026 at 02:57:28PM -0700, Jacob Keller wrote:
> On 10/6/2026 7:16 AM, Seongjun Hong wrote:
> > The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> > block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> > node hint, so they land on the node of whichever CPU happens to create
> > or fill the pool, while the DMA pages they describe are allocated on
> > the pool's node.
> >
> > The pool, page and bitmap are dereferenced on every block allocation
> > and free. Allocate them on the pool's NUMA node as well, so that all
> > of a pool's state lives on one node.
> >
> > These allocations happen on the control path, so the performance gain
> > is expected to be small; the change is mainly for consistency.
> >
> > Signed-off-by: Seongjun Hong <hsj0512@snu.ac.kr>
> > ---
> > Changes in v2:
> > - fix typo and specify the structs are allocated on the control path
> > - add a prefix net-next to the subject
> > - Link to v1: https://lore.kernel.org/r/20261005-net-mlx5-numa-allocate-pool-page-v1-1-01f4283932b4@snu.ac.kr
> > ---
> > drivers/net/ethernet/mellanox/mlx5/core/alloc.c | 11 ++++++-----
> > 1 file changed, 6 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> > index a92cf545bdaf..dcd281c4691d 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> > @@ -110,7 +110,7 @@ static struct mlx5_dma_pool *mlx5_dma_pool_create(struct mlx5_core_dev *dev,
> > {
> > struct mlx5_dma_pool *pool;
> >
> > - pool = kzalloc_obj(*pool);
> > + pool = kzalloc_node(sizeof(*pool), GFP_KERNEL, node);
>
> This removes all the benefits of kzalloc_obj() here...
Do you mean type checking on compile?
>
> But I guess the __alloc_objs API doesn'thave the ability to add a node?
> It seems like we'd benefit from having support for the node parameter..
> but it does seem tricky to add. kzalloc_node_obj() could potentially be
> added. Staring at the implementation I have no idea how complicated that
> would be.
>
> Still, this only really changes two allocations and they're pretty
> obviously correct.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
>
Jacob, thanks for the review.
As far as I know, there is no node-aware variant of _obj APIs yet,
so kzalloc_node() was the only option here.
> > if (!pool)
> > return NULL;
> >
> > @@ -127,19 +127,20 @@ mlx5_dma_pool_page_alloc(struct mlx5_dma_pool *pool)
> > {
> > int blocks_per_page = BIT(PAGE_SHIFT - pool->block_shift);
> > struct mlx5_dma_pool_page *page;
> > + int node = pool->node;
> >
> > - page = kzalloc_obj(*page);
> > + page = kzalloc_node(sizeof(*page), GFP_KERNEL, node);
> > if (!page)
> > goto err_out;
> >
> > page->pool = pool;
> > - page->bitmap = bitmap_zalloc(blocks_per_page, GFP_KERNEL);
> > + page->bitmap = bitmap_zalloc_node(blocks_per_page, GFP_KERNEL, node);
> > if (!page->bitmap)
> > goto err_free_page;
> >
> > bitmap_fill(page->bitmap, blocks_per_page);
> > page->buf = mlx5_dma_zalloc_coherent_node(pool->dev, PAGE_SIZE,
> > - &page->dma, pool->node);
> > + &page->dma, node);
> > if (!page->buf)
> > goto err_free_bitmap;
> >
> > @@ -278,7 +279,7 @@ mlx5_frag_buf_node_pools_create(struct mlx5_core_dev *dev, int node)
> > {
> > struct mlx5_frag_buf_node_pools *node_pools;
> >
> > - node_pools = kzalloc_obj(*node_pools);
> > + node_pools = kzalloc_node(sizeof(*node_pools), GFP_KERNEL, node);
> > if (!node_pools)
> > return NULL;
> >
> >
> > ---
> > base-commit: 8b4e7209c842d8cb9516f1f5ef0a88aa2d8831a6
> > change-id: 20261005-net-mlx5-numa-allocate-pool-page-2ee20a04147e
> >
> > Best regards,
>
>
>
--
Seongjun Hong
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
2026-10-06 14:16 [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node Seongjun Hong
2026-10-06 14:22 ` sashiko-bot
2026-10-06 21:57 ` Jacob Keller
@ 2026-10-11 6:58 ` Tariq Toukan
2026-10-12 14:50 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 6+ messages in thread
From: Tariq Toukan @ 2026-10-11 6:58 UTC (permalink / raw)
To: Seongjun Hong, Saeed Mahameed, Leon Romanovsky, Tariq Toukan,
Mark Bloch, Andrew Lunn, David S. Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni
Cc: netdev, linux-rdma, linux-kernel
On 06/10/2026 17:16, Seongjun Hong wrote:
> The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> node hint, so they land on the node of whichever CPU happens to create
> or fill the pool, while the DMA pages they describe are allocated on
> the pool's node.
>
> The pool, page and bitmap are dereferenced on every block allocation
> and free. Allocate them on the pool's NUMA node as well, so that all
> of a pool's state lives on one node.
>
> These allocations happen on the control path, so the performance gain
> is expected to be small; the change is mainly for consistency.
>
> Signed-off-by: Seongjun Hong <hsj0512@snu.ac.kr>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Thanks for your patch.
> ---
> Changes in v2:
> - fix typo and specify the structs are allocated on the control path
> - add a prefix net-next to the subject
> - Link to v1: https://lore.kernel.org/r/20261005-net-mlx5-numa-allocate-pool-page-v1-1-01f4283932b4@snu.ac.kr
> ---
> drivers/net/ethernet/mellanox/mlx5/core/alloc.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> index a92cf545bdaf..dcd281c4691d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/alloc.c
> @@ -110,7 +110,7 @@ static struct mlx5_dma_pool *mlx5_dma_pool_create(struct mlx5_core_dev *dev,
> {
> struct mlx5_dma_pool *pool;
>
> - pool = kzalloc_obj(*pool);
> + pool = kzalloc_node(sizeof(*pool), GFP_KERNEL, node);
> if (!pool)
> return NULL;
>
> @@ -127,19 +127,20 @@ mlx5_dma_pool_page_alloc(struct mlx5_dma_pool *pool)
> {
> int blocks_per_page = BIT(PAGE_SHIFT - pool->block_shift);
> struct mlx5_dma_pool_page *page;
> + int node = pool->node;
>
> - page = kzalloc_obj(*page);
> + page = kzalloc_node(sizeof(*page), GFP_KERNEL, node);
> if (!page)
> goto err_out;
>
> page->pool = pool;
> - page->bitmap = bitmap_zalloc(blocks_per_page, GFP_KERNEL);
> + page->bitmap = bitmap_zalloc_node(blocks_per_page, GFP_KERNEL, node);
> if (!page->bitmap)
> goto err_free_page;
>
> bitmap_fill(page->bitmap, blocks_per_page);
> page->buf = mlx5_dma_zalloc_coherent_node(pool->dev, PAGE_SIZE,
> - &page->dma, pool->node);
> + &page->dma, node);
> if (!page->buf)
> goto err_free_bitmap;
>
> @@ -278,7 +279,7 @@ mlx5_frag_buf_node_pools_create(struct mlx5_core_dev *dev, int node)
> {
> struct mlx5_frag_buf_node_pools *node_pools;
>
> - node_pools = kzalloc_obj(*node_pools);
> + node_pools = kzalloc_node(sizeof(*node_pools), GFP_KERNEL, node);
> if (!node_pools)
> return NULL;
>
>
> ---
> base-commit: 8b4e7209c842d8cb9516f1f5ef0a88aa2d8831a6
> change-id: 20261005-net-mlx5-numa-allocate-pool-page-2ee20a04147e
>
> Best regards,
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
2026-10-06 14:16 [PATCH net-next v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node Seongjun Hong
` (2 preceding siblings ...)
2026-10-11 6:58 ` Tariq Toukan
@ 2026-10-12 14:50 ` patchwork-bot+netdevbpf
3 siblings, 0 replies; 6+ messages in thread
From: patchwork-bot+netdevbpf @ 2026-10-12 14:50 UTC (permalink / raw)
To: Seongjun Hong
Cc: saeedm, leon, tariqt, mbloch, andrew+netdev, davem, edumazet,
kuba, pabeni, netdev, linux-rdma, linux-kernel
Hello:
This patch was applied to netdev/net-next.git (main)
by Paolo Abeni <pabeni@redhat.com>:
On Tue, 06 Oct 2026 14:16:45 +0000 you wrote:
> The mlx5 DMA pool structs (mlx5_dma_pool, mlx5_dma_pool_page and its
> block bitmap, and mlx5_frag_buf_node_pools) are allocated without a
> node hint, so they land on the node of whichever CPU happens to create
> or fill the pool, while the DMA pages they describe are allocated on
> the pool's node.
>
> The pool, page and bitmap are dereferenced on every block allocation
> and free. Allocate them on the pool's NUMA node as well, so that all
> of a pool's state lives on one node.
>
> [...]
Here is the summary with links:
- [net-next,v2] net/mlx5: allocate DMA pool structs on the pool's NUMA node
https://git.kernel.org/netdev/net-next/c/29f144016f03
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 6+ messages in thread