[PATCH net v3] net/mlx5: Use effective affinity mask for IRQ selection

[PATCH net v3] net/mlx5: Use effective affinity mask for IRQ selection

[Fushuai Wang]

From: Fushuai Wang <wangfushuai@baidu.com>

When a sf is created after a CPU has been taken offline, the IRQ pool may
contain IRQs with affinity masks that include the offline CPU. Since only
online CPUs should be considered for IRQ placement, cpumask_subset() check
would fail because the iter_mask contains offline CPUs that are not present
in req_mask, causing sf creation to fail.

This is an example:
  1. When mlx5 driver loads, it initializes the IRQ pools.
     For sf_ctrl_pool with ≤64 sf:
     - xa_num_irqs = {N, N} (There is only one slot)
  2. When the first SF is created:
     - The ctrl IRQ is allocated with mask=cpu_online_mask={0-191}
  2. We take CPU 20 offline
  3. Existing ctl irq still have mask={0-191}
  4. Create a new SF:
     - req_mask={0-19,21-191}
     - iter_mask={0-191}
     - {0-191} is NOT a subset of {0-19,21-191}
     - least_loaded_irq=NULL
  5. Try to allocate a new irq via irq_pool_request_irq()
  6. xa_alloc() fails because the pool is full(There is only one slot)
  7. sf creation fails with error

Use irq_get_effective_affinity_mask() instead, which returns the IRQ's
actual effective affinity that already excludes offline CPUs.

Fixes: 061f5b23588a ("net/mlx5: SF, Use all available cpu for setting cpu affinity")
Suggested-by: Shay Drory <shayd@nvidia.com>
Signed-off-by: Fushuai Wang <wangfushuai@baidu.com>
---
v2->v3: Separate the patchset to two patches, reverse xmas tree coding style fix.
v1->v2: Use mlx5_irq_get_affinity_mask() api

previous discussion:
https://lore.kernel.org/all/20260603072657.10868-1-fushuai.wang@linux.dev/T/#u
https://lore.kernel.org/all/20260604125705.21241-1-fushuai.wang@linux.dev/T/#u

 drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
index 994fe83da4be..a0bb8ee44e35 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
@@ -105,9 +105,12 @@ irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req
 
 	lockdep_assert_held(&pool->lock);
 	xa_for_each_range(&pool->irqs, index, iter, start, end) {
-		struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
 		int iter_refcount = mlx5_irq_read_locked(iter);
+		const struct cpumask *iter_mask;
 
+		iter_mask = irq_get_effective_affinity_mask(mlx5_irq_get_irq(iter));
+		if (!iter_mask)
+			continue;
 		if (!cpumask_subset(iter_mask, req_mask))
 			/* skip IRQs with a mask which is not subset of req_mask */
 			continue;
-- 
2.36.1

Re: [PATCH net v3] net/mlx5: Use effective affinity mask for IRQ selection

[Shay Drori]

On 05/06/2026 13:21, Fushuai Wang wrote:
> External email: Use caution opening links or attachments
> 
> 
> From: Fushuai Wang <wangfushuai@baidu.com>
> 
> When a sf is created after a CPU has been taken offline, the IRQ pool may
> contain IRQs with affinity masks that include the offline CPU. Since only
> online CPUs should be considered for IRQ placement, cpumask_subset() check
> would fail because the iter_mask contains offline CPUs that are not present
> in req_mask, causing sf creation to fail.
> 
> This is an example:
>    1. When mlx5 driver loads, it initializes the IRQ pools.
>       For sf_ctrl_pool with ≤64 sf:
>       - xa_num_irqs = {N, N} (There is only one slot)
>    2. When the first SF is created:
>       - The ctrl IRQ is allocated with mask=cpu_online_mask={0-191}
>    2. We take CPU 20 offline
>    3. Existing ctl irq still have mask={0-191}
>    4. Create a new SF:
>       - req_mask={0-19,21-191}
>       - iter_mask={0-191}
>       - {0-191} is NOT a subset of {0-19,21-191}
>       - least_loaded_irq=NULL
>    5. Try to allocate a new irq via irq_pool_request_irq()
>    6. xa_alloc() fails because the pool is full(There is only one slot)
>    7. sf creation fails with error
> 
> Use irq_get_effective_affinity_mask() instead, which returns the IRQ's
> actual effective affinity that already excludes offline CPUs.
> 
> Fixes: 061f5b23588a ("net/mlx5: SF, Use all available cpu for setting cpu affinity")
> Suggested-by: Shay Drory <shayd@nvidia.com>

Reviewed-by: Shay Drory <shayd@nvidia.com>

> Signed-off-by: Fushuai Wang <wangfushuai@baidu.com>
> ---
> v2->v3: Separate the patchset to two patches, reverse xmas tree coding style fix.
> v1->v2: Use mlx5_irq_get_affinity_mask() api
> 
> previous discussion:
> https://lore.kernel.org/all/20260603072657.10868-1-fushuai.wang@linux.dev/T/#u
> https://lore.kernel.org/all/20260604125705.21241-1-fushuai.wang@linux.dev/T/#u
> 
>   drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> index 994fe83da4be..a0bb8ee44e35 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> @@ -105,9 +105,12 @@ irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req
> 
>          lockdep_assert_held(&pool->lock);
>          xa_for_each_range(&pool->irqs, index, iter, start, end) {
> -               struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
>                  int iter_refcount = mlx5_irq_read_locked(iter);
> +               const struct cpumask *iter_mask;
> 
> +               iter_mask = irq_get_effective_affinity_mask(mlx5_irq_get_irq(iter));
> +               if (!iter_mask)
> +                       continue;
>                  if (!cpumask_subset(iter_mask, req_mask))
>                          /* skip IRQs with a mask which is not subset of req_mask */
>                          continue;
> --
> 2.36.1
> 

Re: [PATCH net v3] net/mlx5: Use effective affinity mask for IRQ selection

[Tariq Toukan]

On 05/06/2026 13:21, Fushuai Wang wrote:
> From: Fushuai Wang <wangfushuai@baidu.com>
> 
> When a sf is created after a CPU has been taken offline, the IRQ pool may
> contain IRQs with affinity masks that include the offline CPU. Since only
> online CPUs should be considered for IRQ placement, cpumask_subset() check
> would fail because the iter_mask contains offline CPUs that are not present
> in req_mask, causing sf creation to fail.
> 
> This is an example:
>    1. When mlx5 driver loads, it initializes the IRQ pools.
>       For sf_ctrl_pool with ≤64 sf:
>       - xa_num_irqs = {N, N} (There is only one slot)
>    2. When the first SF is created:
>       - The ctrl IRQ is allocated with mask=cpu_online_mask={0-191}
>    2. We take CPU 20 offline
>    3. Existing ctl irq still have mask={0-191}
>    4. Create a new SF:
>       - req_mask={0-19,21-191}
>       - iter_mask={0-191}
>       - {0-191} is NOT a subset of {0-19,21-191}
>       - least_loaded_irq=NULL
>    5. Try to allocate a new irq via irq_pool_request_irq()
>    6. xa_alloc() fails because the pool is full(There is only one slot)
>    7. sf creation fails with error
> 
> Use irq_get_effective_affinity_mask() instead, which returns the IRQ's
> actual effective affinity that already excludes offline CPUs.
> 
> Fixes: 061f5b23588a ("net/mlx5: SF, Use all available cpu for setting cpu affinity")
> Suggested-by: Shay Drory <shayd@nvidia.com>
> Signed-off-by: Fushuai Wang <wangfushuai@baidu.com>
> ---
> v2->v3: Separate the patchset to two patches, reverse xmas tree coding style fix.
> v1->v2: Use mlx5_irq_get_affinity_mask() api
> 
> previous discussion:
> https://lore.kernel.org/all/20260603072657.10868-1-fushuai.wang@linux.dev/T/#u
> https://lore.kernel.org/all/20260604125705.21241-1-fushuai.wang@linux.dev/T/#u
> 
>   drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> index 994fe83da4be..a0bb8ee44e35 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/irq_affinity.c
> @@ -105,9 +105,12 @@ irq_pool_find_least_loaded(struct mlx5_irq_pool *pool, const struct cpumask *req
>   
>   	lockdep_assert_held(&pool->lock);
>   	xa_for_each_range(&pool->irqs, index, iter, start, end) {
> -		struct cpumask *iter_mask = mlx5_irq_get_affinity_mask(iter);
>   		int iter_refcount = mlx5_irq_read_locked(iter);
> +		const struct cpumask *iter_mask;
>   
> +		iter_mask = irq_get_effective_affinity_mask(mlx5_irq_get_irq(iter));
> +		if (!iter_mask)
> +			continue;
>   		if (!cpumask_subset(iter_mask, req_mask))
>   			/* skip IRQs with a mask which is not subset of req_mask */
>   			continue;

Reviewed-by: Tariq Toukan <tariqt@nvidia.com>

Thanks for your patch.

Re: [PATCH net v3] net/mlx5: Use effective affinity mask for IRQ selection

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (main)
by Jakub Kicinski <kuba@kernel.org>:

On Fri,  5 Jun 2026 18:21:12 +0800 you wrote:
> From: Fushuai Wang <wangfushuai@baidu.com>
> 
> When a sf is created after a CPU has been taken offline, the IRQ pool may
> contain IRQs with affinity masks that include the offline CPU. Since only
> online CPUs should be considered for IRQ placement, cpumask_subset() check
> would fail because the iter_mask contains offline CPUs that are not present
> in req_mask, causing sf creation to fail.
> 
> [...]

Here is the summary with links:
  - [net,v3] net/mlx5: Use effective affinity mask for IRQ selection
    https://git.kernel.org/netdev/net/c/a7767290e77c

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html