Re: [Intel-wired-lan] [PATCH net-next v2 0/8] net: napi: add CPU affinity to napi->config

[Shay Drori <shayd@nvidia.com>]

On 18/12/2024 18:58, Ahmed Zaki wrote:
> External email: Use caution opening links or attachments
> 
> 
> Move the IRQ affinity management to the napi struct. All drivers that are
> already using netif_napi_set_irq() are modified to the new API. Except
> mlx5 because it is implementing IRQ pools and moving to the new API does
> not seem trivial.
> 
> Tested on bnxt, ice and idpf.
> ---
> Opens: is cpu_online_mask the best default mask? drivers do this differently

cpu_online_mask is not the best default mask for IRQ affinity management.
Here are two reasons:
- Performance Gains from Driver-Specific CPU Assignment: Many drivers
   assign different CPUs to each IRQ to optimize performance. This plays
a crucial role in CPU utilization.
- Impact of NUMA Node Distance on Traffic Performance: NUMA topology
   plays a crucial role in IRQ performance. Assigning IRQs to CPUs on
   the same NUMA node as the associated device minimizes latency caused
   by remote memory access.[1]

[1]
for more details on NUMA preference, you can look at commit 
2acda57736de1e486036b90a648e67a3599080a1


> 
> v2:
>      - Also move the ARFS IRQ affinity management from drivers to core. Via
>        netif_napi_set_irq(), drivers can ask the core to add the IRQ to the
>        ARFS rmap (already allocated by the driver).
> 
> RFC -> v1:
>      - https://lore.kernel.org/netdev/20241210002626.366878-1-ahmed.zaki@intel.com/
>      - move static inline affinity functions to net/dev/core.c
>      - add the new napi->irq_flags (patch 1)
>      - add code changes to bnxt, mlx4 and ice.
> 
> Ahmed Zaki (8):
>    net: napi: add irq_flags to napi struct
>    net: allow ARFS rmap management in core
>    lib: cpu_rmap: allow passing a notifier callback
>    net: napi: add CPU affinity to napi->config
>    bnxt: use napi's irq affinity
>    ice: use napi's irq affinity
>    idpf: use napi's irq affinity
>    mlx4: use napi's irq affinity
> 
>   drivers/net/ethernet/amazon/ena/ena_netdev.c  | 21 ++---
>   drivers/net/ethernet/broadcom/bnxt/bnxt.c     | 51 +++--------
>   drivers/net/ethernet/broadcom/bnxt/bnxt.h     |  2 -
>   drivers/net/ethernet/broadcom/tg3.c           |  2 +-
>   drivers/net/ethernet/cisco/enic/enic_main.c   |  3 +-
>   drivers/net/ethernet/google/gve/gve_utils.c   |  2 +-
>   .../net/ethernet/hisilicon/hns3/hns3_enet.c   |  2 +-
>   drivers/net/ethernet/intel/e1000/e1000_main.c |  2 +-
>   drivers/net/ethernet/intel/e1000e/netdev.c    |  2 +-
>   drivers/net/ethernet/intel/ice/ice.h          |  3 -
>   drivers/net/ethernet/intel/ice/ice_arfs.c     | 10 +--
>   drivers/net/ethernet/intel/ice/ice_base.c     |  7 +-
>   drivers/net/ethernet/intel/ice/ice_lib.c      | 14 +--
>   drivers/net/ethernet/intel/ice/ice_main.c     | 44 ----------
>   drivers/net/ethernet/intel/idpf/idpf_txrx.c   | 19 ++--
>   drivers/net/ethernet/intel/idpf/idpf_txrx.h   |  6 +-
>   drivers/net/ethernet/mellanox/mlx4/en_cq.c    |  8 +-
>   .../net/ethernet/mellanox/mlx4/en_netdev.c    | 33 +------
>   drivers/net/ethernet/mellanox/mlx4/eq.c       | 24 +----
>   drivers/net/ethernet/mellanox/mlx4/main.c     | 42 +--------
>   drivers/net/ethernet/mellanox/mlx4/mlx4.h     |  1 -
>   drivers/net/ethernet/mellanox/mlx4/mlx4_en.h  |  1 -
>   .../net/ethernet/mellanox/mlx5/core/en_main.c |  2 +-
>   .../net/ethernet/mellanox/mlx5/core/pci_irq.c |  2 +-
>   drivers/net/ethernet/meta/fbnic/fbnic_txrx.c  |  3 +-
>   drivers/net/ethernet/qlogic/qede/qede_main.c  | 28 +++---
>   drivers/net/ethernet/sfc/falcon/efx.c         |  9 ++
>   drivers/net/ethernet/sfc/falcon/nic.c         | 10 ---
>   drivers/net/ethernet/sfc/nic.c                |  2 +-
>   drivers/net/ethernet/sfc/siena/efx_channels.c |  9 ++
>   drivers/net/ethernet/sfc/siena/nic.c          | 10 ---
>   include/linux/cpu_rmap.h                      | 13 ++-
>   include/linux/netdevice.h                     | 23 ++++-
>   lib/cpu_rmap.c                                | 20 ++---
>   net/core/dev.c                                | 87 ++++++++++++++++++-
>   35 files changed, 215 insertions(+), 302 deletions(-)
> 
> --
> 2.43.0
>