* [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
@ 2025-10-16 6:22 Michal Swiatkowski
2025-10-16 7:44 ` [Intel-wired-lan] " Paul Menzel
2025-10-16 15:36 ` Alexander Lobakin
0 siblings, 2 replies; 8+ messages in thread
From: Michal Swiatkowski @ 2025-10-16 6:22 UTC (permalink / raw)
To: intel-wired-lan; +Cc: netdev, Michal Swiatkowski, Jacob Keller
On some high-core systems loading ice driver with default values can
lead to queue/irq exhaustion. It will result in no additional resources
for SR-IOV.
In most cases there is no performance reason for more than 64 queues.
Limit the default value to 64. Still, using ethtool the number of
queues can be changed up to num_online_cpus().
This change affects only the default queue amount on systems with more
than 64 cores.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
3 files changed, 28 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 3d4d8b88631b..354ec2950ff3 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
else
return &pf->adapter->ctrl_pf->hw;
}
+
+/**
+ * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
+ *
+ * This function returns the number of online CPUs, but caps it at suitable
+ * default to prevent excessive resource allocation on systems with very high
+ * CPU counts.
+ *
+ * Note: suitable default is currently at 64, which is reflected in default_cpus
+ * constant. In most cases there is no much benefit for more than 64 and it is a
+ * power of 2 number.
+ *
+ * Return: number of online CPUs, capped at suitable default.
+ */
+static inline u16 ice_capped_num_cpus(void)
+{
+ const int default_cpus = 64;
+
+ return min(num_online_cpus(), default_cpus);
+}
#endif /* _ICE_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index 30801fd375f0..df4d847ca858 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -106,9 +106,11 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf,
#define ICE_RDMA_AEQ_MSIX 1
static int ice_get_default_msix_amount(struct ice_pf *pf)
{
- return ICE_MIN_LAN_OICR_MSIX + num_online_cpus() +
+ u16 cpus = ice_capped_num_cpus();
+
+ return ICE_MIN_LAN_OICR_MSIX + cpus +
(test_bit(ICE_FLAG_FD_ENA, pf->flags) ? ICE_FDIR_MSIX : 0) +
- (ice_is_rdma_ena(pf) ? num_online_cpus() + ICE_RDMA_AEQ_MSIX : 0);
+ (ice_is_rdma_ena(pf) ? cpus + ICE_RDMA_AEQ_MSIX : 0);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index bac481e8140d..3c5f8a4b6c6d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -159,12 +159,12 @@ static void ice_vsi_set_num_desc(struct ice_vsi *vsi)
static u16 ice_get_rxq_count(struct ice_pf *pf)
{
- return min(ice_get_avail_rxq_count(pf), num_online_cpus());
+ return min(ice_get_avail_rxq_count(pf), ice_capped_num_cpus());
}
static u16 ice_get_txq_count(struct ice_pf *pf)
{
- return min(ice_get_avail_txq_count(pf), num_online_cpus());
+ return min(ice_get_avail_txq_count(pf), ice_capped_num_cpus());
}
/**
@@ -907,13 +907,13 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
if (vsi->type == ICE_VSI_CHNL)
vsi->rss_size = min_t(u16, vsi->num_rxq, max_rss_size);
else
- vsi->rss_size = min_t(u16, num_online_cpus(),
+ vsi->rss_size = min_t(u16, ice_capped_num_cpus(),
max_rss_size);
vsi->rss_lut_type = ICE_LUT_PF;
break;
case ICE_VSI_SF:
vsi->rss_table_size = ICE_LUT_VSI_SIZE;
- vsi->rss_size = min_t(u16, num_online_cpus(), max_rss_size);
+ vsi->rss_size = min_t(u16, ice_capped_num_cpus(), max_rss_size);
vsi->rss_lut_type = ICE_LUT_VSI;
break;
case ICE_VSI_VF:
--
2.49.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-16 6:22 [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems Michal Swiatkowski
@ 2025-10-16 7:44 ` Paul Menzel
2025-10-16 8:45 ` Michal Swiatkowski
2025-10-16 15:36 ` Alexander Lobakin
1 sibling, 1 reply; 8+ messages in thread
From: Paul Menzel @ 2025-10-16 7:44 UTC (permalink / raw)
To: Michal Swiatkowski; +Cc: intel-wired-lan, netdev, Jacob Keller
Dear Michal,
Thank you for the patch. I’d mention the 64 in the summary:
> ice: lower default irq/queue counts to 64 on > 64 core systems
Am 16.10.25 um 08:22 schrieb Michal Swiatkowski:
> On some high-core systems loading ice driver with default values can
> lead to queue/irq exhaustion. It will result in no additional resources
> for SR-IOV.
>
> In most cases there is no performance reason for more than 64 queues.
> Limit the default value to 64. Still, using ethtool the number of
> queues can be changed up to num_online_cpus().
>
> This change affects only the default queue amount on systems with more
> than 64 cores.
Please document a specific system and steps to reproduce the issue.
Please also document how to override the value.
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
> drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
> drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
> 3 files changed, 28 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 3d4d8b88631b..354ec2950ff3 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
> else
> return &pf->adapter->ctrl_pf->hw;
> }
> +
> +/**
> + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
> + *
> + * This function returns the number of online CPUs, but caps it at suitable
> + * default to prevent excessive resource allocation on systems with very high
> + * CPU counts.
> + *
> + * Note: suitable default is currently at 64, which is reflected in default_cpus
> + * constant. In most cases there is no much benefit for more than 64 and it is a
no*t* much
> + * power of 2 number.
> + *
> + * Return: number of online CPUs, capped at suitable default.
> + */
> +static inline u16 ice_capped_num_cpus(void)
Why not return `unsigned int` or `size_t`?
> +{
> + const int default_cpus = 64;
> +
> + return min(num_online_cpus(), default_cpus);
> +}
> #endif /* _ICE_H_ */
> diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> index 30801fd375f0..df4d847ca858 100644
> --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> @@ -106,9 +106,11 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf,
> #define ICE_RDMA_AEQ_MSIX 1
> static int ice_get_default_msix_amount(struct ice_pf *pf)
> {
> - return ICE_MIN_LAN_OICR_MSIX + num_online_cpus() +
> + u16 cpus = ice_capped_num_cpus();
> +
> + return ICE_MIN_LAN_OICR_MSIX + cpus +
> (test_bit(ICE_FLAG_FD_ENA, pf->flags) ? ICE_FDIR_MSIX : 0) +
> - (ice_is_rdma_ena(pf) ? num_online_cpus() + ICE_RDMA_AEQ_MSIX : 0);
> + (ice_is_rdma_ena(pf) ? cpus + ICE_RDMA_AEQ_MSIX : 0);
> }
>
> /**
> diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
> index bac481e8140d..3c5f8a4b6c6d 100644
> --- a/drivers/net/ethernet/intel/ice/ice_lib.c
> +++ b/drivers/net/ethernet/intel/ice/ice_lib.c
> @@ -159,12 +159,12 @@ static void ice_vsi_set_num_desc(struct ice_vsi *vsi)
>
> static u16 ice_get_rxq_count(struct ice_pf *pf)
> {
> - return min(ice_get_avail_rxq_count(pf), num_online_cpus());
> + return min(ice_get_avail_rxq_count(pf), ice_capped_num_cpus());
> }
>
> static u16 ice_get_txq_count(struct ice_pf *pf)
> {
> - return min(ice_get_avail_txq_count(pf), num_online_cpus());
> + return min(ice_get_avail_txq_count(pf), ice_capped_num_cpus());
> }
>
> /**
> @@ -907,13 +907,13 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
> if (vsi->type == ICE_VSI_CHNL)
> vsi->rss_size = min_t(u16, vsi->num_rxq, max_rss_size);
> else
> - vsi->rss_size = min_t(u16, num_online_cpus(),
> + vsi->rss_size = min_t(u16, ice_capped_num_cpus(),
> max_rss_size);
> vsi->rss_lut_type = ICE_LUT_PF;
> break;
> case ICE_VSI_SF:
> vsi->rss_table_size = ICE_LUT_VSI_SIZE;
> - vsi->rss_size = min_t(u16, num_online_cpus(), max_rss_size);
> + vsi->rss_size = min_t(u16, ice_capped_num_cpus(), max_rss_size);
> vsi->rss_lut_type = ICE_LUT_VSI;
> break;
> case ICE_VSI_VF:
With the changes addressed, feel free to add:
Reviewed-by: Paul Menzel <pmenzel@molgen.mpg.de>
Kind regards,
Paul
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-16 7:44 ` [Intel-wired-lan] " Paul Menzel
@ 2025-10-16 8:45 ` Michal Swiatkowski
0 siblings, 0 replies; 8+ messages in thread
From: Michal Swiatkowski @ 2025-10-16 8:45 UTC (permalink / raw)
To: Paul Menzel; +Cc: Michal Swiatkowski, intel-wired-lan, netdev, Jacob Keller
On Thu, Oct 16, 2025 at 09:44:43AM +0200, Paul Menzel wrote:
> Dear Michal,
>
>
> Thank you for the patch. I’d mention the 64 in the summary:
>
Sure, I will add it.
> > ice: lower default irq/queue counts to 64 on > 64 core systems
>
>
> Am 16.10.25 um 08:22 schrieb Michal Swiatkowski:
> > On some high-core systems loading ice driver with default values can
> > lead to queue/irq exhaustion. It will result in no additional resources
> > for SR-IOV.
> >
> > In most cases there is no performance reason for more than 64 queues.
> > Limit the default value to 64. Still, using ethtool the number of
> > queues can be changed up to num_online_cpus().
> >
> > This change affects only the default queue amount on systems with more
> > than 64 cores.
>
> Please document a specific system and steps to reproduce the issue.
>
> Please also document how to override the value.
Ok, will add.
>
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > ---
> > drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
> > drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
> > drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
> > 3 files changed, 28 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> > index 3d4d8b88631b..354ec2950ff3 100644
> > --- a/drivers/net/ethernet/intel/ice/ice.h
> > +++ b/drivers/net/ethernet/intel/ice/ice.h
> > @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
> > else
> > return &pf->adapter->ctrl_pf->hw;
> > }
> > +
> > +/**
> > + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
> > + *
> > + * This function returns the number of online CPUs, but caps it at suitable
> > + * default to prevent excessive resource allocation on systems with very high
> > + * CPU counts.
> > + *
> > + * Note: suitable default is currently at 64, which is reflected in default_cpus
> > + * constant. In most cases there is no much benefit for more than 64 and it is a
>
> no*t* much
>
Will fix
> > + * power of 2 number.
> > + *
> > + * Return: number of online CPUs, capped at suitable default.
> > + */
> > +static inline u16 ice_capped_num_cpus(void)
>
> Why not return `unsigned int` or `size_t`?
>
Just because u16 is used for queue counts, but I can go with unsigned
int, makes more sense as num_online_cpus() is returning unsigned int.
> > +{
> > + const int default_cpus = 64;
> > +
> > + return min(num_online_cpus(), default_cpus);
> > +}
> > #endif /* _ICE_H_ */
> > diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> > index 30801fd375f0..df4d847ca858 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> > @@ -106,9 +106,11 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf,
> > #define ICE_RDMA_AEQ_MSIX 1
> > static int ice_get_default_msix_amount(struct ice_pf *pf)
> > {
> > - return ICE_MIN_LAN_OICR_MSIX + num_online_cpus() +
> > + u16 cpus = ice_capped_num_cpus();
> > +
> > + return ICE_MIN_LAN_OICR_MSIX + cpus +
> > (test_bit(ICE_FLAG_FD_ENA, pf->flags) ? ICE_FDIR_MSIX : 0) +
> > - (ice_is_rdma_ena(pf) ? num_online_cpus() + ICE_RDMA_AEQ_MSIX : 0);
> > + (ice_is_rdma_ena(pf) ? cpus + ICE_RDMA_AEQ_MSIX : 0);
> > }
> > /**
> > diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
> > index bac481e8140d..3c5f8a4b6c6d 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_lib.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_lib.c
> > @@ -159,12 +159,12 @@ static void ice_vsi_set_num_desc(struct ice_vsi *vsi)
> > static u16 ice_get_rxq_count(struct ice_pf *pf)
> > {
> > - return min(ice_get_avail_rxq_count(pf), num_online_cpus());
> > + return min(ice_get_avail_rxq_count(pf), ice_capped_num_cpus());
> > }
> > static u16 ice_get_txq_count(struct ice_pf *pf)
> > {
> > - return min(ice_get_avail_txq_count(pf), num_online_cpus());
> > + return min(ice_get_avail_txq_count(pf), ice_capped_num_cpus());
> > }
> > /**
> > @@ -907,13 +907,13 @@ static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
> > if (vsi->type == ICE_VSI_CHNL)
> > vsi->rss_size = min_t(u16, vsi->num_rxq, max_rss_size);
> > else
> > - vsi->rss_size = min_t(u16, num_online_cpus(),
> > + vsi->rss_size = min_t(u16, ice_capped_num_cpus(),
> > max_rss_size);
> > vsi->rss_lut_type = ICE_LUT_PF;
> > break;
> > case ICE_VSI_SF:
> > vsi->rss_table_size = ICE_LUT_VSI_SIZE;
> > - vsi->rss_size = min_t(u16, num_online_cpus(), max_rss_size);
> > + vsi->rss_size = min_t(u16, ice_capped_num_cpus(), max_rss_size);
> > vsi->rss_lut_type = ICE_LUT_VSI;
> > break;
> > case ICE_VSI_VF:
>
> With the changes addressed, feel free to add:
>
> Reviewed-by: Paul Menzel <pmenzel@molgen.mpg.de>
>
Thanks
>
> Kind regards,
>
> Paul
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-16 6:22 [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems Michal Swiatkowski
2025-10-16 7:44 ` [Intel-wired-lan] " Paul Menzel
@ 2025-10-16 15:36 ` Alexander Lobakin
2025-10-17 5:03 ` Przemek Kitszel
1 sibling, 1 reply; 8+ messages in thread
From: Alexander Lobakin @ 2025-10-16 15:36 UTC (permalink / raw)
To: Michal Swiatkowski; +Cc: intel-wired-lan, netdev, Jacob Keller
From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Date: Thu, 16 Oct 2025 08:22:50 +0200
> On some high-core systems loading ice driver with default values can
> lead to queue/irq exhaustion. It will result in no additional resources
> for SR-IOV.
>
> In most cases there is no performance reason for more than 64 queues.
> Limit the default value to 64. Still, using ethtool the number of
> queues can be changed up to num_online_cpus().
>
> This change affects only the default queue amount on systems with more
> than 64 cores.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
> drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
> drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
> 3 files changed, 28 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 3d4d8b88631b..354ec2950ff3 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
> else
> return &pf->adapter->ctrl_pf->hw;
> }
> +
> +/**
> + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
> + *
> + * This function returns the number of online CPUs, but caps it at suitable
> + * default to prevent excessive resource allocation on systems with very high
> + * CPU counts.
> + *
> + * Note: suitable default is currently at 64, which is reflected in default_cpus
> + * constant. In most cases there is no much benefit for more than 64 and it is a
> + * power of 2 number.
> + *
> + * Return: number of online CPUs, capped at suitable default.
> + */
> +static inline u16 ice_capped_num_cpus(void)
> +{
> + const int default_cpus = 64;
Maybe we should just use netif_get_num_default_rss_queues() like I did
in idpf?
Or it still can be too high e.g. on clusters with > 256 CPUs?
> +
> + return min(num_online_cpus(), default_cpus);
> +}
> #endif /* _ICE_H_ */
Thanks,
Olek
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-16 15:36 ` Alexander Lobakin
@ 2025-10-17 5:03 ` Przemek Kitszel
2025-10-17 7:30 ` Michal Swiatkowski
0 siblings, 1 reply; 8+ messages in thread
From: Przemek Kitszel @ 2025-10-17 5:03 UTC (permalink / raw)
To: Alexander Lobakin, Michal Swiatkowski
Cc: intel-wired-lan, netdev, Jacob Keller
On 10/16/25 17:36, Alexander Lobakin wrote:
> From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Date: Thu, 16 Oct 2025 08:22:50 +0200
>
>> On some high-core systems loading ice driver with default values can
>> lead to queue/irq exhaustion. It will result in no additional resources
>> for SR-IOV.
>>
>> In most cases there is no performance reason for more than 64 queues.
>> Limit the default value to 64. Still, using ethtool the number of
>> queues can be changed up to num_online_cpus().
>>
>> This change affects only the default queue amount on systems with more
>> than 64 cores.
>>
>> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
>> Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>> ---
>> drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
>> drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
>> drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
>> 3 files changed, 28 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
>> index 3d4d8b88631b..354ec2950ff3 100644
>> --- a/drivers/net/ethernet/intel/ice/ice.h
>> +++ b/drivers/net/ethernet/intel/ice/ice.h
>> @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
>> else
>> return &pf->adapter->ctrl_pf->hw;
>> }
>> +
>> +/**
>> + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
>> + *
>> + * This function returns the number of online CPUs, but caps it at suitable
>> + * default to prevent excessive resource allocation on systems with very high
>> + * CPU counts.
>> + *
>> + * Note: suitable default is currently at 64, which is reflected in default_cpus
>> + * constant. In most cases there is no much benefit for more than 64 and it is a
>> + * power of 2 number.
>> + *
>> + * Return: number of online CPUs, capped at suitable default.
>> + */
>> +static inline u16 ice_capped_num_cpus(void)
>> +{
>> + const int default_cpus = 64;
>
> Maybe we should just use netif_get_num_default_rss_queues() like I did
> in idpf?
>
> Or it still can be too high e.g. on clusters with > 256 CPUs?
good point,
perhaps we should both use it and change the (kernel) func to cap at 64
>
>> +
>> + return min(num_online_cpus(), default_cpus);
>> +}
>> #endif /* _ICE_H_ */
> Thanks,
> Olek
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-17 5:03 ` Przemek Kitszel
@ 2025-10-17 7:30 ` Michal Swiatkowski
2025-10-17 14:35 ` Alexander Lobakin
0 siblings, 1 reply; 8+ messages in thread
From: Michal Swiatkowski @ 2025-10-17 7:30 UTC (permalink / raw)
To: Przemek Kitszel
Cc: Alexander Lobakin, Michal Swiatkowski, intel-wired-lan, netdev,
Jacob Keller
On Fri, Oct 17, 2025 at 07:03:31AM +0200, Przemek Kitszel wrote:
> On 10/16/25 17:36, Alexander Lobakin wrote:
> > From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Date: Thu, 16 Oct 2025 08:22:50 +0200
> >
> > > On some high-core systems loading ice driver with default values can
> > > lead to queue/irq exhaustion. It will result in no additional resources
> > > for SR-IOV.
> > >
> > > In most cases there is no performance reason for more than 64 queues.
> > > Limit the default value to 64. Still, using ethtool the number of
> > > queues can be changed up to num_online_cpus().
> > >
> > > This change affects only the default queue amount on systems with more
> > > than 64 cores.
> > >
> > > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > > Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > > ---
> > > drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
> > > drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
> > > drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
> > > 3 files changed, 28 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> > > index 3d4d8b88631b..354ec2950ff3 100644
> > > --- a/drivers/net/ethernet/intel/ice/ice.h
> > > +++ b/drivers/net/ethernet/intel/ice/ice.h
> > > @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
> > > else
> > > return &pf->adapter->ctrl_pf->hw;
> > > }
> > > +
> > > +/**
> > > + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
> > > + *
> > > + * This function returns the number of online CPUs, but caps it at suitable
> > > + * default to prevent excessive resource allocation on systems with very high
> > > + * CPU counts.
> > > + *
> > > + * Note: suitable default is currently at 64, which is reflected in default_cpus
> > > + * constant. In most cases there is no much benefit for more than 64 and it is a
> > > + * power of 2 number.
> > > + *
> > > + * Return: number of online CPUs, capped at suitable default.
> > > + */
> > > +static inline u16 ice_capped_num_cpus(void)
> > > +{
> > > + const int default_cpus = 64;
> >
> > Maybe we should just use netif_get_num_default_rss_queues() like I did
> > in idpf?
> >
> > Or it still can be too high e.g. on clusters with > 256 CPUs?
>
> good point,
> perhaps we should both use it and change the (kernel) func to cap at 64
>
Sounds good, thanks for pointing the function.
Do you think it is ok to cap the generic function? Maybe other vendors
want more default queues.
What about capping netif_get_num_default_rss_queues() at 64 just for
ice?
> >
> > > +
> > > + return min(num_online_cpus(), default_cpus);
> > > +}
> > > #endif /* _ICE_H_ */
> > Thanks,
> > Olek
> >
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-17 7:30 ` Michal Swiatkowski
@ 2025-10-17 14:35 ` Alexander Lobakin
2025-10-22 7:09 ` Michal Swiatkowski
0 siblings, 1 reply; 8+ messages in thread
From: Alexander Lobakin @ 2025-10-17 14:35 UTC (permalink / raw)
To: Michal Swiatkowski; +Cc: Przemek Kitszel, intel-wired-lan, netdev, Jacob Keller
From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Date: Fri, 17 Oct 2025 09:30:44 +0200
> On Fri, Oct 17, 2025 at 07:03:31AM +0200, Przemek Kitszel wrote:
>> On 10/16/25 17:36, Alexander Lobakin wrote:
>>> From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>>> Date: Thu, 16 Oct 2025 08:22:50 +0200
>>>
>>>> On some high-core systems loading ice driver with default values can
>>>> lead to queue/irq exhaustion. It will result in no additional resources
>>>> for SR-IOV.
>>>>
>>>> In most cases there is no performance reason for more than 64 queues.
>>>> Limit the default value to 64. Still, using ethtool the number of
>>>> queues can be changed up to num_online_cpus().
>>>>
>>>> This change affects only the default queue amount on systems with more
>>>> than 64 cores.
>>>>
>>>> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
>>>> Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>>>> ---
>>>> drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
>>>> drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
>>>> drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
>>>> 3 files changed, 28 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
>>>> index 3d4d8b88631b..354ec2950ff3 100644
>>>> --- a/drivers/net/ethernet/intel/ice/ice.h
>>>> +++ b/drivers/net/ethernet/intel/ice/ice.h
>>>> @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
>>>> else
>>>> return &pf->adapter->ctrl_pf->hw;
>>>> }
>>>> +
>>>> +/**
>>>> + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
>>>> + *
>>>> + * This function returns the number of online CPUs, but caps it at suitable
>>>> + * default to prevent excessive resource allocation on systems with very high
>>>> + * CPU counts.
>>>> + *
>>>> + * Note: suitable default is currently at 64, which is reflected in default_cpus
>>>> + * constant. In most cases there is no much benefit for more than 64 and it is a
>>>> + * power of 2 number.
>>>> + *
>>>> + * Return: number of online CPUs, capped at suitable default.
>>>> + */
>>>> +static inline u16 ice_capped_num_cpus(void)
>>>> +{
>>>> + const int default_cpus = 64;
>>>
>>> Maybe we should just use netif_get_num_default_rss_queues() like I did
>>> in idpf?
>>>
>>> Or it still can be too high e.g. on clusters with > 256 CPUs?
>>
>> good point,
>> perhaps we should both use it and change the (kernel) func to cap at 64
>>
>
> Sounds good, thanks for pointing the function.
>
> Do you think it is ok to cap the generic function? Maybe other vendors
> want more default queues.
Nah I don't think it's a good idea to hardcode any numbers in the
generic function.
>
> What about capping netif_get_num_default_rss_queues() at 64 just for
> ice?
netif_get_num_default_rss_queues() returns *half* of the number of
*physical* cores. I.e. it will return something bigger than 64 only in
case of > 256 threads in the system (considering SMT).
Do we need to still cap this to 64 in ice at all?
Thanks,
Olek
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Intel-wired-lan] [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems
2025-10-17 14:35 ` Alexander Lobakin
@ 2025-10-22 7:09 ` Michal Swiatkowski
0 siblings, 0 replies; 8+ messages in thread
From: Michal Swiatkowski @ 2025-10-22 7:09 UTC (permalink / raw)
To: Alexander Lobakin
Cc: Michal Swiatkowski, Przemek Kitszel, intel-wired-lan, netdev,
Jacob Keller
On Fri, Oct 17, 2025 at 04:35:18PM +0200, Alexander Lobakin wrote:
> From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Date: Fri, 17 Oct 2025 09:30:44 +0200
>
> > On Fri, Oct 17, 2025 at 07:03:31AM +0200, Przemek Kitszel wrote:
> >> On 10/16/25 17:36, Alexander Lobakin wrote:
> >>> From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> >>> Date: Thu, 16 Oct 2025 08:22:50 +0200
> >>>
> >>>> On some high-core systems loading ice driver with default values can
> >>>> lead to queue/irq exhaustion. It will result in no additional resources
> >>>> for SR-IOV.
> >>>>
> >>>> In most cases there is no performance reason for more than 64 queues.
> >>>> Limit the default value to 64. Still, using ethtool the number of
> >>>> queues can be changed up to num_online_cpus().
> >>>>
> >>>> This change affects only the default queue amount on systems with more
> >>>> than 64 cores.
> >>>>
> >>>> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> >>>> Signed-off-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> >>>> ---
> >>>> drivers/net/ethernet/intel/ice/ice.h | 20 ++++++++++++++++++++
> >>>> drivers/net/ethernet/intel/ice/ice_irq.c | 6 ++++--
> >>>> drivers/net/ethernet/intel/ice/ice_lib.c | 8 ++++----
> >>>> 3 files changed, 28 insertions(+), 6 deletions(-)
> >>>>
> >>>> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> >>>> index 3d4d8b88631b..354ec2950ff3 100644
> >>>> --- a/drivers/net/ethernet/intel/ice/ice.h
> >>>> +++ b/drivers/net/ethernet/intel/ice/ice.h
> >>>> @@ -1133,4 +1133,24 @@ static inline struct ice_hw *ice_get_primary_hw(struct ice_pf *pf)
> >>>> else
> >>>> return &pf->adapter->ctrl_pf->hw;
> >>>> }
> >>>> +
> >>>> +/**
> >>>> + * ice_capped_num_cpus - normalize the number of CPUs to a reasonable limit
> >>>> + *
> >>>> + * This function returns the number of online CPUs, but caps it at suitable
> >>>> + * default to prevent excessive resource allocation on systems with very high
> >>>> + * CPU counts.
> >>>> + *
> >>>> + * Note: suitable default is currently at 64, which is reflected in default_cpus
> >>>> + * constant. In most cases there is no much benefit for more than 64 and it is a
> >>>> + * power of 2 number.
> >>>> + *
> >>>> + * Return: number of online CPUs, capped at suitable default.
> >>>> + */
> >>>> +static inline u16 ice_capped_num_cpus(void)
> >>>> +{
> >>>> + const int default_cpus = 64;
> >>>
> >>> Maybe we should just use netif_get_num_default_rss_queues() like I did
> >>> in idpf?
> >>>
> >>> Or it still can be too high e.g. on clusters with > 256 CPUs?
> >>
> >> good point,
> >> perhaps we should both use it and change the (kernel) func to cap at 64
> >>
> >
> > Sounds good, thanks for pointing the function.
> >
> > Do you think it is ok to cap the generic function? Maybe other vendors
> > want more default queues.
>
> Nah I don't think it's a good idea to hardcode any numbers in the
> generic function.
>
> >
> > What about capping netif_get_num_default_rss_queues() at 64 just for
> > ice?
>
> netif_get_num_default_rss_queues() returns *half* of the number of
> *physical* cores. I.e. it will return something bigger than 64 only in
> case of > 256 threads in the system (considering SMT).
>
> Do we need to still cap this to 64 in ice at all?
That can be good enough. I will send next version with just call to this
function.
>
> Thanks,
> Olek
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-10-22 7:11 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-16 6:22 [PATCH iwl-next v1] ice: lower default irq/queue counts on high-core systems Michal Swiatkowski
2025-10-16 7:44 ` [Intel-wired-lan] " Paul Menzel
2025-10-16 8:45 ` Michal Swiatkowski
2025-10-16 15:36 ` Alexander Lobakin
2025-10-17 5:03 ` Przemek Kitszel
2025-10-17 7:30 ` Michal Swiatkowski
2025-10-17 14:35 ` Alexander Lobakin
2025-10-22 7:09 ` Michal Swiatkowski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).