[Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

[Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

[Jedrzej Jagielski]

There was a wrong queues representation in sysfs during
driver's reinitialization in case of online cpus number is
less than combined queues. It was caused by stopped
NetworkManager, which is responsible for calling vsi_open
function during driver's initialization.
In specific situation (ex. 12 cpus online) there were 16 queues
in /sys/class/net/<iface>/queues. In case of modifying queues with
value higher, than number of online cpus, then it caused write
errors and other errors.
Add updating of sysfs's queues representation during driver
initialization.

Fixes: 41c445ff0f48 ("i40e: main driver core")
Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki@intel.com>
Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
---
v2:Fix "CHECK: Lines should not end with a '('" warning
---
 drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
index 4ff1c9b9217b..1b6f03cc41da 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_main.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
@@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
 	return NULL;
 }
 
+/**
+ * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
+ * @vsi: vsi structure
+ *
+ * This updates netdev's number of tx/rx queues
+ *
+ * Returns status of setting tx/rx queues
+ **/
+static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
+{
+	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);
+
+	return netif_set_real_num_tx_queues(vsi->netdev,
+					    vsi->num_queue_pairs);
+}
+
 /**
  * i40e_vsi_setup - Set up a VSI by a given type
  * @pf: board private structure
@@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
 	case I40E_VSI_MAIN:
 	case I40E_VSI_VMDQ2:
 		ret = i40e_config_netdev(vsi);
+		if (ret)
+			goto err_netdev;
+		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
 		if (ret)
 			goto err_netdev;
 		ret = register_netdev(vsi->netdev);
-- 
2.27.0

[Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

[Maciej Fijalkowski]

On Wed, Dec 15, 2021 at 10:45:54AM +0000, Jedrzej Jagielski wrote:
> There was a wrong queues representation in sysfs during
> driver's reinitialization in case of online cpus number is
> less than combined queues. It was caused by stopped
> NetworkManager, which is responsible for calling vsi_open
> function during driver's initialization.
> In specific situation (ex. 12 cpus online) there were 16 queues
> in /sys/class/net/<iface>/queues. In case of modifying queues with
> value higher, than number of online cpus, then it caused write
> errors and other errors.
> Add updating of sysfs's queues representation during driver
> initialization.

Description is not very clear to me - how did you get the value of 16
queues? Do you have 16 cpu system and then modified the count of cpus
being online?

> 
> Fixes: 41c445ff0f48 ("i40e: main driver core")
> Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki@intel.com>
> Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
> ---
> v2:Fix "CHECK: Lines should not end with a '('" warning
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 4ff1c9b9217b..1b6f03cc41da 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
>  	return NULL;
>  }
>  
> +/**
> + * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
> + * @vsi: vsi structure
> + *
> + * This updates netdev's number of tx/rx queues
> + *
> + * Returns status of setting tx/rx queues
> + **/
> +static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
> +{
> +	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);

Why return value of above is not checked?

Also would be good to convert the i40e_vsi_open() to make use of this
function you're introducing, I guess.

> +
> +	return netif_set_real_num_tx_queues(vsi->netdev,
> +					    vsi->num_queue_pairs);
> +}
> +
>  /**
>   * i40e_vsi_setup - Set up a VSI by a given type
>   * @pf: board private structure
> @@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
>  	case I40E_VSI_MAIN:
>  	case I40E_VSI_VMDQ2:
>  		ret = i40e_config_netdev(vsi);
> +		if (ret)
> +			goto err_netdev;
> +		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
>  		if (ret)
>  			goto err_netdev;
>  		ret = register_netdev(vsi->netdev);
> -- 
> 2.27.0
> 
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan at osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

[Jagielski, Jedrzej]

>> There was a wrong queues representation in sysfs during
>> driver's reinitialization in case of online cpus number is
>> less than combined queues. It was caused by stopped
>> NetworkManager, which is responsible for calling vsi_open
>> function during driver's initialization.
>> In specific situation (ex. 12 cpus online) there were 16 queues
>> in /sys/class/net/<iface>/queues. In case of modifying queues with
>> value higher, than number of online cpus, then it caused write
>> errors and other errors.
>> Add updating of sysfs's queues representation during driver
>> initialization.
>
>Description is not very clear to me - how did you get the value of 16
>queues? Do you have 16 cpu system and then modified the count of cpus
>being online?
>

When You have less than 16 available/online cpus and network manager
is disabled, the maximum preset number of combined queues of the
interface goes to 16. So you can set combined queues up to 16.
Then there is sysfs representation of TX/RX queues index greater than
available/online cpus. Then trying to write some value to (for example)
/sys/class/net/ethx/queues/tx-15/xps_cpus causes Invalid argument error.

>> 
>> Fixes: 41c445ff0f48 ("i40e: main driver core")
>> Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki@intel.com>
>> Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
>> ---
>> v2:Fix "CHECK: Lines should not end with a '('" warning
>> ---
>>  drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
>>  1 file changed, 19 insertions(+)
>> 
>> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
>> index 4ff1c9b9217b..1b6f03cc41da 100644
>> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
>> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
>> @@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
>>  	return NULL;
>>  }
>>  
>> +/**
>> + * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
>> + * @vsi: vsi structure
>> + *
>> + * This updates netdev's number of tx/rx queues
>> + *
>> + * Returns status of setting tx/rx queues
>> + **/
>> +static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
>> +{
>> +	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);
>
>Why return value of above is not checked?
>
>Also would be good to convert the i40e_vsi_open() to make use of this
>function you're introducing, I guess.
>
Sure, good point.
>> +
>> +	return netif_set_real_num_tx_queues(vsi->netdev,
>> +					    vsi->num_queue_pairs);
>> +}
>> +
>>  /**
>>   * i40e_vsi_setup - Set up a VSI by a given type
>>   * @pf: board private structure
>> @@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
>>  	case I40E_VSI_MAIN:
>>  	case I40E_VSI_VMDQ2:
>>  		ret = i40e_config_netdev(vsi);
>> +		if (ret)
>> +			goto err_netdev;
>> +		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
>>  		if (ret)
>>  			goto err_netdev;
>>  		ret = register_netdev(vsi->netdev);
>> -- 
>> 2.27.0

Thanks for your suggestions,
Jedrzej


-----Original Message-----
From: Fijalkowski, Maciej <maciej.fijalkowski@intel.com> 
Sent: ?roda, 15 grudnia 2021 11:58
To: Jagielski, Jedrzej <jedrzej.jagielski@intel.com>
Cc: intel-wired-lan at lists.osuosl.org; Cieplicki, LukaszX <lukaszx.cieplicki@intel.com>
Subject: Re: [Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

On Wed, Dec 15, 2021 at 10:45:54AM +0000, Jedrzej Jagielski wrote:
> There was a wrong queues representation in sysfs during
> driver's reinitialization in case of online cpus number is
> less than combined queues. It was caused by stopped
> NetworkManager, which is responsible for calling vsi_open
> function during driver's initialization.
> In specific situation (ex. 12 cpus online) there were 16 queues
> in /sys/class/net/<iface>/queues. In case of modifying queues with
> value higher, than number of online cpus, then it caused write
> errors and other errors.
> Add updating of sysfs's queues representation during driver
> initialization.

Description is not very clear to me - how did you get the value of 16
queues? Do you have 16 cpu system and then modified the count of cpus
being online?

> 
> Fixes: 41c445ff0f48 ("i40e: main driver core")
> Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki@intel.com>
> Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski@intel.com>
> ---
> v2:Fix "CHECK: Lines should not end with a '('" warning
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 4ff1c9b9217b..1b6f03cc41da 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
>  	return NULL;
>  }
>  
> +/**
> + * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
> + * @vsi: vsi structure
> + *
> + * This updates netdev's number of tx/rx queues
> + *
> + * Returns status of setting tx/rx queues
> + **/
> +static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
> +{
> +	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);

Why return value of above is not checked?

Also would be good to convert the i40e_vsi_open() to make use of this
function you're introducing, I guess.

> +
> +	return netif_set_real_num_tx_queues(vsi->netdev,
> +					    vsi->num_queue_pairs);
> +}
> +
>  /**
>   * i40e_vsi_setup - Set up a VSI by a given type
>   * @pf: board private structure
> @@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
>  	case I40E_VSI_MAIN:
>  	case I40E_VSI_VMDQ2:
>  		ret = i40e_config_netdev(vsi);
> +		if (ret)
> +			goto err_netdev;
> +		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
>  		if (ret)
>  			goto err_netdev;
>  		ret = register_netdev(vsi->netdev);
> -- 
> 2.27.0
> 
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan at osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan