[PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[Paul Barker]

The TX queue should be serviced each time the poll function is called,
even if the full RX work budget has been consumed. This prevents
starvation of the TX queue when RX bandwidth usage is high.

Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
---
Changes from v1:
  * Use the correct 'Fixes' tag.
  * Call the new variable 'unmask' and drop the unnecessary initializer,
    as requested by Sergey.

 drivers/net/ethernet/renesas/ravb_main.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
index d1be030c8848..48803050abdb 100644
--- a/drivers/net/ethernet/renesas/ravb_main.c
+++ b/drivers/net/ethernet/renesas/ravb_main.c
@@ -1324,12 +1324,12 @@ static int ravb_poll(struct napi_struct *napi, int budget)
 	int q = napi - priv->napi;
 	int mask = BIT(q);
 	int quota = budget;
+	bool unmask;
 
 	/* Processing RX Descriptor Ring */
 	/* Clear RX interrupt */
 	ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0);
-	if (ravb_rx(ndev, &quota, q))
-		goto out;
+	unmask = !ravb_rx(ndev, &quota, q);
 
 	/* Processing TX Descriptor Ring */
 	spin_lock_irqsave(&priv->lock, flags);
@@ -1339,6 +1339,9 @@ static int ravb_poll(struct napi_struct *napi, int budget)
 	netif_wake_subqueue(ndev, q);
 	spin_unlock_irqrestore(&priv->lock, flags);
 
+	if (!unmask)
+		goto out;
+
 	napi_complete(napi);
 
 	/* Re-enable RX/TX interrupts */

base-commit: ea2a1cfc3b2019bdea6324acd3c03606b60d71ad
-- 
2.39.2

[PATCH v2 2/2] net: ravb: Always update error counters

[Paul Barker]

The error statistics should be updated each time the poll function is
called, even if the full RX work budget has been consumed. This prevents
the counts from becoming stuck when RX bandwidth usage is high.

This also ensures that error counters are not updated after we've
re-enabled interrupts as that could result in a race condition.

Also drop an unnecessary space.

Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
---
Changes from v1:
  * Use the correct 'Fixes' tag.
  * Actually drop the space described in the commit message!
    (I must have lost this when rebasing things)

 drivers/net/ethernet/renesas/ravb_main.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
index 48803050abdb..ba01c8cc3c90 100644
--- a/drivers/net/ethernet/renesas/ravb_main.c
+++ b/drivers/net/ethernet/renesas/ravb_main.c
@@ -1339,6 +1339,15 @@ static int ravb_poll(struct napi_struct *napi, int budget)
 	netif_wake_subqueue(ndev, q);
 	spin_unlock_irqrestore(&priv->lock, flags);
 
+	/* Receive error message handling */
+	priv->rx_over_errors = priv->stats[RAVB_BE].rx_over_errors;
+	if (info->nc_queues)
+		priv->rx_over_errors += priv->stats[RAVB_NC].rx_over_errors;
+	if (priv->rx_over_errors != ndev->stats.rx_over_errors)
+		ndev->stats.rx_over_errors = priv->rx_over_errors;
+	if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors)
+		ndev->stats.rx_fifo_errors = priv->rx_fifo_errors;
+
 	if (!unmask)
 		goto out;
 
@@ -1355,14 +1364,6 @@ static int ravb_poll(struct napi_struct *napi, int budget)
 	}
 	spin_unlock_irqrestore(&priv->lock, flags);
 
-	/* Receive error message handling */
-	priv->rx_over_errors =  priv->stats[RAVB_BE].rx_over_errors;
-	if (info->nc_queues)
-		priv->rx_over_errors += priv->stats[RAVB_NC].rx_over_errors;
-	if (priv->rx_over_errors != ndev->stats.rx_over_errors)
-		ndev->stats.rx_over_errors = priv->rx_over_errors;
-	if (priv->rx_fifo_errors != ndev->stats.rx_fifo_errors)
-		ndev->stats.rx_fifo_errors = priv->rx_fifo_errors;
 out:
 	return budget - quota;
 }
-- 
2.39.2

Re: [PATCH v2 2/2] net: ravb: Always update error counters

[Sergey Shtylyov]

On 4/2/24 5:53 PM, Paul Barker wrote:

> The error statistics should be updated each time the poll function is
> called, even if the full RX work budget has been consumed. This prevents
> the counts from becoming stuck when RX bandwidth usage is high.
> 
> This also ensures that error counters are not updated after we've
> re-enabled interrupts as that could result in a race condition.
> 
> Also drop an unnecessary space.
> 
> Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
> Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>

Reviewed-by: Sergey Shtylyov <s.shtylyov@omp.ru>

[...]

MBR, Sergey

Re: [PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[Sergey Shtylyov]

On 4/2/24 5:53 PM, Paul Barker wrote:

> The TX queue should be serviced each time the poll function is called,
> even if the full RX work budget has been consumed. This prevents
> starvation of the TX queue when RX bandwidth usage is high.
> 
> Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
> Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>

Reviewed-by: Sergey Shtylyov <s.shtylyov@omp.ru>

[...]

MBR, Sergey

Re: [PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[Ratheesh Kannoth]

On 2024-04-02 at 20:23:04, Paul Barker (paul.barker.ct@bp.renesas.com) wrote:
> The TX queue should be serviced each time the poll function is called,
> even if the full RX work budget has been consumed. This prevents
> starvation of the TX queue when RX bandwidth usage is high.
>
> Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
> Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
> ---
> Changes from v1:
>   * Use the correct 'Fixes' tag.
>   * Call the new variable 'unmask' and drop the unnecessary initializer,
>     as requested by Sergey.
>
>  drivers/net/ethernet/renesas/ravb_main.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
> index d1be030c8848..48803050abdb 100644
> --- a/drivers/net/ethernet/renesas/ravb_main.c
> +++ b/drivers/net/ethernet/renesas/ravb_main.c
> @@ -1324,12 +1324,12 @@ static int ravb_poll(struct napi_struct *napi, int budget)
>  	int q = napi - priv->napi;
>  	int mask = BIT(q);
>  	int quota = budget;
> +	bool unmask;
>
>  	/* Processing RX Descriptor Ring */
>  	/* Clear RX interrupt */
>  	ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0);
> -	if (ravb_rx(ndev, &quota, q))
> -		goto out;
> +	unmask = !ravb_rx(ndev, &quota, q);
>
>  	/* Processing TX Descriptor Ring */
AFAIU, TX is processed without any budget. This wont result in rx work starvation if
TX traffic is more ?

>  	spin_lock_irqsave(&priv->lock, flags);
> @@ -1339,6 +1339,9 @@ static int ravb_poll(struct napi_struct *napi, int budget)
>  	netif_wake_subqueue(ndev, q);
>  	spin_unlock_irqrestore(&priv->lock, flags);
>
> +	if (!unmask)
> +		goto out;
> +
>  	napi_complete(napi);
>
>  	/* Re-enable RX/TX interrupts */
>
> base-commit: ea2a1cfc3b2019bdea6324acd3c03606b60d71ad
> --
> 2.39.2
>

Re: [PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[Paul Barker]

[-- Attachment #1.1.1: Type: text/plain, Size: 2275 bytes --]

On 03/04/2024 04:39, Ratheesh Kannoth wrote:
> On 2024-04-02 at 20:23:04, Paul Barker (paul.barker.ct@bp.renesas.com) wrote:
>> The TX queue should be serviced each time the poll function is called,
>> even if the full RX work budget has been consumed. This prevents
>> starvation of the TX queue when RX bandwidth usage is high.
>>
>> Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
>> Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
>> ---
>> Changes from v1:
>>   * Use the correct 'Fixes' tag.
>>   * Call the new variable 'unmask' and drop the unnecessary initializer,
>>     as requested by Sergey.
>>
>>  drivers/net/ethernet/renesas/ravb_main.c | 7 +++++--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c
>> index d1be030c8848..48803050abdb 100644
>> --- a/drivers/net/ethernet/renesas/ravb_main.c
>> +++ b/drivers/net/ethernet/renesas/ravb_main.c
>> @@ -1324,12 +1324,12 @@ static int ravb_poll(struct napi_struct *napi, int budget)
>>  	int q = napi - priv->napi;
>>  	int mask = BIT(q);
>>  	int quota = budget;
>> +	bool unmask;
>>
>>  	/* Processing RX Descriptor Ring */
>>  	/* Clear RX interrupt */
>>  	ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0);
>> -	if (ravb_rx(ndev, &quota, q))
>> -		goto out;
>> +	unmask = !ravb_rx(ndev, &quota, q);
>>
>>  	/* Processing TX Descriptor Ring */
> AFAIU, TX is processed without any budget. This wont result in rx work starvation if
> TX traffic is more ?

Quoting the docs in Documentation/networking/napi.rst:

    The method takes a ``budget`` argument - drivers can process
    completions for any number of Tx packets but should only process up
    to ``budget`` number of Rx packets.

    skb Tx processing should happen regardless of the ``budget``

I take that to mean that the RX work budget running out should not stop
processing of TX packets.

Other drivers with a combined TX/RX poll function follow the same
pattern of processing TX packets regardless of RX budget exhaustion, for
example see ixgbe_poll() which processes TX packets first (in
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c).

Thanks,

-- 
Paul Barker

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3577 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

RE: Re: [PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[Ratheesh Kannoth]

> From: Paul Barker <paul.barker.ct@bp.renesas.com>
> > AFAIU, TX is processed without any budget. This wont result in rx work
> > starvation if TX traffic is more ?
> 
> Quoting the docs in Documentation/networking/napi.rst:
> 
>     The method takes a ``budget`` argument - drivers can process
>     completions for any number of Tx packets but should only process up
>     to ``budget`` number of Rx packets.
> 
>     skb Tx processing should happen regardless of the ``budget``
> 
> I take that to mean that the RX work budget running out should not stop
> processing of TX packets.
> 
> Other drivers with a combined TX/RX poll function follow the same pattern of
> processing TX packets regardless of RX budget exhaustion, for example see
> ixgbe_poll() which processes TX packets first (in
> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c).
ACK.  Thanks for pointing out. I had seen some drivers, TX NAPI routines exist to bring fairness to rx and tx. 
Just wondering, if there is a lot of tx traffic, would RX budget alone can do fairness among NAPI
Routines. 

Re: [PATCH v2 1/2] net: ravb: Always process TX descriptor ring

[patchwork-bot+netdevbpf]

Hello:

This series was applied to netdev/net.git (main)
by Paolo Abeni <pabeni@redhat.com>:

On Tue,  2 Apr 2024 15:53:04 +0100 you wrote:
> The TX queue should be serviced each time the poll function is called,
> even if the full RX work budget has been consumed. This prevents
> starvation of the TX queue when RX bandwidth usage is high.
> 
> Fixes: c156633f1353 ("Renesas Ethernet AVB driver proper")
> Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com>
> 
> [...]

Here is the summary with links:
  - [v2,1/2] net: ravb: Always process TX descriptor ring
    https://git.kernel.org/netdev/net/c/596a4254915f
  - [v2,2/2] net: ravb: Always update error counters
    https://git.kernel.org/netdev/net/c/101b76418d71

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html