[PATCH net] r8152: avoid the driver drops a lot of packets

[PATCH net] r8152: avoid the driver drops a lot of packets

[Hayes Wang]

Stop submitting rx, if the driver queue more than 256 packets.

If the hardware is more fast than the software, the driver would start
queuing the packets. And, the driver starts dropping the packets, if it
queues more than 1000 packets.

Increase the weight of NAPI could improve the situation. However, the
weight has been changed to 64, so we have to stop submitting rx when the
driver queues too many packets. Then,the device may send the pause frame
to slow down the receiving, when the FIFO of the device is full.

Fixes: cf74eb5a5bc8 ("eth: r8152: try to use a normal budget")
Signed-off-by: Hayes Wang <hayeswang@realtek.com>
---
 drivers/net/usb/r8152.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index 332c853ca99b..b5ed55938b1c 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -2484,10 +2484,6 @@ static int rx_bottom(struct r8152 *tp, int budget)
 			unsigned int pkt_len, rx_frag_head_sz;
 			struct sk_buff *skb;
 
-			/* limit the skb numbers for rx_queue */
-			if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000))
-				break;
-
 			pkt_len = le32_to_cpu(rx_desc->opts1) & RX_LEN_MASK;
 			if (pkt_len < ETH_ZLEN)
 				break;
@@ -2556,7 +2552,7 @@ static int rx_bottom(struct r8152 *tp, int budget)
 		}
 
 submit:
-		if (!ret) {
+		if (!ret && likely(skb_queue_len(&tp->rx_queue) < 256)) {
 			ret = r8152_submit_rx(tp, agg, GFP_ATOMIC);
 		} else {
 			urb->actual_length = 0;
-- 
2.41.0

Re: [PATCH net] r8152: avoid the driver drops a lot of packets

[Paolo Abeni]

Hello,

On Mon, 2023-09-04 at 20:17 +0800, Hayes Wang wrote:
> Stop submitting rx, if the driver queue more than 256 packets.
> 
> If the hardware is more fast than the software, the driver would start
> queuing the packets. And, the driver starts dropping the packets, if it
> queues more than 1000 packets.
> 
> Increase the weight of NAPI could improve the situation. However, the
> weight has been changed to 64, so we have to stop submitting rx when the
> driver queues too many packets. Then,the device may send the pause frame
> to slow down the receiving, when the FIFO of the device is full.
> 
> Fixes: cf74eb5a5bc8 ("eth: r8152: try to use a normal budget")
> Signed-off-by: Hayes Wang <hayeswang@realtek.com>
> ---
>  drivers/net/usb/r8152.c | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
> index 332c853ca99b..b5ed55938b1c 100644
> --- a/drivers/net/usb/r8152.c
> +++ b/drivers/net/usb/r8152.c
> @@ -2484,10 +2484,6 @@ static int rx_bottom(struct r8152 *tp, int budget)
>  			unsigned int pkt_len, rx_frag_head_sz;
>  			struct sk_buff *skb;
>  
> -			/* limit the skb numbers for rx_queue */
> -			if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000))
> -				break;
> -

Dropping this check looks dangerous to me. What if pause frames are
disabled on the other end or dropped? It looks like this would cause
unlimited memory consumption?!?

If this limit is not supposed to be reached under normal conditions,
perhaps is worthy changing it into a WARN_ON_ONCE()?

Thanks!

Paolo

RE: [PATCH net] r8152: avoid the driver drops a lot of packets

[Hayes Wang]

Paolo Abeni <pabeni@redhat.com>
> Sent: Tuesday, September 5, 2023 6:11 PM
[...]
> > -                     /* limit the skb numbers for rx_queue */
> > -                     if (unlikely(skb_queue_len(&tp->rx_queue) >=
> 1000))
> > -                             break;
> > -
> 
> Dropping this check looks dangerous to me. What if pause frames are
> disabled on the other end or dropped? It looks like this would cause
> unlimited memory consumption?!?

When the driver stops submitting rx, the driver wouldn't get any packet
from the device after the previous urbs which have been submitted return.
That is, skb_queue_len(&tp->rx_queue) wouldn't increase any more until
the driver starts submitting rx again.

Now, the driver stops submitting rx when the skb_queue_len more than 256,
so the check becomes redundant. The skb_queue_len has been limited less
than 1000.

Besides, if the flow control is disabled, the packets may be dropped by
the hardware when the FIFO of the device is full, after the driver stops
submitting rx.

Best Regards,
Hayes

Re: [PATCH net] r8152: avoid the driver drops a lot of packets

[Paolo Abeni]

On Tue, 2023-09-05 at 10:37 +0000, Hayes Wang wrote:
> Paolo Abeni <pabeni@redhat.com>
> > Sent: Tuesday, September 5, 2023 6:11 PM
> [...]
> > > -                     /* limit the skb numbers for rx_queue */
> > > -                     if (unlikely(skb_queue_len(&tp->rx_queue) >=
> > 1000))
> > > -                             break;
> > > -
> > 
> > Dropping this check looks dangerous to me. What if pause frames are
> > disabled on the other end or dropped? It looks like this would cause
> > unlimited memory consumption?!?
> 
> When the driver stops submitting rx, the driver wouldn't get any packet
> from the device after the previous urbs which have been submitted return.
> That is, skb_queue_len(&tp->rx_queue) wouldn't increase any more until
> the driver starts submitting rx again.
> 
> Now, the driver stops submitting rx when the skb_queue_len more than 256,
> so the check becomes redundant. The skb_queue_len has been limited less
> than 1000.

I'm sorry, I have a very superficial knowledge of the USB layer, but it
looks like that when such condition is reached, in the worst condition
there could be up to urbs in flight. AFAICS each of them carries a 16K
buffer, can be up to 10 standard-mtu packets - or much more small ones.

Setting an upper limits to the rx_queue still looks like a reasonable
safeguard.

> Besides, if the flow control is disabled, the packets may be dropped by
> the hardware when the FIFO of the device is full, after the driver stops
> submitting rx.

If the incoming rate exceeds the H/W processing capacity, packets are
dropped: that is expected and unavoidable.

Possibly exposing the root cause for such drops to user space via
appropriate stats would be useful.

Cheers,

Paolo

RE: [PATCH net] r8152: avoid the driver drops a lot of packets

[Hayes Wang]

Paolo Abeni <pabeni@redhat.com>
> Sent: Tuesday, September 5, 2023 7:21 PM
[...]
> I'm sorry, I have a very superficial knowledge of the USB layer, but it
> looks like that when such condition is reached, in the worst condition
> there could be up to urbs in flight. AFAICS each of them carries a 16K
> buffer, can be up to 10 standard-mtu packets - or much more small ones.
> 
> Setting an upper limits to the rx_queue still looks like a reasonable
> safeguard.

I think it is very hard to queue more than 1000 packets. The NAPI continues
consuming the queued packets. And, the hardware wouldn't complete all
urbs at one time. However, I would add WARN_ON_ONCE() to observe if 
any exception would occur.

> > Besides, if the flow control is disabled, the packets may be dropped by
> > the hardware when the FIFO of the device is full, after the driver stops
> > submitting rx.
> 
> If the incoming rate exceeds the H/W processing capacity, packets are
> dropped: that is expected and unavoidable.
> 
> Possibly exposing the root cause for such drops to user space via
> appropriate stats would be useful.

The number of packet which the device drops could be got through
ethtool.

Best Regards,
Hayes