Packet loss when txqueuelen is zero

Packet loss when txqueuelen is zero

[Stefan Wahren]

Hi,

i'm new to Linux network driver development and currently i want to port 
the QCA7000 network driver to mainline [1]. I concentrate my tests on tx 
buffering since my last QCA7000 patch RFC [2]. Now i've found a test 
scenario which leads to packet loss:

host A          Powerline       host B
(QCA7000)       Ethernet        Ethernet
192.168.1.3     adaptor         192.168.1.5
|------------------|---------------|
      Homeplug         Ethernet
      10 Mbit          100 Mbit

1. Reduce the txqueuelen from 100 (default value) to 0
2. Run ping in flood mode on host A to host B

ping -c 200 -s 10000 -f 192.168.1.5

3. ping reports a high packet loss

Additional information:
- QCA7000 network driver has a tx ring size of 10 packets
- the packet loss doesn't appear when txqueuelen is 100

Here are my questions:

Is the packet loss a expected result for this scenario?
If not what could be the cause of the packet loss?

[1] - https://github.com/I2SE/qca7000/tree/linux-mainline
[2] - http://www.spinics.net/lists/netdev/msg280751.html

Re: Packet loss when txqueuelen is zero

[Eric Dumazet]

On Sat, 2014-06-28 at 12:59 +0200, Stefan Wahren wrote:
> Hi,
> 
> i'm new to Linux network driver development and currently i want to port 
> the QCA7000 network driver to mainline [1]. I concentrate my tests on tx 
> buffering since my last QCA7000 patch RFC [2]. Now i've found a test 
> scenario which leads to packet loss:
> 
> host A          Powerline       host B
> (QCA7000)       Ethernet        Ethernet
> 192.168.1.3     adaptor         192.168.1.5
> |------------------|---------------|
>       Homeplug         Ethernet
>       10 Mbit          100 Mbit
> 
> 1. Reduce the txqueuelen from 100 (default value) to 0
> 2. Run ping in flood mode on host A to host B
> 
> ping -c 200 -s 10000 -f 192.168.1.5
> 
> 3. ping reports a high packet loss
> 
> Additional information:
> - QCA7000 network driver has a tx ring size of 10 packets
> - the packet loss doesn't appear when txqueuelen is 100
> 
> Here are my questions:
> 
> Is the packet loss a expected result for this scenario?

Sure it is totally expected.

-f is a flood ping, and you remove ability to store packets in the Qdisc
(pifo_fast limit is device txqueuelen).

If you have txqueuelen = 100, then the socket used by ping will more
likely hit its SO_SNDBUF limit and ping will handle this properly (it
detects that a sendmsg() returns -1, errno = ENOBUF

If QCA7000 network driver has a tx ring size of 10 packets, you really
want a qdisc being able to store bursts.

If bufferbloat is your concern, you can switch pfifo_fast to fq_codel or
fq

tc qdisc replace dev eth0 root fq_codel  (or fq to get TCP pacing)

Re: Packet loss when txqueuelen is zero

[Stefan Wahren]

Am 28.06.2014 16:40, schrieb Eric Dumazet:
>>
>> Is the packet loss a expected result for this scenario?
>
> Sure it is totally expected.
>
> -f is a flood ping, and you remove ability to store packets in the Qdisc
> (pifo_fast limit is device txqueuelen).
>
> If you have txqueuelen = 100, then the socket used by ping will more
> likely hit its SO_SNDBUF limit and ping will handle this properly (it
> detects that a sendmsg() returns -1, errno = ENOBUF
>
> If QCA7000 network driver has a tx ring size of 10 packets, you really
> want a qdisc being able to store bursts.

Thanks for your explanation. So it's not a network driver bug.

>
> If bufferbloat is your concern, you can switch pfifo_fast to fq_codel or
> fq
>
> tc qdisc replace dev eth0 root fq_codel  (or fq to get TCP pacing)
>

But fq_codel requires BQL in the QCA7000 network driver. Right?

Re: Packet loss when txqueuelen is zero

[Eric Dumazet]

On Sun, 2014-06-29 at 15:51 +0200, Stefan Wahren wrote:

> But fq_codel requires BQL in the QCA7000 network driver. Right?

Not at all. 10 skb in TX ring is very small, BQL wont really help in
this case.

BQL is good for NIC with TSO support, mostly.

Re: Packet loss when txqueuelen is zero

[Stefan Wahren]

Hi Eric,

Am 30.06.2014 07:49, schrieb Eric Dumazet:
> On Sun, 2014-06-29 at 15:51 +0200, Stefan Wahren wrote:
>
>> But fq_codel requires BQL in the QCA7000 network driver. Right?
>
> Not at all. 10 skb in TX ring is very small, BQL wont really help in
> this case.
>
> BQL is good for NIC with TSO support, mostly.
>

Dave Täht made the suggestion to reduce the skbs to 2. But i'm not sure 
if it's a good solution, because the SPI overhead becomes bigger.

I have another idea. The QCA has an internal tx buffer something above 
3000 bytes. So it would be helpful if the tx ring stores the estimated 
byte size of all skbs. In the case this sum is so big that skb with a 
MTU (1500 byte) won't fit into the QCA buffer, the tx queue can be stopped.

I commited my implementation [1] in the morning.

[1] - 
https://github.com/I2SE/qca7000/commit/4539475a4ce1d0f5ab6ed111df9d2a9ab5f9bcc3