RE: Re: drop counts

RE: Re: drop counts

[Brandeburg, Jesse]

I apologize for my misconfigured email client, this is my correct
address

PS machine rebuilds suck.

-----Original Message-----
From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org]
On Behalf Of Jesse Brandeburg



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id\x16492&op=click

Re: Re: drop counts

[P]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

Jesse Brandeburg wrote:
> 
> dropped: rnbc does not mean dropped, see above.  mpc means dropped.
> 
> fifo: mpc means the fifo overflowed, and the packet was dropped (the 
> only time we drop).
> 
> missed: this should also be mpc because it shows we missed the packet.

Wow, thanks! So there we were reporting drops prior
to 2.6.12 when there possibly were none!
I've confirmed with a traffic generator here that
rnbc does not include packets actually dropped.
It's interesting that at about 1Mpps sustained for 10s,
mpc was 548,888, while rnbc was 4,331,730 suggesting
that the nic was trying to DMA to the buffer at a
higher rate than the packet receive rate.

Anway recapping...

2.6.11 stats were mapped like:

packets = gprc
dropped = rnbc
fifo = mpc
missed = mpc

In 2.6.12 this changed to:

packets = gprc
dropped = mpc
fifo = mpc
missed = mpc

> now, its debatable if we should be reporting the same stat 3 times, but 
> i still think it is correct, and its how our hardware maps stats and the 
> best we can overlay onto linux's definitions as we interpret them.

I think there still is a problem. Notice that /proc/net/dev
reports rx_dropped + rx_missed_errors for the drop column,
and so will be double counting drops currently.
Therefore I think we should be doing:

packets = gprc
dropped = 0
fifo = rnbc
missed = mpc

I'm putting rnbc in fifo as it's not accumulated into
anything else in the kernel. Also it's good to have
it reported as one can see when there is pressure on
the e1000 driver buffer. This is useful for tuning its size
to protect against drops and latency.

thanks,
Pádraig.

[-- Attachment #2: e1000-drops.diff --]
[-- Type: application/x-texinfo, Size: 701 bytes --]

Re: Re: drop counts

[Jesse Brandeburg]

On Thu, 21 Jul 2005, P@draigBrady.com wrote:
> > dropped: rnbc does not mean dropped, see above.  mpc means dropped.
> >
> > fifo: mpc means the fifo overflowed, and the packet was dropped (the
> > only time we drop).
> >
> > missed: this should also be mpc because it shows we missed the packet.
> 
> Wow, thanks! So there we were reporting drops prior
> to 2.6.12 when there possibly were none!

yeah, that was kind of ugly.

> It's interesting that at about 1Mpps sustained for 10s,
> mpc was 548,888, while rnbc was 4,331,730 suggesting
> that the nic was trying to DMA to the buffer at a
> higher rate than the packet receive rate.

Not really that interesting because the NIC may have been trying multiple 
times internally to DMA the data (possibly as each packet was received) 
and failing each time.

> 2.6.11 stats were mapped like:
> 
> packets = gprc
> dropped = rnbc
> fifo = mpc
> missed = mpc
> 
> In 2.6.12 this changed to:
> 
> packets = gprc
> dropped = mpc
> fifo = mpc
> missed = mpc
> 
> > now, its debatable if we should be reporting the same stat 3 times, but
> > i still think it is correct, and its how our hardware maps stats and the
> > best we can overlay onto linux's definitions as we interpret them.
> 
> I think there still is a problem. Notice that /proc/net/dev
> reports rx_dropped + rx_missed_errors for the drop column,
> and so will be double counting drops currently.
> Therefore I think we should be doing:
> 
> packets = gprc
> dropped = 0
> fifo = rnbc
> missed = mpc

Double counting drops may be an issue, and I believe we should make that 
change.  but I don't think its the right thing to report rnbc in fifo 
errors (there was actually no error).

> I'm putting rnbc in fifo as it's not accumulated into
> anything else in the kernel. Also it's good to have
> it reported as one can see when there is pressure on
> the e1000 driver buffer. This is useful for tuning its size
> to protect against drops and latency.

Whenever I want the RNBC numbers I simply look into ethtool -S eth0, and 
that allows me to tune.  I see no reason to "reuse" the fifo errors stat 
with misleading data.

Once again, having clear guidelines on what these stats actually mean for 
drivers and when they should use them for ethernet would be *really* 
useful.

Jesse


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click

Re: Re: drop counts

[P]

Jesse Brandeburg wrote:
> On Thu, 21 Jul 2005, P@draigBrady.com wrote:
> 
>> In 2.6.12 this changed to:
>>
>> packets = gprc
>> dropped = mpc
>> fifo = mpc
>> missed = mpc
>>
>> I think there still is a problem. Notice that /proc/net/dev
>> reports rx_dropped + rx_missed_errors for the drop column,
>> and so will be double counting drops currently.
>> Therefore I think we should be doing:
>>
>> packets = gprc
>> dropped = 0
>> fifo = rnbc
>> missed = mpc
> 
> 
> Double counting drops may be an issue, and I believe we should make that 
> change.

great.

> but I don't think its the right thing to report rnbc in fifo 
> errors (there was actually no error).
> 
>> I'm putting rnbc in fifo as it's not accumulated into
>> anything else in the kernel. Also it's good to have
>> it reported as one can see when there is pressure on
>> the e1000 driver buffer. This is useful for tuning its size
>> to protect against drops and latency.
> 
> 
> Whenever I want the RNBC numbers I simply look into ethtool -S eth0, and 
> that allows me to tune.  I see no reason to "reuse" the fifo errors stat 
> with misleading data.

Fair enough if rnbc is actually reported.
ethtool -S doesn't report it with my driver,
but I see this stat has been added to the
latest driver at least.

So I agree on both points.

thanks!

-- 
Pádraig Brady - http://www.pixelbeat.org
--


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id\x16492&op=click