rx_dropped count for USB ethernet interfaces

rx_dropped count for USB ethernet interfaces

[David Laight]

We are seeing the 'rx_dropped' count increasing on an USB ethernet interfaces
smsc95xx, but not on the e1000 interface connected to the same LAN.

Now I thought that rx_dropped was a count of the number of packets that the
MAC driver/hardware discarded - typically because the rx ring had no buffers.
This could include packets dropped in the receive stack due to other memory
limits.

However it looks as though Linux is also counting rx_dropped if the packet
can't be delivered to a protocol (at the end of __netif_receive_skb).

The LAN definitely has broadcast packets with an unknown 'ethertype',
I'd expect these to be silently discarded not counted as rx_dropped.

If I run tcpdump on the interface (even with a filter that passes nothing)
then rx_dropped doesn't change.

I think I've just worked out why I don't see these error counts
on the e1000 interface. I suspect that DHCP is getting a copy of every
packet (that can't help network performance?)

	David

Re: rx_dropped count for USB ethernet interfaces

[David Miller]

From: "David Laight" <David.Laight@ACULAB.COM>
Date: Wed, 25 Sep 2013 17:22:38 +0100

> However it looks as though Linux is also counting rx_dropped if the packet
> can't be delivered to a protocol (at the end of __netif_receive_skb).

This is expected and correct behavior.

> I suspect that DHCP is getting a copy of every packet (that can't
> help network performance?)

This is a long standing and well known issue.

The problem is that it used to be the case that before you put an IP
address on an interface, you couldn't even receive broadcast UDP
frames on a UDP socket.

So all DHCP implementations use AF_PACKET sockets in promiscuous mode
to deal with this.

These days you can actually receive broadcast frames on an interface
which hasn't had an IP assigned to it yet, due to some routing check
changes that were made a few years ago, but this support obviously not
present in older kernels.

Compound this with the pain that it would take to adjust all the
various DHCP implementations to not only support this, but have
the fallback code too.

RE: rx_dropped count for USB ethernet interfaces

[David Laight]

> From: David Miller [mailto:davem@davemloft.net]

> > Edited slightly...
> > We are seeing the 'rx_dropped' count increasing on ethernet interfaces.
> > 
> > Now I thought that rx_dropped was a count of the number of packets
> > that the MAC driver/hardware discarded - typically because the rx
> > ring had no buffers. This could include packets dropped in the
> > receive stack due to other memory limits.
> >
> > However it looks as though Linux is also counting rx_dropped if
> > the packet can't be delivered to a protocol.
> 
> This is expected and correct behavior.

It isn't exactly useful behaviour though.
Two different things are being counted together.

Packets dropped by the hardware (or by ENOMEM errors) are packets
that haven't been delivered to the protocol stack - so cause
some kind of protocol recovery actions.
If any of these are happening there is usually something wrong
with the system/network.

Packets for which there is no protocol are largely boring.
Especially if they are multicast packets.

A compromise might be to only count packets that are directed
to the interface's MAC address.
In this case there is at least some expectation that the system
should process the packet.

	David

Re: rx_dropped count for USB ethernet interfaces

[David Miller]

From: "David Laight" <David.Laight@ACULAB.COM>
Date: Thu, 26 Sep 2013 10:32:24 +0100

> It isn't exactly useful behaviour though.

In your opinion.

We have tracepoints for people who want more fine grained drops
in these kinds of situations.

Also, the behavior of this statistic has existed for more than a
decade so changing it is really not in the cards.  Therefore any
discussion about what would have been the best samentic to choose from
the beginning is moot.

RE: rx_dropped count for USB ethernet interfaces

[David Laight]

> From: "David Laight" <David.Laight@ACULAB.COM>
> Date: Thu, 26 Sep 2013 10:32:24 +0100
> 
> > It isn't exactly useful behaviour though.
> 
> In your opinion.
> 
> We have tracepoints for people who want more fine grained drops
> in these kinds of situations.
> 
> Also, the behavior of this statistic has existed for more than a
> decade so changing it is really not in the cards.  Therefore any
> discussion about what would have been the best samentic to choose from
> the beginning is moot.

I suspect that not many people have networks with broadcast packets
for non-IP protocols - so don't ever see rx_dropped being incremented
because there wasn't a protocol stack looking for the packet.

A similar problem will arise if promiscuous mode is enabled
by someone who only wants a specific protocol - eg a specific
ethertype or a single LLC SAP value.

	David