a maze of twisty stats, most different

a maze of twisty stats, most different

[Rick Jones]

It seems that every driver, when providing support for ethtool -S 
functionality, has considerable lattitude when it comes to the stats 
provided.  Clearly this is very nice for the driver writer(s) as it 
allows them to provide whatever stats they feel are most "natural" for 
their NIC(s) and name them as they see fit.

However :)

 From the standpoint of someone looking from the outside, say someone 
wanting to consume ethtool -S statistics, it seems to be a big jumble.

Might there be a way to bring those two camps together?  Is there 
already, with the same wide availabilty of ethtool, and I've just not 
seen it?

rick jones

Re: a maze of twisty stats, most different

[Chris Snook]

Rick Jones wrote:
> It seems that every driver, when providing support for ethtool -S 
> functionality, has considerable lattitude when it comes to the stats 
> provided.  Clearly this is very nice for the driver writer(s) as it 
> allows them to provide whatever stats they feel are most "natural" for 
> their NIC(s) and name them as they see fit.

This is deliberate.  Ethtool operates at a very primitive level, and generally 
does little or no translation between the hardware registers and userspace.

> However :)
> 
>  From the standpoint of someone looking from the outside, say someone 
> wanting to consume ethtool -S statistics, it seems to be a big jumble.
> 
> Might there be a way to bring those two camps together?  Is there 
> already, with the same wide availabilty of ethtool, and I've just not 
> seen it?

ifconfig comes to mind, for one.  What statistics are you looking for exactly?

	-- Chris

Re: a maze of twisty stats, most different

[Rick Jones]

Chris Snook wrote:
> Rick Jones wrote:
> 
>> It seems that every driver, when providing support for ethtool -S 
>> functionality, has considerable lattitude when it comes to the stats 
>> provided.  Clearly this is very nice for the driver writer(s) as it 
>> allows them to provide whatever stats they feel are most "natural" for 
>> their NIC(s) and name them as they see fit.
> 
> 
> This is deliberate.  Ethtool operates at a very primitive level, and 
> generally does little or no translation between the hardware registers 
> and userspace.

Noted.

> 
>> However :)
>>
>>  From the standpoint of someone looking from the outside, say someone 
>> wanting to consume ethtool -S statistics, it seems to be a big jumble.
>>
>> Might there be a way to bring those two camps together?  Is there 
>> already, with the same wide availabilty of ethtool, and I've just not 
>> seen it?
> 
> 
> ifconfig comes to mind, for one.  What statistics are you looking for 
> exactly?

This would be my strawman - in terms of content not necessarily format, 
and not all the things would be applicable I'm sure.  Still, it covers 
the basics which are (IMO) sufficint for initial troubleshooting.

$ lanadmin -g mibstats 0

                       LAN INTERFACE STATUS DISPLAY
                        Thu, Jun 28,2007  10:24:02

PPA Number                      = 0
Description                     = lan0 HP PCI 10/100Base-TX Core 
[100BASE-TX,FD,AUTO,TT=1500]
Type (value)                    = ethernet-csmacd(6)
MTU Size                        = 1500
Speed                           = 100000000
Station Address                 = 0x1083cf5502
Administration Status (value)   = up(1)
Operation Status (value)        = up(1)
Last Change                     = 678
Inbound Octets                  = 2816678451
Inbound Unicast Packets         = 5557984
Inbound Non-Unicast Packets     = 6857814
Inbound Discards                = 0
Inbound Errors                  = 0
Inbound Unknown Protocols       = 798022
Outbound Octets                 = 1833385875
Outbound Unicast Packets        = 5744993
Outbound Non-Unicast Packets    = 3845
Outbound Discards               = 0
Outbound Errors                 = 0
Outbound Queue Length           = 0
Specific                        = 655367

Ethernet-like Statistics Group

Index                           = 1
Alignment Errors                = 0
FCS Errors                      = 0
Single Collision Frames         = 0
Multiple Collision Frames       = 0
Deferred Transmissions          = 0
Late Collisions                 = 0
Excessive Collisions            = 0
Internal MAC Transmit Errors    = 0
Carrier Sense Errors            = 0
Frames Too Long                 = 0
Internal MAC Receive Errors     = 0

What I'm thinking of might be as simple as a convention that the first N 
stats each driver provides are something akin to the above - consistent 
names and semantics for the stats, and then a driver/NIC-specific 
portion following

There is indeed pretty decent overlap with ifconfig output:

[root@hpcpc107 netperf2_work]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:17:A4:AB:40:FB
           inet addr:16.89.84.107  Bcast:16.89.84.127  Mask:255.255.255.128
           inet6 addr: fe80::217:a4ff:feab:40fb/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:12448920 errors:0 dropped:0 overruns:0 frame:0
           TX packets:6075025 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:1000
           RX bytes:18505185007 (17.2 GiB)  TX bytes:2750351151 (2.5 GiB)
           Interrupt:55

The break-out of the stats is nice - "late collisions" in the lanadmin 
is quite helpful when looking for duplex mismatch.

I'm not sure if the "UP" and "RUNNING" are matched with administrative 
and operational state (what we want and what we have respectively).  The 
queue length in the lanadmin is the current queue length, not the limit. 
  Serves as a guesstimate as to the utilization of the NIC if one wants 
to abuse little's law and all that.

Mostly I'm looking for a bit broader set of link-level stats which would 
be reported consistenly regardless of NIC type.

rick jones

Re: a maze of twisty stats, most different

[David Stevens]

I think there's a more general problem that's a huge hassle. There are 
lots of
new SNMP MIB's, but no conventions (that I'm aware of, at least) to allow 
for
changes to the /proc entries that get them to applications. For example, 
the
route age data recently submitted. I agree that existing apps rely on 
these
and are generally very fragile, but in practice it means it isn't easy to 
do any
changes for RFC compliance with new MIBS, short of replicating existing 
data
in a new /proc entry along with the old one.

How do people feel about adding a MIB subtree in /proc that tracks MIB
updates with some basic rules:

1) all fields must be tagged with a label
2) all apps using it must match the label, and ignore anything they don't
        match

Then future additions to a row just mean adding a label (like route age),
and items like the one-per-line v6 statistics could add new ones easily, 
too.
And the existing MIB and non-MIB stats can be the same format, though
possibly derived from different data.

I've got a patch for doing ICMPMsgStats, which replaces the individual
ICMP counters now defined (and which are deprecated), but changing
/proc/net/snmp6 doesn't seem so wise, given how fragile some of these
apps are.

Defined device stats could live in that proposed tree, too, and with basic
rules for users would allow Linux-only useful extensions without breaking
the rules that apps using it should follow (and therefore without breaking 
the
apps). That's independent of ethtool extensions, but at least marginally 
related...

Comments?

                                                +-DLS

Re: a maze of twisty stats, most different

[David Miller]

From: David Stevens <dlstevens@us.ibm.com>
Date: Thu, 28 Jun 2007 11:17:51 -0700

> Comments?

I think sysfs is a better model for MIBs, the extensibility we
get for free since each SNMP MIB entry we want to add is simply
a new file.

I'd be quite thrilled to apply a patch which implemented this.

Re: a maze of twisty stats, most different

[Rick Jones]

David Miller wrote:
> From: David Stevens <dlstevens@us.ibm.com>
> Date: Thu, 28 Jun 2007 11:17:51 -0700
> 
> 
>>Comments?
> 
> 
> I think sysfs is a better model for MIBs, the extensibility we
> get for free since each SNMP MIB entry we want to add is simply
> a new file.
> 
> I'd be quite thrilled to apply a patch which implemented this.


OK, lets suppose this was done.  Assume we were speaking of a link-level 
stats  MIB,  would this then be something that did an ethtool-like call 
into each driver and so we are back to per-driver differences having to 
be dealt with by higher layers, or would we be having the drivers stick 
stats in multiple places?  Or are we back to wanting drivers to have a 
common set of stats they keep?

rick jones
unafraid to show his ignorance of how things work, and how :)

Re: a maze of twisty stats, most different

[Andi Kleen]

David Stevens <dlstevens@us.ibm.com> writes:

> I think there's a more general problem that's a huge hassle. There are 
> lots of
> new SNMP MIB's, but no conventions (that I'm aware of, at least) to allow 
> for
> changes to the /proc entries that get them to applications. 

Actually /proc/net/snmp and netstat -s are extensible. If you add a new
MIB or field there it should just show up in netstat -s. It won't know about
the text decoding that is done for the early traditional MIBs, but that
is already not there for most of the newer fields.

-Andi

Re: a maze of twisty stats, most different

[David Stevens]

ak@suse.de wrote on 06/29/2007 10:30:23 AM:

> David Stevens <dlstevens@us.ibm.com> writes:
> 
> > I think there's a more general problem that's a huge hassle. There are 

> > lots of
> > new SNMP MIB's, but no conventions (that I'm aware of, at least) to 
allow 
> > for
> > changes to the /proc entries that get them to applications. 
> 
> Actually /proc/net/snmp and netstat -s are extensible. If you add a new
> MIB or field there it should just show up in netstat -s. It won't know 
about
> the text decoding that is done for the early traditional MIBs, but that
> is already not there for most of the newer fields.

That works ok for some things, like new global counters, but some
items really fit best in existing files and the concern there is about
other uses of them beyond the standard tools.
        Examples:
-addition of route age in /proc/net/route and /proc/net/ipv6_route
-per-group data in /proc/net/igmp & igmp6
-per-interface MLD MIB info, which ought to go with other per-interface 
data

        I think everything that uses this kind of interface ought to do
label matching, so additional columns in a row (anywhere in the row)
would just be skipped/ignored by things that don't understand them,
and similarlarly for single-row tagged items. You can do that in scripts
with awk, but if existing items don't, they'll break.
        On the other hand, we don't want to create a new instance for
every addition or modification, so it seems the middle ground is to
go with new entries now and state that scripts and code that don't
match tags and handle changes in the format in the future are broken.
        We could dump everything in /proc/net/snmp{,6}, of course,
but I'm not sure it's a good idea to replicate all the routing and 
interface
data that is not there now just to get a few additional fields for those
items.

                                                +-DLS

Re: a maze of twisty stats, most different

[Andi Kleen]

> That works ok for some things, like new global counters, but some
> items really fit best in existing files and the concern there is about
> other uses of them beyond the standard tools.
>         Examples:
> -addition of route age in /proc/net/route and /proc/net/ipv6_route

Routing information belongs into netlink imho. The /proc setup
for it already only shows a small subset of it. Netlink is easily
extensible -- just add new headers.

> -per-group data in /proc/net/igmp & igmp6

Don't know.  But you can probably just add more fields there,
multicasting is not exactly something that a lot of people care about
so there are likely not many scripts that might get broken.

> -per-interface MLD MIB info, which ought to go with other per-interface 
> data

ethtool ?  It's also extensible, although you have to change
the userland. But I don't think there is a risk of someone's script
breaking.

> 
>         I think everything that uses this kind of interface ought to do
> label matching, so additional columns in a row (anywhere in the row)
> would just be skipped/ignored by things that don't understand them,
> and similarlarly for single-row tagged items. You can do that in scripts
> with awk, but if existing items don't, they'll break.

I would expect shell scripts to generally do netstat -s | ..., which
is easily matchable.


-Andi> 
>