how to report network device errors to userspace?

how to report network device errors to userspace?

[Chris Friesen]

I'm curious about the proper way for a network device driver to report faults to
userspace.

The network devices don't seem to have /dev entries as a rule, so that makes it
difficult to use select/poll to alert a user app.

I haven't been able to find a standard interface for this--does nobody care
about this kind of thing?

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to report network device errors to userspace?

[Donald Becker]

On Wed, 19 Jun 2002, Chris Friesen wrote:

> I'm curious about the proper way for a network device driver to report
> faults to userspace.

What type of fault?

Errors are counted in /proc/net/dev.

> I haven't been able to find a standard interface for this--does nobody care
> about this kind of thing?

People very much care.  There are sufficient reporting mechanisms for
most errors -- it's much more consistent, orthogonal and thorough than
other device types.  Compare /proc/net/dev with the errors reported for
disks, serial, USB, keyboard...

-- 
Donald Becker				becker@scyld.com
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993

Re: how to report network device errors to userspace?

[Chris Friesen]

Donald Becker wrote:
> 
> On Wed, 19 Jun 2002, Chris Friesen wrote:
> 
> > I'm curious about the proper way for a network device driver to report
> > faults to userspace.
> 
> What type of fault?

I'm looking for asynchronous notification of events such as loss of ethernet
carrier or SONET LOS/LOF/LCD/RDI/AIS.  When the driver figures out (whether
interrupt or poll-based) that it's lost connectivity it then somehow notifies a
userspace app that something has happened, so the userspace app can deal with
it.

Currently the way we are doing it for ethernet is that the userspace app calls a
device ioctl() with SIOCGMIIREG multiple times per second, but it would be nice
to have the driver notify us asynchronously.

> People very much care.  There are sufficient reporting mechanisms for
> most errors -- it's much more consistent, orthogonal and thorough than
> other device types.  Compare /proc/net/dev with the errors reported for
> disks, serial, USB, keyboard...

Is it possible to select() on an entry in /proc/net becoming readable?  Is
reading from /proc/net and converting from ASCII faster than doing an ioctl() to
the driver and getting binary data?

Basically, I want a userspace app to be notified somehow by the kernel that a
link that I've expressed interest in has lost physical connectivity, and I want
to get that information as fast as possible after the event.

I've hacked a SONET driver to allow a userspace app to register its pid, which
then gets sent SIGUSR1 on critical events to tell it to use ioctl() to get the
actual information, but this is not very clean nor is it scalable.

Thanks,

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to report network device errors to userspace?

[Andi Kleen]

On Wed, Jun 19, 2002 at 05:19:14PM -0400, Chris Friesen wrote:
> Donald Becker wrote:
> > 
> > On Wed, 19 Jun 2002, Chris Friesen wrote:
> > 
> > > I'm curious about the proper way for a network device driver to report
> > > faults to userspace.
> > 
> > What type of fault?
> 
> I'm looking for asynchronous notification of events such as loss of ethernet
> carrier or SONET LOS/LOF/LCD/RDI/AIS.  When the driver figures out (whether
> interrupt or poll-based) that it's lost connectivity it then somehow notifies a
> userspace app that something has happened, so the userspace app can deal with
> it.
> 
> Currently the way we are doing it for ethernet is that the userspace app calls a
> device ioctl() with SIOCGMIIREG multiple times per second, but it would be nice
> to have the driver notify us asynchronously.

I would suggest sending a netlink message from the driver.  That's really
what netlink was designed for.

> 
> > People very much care.  There are sufficient reporting mechanisms for
> > most errors -- it's much more consistent, orthogonal and thorough than
> > other device types.  Compare /proc/net/dev with the errors reported for
> > disks, serial, USB, keyboard...
> 
> Is it possible to select() on an entry in /proc/net becoming readable?  Is

Yes, but it's very ugly to implement and not recommended.

> reading from /proc/net and converting from ASCII faster than doing an ioctl() to
> the driver and getting binary data?

It's quite slow. When you plan to do this often better go with some 
binary interface (experience has shown for other statistics that /proc
has a huge overhead) 


-Andi

Re: how to report network device errors to userspace?

[Chris Friesen]

Andi Kleen wrote:

> I would suggest sending a netlink message from the driver.  That's really
> what netlink was designed for.

Can you point me to a driver that's done this cleanly as an example to follow?

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how to report network device errors to userspace?

[Andi Kleen]

On Wed, Jun 19, 2002 at 05:44:05PM -0400, Chris Friesen wrote:
> Andi Kleen wrote:
> 
> > I would suggest sending a netlink message from the driver.  That's really
> > what netlink was designed for.
> 
> Can you point me to a driver that's done this cleanly as an example to follow?

Not exactly a driver, but you can e.g. look at the ip_queue netfilter 
module.

-Andi

Re: how to report network device errors to userspace?

[James Morris]

On Wed, 19 Jun 2002, Andi Kleen wrote:

> On Wed, Jun 19, 2002 at 05:44:05PM -0400, Chris Friesen wrote:
> > Andi Kleen wrote:
> > 
> > > I would suggest sending a netlink message from the driver.  That's really
> > > what netlink was designed for.
> > 
> > Can you point me to a driver that's done this cleanly as an example to follow?
> 
> Not exactly a driver, but you can e.g. look at the ip_queue netfilter 
> module.
> 

It's not a very clean example at this stage, unfortunately, as it contains 
some fairly ugly logic for maintaining unicast sessions.

If you don't specifically need unicast Netlink (which I don't think you 
do), try also looking at the tcpdiag & ctnetlink code.


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: how to report network device errors to userspace?

[jamal]

Before someone reinvents the wheel, did anyone actually bother following
the discussion that happened here on netdev:
http://marc.theaimsgroup.com/?t=101751064400001&r=1&w=2

If Stefan Rompf hasnt disapeared or is buried in work somewhere,
all pieces are already in place. thats what the device netcarrier state
is for.

cheers,
jamal

On Thu, 20 Jun 2002, James Morris wrote:

> On Wed, 19 Jun 2002, Andi Kleen wrote:
>
> > On Wed, Jun 19, 2002 at 05:44:05PM -0400, Chris Friesen wrote:
> > > Andi Kleen wrote:
> > >
> > > > I would suggest sending a netlink message from the driver.  That's really
> > > > what netlink was designed for.
> > >
> > > Can you point me to a driver that's done this cleanly as an example to follow?
> >
> > Not exactly a driver, but you can e.g. look at the ip_queue netfilter
> > module.
> >
>
> It's not a very clean example at this stage, unfortunately, as it contains
> some fairly ugly logic for maintaining unicast sessions.
>
> If you don't specifically need unicast Netlink (which I don't think you
> do), try also looking at the tcpdiag & ctnetlink code.
>
>
> - James
> --
> James Morris
> <jmorris@intercode.com.au>
>
>

Re: how to report network device errors to userspace?

[Jes Sorensen]

>>>>> "Andi" == Andi Kleen <ak@suse.de> writes:

Andi> On Wed, Jun 19, 2002 at 05:19:14PM -0400, Chris Friesen wrote:
>> Currently the way we are doing it for ethernet is that the
>> userspace app calls a device ioctl() with SIOCGMIIREG multiple
>> times per second, but it would be nice to have the driver notify us
>> asynchronously.

Andi> I would suggest sending a netlink message from the driver.
Andi> That's really what netlink was designed for.

Hmmm, the broader issue here is that we should aim for a generic
interface - if netlink is the way, maybe it would be worth documenting
it and moving towards getting it into more/all drivers.

I have had this question from other sides as well - hadn't thought of
netlink.

Cheers,
Jes

Re: how to report network device errors to userspace?

[Chris Friesen]

Jes Sorensen wrote:

> Hmmm, the broader issue here is that we should aim for a generic
> interface - if netlink is the way, maybe it would be worth documenting
> it and moving towards getting it into more/all drivers.

I agree with this.

Also, if we're going to make it generic, lets plan on using it for stuff other
than just ethernet--I'd like to have it available for atm/sonet cards as well.

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com