Netlink for kernel<->user space communication?

Netlink for kernel<->user space communication?

[Arvid Brodin]

Hi.

I'm writing a kernel driver for the HSR protocol, a standard for high availability
networks. I want to send messages from the kernel to user space about broken network
links. I also want user space to be able to ask the kernel about its view of the status of
nodes on the network.

Netlink seems like a good tool for this. (Is it?)

But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
this seems a bit out of date, the kernel API description differs from today's kernel
implementation.)
Or do I use the "Kernel Connector" (Documentation/connector/connector.txt)?
Do I use libnetlink?
Or do I use libnl? (This seems to be actively maintained.)
Or are there more and even better ways to do this?

I'm having a case of information overload here and would be grateful for some guidelines.

Thanks,
Arvid Brodin
Enea Services Stockholm AB - since February 16 a part of Xdin in the Alten Group. Soon we
will be working under the common brand name Xdin. Read more at www.xdin.com.

Re: Netlink for kernel<->user space communication?

[Stephen Hemminger]

On Tue, 24 Apr 2012 23:52:34 +0000
Arvid Brodin <Arvid.Brodin@xdin.com> wrote:

> Hi.
> 
> I'm writing a kernel driver for the HSR protocol, a standard for high availability
> networks. I want to send messages from the kernel to user space about broken network
> links. I also want user space to be able to ask the kernel about its view of the status of
> nodes on the network.
> 
> Netlink seems like a good tool for this. (Is it?)

Yes.

> But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
> this seems a bit out of date, the kernel API description differs from today's kernel
> implementation.)

No. Your driver probably looks like a device so you should be
using rtnetlink messages.

> Or do I use the "Kernel Connector" (Documentation/connector/connector.txt)?
no.

> Do I use libnetlink?
if you extend iproute2 to support your link type, then yes,
but use the version inside iproute2 (rather than the older extracted libnetlink
in some distros).

> Or do I use libnl? (This seems to be actively maintained.)
If you aren't going to be in iproute2 then use libmnl. libnl does lots
of caching etc, which makes it good for monitoring and gui tools but a pain
for simple management.

Advice is free, code is what counts.

Re: Netlink for kernel<->user space communication?

[Arvid Brodin]

On Tue, 24 Apr 2012 16:57:55 -0700
Stephen Hemminger <shemminger@xxxxxxxxxx> wrote:
> On Tue, 24 Apr 2012 23:52:34 +0000
> Arvid Brodin <Arvid.Brodin@xxxxxxxx> wrote:
> 
>> Hi.
>> 
>> I'm writing a kernel driver for the HSR protocol, a standard for high availability
>> networks. I want to send messages from the kernel to user space about broken network
>> links. I also want user space to be able to ask the kernel about its view of the status of
>> nodes on the network.
>> 
>> Netlink seems like a good tool for this. (Is it?)
> 
> Yes.
> 
>> But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
>> this seems a bit out of date, the kernel API description differs from today's kernel
>> implementation.)
> 
> No. Your driver probably looks like a device so you should be
> using rtnetlink messages.

I'm already using rtnetlink messages to add and remove my device, which works fine (see
e.g. http://www.spinics.net/lists/netdev/msg192817.html - although I didn't think it
meaningful to include the iproute2 patch here, until the kernel part is ready).

The protocol specifies transmission of "supervision frames" every 2 seconds, e.g. to check
link integrity. Every such frame should be received from two directions in the ring - if
only one is received, then there is a link problem.

I'd like to notify user space about every such occurence. Is there a rtnetlink message
type that fits this? The stuff in rtnetlink.h seems to be mostly concerned with specific
user space commands (there is something called RTNLGRP_NOTIFY but I couldn't find any
instances of it being used in the kernel, nor any documentation).


>> Or do I use the "Kernel Connector" (Documentation/connector/connector.txt)?
> no.

Your reply didn't reach me for some reason - I found it just yesterday on spinics - and in
the meantime I've implemented the notification using the connector protocol... :-|


-- 
Arvid Brodin
Enea Services Stockholm AB - since February 16 a part of Xdin in the Alten Group. Soon we
will be working under the common brand name Xdin. Read more at www.xdin.com.

Re: Netlink for kernel<->user space communication?

[Stephen Hemminger]

On Mon, 7 May 2012 18:43:23 +0000
Arvid Brodin <Arvid.Brodin@xdin.com> wrote:

> On Tue, 24 Apr 2012 16:57:55 -0700
> Stephen Hemminger <shemminger@xxxxxxxxxx> wrote:
> > On Tue, 24 Apr 2012 23:52:34 +0000
> > Arvid Brodin <Arvid.Brodin@xxxxxxxx> wrote:
> > 
> >> Hi.
> >> 
> >> I'm writing a kernel driver for the HSR protocol, a standard for high availability
> >> networks. I want to send messages from the kernel to user space about broken network
> >> links. I also want user space to be able to ask the kernel about its view of the status of
> >> nodes on the network.
> >> 
> >> Netlink seems like a good tool for this. (Is it?)
> > 
> > Yes.
> > 
> >> But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
> >> this seems a bit out of date, the kernel API description differs from today's kernel
> >> implementation.)
> > 
> > No. Your driver probably looks like a device so you should be
> > using rtnetlink messages.
> 
> I'm already using rtnetlink messages to add and remove my device, which works fine (see
> e.g. http://www.spinics.net/lists/netdev/msg192817.html - although I didn't think it
> meaningful to include the iproute2 patch here, until the kernel part is ready).
> 
> The protocol specifies transmission of "supervision frames" every 2 seconds, e.g. to check
> link integrity. Every such frame should be received from two directions in the ring - if
> only one is received, then there is a link problem.

Why not just manipulate the carrier or operational state (see Documentation/networking/operstate)
and use the existing notification on link changes. If you don't get heartbeat then change
the state of the device to indicate lower device is down with set_operstate(), the necessary
link everts propgate back as netlink events.

> I'd like to notify user space about every such occurence. Is there a rtnetlink message
> type that fits this? The stuff in rtnetlink.h seems to be mostly concerned with specific
> user space commands (there is something called RTNLGRP_NOTIFY but I couldn't find any
> instances of it being used in the kernel, nor any documentation).
> 

I am trying to steer you to use existing API's because then existing programs and
infrastructure can deal with the new device type.

Re: Netlink for kernel<->user space communication?

[Arvid Brodin]

On 2012-05-08 00:33, Stephen Hemminger wrote:
> On Mon, 7 May 2012 18:43:23 +0000
> Arvid Brodin <Arvid.Brodin@xdin.com> wrote:
> 
>> On Tue, 24 Apr 2012 16:57:55 -0700
>> Stephen Hemminger <shemminger@xxxxxxxxxx> wrote:
>>> On Tue, 24 Apr 2012 23:52:34 +0000
>>> Arvid Brodin <Arvid.Brodin@xxxxxxxx> wrote:
>>>
>>>> Hi.
>>>>
>>>> I'm writing a kernel driver for the HSR protocol, a standard for high availability
>>>> networks. I want to send messages from the kernel to user space about broken network
>>>> links. I also want user space to be able to ask the kernel about its view of the status of
>>>> nodes on the network.
>>>>
>>>> Netlink seems like a good tool for this. (Is it?)
>>>
>>> Yes.
>>>
>>>> But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
>>>> this seems a bit out of date, the kernel API description differs from today's kernel
>>>> implementation.)
>>>
>>> No. Your driver probably looks like a device so you should be
>>> using rtnetlink messages.
>>
>> I'm already using rtnetlink messages to add and remove my device, which works fine (see
>> e.g. http://www.spinics.net/lists/netdev/msg192817.html - although I didn't think it
>> meaningful to include the iproute2 patch here, until the kernel part is ready).
>>
>> The protocol specifies transmission of "supervision frames" every 2 seconds, e.g. to check
>> link integrity. Every such frame should be received from two directions in the ring - if
>> only one is received, then there is a link problem.
> 
> Why not just manipulate the carrier or operational state (see Documentation/networking/operstate)
> and use the existing notification on link changes. If you don't get heartbeat then change
> the state of the device to indicate lower device is down with set_operstate(), the necessary
> link everts propgate back as netlink events.

With HSR, all nodes in the network ring can detect a link problem anywhere in the ring. So
I need a way to communicate link problems that does not concern the host's devices at all,
but rather the state of the network as a whole. A typical message might say "Frames from
node 01:23:45:67:89:AB is only received over Slave Interface 1!" This indicates a problem
since all frames should be received over both slave interfaces. The broken link can be
anywhere between this node and the indicated node. If user space is aware of the network
topology, it can figure out exactly where the damage is by looking at which nodes' frames
are received over which slave interface.

(Thanks for the operstates info though, I hadn't discovered IF_OPER_LOWERLAYERDOWN! I will
use it to indicate a local slave is down.)


>> I'd like to notify user space about every such occurence. Is there a rtnetlink message
>> type that fits this? The stuff in rtnetlink.h seems to be mostly concerned with specific
>> user space commands (there is something called RTNLGRP_NOTIFY but I couldn't find any
>> instances of it being used in the kernel, nor any documentation).
>>
> 
> I am trying to steer you to use existing API's because then existing programs and
> infrastructure can deal with the new device type.

I really appreciate that! I want to use existing API's as far as possible. That's why I
keep sending you all these questions. :)


-- 
Arvid Brodin
XDIN AB | Jan Stenbecks Torg 17 | SE-164 40 Kista | Sweden | xdin.com

Re: Netlink for kernel<->user space communication?

[Stephen Hemminger]

On Wed, 9 May 2012 23:32:08 +0000
Arvid Brodin <Arvid.Brodin@xdin.com> wrote:

> On 2012-05-08 00:33, Stephen Hemminger wrote:
> > On Mon, 7 May 2012 18:43:23 +0000
> > Arvid Brodin <Arvid.Brodin@xdin.com> wrote:
> > 
> >> On Tue, 24 Apr 2012 16:57:55 -0700
> >> Stephen Hemminger <shemminger@xxxxxxxxxx> wrote:
> >>> On Tue, 24 Apr 2012 23:52:34 +0000
> >>> Arvid Brodin <Arvid.Brodin@xxxxxxxx> wrote:
> >>>
> >>>> Hi.
> >>>>
> >>>> I'm writing a kernel driver for the HSR protocol, a standard for high availability
> >>>> networks. I want to send messages from the kernel to user space about broken network
> >>>> links. I also want user space to be able to ask the kernel about its view of the status of
> >>>> nodes on the network.
> >>>>
> >>>> Netlink seems like a good tool for this. (Is it?)
> >>>
> >>> Yes.
> >>>
> >>>> But do I use raw netlink? (Described here: http://www.linuxjournal.com/article/7356 - but
> >>>> this seems a bit out of date, the kernel API description differs from today's kernel
> >>>> implementation.)
> >>>
> >>> No. Your driver probably looks like a device so you should be
> >>> using rtnetlink messages.
> >>
> >> I'm already using rtnetlink messages to add and remove my device, which works fine (see
> >> e.g. http://www.spinics.net/lists/netdev/msg192817.html - although I didn't think it
> >> meaningful to include the iproute2 patch here, until the kernel part is ready).
> >>
> >> The protocol specifies transmission of "supervision frames" every 2 seconds, e.g. to check
> >> link integrity. Every such frame should be received from two directions in the ring - if
> >> only one is received, then there is a link problem.
> > 
> > Why not just manipulate the carrier or operational state (see Documentation/networking/operstate)
> > and use the existing notification on link changes. If you don't get heartbeat then change
> > the state of the device to indicate lower device is down with set_operstate(), the necessary
> > link everts propgate back as netlink events.
> 
> With HSR, all nodes in the network ring can detect a link problem anywhere in the ring. So
> I need a way to communicate link problems that does not concern the host's devices at all,
> but rather the state of the network as a whole. A typical message might say "Frames from
> node 01:23:45:67:89:AB is only received over Slave Interface 1!" This indicates a problem
> since all frames should be received over both slave interfaces. The broken link can be
> anywhere between this node and the indicated node. If user space is aware of the network
> topology, it can figure out exactly where the damage is by looking at which nodes' frames
> are received over which slave interface.
> 
> (Thanks for the operstates info though, I hadn't discovered IF_OPER_LOWERLAYERDOWN! I will
> use it to indicate a local slave is down.)

Sounds like a message that is specific to the protocol. Maybe just a log
message would suffice, or having a protocol specific event channel.

> >> I'd like to notify user space about every such occurence. Is there a rtnetlink message
> >> type that fits this? The stuff in rtnetlink.h seems to be mostly concerned with specific
> >> user space commands (there is something called RTNLGRP_NOTIFY but I couldn't find any
> >> instances of it being used in the kernel, nor any documentation).
> >>
> > 
> > I am trying to steer you to use existing API's because then existing programs and
> > infrastructure can deal with the new device type.
> 
> I really appreciate that! I want to use existing API's as far as possible. That's why I
> keep sending you all these questions. :)
> 
>