The Linux Staging tree, what it is and is not.

The Linux Staging tree, what it is and is not.

[Greg KH]

Hi all,

It's been many months since the Linux Kernel developers conference, where the
linux-staging tree was discussed and role changed.  It turns out that people
are still a bit confused as to what the staging tree is for, and how it works.

So here's a short summary, I'm not going into the history or background here,
that's a much longer writeup that I'd be glad to do if people are interested.


The Linux Staging Tree, what it is and is not.

What the Linux Staging tree is:
  The Linux Staging tree (or just "staging" from now on) is used to hold
  stand-alone[1] drivers and filesystems that are not ready to be merged into
  the main portion of the Linux kernel tree at this point in time for various
  technical reasons.  It is contained within the main Linux kernel tree so
  that users can get access to the drivers much easier than before, and to
  provide a common place for the development to happen, resolving the
  "hundreds of different download sites" problem that most out-of-tree drivers
  have had in the past.

What the Linux Staging tree is not:
  The staging tree is not a place to dump code and run away, hoping that
  someone else will to the cleanup work for you.  While there are developers
  available and willing to do this kind of work, you need to get them to agree
  to "babysit" the code in order for it to be accepted.

Location and Development:
  The staging tree is now contained within the main Linux kernel source tree
  at the location drivers/staging/.  All development happens within the main
  kernel source tree, like any other subsystem within the kernel.  This means:
	- the linux-next tree contains the latest version of the staging tree,
	  with bugfixes that are about to be merged into Linus's tree, as well
	  as the patches that are to be merged into the next major kernel
	  release.
	- if you wish to do work on the staging tree, checkout the linux-next
	  tree and send patches based on that.

Runtime:
  When code from the staging tree is loaded in the kernel, a warning message
  will be printed to the kernel log saying:
    MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned.
  and the kernel will be tainted with the TAINT_CRAP flag.  This flag shows up
  in any kernel oops that might be produced after the driver has been loaded.

  Note, most kernel developers have expressed the warning that they will not
  work on bugs for when this taint flag has happened, so if you run into a
  kernel problem after loading such a module, please work to reproduce the
  issue without a staging module loaded in order to be able to get help from
  the community.

If anyone has any questions that this summary doesn't answer, please let me
know.

thanks,

greg k-h


[1] stand-alone means no changes needed in other places within the kernel.
There are some exceptions to this:
  - Documenation for the code can live in the Documentation/ directory
    although this isn't recommended.
  - Firmware for the drivers can live in the firmware/ directory, this is
    recommended.
  - Some symbols might need to be exported from the main portion of the kernel
    source tree.  This is acceptable as long as the relevant subsystem
    maintainer agrees with this export.

Re: The Linux Staging tree, what it is and is not.

[Robert Schwebel]

Greg,

On Wed, Mar 18, 2009 at 11:32:32AM -0700, Greg KH wrote:
> If anyone has any questions that this summary doesn't answer, please let me
> know.

Let me take this as an opportunity to discuss the epl (Ethernet
Powerlink) driver in staging. Taken aside the eye-cancer thing while
looking at the code (this could be fixed in the staging model), I
suppose the whole design is questionable.

We have ported similar commercial EPL stacks to linux-rt in the past,
and what we simply did is to implement the code completely in userspace,
ontop of a raw socket. It worked out pretty well and with reasonable
realtime performance. For the high level API, we used the process data
model provided by libpv [1], which gives you an abstraction that follows
both, automation-people's "process variable" concept and a modern object
oriented desing (in C, modeled after the POSIX object model for example
like pthread_create() & friends).

Doing this kind of network protocols in kernel space may be possible in
general, but IMHO the first thing that has to be done for a sane design
is:

	"Invent proper APIs and abstractions"

Unfortunately, industry people have somehow missed the last 10 years of
software engineering, so even recent ethernet fieldbus designs like
PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their
"abstraction" between the stack and the application. So writing
applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on
a variable-length global variable array. Welcome to software design of
the 80es. Nevertheless, "Object Dictionary" is a standard API for
industry people which cannot be discussed away, because automation
people are used to this terminology.

So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel,
let's start with the question what it buys us in comparism with a pure
userspace solution like outlined above.

One interesting thing could be shared APIs among this group of
fieldbusses, for example one ObjectDictionary API that works for
CANopen, EPL and EtherCAT. If we put that into the kernel, I'm wondering
how this API could look like; the problem with objdict is that it is
variable length, here's an example:

  /-------------> 16 bit index
 /    /---------> length
/--\ /  /--/----> data, addressed with 8 bit subindex
0000 02 aa bb
0001 00
0002 05 aa bb cc dd ee
....
FFFF ..

Using that "API" more or less works by poking the right magic bits to
several locations and things start happening.

So what would be the right kernel-userspace API for that? mmap() on a
device node does not work because of the variable length nature. On the
other hand, accessing the objdict shall be fast, because it will happen
in the fast path of a control application. Another possibility could be
sysfs, with the indices being the file names. That could at least be
used for manual access (development, debugging) or initial setup. Is
that fast enough? I don't know. Usually the initial objdict is worked
out once during the design phase of an application, so you'll be able to
load a whole objdict into the system on boot time. Later on, during
operational phase of the stack, netlink may be used to propagate events
and changes, at least until somebody is brave enough to write an
in-kernel dbus server :-)

The objdict is IMHO the most complicated part, because other
interactions like NMT, PDO and SDO could be easily done with sysfs +
netlink.

However, I'm still not convinced that this is the right way to go. What
you want to do for the main APIs is provide some nice userspace library
API for your applications. Why not put the whole "stack" into userspace
then and connect it to the kernel only via raw socket.

What do others think? Is it worth the effort to invent a proper objdict
API for linux?

rsc

[1] http://www.pengutronix.de/software/libpv/index_en.html
[2] http://en.wikipedia.org/wiki/CANopen#Object_dictionary
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: The Linux Staging tree, what it is and is not.

[Greg KH]

On Fri, Mar 20, 2009 at 01:58:08AM +0100, Robert Schwebel wrote:
> Greg,
> 
> On Wed, Mar 18, 2009 at 11:32:32AM -0700, Greg KH wrote:
> > If anyone has any questions that this summary doesn't answer, please let me
> > know.
> 
> Let me take this as an opportunity to discuss the epl (Ethernet
> Powerlink) driver in staging. Taken aside the eye-cancer thing while
> looking at the code (this could be fixed in the staging model), I
> suppose the whole design is questionable.

Sure it's questionable, and it's horrid code, but it is being used by
people and is the only public implementation of EPL on Linux that I can
find.

> We have ported similar commercial EPL stacks to linux-rt in the past,
> and what we simply did is to implement the code completely in userspace,
> ontop of a raw socket. It worked out pretty well and with reasonable
> realtime performance. For the high level API, we used the process data
> model provided by libpv [1], which gives you an abstraction that follows
> both, automation-people's "process variable" concept and a modern object
> oriented desing (in C, modeled after the POSIX object model for example
> like pthread_create() & friends).
> 
> Doing this kind of network protocols in kernel space may be possible in
> general, but IMHO the first thing that has to be done for a sane design
> is:
> 
> 	"Invent proper APIs and abstractions"

Agreed.

> Unfortunately, industry people have somehow missed the last 10 years of
> software engineering, so even recent ethernet fieldbus designs like
> PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their
> "abstraction" between the stack and the application. So writing
> applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on
> a variable-length global variable array. Welcome to software design of
> the 80es. Nevertheless, "Object Dictionary" is a standard API for
> industry people which cannot be discussed away, because automation
> people are used to this terminology.
> 
> So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel,
> let's start with the question what it buys us in comparism with a pure
> userspace solution like outlined above.

Are userspace solutions for this opensource today?

<snip>

> What do others think? Is it worth the effort to invent a proper objdict
> API for linux?

I suggest discussing this on netdev, that's the proper place for network
protocol discussions like this, right?

thanks,

greg k-h

Re: The Linux Staging tree, what it is and is not.

[Daniel Krüger]

Greg KH schrieb:
> On Fri, Mar 20, 2009 at 01:58:08AM +0100, Robert Schwebel wrote:
>> Greg,
>>
>> On Wed, Mar 18, 2009 at 11:32:32AM -0700, Greg KH wrote:
>>> If anyone has any questions that this summary doesn't answer, please let me
>>> know.
>> Let me take this as an opportunity to discuss the epl (Ethernet
>> Powerlink) driver in staging. Taken aside the eye-cancer thing while
>> looking at the code (this could be fixed in the staging model), I
>> suppose the whole design is questionable.
> 
> Sure it's questionable, and it's horrid code, but it is being used by
> people and is the only public implementation of EPL on Linux that I can
> find.

BTW, the implementation does not follow the kernel style guide, because 
our company has its own code style guide. But what is that you don't like?

>> We have ported similar commercial EPL stacks to linux-rt in the past,
>> and what we simply did is to implement the code completely in userspace,
>> ontop of a raw socket. It worked out pretty well and with reasonable
>> realtime performance. For the high level API, we used the process data
>> model provided by libpv [1], which gives you an abstraction that follows
>> both, automation-people's "process variable" concept and a modern object
>> oriented desing (in C, modeled after the POSIX object model for example
>> like pthread_create() & friends).
>>
>> Doing this kind of network protocols in kernel space may be possible in
>> general, but IMHO the first thing that has to be done for a sane design
>> is:
>>
>> 	"Invent proper APIs and abstractions"
> 
> Agreed.

I can only second that. But it is no easy task to find a common API for 
all field busses.

>> Unfortunately, industry people have somehow missed the last 10 years of
>> software engineering, so even recent ethernet fieldbus designs like
>> PowerLink or EtherCAT use the CANopen Object Dictionary [1] as their
>> "abstraction" between the stack and the application. So writing
>> applications in the EPL/CANopen/EtherCAT world works by PEEK and POKE on
>> a variable-length global variable array. Welcome to software design of
>> the 80es. Nevertheless, "Object Dictionary" is a standard API for
>> industry people which cannot be discussed away, because automation
>> people are used to this terminology.
>>
>> So if we want to do any kind of EPL/CANopen/EtherCAT work in the kernel,
>> let's start with the question what it buys us in comparism with a pure
>> userspace solution like outlined above.
> 
> Are userspace solutions for this opensource today?

No. But openPOWERLINK can be ported to userspace.

I would propose another solution.
Just leave the high priority tasks in the kernel which directly deal 
with the Ethernet frames and implement all other modules in userspace. 
Maybe we can enhance the standard network driver interface and directly 
connect the data link layer of POWERLINK to it. Additionally I would put 
the core NMT state machine and the PDO processing (encode and decode the 
frames with the process data from and into the process image) into the 
kernel. The process image can be accessed via mmap from userspace. All 
other modules like the object dictionary, SDO module, network management 
of the MN, etc can be put in userspace. I think the described kernel 
part can be done without the knowledge of an object dictionary at all. 
It's just a matter of "proper APIs".

> <snip>
> 
>> What do others think? Is it worth the effort to invent a proper objdict
>> API for linux?
> 
> I suggest discussing this on netdev, that's the proper place for network
> protocol discussions like this, right?

Ok, I will subscribe it.

cu,
   Daniel

--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz

Telefon  : +49 (0) 3661 6279 0
Fax      : +49 (0) 3661 6279 99
Email    : daniel.krueger@systec-electronic.com
Internet : http://www.systec-electronic.com

Managing Director   : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563

Re: The Linux Staging tree, what it is and is not.

[Robert Schwebel]

On Fri, Mar 20, 2009 at 09:34:29AM +0100, Daniel Krüger wrote:
> > Sure it's questionable, and it's horrid code, but it is being used by
> > people and is the only public implementation of EPL on Linux that I
> > can find.
>
> BTW, the implementation does not follow the kernel style guide,
> because  our company has its own code style guide. But what is that
> you don't like?

If you do your private stuff, you can do whatever you like. If you want
to bring something into the kernel, you have to follow the kernel style.

But it's not only a matter of coding style. Please let's not start with
implementation details as long as the overall design is unclear. So I
suggest that following Greg's advise and moving the discussion to netdev
is what we should do.

> > > Doing this kind of network protocols in kernel space may be
> > > possible in general, but IMHO the first thing that has to be done
> > > for a sane design is:
> > >
> > > 	"Invent proper APIs and abstractions"
> >
> > Agreed.
>
> I can only second that. But it is no easy task to find a common API
> for all field busses.

Finding a common API for all field busses is somethign we'll do on a
higher level in the OSADL Fieldbus Framework project, but in userspace.

What matters for the kernel is that, if we want to have things in
mainline, we must answer the question "what have these things in
common". It doesn't make any sense to invent infrastructure for things
that don't have anything in common. Kernel frameworks are all about
abstraction.

> > Are userspace solutions for this opensource today?
>
> No. But openPOWERLINK can be ported to userspace.

Sure.

> I would propose another solution. Just leave the high priority tasks
> in the kernel which directly deal with the Ethernet frames and
> implement all other modules in userspace.

So why not work completely ontop of raw sockets? What else do you need
in the kernel?

The only thing I can imagine is a fast packet switch that separates high
priority realtime packets from normal non-rt traffic in the network
stack. As far as I know, tglx has a patch for that.

Let's move the discussion to netdev.

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: The Linux Staging tree, what it is and is not.

[Greg KH]

On Fri, Mar 20, 2009 at 09:34:29AM +0100, Daniel Krüger wrote:
> Greg KH schrieb:
>> On Fri, Mar 20, 2009 at 01:58:08AM +0100, Robert Schwebel wrote:
>>> Greg,
>>>
>>> On Wed, Mar 18, 2009 at 11:32:32AM -0700, Greg KH wrote:
>>>> If anyone has any questions that this summary doesn't answer, please let 
>>>> me
>>>> know.
>>> Let me take this as an opportunity to discuss the epl (Ethernet
>>> Powerlink) driver in staging. Taken aside the eye-cancer thing while
>>> looking at the code (this could be fixed in the staging model), I
>>> suppose the whole design is questionable.
>> Sure it's questionable, and it's horrid code, but it is being used by
>> people and is the only public implementation of EPL on Linux that I can
>> find.
>
> BTW, the implementation does not follow the kernel style guide, because our 
> company has its own code style guide. But what is that you don't like?

It's not an issue of "not liking" it, it just doesn't match the kernel
style guide.

I'll go make up a patch today to fix the layout and lots of the
structure issues of the code to match our rules, so that people can then
start working on the real functionality much easier.

thanks,

greg k-h

Re: The Linux Staging tree, what it is and is not.

[Alan Cox]

> However, I'm still not convinced that this is the right way to go. What
> you want to do for the main APIs is provide some nice userspace library
> API for your applications. Why not put the whole "stack" into userspace
> then and connect it to the kernel only via raw socket.

- Security
- Compartmentalisation
- Sharing the device/interconnect
- Real time prioritisation

and a few others.

> What do others think? Is it worth the effort to invent a proper objdict
> API for linux?

Objdict appears to be a legacy wart and can stay where it belongs I
suspect - you might need to have a objdict -> 21st century conversion
library for the afflicted. We don't put specialised COBOL support in the
kernel either we let user space figure out how to cope with the funnies.

Alan

Re: The Linux Staging tree, what it is and is not.

[Robert Schwebel]

Hi Alan,

Thanks for your feedback! :-)

On Fri, Mar 20, 2009 at 10:01:46AM +0000, Alan Cox wrote:
> > However, I'm still not convinced that this is the right way to go. What
> > you want to do for the main APIs is provide some nice userspace library
> > API for your applications. Why not put the whole "stack" into userspace
> > then and connect it to the kernel only via raw socket.
>
> - Security

For industrial I/O? We are talking about transportation of analog and
digital I/O, relais, ADC input etc. The only part that I'd consider
security related is tunnel traffic; EPL and EtherCAT have the
possibiltiy to do ethernet-over-$TECHNOLOGY. In the userspace variant,
tunnel trafic would be pushed back into the stack via tun/tap.

> - Compartmentalisation

Yes, that's a good and valid point, IMHO. Also, from a political point
of view, it would be very good to have *the one and only* kernel way to
do these things, instead of myriards of incompatible userspace hacks.

> - Sharing the device/interconnect

Sharing the device works also with userspace, because the device is a
network card and the interface is raw ethernet frames. What do you mean
with interconnect?

> - Real time prioritisation

Would be done by giving the kernel task (with threaded interrupts) or
userspace task the right POSIX relatime priorities, so I don't see a
difference here.

> > What do others think? Is it worth the effort to invent a proper objdict
> > API for linux?
>
> Objdict appears to be a legacy wart

Don't tell that to the industrial networking people - they are very
proud of the fact that they use a "standardized application API", so
even the fanciest and newest industrial ethernet technologies are based
on that legacy wart objdict, and it is officially considered to be a
*feature*.

> and can stay where it belongs I suspect -

So where does it belong, kernel or userspace? I'd say userspace, but
then you can put the whole stack into userspace. If kernel, we would
need an objdict API to userspace...?

One thing that has to be considered is that messages often have to be
handled fast and under realtime constraints; for example, it must be
possible to hook up the userspace to certain changes in the objdict.
Something like:

  -------   ---------     --------------
    eth      objdict       process data
    can                        layer
  -------   ---------     --------------
     |          |                |
     | rx()     |                |
     |--------->|                |
     |          # write pdo into |
     |          # position in    |
     |          # objdict        |
     |          |                |
     |          |  pdo_changed() |
     |          |--------------->|
     |          |                # pull process data out of certain
     |          |                # position in the objdict and make
     |          |                # proper process variable objects
     |          |                #
     |          |                |

Note that what they are doing is to use the objdict as a binary config
space (think registers) to configure the whole network workflow; so if
for example somebody wants to send out a temperature every delta-t
seconds, he configures that by setting bits in the objdict, pushing the
process data (temperature) into other bits in the objdict and the stack
sends out can/eth frames with that data. It's a complete mixture of
ISO/OSI layers, because they have things like "what is to be sent when"
in the same binary blob space as "payload".

> you might need to have a objdict -> 21st century conversion library
> for the afflicted.

That's what we are working on in the OSADL Fieldbus Framework, in
userspace. The basic idea is to bring the fieldbusses together with the
mathematics developed in the Ptolemy II project [1], in a way that you
can do classic soft-PLC [2] / IEC61131 stuff as well as modern
heterogenous model based design (thing LabVIEW, Matlab/Simulink,
Rhapsody, ...).

> We don't put specialised COBOL support in the kernel either we let
> user space figure out how to cope with the funnies.

Industrial communication people are in some strange time loop that
started looping in the 80es, but didn't notice yet :-)

rsc

[1] http://ptolemy.berkeley.edu/ptolemyII/
[2] http://en.wikipedia.org/wiki/Programmable_logic_controller
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: The Linux Staging tree, what it is and is not.

[Alan Cox]

> > - Security
> 
> For industrial I/O? We are talking about transportation of analog and
> digital I/O, relais, ADC input etc. The only part that I'd consider
> security related is tunnel traffic; EPL and EtherCAT have the

Maybe in your case but if that analogue I/O controls parts of a power
plant or weapons system you really really don't want a worm or trojan
getting involved with it.

> > - Sharing the device/interconnect
> 
> Sharing the device works also with userspace, because the device is a
> network card and the interface is raw ethernet frames. What do you mean
> with interconnect?

Load two user space fieldbus stacks on the same network card (remembering
this could be done by accident)

> > - Real time prioritisation
> 
> Would be done by giving the kernel task (with threaded interrupts) or
> userspace task the right POSIX relatime priorities, so I don't see a
> difference here.

There may be multiple user space tasks with differing priorities beyond
the message priority. That in turn means you want them to have different
priorities to the scheduler which means you want them to be different
processes really.

> Don't tell that to the industrial networking people - they are very
> proud of the fact that they use a "standardized application API", so
> even the fanciest and newest industrial ethernet technologies are based
> on that legacy wart objdict, and it is officially considered to be a
> *feature*.

It's a standard yes, but so once was the horse and cart.

> then you can put the whole stack into userspace. If kernel, we would
> need an objdict API to userspace...?

It depends where you split the stack between kernel and user space. There
needs to be enough in the kernel to allow reasonably efficient
multi-process models but that can be a small part of the stack. The
Appletalk stack is a good example. Appletalk has a single address/port
system for all higher level protocols and the higher level protocols
don't interact much with each other. It's actually very elegant in that
way. One result of this is that the AF_APPLETALK kernel stack deals only
with low level packet routing and delivery. It doesn't have to care about
higher stuff to provide the needed separation and sharing. IP by
comparison has all sorts of interactions, multiple port addressing
schemes and ICMP interactions which mean the kernel has to hold the
entire TCP and UDP layers.

> One thing that has to be considered is that messages often have to be
> handled fast and under realtime constraints; for example, it must be
> possible to hook up the userspace to certain changes in the objdict.
> Something like:

One model would be

kernel demux by objdict id -> forwards packet to the right user

a completely and gloriously insane one might be to have the objdict in
mmap space as you suggest but with a futex on each objdict entry.

> Note that what they are doing is to use the objdict as a binary config
> space (think registers) to configure the whole network workflow; so if

I tend to think of it as a rather peculiar shared event space - much like
the device end of SNMP. I guess we should be thankful for small mercies
that it wasn't invented today and using XML ;)

None of which is to say that an entirely userspace stack might not be the
right model, but there are distinct appeals to the kernel doing some of
the demux work and separation of stuff.

I shall be interested to see what turns out best as there are other
busses where the design really expects to set up states and the stack to
do regular repeating.

Re: The Linux Staging tree, what it is and is not.

[Robert Schwebel]

On Fri, Mar 20, 2009 at 10:55:56AM +0000, Alan Cox wrote:
> > > - Security
> >
> > For industrial I/O? We are talking about transportation of analog and
> > digital I/O, relais, ADC input etc. The only part that I'd consider
> > security related is tunnel traffic; EPL and EtherCAT have the
>
> Maybe in your case but if that analogue I/O controls parts of a power
> plant or weapons system you really really don't want a worm or trojan
> getting involved with it.

right :) But is there any difference between the stack and the
application here? Applications must be able to set process variables, so
if you compromize them and write the wrong ones to the stack, your power
plant goes baaaang as well.

> > > - Sharing the device/interconnect
> >
> > Sharing the device works also with userspace, because the device is a
> > network card and the interface is raw ethernet frames. What do you mean
> > with interconnect?
>
> Load two user space fieldbus stacks on the same network card (remembering
> this could be done by accident)

ok

> > > - Real time prioritisation
> >
> > Would be done by giving the kernel task (with threaded interrupts) or
> > userspace task the right POSIX relatime priorities, so I don't see a
> > difference here.
>
> There may be multiple user space tasks with differing priorities beyond
> the message priority. That in turn means you want them to have different
> priorities to the scheduler which means you want them to be different
> processes really.

right, but that's the same, no matter if it runs in kernel or user
space.

> > Don't tell that to the industrial networking people - they are very
> > proud of the fact that they use a "standardized application API", so
> > even the fanciest and newest industrial ethernet technologies are based
> > on that legacy wart objdict, and it is officially considered to be a
> > *feature*.
>
> It's a standard yes, but so once was the horse and cart.

/me puts this into his cites file :)

> > then you can put the whole stack into userspace. If kernel, we would
> > need an objdict API to userspace...?
>
> It depends where you split the stack between kernel and user space. There
> needs to be enough in the kernel to allow reasonably efficient
> multi-process models but that can be a small part of the stack. The
> Appletalk stack is a good example. Appletalk has a single address/port
> system for all higher level protocols and the higher level protocols
> don't interact much with each other. It's actually very elegant in that
> way. One result of this is that the AF_APPLETALK kernel stack deals only
> with low level packet routing and delivery. It doesn't have to care about
> higher stuff to provide the needed separation and sharing. IP by
> comparison has all sorts of interactions, multiple port addressing
> schemes and ICMP interactions which mean the kernel has to hold the
> entire TCP and UDP layers.

So what "layers" do we have for CANopen, EtherCAT and PowerLink?

- the lower api is getting ethernet frames
- the stack itself is a state machine that pushes things to objdict
- objdict needs to notify applications on certain changes
- userspace needs to trigger state machine on certain objcict changes by
  the application

We can either put the state machine into the kernel or into userspace;
if userspace, the kernel API is already there (socket), if kernel we
would need an objdict API.

> One model would be
>
> kernel demux by objdict id -> forwards packet to the right user

How would a user attach to the "right" thing? Sounds like some
publish/subscribe pattern.

> a completely and gloriously insane one might be to have the objdict in
> mmap space as you suggest but with a futex on each objdict entry.

uuuuh :)

The problem with mmap is that the objdict is not "rectangular", so you
would need an mmapped blob + description of the variable length content,
plus access serialization. Sounds indeed gloriously insane :)

> > Note that what they are doing is to use the objdict as a binary
> > config space (think registers) to configure the whole network
> > workflow; so if
>
> I tend to think of it as a rather peculiar shared event space - much
> like the device end of SNMP. I guess we should be thankful for small
> mercies that it wasn't invented today and using XML ;)
>
> None of which is to say that an entirely userspace stack might not be
> the right model, but there are distinct appeals to the kernel doing
> some of the demux work and separation of stuff.

seconded, we'll put some brain into that idea ...

> I shall be interested to see what turns out best as there are other
> busses where the design really expects to set up states and the stack
> to do regular repeating.

That's not different here.

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: The Linux Staging tree, what it is and is not.

[Daniel Krüger]

Robert Schwebel schrieb:
> On Fri, Mar 20, 2009 at 10:55:56AM +0000, Alan Cox wrote:
>> It depends where you split the stack between kernel and user space. There
>> needs to be enough in the kernel to allow reasonably efficient
>> multi-process models but that can be a small part of the stack. The
>> Appletalk stack is a good example. Appletalk has a single address/port
>> system for all higher level protocols and the higher level protocols
>> don't interact much with each other. It's actually very elegant in that
>> way. One result of this is that the AF_APPLETALK kernel stack deals only
>> with low level packet routing and delivery. It doesn't have to care about
>> higher stuff to provide the needed separation and sharing. IP by
>> comparison has all sorts of interactions, multiple port addressing
>> schemes and ICMP interactions which mean the kernel has to hold the
>> entire TCP and UDP layers.
> 
> So what "layers" do we have for CANopen, EtherCAT and PowerLink?
> 
> - the lower api is getting ethernet frames
> - the stack itself is a state machine that pushes things to objdict
> - objdict needs to notify applications on certain changes
> - userspace needs to trigger state machine on certain objcict changes by
>   the application
> 
> We can either put the state machine into the kernel or into userspace;
> if userspace, the kernel API is already there (socket), if kernel we
> would need an objdict API.

My understanding of an object dictionary is that it is only a structured 
view to a set of configuration variables, process variables and 
diagnostic information. It could also be an XML file or a simple C 
struct. In our implementation the object dictionary is not necessarily 
the backing store of the data. The process variables for example are 
just linked into the objdict. The application or the process image 
provides the backing store. In the end we can find any appropriate 
structure to pass the configuration from userspace to the data link 
layer and the PDO module in the kernel.

The PDO module is in fact a programmable scatter-gather copy machine 
(like DMA) between process image and Ethernet frames and vice-versa. The 
information of what to copy to where comes from the object dictionary. 
But it is possible to extract this information from the objdict and pass 
it via generic structures (e.g. something like IO vectors) to the PDO 
module in kernel. The information flow of the configuration data is 
one-way only, from userspace to kernel. This is what I plan to implement 
in openPOWERLINK, because the copy information can also be passed to a 
special hardware like a DMA controller.

So you can implement the objdict in userspace and access the process 
image via mmap (e.g. libpv).

cu,
   Daniel

--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz

Telefon  : +49 (0) 3661 6279 0
Fax      : +49 (0) 3661 6279 99
Email    : daniel.krueger@systec-electronic.com
Internet : http://www.systec-electronic.com

Managing Director   : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563

Re: The Linux Staging tree, what it is and is not.

[Alan Cox]

> My understanding of an object dictionary is that it is only a structured 
> view to a set of configuration variables, process variables and 
> diagnostic information. It could also be an XML file or a simple C 
> struct. In our implementation the object dictionary is not necessarily 
> the backing store of the data. The process variables for example are 

It's also possible I think to look at the object dictionary as no
different from port numbers. The object identifiers tell you who owns the
data and thus who needs to be told. I admit to being a bit rusty on this.
My last near encounters with fieldbus type systems were some years ago
when I²IT was involved in EMUG and OLCHFA and I managed to keep out of
the blast radius of those ;)

> information of what to copy to where comes from the object dictionary. 
> But it is possible to extract this information from the objdict and pass 
> it via generic structures (e.g. something like IO vectors) to the PDO 
> module in kernel. The information flow of the configuration data is 
> one-way only, from userspace to kernel. This is what I plan to implement 
> in openPOWERLINK, because the copy information can also be passed to a 
> special hardware like a DMA controller.

Ok so your hardware in fact really is oriented around an mmap type
interface ?

Alan

Re: The Linux Staging tree, what it is and is not.

[david]

On Fri, 20 Mar 2009, Alan Cox wrote:

>> information of what to copy to where comes from the object dictionary.
>> But it is possible to extract this information from the objdict and pass
>> it via generic structures (e.g. something like IO vectors) to the PDO
>> module in kernel. The information flow of the configuration data is
>> one-way only, from userspace to kernel. This is what I plan to implement
>> in openPOWERLINK, because the copy information can also be passed to a
>> special hardware like a DMA controller.
>
> Ok so your hardware in fact really is oriented around an mmap type
> interface ?

what I understand from this thread is that the hardware is a standard 
ethernet card.

David Lang

Re: The Linux Staging tree, what it is and is not.

[Daniel Krüger]

david@lang.hm schrieb:
> On Fri, 20 Mar 2009, Alan Cox wrote:
> 
>>> information of what to copy to where comes from the object dictionary.
>>> But it is possible to extract this information from the objdict and pass
>>> it via generic structures (e.g. something like IO vectors) to the PDO
>>> module in kernel. The information flow of the configuration data is
>>> one-way only, from userspace to kernel. This is what I plan to implement
>>> in openPOWERLINK, because the copy information can also be passed to a
>>> special hardware like a DMA controller.
>>
>> Ok so your hardware in fact really is oriented around an mmap type
>> interface ?
> 
> what I understand from this thread is that the hardware is a standard 
> ethernet card.

It may be a standard Ethernet controller. But it could also be a 
controller with a special hardware acceleration.

cu,
   Daniel

--
SYS TEC electronic GmbH
August-Bebel-Str. 29
D-07973 Greiz

Telefon  : +49 (0) 3661 6279 0
Fax      : +49 (0) 3661 6279 99
Email    : daniel.krueger@systec-electronic.com
Internet : http://www.systec-electronic.com

Managing Director   : Dipl.-Phys. Siegmar Schmidt
Commercial registry : Amtsgericht Jena, HRB 205563

Re: The Linux Staging tree, what it is and is not.

[Robert Schwebel]

On Mon, Mar 23, 2009 at 10:03:23AM +0100, Daniel Krüger wrote:
>> what I understand from this thread is that the hardware is a standard
>> ethernet card.
>
> It may be a standard Ethernet controller. But it could also be a
> controller with a special hardware acceleration.

Industrial Ethernets or CANopen can also be implemented with hardware
solutions like a netX or an FPGA, which both would have shared memory to
the CPU plus a part of the stack in userspace; however, this is nothing
that should be abstracted with this driver infrastructure. It is an
issue for a more higher level interface, which is what we will provide
with the OSADL Fieldbus Framework:

+------------------------------------+
I            Application             I
+------------------------------------+

+------------------------------------+
I         Process Data Library       I
+------------------------------------+

+--------------------+ +-------------+
I   $FIELDBUS Stack  I I  Userspace  I
I                    I I    Driver   I
+--------------------+ +-------------+

+--------------------+ +-------------+
I AF_CAN / AF_PACKET I I     UIO     I
I     Socket         I I             I
+--------------------+ +-------------+ Userspace
                                        ___^___
+--------------------+ +-------------+     v
I    eth/can Card    I I UIO driver  I  Kernel
I with normal driver I I             I
+--------------------+ +-------------+

rsc
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: The Linux Staging tree, what it is and is not.

[Dave Airlie]

On Thu, Mar 19, 2009 at 4:32 AM, Greg KH <greg@kroah.com> wrote:
> Hi all,
>
> It's been many months since the Linux Kernel developers conference, where the
> linux-staging tree was discussed and role changed.  It turns out that people
> are still a bit confused as to what the staging tree is for, and how it works.
>
> So here's a short summary, I'm not going into the history or background here,
> that's a much longer writeup that I'd be glad to do if people are interested.
>
>
> The Linux Staging Tree, what it is and is not.
>
> What the Linux Staging tree is:
>  The Linux Staging tree (or just "staging" from now on) is used to hold
>  stand-alone[1] drivers and filesystems that are not ready to be merged into
>  the main portion of the Linux kernel tree at this point in time for various
>  technical reasons.  It is contained within the main Linux kernel tree so
>  that users can get access to the drivers much easier than before, and to
>  provide a common place for the development to happen, resolving the
>  "hundreds of different download sites" problem that most out-of-tree drivers
>  have had in the past.
>
> What the Linux Staging tree is not:
>  The staging tree is not a place to dump code and run away, hoping that
>  someone else will to the cleanup work for you.  While there are developers
>  available and willing to do this kind of work, you need to get them to agree
>  to "babysit" the code in order for it to be accepted.
>
> Location and Development:
>  The staging tree is now contained within the main Linux kernel source tree
>  at the location drivers/staging/.  All development happens within the main
>  kernel source tree, like any other subsystem within the kernel.  This means:
>        - the linux-next tree contains the latest version of the staging tree,
>          with bugfixes that are about to be merged into Linus's tree, as well
>          as the patches that are to be merged into the next major kernel
>          release.
>        - if you wish to do work on the staging tree, checkout the linux-next
>          tree and send patches based on that.
>
> Runtime:
>  When code from the staging tree is loaded in the kernel, a warning message
>  will be printed to the kernel log saying:
>    MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned.
>  and the kernel will be tainted with the TAINT_CRAP flag.  This flag shows up
>  in any kernel oops that might be produced after the driver has been loaded.
>
>  Note, most kernel developers have expressed the warning that they will not
>  work on bugs for when this taint flag has happened, so if you run into a
>  kernel problem after loading such a module, please work to reproduce the
>  issue without a staging module loaded in order to be able to get help from
>  the community.
>
> If anyone has any questions that this summary doesn't answer, please let me
> know.

What is the target audience for staging?

It doesn't fill the out-of-tree modules void as far as I can tell.

The model most users are stuck with are
a) distro kernel
b) hw not supported.

Distro kernel could be any revision from 2.6.15 or something crazy up until now.

Generally you can pull the out of tree modules zip file or repo, build
it against
the kernel you have installed and know works with all your other hw
and your nvidia
binary driver, and the out of tree people have wrapped all the API changes so
that you don't have to do much in theory for it to just build and
install on your
kernel.

Staging doesn't fulfill this role from my POV:
a) what sane distro will enable staging drivers? do they care about their users?
b) if your distro doesn't enable staging drivers you now have two options:
1. distro kernel rebuild with staging drivers enabled if new enough
distro kernel
2. kernel.org kernel build with staging drivers enabled.

For a normal user who just wants his hw to work, these options are possibly
most unappealing. They still want do just download something from out-of-tree
which does all the nice API break wrapping for them and is 1MB instead of 300MB.

So hence my target audience question.

Dave.

Re: The Linux Staging tree, what it is and is not.

[Greg KH]

On Fri, Mar 20, 2009 at 02:26:10PM +1000, Dave Airlie wrote:
> On Thu, Mar 19, 2009 at 4:32 AM, Greg KH <greg@kroah.com> wrote:
> > Hi all,
> >
> > It's been many months since the Linux Kernel developers conference, where the
> > linux-staging tree was discussed and role changed.  It turns out that people
> > are still a bit confused as to what the staging tree is for, and how it works.
> >
> > So here's a short summary, I'm not going into the history or background here,
> > that's a much longer writeup that I'd be glad to do if people are interested.
> >
> >
> > The Linux Staging Tree, what it is and is not.
> >
> > What the Linux Staging tree is:
> >  The Linux Staging tree (or just "staging" from now on) is used to hold
> >  stand-alone[1] drivers and filesystems that are not ready to be merged into
> >  the main portion of the Linux kernel tree at this point in time for various
> >  technical reasons.  It is contained within the main Linux kernel tree so
> >  that users can get access to the drivers much easier than before, and to
> >  provide a common place for the development to happen, resolving the
> >  "hundreds of different download sites" problem that most out-of-tree drivers
> >  have had in the past.
> >
> > What the Linux Staging tree is not:
> >  The staging tree is not a place to dump code and run away, hoping that
> >  someone else will to the cleanup work for you.  While there are developers
> >  available and willing to do this kind of work, you need to get them to agree
> >  to "babysit" the code in order for it to be accepted.
> >
> > Location and Development:
> >  The staging tree is now contained within the main Linux kernel source tree
> >  at the location drivers/staging/.  All development happens within the main
> >  kernel source tree, like any other subsystem within the kernel.  This means:
> >        - the linux-next tree contains the latest version of the staging tree,
> >          with bugfixes that are about to be merged into Linus's tree, as well
> >          as the patches that are to be merged into the next major kernel
> >          release.
> >        - if you wish to do work on the staging tree, checkout the linux-next
> >          tree and send patches based on that.
> >
> > Runtime:
> >  When code from the staging tree is loaded in the kernel, a warning message
> >  will be printed to the kernel log saying:
> >    MODULE_NAME: module is from the staging directory, the quality is unknown, you have been warned.
> >  and the kernel will be tainted with the TAINT_CRAP flag.  This flag shows up
> >  in any kernel oops that might be produced after the driver has been loaded.
> >
> >  Note, most kernel developers have expressed the warning that they will not
> >  work on bugs for when this taint flag has happened, so if you run into a
> >  kernel problem after loading such a module, please work to reproduce the
> >  issue without a staging module loaded in order to be able to get help from
> >  the community.
> >
> > If anyone has any questions that this summary doesn't answer, please let me
> > know.
> 
> What is the target audience for staging?

two different groups:
  - users who want to use their hardware
  - developers working on the code in a common area.

> It doesn't fill the out-of-tree modules void as far as I can tell.

Hm, I think it does.

> The model most users are stuck with are
> a) distro kernel
> b) hw not supported.
> 
> Distro kernel could be any revision from 2.6.15 or something crazy up
> until now.
> 
> Generally you can pull the out of tree modules zip file or repo, build
> it against the kernel you have installed and know works with all your
> other hw and your nvidia binary driver, and the out of tree people
> have wrapped all the API changes so that you don't have to do much in
> theory for it to just build and install on your kernel.

Um, no, I don't think you have looked at these out-of-tree modules much.

> Staging doesn't fulfill this role from my POV:
> a) what sane distro will enable staging drivers? do they care about
> their users?

If the distros look, most of these drivers were already included in
them, I've merged all of the Fedora and Ubuntu and openSUSE out-of-tree
drivers that were previously in their kernels.

So I'm pretty sure they will enable them, just to keep parity with what
they are currently shipping :)

openSUSE enables them for this very reason.

> b) if your distro doesn't enable staging drivers you now have two options:

> 1. distro kernel rebuild with staging drivers enabled if new enough
> distro kernel
> 2. kernel.org kernel build with staging drivers enabled.

That's up to a distro as to what they do.

> For a normal user who just wants his hw to work, these options are
> possibly most unappealing. They still want do just download something
> from out-of-tree which does all the nice API break wrapping for them
> and is 1MB instead of 300MB.

A lot of the work that I've done getting these drivers in to the tree,
is just finding all of the proper patches for the drivers, and getting
the code to build.  A "normal" user wouldn't be able to do that at all.

Take the PSB video driver as an example, just finding the proper patches
took weeks.

thanks,

greg k-h

Re: The Linux Staging tree, what it is and is not.

[Lubomir Rintel]

On Fri, 2009-03-20 at 14:26 +1000, Dave Airlie wrote:
> Staging doesn't fulfill this role from my POV:
> a) what sane distro will enable staging drivers? do they care about their users?

If the code there actually builds, I don't see a problem with packaging
the staging drivers into a -crap subpackage, or maybe blacklisting them
all by default, but saving the user who wants to cut his head off and
watch himself bleed from having to recompile the kernel package for
that.

-- 
Lubomir Rintel <lkundrak@v3.sk>