Is there no way to shared code with Linux and other OSes?

Is there no way to shared code with Linux and other OSes?

[Chad Reese]

Watching the discussion about Octeon patches submitted by Cavium
Networks, it seems apparent the majority of the problems simply come
from the fact that the code was written to be shared between multiple
operating systems. Code for programming the low level details of
hardware doesn't really change if the OS is Linux, VxWorks, BSD, or
something else. I've found it very depressing that most of the comments
basically come down to "this doesn't match the kernel coding standard,
change it". Obviously rewriting code for every coding standard and OS is
just a bug farm. Fixes will never get merged into all the rewrites.

Cavium can't be the first to want to share code. We'd like Octeon to be
well supported in the Linux kernel, but we'd also like other OSes to
work well too. There has to be some sort of middle ground here. Our base
"library" that is completely OS agnostic is actually license under the
BSD license to allow maximum portability between various OSes. What have
other people done before?

Through the discussion on the Octeon patches a number of bugs have been
uncovered and code has been improved. This part of the kernel submit
process is truly great. It just bothers me that so much needs to be
rewritten for arbitrary reasons.

For example, there has been lots of complaints that we use typedefs
throughout our code. Some people may not like them, but they have been
useful in the past. Some code used to use structures to reference chip
registers. Later due to new features, we found it necessary to change
the struct to a union with anonymous members. Because of the typedefs we
were able to change the fields for the new features without breaking
compatibility with existing code. If we'd used "struct" everywhere
instead of a typedef, all existing code would have to change for no
other reason except to substitute "union" for "struct". Not everyone has
 the freedom of the kernel programmers to ignore code outside of the
project.

So far the code submitted for Octeon is fairly trivial. The idea of
duplicating all of the network setup code for RGMII, GMII, MII, SGMII,
1000 base X, PICMG, XAUI, Higig, Higig2, etc is just plain scary. I'm
sure I forgot a few in this list. There has to be a better way.

Chad

Re: Is there no way to shared code with Linux and other OSes?

[Kevin D. Kissell]

[This should be good for some useless weekend flaming.]

Chad Reese wrote:
> Watching the discussion about Octeon patches submitted by Cavium
> Networks, it seems apparent the majority of the problems simply come
> from the fact that the code was written to be shared between multiple
> operating systems. Code for programming the low level details of
> hardware doesn't really change if the OS is Linux, VxWorks, BSD, or
> something else. I've found it very depressing that most of the comments
> basically come down to "this doesn't match the kernel coding standard,
> change it". Obviously rewriting code for every coding standard and OS is
> just a bug farm. Fixes will never get merged into all the rewrites.
>   
If one had a fixed list of OSes that one wanted to support, each of 
which had a stable set of coding standards, then in principle it might 
be possible to derive some lowest-common-denominator coding standards 
from the intersection of sets.  As you point out, the resulting 
constraints (no typedefs for Linux, no identifiers more than N 
characters for other environments, etc.) may directly with efficient 
coding and maintenance.  That's a trade-off you get to make versus 
maintaining multiple variants.
> Cavium can't be the first to want to share code. We'd like Octeon to be
> well supported in the Linux kernel, but we'd also like other OSes to
> work well too. There has to be some sort of middle ground here. Our base
> "library" that is completely OS agnostic is actually license under the
> BSD license to allow maximum portability between various OSes. What have
> other people done before?
>
> Through the discussion on the Octeon patches a number of bugs have been
> uncovered and code has been improved. This part of the kernel submit
> process is truly great. It just bothers me that so much needs to be
> rewritten for arbitrary reasons.
>   
A consistent coding style is, I think you'll agree, an aid to coding and 
maintenence in large-scale programming, and while the Linux kernel isn't 
really all that big as software systems go, it's big enough to warrant a 
consistent style.  What's disconcerting is the "feature creep" in the 
coding standard.  Typedefs weren't banned by Linux in the beginning, and 
there are legacy typedefs in the system for exactly the reasons why 
software engineers working in C have used them for generations.  At some 
point, if Linux isn't going to become the Latin liturgy of operating 
systems, the standard will need to move away from such arbitrary 
dogmatism.  The argument given for banning typedefs altogether is that 
nested typedefs are confusing to programmers.  I strongly suspect that 
there's a coding rule that would exclude the kinds of abuses that 
provoked the rule while allowing sensible use of typedefs for 
portability and future-proofing. But that's not going to happen any time 
real soon.
> For example, there has been lots of complaints that we use typedefs
> throughout our code. Some people may not like them, but they have been
> useful in the past. Some code used to use structures to reference chip
> registers. Later due to new features, we found it necessary to change
> the struct to a union with anonymous members. Because of the typedefs we
> were able to change the fields for the new features without breaking
> compatibility with existing code. If we'd used "struct" everywhere
> instead of a typedef, all existing code would have to change for no
> other reason except to substitute "union" for "struct". Not everyone has
>  the freedom of the kernel programmers to ignore code outside of the
> project.
>   
I had a similar experience of annoyance with Linux dogma years ago, so I 
very much sympathize with your reaction.  However, isn't there a less 
elegant but functional alternative, such as passing pointers to void 
around and casting to type as appropriate, that you could have used had 
you known in advance that the Linux priesthood would reject typedefs as 
heresy?

          Regards,

          Kevin K.

Re: Is there no way to shared code with Linux and other OSes?

[Geert Uytterhoeven]

On Sat, 22 Nov 2008, Kevin D. Kissell wrote:
> [This should be good for some useless weekend flaming.]

Yeah! ;-)

> Chad Reese wrote:
> to move away from such arbitrary dogmatism.  The argument given for banning
> typedefs altogether is that nested typedefs are confusing to programmers.  I

I thought the main reason was that you can't have forward declarations of
typedefs, while you can have for structs.

Hence

    typedef struct {
	...
	b_t *p;
    } a_t;

    typedef struct {
	...
	a_t *p;
    } b_t;

won't fly.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Is there no way to shared code with Linux and other OSes?

[Kevin D. Kissell]

Geert Uytterhoeven wrote:
> On Sat, 22 Nov 2008, Kevin D. Kissell wrote:
>   
>> [This should be good for some useless weekend flaming.]
>>     
>
> Yeah! ;-)
>
>   
>> Chad Reese wrote:
>>     
Don't blame Chad for this quote, it was me!
>> to move away from such arbitrary dogmatism.  The argument given for banning
>> typedefs altogether is that nested typedefs are confusing to programmers.  I
>>     
>
> I thought the main reason was that you can't have forward declarations of
> typedefs, while you can have for structs.
>   
That's a better argument than the one in the HTML version of 
Documentation/CodingStyle.txt that I had bookmarked (which was what I 
cited).  Interestingly, if I look at the *current* Linux 
Documentation/CodingStyle.txt for 2.6.28-rc6, the blanket interdiction 
of typedefs is no longer there!  Things *have* evolved, as I said they'd 
have to, to recognize 5 (a good Illuminati number) cases where typedefs 
are permitted.  Superficially, based on Chad's description (I admit that 
I haven't been reviewing his patches) the Cavium case would seem to fall 
into the first category. Is the MIPS Linux community now some kind of 
ultra-orthodox sub-sect of the Linux cult? ;o)

          Regards,

          Kevin K.

Re: Is there no way to shared code with Linux and other OSes?

[Ralf Baechle]

On Sat, Nov 22, 2008 at 04:44:52AM -0600, Kevin D. Kissell wrote:

>>   
>>> [This should be good for some useless weekend flaming.]
>>>     
>>
>> Yeah! ;-)

With the oil price down so far we gotta make some good use of that ;-)

> That's a better argument than the one in the HTML version of  
> Documentation/CodingStyle.txt that I had bookmarked (which was what I  
> cited).  Interestingly, if I look at the *current* Linux  
> Documentation/CodingStyle.txt for 2.6.28-rc6, the blanket interdiction  
> of typedefs is no longer there!  Things *have* evolved, as I said they'd  
> have to, to recognize 5 (a good Illuminati number) cases where typedefs  
> are permitted.  Superficially, based on Chad's description (I admit that  
> I haven't been reviewing his patches) the Cavium case would seem to fall  
> into the first category. Is the MIPS Linux community now some kind of  
> ultra-orthodox sub-sect of the Linux cult? ;o)

There was never a blanket interdiction of typedefs though it may seem to
and unfortunately scripts/checkpatch.pl also bitches about every typedef,
no matter what.  It was rather a restriction to only using typedefs for
simple types such as char, int or pointers but not on structs - otherwise
the definitions for u32 etc. would not have made it in.  In a 2002 OLS
paper giving a bit of a rationale for the Linux coding standard Greg KH
writes:

[...]
typedef is evil

typedef should not be used in naming any of your structures. Almost all main
kernel structures do not have a typedef to shorten their usage. This
includes the following:

	struct inode
	struct dentry
	struct file
	struct buffer_head
	struct user
	struct task_struct

Using typedef tries to hide the real type of a variable. There have been
records of some kernel code using typedefs nested up to 4 layers deep,
preventing the programmer from telling what type of variable they are really
using. This can easily cause very large structures to be accidentally
declared on the stack, or to be returned from functions if the programmer
does not realize the size of the structure.  typedef can also be used as a
crutch to keep from typing long structure denitions. If this is the case,
the structure names should be made shorter, according to the above listed
naming rules.  Never define a typedef to just signify a pointer to a
structure, as in the following example:

	typedef struct foo
	{
		int bar;
		int baz;
	} foo t, *pfoo_t;

This again hides the true type of the variable, and is using the name of
the variable type to define what is is (see the comment about Hungarian
notation previously.

Some examples of where typedef is badly used are in the include/raid/md*.h
files where every structure has a typedef assigned to it, and in the
drivers/acpi/include/*.h files, where a lot of the structures do not even
have a name assigned to them, only a typedef.

The only place that using typedef is acceptable, is in declaring function
prototypes. These can be difcult to type out every time, so declaring a
typedef for these is nice to do. An example of this is the bh_end_io_t
typedef which is used as a parameter in the init buffer() call. This is
defined in include/fs.h as:

	typedef void (bh_end_io_t)(struct buffer_head *bh, int uptodate);

[...]

The full paper is available at http://www.kroah.com/linux/talks/ols_2002_kernel_codingstyle_paper/codingstyle.ps

  Ralf

Re: Is there no way to shared code with Linux and other OSes?

[Chad Reese]

I appreciate the info about the typedefs, but I didn't really mean to
signal them out. They just happened to be an example I picked. The
fundamental issue is that we have a large number of files written using
a fairly common corporate coding standard (4 space indents, no tabs,
javadoc comments) that is used by a number of OSes. We generate our API
documentation from it using doxygen, so the comments at least attempt to
have useful info in them. In order to use this code in the kernel, we're
having to basically run it through the evil indent program and strip
quite a bit of stuff out. Obviously indent will make a mess of some
stuff, so lots of hand editing is needed. The end result is code that
has diverged from the original in such a way that diffs are basically
impossible.

We're already having issues with bugs between the different versions and
the kernel submit stuff has just started. Maybe the only solution is two
divergent code bases, but it just feels like the gun is pointed at my
foot and cocked. I really don't want to pull the trigger knowing what is
about to happen.

Obviously I don't target these comments at true Linux drivers. The CF
libata driver is obviously a Linux specific driver and should follow all
Linux standards. Code for bootloader communication and networks setup is
completely OS agnostic, so sharing it between systems makes sense.

Chad

Ralf Baechle wrote:
> On Sat, Nov 22, 2008 at 04:44:52AM -0600, Kevin D. Kissell wrote:
> 
>>>   
>>>> [This should be good for some useless weekend flaming.]
>>>>     
>>> Yeah! ;-)
> 
> With the oil price down so far we gotta make some good use of that ;-)
> 
>> That's a better argument than the one in the HTML version of  
>> Documentation/CodingStyle.txt that I had bookmarked (which was what I  
>> cited).  Interestingly, if I look at the *current* Linux  
>> Documentation/CodingStyle.txt for 2.6.28-rc6, the blanket interdiction  
>> of typedefs is no longer there!  Things *have* evolved, as I said they'd  
>> have to, to recognize 5 (a good Illuminati number) cases where typedefs  
>> are permitted.  Superficially, based on Chad's description (I admit that  
>> I haven't been reviewing his patches) the Cavium case would seem to fall  
>> into the first category. Is the MIPS Linux community now some kind of  
>> ultra-orthodox sub-sect of the Linux cult? ;o)
> 
> There was never a blanket interdiction of typedefs though it may seem to
> and unfortunately scripts/checkpatch.pl also bitches about every typedef,
> no matter what.  It was rather a restriction to only using typedefs for
> simple types such as char, int or pointers but not on structs - otherwise
> the definitions for u32 etc. would not have made it in.  In a 2002 OLS
> paper giving a bit of a rationale for the Linux coding standard Greg KH
> writes:
> 
> [...]
> typedef is evil
> 
> typedef should not be used in naming any of your structures. Almost all main
> kernel structures do not have a typedef to shorten their usage. This
> includes the following:
> 
> 	struct inode
> 	struct dentry
> 	struct file
> 	struct buffer_head
> 	struct user
> 	struct task_struct
> 
> Using typedef tries to hide the real type of a variable. There have been
> records of some kernel code using typedefs nested up to 4 layers deep,
> preventing the programmer from telling what type of variable they are really
> using. This can easily cause very large structures to be accidentally
> declared on the stack, or to be returned from functions if the programmer
> does not realize the size of the structure.  typedef can also be used as a
> crutch to keep from typing long structure denitions. If this is the case,
> the structure names should be made shorter, according to the above listed
> naming rules.  Never define a typedef to just signify a pointer to a
> structure, as in the following example:
> 
> 	typedef struct foo
> 	{
> 		int bar;
> 		int baz;
> 	} foo t, *pfoo_t;
> 
> This again hides the true type of the variable, and is using the name of
> the variable type to define what is is (see the comment about Hungarian
> notation previously.
> 
> Some examples of where typedef is badly used are in the include/raid/md*.h
> files where every structure has a typedef assigned to it, and in the
> drivers/acpi/include/*.h files, where a lot of the structures do not even
> have a name assigned to them, only a typedef.
> 
> The only place that using typedef is acceptable, is in declaring function
> prototypes. These can be difcult to type out every time, so declaring a
> typedef for these is nice to do. An example of this is the bh_end_io_t
> typedef which is used as a parameter in the init buffer() call. This is
> defined in include/fs.h as:
> 
> 	typedef void (bh_end_io_t)(struct buffer_head *bh, int uptodate);
> 
> [...]
> 
> The full paper is available at http://www.kroah.com/linux/talks/ols_2002_kernel_codingstyle_paper/codingstyle.ps
> 
>   Ralf

-- 

Chad Reese <kreese@caviumnetworks.com>
Cavium Networks
Phone: 650 - 623 - 7038
Cell: 321 - 438 - 7753

Re: Is there no way to shared code with Linux and other OSes?

[Ralf Baechle]

On Sat, Nov 22, 2008 at 11:08:05AM -0800, Chad Reese wrote:

> 	linux-mips@linux-mips.org
> Subject: Re: Is there no way to shared code with Linux and other OSes?
> Content-Type: text/plain; charset=ISO-8859-1

Adding linux-kernel since this is probably of more general interest than
just the linux-mips list.

> I appreciate the info about the typedefs, but I didn't really mean to
> signal them out. They just happened to be an example I picked.

Thus it received a specific answer.

> The
> fundamental issue is that we have a large number of files written using
> a fairly common corporate coding standard (4 space indents, no tabs,
> javadoc comments) that is used by a number of OSes. We generate our API
> documentation from it using doxygen, so the comments at least attempt to
> have useful info in them. In order to use this code in the kernel, we're
> having to basically run it through the evil indent program and strip
> quite a bit of stuff out. Obviously indent will make a mess of some
> stuff, so lots of hand editing is needed. The end result is code that
> has diverged from the original in such a way that diffs are basically
> impossible.
> 
> We're already having issues with bugs between the different versions and
> the kernel submit stuff has just started. Maybe the only solution is two
> divergent code bases, but it just feels like the gun is pointed at my
> foot and cocked. I really don't want to pull the trigger knowing what is
> about to happen.
> 
> Obviously I don't target these comments at true Linux drivers. The CF
> libata driver is obviously a Linux specific driver and should follow all
> Linux standards. Code for bootloader communication and networks setup is
> completely OS agnostic, so sharing it between systems makes sense.

I'll start off my answer with a bit of history.  A few years ago the Linux
codebase was growing at exponentional speed not last from contributions
from other projects and there were few standards in the Linux world.  As
the result the code base was highly inconsistent in many aspects,
including coding style and typedefs.  It made maintenance and even just
understanding the code base painful.  Then on a few occasions Linus
decreed certain standards such as code to be formatted to fit into 80
colums.  Other standards arose from a common understanding.

The primary purpose of all these standards, both written in the kernel
documentation and unwritten is to improve maintainability of the code and
as such there is little point in blind enforcing them even though the
noise generated by automated checkers such as checkpatch.pl encourages
that.  So take that with a grain of salt.

Re divergent code base - I see your issue but the problem is both ways.
Historic example is Broadcom which contributed the headers for the Sibyte
SOC family.  The headers are generated using inhouse tools just as yours
and we'd like to change them - but all those changes conflict with
Broadcom's bi-annual contribution which are bascially are versions
re-generated with their latest tools and from their latest RTL.

OS compatibility layers are another subject.  They simplify maintenance
for the one and make it harder for the other.  In general in the Linux
world we've made not so positive experience with such glue code layers,
so we tend to limit if not avoid their use.

Another factor is that usually long after the contributing company has
already forgotten about their code I'm the guy who still is stuck with it.
For many more years ...

  Ralf

Re: Is there no way to shared code with Linux and other OSes?

[Chad Reese]

Ralf Baechle wrote:
> I'll start off my answer with a bit of history.  A few years ago the Linux
> codebase was growing at exponentional speed not last from contributions
> from other projects and there were few standards in the Linux world.  As
> the result the code base was highly inconsistent in many aspects,
> including coding style and typedefs.  It made maintenance and even just
> understanding the code base painful.  Then on a few occasions Linus
> decreed certain standards such as code to be formatted to fit into 80
> colums.  Other standards arose from a common understanding.

I appreciate the history and agree that coding standards are a necessary
thing. To be clear, I'm not looking for some one to say "ok, you can put
your stuff in". I'm more looking for other people that have already had
this problem and may have come up with a better solution. Even something
a simple as a smarter indent program might be a good idea. I've always
felt that an automated formatter normally causes more harm than good,
but sometimes they are a necessary evil.

> Re divergent code base - I see your issue but the problem is both ways.
> Historic example is Broadcom which contributed the headers for the Sibyte
> SOC family.  The headers are generated using inhouse tools just as yours
> and we'd like to change them - but all those changes conflict with
> Broadcom's bi-annual contribution which are bascially are versions
> re-generated with their latest tools and from their latest RTL.

In principle I agree, but in the specifics for the Octeon code I have
trouble with this. Currently we put all hardware register definitions in
two files, one for bit typedefs and one for the corresponding physical
address. These two files contain nothing other than raw register
definitions and are generated from RTL. Lots of the feedback we've
gotten from the community want this broken up such that registers are
defined in smaller groups where they are used. This makes sense if
you're talking about stuff that is hand maintained. Across the various
Octeon chip there are something around 1500 hardware specific registers.
Nobody is going to maintain something like that by hand. I'm sure years
from now when someone is trying to figure out how to program some Octeon
feature having the register definitions already in the kernel will help
tremendously.

> OS compatibility layers are another subject.  They simplify maintenance
> for the one and make it harder for the other.  In general in the Linux
> world we've made not so positive experience with such glue code layers,
> so we tend to limit if not avoid their use.
> 
> Another factor is that usually long after the contributing company has
> already forgotten about their code I'm the guy who still is stuck with it.
> For many more years ...

The unfortunate facts of life is that this will likely happen again.
Luckily Linux is currently requested by our customers heavily, so it
will be supported better than other open source OSes. I'm just worried
that code fixes will not propagate to other OSes simply because it isn't
possible to share code. The Octeon FreeBSD port currently shares quite a
bit of low level Octeon code with Linux. If the current Octeon Linux
submit process continues, this will not be possible.

After spending quite a bit of time making sure the Octeon "hal" code
compiles and works cleanly under the four different mips abis, with
either a cavium compiler or a standard one for multiple OSes, I hate to
see that work be for nothing.

Ok, I'll stop whining now...

Chad