Re: Is there no way to shared code with Linux and other OSes?

Re: Is there no way to shared code with Linux and other OSes?

[Ralf Baechle]

On Sat, Nov 22, 2008 at 11:08:05AM -0800, Chad Reese wrote:

> 	linux-mips@linux-mips.org
> Subject: Re: Is there no way to shared code with Linux and other OSes?
> Content-Type: text/plain; charset=ISO-8859-1

Adding linux-kernel since this is probably of more general interest than
just the linux-mips list.

> I appreciate the info about the typedefs, but I didn't really mean to
> signal them out. They just happened to be an example I picked.

Thus it received a specific answer.

> The
> fundamental issue is that we have a large number of files written using
> a fairly common corporate coding standard (4 space indents, no tabs,
> javadoc comments) that is used by a number of OSes. We generate our API
> documentation from it using doxygen, so the comments at least attempt to
> have useful info in them. In order to use this code in the kernel, we're
> having to basically run it through the evil indent program and strip
> quite a bit of stuff out. Obviously indent will make a mess of some
> stuff, so lots of hand editing is needed. The end result is code that
> has diverged from the original in such a way that diffs are basically
> impossible.
> 
> We're already having issues with bugs between the different versions and
> the kernel submit stuff has just started. Maybe the only solution is two
> divergent code bases, but it just feels like the gun is pointed at my
> foot and cocked. I really don't want to pull the trigger knowing what is
> about to happen.
> 
> Obviously I don't target these comments at true Linux drivers. The CF
> libata driver is obviously a Linux specific driver and should follow all
> Linux standards. Code for bootloader communication and networks setup is
> completely OS agnostic, so sharing it between systems makes sense.

I'll start off my answer with a bit of history.  A few years ago the Linux
codebase was growing at exponentional speed not last from contributions
from other projects and there were few standards in the Linux world.  As
the result the code base was highly inconsistent in many aspects,
including coding style and typedefs.  It made maintenance and even just
understanding the code base painful.  Then on a few occasions Linus
decreed certain standards such as code to be formatted to fit into 80
colums.  Other standards arose from a common understanding.

The primary purpose of all these standards, both written in the kernel
documentation and unwritten is to improve maintainability of the code and
as such there is little point in blind enforcing them even though the
noise generated by automated checkers such as checkpatch.pl encourages
that.  So take that with a grain of salt.

Re divergent code base - I see your issue but the problem is both ways.
Historic example is Broadcom which contributed the headers for the Sibyte
SOC family.  The headers are generated using inhouse tools just as yours
and we'd like to change them - but all those changes conflict with
Broadcom's bi-annual contribution which are bascially are versions
re-generated with their latest tools and from their latest RTL.

OS compatibility layers are another subject.  They simplify maintenance
for the one and make it harder for the other.  In general in the Linux
world we've made not so positive experience with such glue code layers,
so we tend to limit if not avoid their use.

Another factor is that usually long after the contributing company has
already forgotten about their code I'm the guy who still is stuck with it.
For many more years ...

  Ralf

Re: Is there no way to shared code with Linux and other OSes?

[Chad Reese]

Ralf Baechle wrote:
> I'll start off my answer with a bit of history.  A few years ago the Linux
> codebase was growing at exponentional speed not last from contributions
> from other projects and there were few standards in the Linux world.  As
> the result the code base was highly inconsistent in many aspects,
> including coding style and typedefs.  It made maintenance and even just
> understanding the code base painful.  Then on a few occasions Linus
> decreed certain standards such as code to be formatted to fit into 80
> colums.  Other standards arose from a common understanding.

I appreciate the history and agree that coding standards are a necessary
thing. To be clear, I'm not looking for some one to say "ok, you can put
your stuff in". I'm more looking for other people that have already had
this problem and may have come up with a better solution. Even something
a simple as a smarter indent program might be a good idea. I've always
felt that an automated formatter normally causes more harm than good,
but sometimes they are a necessary evil.

> Re divergent code base - I see your issue but the problem is both ways.
> Historic example is Broadcom which contributed the headers for the Sibyte
> SOC family.  The headers are generated using inhouse tools just as yours
> and we'd like to change them - but all those changes conflict with
> Broadcom's bi-annual contribution which are bascially are versions
> re-generated with their latest tools and from their latest RTL.

In principle I agree, but in the specifics for the Octeon code I have
trouble with this. Currently we put all hardware register definitions in
two files, one for bit typedefs and one for the corresponding physical
address. These two files contain nothing other than raw register
definitions and are generated from RTL. Lots of the feedback we've
gotten from the community want this broken up such that registers are
defined in smaller groups where they are used. This makes sense if
you're talking about stuff that is hand maintained. Across the various
Octeon chip there are something around 1500 hardware specific registers.
Nobody is going to maintain something like that by hand. I'm sure years
from now when someone is trying to figure out how to program some Octeon
feature having the register definitions already in the kernel will help
tremendously.

> OS compatibility layers are another subject.  They simplify maintenance
> for the one and make it harder for the other.  In general in the Linux
> world we've made not so positive experience with such glue code layers,
> so we tend to limit if not avoid their use.
> 
> Another factor is that usually long after the contributing company has
> already forgotten about their code I'm the guy who still is stuck with it.
> For many more years ...

The unfortunate facts of life is that this will likely happen again.
Luckily Linux is currently requested by our customers heavily, so it
will be supported better than other open source OSes. I'm just worried
that code fixes will not propagate to other OSes simply because it isn't
possible to share code. The Octeon FreeBSD port currently shares quite a
bit of low level Octeon code with Linux. If the current Octeon Linux
submit process continues, this will not be possible.

After spending quite a bit of time making sure the Octeon "hal" code
compiles and works cleanly under the four different mips abis, with
either a cavium compiler or a standard one for multiple OSes, I hate to
see that work be for nothing.

Ok, I'll stop whining now...

Chad