Adding a new platform

Adding a new platform

[David VomLehn]

I'm working to educate our management on the need to get our platform in the 
kernel mainline. I expect I will be asked to tell them how much work this is. 
What do we need to do to add a new MIPS platform?
--
David VomLehn

Re: Adding a new platform

[Ralf Baechle]

On Tue, Aug 19, 2008 at 11:01:30AM -0700, David VomLehn wrote:

> I'm working to educate our management on the need to get our platform in 
> the kernel mainline. I expect I will be asked to tell them how much work 
> this is. What do we need to do to add a new MIPS platform?

Basically a patchset that adds support for the base platform and all
necessary drivers.  Send that to the respective mailing lists and
maintainers (see MAINTAINERS file) and wait for the feedback.  Expect to
polish your patchset a few times until everybody is happy.  That's the
basic receipe.  See also Documentation/SubmittingPatches,
Documentation/SubmittingDrivers and Documentation/SubmitChecklist.

  Ralf

Re: Adding a new platform

[Paul Gortmaker]

On Tue, Aug 19, 2008 at 2:01 PM, David VomLehn <dvomlehn@cisco.com> wrote:
> I'm working to educate our management on the need to get our platform in the
> kernel mainline. I expect I will be asked to tell them how much work this
> is. What do we need to do to add a new MIPS platform?

Based on the phrase "educate our management"  -- I would strongly suggest you
have a look at Jonathan Corbet's document that describes the process and the
eventual value-add of having things in-kernel -- with an audience of non-kernel
hackers in mind.  I think this will really assist you in your efforts,
and I'm glad that
Jonathan put the time into this that he did.

His original post can be viewed here:

http://lkml.org/lkml/2008/7/29/363

or here:

http://lwn.net/Articles/291967/

Paul.

> --
> David VomLehn
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-embedded" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Adding a new platform

[vb]

On Tue, Aug 19, 2008 at 8:19 PM, Paul Gortmaker
<paul.gortmaker@gmail.com> wrote:
> On Tue, Aug 19, 2008 at 2:01 PM, David VomLehn <dvomlehn@cisco.com> wrote:
>> I'm working to educate our management on the need to get our platform in the
>> kernel mainline. I expect I will be asked to tell them how much work this
>> is. What do we need to do to add a new MIPS platform?
>
> Based on the phrase "educate our management"  -- I would strongly suggest you
> have a look at Jonathan Corbet's document that describes the process and the
> eventual value-add of having things in-kernel -- with an audience of non-kernel
> hackers in mind.  I think this will really assist you in your efforts,
> and I'm glad that
> Jonathan put the time into this that he did.
>
> His original post can be viewed here:
>
> http://lkml.org/lkml/2008/7/29/363
>
> or here:
>
> http://lwn.net/Articles/291967/
>

This indeed is a very interesting document. I can hardly agree with
the below point, however:

+- While kernel developers strive to maintain a stable interface to user
+  space, the internal kernel API is in constant flux.  The lack of a stable
+  internal interface is a deliberate design decision; it allows fundamental
+  improvements to be made at any time and results in higher-quality code.
+  But one result of that policy is that any out-of-tree code requires
+  constant upkeep if it is to work with new kernels.  Maintaining
+  out-of-tree code requires significant amounts of work just to keep that
+  code working.

so, say a developer submits a proprietary driver and it gets accepted.
Then some internal kernel interface gets changed. Who now has to
modify and retest the driver? Is it the person who introduces the
change (how can he even do this, as he does not have the proprietary
hardware available)? Or is this the original submitter - then where is
the benefit of saving on upkeep?

This constant change of the kernel innards is hardly a good selling
point, would you agree?

cheers,
/vb

Re: Adding a new platform

[Paul Gortmaker]

On Tue, Aug 19, 2008 at 11:57 PM, vb <vb@vsbe.com> wrote:
> On Tue, Aug 19, 2008 at 8:19 PM, Paul Gortmaker
> <paul.gortmaker@gmail.com> wrote:
>> On Tue, Aug 19, 2008 at 2:01 PM, David VomLehn <dvomlehn@cisco.com> wrote:
>>> I'm working to educate our management on the need to get our platform in the
>>> kernel mainline. I expect I will be asked to tell them how much work this
>>> is. What do we need to do to add a new MIPS platform?
>>
>> Based on the phrase "educate our management"  -- I would strongly suggest you
>> have a look at Jonathan Corbet's document that describes the process and the
>> eventual value-add of having things in-kernel -- with an audience of non-kernel
>> hackers in mind.  I think this will really assist you in your efforts,
>> and I'm glad that
>> Jonathan put the time into this that he did.
>>
>> His original post can be viewed here:
>>
>> http://lkml.org/lkml/2008/7/29/363
>>
>> or here:
>>
>> http://lwn.net/Articles/291967/
>>
>
> This indeed is a very interesting document. I can hardly agree with
> the below point, however:
>
> +- While kernel developers strive to maintain a stable interface to user
> +  space, the internal kernel API is in constant flux.  The lack of a stable
> +  internal interface is a deliberate design decision; it allows fundamental
> +  improvements to be made at any time and results in higher-quality code.
> +  But one result of that policy is that any out-of-tree code requires
> +  constant upkeep if it is to work with new kernels.  Maintaining
> +  out-of-tree code requires significant amounts of work just to keep that
> +  code working.
>
> so, say a developer submits a proprietary driver and it gets accepted.

Doesn't happen.  By design.  If the driver is proprietary then it is presumably
not meant for open distribution, and hence not compatible with GPL and
widespread distribution into 100,000 public git repositories.  So it won't
get submitted and it won't get accepted.

> Then some internal kernel interface gets changed. Who now has to
> modify and retest the driver? Is it the person who introduces the
> change (how can he even do this, as he does not have the proprietary
> hardware available)? Or is this the original submitter - then where is
> the benefit of saving on upkeep?

I'd suggest you feed google "stable API nonsense" and continue reading
from there if you are interested in the topic.

Paul.

>
> This constant change of the kernel innards is hardly a good selling
> point, would you agree?
>
> cheers,
> /vb
>

Re: Adding a new platform

[vb]

On Tue, Aug 19, 2008 at 9:29 PM, Paul Gortmaker
<paul.gortmaker@gmail.com> wrote:
> On Tue, Aug 19, 2008 at 11:57 PM, vb <vb@vsbe.com> wrote:
>>
>> so, say a developer submits a proprietary driver and it gets accepted.
>
> Doesn't happen.  By design.  If the driver is proprietary then it is presumably
> not meant for open distribution, and hence not compatible with GPL and
> widespread distribution into 100,000 public git repositories.  So it won't
> get submitted and it won't get accepted.
>

I guess 'proprietary' is not the right term then, how do you call a
driver which is not a secret and not a problem to release, but
controls some hardware present in only in certain devices of a certain
company.

Would such a driver be accepted? Wouldn't such a driver get stale
after a few kernel releases?

Re: Adding a new platform

[Haavard Skinnemoen]

vb <vb@vsbe.com> wrote:
> Would such a driver be accepted? Wouldn't such a driver get stale
> after a few kernel releases?

Yes, the kernel contains tons of such platform-specific drivers. No, it
won't get stale as long as someone who has access to the hardware is
willing to test a -rc kernel every once in a while.

You can't really expect to get away with no maintenance at all, but in
my experience, it takes much less effort to maintain a driver in-tree
than out-of-tree. It's also much easier to get help if it's in-tree.

Haavard

Re: Adding a new platform

[vb]

On Wed, Aug 20, 2008 at 1:10 AM, Haavard Skinnemoen
<haavard.skinnemoen@atmel.com> wrote:
> vb <vb@vsbe.com> wrote:
>> Would such a driver be accepted? Wouldn't such a driver get stale
>> after a few kernel releases?
>
> Yes, the kernel contains tons of such platform-specific drivers. No, it
> won't get stale as long as someone who has access to the hardware is
> willing to test a -rc kernel every once in a while.
>
> You can't really expect to get away with no maintenance at all, but in
> my experience, it takes much less effort to maintain a driver in-tree
> than out-of-tree. It's also much easier to get help if it's in-tree.
>

fair enough, maybe this paragraph should be added to the original document.

cheers,
/v

> Haavard
>

Re: Adding a new platform

[Charles Manning]

On Wednesday 20 August 2008 17:15:01 vb wrote:
> On Tue, Aug 19, 2008 at 9:29 PM, Paul Gortmaker
>
> <paul.gortmaker@gmail.com> wrote:
> > On Tue, Aug 19, 2008 at 11:57 PM, vb <vb@vsbe.com> wrote:
> >> so, say a developer submits a proprietary driver and it gets accepted.
> >
> > Doesn't happen.  By design.  If the driver is proprietary then it is
> > presumably not meant for open distribution, and hence not compatible with
> > GPL and widespread distribution into 100,000 public git repositories.  So
> > it won't get submitted and it won't get accepted.
>
> I guess 'proprietary' is not the right term then, how do you call a
> driver which is not a secret and not a problem to release, but
> controls some hardware present in only in certain devices of a certain
> company.
>
> Would such a driver be accepted? Wouldn't such a driver get stale
> after a few kernel releases?

In-tree is no silver bullet.

When people modify internal APIs they will likely fix anything that breaks 
compilation. However many things are more subtle than that and it is very 
easy to end up with a driver or other code that compiles but does not work 
properly.

Luckily APIs for drivers (the most common stuff that people work on) don't 
change that much, and the interfaces are reasonably clear. If you want some 
hell then try working on file systems :-).

Re: Adding a new platform

[David Woodhouse]

On Thu, 2008-08-21 at 15:02 +1200, Charles Manning wrote:
> 
> In-tree is no silver bullet.
> 
> When people modify internal APIs they will likely fix anything that breaks 
> compilation. However many things are more subtle than that and it is very 
> easy to end up with a driver or other code that compiles but does not work 
> properly.
> 
> Luckily APIs for drivers (the most common stuff that people work on) don't 
> change that much, and the interfaces are reasonably clear. If you want some 
> hell then try working on file systems :-).

File systems show an excellent example of why you're wrong, in fact. The
API for file systems changes quite a lot, and the people who make those
changes do tend to fix up all the in-tree file systems simultaneously,
-- and since they're doing it with a full understanding of the
implications of the change they're making, they usually manage to get
the change _right_.

I find it's more common to introduce subtle errors when the person
modifying the core API _hasn't_ also modified the driver/fs code which
uses that API. Which is what happens when you have out-of-tree code.

I'd rather remove my gonads with a rusty spoon than maintain an
out-of-tree file system.

-- 
dwmw2

Re: Adding a new platform

[Geert Uytterhoeven]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2095 bytes --]

On Thu, 21 Aug 2008, Charles Manning wrote:
> On Wednesday 20 August 2008 17:15:01 vb wrote:
> > On Tue, Aug 19, 2008 at 9:29 PM, Paul Gortmaker
> >
> > <paul.gortmaker@gmail.com> wrote:
> > > On Tue, Aug 19, 2008 at 11:57 PM, vb <vb@vsbe.com> wrote:
> > >> so, say a developer submits a proprietary driver and it gets accepted.
> > >
> > > Doesn't happen.  By design.  If the driver is proprietary then it is
> > > presumably not meant for open distribution, and hence not compatible with
> > > GPL and widespread distribution into 100,000 public git repositories.  So
> > > it won't get submitted and it won't get accepted.
> >
> > I guess 'proprietary' is not the right term then, how do you call a
> > driver which is not a secret and not a problem to release, but
> > controls some hardware present in only in certain devices of a certain
> > company.
> >
> > Would such a driver be accepted? Wouldn't such a driver get stale
> > after a few kernel releases?
> 
> In-tree is no silver bullet.
> 
> When people modify internal APIs they will likely fix anything that breaks 
> compilation. However many things are more subtle than that and it is very 
> easy to end up with a driver or other code that compiles but does not work 
> properly.

My experience with Linux/m68k tells me that +95% of the breakage is visible
by compiler warnings and errors. If it still compiles, it still works ;-)

> Luckily APIs for drivers (the most common stuff that people work on) don't 
> change that much, and the interfaces are reasonably clear. If you want some 
> hell then try working on file systems :-).

Really? So how come so few changes are needed to keep squashfs working?

With kind regards,

Geert Uytterhoeven
Software Architect

Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium

Phone:    +32 (0)2 700 8453
Fax:      +32 (0)2 700 8622
E-mail:   Geert.Uytterhoeven@sonycom.com
Internet: http://www.sony-europe.com/

A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010

Re: Adding a new platform

[David Woodhouse]

On Thu, 2008-08-21 at 10:46 +0200, Geert Uytterhoeven wrote:
> Really? So how come so few changes are needed to keep squashfs
> working?

It's read-only.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: Adding a new platform

[Phillip Lougher]

Geert Uytterhoeven wrote:
> On Thu, 21 Aug 2008, Charles Manning wrote:
> 
>> Luckily APIs for drivers (the most common stuff that people work on) don't 
>> change that much, and the interfaces are reasonably clear. If you want some 
>> hell then try working on file systems :-).
> 
> Really? So how come so few changes are needed to keep squashfs working?
> 

It only uses a relatively small subset of the VFS, and goes nowhere near 
the MTD subsystem.  Recently dusting off an MTD patch I wrote for 
Squashfs a couple of years ago, I found it had to be completely 
rewritten for modern kernels.

Having said that the bit of the VFS Squashfs is interested in has 
changed in each of 2.6.23, 2.6.24, 2.6.25, 2.6.26 and now 2.6.27, and so 
I wouldn't say it was immune anyway.

Phillip

Re: Adding a new platform

[Peter Korsgaard]

>>>>> "Phillip" == Phillip Lougher <phillip@lougher.demon.co.uk> writes:

Hi,

 Phillip> It only uses a relatively small subset of the VFS, and goes
 Phillip> nowhere near the MTD subsystem.  Recently dusting off an MTD
 Phillip> patch I wrote for Squashfs a couple of years ago, I found it
 Phillip> had to be completely rewritten for modern kernels.

Can I see it? I've recently pondered patching squashfs to talk
directly to MTD instead of through mtdblock as squashfs often is the
only thing using the block layer on embedded systems.

Compiling with CONFIG_BLOCK=n seems to save around 125kb here.

 Phillip> Having said that the bit of the VFS Squashfs is interested
 Phillip> in has changed in each of 2.6.23, 2.6.24, 2.6.25, 2.6.26 and
 Phillip> now 2.6.27, and so I wouldn't say it was immune anyway.

Yeah, but it has always been pretty easy to fixup.

-- 
Bye, Peter Korsgaard

Re: Adding a new platform

[Paul Mundt]

On Tue, Aug 19, 2008 at 08:57:59PM -0700, vb wrote:
> On Tue, Aug 19, 2008 at 8:19 PM, Paul Gortmaker
> <paul.gortmaker@gmail.com> wrote:
> > On Tue, Aug 19, 2008 at 2:01 PM, David VomLehn <dvomlehn@cisco.com> wrote:
> >> I'm working to educate our management on the need to get our platform in the
> >> kernel mainline. I expect I will be asked to tell them how much work this
> >> is. What do we need to do to add a new MIPS platform?
> >
> > Based on the phrase "educate our management"  -- I would strongly suggest you
> > have a look at Jonathan Corbet's document that describes the process and the
> > eventual value-add of having things in-kernel -- with an audience of non-kernel
> > hackers in mind.  I think this will really assist you in your efforts,
> > and I'm glad that
> > Jonathan put the time into this that he did.
> >
> > His original post can be viewed here:
> >
> > http://lkml.org/lkml/2008/7/29/363
> >
> > or here:
> >
> > http://lwn.net/Articles/291967/
> >
> 
> This indeed is a very interesting document. I can hardly agree with
> the below point, however:
> 
> +- While kernel developers strive to maintain a stable interface to user
> +  space, the internal kernel API is in constant flux.  The lack of a stable
> +  internal interface is a deliberate design decision; it allows fundamental
> +  improvements to be made at any time and results in higher-quality code.
> +  But one result of that policy is that any out-of-tree code requires
> +  constant upkeep if it is to work with new kernels.  Maintaining
> +  out-of-tree code requires significant amounts of work just to keep that
> +  code working.
> 
> so, say a developer submits a proprietary driver and it gets accepted.

This doesn't make any sense to begin with. It's not possible to merge a
proprietary driver in to the kernel. If you're talking about a GPLed
driver, then the word proprietary here is meaningless (ie, whether
hardware is widely available or not is not a concern as long as someone
is actively looking after the code). On the other hand, if you are
talking about an out-of-tree driver, then the maintenance burden is
purely on whoever is maintaining that.

> Then some internal kernel interface gets changed. Who now has to
> modify and retest the driver? Is it the person who introduces the
> change (how can he even do this, as he does not have the proprietary
> hardware available)? Or is this the original submitter - then where is
> the benefit of saving on upkeep?
> 
In the general case, interface changes are done carefully, and in-tree
users are converted over as to minimize breakage. Obviously whoever is
making the change is not going to be able to test each driver, so it
usually comes down to common sense. If there's something that's
potentially problematic for a given driver, then testing is solicited
from those that have a vested interest in the continued functionality of
said driver. Likewise, if the change goes in and breaks things, that's a
regression, and is usually reverted or quickly fixed once it's been
identified. People that habitually cause regressions will suddenly find
it much more difficult to get anything applied to the kernel, it's fairly
self-regulating.

People don't inherently go out of their way to break others code, but it
certainly does happen. The main thing is that we have a pretty good
process in place to handle those sorts of problems as they come up, so
there's generally no reason _not_ to keep things changing.

On the embedded side there is obviously the issue that some people will
only test things once or twice a year, but at that level of active
involvement, they may as well just do all of their things out-of-tree
instead.

> This constant change of the kernel innards is hardly a good selling
> point, would you agree?
> 
No, but there are others that might. Have you considered BSD?