re: Kernel Releases

re: Kernel Releases

[Dan Kegel]

David Relson <relson@osagesoftware.com> wrote:
> With the recent problems, the working 
> versions tend to be the -pre1 or -pre2 releases, not the released 
> one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
> releases.  Here's how...
> 
> When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
> next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
> (as in release candidate).  A day or two with -rc1 will quickly show if it 
> has a show stopper.  If so, then the minor fixes (and nothing else) go into 
> -rc2.  A day or two ..., and either -rc3 appears or we have a stable 
> release and 2.4.16 is ready to be released.
> 
> Let's go the extra distance and have the releases be usable, stable 
> kernels!  It's what users want and it's within the abilities of the 
> developers to produce.  Let's do it :-)

Hear, hear!  

Also, Marcelo should get in touch with the nice folks at stp@osdlab.org
and find out what sort of regression tests they will be running on
his kernels.  Perhaps he can wait for their feedback on the -rcX
kernels before declaring them final.  (Not that they would have
found the 2.4.15 problem with their current tests, but they'll
catch a few things.)

- Dan

Re: Kernel Releases

[Nathan Dabney]

On Sat, Nov 24, 2001 at 09:37:38PM -0800, Dan Kegel wrote:
> David Relson <relson@osagesoftware.com> wrote:
> > With the recent problems, the working 
> > versions tend to be the -pre1 or -pre2 releases, not the released 
> > one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
> > releases.  Here's how...
> > 
> > When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
> > next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
> > (as in release candidate).  A day or two with -rc1 will quickly show if it 
> > has a show stopper.  If so, then the minor fixes (and nothing else) go into 
> > -rc2.  A day or two ..., and either -rc3 appears or we have a stable 
> > release and 2.4.16 is ready to be released.
> > 
> > Let's go the extra distance and have the releases be usable, stable 
> > kernels!  It's what users want and it's within the abilities of the 
> > developers to produce.  Let's do it :-)
> 
> Hear, hear!  
> 
> Also, Marcelo should get in touch with the nice folks at stp@osdlab.org
> and find out what sort of regression tests they will be running on
> his kernels.  Perhaps he can wait for their feedback on the -rcX
> kernels before declaring them final.  (Not that they would have
> found the 2.4.15 problem with their current tests, but they'll
> catch a few things.)
> 
> - Dan

We will be running all the available tests (until that list gets too large
to be possible) on each kernel the morning after it's released.  That's
including pre/test/flyingmonkey/whatever.  The test requests after a kernel
release are currently automatic.

The regression tests we have planned are the majority of the tests available
under the Linux Test Project (sourceforge:LTP).  We have not however
finished setting these up for automation.

Marcelo is also already working with the lab on non-stp related projects.

QA on the Linux Kernel will happen.  However just like everything else in the 
Linux world, it won't be what you are used to.  (read: automatic bug
submissions to the LK list will NOT happen.)

One potential issue for waiting for these tests to finish before moving on
to the next release is the fact that while all the tests are /requested/ the
morning after a kernel hits kernel.org, they may not finish for days.  This is
because of other tests in the queues from developers or tests that take 
longer than a single day to complete.  My thoughts on this tend towards
having a short set of tests that can be considered a "smoke test" on the
kernel to verify a short list of basic QA items before the major tests are
ordered against a kernel.  This short list would catch most compile/boot
issues as well as a limited range of driver problems and a decent number of
kernel component performance areas.  I see the potential to have the STP
going in the background from the development effort and only becoming
visible to the LK list when we find problems or other things worth
commenting on.  I don't see a benefit for the development effort to be
driven or controlled by anything the QA process does or doesn't do.

It's a constant game of catch up and verification.  But at least the tools
are getting better.

-Nathan

Re: Kernel Releases

[Keith Owens]

On Sun, 25 Nov 2001 01:25:07 -0800, 
Nathan Dabney <smurf@osdlab.org> wrote:
>We will be running all the available tests (until that list gets too large
>to be possible) on each kernel the morning after it's released.

Have you been following the kbuild 2.5 developments[1]?  Linus has
agreed that this change will go in early in the 2.5 cycle, that will
impact on all automated testing for 2.5.  There will be both good and
bad impacts, the bad is the initial changeover, the good is a much
cleaner build process and the ability to build multiple configurations
from a single source tree.

[1] http://sourceforge.net/projects/kbuild/

Re: Kernel Releases

[Phil Howard]

On Sun, Nov 25, 2001 at 09:24:21PM +1100, Keith Owens wrote:

| Have you been following the kbuild 2.5 developments[1]?  Linus has
| agreed that this change will go in early in the 2.5 cycle, that will
| impact on all automated testing for 2.5.  There will be both good and
| bad impacts, the bad is the initial changeover, the good is a much
| cleaner build process and the ability to build multiple configurations
| from a single source tree.

Some of us have this ability with our own stuff already.  I've been
doing automated multi-config builds since 2.0.  But it will be nice
to not have to track a moving target anymore.  There will be a bigger
initial bump for some of us, but hopefully all the features will be
in there.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam@ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------

Re: Kernel Releases

[Nathan Dabney]

On Sun, Nov 25, 2001 at 09:24:21PM +1100, Keith Owens wrote:
> On Sun, 25 Nov 2001 01:25:07 -0800, 
> Nathan Dabney <smurf@osdlab.org> wrote:
> >We will be running all the available tests (until that list gets too large
> >to be possible) on each kernel the morning after it's released.
> 
> Have you been following the kbuild 2.5 developments[1]?  Linus has
> agreed that this change will go in early in the 2.5 cycle, that will
> impact on all automated testing for 2.5.  There will be both good and
> bad impacts, the bad is the initial changeover, the good is a much
> cleaner build process and the ability to build multiple configurations
> from a single source tree.
> 
> [1] http://sourceforge.net/projects/kbuild/

Yes, we will be able to add another method for (auto) kernel configuration when
this becomes an issue (hopefully RSN).

Currently the method we use to "sanitize" submitted kernel configs for our
hardware is as reliable as you can get under CML1.  It doesn't however, give
us a good way to build multiple configurations.

-Nathan

Re: Kernel Releases

[vda]

On Monday 26 November 2001 16:45, Mike Galbraith wrote:
> On Mon, 26 Nov 2001, vda wrote:
> > On Monday 26 November 2001 13:02, Rik van Riel wrote:
> > > On Mon, 26 Nov 2001, Ian Stirling wrote:
> > > > However, I for one never run a -pre kernel.
> > > >
> > > > I don't run -pre, because rightly or wrongly, I've got the impression
> > > > that these get even less testing than releases.
> > >
> > > I think the opening sentence of your email states
> > > the reason for that pretty well.
> >
> > The only way we can get good testing for new kernels is to stop using
> > -preN prefix in development branch (2.5.x). Just increment that 'x'.
>
> That won't change anything except the number on the kernel.  The people
> who you're trying to turn into bleeding edge testers (those who stay a
> little behind [bignum]) will continue to ride the curve at the point of
> their choosing.

Yes, but they can't tell which 2.5.x is more stable just from version number.
This way Linus will get better test coverage in 2.5.x.

Those who need stability can read lkml and figure out which 2.5.x was 
'glitchless' :-) or stick with 2.4.x
--
vda

Re: Kernel Releases

[Mike Galbraith]

On Tue, 27 Nov 2001, vda wrote:

> On Monday 26 November 2001 16:45, Mike Galbraith wrote:
> > On Mon, 26 Nov 2001, vda wrote:
> > > On Monday 26 November 2001 13:02, Rik van Riel wrote:
> > > > On Mon, 26 Nov 2001, Ian Stirling wrote:
> > > > > However, I for one never run a -pre kernel.
> > > > >
> > > > > I don't run -pre, because rightly or wrongly, I've got the impression
> > > > > that these get even less testing than releases.
> > > >
> > > > I think the opening sentence of your email states
> > > > the reason for that pretty well.
> > >
> > > The only way we can get good testing for new kernels is to stop using
> > > -preN prefix in development branch (2.5.x). Just increment that 'x'.
> >
> > That won't change anything except the number on the kernel.  The people
> > who you're trying to turn into bleeding edge testers (those who stay a
> > little behind [bignum]) will continue to ride the curve at the point of
> > their choosing.
>
> Yes, but they can't tell which 2.5.x is more stable just from version number.
> This way Linus will get better test coverage in 2.5.x.

Wrong.  Anybody who has been around a while will not be foxed.  Those
newcomers who haven't figured out where on the curve they want to sit
will figure it out, so I repeat.. changes nothing.

	-Mike

P.S.  I had deleted the cc list quite intentionally.... :-

Re: Kernel Releases

[Giacomo Catenazzi]

Dan Kegel wrote:

>>On Monday 26 November 2001 16:45, Mike Galbraith wrote:
>>
>>>>The only way we can get good testing for new kernels is to stop using
>>>>-preN prefix in development branch (2.5.x). Just increment that 'x'.
>>>>
>>>That won't change anything except the number on the kernel.  The people
>>>who you're trying to turn into bleeding edge testers (those who stay a
>>>little behind [bignum]) will continue to ride the curve at the point of
>>>their choosing.
>>>
>>Yes, but they can't tell which 2.5.x is more stable just from version number.
>>This way Linus will get better test coverage in 2.5.x.
>>
>>Those who need stability can read lkml and figure out which 2.5.x was 
>>'glitchless' :-) or stick with 2.4.x
>>
> 
> Agreed.  2.5.x should not use -pre.  Just increment X.  
> 


No. In the unstable branch there are frequent 'private' pre-release,
used for test or to syncronize big merges/changes.


Let continue actual status:

the normal release for everybody (restricted to developer) and the

pre reelase for special/merges patch.

	giacomo

Re: Kernel Releases

[Dan Kegel]

> On Monday 26 November 2001 16:45, Mike Galbraith wrote:
> > > The only way we can get good testing for new kernels is to stop using
> > > -preN prefix in development branch (2.5.x). Just increment that 'x'.
> >
> > That won't change anything except the number on the kernel.  The people
> > who you're trying to turn into bleeding edge testers (those who stay a
> > little behind [bignum]) will continue to ride the curve at the point of
> > their choosing.
> 
> Yes, but they can't tell which 2.5.x is more stable just from version number.
> This way Linus will get better test coverage in 2.5.x.
> 
> Those who need stability can read lkml and figure out which 2.5.x was 
> 'glitchless' :-) or stick with 2.4.x

Agreed.  2.5.x should not use -pre.  Just increment X.  

2.4.x should continue to use -preY.
There's no need for a -rcY as some have suggested.
All we need to do to avoid messes like the 2.4.15 debacle
is to insist that a 2.4.X-preY should not be
released as final 2.4.X until the pre's been out for a week,
and there should never be any changes introduced into a final
that didn't cook for a week as a pre.  Marcello, what say ye?

That should give enough time for everyone to
find bugs and yell.
- Dan

Re: Kernel Releases

[François Cami]

Dan Kegel wrote:


> 2.4.x should continue to use -preY.
> There's no need for a -rcY as some have suggested.
> All we need to do to avoid messes like the 2.4.15 debacle
> is to insist that a 2.4.X-preY should not be
> released as final 2.4.X until the pre's been out for a week,
> and there should never be any changes introduced into a final
> that didn't cook for a week as a pre.  


I don't see the difference between a -rc that has cooked for a week,
and a -pre that has cooked for a week, except that -rc sounds more
like "this is *maybe* the next stable kernel", whereas -pre still
sounds "beta".

That said, I think the week long delay is a *good* idea.

François

Re: Kernel Releases

[Dan Kegel]

François Cami wrote:
> 
> Dan Kegel wrote:
> 
> > 2.4.x should continue to use -preY.
> > There's no need for a -rcY as some have suggested.
> > All we need to do to avoid messes like the 2.4.15 debacle
> > is to insist that a 2.4.X-preY should not be
> > released as final 2.4.X until the pre's been out for a week,
> > and there should never be any changes introduced into a final
> > that didn't cook for a week as a pre.
> 
> I don't see the difference between a -rc that has cooked for a week,
> and a -pre that has cooked for a week, except that -rc sounds more
> like "this is *maybe* the next stable kernel", whereas -pre still
> sounds "beta".

The difference between "this is *maybe* the next stable kernel"
and "just another beta" is very slippery.  I don't object
to the -rc idea, but I don't think it's as valuable as all that.

> That said, I think the week long delay is a *good* idea.

It's the key to avoiding bad releases.
- Dan

Re: Kernel Releases

[Vitaly Luban]

Dan Kegel wrote:

> François Cami wrote:
> >
> > Dan Kegel wrote:
> >
> > > 2.4.x should continue to use -preY.
> > > There's no need for a -rcY as some have suggested.
> > > All we need to do to avoid messes like the 2.4.15 debacle
> > > is to insist that a 2.4.X-preY should not be
> > > released as final 2.4.X until the pre's been out for a week,
> > > and there should never be any changes introduced into a final
> > > that didn't cook for a week as a pre.
> >
> > I don't see the difference between a -rc that has cooked for a week,
> > and a -pre that has cooked for a week, except that -rc sounds more
> > like "this is *maybe* the next stable kernel", whereas -pre still
> > sounds "beta".
>
> The difference between "this is *maybe* the next stable kernel"
> and "just another beta" is very slippery.  I don't object
> to the -rc idea, but I don't think it's as valuable as all that.
>

IMHO, -rc is just one extra unnecessary entity and may bring
only some confusion to newbies & extra work for maintaner.

Cheers,
    Vitaly.

Re: Kernel Releases

[Horst von Brand]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

Dan Kegel <dank@kegel.com> said:
> François Cami wrote:

[...]

> > That said, I think the week long delay is a *good* idea.

> It's the key to avoiding bad releases.

Yep. Specially when new 2.5.X-preY or 2.5.X are generated at a rate of 2 or
3 a week. Look at the history of the kernel. This would only create even
_more_ pressure to get new patches in, and that is bad.

When a freeze (or slush) is decreed at the very end, it makes sense.
Remember that a large part of the success of Linux is due to "Release
early, release often".
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Kernel Releases

[Mike Fedyk]

On Wed, Nov 28, 2001 at 01:23:07PM -0300, Horst von Brand wrote:
> Dan Kegel <dank@kegel.com> said:
> > Fran?ois Cami wrote:
> 
> [...]
> 
> > > That said, I think the week long delay is a *good* idea.
> 
> > It's the key to avoiding bad releases.
> 
> Yep. Specially when new 2.5.X-preY or 2.5.X are generated at a rate of 2 or
> 3 a week. Look at the history of the kernel. This would only create even
> _more_ pressure to get new patches in, and that is bad.

2.5 doesn't have releases, just snapshots of the development.

> 
> When a freeze (or slush) is decreed at the very end, it makes sense.
> Remember that a large part of the success of Linux is due to "Release
> early, release often".

Remember we are talking about 2.4, not 2.5.  Having more time between
releases gives more time to test the pre releases.  The point it to have
good point (not like 2.4.11, 12, 14, or 15) releases for what is supposed to
be *stable*.

Release 50 -pre kernels for 2.4, but make sure that the next release has had a
chance to stabalize with the new changes.

MF

Re: Kernel Releases

[Martin Knoblauch]

> Re: Kernel Releases
> 
> 
> > Development kernels are development kernels... nothing else. Look to
> > distributors for high degrees of quality assurance testing. When you run a
> > development kernel you have joined the development team, even if you don't
> > know it. Finding and reporting bugs is your job...
> 
> That's why you stay away from 2.5.x, or 2.4.x-pre, or 2.4.x-ac -- which
> are development kernels. 2.4.x kernels are released kernels.
> 

 The point being made (I believe) is that recently the "released"
kernels have had a low quality with showstopper-quality bugs being
introduced, but not detected,  very late in the -preX cycle. What the
initiator of this thread wants is a longer testing of the last -preX
version. And changes between that an the "release" confined to bug/doc
fixing *only*. Whether this has to be under the "-rcX" label, or not -
the idea behind it is sound.

Martin
-- 
------------------------------------------------------------------
Martin Knoblauch         |    email:  Martin.Knoblauch@TeraPort.de
TeraPort GmbH            |    Phone:  +49-89-510857-309
C+ITS                    |    Fax:    +49-89-510857-111
http://www.teraport.de   |    Mobile: +49-170-4904759

Re: Kernel Releases

[John Jasen]

On Mon, 26 Nov 2001, Martin Knoblauch wrote:

>  The point being made (I believe) is that recently the "released"
> kernels have had a low quality with showstopper-quality bugs being
> introduced, but not detected,  very late in the -preX cycle. What the
> initiator of this thread wants is a longer testing of the last -preX
> version. And changes between that an the "release" confined to bug/doc
> fixing *only*. Whether this has to be under the "-rcX" label, or not -
> the idea behind it is sound.

I tend to agree with the idea of a -rcX labeling for -pre trees that the
maintainer thinks are 'pretty much ready'. But, it is up to the
maintainers of the relevant trees to accept such an idea, and be willing
to hold off on new and exciting patches while testing and bugstomping is
going on.

--
-- John E. Jasen (jjasen1@umbc.edu)
-- In theory, theory and practise are the same. In practise, they aren't.

Re: Kernel Releases

[Horst von Brand]

John Jasen <jjasen1@umbc.edu> said:

[...]

> I tend to agree with the idea of a -rcX labeling for -pre trees that the
> maintainer thinks are 'pretty much ready'. But, it is up to the
> maintainers of the relevant trees to accept such an idea, and be willing
> to hold off on new and exciting patches while testing and bugstomping is
> going on.

AFAIU, if Linus (or whoever) puts out a -preX, it is because according to
_their_ testing (limited, whatever) it is "almost done, please check". If
they put out a non-pre, it is because they don't see any more issues with
it. The _only_ way they can ever know something breaks is that _somebody_
(yes, I _do_ mean you) checks it out and complains beforehand.

Want to have Linux for a free ride? Sorry, pal, no deal. You can use it in
exchange for testing and bug reports. Better yet, patches.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Kernel Releases

[Gerhard Mack]

On Mon, 26 Nov 2001, Horst von Brand wrote:

> AFAIU, if Linus (or whoever) puts out a -preX, it is because according to
> _their_ testing (limited, whatever) it is "almost done, please check". If
> they put out a non-pre, it is because they don't see any more issues with
> it. The _only_ way they can ever know something breaks is that _somebody_
> (yes, I _do_ mean you) checks it out and complains beforehand.

This is not always correct.. I've seen countless -pre kernels released as
synch points with a warning in the changelog.

It is a matter of not messing with pre series kernels unless you are
willing to read the changelogs.
 
	Gerhard

--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

Kernel Releases

[David Relson]

Greetings to All,

Over the past few months, I've been listening in on LKML, with occasional, 
minor comments - mostly to help newbies.  Now, I think it's time for a 
suggestion ...

As we all know, several of the recent releases have had defects that have 
__required__ patches before they could be built (or used safely).  Problems 
with symlinks, loopbacks, and unmount come to mind as being like 
this.  They are all show stoppers that required immediate fixes and the 
creation of a new release or of the next -pre1 version.

I have a tendency to tink that it's better to be running a released kernel, 
than a pre-release kernel.  I'd much rather be running a kernel named 2.4.x 
than a kernel named 2.4.y-pre?.  With the recent problems, the working 
versions tend to be the -pre1 or -pre2 releases, not the released 
one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
releases.  Here's how...

When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
(as in release candidate).  A day or two with -rc1 will quickly show if it 
has a show stopper.  If so, then the minor fixes (and nothing else) go into 
-rc2.  A day or two ..., and either -rc3 appears or we have a stable 
release and 2.4.16 is ready to be released.

Let's go the extra distance and have the releases be usable, stable 
kernels!  It's what users want and it's within the abilities of the 
developers to produce.  Let's do it :-)

David

Re: Kernel Releases

[John Alvord]

Development kernels are development kernels... nothing else. Look to
distributors for high degrees of quality assurance testing. When you run a
development kernel you have joined the development team, even if you don't
know it. Finding  and reporting bugs is your job...

john alvord

On Sat, 24 Nov 2001, David Relson wrote:

> Greetings to All,
> 
> Over the past few months, I've been listening in on LKML, with occasional, 
> minor comments - mostly to help newbies.  Now, I think it's time for a 
> suggestion ...
> 
> As we all know, several of the recent releases have had defects that have 
> __required__ patches before they could be built (or used safely).  Problems 
> with symlinks, loopbacks, and unmount come to mind as being like 
> this.  They are all show stoppers that required immediate fixes and the 
> creation of a new release or of the next -pre1 version.
> 
> I have a tendency to tink that it's better to be running a released kernel, 
> than a pre-release kernel.  I'd much rather be running a kernel named 2.4.x 
> than a kernel named 2.4.y-pre?.  With the recent problems, the working 
> versions tend to be the -pre1 or -pre2 releases, not the released 
> one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
> releases.  Here's how...
> 
> When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
> next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
> (as in release candidate).  A day or two with -rc1 will quickly show if it 
> has a show stopper.  If so, then the minor fixes (and nothing else) go into 
> -rc2.  A day or two ..., and either -rc3 appears or we have a stable 
> release and 2.4.16 is ready to be released.
> 
> Let's go the extra distance and have the releases be usable, stable 
> kernels!  It's what users want and it's within the abilities of the 
> developers to produce.  Let's do it :-)
> 
> David
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Kernel Releases

[CaT]

On Sat, Nov 24, 2001 at 09:49:39PM -0800, John Alvord wrote:
> Development kernels are development kernels... nothing else. Look to
> distributors for high degrees of quality assurance testing. When you run a
> development kernel you have joined the development team, even if you don't
> know it. Finding  and reporting bugs is your job...

2.x.y, where x is even, is not a development kernel. It's a release kernel.
2.x.y, where x is odd is a development kernel
2.x.yprez is a development kernel

I think his point is that recently, 2.x.y where x is even has been of
the same quality as a development kernel. Blatantly nasty bugs are in it
and they really shouldn't be. And so, in order to test things out better
he effectively saying that putting a -rcz tag on it will help as it'll
indicate a release candidate and as such might get more ppl to use it.
Once no obvious issues show up with those, put it as a release.

No real extra work required. You just need to make a decision of 'enough
stuffing about. I wanna make a release soon' and start labelling
acordingly. This doesn't mean -rcz will be release quality. It just
means that it's close to it. It still wont mean it's 100% bug free but
it's an extra chance to catch the bad stuff.

All in all, it's not a bad idea[tm].

-- 
CaT        "As you can expect it's really affecting my sex life. I can't help
           it. Each time my wife initiates sex, these ejaculating hippos keep
           floating through my mind."
                - Mohd. Binatang bin Goncang, Singapore Zoological Gardens

Re: Kernel Releases

[John Jasen]

On Sat, 24 Nov 2001, John Alvord wrote:

> Development kernels are development kernels... nothing else. Look to
> distributors for high degrees of quality assurance testing. When you run a
> development kernel you have joined the development team, even if you don't
> know it. Finding  and reporting bugs is your job...

That's why you stay away from 2.5.x, or 2.4.x-pre, or 2.4.x-ac -- which
are development kernels. 2.4.x kernels are released kernels.

--
-- John E. Jasen (jjasen1@umbc.edu)
-- In theory, theory and practise are the same. In practise, they aren't.

Re: Kernel Releases

[John Alvord]

On Sun, 25 Nov 2001 09:12:10 -0500, John Jasen <jjasen1@umbc.edu>
wrote:

>On Sat, 24 Nov 2001, John Alvord wrote:
>
>> Development kernels are development kernels... nothing else. Look to
>> distributors for high degrees of quality assurance testing. When you run a
>> development kernel you have joined the development team, even if you don't
>> know it. Finding  and reporting bugs is your job...
>
>That's why you stay away from 2.5.x, or 2.4.x-pre, or 2.4.x-ac -- which
>are development kernels. 2.4.x kernels are released kernels.

2.<even>.x are "stablizing" kernels, where (theoretically) only bug
fixes are accepted. But the code hasn't been through any QA cycle at
the moment of release. It is still a developer release in any
traditional software development definition.

I am quite impressed that Linus hangs on to the initial x.<even>
series until it becomes close to production quality. Notice that an
important user (google) came up with a severe error very late in the
day and Linus held things up for a few weeks until that was cleared.
Notice the months long struggle to decide on a VM solution. In this
cycle, it took maybe ten months to shepard 2.4.0 into something of
excellent potential that is a real step forward from the 2.2 series.

 Everyone who participated can be proud.

john alvord

Re: Kernel Releases

[Eric W. Biederman]

John Alvord <jalvo@mbay.net> writes:

> Development kernels are development kernels... nothing else. Look to
> distributors for high degrees of quality assurance testing. When you run a
> development kernel you have joined the development team, even if you don't
> know it. Finding  and reporting bugs is your job...

Bah.  Some of the worst kernels I have seen have been distributors kernels.
Distributors seem to give in to the desire for more features, and don't
address the deep bugs.  

Now stability and correctness are two different (but related) things in code.
I have seen stable programs and some even a few kernels that were rock solid
reliable and stable but were extremely incorrect in places.

Correctness means you can write a proof showing how it meats it's
specifications.  Stability means something passes the test of time.

If you are correct it is relatively easy to pass the test of time.  If
you are incorrect it is still possible but harder.

What the developers produce primarily are correct kernels, especially
Linus.  I have never seen a distribution make a kernel more correct.
Instead what I have seen is distributors testing on a wide variety of
hardware and for those things that don't work they throw in the
current best work around.

Also distributors during development don't have the same number of
people testing a kernel as the global linux kernel project has.  At
the current point in time it appears that Linus Torvalds is finally
satisfied with the basic correctness of 2.4.x.  Enough so that he
doesn't feel the need to keep fixing it.  So even if 2.4.15 is 
slightly damaged it makes a good base for future stability.

To date Alan Cox has done a great job in maintaining the stable 2.2
series.  Giving us all something we can use reliably without fear.
And while I haven't watched it religiously Alan Cox has done the whole
release candidate thing, which really does help to catch stupid
typos..  And hopefully in his one way Marcello Tosatti will as well.
I honestly expect future 2.4.x kernels to be much more stable.

I have the utmost respect for the work Linus, Alan, and now Marcello
are doing.  Taking on a position where you agree to be flooded with
email.  Being buried in so much management of patches that you hardly
have time to develop yourself.  Waiting in a pre patch, for someone
else to catch your typos.  Patience is a hard thing, as is believe
that somehow you goofed and you need a second pair of eyes to look
at it and fix it.

Now I agree that finding and reporting bugs is still the job of
everyone who runs a Linux kernel in the stable series, but that is
just the final stage of quality assurance.  But this is part of a
larger job that is really finding a way to get the best possible
kernel.  Giving up on that process and figuring it will always result
in a buggy and flaky product looks like the wrong attitude to take.

Linus has admitted he really doesn't have the temperament for
maintaining a long term stable release.  But as we have all seen he
does have the temperament to lead a development kernel.  And his
stepping down now from the stable branch now that the core of the
kernel is correct is the best sign for stability for 2.4.x that I have
seen.  I have yet to see Marcello's work in action but if is anywhere
near as good as Alan's 2.4.16+ should be more stable than anything the
distributors release.

Eric

Re: Kernel Releases

[Nathan Walp]

> Correctness means you can write a proof showing how it meats it's
> specifications.  Stability means something passes the test of time.
                                             ^^^^^^^^^^^^^^^^^^^^^^^
I think you hit the nail on the head here.  2.<even>.x kernels are not
stable, they are presumably stable.  Nothing is stable until it proves
itself.

If you run a hours-old kernel on a production machine, you will get
burned sooner or later.  If you run an hours-old kernel on a production
machine, and manage to reboot it so quickly, I start to wonder what kind
of "production" you're running.

Nathan

Re: Kernel Releases

[Bill Davidsen]

On Sun, 25 Nov 2001, Eric W. Biederman wrote:

> Bah.  Some of the worst kernels I have seen have been distributors kernels.
> Distributors seem to give in to the desire for more features, and don't
> address the deep bugs.  

You use the wrong distributor, try Alan Cox ;-)
 
> What the developers produce primarily are correct kernels, especially
> Linus.  I have never seen a distribution make a kernel more correct.
> Instead what I have seen is distributors testing on a wide variety of
> hardware and for those things that don't work they throw in the
> current best work around.

I would regard a stable kernel which doesn't compile, or which corrupts
filesystems in operation normal to all systems (umount) as correctness
issues, which should be caught if even the smallest QA were being done.

> Also distributors during development don't have the same number of
> people testing a kernel as the global linux kernel project has.

A point some folks sem to have missed.

> To date Alan Cox has done a great job in maintaining the stable 2.2
> series.  Giving us all something we can use reliably without fear.
> And while I haven't watched it religiously Alan Cox has done the whole
> release candidate thing, which really does help to catch stupid
> typos..  And hopefully in his one way Marcello Tosatti will as well.
> I honestly expect future 2.4.x kernels to be much more stable.

> Linus has admitted he really doesn't have the temperament for
> maintaining a long term stable release.  But as we have all seen he
> does have the temperament to lead a development kernel.  And his
> stepping down now from the stable branch now that the core of the
> kernel is correct is the best sign for stability for 2.4.x that I have
> seen.

Agreed. But I would like to have seen it sooner, allowing all the
developmental stuff to be fully developed in 2.5 before it was dropped
into 2.4. By putting new features in 2.4.x we had the worst of possible
results, end users got a less stable kernel and developers were slowed
down by release cycles. This didn't really make anyone happy, I believe. I
expect both 2.4 and 2.5 to work better now.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Kernel Releases

[Phil Sorber]

[-- Attachment #1: Type: text/plain, Size: 2845 bytes --]

i know this is an old thread now, but i thought of something that might
be a nice feature, though it means more work for kernel maintainers.

a bug patch seperate from the new kernel patch. so you could only
upgrade from 2.4.0 with the bug fixes in 2.4.1. by the time you're at
something like 2.4.15, 2.4.0 should be rock solid. but you wouldn't have
any new funcionality.

say you wanted the new VM. you could upgrade to the first kernel with
the new VM, then get the bug patch for the latest kernel.

not a simple straigt forward idea, and it could mean a lot of work.
though maybe it is easier than i realize, or a few orders of magnitude
harder than i realize, and thus not worth it.

i've noticed that the 'aa' patch series seems to be something along
these lines.

On Sat, 2001-11-24 at 23:27, David Relson wrote:
> Greetings to All,
> 
> Over the past few months, I've been listening in on LKML, with occasional, 
> minor comments - mostly to help newbies.  Now, I think it's time for a 
> suggestion ...
> 
> As we all know, several of the recent releases have had defects that have 
> __required__ patches before they could be built (or used safely).  Problems 
> with symlinks, loopbacks, and unmount come to mind as being like 
> this.  They are all show stoppers that required immediate fixes and the 
> creation of a new release or of the next -pre1 version.
> 
> I have a tendency to tink that it's better to be running a released kernel, 
> than a pre-release kernel.  I'd much rather be running a kernel named 2.4.x 
> than a kernel named 2.4.y-pre?.  With the recent problems, the working 
> versions tend to be the -pre1 or -pre2 releases, not the released 
> one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
> releases.  Here's how...
> 
> When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
> next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
> (as in release candidate).  A day or two with -rc1 will quickly show if it 
> has a show stopper.  If so, then the minor fixes (and nothing else) go into 
> -rc2.  A day or two ..., and either -rc3 appears or we have a stable 
> release and 2.4.16 is ready to be released.
> 
> Let's go the extra distance and have the releases be usable, stable 
> kernels!  It's what users want and it's within the abilities of the 
> developers to produce.  Let's do it :-)
> 
> David
> 
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
-- 
Phil Sorber
AIM: PSUdaemon
IRC: irc.openprojects.net #psulug PSUdaemon
GnuPG: keyserver - pgp.mit.edu

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: Kernel Releases

[Allan Sandfeld]

On Sunday 25 November 2001 05:27, David Relson wrote:

<snip>
> When the kernel maintainer, now Marcelo for 2.4, is ready to release the
> next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1"
> (as in release candidate).  A day or two with -rc1 will quickly show if it
> has a show stopper.  If so, then the minor fixes (and nothing else) go into
> -rc2.  A day or two ..., and either -rc3 appears or we have a stable
> release and 2.4.16 is ready to be released.

Like Linus said, it's a statistical problem: An unofficial kernel would never 
get the same attention as a released one. We would keep seeing problems arise 
once the kernel has been released. 

One thing we could do, was to do the same as the vendors do, and maintain a 
-post kernel with the most glaring bug-fixed, especially build-ones. 

The need to do it however is not that big..

Re: Kernel Releases

[Ian Stirling]

> 
> On Sunday 25 November 2001 05:27, David Relson wrote:
> 
> <snip>
> > When the kernel maintainer, now Marcelo for 2.4, is ready to release the
> > next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1"
<snip>
> Like Linus said, it's a statistical problem: An unofficial kernel would never 
> get the same attention as a released one. We would keep seeing problems arise 
> once the kernel has been released. 
True.
However, I for one never run a -pre kernel.
I am unfortunately not wealthy, so am at the trailing edge of hardware 
(Athlon 550, P100 laptop, ...) so I rarely need devices only in the 
latest kernels.
I do however occasionally run new kernels on boxes at various times.
I don't run -pre, because rightly or wrongly, I've got the impression that
these get even less testing than releases.
I might well run -rcn kenrels, as these are presumably tested a little more,
and thought to be a little more stable.
This would help make me part of the solution, rather than part of the problem.
(complaining when 2.4.10 falls on it's face whenever something reads lots
of data say, or 2.4.11 has link problems)

Re: Kernel Releases

[Rik van Riel]

On Mon, 26 Nov 2001, Ian Stirling wrote:

> However, I for one never run a -pre kernel.

> I don't run -pre, because rightly or wrongly, I've got the impression
> that these get even less testing than releases.

I think the opening sentence of your email states
the reason for that pretty well.

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/

http://www.surriel.com/		http://distro.conectiva.com/

Re: Kernel Releases

[Ian Stirling]

> 
> On Mon, 26 Nov 2001, Ian Stirling wrote:
> 
> > However, I for one never run a -pre kernel.
> 
> > I don't run -pre, because rightly or wrongly, I've got the impression
> > that these get even less testing than releases.
> 
> I think the opening sentence of your email states
> the reason for that pretty well.

Ok, I should have stated that more accurately.
...and also there is no way to tell how stable it's expected to be.
If there was a way to say that this was a kernel that seemed stable 
for the testers, and might be a candidate for a new version. so some
more testers would be nice, this might be good.

Re: Kernel Releases

[vda]

On Monday 26 November 2001 13:02, Rik van Riel wrote:
> On Mon, 26 Nov 2001, Ian Stirling wrote:
> > However, I for one never run a -pre kernel.
> >
> > I don't run -pre, because rightly or wrongly, I've got the impression
> > that these get even less testing than releases.
>
> I think the opening sentence of your email states
> the reason for that pretty well.

The only way we can get good testing for new kernels is to stop using
-preN prefix in development branch (2.5.x). Just increment that 'x'.

This will eliminate 'people waiting for final 2.5.N' problem.
(2.5 is unstable, so it's normal for it to break even in silly ways
as code gets patched/rewritten)

We can't do that for 2.4, for 2.4 scenario is:

Joe User: I see this oops in 2.4.18!
Maintainer: please try this 2.4.19pre1 I made for you
J: still oopses
M: hmm... try this 2.4.19pre2
J: YES YES YES it works now thanks
   - several days have passed, M may have some afterthoughts -
   - and updated preN's, can feed them to J etc... -
M: Do you see anything unusual since pre2?
J: No
M: He he me is cool, let's announce 2.4.19
--
vda

Re: Kernel Releases

[Bill Davidsen]

On Sun, 25 Nov 2001, David Relson wrote:

I wish you luck. About 18 months ago I offered to set up a testing group
to take kernel source before it with up for ftp and to compile it on
various x86, SPARC{32,64}, and ALPHA machines to be sure it would at least
compile. I was told with varying degrees of rudeness that it would delay
the releases (that is a GOOD thing in stable kernels), and that I should
avoid the 2.4 series and use the "stable" 2.2 kernsls (2.4 IS a stable
kernel of course, although you would never gues it).
 
> Over the past few months, I've been listening in on LKML, with occasional, 
> minor comments - mostly to help newbies.  Now, I think it's time for a 
> suggestion ...
> 
> As we all know, several of the recent releases have had defects that have 
> __required__ patches before they could be built (or used safely).  Problems 
> with symlinks, loopbacks, and unmount come to mind as being like 
> this.  They are all show stoppers that required immediate fixes and the 
> creation of a new release or of the next -pre1 version.
> 
> I have a tendency to tink that it's better to be running a released kernel, 
> than a pre-release kernel.  I'd much rather be running a kernel named 2.4.x 
> than a kernel named 2.4.y-pre?.  With the recent problems, the working 
> versions tend to be the -pre1 or -pre2 releases, not the released 
> one.  With a bit of QA, I think we can have 2.4.x releases be the stable 
> releases.  Here's how...
> 
> When the kernel maintainer, now Marcelo for 2.4, is ready to release the 
> next kernel, for example 2.4.16, I suggest he switch from "pre?" to "-rc1" 
> (as in release candidate).  A day or two with -rc1 will quickly show if it 
> has a show stopper.  If so, then the minor fixes (and nothing else) go into 
> -rc2.  A day or two ..., and either -rc3 appears or we have a stable 
> release and 2.4.16 is ready to be released.

I do agree, of course. There's no reason to avoid -pre releases, but
having -rc versions would be really useful because people who are waiting
for features would then be motivated to test, knowing that only fixes will
be applied. NOTE: this might mean that 2.4.18-pre1 would be out before
2.4.17 was actually released. That may bother some people.

A stable kernel should have only new drivers and not a whole new VM, at
least until it is in a development series and has been tested. If it were
up to me I would have opened 2.5 when 2.4.0 was released, and all the VM
stuff would have happened there. That's just the way I would assure
stability, knowing that there is no lack of people to test Linux kernels
no matter how unstable and patched... But a little truth in labeling would
be nice, recently the "really stable" kernels were -ac, and that's
probably not desirable.

Maybe the time is right to separate -pre and -rc and let people who who
want to test choose which, if either, they want to try. Some stuff really
MUST be tested on production machines (the recent VM versions, for
instance) to see which performs better and provide useful feedback.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: Kernel Releases

[Mike Fedyk]

On Mon, Nov 26, 2001 at 03:42:29PM -0500, Bill Davidsen wrote:
> NOTE: this might mean that 2.4.18-pre1 would be out before
> 2.4.17 was actually released. That may bother some people.
[snip]
> least until it is in a development series and has been tested. If it were
> up to me I would have opened 2.5 when 2.4.0 was released, and all the VM
> stuff would have happened there. That's just the way I would assure

The new VM was integrated too late, and 2.4 released too early.

I agree with Linus...  You do need a _known_ starting point for the next dev
kernel.  2.4.15 is known.

Also, having a 17-rc and 18-pre (at the same time) would load down Marcello
much more, and I haven't seen anyone else try it.

If 2.4 stayed 2.3 a bit longer, maybe it would've been cought maybe not.
Who knows if Andrea would've written his new VM back then.

What is interesting now is how Rik's VM (patch against 2.4.16) looks now
that it is seperate from the hugeness of the -ac patch.

The point is, if 2.4.0 was more mature, Linus *could* have given it over to
a maintainer sooner and started 2.5.  Instead, we ended up with a quasi
dev/stable kernel, and no dev "idiot if you don't expect corruption" kernel.
During that time there was no *rock* in 2.4 (unless it crashed - which
hasn't happened to me).  More like packed dirt. ;)

MF

Re: Kernel Releases

[Helge Hafting]

Bill Davidsen wrote:
> 
> On Sun, 25 Nov 2001, David Relson wrote:
> 
> I wish you luck. About 18 months ago I offered to set up a testing group
> to take kernel source before it with up for ftp and to compile it on
> various x86, SPARC{32,64}, and ALPHA machines to be sure it would at least
> compile. I was told with varying degrees of rudeness that it would delay
> the releases (that is a GOOD thing in stable kernels), and that I should
> avoid the 2.4 series and use the "stable" 2.2 kernsls (2.4 IS a stable
> kernel of course, although you would never gues it).
> 

Nobody's against you running a compile-test setup.  What they don't like
is the part about not releasing it until you have tested it.
That's because many others want to test too!
This is necessary - some may test aspects other than mere compiling.
Or perhaps a compiler different from yours.

Feel free to serve "known good" kernels - the masses who don't read
changelogs
or don't want to risk possible new bugs will surely appreciate this.

Just don't limit those of us who want to test the very latest - even
kernels with known bugs.  Maybe the bug won't bother me because I
don't use that driver/fs and things like that.  I want the kernel even
if, say, minixfs is broken.

Downloading the latest kernel is _not_ for those who can't deal
with occational trouble.  
Pre or no pre - development or "stable" series.
Those who run the latest is the testing team.  If they don't want to
be, they go for kernels at least a couple of weeks old that haven't
gathered trouble reports.  

Helge Hafting