Re: Regression testing of 2.4.x before release?

[Dan Kegel <dank@kegel.com>]

"Timothy D. Witham" wrote:
> ...
>   I agree having the users run their applications and under their usage
> model is a very good way of testing code drops.  Dan, I think that what
> you are trying to say is that it might be a good idea to take a group
> of tests and make them the standard set of "pass/fail" that people
> should look to before doing their own testing.

More like a safety net.  It'd help make sure we didn't forget something
obvious.
>...
>   Regression type pass/fail tests don't tend to have the benchmark
> optimization issue but like any test they usually only find the
> problems that you either already have had in the past or that are
> obvious.  Not complete but they should be dynamic environment that
> things are being added to all the time.  Also the nice part about a
> knows series of tests is that if a problem pops up it is much
> easier to reproduce for debugging purposes.

Yep.

> > 2. The STP at OSDLab seems like a great resource that we might be able
> > to leverage to solve the problem Alan points out.
> 
>   The nice part about the way that STP was designed is that it is
> extensible.  If somebody comes up with another test we can add it.
> If we need to add additional equipment to get the run times down
> to a usable level then that is easy to do also.
> 
> > I'm not suggesting anyone do any less testing.  Just the opposite;
> > if we set things up properly with the STP, we might be able to run
> > many more tests before each final release.
> 
>   We are in the process of setting up the Kernel STP to automatically
> grab the Linus and -ac kernels and run the full setup.  This will
> do part of what Dan is asking for and it will also allow people who
> are looking to supply patches a baseline for there patch testing.

That's super!  Thanks, Tim!

At some point it might be nice to also use the STP to help
speed gcc 3 development, too.  (I personally am really
looking forward to the day when I can use the same compiler
for both c++ and kernel.)

- Dan