Regression testing?

Regression testing?

[Dan Kegel]

Does anyone have access to some hefty benchmarks
suites like SPEC's, or regression tests
like the POSIX compliance test?
It might be a good idea to run 2.2.0-pre-x
through these to look for suprises
before going final with it.

The Posix test suite is downloadable from
http://www.itl.nist.gov/div897/ctg/posix_form.htm
See also http://www.standards.ieee.org/regauth/posix/
Even if we didn't actually pass all the tests,
it'd be good to know that we didn't oops.

The SPEC benchmarks can be purchased at
http://www.spec.org/cgi-bin/osgorder
for $700 or so.  The hard part is
actually having the patience to run them.
(I suspect if Linux or Alan asked nicely, they'd
donate a benchmark or two.   Or we could take
up a collection.  Or Alan's boss could pay.)
Three of the SPEC tests look interesting:

SPECweb96 stresses out HTTP networking.
SPEC SFS97 stresses out NFS networking.
SPEC SDM (System Development: Multitasking) 
stresses the kernel by simulating a large number of 
users running make, cp, diff, etc.

The point of running these benchmarks is not really
to say we're fast, but to see whether we can
actually get all the way through the benchmark
(which might take a day to run) without having
a functional error or a crash.
- Dan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/