What test suites can you tell me about?

What test suites can you tell me about?

[Michael D. Crawford]

Can you tell me about any ready-to-use test suites, for any software package
that should run under Linux, that I can build and run to test the new kernel?

Besides running all these tests myself on my machine, I'm going to document them
in an article at:

http://linuxquality.sunsite.dk/articles/

(there are no articles there yet but I'm composing a couple that will be posted
soon)

For example, if you build Python (http://www.python.org) and say "make check",
it will run a bunch of python programs that test the correctness of the
programming language.

This is of interest in part because lots of the Python tests make system calls,
but also because it tests that the compilers generate correct code under the new
kernel (another test I do is, after I boot off a new kernel, I do "make clean",
build it again and boot off _that_).

"make exec" under the Mesa 3.4 library builds a bunch of graphics demos, a few
of which are kind of whizzy but most of which exercise a few basic functions in
OpenGL.  So one can watch that they don't crash, that the images look correctly
drawn and so on.  This enabled me to realize that DRI wasn't working under 2.4.0
but it was under 2.4.0-prerelease-ac5, which I've detailed in a separate
message.

Another test suite I know about comes with Kaffe (http://www.kaffe.org) and
verifies that Kaffe's implementation of Java is running correctly on your
system.

One I read about somewhere but have no clue where to get it is this memory
stress-testing tool that does lots of DMA and stuff off of the disks.

There must be a lot of these tools available, if only we had them listed all in
one place.

If you maintain such a test tool, it would be helpful if you provided the option
to run the whole suite completely unattended.  Mesa provides a good test for
lots of functions of the kernel, but one problem is that one has to quit the
tests after each one runs, usually by pressing the ESC key.  Unattended testing
also allows one to run lots of tests simultaneously to test a heavily loaded
system. 

In some cases, the tests really do need to have some user input, like navigating
around a 3D world or turning various rendering options on and off, but it's
possible the tests could be extended to allow this input from a script (Python
provides a nice way to bolt a script interpreter to any application).

Mike
-- 
Michael D. Crawford
GoingWare Inc. - Expert Software Development and Consulting
http://www.goingware.com/
crawford@goingware.com

   Tilting at Windmills for a Better Tomorrow.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What test suites can you tell me about?

[Dan Kegel]

Michael D. Crawford (crawford@goingware.com) asked:
> Can you tell me about any ready-to-use test suites, 
> for any software package that should run under Linux, 
> that I can build and run to test the new kernel? 

The LSB has a set of test suites it has gathered for Linux itself;
see http://www.linuxbase.org/test/

- Dan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What test suites can you tell me about?

[Michael D. Crawford]

Nate Straz of the Linux Test Project at SGI (nstraz@sgi.com) said:
> The Linux Test Project (http://oss.sgi.com/projects/ltp/) was set up to 
> create a set of automated tests for Linux. 

Nate,

This is most excellent news!

I'd like you to look at http://linuxquality.sunsite.dk/

The objectives of the Linux Quality Database are:

- to get more people testing the new Linux kernels in a meaningful way
- to give them tips on how to do it effectively, such as a link to your site and
other test suites
- in the long run, to provide an easy-to-use web interface to make it easier for
people to log more meaningful bug reports, with a powerful search interface to
make it easy to find the bugs a developer wants to know about
- to improve the overall quality of free software in general through advocacy
and education

I'd be most interested in your comments about the site and what I'm trying to
achieve.

Also see "Why We Should All Test the New Linux Kernel" at:
http://advogato.org/article/224.html

Unfortunately, Advogato doesn't provide for editing an article once one presses
the final "submit" button.  So I'm going to post an edited and updated version
of that on the Linuxquality site.

Regards,
-- 
Michael D. Crawford
GoingWare Inc. - Expert Software Development and Consulting
http://www.goingware.com/
crawford@goingware.com

   Tilting at Windmills for a Better Tomorrow.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What test suites can you tell me about?

[Nathan Straz]

On Sat, Jan 06, 2001 at 11:21:31PM +0000, Michael D. Crawford wrote:
> Can you tell me about any ready-to-use test suites, for any software
> package that should run under Linux, that I can build and run to test
> the new kernel?

The Linux Test Project (http://oss.sgi.com/projects/ltp/) was set up to
create a set of automated tests for Linux.  The project currently
consists of about 100 tests and a simple test driver.  Grab the source
from anonymous CVS as the tarballs are out of date.  


-- 
Nate Straz                                              nstraz@sgi.com
sgi, inc                                           http://www.sgi.com/
Linux Test Project                    http://oss.sgi.com/projects/ltp/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/