[Qemu-devel] testcases

[Qemu-devel] testcases

[Maggie Hamill]

Hi,

I am a grad student doing research on software reliability. I am
hoping to use qemu as a case study, but I need testcase to run on the
software. Does anyone know any infromation about existing testscases
(ie how many there are, and where I can get them) and whether or not
there is any documentation on the tests?

Thanks,
Maggie*

Re: [Qemu-devel] testcases

[Kyle Hayes]

On Tuesday 24 May 2005 05:23, Maggie Hamill wrote:
> Hi,
>
> I am a grad student doing research on software reliability. I am
> hoping to use qemu as a case study, but I need testcase to run on the
> software. Does anyone know any infromation about existing testscases
> (ie how many there are, and where I can get them) and whether or not
> there is any documentation on the tests?

Hmm, there's not much documentation.  If you check out the full source tree 
for Qemu, you will see a tests directory.  I'm not sure that these are 
fully up to date, but they have been modified very recently, so they 
probably are.

The tests mostly test individual instruction simulation.  Look at 
test-i386.ref.  I think that shows what instruction followed by the inputs 
and outputs.  Maybe.   For other architectures, I do not think there is 
much.  There is an ARM file in that directory.  

Most testing on Qemu is accomplished with very high level black box 
testing.  I.e. load an OS into it and see what happens :-)  The test cases 
tend to be very ad hoc.  Fabrice may have his own suite of tests, but I do 
not see much in the CVS tree.

Fabrice's other projects include tcc, a tiny C compiler.  It has been 
tested using parts of the GNU C compiler test suite.  That is a much 
smaller code base.

As to things like test-driven development or similar agile programming 
processes, I do not think that Fabrice is using them.  I could be wrong 
since I do not sit and look over his shoulder while he codes :-)

However, releasing the code early and often as Fabrice does make sure that 
the number of people testing Qemu exceeds the number of people coding Qemu 
by several orders of magnitude.    This is not a software methodology that 
you will find taught very often for all that it is what makes open source 
go and is very good in practice.  

I'm not sure that this is a good project to use as an example if you are 
hoping to find practical applications of the test and quality 
methodologies you read about.  It is, however, a fairly good one to look 
at for an example of a real open source project that works in all senses 
of the word.

Best,
Kyle


Best,
Kyle