[Qemu-devel] Test Scaffolding

[Qemu-devel] Test Scaffolding

[Tim]

Hello,

I have a bit of free time over the next few weeks, and thought it might
be interesting and useful to implement some type of randomized tests for
Qemu's system emmulation.  One thought I had was to generate a large
number of small disk images with (partially or fully) randomized boot
sectors and then try booting off of those.  Basically it would be a raw
stability test to see if Qemu falls over under weird circumstances. Some
questions I had: 

  - Would this approach be a useful test of the system stability?

  - Would this kind of test pose any significant risk to my underlying
    host system?

  - Can you think of a better randomized test scenario?


The inspriation for this comes from a little program called crashme
(http://people.delphiforums.com/gjc/crashme.html), which is designed to
test an OSes ability to handle random binaries.  Obviously, this could
be run under Linux in a VM, but Linux would likely contain much of the
weird cases and Qemu wouldn't be exposed to the dangerous cases.

Thanks,
tim

Re: [Qemu-devel] Test Scaffolding

[Tim]

> I have a bit of free time over the next few weeks, and thought it might
> be interesting and useful to implement some type of randomized tests for
> Qemu's system emmulation.  One thought I had was to generate a large
> number of small disk images with (partially or fully) randomized boot
> sectors and then try booting off of those.  Basically it would be a raw
> stability test to see if Qemu falls over under weird circumstances.


Ok, so in throwing caution to the wind, I put together this really quick
shell script (I will improve on it later): 

#!/bin/sh

OUTPUT_FILE=$1

qemu-img create -f raw "$OUTPUT_FILE" 1K
( dd bs=1 count=510 if=/dev/urandom 2>/dev/null;\
  echo | sed 's/^$/\x55\xAA/'; )\
  | dd bs=1 count=512 conv=notrunc "of=$OUTPUT_FILE" 2>/dev/null
###EOF####


Which creates a 1K disk image with a random boot sector.  I tested
qemu (i386-softmmu) on a number of these images, and in general it holds
up really well.  However, I did have several images crash qemu.  Two of
these appear to occur when program execution makes it past the main 512
byte boot sector, and begins trying to generate code for 0's, which
appear to disassemble as:

add [bx+si],al

In taking large blocks of these and converting them, qemu crashes.
Perhaps there is a bound checking problem on large blocks?  I don't
know, I haven't looked that deep into it yet.  (What file should I start
in?)  The bzipped tar of three files can be downloaded at
http://www.sentinelchicken.org/projects/qemu/add-crash.tar.bz2 .  
It contains an image that consistently crashes, the logfile which was
written to /tmp/qemu.log with the `-d all' option, and the disassembly
(via ndisasm) of the boot sector.

thanks,
tim