Hello all
It seems that there is agreement that some sort of automated testing is "a good thing" ;-). I'll have some free time in the next couple of days and plan on throwing something like this together on a spare box. I was thinking along the lines:
- several qemu images (one with Debian, one with Windows XP - I can get a free student license for it, etc)
- a script does a checkout of the trunk, checks if the version number is different from the last checkout (to avoid spamming the list :-))
- the script introduces the source in each VM, starts the VM and lets the different compilers available in the VM (like gcc 3.3, 3.4, mingw) compile the source
- if the compile fails, it collects the error logs
- if the compile succeeds, performance and functionality tests are run with the resulting binary - the is the most nebulous part for the moment for me - if I recall Fabrice said that compiling something inside a VM is a good performance test...
- results are sumitted to the list - if you are ok with that, I wouldn't want to spam the list
Please comment if you find the testing methodology good and what performance and functionality test should the process include...
Best regards.
Hi,
using an existing framework for continuous integration tests will
provide many of the
features needed. Here is an example for an automated mail (which might
be sent to
the Qemu mailing list). I used CruiseControl, a Java based framework.
The
configuration and a helper script are appended to this mail and could
be added to
Qemu trunk, so everyone can run his/her own continuous integration.
The current configuration is just a simple compile test for all targets.
It can be extended to add more information to the mail, to inform those
who
changed the code about success or failure, to run a web interface,
to cross compile, to run tests...
Of course, running the continuous integration on a server with access
from the
Internet would be better than running it on my private machine, but
even my
private machine could send mails to the list when something goes wrong.
I tried to run CI on a virtual server but it did not have enough
resources to run Java...
Regards
Stefan
Example of mail sent by CruiseControl:
View results here ->
http://localhost/cruisecontrol/buildresults/qemu
trunk x86?log=log20080615200821Lbuild.7
|
No Tests Run |
|
This project doesn't have any tests |
|
|
| |
| modified |
blueswir1 |
/trunk/target-sparc/translate.c |
06/15/2008
20:06:39 |
Avoid
temporary variable use across basic blocks for udivx |
| modified |
blueswir1 |
/trunk/linux-user/main.c |
06/15/2008
20:02:48 |
Fix
Sparc32plus & Sparc64 debug output |