Re: [Qemu-devel] KVM call minutes for Apr 5

[Stefan Hajnoczi <stefanha@gmail.com>]

On Tue, Apr 5, 2011 at 4:07 PM, Chris Wright <chrisw@redhat.com> wrote:
> kvm-autotest
> - roadmap...refactor to centralize testing (handle the xen-autotest split off)
> - internally at RH, lmr and cleber maintain autotest server to test
>  branches (testing qemu.git daily)
>  - have good automation for installs and testing
> - seems more QA focused than developers
>  - plenty of benefit for developers, so lack of developer use partly
>    cultural/visibility...
>  - kvm-autotest team always looking for feedback to improve for
>    developer use case
> - kvm-autotest day to have folks use it, write test, give feedback?
>  - startup cost is/was steep, the day might be too much handholding
>  - install-fest? (to get it installed and up and running)
> - buildbot or autotest for testing patches to verify building and working
> - one goal is to reduce mailing list load (patch resubmission because
>  they haven't handled basic cases that buildbot or autotest would have
>  caught)
> - fedora-virt test day coming up on April 14th.  lucas will be on hand and
>  we can piggy back on that to include kvm-autotest install and virt testing
> - kvm autotest run before qemu pull request and post merge to track
>  regressions, more frequent testing helps developers see breakage
>  quickly
>  - qemu.git daily testing already, only the "sanity" test subset
>    - run more comprehensive "stable" set of tests on weekends
> - one issue is the large number of known failures, need to make these
>  easier to identify (and fix the failures one way or another)
> - create database and verify (regressions) against that
>  - red/yellow/green (yellow shows area was already broken)
> - autotest can be run against server, not just on laptop

Features that I think are important for a qemu.git kvm-autotest:
* Public results display (sounds like you're working on this)
* Public notifications of breakage, qemu.git/master failures to
qemu-devel mailing list.
* A one-time contributor can get their code tested.  No requirement to
set up a server because contributors may not have the resources.

Perhaps kvm-autotest is a good platform for the automated testing of
ARM TCG.  Paul is CCed, I recently saw the Jenkins qemu build and boot
tests he has set up.  Lucas, do you have ideas on how these efforts
can work together to bring testing to upstream QEMU?
http://validation.linaro.org/jenkins/job/qemu-boot-images/

Stefan