From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:33218) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UDB2P-0004zX-Rq for qemu-devel@nongnu.org; Wed, 06 Mar 2013 04:53:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UDB2M-0000vl-Gv for qemu-devel@nongnu.org; Wed, 06 Mar 2013 04:53:29 -0500 Received: from mail-we0-x22f.google.com ([2a00:1450:400c:c03::22f]:38366) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UDB2M-0000vP-AF for qemu-devel@nongnu.org; Wed, 06 Mar 2013 04:53:26 -0500 Received: by mail-we0-f175.google.com with SMTP id x8so7831444wey.34 for ; Wed, 06 Mar 2013 01:53:25 -0800 (PST) Date: Wed, 6 Mar 2013 10:53:08 +0100 From: Stefan Hajnoczi Message-ID: <20130306095308.GA1954@stefanha-thinkpad.muc.redhat.com> References: <5134CA4A.7040700@redhat.com> <20130305104605.GG1938@stefanha-thinkpad.redhat.com> <5135D443.2070208@redhat.com> <5136165D.6040107@redhat.com> <513619F6.5020607@redhat.com> <51361B7E.2030709@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <51361B7E.2030709@redhat.com> Subject: Re: [Qemu-devel] The state of testing List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Lucas Meneghel Rodrigues Cc: Kevin Wolf , Anthony Liguori , Juan Quintela , Markus Armbruster , Michael Roth , qemu-devel , Gerd Hoffmann , Amit Shah On Tue, Mar 05, 2013 at 01:21:18PM -0300, Lucas Meneghel Rodrigues wrote: > On 03/05/2013 01:14 PM, Gerd Hoffmann wrote: > > Hi, > > > >>>I don't think we have *any* regular autotest coverage for master, have > >>>we? > >> > >>^ Yes, there is coverage. There's a daily job called sanity that takes > >>about 2.5 hours, that builds the latest kvm.git and installs it in the > >>host, then reboots the box, builds the latest qemu master and runs: > > > >Oh, good. I guess it would be a good idea to send reports to > >qemu-devel, maybe a weekly status or mails on regressions simliar to the > >buildbot build failures. So people (a) aware that the service exists in > >the first place and (b) get informed that something broke which should > >be fixed. > > No doubt, I did try that before, but from my experience, the reports > are likely to be ignored and people will complain about SPAM. > > So we ended up creating an internal mailing list where Gleb and > Marcelo are subscribed to, in case they want to look at the reports, > and I'd look at the reports and report on the ML in case something > bad happens. I understand that SPAM issue. Maybe only send failures to the list? The reason why it's important to include qemu-devel@nongnu.org is that in the current mode you are essentially just reporting bugs. This is different from providing maintainers test results and failure notifications so the tests become part of the QEMU development and release process. That's an important distinction. Maintainers need to see the test status (e.g. "99/102 tests passing") and have the ability to dig deeper - especially during the hard freeze period where we shake out bugs. If maintainers have no visibility then the tests are not part of the development process. The way that buildbot does this is and I think the same is possible for kvm autotest: 1. Send failure notifications to qemu-devel. 2. Provide a website to check the status http://buildbot.b1-systems.de/qemu/. If you want help, please let me know. qemu.org could host web-accessible test results. We can check with Anthony and I can help you set it up. Stefan