From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1K3ewR-00057m-Ad for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:25:19 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1K3ewQ-00056f-Rb for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:25:18 -0400 Received: from [199.232.76.173] (port=34796 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1K3ewQ-00056N-Dn for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:25:18 -0400 Received: from mail.codesourcery.com ([65.74.133.4]:47182) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1K3ewQ-0003kB-6Y for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:25:18 -0400 From: Paul Brook Subject: Re: [Qemu-devel] Suggestion for testing framework Date: Tue, 3 Jun 2008 23:25:01 +0100 References: <767386.58386.qm@web57006.mail.re3.yahoo.com> <200806032302.09778.paul@codesourcery.com> <5d6222a80806031505t15f18fe7u256514ccbdc9960@mail.gmail.com> In-Reply-To: <5d6222a80806031505t15f18fe7u256514ccbdc9960@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806032325.02120.paul@codesourcery.com> Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Glauber Costa , Erik de Castro Lopo , "Balazs Attila-Mihaly (Cd-MaN)" On Tuesday 03 June 2008, Glauber Costa wrote: > On Tue, Jun 3, 2008 at 7:02 PM, Paul Brook wrote: > > On Tuesday 03 June 2008, Erik de Castro Lopo wrote: > >> Balazs Attila-Mihaly (Cd-MaN) wrote: > >> > Hello all > >> > > >> > It seems that there is agreement that some sort of automated > >> > testing is "a good thing" ;-). > >> > >> I am a huge fan of testing and think that qemu developers and users > >> would both benefit from more automated testing. > > > > IMHO Automated testing by itself is pretty much worthless. > > The value comes from having someone look at the results, and actively fix > > problems as they are discovered. Once you've allocated resources to do > > this bugfixing setting up the testing is fairly trivial. > > Not at all. A developer writing something new for qemu will have a way > to make sure his code works before submitting it upstream. > Right now, each one has to write its own testing, each time, which can > be failed in itself, and not do a full coverage. We were talking about a tester that does periodic long running tests off svn trunk, and reports the results. Individual developers are not directly involved. You're talking about some sort of testsuite that can be distributed to all developers and reasonably run before every patch is submitted, which is a significantly different beast. I'm pretty certain the proposed tests would not be suitable for routine use by the majority of developers are part of normal developers. They will be too large, probably take a long time to run, and contain proprietary software that can't be redistributed. Paul