From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1K3eof-0001xs-Ps for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:17:17 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1K3eoe-0001xJ-5c for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:17:17 -0400 Received: from [199.232.76.173] (port=40249 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1K3eod-0001xG-UR for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:17:15 -0400 Received: from wx-out-0506.google.com ([66.249.82.237]:24705) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1K3eod-0002Gv-Ox for qemu-devel@nongnu.org; Tue, 03 Jun 2008 18:17:15 -0400 Received: by wx-out-0506.google.com with SMTP id h29so1191557wxd.4 for ; Tue, 03 Jun 2008 15:17:14 -0700 (PDT) Message-ID: <4845C2DD.5080108@codemonkey.ws> Date: Tue, 03 Jun 2008 17:17:01 -0500 From: Anthony Liguori MIME-Version: 1.0 Subject: Re: [Qemu-devel] Suggestion for testing framework References: <767386.58386.qm@web57006.mail.re3.yahoo.com> <20080604075042.aff7c055.mle+tools@mega-nerd.com> <200806032302.09778.paul@codesourcery.com> <5d6222a80806031505t15f18fe7u256514ccbdc9960@mail.gmail.com> In-Reply-To: <5d6222a80806031505t15f18fe7u256514ccbdc9960@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Erik de Castro Lopo , "Balazs Attila-Mihaly (Cd-MaN)" Glauber Costa wrote: > On Tue, Jun 3, 2008 at 7:02 PM, Paul Brook wrote: > >> On Tuesday 03 June 2008, Erik de Castro Lopo wrote: >> >>> Balazs Attila-Mihaly (Cd-MaN) wrote: >>> >>>> Hello all >>>> >>>> It seems that there is agreement that some sort of automated >>>> testing is "a good thing" ;-). >>>> >>> I am a huge fan of testing and think that qemu developers and users >>> would both benefit from more automated testing. >>> >> IMHO Automated testing by itself is pretty much worthless. >> The value comes from having someone look at the results, and actively fix >> problems as they are discovered. Once you've allocated resources to do this >> bugfixing setting up the testing is fairly trivial. >> > > Not at all. A developer writing something new for qemu will have a way > to make sure his code works before submitting it upstream. > Test cases are great. I think the question is how useful is a fully automated testing suite that runs continuously. Without people dedicating time to track down all of the various failures, it tends to suffer the same fate as a poorly maintained bugzilla. But yeah, we want reproducible test cases. That's what KVMTest is all about--automating things that normally require user intervention. Regards, Anthony Liguori > Right now, each one has to write its own testing, each time, which can > be failed in itself, and not do a full coverage. > > So while it may not do much for existing code, it can certainly help > the quality of new code to improve. > > >