From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1M8sot-0005Cj-9k
	for qemu-devel@nongnu.org; Tue, 26 May 2009 05:19:39 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1M8soo-0005C1-Hg
	for qemu-devel@nongnu.org; Tue, 26 May 2009 05:19:38 -0400
Received: from [199.232.76.173] (port=34768 helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1M8soo-0005By-9d
	for qemu-devel@nongnu.org; Tue, 26 May 2009 05:19:34 -0400
Received: from mx2.redhat.com ([66.187.237.31]:56338)
	by monty-python.gnu.org with esmtp (Exim 4.60)
	(envelope-from <dlaor@redhat.com>) id 1M8son-0006sj-U8
	for qemu-devel@nongnu.org; Tue, 26 May 2009 05:19:34 -0400
Message-ID: <4A1BB3C4.5090002@redhat.com>
Date: Tue, 26 May 2009 12:17:56 +0300
From: Dor Laor <dlaor@redhat.com>
MIME-Version: 1.0
Subject: Re: autotest (was Re: [Qemu-devel] Re: [Qemu-commits] [COMMIT f80f9ec]
	Convert	machine	registration to use module initfunctions)
References: <200905211348.n4LDmnYd025976@d01av04.pok.ibm.com>
	<200905212336.02493.paul@codesourcery.com>
	<20090521225039.GA26811@poweredge.glommer>	<4A160138.7050109@us.ibm.com>
	<4A18068F.5020700@redhat.com> <4A19A422.80807@us.ibm.com>
	<4A19B32E.2020201@redhat.com> <4A1BA408.5030400@us.ibm.com>
In-Reply-To: <4A1BA408.5030400@us.ibm.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Reply-To: dlaor@redhat.com
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Anthony Liguori <aliguori@us.ibm.com>
Cc: qemu-devel@nongnu.org, Glauber Costa <glommer@redhat.com>, Avi Kivity <avi@redhat.com>, Paul Brook <paul@codesourcery.com>

Anthony Liguori wrote:
> Avi Kivity wrote:
>> Anthony Liguori wrote:
>>> Autotest doesn't currently test the regressions that I want to test.  
>>
>> Which regressions do you want to test?
>
> For instance, we often have networking regressions.  In particular, 
> with the refactoring going on in slirp and tun/tap, I'd really like to 
> have an automated way to test slirp and tap with TCP transmit/receive, 
> and slirp redir.  I have something locally to do this that I intend on 
> posting in the next few days.
>
Current kvm autotest do test slirp. The step-files for guest 
installation open ssh/telnet access in order
to let the host reach them.
We trapped exactly this regression using autotest.
>>
>> One way to increase coverage is to require each patch (fix or 
>> feature) to come with a test.  On the other hand, it might stop 
>> contributions instead of increasing test coverage.
>>
>> I've used this method for the x86 emulator in kvm, with some success.
>
> I wouldn't require kvm-autotest, but I'd like to have an interest set 
> of unit tests that are then usable within kvm-autotest.  Once we have 
> that, asking politely for tests to be written I think is quite 
> reasonable.
Some features are easily tested using a large framework, for example 
migration, slirp, time drift, etc.
In addition kvm's tests unit tests, qemu-io and similar should be 
written too in order to test various cases that
are only reasonable to be found using low level operations.
No doubt all of the unit tests should be executed from autotest.
>
> Regards,
>
> Anthony Liguori
>