From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:60702) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RgKnB-0006Or-D5 for qemu-devel@nongnu.org; Thu, 29 Dec 2011 13:33:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RgKnA-0008TA-C2 for qemu-devel@nongnu.org; Thu, 29 Dec 2011 13:33:29 -0500 Received: from mail-yx0-f173.google.com ([209.85.213.173]:56151) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RgKnA-0008T6-8J for qemu-devel@nongnu.org; Thu, 29 Dec 2011 13:33:28 -0500 Received: by yenm6 with SMTP id m6so8996396yen.4 for ; Thu, 29 Dec 2011 10:33:27 -0800 (PST) Message-ID: <4EFCB274.5020102@codemonkey.ws> Date: Thu, 29 Dec 2011 12:33:24 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <4EEF70B4.3070109@us.ibm.com> <4EF73EF5.8050606@redhat.com> <4EF88EC0.8020301@codemonkey.ws> <4EF8FC88.70809@redhat.com> <4EFA4829.4000207@redhat.com> <4EFA80EA.3050405@codemonkey.ws> <4EFAA2A2.4000107@redhat.com> <4EFB2764.7040006@codemonkey.ws> <4EFB2F36.2090408@redhat.com> <4EFB35AB.6040003@redhat.com> <4EFB4757.4020504@codemonkey.ws> <4EFB5138.5020502@redhat.com> <4EFC916E.4070902@codemonkey.ws> <4EFC9706.4090500@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Maydell Cc: "lmr@redhat.com" , Stefan Hajnoczi , cleber@redhat.com, dlaor@redhat.com, qemu-devel , Avi Kivity , Cleber Rosa , Paolo Bonzini , Gerd Hoffmann On 12/29/2011 11:22 AM, Peter Maydell wrote: > On 29 December 2011 16:36, Avi Kivity wrote: >> Yes; but using Linux limits you to what it exposes (of course Linux >> exposes quite a lot, so that's mostly a non issue; but we'll have >> counterexamples). > > Actually IME Linux is pretty conservative about how it uses devices. > A lot of the ARM device models have gaping holes in their emulation > which we've never needed to fix because Linux simply doesn't use > those features (eg the PL080 DMA controller which doesn't actually > implement DMA!) > > I think for devices what would be particularly useful would be > if you can write a (simple) test for something at the register > level, which generates an image which you can run on the real > hardware as well as on QEMU. Then you can confirm that your test > case is correct. Otherwise the tests are just going to bake in the > same assumptions/misconceptions about what the hardware does as > the QEMU model. I think that could be useful, but it orthogonal to what we need for Test Driven Development. The problem today is that it is very hard to make broad changes in QEMU simply because it's so difficult to know whether you're breaking another target. Look at the challenges around merging the memory API. We need a simple way that a developer can sanity check that they've not broken other targets and all of the various devices that we support. > > My guess is that a serious attempt at tests covering all the > functionality of a device is probably approximately doubling > the effort required for a device model, incidentally. A > half-hearted attempt probably doesn't buy you much over > automating "boot the guest OS and prod its driver". Right, but "boot the guest OS and prod its driver" is very useful particularly if it can cover all of the various targets we support. I don't think we should focus much on qtest for non-x86 targets. I mean, if you are interested in it for ARM, fantastic, but I don't think we would mandate it. For x86, it's reasonable to have stricter requirements on testing, particularly for new devices. FWIW, I expect virtio-scsi to be the first guinea pig here... I believe Stefan has already started looking at writing some qtest based tests for virtio-scsi. Regards, Anthony Liguori > -- PMM >