From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S623u-0006dA-8x for qemu-devel@nongnu.org; Fri, 09 Mar 2012 10:49:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S623p-0007g6-6T for qemu-devel@nongnu.org; Fri, 09 Mar 2012 10:48:57 -0500 Received: from mail-pw0-f45.google.com ([209.85.160.45]:60987) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S623o-0007da-Vv for qemu-devel@nongnu.org; Fri, 09 Mar 2012 10:48:53 -0500 Received: by pbcuo5 with SMTP id uo5so3062135pbc.4 for ; Fri, 09 Mar 2012 07:48:51 -0800 (PST) Message-ID: <4F5A265F.6060504@codemonkey.ws> Date: Fri, 09 Mar 2012 09:48:47 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <4F582EDB.1040608@redhat.com> <4F58B5CB.8040503@codemonkey.ws> <4F58CDEA.2020506@redhat.com> <4F59010C.2060105@codemonkey.ws> <4F5909B3.4080405@redhat.com> <4F590BD7.6030605@codemonkey.ws> <4F5913F3.3040503@redhat.com> <4F591EB4.1090300@codemonkey.ws> <4F5A076E.9040904@redhat.com> <4F5A0D49.60200@codemonkey.ws> <4F5A13FA.6040606@redhat.com> <4F5A1729.5060206@codemonkey.ws> <4F5A1B25.6070306@redhat.com> <4F5A1B86.4070806@codemonkey.ws> <4F5A1F18.3010006@redhat.com> <4F5A20A5.5050808@codemonkey.ws> <4F5A22FE.2070802@redhat.com> In-Reply-To: <4F5A22FE.2070802@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: Lucas Meneghel Rodrigues , Cleber Rosa , QEMU devel , Ademar Reis On 03/09/2012 09:34 AM, Paolo Bonzini wrote: > Il 09/03/2012 16:24, Anthony Liguori ha scritto: >>> At the very least dump the inquiry pages, mode pages, etc. and see that >>> they make sense and correspond to the device properties. >> >> Is this not something that's reasonably easy to do in qtest? > > Yes (at least with virtio-scsi the libos bits are relatively small; just > think of what it would have been like when the only HBA was LSI), but > with one gotcha... > >> Is it possible to write a C program that does the ioctl and dump the >> inquiry page in a text format conducive to shell parsing? > > ... sg_utils also parses the pages and dumps them in human-readable > format. This is useful because it provides a completely separate > implementation and avoids problems with misinterpretation of the > standard. Of course it would work just as well if someone wrote tests > instead of me. I don't recommend it in the general case, but it should be trivial to add additional packages to qemu-jeos and reuse the toolchain to build them. I don't think it's all that valuable here. I think you really want to test this via qtest. You could easy copy/paste code from sg_utils to do the parsing if you were so inclined... >>>> Are these the sort of tests that would be interesting to also run on >>>> Fedora, Windows, and Ubuntu? >>> >>> They should give exactly the same output on any guest. >> >> Is it valuable to have a per-platform test or since this is mostly >> passthrough to the device (I assume), do you just need a single test? > > Ah, understood. Yeah, a single test is enough for the purpose of > testing QEMU. If you want to test the driver too, running under Windows > would be useful. We should separate integration test (testing multiple components where we want to be able to use different versions/implementations of one component) from functional/unit tests that are strictly testing a single component. There isn't always a clean separation and there may be overlap, but I don't think we should stress out about the overlap. For instance, doing general purpose I/O testing is something where we want to test I/O with a Linux guest, a Windows guest, etc. This is an integration test and we should focus on it. But we probably want some sort of I/O test in qemu-test too since it provides a simple functional test. Yes, there's overlap, but the functional form of the test is so simple that it really isn't that important. But testing how QEMU handles malformatted virtio-blk requests is something that's clearly QEMU specific. There's no value doing that with a Linux and Windows guest (even if it's somehow possible). It's definitely a unit test that's strictly specific to QEMU. I think autotest should be able to execute QEMU's unit and functional tests. By using a test framework like gtest that exposes a test protocol, it should be trivial to add that support. Regards, Anthony Liguori > Paolo