From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:50938) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S63C4-0006BF-Pj for qemu-devel@nongnu.org; Fri, 09 Mar 2012 12:01:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S63Bb-0007gv-8k for qemu-devel@nongnu.org; Fri, 09 Mar 2012 12:01:28 -0500 Received: from mx1.redhat.com ([209.132.183.28]:24636) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S63Bb-0007fv-0O for qemu-devel@nongnu.org; Fri, 09 Mar 2012 12:00:59 -0500 Message-ID: <4F5A37A3.7020705@redhat.com> Date: Fri, 09 Mar 2012 14:02:27 -0300 From: Cleber Rosa MIME-Version: 1.0 References: <4F582EDB.1040608@redhat.com> <4F58B5CB.8040503@codemonkey.ws> <4F58CDEA.2020506@redhat.com> <4F59010C.2060105@codemonkey.ws> <4F5909B3.4080405@redhat.com> <4F590BD7.6030605@codemonkey.ws> <4F5913F3.3040503@redhat.com> <4F591EB4.1090300@codemonkey.ws> <4F5A076E.9040904@redhat.com> <4F5A0D49.60200@codemonkey.ws> <4F5A13FA.6040606@redhat.com> <4F5A1729.5060206@codemonkey.ws> <4F5A1B25.6070306@redhat.com> <4F5A1B86.4070806@codemonkey.ws> <4F5A1F18.3010006@redhat.com> <4F5A20A5.5050808@codemonkey.ws> <4F5A22FE.2070802@redhat.com> <4F5A265F.6060504@codemonkey.ws> In-Reply-To: <4F5A265F.6060504@codemonkey.ws> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests Reply-To: cleber@redhat.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: Lucas Meneghel Rodrigues , Paolo Bonzini , QEMU devel , Ademar Reis On 03/09/2012 12:48 PM, Anthony Liguori wrote: > On 03/09/2012 09:34 AM, Paolo Bonzini wrote: >> Il 09/03/2012 16:24, Anthony Liguori ha scritto: >>>> At the very least dump the inquiry pages, mode pages, etc. and see >>>> that >>>> they make sense and correspond to the device properties. >>> >>> Is this not something that's reasonably easy to do in qtest? >> >> Yes (at least with virtio-scsi the libos bits are relatively small; just >> think of what it would have been like when the only HBA was LSI), but >> with one gotcha... >> >>> Is it possible to write a C program that does the ioctl and dump the >>> inquiry page in a text format conducive to shell parsing? >> >> ... sg_utils also parses the pages and dumps them in human-readable >> format. This is useful because it provides a completely separate >> implementation and avoids problems with misinterpretation of the >> standard. Of course it would work just as well if someone wrote tests >> instead of me. > > I don't recommend it in the general case, but it should be trivial to > add additional packages to qemu-jeos and reuse the toolchain to build > them. > And then the two code bases (now buildroot and qemu-jeos) would have and increasing number of similar features. > I don't think it's all that valuable here. I think you really want to > test this via qtest. You could easy copy/paste code from sg_utils to > do the parsing if you were so inclined... And yet more code duplication. Even if a few KLOCs, but still... Hopefully I'm not the only one that fears those suggestions becoming real. > >>>>> Are these the sort of tests that would be interesting to also run on >>>>> Fedora, Windows, and Ubuntu? >>>> >>>> They should give exactly the same output on any guest. >>> >>> Is it valuable to have a per-platform test or since this is mostly >>> passthrough to the device (I assume), do you just need a single test? >> >> Ah, understood. Yeah, a single test is enough for the purpose of >> testing QEMU. If you want to test the driver too, running under Windows >> would be useful. > > We should separate integration test (testing multiple components where > we want to be able to use different versions/implementations of one > component) from functional/unit tests that are strictly testing a > single component. There isn't always a clean separation and there may > be overlap, but I don't think we should stress out about the overlap. > > For instance, doing general purpose I/O testing is something where we > want to test I/O with a Linux guest, a Windows guest, etc. This is an > integration test and we should focus on it. > > But we probably want some sort of I/O test in qemu-test too since it > provides a simple functional test. Yes, there's overlap, but the > functional form of the test is so simple that it really isn't that > important. > > But testing how QEMU handles malformatted virtio-blk requests is > something that's clearly QEMU specific. There's no value doing that > with a Linux and Windows guest (even if it's somehow possible). It's > definitely a unit test that's strictly specific to QEMU. > > I think autotest should be able to execute QEMU's unit and functional > tests. By using a test framework like gtest that exposes a test > protocol, it should be trivial to add that support. > > Regards, > > Anthony Liguori > >> Paolo >