From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:39026) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5nJu-0005wB-FG for qemu-devel@nongnu.org; Thu, 08 Mar 2012 19:04:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S5nJr-0000Y5-Rx for qemu-devel@nongnu.org; Thu, 08 Mar 2012 19:04:29 -0500 Received: from mail-pz0-f45.google.com ([209.85.210.45]:38665) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5nJr-0000W5-M5 for qemu-devel@nongnu.org; Thu, 08 Mar 2012 19:04:27 -0500 Received: by dadp14 with SMTP id p14so1069376dad.4 for ; Thu, 08 Mar 2012 16:04:25 -0800 (PST) Message-ID: <4F594905.3020202@codemonkey.ws> Date: Thu, 08 Mar 2012 18:04:21 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <4F582EDB.1040608@redhat.com> <4F58B5CB.8040503@codemonkey.ws> <20120308144958.GA25750@t420s.optimusnet> <4F58C897.5020405@codemonkey.ws> <20120308150722.GA30576@t420s.optimusnet> <4F58CCBA.9000702@codemonkey.ws> <20120308160505.GA32360@t420s.optimusnet> <4F58E67A.3050708@codemonkey.ws> <20120308175907.GA4900@t420s.optimusnet> <4F5905AA.3060304@codemonkey.ws> <20120308210209.GA11998@t420s.optimusnet> <4F59237F.6010406@codemonkey.ws> <4F593BAF.1020201@redhat.com> In-Reply-To: <4F593BAF.1020201@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Lucas Meneghel Rodrigues Cc: Scott Zawalski , Cleber Rosa , QEMU devel , "kvm-autotest@redhat.com" , Ademar Reis On 03/08/2012 05:07 PM, Lucas Meneghel Rodrigues wrote: > On 03/08/2012 06:24 PM, Anthony Liguori wrote: >>> >>> Cons: >>> - Lot of code will be duplicated to cover the main code paths: >>> writting tests will require writting/supporting considerable >>> ammount of code (that already exists in autotest). >> >> Again, existence proof that this isn't true. > > Case in point, the virtio test (that uses an auxiliary script to send data to > the host). Can you tell me if both tests cover even remotely the same amount of > functionality? > > https://github.com/autotest/autotest/blob/master/client/tests/kvm/tests/virtio_console.py > > > https://github.com/autotest/autotest/blob/master/client/virt/scripts/virtio_console_guest.py > > > Here is the qemu-test version > > http://git.qemu.org/?p=qemu-test.git;a=blob;f=tests/virtio-serial.sh;h=e95ae6e0b63758262919702d51a9c83bebe2fb08;hb=master I seem to recall us having this same discussion in the past.... But nevertheless, I started to respond in detail but decided to wait until tomorrow. It's been a long day and I think this thread is a bit too heated already. I think I'll be a bit less grumpy in the morning :-) But there's one thing I want to point out... > What the qemu-test version covers: > * host starts qemu with one virtio console device backed by a file > * guest verifies if the name of the device is correct > * guest writes to the console device > * host verifies if guest wrote to the virtio console > > What the virtio-console covers: > > * Sends data between host and guest back and forth, validates the data being > sent, for both small and large amounts of data, both random or sequential. > * Tests write/send in blocking, polling, selecting mode, with port mode sync/async This bullet (and many of the bullets) below are not tests of QEMU. They are tests of the Linux kernel. This is an integration test, not a functional test of QEMU. We absolutely need integration tests and hands down, that's where autotest is going to shine. This is clearly not in the scope of qemu-test and there's been a few discussions on the ML on this including me rejecting a patch that crossed this boundary. Right now autotest effectively does everything because we (QEMU) have not fulfilled our responsibility here. Regards, Anthony Liguori