From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:54582) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5cRT-00071Q-Uo for qemu-devel@nongnu.org; Thu, 08 Mar 2012 07:27:45 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S5cRL-0001nl-14 for qemu-devel@nongnu.org; Thu, 08 Mar 2012 07:27:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52239) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5cRK-0001nG-OO for qemu-devel@nongnu.org; Thu, 08 Mar 2012 07:27:26 -0500 Message-ID: <4F58A604.80209@redhat.com> Date: Thu, 08 Mar 2012 09:28:52 -0300 From: Cleber Rosa MIME-Version: 1.0 References: <4F582EDB.1040608@redhat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests Reply-To: cleber@redhat.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Lucas Meneghel Rodrigues , QEMU devel , Scott Zawalski , Autotest Mailing List , "kvm-autotest@redhat.com" , Ademar Reis On 03/08/2012 08:54 AM, Stefan Hajnoczi wrote: > On Thu, Mar 8, 2012 at 11:44 AM, Stefan Hajnoczi wrote: >> On Thu, Mar 8, 2012 at 4:00 AM, Lucas Meneghel Rodrigues wrote: >>> One of our main goals is to provide useful tools for the qemu community, >>> since we have a good number of tests and libraries written to perform >>> integration/QA testing for that tool, being successfuly used by a number of >>> QA teams that work on qemu. Also, we recently provided a subset of that >>> infrastructure to test libvirt, one of our virtualization projects of >>> interest. >> Thanks for sharing. > One thing I should have added is that my message is about what it > would take for me to use autotest and contribute tests. But now I > realize that you might be going for a different model: > > If you're aiming for a different model where autotest integrates > external test suites (i.e. tests wouldn't be written in autotest.git, > instead autotest.git would contain snapshots of external test suites), > then this proposal seems fine. Upstream projects like QEMU would > develop their own test suite and it would be dropped into autotest or > a specific autotest instance. > > I'm not 100% sure which of these two models you're going for? Autotest will continue to integrate and ship with external test suites, even though that's an option at packaging time. But the point here is that we also want to cover the other use cases, which includes being able to run tests that are hosted within external projects, such as QEMU itself. The idea is to put two things in a state that's easier to be consumed by individual developers: * The test runner * The (optional) autotest API So, by making the autotest API optional, and even the language of the test script your own choice, you can keep you current test code, using your own mini-framework and still use the autotest test runner for running the tests and gathering the results and important system information. By improving the API, which basically means making it more visible, better organized and documented, we hope that users writing instrumented tests (using serial or ssh sessions, sending either HMP or QMP monitor commands, etc) will choose to use it. CR. > > Stefan