From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:60290) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5yJm-0007qE-Ms for qemu-devel@nongnu.org; Fri, 09 Mar 2012 06:49:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S5yJO-0006dD-1I for qemu-devel@nongnu.org; Fri, 09 Mar 2012 06:49:06 -0500 Received: from mx1.redhat.com ([209.132.183.28]:29725) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S5yJN-0006cf-QR for qemu-devel@nongnu.org; Fri, 09 Mar 2012 06:48:41 -0500 Message-ID: <4F59EE0E.9070202@redhat.com> Date: Fri, 09 Mar 2012 19:48:30 +0800 From: Osier Yang MIME-Version: 1.0 References: <4F582EDB.1040608@redhat.com> <20120308121742.GA3509@t420s.optimusnet> In-Reply-To: <20120308121742.GA3509@t420s.optimusnet> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [KVM-AUTOTEST] [RFC] Future goals for autotest and virtualization tests List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Ademar Reis Cc: Autotest Mailing List , Stefan Hajnoczi , Scott Zawalski , QEMU devel , "kvm-autotest@redhat.com" On 2012=E5=B9=B403=E6=9C=8808=E6=97=A5 20:17, Ademar Reis wrote: > On Thu, Mar 08, 2012 at 11:54:31AM +0000, Stefan Hajnoczi wrote: >> On Thu, Mar 8, 2012 at 11:44 AM, Stefan Hajnoczi = wrote: >>> On Thu, Mar 8, 2012 at 4:00 AM, Lucas Meneghel Rodrigues wrote: >>>> One of our main goals is to provide useful tools for the qemu commun= ity, >>>> since we have a good number of tests and libraries written to perfor= m >>>> integration/QA testing for that tool, being successfuly used by a nu= mber of >>>> QA teams that work on qemu. Also, we recently provided a subset of t= hat >>>> infrastructure to test libvirt, one of our virtualization projects o= f >>>> interest. >>> >>> Thanks for sharing. >> >> One thing I should have added is that my message is about what it >> would take for me to use autotest and contribute tests. But now I >> realize that you might be going for a different model: >> >> If you're aiming for a different model where autotest integrates >> external test suites (i.e. tests wouldn't be written in autotest.git, >> instead autotest.git would contain snapshots of external test suites), >> then this proposal seems fine. Upstream projects like QEMU would >> develop their own test suite and it would be dropped into autotest or >> a specific autotest instance. >> > > Yes, that's the idea. We want autotest to be the framework, not > (just a) collection of tests. We also want each development team > to implement and maintain their own set of tests, using (or not) > the goodies from autotest at their discretion. > > In summary, autotest is (or is going to be) a framework that > provides: > > - A test runner, with grid/cluster support and advanced > instrumentation > - A devel library and set of utilities for test writers > - A set of pre-built images (JeOS =E2=80=93 Just Enough OS) for > test writers > > (attached is a picture showing what we want to achieve) > > If a project has an internal library or set of utilities that can > be of general use, they can be submitted to autotest.git for > inclusion, thus reaching a broader audience. > > A short summary of the plans: > > - Tests can live anywhere and each devel team implements and > maintains their own set of tests > - Usage of the autotest library by test writers is optional > - Tests are scripts returning 0 or error (any language) > - Tests can be run individually or in sets > - Tests should run fast, our target is seconds or a few minutes > - The test runner is smart and =E2=80=9Cjust works=E2=80=9D by defaul= t > - Trivial standard output (FAIL, PASS, SKIPPED) > - Collect logs, OS data and other stuff (e.g. --record-video!) Is there a way to collect the logs generated by external test suite itself? It could be in the specific path. Osier