From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ryan Harper Subject: Re: [Autotest] [PATCH] [RFC] KVM test: Major control file cleanup Date: Mon, 2 Nov 2009 16:10:40 -0600 Message-ID: <20091102221040.GQ13808@us.ibm.com> References: <1285393002.1333521257019268689.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> <1276240077.1333571257019276425.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Ryan Harper , autotest@test.kernel.org, uril@redhat.com, kvm@vger.kernel.org, Lucas Meneghel Rodrigues To: Michael Goldish Return-path: Received: from e39.co.us.ibm.com ([32.97.110.160]:38292 "EHLO e39.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756769AbZKBWK6 (ORCPT ); Mon, 2 Nov 2009 17:10:58 -0500 Received: from d03relay05.boulder.ibm.com (d03relay05.boulder.ibm.com [9.17.195.107]) by e39.co.us.ibm.com (8.14.3/8.13.1) with ESMTP id nA2M50BQ029251 for ; Mon, 2 Nov 2009 15:05:00 -0700 Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by d03relay05.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id nA2MAkTf116432 for ; Mon, 2 Nov 2009 15:10:50 -0700 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id nA2M7uOi030136 for ; Mon, 2 Nov 2009 15:07:56 -0700 Content-Disposition: inline In-Reply-To: <1276240077.1333571257019276425.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: * Michael Goldish [2009-10-31 16:02]: > > > > If I'm understanding things, we are talking about moving the large > > body > > of kvm_tests.cfg test definitions, guest definitions into a > > "library", > > and then moving the requested test config (bottom on kvm_tests.cfg) > > into > > the control file itself which means the autotest webui would be able > > to > > control which tests get run; I like this idea very well. My concern > > that I mentioned is that as you edit the "library" it can be > > difficult > > to ensure you described exactly which set of tests on which guests > > you > > want to run and kvm_config.py is invaluable in the process of getting > > it > > right. > > > > Why not have kvm_config.py , or some other wrapper generate a > > "kvm_tests.cfg" file dynamically from the "library" and the strings > > from > > the control file? That way we could still debug configuration via > > kvm_config.py? I much perfer this over queueing up jobs in the > > webiu, > > waiting for it to run, checking the results in the DEBUG dir, > > adjusting, > > repeat. > > I'm not sure I understand your idea: you want some program to read the > control file and generate a new file (kvm_tests.cfg or something) from > the control file and the library file, so that this file can be debugged > with kvm_config.py? I wanted something that would take the test description string from the control file, run that through the library such that it can print out the tests that will run -- the equivalent of running kvm_config.py on a kvm_tests.cfg file... > > IMO this solution is "dirty" because the control file is python code, not > our own format, so it's not nice to automatically extract stuff from it. > It would be nice to do something that eases debugging, but if you ask me, > I'd rather have something as clean as possible. > > Here's another idea, which I suggested but haven't received any feedback > on: let's write a little proggie that runs the control file just like > client/bin/autotest does. The proggie will supply the control file with > a fake job object that has nothing but a run_test() method, but instead > of running a test, that method will simply nicely print out the test > params, like kvm_config.py does. So the user will be able to do something > like './dry_run.py control.mine' which will list all the tests to be > executed. We might also want to implement job.parallel() in addition to > job.run_test() but that should be very easy (it doesn't really have to be > parallel at all). > Does that make any sense? It does, though I don't see how it's different then what I suggested; I don't really care how we do it; I still need something to debug test config files outside of cycling through job output via the webui; -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@us.ibm.com