Re: [Autotest] [PATCH] [RFC] KVM test: Major control file cleanup

[Ryan Harper <ryanh@us.ibm.com>]

* Michael Goldish <mgoldish@redhat.com> [2009-10-31 16:02]:
> > 
> > If I'm understanding things, we are talking about moving the large
> > body
> > of kvm_tests.cfg test definitions, guest definitions into a
> > "library",
> > and then moving the requested test config (bottom on kvm_tests.cfg)
> > into
> > the control file itself which means the autotest webui would be able
> > to
> > control which tests get run;  I like this idea very well.  My concern
> > that I mentioned is that as you edit the "library" it can be
> > difficult
> > to ensure you described exactly which set of tests on which guests
> > you
> > want to run and kvm_config.py is invaluable in the process of getting
> > it
> > right.
> > 
> > Why not have kvm_config.py , or some other wrapper generate a
> > "kvm_tests.cfg" file dynamically from the "library" and the strings
> > from
> > the control file?  That way we could still debug configuration via
> > kvm_config.py?  I much perfer this over queueing up jobs in the
> > webiu,
> > waiting for it to run, checking the results in the DEBUG dir,
> > adjusting,
> > repeat.
> 
> I'm not sure I understand your idea: you want some program to read the
> control file and generate a new file (kvm_tests.cfg or something) from
> the control file and the library file, so that this file can be debugged
> with kvm_config.py?

I wanted something that would take the test description string from the
control file, run that through the library such that it can print out
the tests that will run -- the equivalent of running kvm_config.py on a
kvm_tests.cfg file... 

> 
> IMO this solution is "dirty" because the control file is python code, not
> our own format, so it's not nice to automatically extract stuff from it.
> It would be nice to do something that eases debugging, but if you ask me,
> I'd rather have something as clean as possible.
> 
> Here's another idea, which I suggested but haven't received any feedback
> on: let's write a little proggie that runs the control file just like
> client/bin/autotest does.  The proggie will supply the control file with
> a fake job object that has nothing but a run_test() method, but instead
> of running a test, that method will simply nicely print out the test
> params, like kvm_config.py does.  So the user will be able to do something
> like './dry_run.py control.mine' which will list all the tests to be
> executed.  We might also want to implement job.parallel() in addition to
> job.run_test() but that should be very easy (it doesn't really have to be
> parallel at all).
> Does that make any sense?

It does, though I don't see how it's different then what I suggested; I
don't really care how we do it;  I still need something to debug test
config files outside of cycling through job output via the webui;


-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
ryanh@us.ibm.com