Re: [PATCH kvm-unit-tests 5/5] x86-run: Pull extra arguments from unittests.cfg

[Gleb Natapov <gleb@redhat.com>]

On Mon, Mar 25, 2013 at 02:30:27PM -0400, Cole Robinson wrote:
> On 03/20/2013 03:06 PM, Marcelo Tosatti wrote:
> > On Sun, Mar 17, 2013 at 07:58:56PM -0400, Cole Robinson wrote:
> >> On 03/17/2013 11:25 AM, Gleb Natapov wrote:
> >>> On Fri, Mar 15, 2013 at 08:09:09PM -0400, Cole Robinson wrote:
> >>>> Some tests want extra arguments as enumerated in unittests.cfg,
> >>>> use them.
> >>>>
> >>>> unittests.cfg also has a few sections about invoking certains tests
> >>>> with different combinations of options, but x86-run doesn't do
> >>>> anything with that.
> >>> With this it will not be possible to use x86-run outside of autotest,
> >>> no?
> >>>
> >>
> >> Not true, x86-run is still meant to be the standalone helper script for
> >> running unittests. autotest doesn't care about x86-run, and ConfigParser is a
> >> standard python module.
> >>
> >> x86/unittests.cfg already exists in the kvm-unit-tests repo, I assumed it was
> >> encoding required test options but maybe I'm wrong about that. It's still
> >> useful to build off of if there's value in running some tests with different
> >> combinations of parameters.
> > 
> > I fail to see what is the point here.
> > 
> > unittests.cfg has been originally (and continues to be, AFAIK), intended
> > for autotest:
> > http://kerneltrap.org/mailarchive/linux-kvm/2010/6/24/6264146
> > 
> > Please don't remove manual execution from README.
> 
> I guess I'm misunderstanding things here.
> 
> I understood that unittests.cfg was only being used by autotest. What confused
> me was that this config enumerates different command line options for some of
> the tests. Why is autotest invoking these test cases with different options
> than the README suggests? Which one is right?
> 
Neither. README just have examples how to enable test device in various
versions of QEMU. unittests.cfg suppose to have reasonable configuration
for each test which is less likely to fail on most kernel. For instance
I may add a fix for a kernel bug and a test case for the bug. Since the
bug is not fixed on any released kernel autotest will start to fail,
so it may sense to disable the test in unittests.cfg meanwhile. Or I can
add a test that crashes kernel if it fails. It is probably not wise to
enable it for autotest, but running it manually will be still possible.

> If unittests.cfg is the canonical list of options, than the README needs to
> document different command lines for each test. If the README is canonical,
> then I don't understand why autotest is using their own options.
> 
I do not quite understand what do you mean here. README just have an
example on how to enable test device in QEMU. unittests.cfg does not
control that.

> My intent was to unify the two, so that autotest wouldn't reproduce an error
> that the kvm-unit-test docs might miss. And if there's ever a new unit test
> added that requires non-default qemu options, or is useful to run with a few
> permutations of cli options, unittests.cfg can be extended and x86-run will
> just do the right thing.
> 
> Does that make sense?
> 
> Thanks,
> Cole

--
			Gleb.