Re: [PATCH v3 0/9] tests: Add test cases for TPM 1.2 ACPI tables

[Igor Mammedov <imammedo@redhat.com>]

On Mon, 19 Jul 2021 15:56:16 +0200
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> On Mon, Jul 19, 2021 at 3:50 PM Markus Armbruster <armbru@redhat.com> wrote:
> > Igor Mammedov <imammedo@redhat.com> writes:  
> > > On Thu, 15 Jul 2021 07:49:13 +0200
> > > Markus Armbruster <armbru@redhat.com> wrote:  
> > >> Philippe Mathieu-Daudé <philmd@redhat.com> writes:  
> 
> > >> >>> IMO the "right" solution is to check via QMP if TMP is supported
> > >> >>> or not. This is now doable since commit caff255a546 ("tpm: Return
> > >> >>> QMP error when TPM is disabled in build").
> > >> >>>
> > >> >>> Long term we'd like to decouple the tests/ build from the various
> > >> >>> QEMU configurations, and build the tests once.  
> > >> >>
> > >> >> This argument applies only to macros from target-specific headers like
> > >> >> $TARGET-config-target.h, not to macros from config-host.h.  #ifdef
> > >> >> CONFIG_TPM should be fine, shouldn't it?  
> > >> >
> > >> > Some definitions depend on the host (OS, libraries installed, ...),
> > >> > others depend on the --enable/--disable ./configure options.
> > >> >
> > >> > IMO it would be nice if we could get qtests independent of the latter.  
> > >>
> > >> Why?  
> > >
> > > In another mail-thread Philippe mentioned that there is desire
> > > to use qtest out of tree to test other QEMU binaries.
> > >
> > > However, just probing for features at runtime aren't going
> > > to help with the goal as tests are tailored for the latest
> > > CLI/QMP/ABI. To make it work we would have practically
> > > introduce versioned tests.
> > >
> > > So I wonder why one external acceptance-tests suite is not
> > > sufficient, that we would want to hijack relatively simple
> > > internal qtest at expense of increased resources needed to
> > > run/write unit tests.  
> >
> > Yes.  qtest was not designed for use with anything but HEAD, and I doubt
> > we can make it fit such uses at reasonable expense.  
> 
> One HEAD but multiple configurations...

Even assuming reconfigure won't cause world rebuild,
It will be a win only if number of configuration probes
is small.
However it doesn't scale for large numbers and it might be
faster to rebuild affected tests in the end. (worst case: #probes * #targets)
I wonder if we can do probing once & cache it somewhere to avoid ^^^.


> If you want to simplify human time, can we simply run qtests once per
> arch/OS but with all features enabled? Otherwise skip qtests?
>