Re: [PATCH v6 00/10] Introduce new acpi/smbios avocado tests using biosbits

["Michael S. Tsirkin" <mst@redhat.com>]

On Sat, Oct 22, 2022 at 10:13:13PM +0530, Ani Sinha wrote:
> 
> 
> On Sat, Oct 22, 2022 at 22:05 Michael S. Tsirkin <mst@redhat.com> wrote:
> 
>     On Sat, Oct 22, 2022 at 06:28:32AM +0530, Ani Sinha wrote:
>     >
>     >
>     > On Fri, Oct 21, 2022 at 21:32 Alex Bennée <alex.bennee@linaro.org> wrote:
>     >
>     >
>     >     Ani Sinha <ani@anisinha.ca> writes:
>     >
>     >     > On Fri, 21 Oct, 2022, 5:52 pm Ani Sinha, <ani@anisinha.ca> wrote:
>     >     >
>     >     >  On Fri, 21 Oct, 2022, 5:26 pm Alex Bennée, <alex.bennee@linaro.org
>     >
>     >     wrote:
>     >     >
>     >     >  Ani Sinha <ani@anisinha.ca> writes:
>     >     >
>     >     >  > On Fri, Oct 21, 2022 at 3:10 PM Michael S. Tsirkin <
>     mst@redhat.com>
>     >     wrote:
>     >     >  >>
>     >     >  >> On Fri, Oct 21, 2022 at 10:30:09AM +0100, Alex Bennée wrote:
>     >     >  >> >
>     >     >  >> > Ani Sinha <ani@anisinha.ca> writes:
>     >     >  >> >
>     >     >  >> > > On Fri, Oct 21, 2022 at 2:02 PM Michael S. Tsirkin <
>     >     mst@redhat.com> wrote:
>     >     >  >> > >>
>     >     >  >> > >> On Fri, Oct 21, 2022 at 05:45:15AM +0530, Ani Sinha wrote:
>     >     >  >> > >> > And have multiple platform specific branches in bits
>     that have
>     >     fixes for those
>     >     >  >> > >> > platforms so that bits can run there. Plus the existing
>     test
>     >     can be enhanced to
>     >     >  >> > >> > pull in binaries from those branches based on the
>     platform on
>     >     which it is being
>     >     >  >> > >> > run.
>     >     >  >> > >> >
>     >     >  >> > >>
>     >     >  >> > >> What a mess.
>     >     >  >> > >> Who is going to be testing all these million platforms?
>     >     >  >> > >
>     >     >  >> > > I am not talking about branches in QEMU but branches in
>     bits.
>     >     >  >> > > If you are going to test multiple platforms, you do need to
>     build
>     >     bits
>     >     >  >> > > binaries for them. There is no way around it.
>     >     >  >> > > bits is not all platform independent python. It does have
>     binary
>     >     executables.
>     >     >  >> > >
>     >     >  >> > > Currently bits is built only for the x86 platform. Other
>     >     platforms are
>     >     >  >> > > not tested. I doubt if anyone even tried building bits for
>     arm or
>     >     >  >> > > mips.
>     >     >  >> >
>     >     >  >> > I'm not worried about test bits on other targets, but we do
>     run x86
>     >     >  >> > targets on a number of hosts. The current reliance on a
>     special
>     >     patched
>     >     >  >> > host build tool for only one architecture is the problem. If 
>     we
>     >     just
>     >     >  >> > download the iso that problem goes away.
>     >     >  >>
>     >     >  >> 👍what he said.
>     >     >  >
>     >     >  > Yes, in that case the problem is that upstream bits does not
>     pass all
>     >     >  > the test out of the box. Hence we are taking this approach of
>     keeping
>     >     >  > some test scripts in QEMU repo and modifying them. Then
>     generating the
>     >     >  > iso with the modified scripts. It also helps developers who want
>     to
>     >     >  > write new tests or make enhancements to existing tests.
>     >     >  > If modifications need to be made to tests, they need to be
>     versioned.
>     >     >  > We have gone through the route of not using submodules and I am
>     not
>     >     >  > going to open that can of worms again.
>     >     >
>     >     >  We have added a mirror of biosbits to the QEMU project so there is
>     no
>     >     >  reason why we can't track changes and modifications there (we do
>     this
>     >     >  for TestFloat which is forked from the upstream SoftFloat code).
>     >
>     >
>     > One last option. Commit this patch set but also double commit patch 3 to
>     the
>     > bits repo so that we can build an iso that would successfully run all
>     tests for
>     > a separate platform independent test to be written later.
>     >
>     > Then we will have two tests:
>     >
>     > - this one for developers writing new test.
>     > - platform independent one for a basic sanity.
>     >
>     > I’m just documenting the fact that I have proposed ideas that can work
>     where
>     > all can be happy. It’s up to others to take it or keep objecting and
>     killing
>     > motivations for freelance contributors. 
> 
>     I think it's ok to apply this as is for starters.
>     Anyone has objections?
> 
>     Down the road I think things should be refactored slightly to work as
>     follows:
>     - test developers can check out biosbits repo to create the iso
>     - everyone else gets iso downloaded
> 
> 
> It will be difficult to convince test developers to check out another repo and
> go back and forth between two repos. If the bits repo was a sub module that’s
> another story.
> 
> Test developers should use the test scripts from qemu repo. Someone then later
> can incrementally commit these new tests into bits repo and generate newer iso
> at some periodic intervals. Since I am the maintainer of the bits repo I can do
> the second part.

Sounds like a plan. Anyway, I think we can worry about that down the road
after something is merged.


> 
> 
> 
> 
>     Objections to this plan?
> 
> 
>     >
>     >
>     >
>     >     >
>     >     > 
>     >
> 
>