Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)

["Michael S. Tsirkin" <mst@redhat.com>]

On Fri, Jul 01, 2022 at 12:58:33PM +0530, Ani Sinha wrote:
> On Fri, Jul 1, 2022 at 12:23 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Fri, Jul 01, 2022 at 06:12:14AM +0200, Thomas Huth wrote:
> > > I even wouldn't mind if you put your python stuff in a new directory like
> > > tests/pytests/ for example, as long as it downloads your binaries separately
> > > - as I wrote in another mail, the avocado framework rather looks like an
> > > oddball in our test framework nowadays since it uses a separate test runner
> > > and not the meson test harness, so having a new approach for python-based
> > > tests is maybe even a good idea. I just really want to avoid that this goes
> > > into tests/qtest (since it really does not belong there), and please don't
> > > add more external stuff via git submodules, that's really the wrong approach
> > > for this.
> >
> > I get it, people hate submodules with passion.  I think trying another
> > approach for testing that is neither avocado nor qtest is
> > not too bad. As long as this is not user visible, we can
> > allow ourselves space to experiment.
> >
> > OK so, how about this:
> > - put it in a new directory: tests/roms?
> > - create repo for a fork of biosbits under git.qemu.org
> > - roll our own analog to git submodules: a script
> >   that clones the repo
> 
> No need to clone the whole repo. We can simply download the binaries
> that the girlab CI job would generate from the bits sources in that
> repo.
> We need to clone if we are always building bits from source for every
> test. That is not necessary IMHO since much of the bits package would
> remain as is without modification.

IMHO CI job idea isn't great since isn't versioned at all, is it?
Also as long as test passes, sure. But if it fails one will
need the sources to investigate.

Let's start with building things from source. Add an option
of prebuilt binaries as an optimization once things
stabilize.


> > - new target make check-roms,
> 
> I think make pytest or some such is better and more generic if other
> such tests in other areas follow suit.

The name is not critical in my mind, but I think we need to decide
what exactly differentiates it from other tests.


> 
> if the clone exists locally -
> >   run the test, if not - skip it
> 
> if download of the bits binaries fail, skip it.

You seem to be recreating either git or avocado or both here.

Personally I want something that works offline.



> > - as for using pre-generates ISOs as an optimization,
> >   I'm not sure how important that is, if yes -
> >   we can add another repo and another make target along the
> >   same lines
> >
> >
> >
> > --
> > MST
> >