Re: [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment

[Paolo Bonzini <pbonzini@redhat.com>]

On 5/13/22 02:06, John Snow wrote:
> The only downside I am really frowning at is that I will have to
> replicate some "update the venv if it's outdated" logic that is usually
> handled by the Make system in the venv bootstrapper. Still, I think it's
> probably the only way to hit all of the requirements here without trying
> to concoct a fairly complex Makefile.
> 
> any thoughts? If not, I'll just move on to trying to hack up that
> version next instead.

I would not even bother with keeping the venv up to date.  Just initialize
it in configure, this is exactly what configure remains useful for in the
Meson-based world:

- add configure options --enable-python-qemu={enabled,system,internal,pip,
auto}/--disable-python-qemu (auto means system>internal>pip>disabled; enabled means
system>internal>pip>error) and matching CONFIG_PYTHON_QEMU=y to
config-host.mak

- use CONFIG_PYTHON_QEMU to enable/disable iotests in tests/qemu-iotests/meson.build

- add a configure option --enable-avocado=
{system,pip,auto,enabled}/--disable-avocado and matching
CONFIG_AVOCADO=y to config-host.mak

- use it to enable/disable acceptance tests in tests/Makefile.include

- build the venv in configure and use the options to pick the right pip install
commands, like

has_python_module() {
   $python -c "import $1" > /dev/null 2>&1
}

# do_pip VENV-PATH VAR PACKAGE [PATH] -- PIP-OPTIONS
do_pip() {
     local num_args source
     num_args=5
     test $4 = '--' && num_args=4
     eval source=\$$2
     # Try to resolve the package using a system install
     case $source in
       enabled|auto|system)
         if has_python_module $3; then
           source=system
         elif test $source = system; then
           error_exit "Python package $3 not installed"
         fi
     esac
     # See if a bundled copy is present
     case $source in
       enabled|auto|internal)
         if test $num_args = 5 && test -f $4/setup.cfg; then
           source=internal
         elif test $source = internal; then
           error_exit "Sources for Python package $3 not found in the QEMU source tree"
         fi
     esac
     # Install the bundled copy or let pip download the package
     case $source in
       internal)
         # The Pip error message should be clear enough
         (cd $1 && . bin/activate && pip install "$@") || exit 1
       ;;
       enabled|auto|pip)
         shift $num_args
         if (cd $1 && . bin/activate && pip install "$@"); then
           source=pip
         elif test $source = auto; then
           source=disabled
         else
           # The Pip error message should be clear enough
           exit 1
         fi
       ;;
     esac
     eval $2=\$source
}

rm -rf venv/
$python -m venv venv/
do_pip venv/ enable_python_qemu qemu.qmp python/qemu -- qemu.qmp
do_pip venv/ enable_avocado avocado -- -r tests/requirements.txt