qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Willian Rampazzo <wrampazz@redhat.com>
To: Cleber Rosa <crosa@redhat.com>
Cc: qemu-ppc@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
	"Beraldo Leal" <bleal@redhat.com>,
	"Yoshinori Sato" <ysato@users.sourceforge.jp>,
	"Pavel Dovgalyuk" <pavel.dovgaluk@ispras.ru>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	qemu-devel <qemu-devel@nongnu.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Greg Kurz" <groug@kaod.org>,
	"Auger Eric" <eric.auger@redhat.com>,
	"Kamil Rytarowski" <kamil@netbsd.org>,
	"Hervé Poussineau" <hpoussin@reactos.org>,
	"Reinoud Zandijk" <reinoud@netbsd.org>,
	"David Gibson" <david@gibson.dropbear.id.au>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Alexandre Iooss" <erdnaxe@crans.org>,
	"Mahmoud Mandour" <ma.mandourr@gmail.com>,
	"Ryo ONODERA" <ryoon@netbsd.org>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Eduardo Habkost" <ehabkost@redhat.com>
Subject: Re: [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0
Date: Wed, 6 Oct 2021 15:39:59 -0300	[thread overview]
Message-ID: <CAKJDGDagReANLxsA5tnqQ0sjU6qrx=LiMbB4Jq2LUc-UtHpdMA@mail.gmail.com> (raw)
In-Reply-To: <20210924185506.2542588-2-crosa@redhat.com>

On Fri, Sep 24, 2021 at 3:57 PM Cleber Rosa <crosa@redhat.com> wrote:
>
> Avocado 91.0, brings, among other changes, a switch to a new runner
> implementation, known as "nrunner".  While my personal testing shows
> that it's suitable for running the QEMU tests, there are some
> considerations that should be addressed before the change.
>
> For instance, the "nrunner" implementation will run tests in parallel
> by default.  With more tests running, and resources shared, some tests
> may fail with timeouts on some environments.
>
> So, for now, let's bump the Avocado version to allow for the other
> features to be used.  And the "nrunner" implementation to be used by
> those that choose to do so.
>
> More information can be found at the release notes at:
>    https://avocado-framework.readthedocs.io/en/latest/releases/91_0.html
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  docs/devel/testing.rst | 12 ++++++++++++
>  tests/Makefile.include |  2 ++
>  tests/requirements.txt |  2 +-
>  3 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 4a0abbf23d..2818c58ff8 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -740,6 +740,18 @@ may be invoked by running:
>
>    tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
>
> +Running tests in parallel
> +-------------------------
> +
> +The current ``make check-acceptance`` target continues to use the
> +Avocado runner implementation simply known as "runner".  But, it's now
> +possible to opt-in and choose the "nrunner" implementation, which,
> +among other things, allows for parallel execution of tests:
> +
> +.. code::
> +
> +  make AVOCADO_RUNNER=nrunner check-acceptance

The examples in the documentation are all setting the environment
variable at the end of the `make` command. For consistency, I would
move the definition of the AVOCADO_RUNNER to the end of the command.

> +
>  Manual Installation
>  -------------------
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 6e16c05f10..ac289a2e41 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -92,6 +92,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
>  # Any number of command separated loggers are accepted.  For more
>  # information please refer to "avocado --help".
>  AVOCADO_SHOW=app
> +AVOCADO_RUNNER=runner
>  AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
>
>  $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> @@ -128,6 +129,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>         $(call quiet-command, \
>              $(TESTS_VENV_DIR)/bin/python -m avocado \
>              --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> +            --test-runner=$(AVOCADO_RUNNER) \
>              --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
>              $(AVOCADO_TAGS) \
>              $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index a21b59b443..40af24c664 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -1,5 +1,5 @@
>  # Add Python module requirements, one per line, to be installed
>  # in the tests/venv Python virtual environment. For more info,
>  # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> -avocado-framework==88.1
> +avocado-framework==91.0
>  pycdlib==1.11.0
> --
> 2.31.1
>

Apart from a tiny documentation change, the code is welcome as it
allows using the new Avocado runner.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>



  reply	other threads:[~2021-10-06 18:42 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-24 18:54 [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Cleber Rosa
2021-09-24 18:54 ` [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0 Cleber Rosa
2021-10-06 18:39   ` Willian Rampazzo [this message]
2021-09-24 18:54 ` [PATCH 02/16] Acceptance Tests: improve check-acceptance description Cleber Rosa
2021-09-27 14:40   ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 03/16] Acceptance Tests: add mechanism for listing tests Cleber Rosa
2021-10-06 18:47   ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 04/16] Acceptance Tests: keep track and disable tests with known issues Cleber Rosa
2021-09-24 18:54 ` [PATCH 05/16] Acceptance Tests: add standard clean up at test tearDown() Cleber Rosa
2021-09-24 19:38   ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 06/16] Acceptance Tests: use extract from package from avocado.utils Cleber Rosa
2021-09-24 18:54 ` [PATCH 07/16] Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert Cleber Rosa
2021-09-24 18:54 ` [PATCH 08/16] acceptance/tests/vnc.py: use explicit syntax for enabling passwords Cleber Rosa
2021-09-27 14:41   ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 09/16] tests/acceptance/boot_xen.py: merge base classes Cleber Rosa
2021-10-06 18:57   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 10/16] tests/acceptance/boot_xen.py: unify tags Cleber Rosa
2021-10-06 18:56   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 11/16] tests/acceptance/boot_xen.py: fetch kernel during test setUp() Cleber Rosa
2021-10-06 18:55   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 12/16] tests/acceptance/boot_xen.py: removed unused import Cleber Rosa
2021-09-27 14:43   ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 13/16] tests/acceptance/boot_xen.py: use class attribute Cleber Rosa
2021-10-06 18:58   ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 14/16] tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update Cleber Rosa
2021-09-27 15:26   ` Philippe Mathieu-Daudé
2021-09-30 19:19     ` Reinoud Zandijk
2021-09-30 19:44       ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 15/16] tests/acceptance/ppc_prep_40p.py: clean up unused import Cleber Rosa
2021-09-27 14:42   ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 16/16] tests/acceptance/ppc_prep_40p.py: unify tags Cleber Rosa
2021-09-27 14:45   ` Philippe Mathieu-Daudé
2021-10-06 19:17   ` Willian Rampazzo
2021-10-06 19:19     ` Willian Rampazzo
2021-09-27  9:09 ` [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Pavel Dovgalyuk
2021-09-27 16:46 ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAKJDGDagReANLxsA5tnqQ0sjU6qrx=LiMbB4Jq2LUc-UtHpdMA@mail.gmail.com' \
    --to=wrampazz@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=bleal@redhat.com \
    --cc=crosa@redhat.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=ehabkost@redhat.com \
    --cc=erdnaxe@crans.org \
    --cc=eric.auger@redhat.com \
    --cc=groug@kaod.org \
    --cc=hpoussin@reactos.org \
    --cc=kamil@netbsd.org \
    --cc=ma.mandourr@gmail.com \
    --cc=pavel.dovgaluk@ispras.ru \
    --cc=pbonzini@redhat.com \
    --cc=philmd@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=reinoud@netbsd.org \
    --cc=ryoon@netbsd.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=ysato@users.sourceforge.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).