From: Willian Rampazzo <wrampazz@redhat.com>
To: Cleber Rosa <crosa@redhat.com>
Cc: qemu-ppc@nongnu.org, "Thomas Huth" <thuth@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>,
"Yoshinori Sato" <ysato@users.sourceforge.jp>,
"Pavel Dovgalyuk" <pavel.dovgaluk@ispras.ru>,
"Alex Bennée" <alex.bennee@linaro.org>,
qemu-devel <qemu-devel@nongnu.org>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Greg Kurz" <groug@kaod.org>,
"Auger Eric" <eric.auger@redhat.com>,
"Kamil Rytarowski" <kamil@netbsd.org>,
"Hervé Poussineau" <hpoussin@reactos.org>,
"Reinoud Zandijk" <reinoud@netbsd.org>,
"David Gibson" <david@gibson.dropbear.id.au>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Alexandre Iooss" <erdnaxe@crans.org>,
"Mahmoud Mandour" <ma.mandourr@gmail.com>,
"Ryo ONODERA" <ryoon@netbsd.org>,
"Philippe Mathieu-Daudé" <philmd@redhat.com>,
"Eduardo Habkost" <ehabkost@redhat.com>
Subject: Re: [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0
Date: Wed, 6 Oct 2021 15:39:59 -0300 [thread overview]
Message-ID: <CAKJDGDagReANLxsA5tnqQ0sjU6qrx=LiMbB4Jq2LUc-UtHpdMA@mail.gmail.com> (raw)
In-Reply-To: <20210924185506.2542588-2-crosa@redhat.com>
On Fri, Sep 24, 2021 at 3:57 PM Cleber Rosa <crosa@redhat.com> wrote:
>
> Avocado 91.0, brings, among other changes, a switch to a new runner
> implementation, known as "nrunner". While my personal testing shows
> that it's suitable for running the QEMU tests, there are some
> considerations that should be addressed before the change.
>
> For instance, the "nrunner" implementation will run tests in parallel
> by default. With more tests running, and resources shared, some tests
> may fail with timeouts on some environments.
>
> So, for now, let's bump the Avocado version to allow for the other
> features to be used. And the "nrunner" implementation to be used by
> those that choose to do so.
>
> More information can be found at the release notes at:
> https://avocado-framework.readthedocs.io/en/latest/releases/91_0.html
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
> docs/devel/testing.rst | 12 ++++++++++++
> tests/Makefile.include | 2 ++
> tests/requirements.txt | 2 +-
> 3 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 4a0abbf23d..2818c58ff8 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -740,6 +740,18 @@ may be invoked by running:
>
> tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
>
> +Running tests in parallel
> +-------------------------
> +
> +The current ``make check-acceptance`` target continues to use the
> +Avocado runner implementation simply known as "runner". But, it's now
> +possible to opt-in and choose the "nrunner" implementation, which,
> +among other things, allows for parallel execution of tests:
> +
> +.. code::
> +
> + make AVOCADO_RUNNER=nrunner check-acceptance
The examples in the documentation are all setting the environment
variable at the end of the `make` command. For consistency, I would
move the definition of the AVOCADO_RUNNER to the end of the command.
> +
> Manual Installation
> -------------------
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 6e16c05f10..ac289a2e41 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -92,6 +92,7 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> # Any number of command separated loggers are accepted. For more
> # information please refer to "avocado --help".
> AVOCADO_SHOW=app
> +AVOCADO_RUNNER=runner
> AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
>
> $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> @@ -128,6 +129,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> $(call quiet-command, \
> $(TESTS_VENV_DIR)/bin/python -m avocado \
> --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> + --test-runner=$(AVOCADO_RUNNER) \
> --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
> $(AVOCADO_TAGS) \
> $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> index a21b59b443..40af24c664 100644
> --- a/tests/requirements.txt
> +++ b/tests/requirements.txt
> @@ -1,5 +1,5 @@
> # Add Python module requirements, one per line, to be installed
> # in the tests/venv Python virtual environment. For more info,
> # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> -avocado-framework==88.1
> +avocado-framework==91.0
> pycdlib==1.11.0
> --
> 2.31.1
>
Apart from a tiny documentation change, the code is welcome as it
allows using the new Avocado runner.
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
next prev parent reply other threads:[~2021-10-06 18:42 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-24 18:54 [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Cleber Rosa
2021-09-24 18:54 ` [PATCH 01/16] Acceptance Tests: bump Avocado requirement to 91.0 Cleber Rosa
2021-10-06 18:39 ` Willian Rampazzo [this message]
2021-09-24 18:54 ` [PATCH 02/16] Acceptance Tests: improve check-acceptance description Cleber Rosa
2021-09-27 14:40 ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 03/16] Acceptance Tests: add mechanism for listing tests Cleber Rosa
2021-10-06 18:47 ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 04/16] Acceptance Tests: keep track and disable tests with known issues Cleber Rosa
2021-09-24 18:54 ` [PATCH 05/16] Acceptance Tests: add standard clean up at test tearDown() Cleber Rosa
2021-09-24 19:38 ` Willian Rampazzo
2021-09-24 18:54 ` [PATCH 06/16] Acceptance Tests: use extract from package from avocado.utils Cleber Rosa
2021-09-24 18:54 ` [PATCH 07/16] Acceptance Tests: workaround expired mipsdistros.mips.com HTTPS cert Cleber Rosa
2021-09-24 18:54 ` [PATCH 08/16] acceptance/tests/vnc.py: use explicit syntax for enabling passwords Cleber Rosa
2021-09-27 14:41 ` Philippe Mathieu-Daudé
2021-09-24 18:54 ` [PATCH 09/16] tests/acceptance/boot_xen.py: merge base classes Cleber Rosa
2021-10-06 18:57 ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 10/16] tests/acceptance/boot_xen.py: unify tags Cleber Rosa
2021-10-06 18:56 ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 11/16] tests/acceptance/boot_xen.py: fetch kernel during test setUp() Cleber Rosa
2021-10-06 18:55 ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 12/16] tests/acceptance/boot_xen.py: removed unused import Cleber Rosa
2021-09-27 14:43 ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 13/16] tests/acceptance/boot_xen.py: use class attribute Cleber Rosa
2021-10-06 18:58 ` Willian Rampazzo
2021-09-24 18:55 ` [PATCH 14/16] tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update Cleber Rosa
2021-09-27 15:26 ` Philippe Mathieu-Daudé
2021-09-30 19:19 ` Reinoud Zandijk
2021-09-30 19:44 ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 15/16] tests/acceptance/ppc_prep_40p.py: clean up unused import Cleber Rosa
2021-09-27 14:42 ` Philippe Mathieu-Daudé
2021-09-24 18:55 ` [PATCH 16/16] tests/acceptance/ppc_prep_40p.py: unify tags Cleber Rosa
2021-09-27 14:45 ` Philippe Mathieu-Daudé
2021-10-06 19:17 ` Willian Rampazzo
2021-10-06 19:19 ` Willian Rampazzo
2021-09-27 9:09 ` [PATCH 00/16] Acceptance Tests: use Avocado 91.0 features and other improvements Pavel Dovgalyuk
2021-09-27 16:46 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKJDGDagReANLxsA5tnqQ0sjU6qrx=LiMbB4Jq2LUc-UtHpdMA@mail.gmail.com' \
--to=wrampazz@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=bleal@redhat.com \
--cc=crosa@redhat.com \
--cc=david@gibson.dropbear.id.au \
--cc=ehabkost@redhat.com \
--cc=erdnaxe@crans.org \
--cc=eric.auger@redhat.com \
--cc=groug@kaod.org \
--cc=hpoussin@reactos.org \
--cc=kamil@netbsd.org \
--cc=ma.mandourr@gmail.com \
--cc=pavel.dovgaluk@ispras.ru \
--cc=pbonzini@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=qemu-ppc@nongnu.org \
--cc=reinoud@netbsd.org \
--cc=ryoon@netbsd.org \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
--cc=ysato@users.sourceforge.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).