* [PATCH] tests/acceptance: allow control over tags during check-acceptance
@ 2021-07-06 19:06 Willian Rampazzo
2021-07-06 20:46 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 3+ messages in thread
From: Willian Rampazzo @ 2021-07-06 19:06 UTC (permalink / raw)
To: alex.bennee, f4bug, crosa, thuth, wainersm, qemu-devel, philmd
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.
This allows using the TAGS environment variable where the user takes
total control of which tests should run based on the tags defined.
This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.
e.g.:
TAGS="foo bar baz" make check-acceptance
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
---
docs/devel/testing.rst | 7 +++++++
tests/Makefile.include | 10 ++++++++--
2 files changed, 15 insertions(+), 2 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 4e42392810..6e03c3449b 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -760,6 +760,13 @@ in the current directory, tagged as "quick", run:
avocado run -t quick .
+It is also possible to run tests based on tags using the
+``make check-acceptance`` command and the ``TAGS`` environment variable:
+
+.. code::
+
+ TAGS=quick make check-acceptance
+
The ``avocado_qemu.Test`` base test class
-----------------------------------------
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8f220e15d1..5869ab8a04 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -92,7 +92,11 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
# Any number of command separated loggers are accepted. For more
# information please refer to "avocado --help".
AVOCADO_SHOW=app
-AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+ifndef TAGS
+ AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
+else
+ AVOCADO_TAGS=$(addprefix -t , $(TAGS))
+endif
$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
$(call quiet-command, \
@@ -125,10 +129,12 @@ get-vm-image-fedora-31-%: check-venv
get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
+ @echo "AVOCADO_TAGS $(AVOCADO_TAGS)"
$(call quiet-command, \
$(TESTS_VENV_DIR)/bin/python -m avocado \
--show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
- --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
+ $(if $(TAGS),, --filter-by-tags-include-empty \
+ --filter-by-tags-include-empty-key) \
$(AVOCADO_TAGS) \
$(if $(GITLAB_CI),,--failfast) tests/acceptance, \
"AVOCADO", "tests/acceptance")
--
2.31.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] tests/acceptance: allow control over tags during check-acceptance
2021-07-06 19:06 [PATCH] tests/acceptance: allow control over tags during check-acceptance Willian Rampazzo
@ 2021-07-06 20:46 ` Philippe Mathieu-Daudé
2021-07-06 21:10 ` Willian Rampazzo
0 siblings, 1 reply; 3+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-07-06 20:46 UTC (permalink / raw)
To: Willian Rampazzo, alex.bennee, crosa, thuth, wainersm, qemu-devel,
philmd
On 7/6/21 9:06 PM, Willian Rampazzo wrote:
> Although it is possible to run a specific test using the avocado
> command-line, a user may want to use a specific tag while running the
> ``make check-acceptance`` during the development or debugging.
>
> This allows using the TAGS environment variable where the user takes
> total control of which tests should run based on the tags defined.
>
> This also makes the check-acceptance command flexible to restrict tests
> based on tags while running on CI.
>
> e.g.:
>
> TAGS="foo bar baz" make check-acceptance
I'm worried 'TAGS' is a bit too generic...
Maybe rename AVOCADO_TAGS -> AVOCADO_CMDLINE_TAGS and
TAGS -> AVOCADO_TAGS?
> Signed-off-by: Willian Rampazzo <willianr@redhat.com>
> ---
> docs/devel/testing.rst | 7 +++++++
> tests/Makefile.include | 10 ++++++++--
> 2 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 4e42392810..6e03c3449b 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -760,6 +760,13 @@ in the current directory, tagged as "quick", run:
>
> avocado run -t quick .
>
> +It is also possible to run tests based on tags using the
> +``make check-acceptance`` command and the ``TAGS`` environment variable:
> +
> +.. code::
> +
> + TAGS=quick make check-acceptance
> +
> The ``avocado_qemu.Test`` base test class
> -----------------------------------------
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 8f220e15d1..5869ab8a04 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -92,7 +92,11 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> # Any number of command separated loggers are accepted. For more
> # information please refer to "avocado --help".
> AVOCADO_SHOW=app
> -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> +ifndef TAGS
> + AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> +else
> + AVOCADO_TAGS=$(addprefix -t , $(TAGS))
> +endif
>
> $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> $(call quiet-command, \
> @@ -125,10 +129,12 @@ get-vm-image-fedora-31-%: check-venv
> get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
>
> check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> + @echo "AVOCADO_TAGS $(AVOCADO_TAGS)"
> $(call quiet-command, \
> $(TESTS_VENV_DIR)/bin/python -m avocado \
> --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> - --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
> + $(if $(TAGS),, --filter-by-tags-include-empty \
> + --filter-by-tags-include-empty-key) \
> $(AVOCADO_TAGS) \
> $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> "AVOCADO", "tests/acceptance")
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] tests/acceptance: allow control over tags during check-acceptance
2021-07-06 20:46 ` Philippe Mathieu-Daudé
@ 2021-07-06 21:10 ` Willian Rampazzo
0 siblings, 0 replies; 3+ messages in thread
From: Willian Rampazzo @ 2021-07-06 21:10 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Alex Bennée, qemu-devel, Wainer Moschetta,
Cleber Rosa Junior
On Tue, Jul 6, 2021 at 5:46 PM Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> On 7/6/21 9:06 PM, Willian Rampazzo wrote:
> > Although it is possible to run a specific test using the avocado
> > command-line, a user may want to use a specific tag while running the
> > ``make check-acceptance`` during the development or debugging.
> >
> > This allows using the TAGS environment variable where the user takes
> > total control of which tests should run based on the tags defined.
> >
> > This also makes the check-acceptance command flexible to restrict tests
> > based on tags while running on CI.
> >
> > e.g.:
> >
> > TAGS="foo bar baz" make check-acceptance
>
> I'm worried 'TAGS' is a bit too generic...
> Maybe rename AVOCADO_TAGS -> AVOCADO_CMDLINE_TAGS and
> TAGS -> AVOCADO_TAGS?
I'm fine with those changes.
>
> > Signed-off-by: Willian Rampazzo <willianr@redhat.com>
> > ---
> > docs/devel/testing.rst | 7 +++++++
> > tests/Makefile.include | 10 ++++++++--
> > 2 files changed, 15 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> > index 4e42392810..6e03c3449b 100644
> > --- a/docs/devel/testing.rst
> > +++ b/docs/devel/testing.rst
> > @@ -760,6 +760,13 @@ in the current directory, tagged as "quick", run:
> >
> > avocado run -t quick .
> >
> > +It is also possible to run tests based on tags using the
> > +``make check-acceptance`` command and the ``TAGS`` environment variable:
> > +
> > +.. code::
> > +
> > + TAGS=quick make check-acceptance
> > +
> > The ``avocado_qemu.Test`` base test class
> > -----------------------------------------
> >
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 8f220e15d1..5869ab8a04 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -92,7 +92,11 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> > # Any number of command separated loggers are accepted. For more
> > # information please refer to "avocado --help".
> > AVOCADO_SHOW=app
> > -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> > +ifndef TAGS
> > + AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> > +else
> > + AVOCADO_TAGS=$(addprefix -t , $(TAGS))
> > +endif
> >
> > $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> > $(call quiet-command, \
> > @@ -125,10 +129,12 @@ get-vm-image-fedora-31-%: check-venv
> > get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
> >
> > check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> > + @echo "AVOCADO_TAGS $(AVOCADO_TAGS)"
> > $(call quiet-command, \
> > $(TESTS_VENV_DIR)/bin/python -m avocado \
> > --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> > - --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
> > + $(if $(TAGS),, --filter-by-tags-include-empty \
> > + --filter-by-tags-include-empty-key) \
> > $(AVOCADO_TAGS) \
> > $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> > "AVOCADO", "tests/acceptance")
> >
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-07-06 21:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-07-06 19:06 [PATCH] tests/acceptance: allow control over tags during check-acceptance Willian Rampazzo
2021-07-06 20:46 ` Philippe Mathieu-Daudé
2021-07-06 21:10 ` Willian Rampazzo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).