* [PATCH 0/2] tests: Add 'fetch-acceptance' rule @ 2020-07-24 7:35 Philippe Mathieu-Daudé 2020-07-24 7:35 ` [PATCH 1/2] " Philippe Mathieu-Daudé 2020-07-24 7:35 ` [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule Philippe Mathieu-Daudé 0 siblings, 2 replies; 6+ messages in thread From: Philippe Mathieu-Daudé @ 2020-07-24 7:35 UTC (permalink / raw) To: qemu-devel Cc: Thomas Huth, Philippe Mathieu-Daudé, Wainer dos Santos Moschetta, Willian Rampazzo, Cleber Rosa, Alex Bennée Add a rule to fetch acceptance test assets. This is particularly useful in a CI context, when a single job can fetch and save the cache so other jobs reuse it directly. It is also useful to measure the time spent downloading the assets versus the time spent running the tests. For now we have to exclude the 'boot_linux.py' which triggers an exception. Philippe Mathieu-Daudé (2): tests: Add 'fetch-acceptance' rule tests: Exclude 'boot_linux.py' from fetch-acceptance rule tests/Makefile.include | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.21.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] tests: Add 'fetch-acceptance' rule 2020-07-24 7:35 [PATCH 0/2] tests: Add 'fetch-acceptance' rule Philippe Mathieu-Daudé @ 2020-07-24 7:35 ` Philippe Mathieu-Daudé 2020-07-24 19:00 ` Wainer dos Santos Moschetta 2020-07-24 7:35 ` [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule Philippe Mathieu-Daudé 1 sibling, 1 reply; 6+ messages in thread From: Philippe Mathieu-Daudé @ 2020-07-24 7:35 UTC (permalink / raw) To: qemu-devel Cc: Thomas Huth, Philippe Mathieu-Daudé, Wainer dos Santos Moschetta, Willian Rampazzo, Cleber Rosa, Alex Bennée Add a rule to fetch acceptance test assets. This is particularly useful in a CI context, when a single job can fetch and save the cache so other jobs reuse it directly. It is also useful to measure the time spent downloading the assets versus the time spent running the tests. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> --- tests/Makefile.include | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/Makefile.include b/tests/Makefile.include index c7e4646ded..238974d8da 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -944,6 +944,15 @@ get-vm-image-fedora-31-%: check-venv # download all vm images, according to defined targets get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD)) +# fetch acceptance test assets +fetch-acceptance: check-venv + $(call quiet-command, \ + $(TESTS_VENV_DIR)/bin/python -m avocado \ + $(if $(V),--show=$(AVOCADO_SHOW)) \ + assets fetch \ + $(wildcard tests/acceptance/*.py), \ + "AVOCADO", "tests/acceptance") + check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images $(call quiet-command, \ $(TESTS_VENV_DIR)/bin/python -m avocado \ -- 2.21.3 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] tests: Add 'fetch-acceptance' rule 2020-07-24 7:35 ` [PATCH 1/2] " Philippe Mathieu-Daudé @ 2020-07-24 19:00 ` Wainer dos Santos Moschetta 0 siblings, 0 replies; 6+ messages in thread From: Wainer dos Santos Moschetta @ 2020-07-24 19:00 UTC (permalink / raw) To: Philippe Mathieu-Daudé, qemu-devel Cc: Willian Rampazzo, Thomas Huth, Alex Bennée, Cleber Rosa Hello, On 7/24/20 4:35 AM, Philippe Mathieu-Daudé wrote: > Add a rule to fetch acceptance test assets. > > This is particularly useful in a CI context, when a single job > can fetch and save the cache so other jobs reuse it directly. > > It is also useful to measure the time spent downloading the > assets versus the time spent running the tests. > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > tests/Makefile.include | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index c7e4646ded..238974d8da 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -944,6 +944,15 @@ get-vm-image-fedora-31-%: check-venv > # download all vm images, according to defined targets > get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD)) > > +# fetch acceptance test assets > +fetch-acceptance: check-venv This new target misses an entry on check-help. > + $(call quiet-command, \ > + $(TESTS_VENV_DIR)/bin/python -m avocado \ > + $(if $(V),--show=$(AVOCADO_SHOW)) \ > + assets fetch \ Perhaps pass '--ignore-errors' so that intermittent network failurse won't disturb the execution (the test will have a second chance to download the asset later when it executes). > + $(wildcard tests/acceptance/*.py), \ > + "AVOCADO", "tests/acceptance") nit: print "Downloading acceptance tests assets" (similar to get-vm-image-fedora-32-% target). Talking about get-vm-images...that target is pre-req of check-acceptance, which makes me think that fetch-acceptance should be either (for the sake of consistency.) The downside is that - as a developer running it on my machine - `avocado assets fetch` will attempt to download artifacts even for those tests which I'm not going to run anyway. Any opinion? Regards, Wainer > + > check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images > $(call quiet-command, \ > $(TESTS_VENV_DIR)/bin/python -m avocado \ ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule 2020-07-24 7:35 [PATCH 0/2] tests: Add 'fetch-acceptance' rule Philippe Mathieu-Daudé 2020-07-24 7:35 ` [PATCH 1/2] " Philippe Mathieu-Daudé @ 2020-07-24 7:35 ` Philippe Mathieu-Daudé 2020-07-24 18:26 ` Wainer dos Santos Moschetta 1 sibling, 1 reply; 6+ messages in thread From: Philippe Mathieu-Daudé @ 2020-07-24 7:35 UTC (permalink / raw) To: qemu-devel Cc: Thomas Huth, Philippe Mathieu-Daudé, Wainer dos Santos Moschetta, Willian Rampazzo, Cleber Rosa, Alex Bennée The boot_linux.py file triggers an exception: $ make fetch-acceptance AVOCADO tests/acceptance Fetching assets from tests/acceptance/empty_cpu_model.py. Fetching assets from tests/acceptance/vnc.py. Fetching assets from tests/acceptance/boot_linux_console.py. Fetching assets from tests/acceptance/boot_linux.py. Traceback (most recent call last): File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/__main__.py", line 11, in <module> sys.exit(main.run()) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/core/app.py", line 91, in run return method(self.parser.config) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 291, in run success, fail = fetch_assets(test_file) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 200, in fetch_assets handler = FetchAssetHandler(test_file, klass, method) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 65, in __init__ self.visit(self.tree) File "/usr/lib64/python3.7/ast.py", line 271, in visit return visitor(node) File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit self.visit(item) File "/usr/lib64/python3.7/ast.py", line 271, in visit return visitor(node) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 139, in visit_ClassDef self.generic_visit(node) File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit self.visit(item) File "/usr/lib64/python3.7/ast.py", line 271, in visit return visitor(node) File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 171, in visit_Assign self.asgmts[cur_klass][cur_method][name] = node.value.s KeyError: 'launch_and_wait' make: *** [tests/Makefile.include:949: fetch-acceptance] Error 1 Exclude it for now. We will revert this commit once the script is fixed. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> --- tests/Makefile.include | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 238974d8da..7c9cf7a818 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -950,7 +950,7 @@ fetch-acceptance: check-venv $(TESTS_VENV_DIR)/bin/python -m avocado \ $(if $(V),--show=$(AVOCADO_SHOW)) \ assets fetch \ - $(wildcard tests/acceptance/*.py), \ + $(filter-out tests/acceptance/boot_linux.py,$(wildcard tests/acceptance/*.py)), \ "AVOCADO", "tests/acceptance") check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images -- 2.21.3 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule 2020-07-24 7:35 ` [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule Philippe Mathieu-Daudé @ 2020-07-24 18:26 ` Wainer dos Santos Moschetta 2020-07-24 18:47 ` Willian Rampazzo 0 siblings, 1 reply; 6+ messages in thread From: Wainer dos Santos Moschetta @ 2020-07-24 18:26 UTC (permalink / raw) To: Philippe Mathieu-Daudé, qemu-devel Cc: Willian Rampazzo, Thomas Huth, Alex Bennée, Cleber Rosa Hi Philippe, On 7/24/20 4:35 AM, Philippe Mathieu-Daudé wrote: > The boot_linux.py file triggers an exception: > > $ make fetch-acceptance > AVOCADO tests/acceptance > Fetching assets from tests/acceptance/empty_cpu_model.py. > Fetching assets from tests/acceptance/vnc.py. > Fetching assets from tests/acceptance/boot_linux_console.py. > Fetching assets from tests/acceptance/boot_linux.py. > Traceback (most recent call last): > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/__main__.py", line 11, in <module> > sys.exit(main.run()) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/core/app.py", line 91, in run > return method(self.parser.config) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 291, in run > success, fail = fetch_assets(test_file) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 200, in fetch_assets > handler = FetchAssetHandler(test_file, klass, method) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 65, in __init__ > self.visit(self.tree) > File "/usr/lib64/python3.7/ast.py", line 271, in visit > return visitor(node) > File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit > self.visit(item) > File "/usr/lib64/python3.7/ast.py", line 271, in visit > return visitor(node) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 139, in visit_ClassDef > self.generic_visit(node) > File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit > self.visit(item) > File "/usr/lib64/python3.7/ast.py", line 271, in visit > return visitor(node) > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 171, in visit_Assign > self.asgmts[cur_klass][cur_method][name] = node.value.s > KeyError: 'launch_and_wait' > make: *** [tests/Makefile.include:949: fetch-acceptance] Error 1 Currently the acceptance tests use Avocado 7.6. I bumped to 80.0 (latest released) here and that error is gone. Could you double check? Regards, Wainer > > Exclude it for now. We will revert this commit once the script is > fixed. > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > tests/Makefile.include | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 238974d8da..7c9cf7a818 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -950,7 +950,7 @@ fetch-acceptance: check-venv > $(TESTS_VENV_DIR)/bin/python -m avocado \ > $(if $(V),--show=$(AVOCADO_SHOW)) \ > assets fetch \ > - $(wildcard tests/acceptance/*.py), \ > + $(filter-out tests/acceptance/boot_linux.py,$(wildcard tests/acceptance/*.py)), \ > "AVOCADO", "tests/acceptance") > > check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule 2020-07-24 18:26 ` Wainer dos Santos Moschetta @ 2020-07-24 18:47 ` Willian Rampazzo 0 siblings, 0 replies; 6+ messages in thread From: Willian Rampazzo @ 2020-07-24 18:47 UTC (permalink / raw) To: Wainer dos Santos Moschetta Cc: Thomas Huth, Philippe Mathieu-Daudé, qemu-devel, Willian Rampazzo, Cleber Rosa, Alex Bennée On Fri, Jul 24, 2020 at 3:26 PM Wainer dos Santos Moschetta <wainersm@redhat.com> wrote: > > Hi Philippe, > > On 7/24/20 4:35 AM, Philippe Mathieu-Daudé wrote: > > The boot_linux.py file triggers an exception: > > > > $ make fetch-acceptance > > AVOCADO tests/acceptance > > Fetching assets from tests/acceptance/empty_cpu_model.py. > > Fetching assets from tests/acceptance/vnc.py. > > Fetching assets from tests/acceptance/boot_linux_console.py. > > Fetching assets from tests/acceptance/boot_linux.py. > > Traceback (most recent call last): > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/__main__.py", line 11, in <module> > > sys.exit(main.run()) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/core/app.py", line 91, in run > > return method(self.parser.config) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 291, in run > > success, fail = fetch_assets(test_file) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 200, in fetch_assets > > handler = FetchAssetHandler(test_file, klass, method) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 65, in __init__ > > self.visit(self.tree) > > File "/usr/lib64/python3.7/ast.py", line 271, in visit > > return visitor(node) > > File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit > > self.visit(item) > > File "/usr/lib64/python3.7/ast.py", line 271, in visit > > return visitor(node) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 139, in visit_ClassDef > > self.generic_visit(node) > > File "/usr/lib64/python3.7/ast.py", line 279, in generic_visit > > self.visit(item) > > File "/usr/lib64/python3.7/ast.py", line 271, in visit > > return visitor(node) > > File "/var/tmp/qemu-builddir/tests/venv/lib64/python3.7/site-packages/avocado/plugins/assets.py", line 171, in visit_Assign > > self.asgmts[cur_klass][cur_method][name] = node.value.s > > KeyError: 'launch_and_wait' > > make: *** [tests/Makefile.include:949: fetch-acceptance] Error 1 > > Currently the acceptance tests use Avocado 7.6. I bumped to 80.0 (latest > released) here and that error is gone. Could you double check? > > Regards, > > Wainer Hi Wainer, thanks for looking at this problem. This bug was fixed here https://github.com/avocado-framework/avocado/pull/3665, on release 78 of Avocado. It was reported by Philippe at that time. I think we forgot to bump the Avocado version here. > > > > > Exclude it for now. We will revert this commit once the script is > > fixed. > > > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > > --- > > tests/Makefile.include | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/tests/Makefile.include b/tests/Makefile.include > > index 238974d8da..7c9cf7a818 100644 > > --- a/tests/Makefile.include > > +++ b/tests/Makefile.include > > @@ -950,7 +950,7 @@ fetch-acceptance: check-venv > > $(TESTS_VENV_DIR)/bin/python -m avocado \ > > $(if $(V),--show=$(AVOCADO_SHOW)) \ > > assets fetch \ > > - $(wildcard tests/acceptance/*.py), \ > > + $(filter-out tests/acceptance/boot_linux.py,$(wildcard tests/acceptance/*.py)), \ > > "AVOCADO", "tests/acceptance") > > > > check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-07-24 19:02 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-07-24 7:35 [PATCH 0/2] tests: Add 'fetch-acceptance' rule Philippe Mathieu-Daudé 2020-07-24 7:35 ` [PATCH 1/2] " Philippe Mathieu-Daudé 2020-07-24 19:00 ` Wainer dos Santos Moschetta 2020-07-24 7:35 ` [PATCH 2/2] tests: Exclude 'boot_linux.py' from fetch-acceptance rule Philippe Mathieu-Daudé 2020-07-24 18:26 ` Wainer dos Santos Moschetta 2020-07-24 18:47 ` Willian Rampazzo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).