[PATCH v2 0/2] Acceptance Tests: update assets location and cancel tests if missing

[PATCH v2 0/2] Acceptance Tests: update assets location and cancel tests if missing

[Philippe Mathieu-Daudé]

Cover from Cleber's v1:

  To minimize the inconvenciente caused by test and job failures in the
  future, an option is enabled that will cancel (AKA skip) tests early
  when those assets are not available.

Pavel Dovgaluk (1):
  tests: bump avocado version

Philippe Mathieu-Daudé (1):
  tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)

 tests/Makefile.include                    |  2 +-
 tests/acceptance/avocado_qemu/__init__.py | 12 ++++++++++++
 tests/requirements.txt                    |  2 +-
 3 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.26.2

[PATCH v2 1/2] tests: bump avocado version

[Philippe Mathieu-Daudé]

From: Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com>

Reverse debugging test uses gdb remote client of avocado framework.
This client was fixed since the currently used version 76.
Therefore this patch bumps the version to 81 and fixes command
line version compatibility issue.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <159903462803.28509.16851113546106095750.stgit@pasha-ThinkPad-X280>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/Makefile.include | 2 +-
 tests/requirements.txt | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index f93e6112202..2baebc179e7 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -127,7 +127,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
             --filter-by-tags-include-empty --filter-by-tags-include-empty-key \
             $(AVOCADO_TAGS) \
-            $(if $(GITLAB_CI),,--failfast=on) tests/acceptance, \
+            $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
             "AVOCADO", "tests/acceptance")
 
 # Consolidated targets
diff --git a/tests/requirements.txt b/tests/requirements.txt
index f9c84b4ba1a..036691c9221 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,5 +1,5 @@
 # Add Python module requirements, one per line, to be installed
 # in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-avocado-framework==76.0
+avocado-framework==81.0
 pycdlib==1.9.0
-- 
2.26.2

[PATCH v2 2/2] tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)

[Philippe Mathieu-Daudé]

Invert the default of avocado.Test.fetch_asset 'cancel_on_missing'
keyword: accept missing artefacts by default. If a test is certain
an artifact can't be missing, it will set cancel_on_missing=False.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/acceptance/avocado_qemu/__init__.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index db9c0f5d79d..4cda0371879 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -192,3 +192,15 @@ def get_vm(self, *args, name=None):
     def tearDown(self):
         for vm in self._vms.values():
             vm.shutdown()
+
+    def fetch_asset(self, name,
+                    asset_hash=None, algorithm=None,
+                    locations=None, expire=None,
+                    find_only=False, cancel_on_missing=True):
+        return super(Test, self).fetch_asset(name,
+                        asset_hash=asset_hash,
+                        algorithm=algorithm,
+                        locations=locations,
+                        expire=expire,
+                        find_only=find_only,
+                        cancel_on_missing=cancel_on_missing)
-- 
2.26.2

Re: [PATCH v2 0/2] Acceptance Tests: update assets location and cancel tests if missing

[Philippe Mathieu-Daudé]

On 9/8/20 10:23 PM, Philippe Mathieu-Daudé wrote:
> Cover from Cleber's v1:
> 
>   To minimize the inconvenciente caused by test and job failures in the
>   future, an option is enabled that will cancel (AKA skip) tests early
>   when those assets are not available.
> 
> Pavel Dovgaluk (1):
>   tests: bump avocado version
> 
> Philippe Mathieu-Daudé (1):
>   tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)
> 
>  tests/Makefile.include                    |  2 +-
>  tests/acceptance/avocado_qemu/__init__.py | 12 ++++++++++++
>  tests/requirements.txt                    |  2 +-
>  3 files changed, 14 insertions(+), 2 deletions(-)
> 

CI:
https://gitlab.com/philmd/qemu/-/pipelines/187519869

Re: [PATCH v2 0/2] Acceptance Tests: update assets location and cancel tests if missing

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> Cover from Cleber's v1:
>
>   To minimize the inconvenciente caused by test and job failures in the
>   future, an option is enabled that will cancel (AKA skip) tests early
>   when those assets are not available.
>
> Pavel Dovgaluk (1):
>   tests: bump avocado version
>
> Philippe Mathieu-Daudé (1):
>   tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)
>
>  tests/Makefile.include                    |  2 +-
>  tests/acceptance/avocado_qemu/__init__.py | 12 ++++++++++++
>  tests/requirements.txt                    |  2 +-
>  3 files changed, 14 insertions(+), 2 deletions(-)

Queued to testing/next, thanks.

-- 
Alex Bennée