[PATCH 0/4] Test more "check-acceptance" tests in the gitlab-CI

[PATCH 0/4] Test more "check-acceptance" tests in the gitlab-CI

[Thomas Huth]

These patches enable more "check-acceptance" tests in the gitlab-CI,
fixing a broken URL, setting AVOCADO_ALLOW_UNTRUSTED_CODE and adding
some missing packages to the containers.

Thomas Huth (4):
  tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
  test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
  tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
  test/docker/dockerfiles: Add missing packages for acceptance tests

 .gitlab-ci.yml                               | 1 +
 tests/acceptance/ppc_prep_40p.py             | 4 +---
 tests/docker/dockerfiles/centos8.docker      | 1 +
 tests/docker/dockerfiles/debian-amd64.docker | 3 +++
 tests/docker/dockerfiles/fedora.docker       | 1 +
 tests/docker/dockerfiles/ubuntu2004.docker   | 1 +
 6 files changed, 8 insertions(+), 3 deletions(-)

-- 
2.18.2

[PATCH 1/4] tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive

[Thomas Huth]

The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
instead seems to be fine.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/acceptance/ppc_prep_40p.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/acceptance/ppc_prep_40p.py
index 1515561249..e82755c9cf 100644
--- a/tests/acceptance/ppc_prep_40p.py
+++ b/tests/acceptance/ppc_prep_40p.py
@@ -35,7 +35,7 @@ class IbmPrep40pMachine(Test):
                     '7020-40p/P12H0456.IMG')
         bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03'
         bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
-        drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/'
+        drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
                      'NetBSD-4.0/prep/installation/floppy/generic_com0.fs')
         drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb'
         drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
-- 
2.18.2

Re: [PATCH 1/4] tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive

[Philippe Mathieu-Daudé]

On 10/23/20 9:33 AM, Thomas Huth wrote:
> The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
> instead seems to be fine.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/acceptance/ppc_prep_40p.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/acceptance/ppc_prep_40p.py
> index 1515561249..e82755c9cf 100644
> --- a/tests/acceptance/ppc_prep_40p.py
> +++ b/tests/acceptance/ppc_prep_40p.py
> @@ -35,7 +35,7 @@ class IbmPrep40pMachine(Test):
>                       '7020-40p/P12H0456.IMG')
>           bios_hash = '1775face4e6dc27f3a6ed955ef6eb331bf817f03'
>           bios_path = self.fetch_asset(bios_url, asset_hash=bios_hash)
> -        drive_url = ('https://cdn.netbsd.org/pub/NetBSD/NetBSD-archive/'
> +        drive_url = ('https://archive.netbsd.org/pub/NetBSD-archive/'
>                        'NetBSD-4.0/prep/installation/floppy/generic_com0.fs')
>           drive_hash = 'dbcfc09912e71bd5f0d82c7c1ee43082fb596ceb'
>           drive_path = self.fetch_asset(drive_url, asset_hash=drive_hash)
> 

Thanks!

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

[PATCH 2/4] test/acceptance: Remove the CONTINUOUS_INTEGRATION tags

[Thomas Huth]

We are not running the acceptance tests on Travis anymore, so these
checks can be removed now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/acceptance/ppc_prep_40p.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tests/acceptance/ppc_prep_40p.py b/tests/acceptance/ppc_prep_40p.py
index e82755c9cf..96ba13b894 100644
--- a/tests/acceptance/ppc_prep_40p.py
+++ b/tests/acceptance/ppc_prep_40p.py
@@ -22,7 +22,6 @@ class IbmPrep40pMachine(Test):
     # All rights reserved.
     # U.S. Government Users Restricted Rights - Use, duplication or disclosure
     # restricted by GSA ADP Schedule Contract with IBM Corp.
-    @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
     def test_factory_firmware_and_netbsd(self):
         """
@@ -61,7 +60,6 @@ class IbmPrep40pMachine(Test):
         wait_for_console_pattern(self, '>> Memory: 192M')
         wait_for_console_pattern(self, '>> CPU type PowerPC,604')
 
-    @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_openbios_and_netbsd(self):
         """
         :avocado: tags=arch:ppc
-- 
2.18.2

Re: [PATCH 2/4] test/acceptance: Remove the CONTINUOUS_INTEGRATION tags

[Philippe Mathieu-Daudé]

On 10/23/20 9:33 AM, Thomas Huth wrote:
> We are not running the acceptance tests on Travis anymore, so these
> checks can be removed now.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/acceptance/ppc_prep_40p.py | 2 --
>   1 file changed, 2 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

[PATCH 3/4] tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI

[Thomas Huth]

The tests are running in containers here, so it should be OK to
run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 66ad7aa5c2..5d6773efd2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -66,6 +66,7 @@ include:
     - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
       fi
+    - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
   after_script:
     - cd build
     - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP", "CANCEL")]' | xargs cat
-- 
2.18.2

Re: [PATCH 3/4] tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI

[Philippe Mathieu-Daudé]

On 10/23/20 9:33 AM, Thomas Huth wrote:
> The tests are running in containers here, so it should be OK to
> run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   .gitlab-ci.yml | 1 +
>   1 file changed, 1 insertion(+)

Thanks :)

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

[PATCH 4/4] test/docker/dockerfiles: Add missing packages for acceptance tests

[Thomas Huth]

Some of the "check-acceptance" tests are still skipped in the CI
since the docker images do not provide the necessary packages, e.g.
the netcat binary. Add them to get more test coverage.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/docker/dockerfiles/centos8.docker      | 1 +
 tests/docker/dockerfiles/debian-amd64.docker | 3 +++
 tests/docker/dockerfiles/fedora.docker       | 1 +
 tests/docker/dockerfiles/ubuntu2004.docker   | 1 +
 4 files changed, 6 insertions(+)

diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index 585dfad9be..a589142114 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -18,6 +18,7 @@ ENV PACKAGES \
     lzo-devel \
     make \
     mesa-libEGL-devel \
+    nmap-ncat \
     nettle-devel \
     ninja-build \
     perl-Test-Harness \
diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index 314c6bae83..55075d9fce 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -23,6 +23,9 @@ RUN apt update && \
         libsnappy-dev \
         libvte-dev \
         netcat-openbsd \
+        openssh-client \
+        python3-numpy \
+        python3-opencv \
         python3-venv
 
 # virgl
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index ac79d95418..0b5053f2d0 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -73,6 +73,7 @@ ENV PACKAGES \
     mingw64-pixman \
     mingw64-pkg-config \
     mingw64-SDL2 \
+    nmap-ncat \
     ncurses-devel \
     nettle-devel \
     ninja-build \
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 17b37cda38..355bbb3c63 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -47,6 +47,7 @@ ENV PACKAGES flex bison \
     libxen-dev \
     libzstd-dev \
     make \
+    netcat-openbsd \
     ninja-build \
     python3-numpy \
     python3-opencv \
-- 
2.18.2

Re: [PATCH 4/4] test/docker/dockerfiles: Add missing packages for acceptance tests

[Philippe Mathieu-Daudé]

On 10/23/20 9:33 AM, Thomas Huth wrote:
> Some of the "check-acceptance" tests are still skipped in the CI
> since the docker images do not provide the necessary packages, e.g.
> the netcat binary. Add them to get more test coverage.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/docker/dockerfiles/centos8.docker      | 1 +
>   tests/docker/dockerfiles/debian-amd64.docker | 3 +++
>   tests/docker/dockerfiles/fedora.docker       | 1 +
>   tests/docker/dockerfiles/ubuntu2004.docker   | 1 +
>   4 files changed, 6 insertions(+)

Finally...

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 0/4] Test more "check-acceptance" tests in the gitlab-CI

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> These patches enable more "check-acceptance" tests in the gitlab-CI,
> fixing a broken URL, setting AVOCADO_ALLOW_UNTRUSTED_CODE and adding
> some missing packages to the containers.

Queued to pr/261020-testing-and-misc-1, thanks.

-- 
Alex Bennée

Re: [PATCH 0/4] Test more "check-acceptance" tests in the gitlab-CI

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Thomas Huth <thuth@redhat.com> writes:
>
>> These patches enable more "check-acceptance" tests in the gitlab-CI,
>> fixing a broken URL, setting AVOCADO_ALLOW_UNTRUSTED_CODE and adding
>> some missing packages to the containers.
>
> Queued to pr/261020-testing-and-misc-1, thanks.

Ahh I shall drop as your PR should be in first anyway.

-- 
Alex Bennée