[PULL v2 00/24] Tests and misc patches

[PULL v2 00/24] Tests and misc patches

[Thomas Huth]

 Hi Peter!

The following changes since commit a66f28df650166ae8b50c992eea45e7b247f4143:

  Merge tag 'migration-20240909-pull-request' of https://gitlab.com/peterx/qemu into staging (2024-09-10 11:19:22 +0100)

are available in the Git repository at:

  https://gitlab.com/thuth/qemu.git tags/pull-request-2024-09-11

for you to fetch changes up to b4358ed4fd29c21c69e492d814f0926c58caa10f:

  Update FreeBSD CI jobs FreeBSD 14.1 (2024-09-11 11:26:41 +0200)

----------------------------------------------------------------
* Split --enable-sanitizers to --enable-{asan, ubsan}
* Build MSYS2 job using multiple CPUs
* Fix "make distclean" wrt contrib/plugins/
* Convert more Avocado tests to plain standalone functional tests
* Fix bug that breaks "make check-functional" when tesseract is missing
* Use builtin hashlib of Python in the functional tests
* Update the FreeBSD CI jobs to 14.1

v2:
 - Use python hashlib instead of external programs
 - Update FreeBSD CI jobs FreeBSD 14.1

----------------------------------------------------------------
Beraldo Leal (1):
      MAINTAINERS: Remove myself as reviewer

Philippe Mathieu-Daudé (10):
      gitlab-ci: Build MSYS2 job using multiple CPUs
      tests/functional: Convert mips64el Fuloong2e avocado test (2/2)
      tests/functional: Convert mips64el I6400 Malta avocado tests
      tests/functional: Convert mips64el 5KEc Malta avocado tests
      tests/functional: Convert mips32el Malta YAMON avocado test
      tests/functional: Convert nanomips Malta avocado tests
      tests/functional: Convert mips32eb 4Kc Malta avocado tests
      tests/functional: Convert ARM Raspi2 avocado tests
      tests/functional: Convert Aarch64 Raspi3 avocado tests
      tests/functional: Convert Aarch64 Raspi4 avocado tests

Richard Henderson (2):
      meson: Split --enable-sanitizers to --enable-{asan, ubsan}
      meson: Move -fsanitize=undefined into normal configuraton

Thomas Huth (11):
      contrib/plugins/Makefile: Add a 'distclean' target
      MAINTAINERS: Remove myself from the Meson section
      tests/functional: Add the LinuxKernelTest for testing the Linux boot process
      tests/functional: Convert the m68k Q800 Avocado test into a functional test
      tests/functional: Convert the Alpha Clipper Avocado test
      tests/functional: Convert the m68k MCF5208EVB Avocado test
      tests/functional: Convert the or1k-sim Avocado test
      tests/functional: Convert the multiprocess avocado test into a standalone test
      tests/functional: Fix bad usage of has_cmd
      tests/functional/qemu_test: Use Python hashlib instead of external programs
      Update FreeBSD CI jobs FreeBSD 14.1

 MAINTAINERS                                        |  12 +-
 docs/devel/testing/fuzzing.rst                     |   4 +-
 meson.build                                        |  26 +-
 tests/qtest/fdc-test.c                             |   2 +-
 .gitlab-ci.d/buildtest.yml                         |  12 +-
 .gitlab-ci.d/cirrus.yml                            |   6 +-
 .../cirrus/{freebsd-13.vars => freebsd-14.vars}    |   2 +-
 .../custom-runners/ubuntu-22.04-aarch64.yml        |   2 +-
 .gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml |   2 +-
 .gitlab-ci.d/windows.yml                           |   3 +-
 contrib/plugins/Makefile                           |   2 +-
 meson_options.txt                                  |   6 +-
 scripts/meson-buildoptions.sh                      |   9 +-
 tests/avocado/avocado_qemu/__init__.py             |  10 -
 tests/avocado/boot_linux_console.py                | 483 ---------------------
 tests/avocado/machine_mips_malta.py                | 162 -------
 tests/avocado/multiprocess.py                      | 102 -----
 tests/docker/test-debug                            |   4 +-
 tests/functional/meson.build                       |  26 +-
 tests/functional/qemu_test/__init__.py             |   1 +
 tests/functional/qemu_test/asset.py                |  16 +-
 tests/functional/qemu_test/cmd.py                  |   6 +-
 tests/functional/qemu_test/linuxkernel.py          |  41 ++
 tests/functional/qemu_test/tesseract.py            |   3 +-
 tests/functional/test_aarch64_raspi3.py            |  41 ++
 tests/functional/test_aarch64_raspi4.py            |  99 +++++
 tests/functional/test_alpha_clipper.py             |  38 ++
 tests/functional/test_arm_raspi2.py                |  95 ++++
 tests/functional/test_m68k_mcf5208evb.py           |  29 ++
 tests/functional/test_m68k_q800.py                 |  37 ++
 tests/functional/test_mips64el_fuloong2e.py        |  25 +-
 tests/functional/test_mips64el_malta.py            | 186 ++++++++
 tests/functional/test_mips_malta.py                |  81 ++++
 tests/functional/test_mipsel_malta.py              |  96 ++++
 tests/functional/test_multiprocess.py              | 100 +++++
 tests/functional/test_or1k_sim.py                  |  29 ++
 tests/lcitool/refresh                              |   4 +-
 tests/vm/freebsd                                   |   6 +-
 38 files changed, 998 insertions(+), 810 deletions(-)
 rename .gitlab-ci.d/cirrus/{freebsd-13.vars => freebsd-14.vars} (95%)
 delete mode 100644 tests/avocado/machine_mips_malta.py
 delete mode 100644 tests/avocado/multiprocess.py
 create mode 100644 tests/functional/qemu_test/linuxkernel.py
 create mode 100755 tests/functional/test_aarch64_raspi3.py
 create mode 100755 tests/functional/test_aarch64_raspi4.py
 create mode 100755 tests/functional/test_alpha_clipper.py
 create mode 100755 tests/functional/test_arm_raspi2.py
 create mode 100755 tests/functional/test_m68k_mcf5208evb.py
 create mode 100755 tests/functional/test_m68k_q800.py
 create mode 100755 tests/functional/test_mips64el_malta.py
 create mode 100755 tests/functional/test_mips_malta.py
 create mode 100755 tests/functional/test_mipsel_malta.py
 create mode 100755 tests/functional/test_multiprocess.py
 create mode 100755 tests/functional/test_or1k_sim.py

[PULL v2 23/24] tests/functional/qemu_test: Use Python hashlib instead of external programs

[Thomas Huth]

Some systems (like OpenBSD) do not have the sha256sum or sha512sum programs
installed by default, or use different names for those. Use the Python
hashlib instead so we don't have to rely on the external programs.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20240910201742.239559-1-thuth@redhat.com>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/asset.py | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index d3be2aff82..3ec429217e 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -43,15 +43,21 @@ def _check(self, cache_file):
         if self.hash is None:
             return True
         if len(self.hash) == 64:
-            sum_prog = 'sha256sum'
+            hl = hashlib.sha256()
         elif len(self.hash) == 128:
-            sum_prog = 'sha512sum'
+            hl = hashlib.sha512()
         else:
             raise Exception("unknown hash type")
 
-        checksum = subprocess.check_output(
-            [sum_prog, str(cache_file)]).split()[0]
-        return self.hash == checksum.decode("utf-8")
+        # Calculate the hash of the file:
+        with open(cache_file, 'rb') as file:
+            while True:
+                chunk = file.read(1 << 20)
+                if not chunk:
+                    break
+                hl.update(chunk)
+
+        return  hl.hexdigest()
 
     def valid(self):
         return self.cache_file.exists() and self._check(self.cache_file)
-- 
2.46.0

[PULL v2 24/24] Update FreeBSD CI jobs FreeBSD 14.1

[Thomas Huth]

The current FreeBSD CI jobs are failing installation since the
"opencv" package is now missing there. Updating to 14.1 fixes
the issue.

Message-Id: <20240911090149.286257-1-thuth@redhat.com>
Reviewed-by: Li-Wen Hsu <lwhsu@FreeBSD.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/cirrus.yml                                  | 6 +++---
 .gitlab-ci.d/cirrus/{freebsd-13.vars => freebsd-14.vars} | 2 +-
 tests/lcitool/refresh                                    | 4 ++--
 tests/vm/freebsd                                         | 6 +++---
 4 files changed, 9 insertions(+), 9 deletions(-)
 rename .gitlab-ci.d/cirrus/{freebsd-13.vars => freebsd-14.vars} (95%)

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 75df1273bc..92c97eefc1 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -46,13 +46,13 @@
   variables:
     QEMU_JOB_CIRRUS: 1
 
-x64-freebsd-13-build:
+x64-freebsd-14-build:
   extends: .cirrus_build_job
   variables:
-    NAME: freebsd-13
+    NAME: freebsd-14
     CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
     CIRRUS_VM_IMAGE_SELECTOR: image_family
-    CIRRUS_VM_IMAGE_NAME: freebsd-13-3
+    CIRRUS_VM_IMAGE_NAME: freebsd-14-1
     CIRRUS_VM_CPUS: 8
     CIRRUS_VM_RAM: 8G
     UPDATE_COMMAND: pkg update; pkg upgrade -y
diff --git a/.gitlab-ci.d/cirrus/freebsd-13.vars b/.gitlab-ci.d/cirrus/freebsd-14.vars
similarity index 95%
rename from .gitlab-ci.d/cirrus/freebsd-13.vars
rename to .gitlab-ci.d/cirrus/freebsd-14.vars
index 69c948b503..aba0eff4b9 100644
--- a/.gitlab-ci.d/cirrus/freebsd-13.vars
+++ b/.gitlab-ci.d/cirrus/freebsd-14.vars
@@ -1,6 +1,6 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool variables freebsd-13 qemu
+#  $ lcitool variables freebsd-14 qemu
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index ac803e34f1..290804f6be 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -207,14 +207,14 @@ try:
     #
     # Cirrus packages lists for GitLab
     #
-    generate_cirrus("freebsd-13")
+    generate_cirrus("freebsd-14")
     generate_cirrus("macos-13")
     generate_cirrus("macos-14")
 
     #
     # VM packages lists
     #
-    generate_pkglist("freebsd", "freebsd-13")
+    generate_pkglist("freebsd", "freebsd-14")
 
     #
     # Ansible package lists
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 1247f40a38..74b3b1e520 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -28,8 +28,8 @@ class FreeBSDVM(basevm.BaseVM):
     name = "freebsd"
     arch = "x86_64"
 
-    link = "https://download.freebsd.org/releases/CI-IMAGES/13.2-RELEASE/amd64/Latest/FreeBSD-13.2-RELEASE-amd64-BASIC-CI.raw.xz"
-    csum = "a4fb3b6c7b75dd4d58fb0d75e4caf72844bffe0ca00e66459c028b198ffb3c0e"
+    link = "https://download.freebsd.org/releases/CI-IMAGES/14.1-RELEASE/amd64/Latest/FreeBSD-14.1-RELEASE-amd64-BASIC-CI.raw.xz"
+    csum = "202fe27a05427f0a86d3ebb97712745186f2776ccc4f70d95466dd99a0238ba5"
     size = "20G"
 
     BUILD_SCRIPT = """
@@ -39,7 +39,7 @@ class FreeBSDVM(basevm.BaseVM):
         mkdir src build; cd src;
         tar -xf /dev/vtbd1;
         cd ../build;
-        ../src/configure --python=python3.9  --extra-ldflags=-L/usr/local/lib \
+        ../src/configure --extra-ldflags=-L/usr/local/lib \
                          --extra-cflags=-I/usr/local/include {configure_opts};
         gmake --output-sync -j{jobs} {target} {verbose};
     """
-- 
2.46.0

Re: [PULL v2 00/24] Tests and misc patches

[Peter Maydell]

On Wed, 11 Sept 2024 at 10:34, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Peter!
>
> The following changes since commit a66f28df650166ae8b50c992eea45e7b247f4143:
>
>   Merge tag 'migration-20240909-pull-request' of https://gitlab.com/peterx/qemu into staging (2024-09-10 11:19:22 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/thuth/qemu.git tags/pull-request-2024-09-11
>
> for you to fetch changes up to b4358ed4fd29c21c69e492d814f0926c58caa10f:
>
>   Update FreeBSD CI jobs FreeBSD 14.1 (2024-09-11 11:26:41 +0200)
>
> ----------------------------------------------------------------
> * Split --enable-sanitizers to --enable-{asan, ubsan}
> * Build MSYS2 job using multiple CPUs
> * Fix "make distclean" wrt contrib/plugins/
> * Convert more Avocado tests to plain standalone functional tests
> * Fix bug that breaks "make check-functional" when tesseract is missing
> * Use builtin hashlib of Python in the functional tests
> * Update the FreeBSD CI jobs to 14.1
>
> v2:
>  - Use python hashlib instead of external programs
>  - Update FreeBSD CI jobs FreeBSD 14.1
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/9.2
for any user-visible changes.

-- PMM