[PATCH 0/5] tests: Update Fedora and Alpine containers via lcitool

[PATCH 0/5] tests: Update Fedora and Alpine containers via lcitool

[Thomas Huth]

According to QEMU's support policy, Fedora 38 and Alpine 3.18 are not
supported anymore, so let's bump the containers to a newer version.

Both, Alpine 3.20 and Fedora 40 ship with Python 3.12 that breaks our
old version of Avocado since the "imp" module has been removed there.
To work around this problem, Fedora fortunately still ships a separate
"python3-zombi-imp" package that we can install. And with regards to
Alpine, we only upgrade to 3.19 that is still using Python 3.11.

Another problem are improvements in the undefined-behavior sanitizer
of the latest versions of Clang that we use in the "clang-system"
Fedora container: We now need to compile with -fno-sanitize=function
there until all spots in the source code have been fixed (and that
might take while since many of the issues are not trivial).

Thomas Huth (5):
  tests/lcitool: Delete obsolete centos-stream-8.yml file
  tests/lcitool: Bump to latest libvirt-ci and update Fedora and Alpine
    version
  .gitlab-ci.d/buildtest.yml: Use -fno-sanitize=function in the
    clang-system job
  tests/lcitool: Install mingw-w64-tools for the Windows cross-builds
  tests/docker/dockerfiles: Run lcitool-refresh to update Fedora and
    Alpine

 .gitlab-ci.d/buildtest.yml                         | 1 +
 tests/docker/dockerfiles/alpine.docker             | 4 ++--
 tests/docker/dockerfiles/fedora-win64-cross.docker | 6 ++++--
 tests/docker/dockerfiles/fedora.docker             | 5 +++--
 tests/lcitool/libvirt-ci                           | 2 +-
 tests/lcitool/projects/qemu-win-installer.yml      | 1 +
 tests/lcitool/projects/qemu.yml                    | 1 +
 tests/lcitool/refresh                              | 6 +++---
 tests/lcitool/targets/centos-stream-8.yml          | 3 ---
 9 files changed, 16 insertions(+), 13 deletions(-)
 delete mode 100644 tests/lcitool/targets/centos-stream-8.yml

-- 
2.45.1

[PATCH 1/5] tests/lcitool: Delete obsolete centos-stream-8.yml file

[Thomas Huth]

We've missed to delete this file when removing support for CentOS 8.
Since the current upstream version of the lcitool removed support
for CentOS 8 now, too, we have to remove the file before updating.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/lcitool/targets/centos-stream-8.yml | 3 ---
 1 file changed, 3 deletions(-)
 delete mode 100644 tests/lcitool/targets/centos-stream-8.yml

diff --git a/tests/lcitool/targets/centos-stream-8.yml b/tests/lcitool/targets/centos-stream-8.yml
deleted file mode 100644
index 6b11160fd1..0000000000
--- a/tests/lcitool/targets/centos-stream-8.yml
+++ /dev/null
@@ -1,3 +0,0 @@
-paths:
-  pip3: /usr/bin/pip3.8
-  python: /usr/bin/python3.8
-- 
2.45.1

Re: [PATCH 1/5] tests/lcitool: Delete obsolete centos-stream-8.yml file

[Daniel P. Berrangé]

On Sat, Jun 01, 2024 at 09:05:39AM +0200, Thomas Huth wrote:
> We've missed to delete this file when removing support for CentOS 8.
> Since the current upstream version of the lcitool removed support
> for CentOS 8 now, too, we have to remove the file before updating.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/lcitool/targets/centos-stream-8.yml | 3 ---
>  1 file changed, 3 deletions(-)
>  delete mode 100644 tests/lcitool/targets/centos-stream-8.yml

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH 2/5] tests/lcitool: Bump to latest libvirt-ci and update Fedora and Alpine version

[Thomas Huth]

Update to the latest version of lcitool. It dropped support for Fedora 38
and Alpine 3.18, so we have to update these to newer versions here, too.

Python 3.12 dropped the "imp" module which we still need for running
Avocado. Fortunately Fedora 40 still ships with a work-around package
that we can use until somebody updates our Avocado to a newer version.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/lcitool/libvirt-ci        | 2 +-
 tests/lcitool/projects/qemu.yml | 1 +
 tests/lcitool/refresh           | 6 +++---
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
index cec6703971..0e9490cebc 160000
--- a/tests/lcitool/libvirt-ci
+++ b/tests/lcitool/libvirt-ci
@@ -1 +1 @@
-Subproject commit cec67039719becbfbab866f9c23574f389cf9559
+Subproject commit 0e9490cebc726ef772b6c9e27dac32e7ae99f9b2
diff --git a/tests/lcitool/projects/qemu.yml b/tests/lcitool/projects/qemu.yml
index 7511ec7ccb..070d7f4706 100644
--- a/tests/lcitool/projects/qemu.yml
+++ b/tests/lcitool/projects/qemu.yml
@@ -89,6 +89,7 @@ packages:
  - pkg-config
  - pulseaudio
  - python3
+ - python3-imp
  - python3-numpy
  - python3-opencv
  - python3-pillow
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 789acefb75..9d8e9c6a4a 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -124,11 +124,11 @@ try:
     #
     # Standard native builds
     #
-    generate_dockerfile("alpine", "alpine-318")
+    generate_dockerfile("alpine", "alpine-319")
     generate_dockerfile("centos9", "centos-stream-9")
     generate_dockerfile("debian", "debian-12",
                         trailer="".join(debian12_extras))
-    generate_dockerfile("fedora", "fedora-38")
+    generate_dockerfile("fedora", "fedora-40")
     generate_dockerfile("opensuse-leap", "opensuse-leap-15")
     generate_dockerfile("ubuntu2204", "ubuntu-2204")
 
@@ -191,7 +191,7 @@ try:
                         trailer=cross_build("s390x-linux-gnu-",
                                             "s390x-softmmu,s390x-linux-user"))
 
-    generate_dockerfile("fedora-win64-cross", "fedora-38",
+    generate_dockerfile("fedora-win64-cross", "fedora-40",
                         project='qemu,qemu-win-installer',
                         cross="mingw64",
                         trailer=cross_build("x86_64-w64-mingw32-",
-- 
2.45.1

Re: [PATCH 2/5] tests/lcitool: Bump to latest libvirt-ci and update Fedora and Alpine version

[Daniel P. Berrangé]

On Sat, Jun 01, 2024 at 09:05:40AM +0200, Thomas Huth wrote:
> Update to the latest version of lcitool. It dropped support for Fedora 38
> and Alpine 3.18, so we have to update these to newer versions here, too.
> 
> Python 3.12 dropped the "imp" module which we still need for running
> Avocado. Fortunately Fedora 40 still ships with a work-around package
> that we can use until somebody updates our Avocado to a newer version.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/lcitool/libvirt-ci        | 2 +-
>  tests/lcitool/projects/qemu.yml | 1 +
>  tests/lcitool/refresh           | 6 +++---
>  3 files changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH 3/5] .gitlab-ci.d/buildtest.yml: Use -fno-sanitize=function in the clang-system job

[Thomas Huth]

The latest version of Clang (version 18 from Fedora 40) now reports
bad function pointer casts as undefined behavior. Unfortunately, we are
still doing this in quite a lot of places in the QEMU code and some of
them are not easy to fix. So for the time being, temporarily switch this
off in the failing clang-system job until all spots in the QEMU sources
have been tackled.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 91c57efded..0eec570310 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -432,6 +432,7 @@ clang-system:
     IMAGE: fedora
     CONFIGURE_ARGS: --cc=clang --cxx=clang++
       --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
+      --extra-cflags=-fno-sanitize=function
     TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu s390x-softmmu
     MAKE_CHECK_ARGS: check-qtest check-tcg
 
-- 
2.45.1

Re: [PATCH 3/5] .gitlab-ci.d/buildtest.yml: Use -fno-sanitize=function in the clang-system job

[Daniel P. Berrangé]

On Sat, Jun 01, 2024 at 09:05:41AM +0200, Thomas Huth wrote:
> The latest version of Clang (version 18 from Fedora 40) now reports
> bad function pointer casts as undefined behavior. Unfortunately, we are
> still doing this in quite a lot of places in the QEMU code and some of
> them are not easy to fix. So for the time being, temporarily switch this
> off in the failing clang-system job until all spots in the QEMU sources
> have been tackled.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci.d/buildtest.yml | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index 91c57efded..0eec570310 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -432,6 +432,7 @@ clang-system:
>      IMAGE: fedora
>      CONFIGURE_ARGS: --cc=clang --cxx=clang++
>        --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
> +      --extra-cflags=-fno-sanitize=function
>      TARGETS: alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu s390x-softmmu
>      MAKE_CHECK_ARGS: check-qtest check-tcg

I think we have a gitlab issue for the functio ncast problems - good
to have a comment linking to those, to remind us to remove -fno-sanitize
when they're fixed.

Regardless

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH 4/5] tests/lcitool: Install mingw-w64-tools for the Windows cross-builds

[Thomas Huth]

Beside g++ we also need the mingw-w64-tools for properly building
the code in qga/vss-win32/ , so let's install that package now, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/lcitool/projects/qemu-win-installer.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/lcitool/projects/qemu-win-installer.yml b/tests/lcitool/projects/qemu-win-installer.yml
index 86aa22297c..f3663ba030 100644
--- a/tests/lcitool/projects/qemu-win-installer.yml
+++ b/tests/lcitool/projects/qemu-win-installer.yml
@@ -2,3 +2,4 @@
 ---
 packages:
  - g++
+ - mingw-w64-tools
-- 
2.45.1

Re: [PATCH 4/5] tests/lcitool: Install mingw-w64-tools for the Windows cross-builds

[Daniel P. Berrangé]

On Sat, Jun 01, 2024 at 09:05:42AM +0200, Thomas Huth wrote:
> Beside g++ we also need the mingw-w64-tools for properly building
> the code in qga/vss-win32/ , so let's install that package now, too.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/lcitool/projects/qemu-win-installer.yml | 1 +
>  1 file changed, 1 insertion(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 4/5] tests/lcitool: Install mingw-w64-tools for the Windows cross-builds

[Philippe Mathieu-Daudé]

On 1/6/24 09:05, Thomas Huth wrote:
> Beside g++ we also need the mingw-w64-tools for properly building
> the code in qga/vss-win32/ , so let's install that package now, too.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>   tests/lcitool/projects/qemu-win-installer.yml | 1 +
>   1 file changed, 1 insertion(+)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

[PATCH 5/5] tests/docker/dockerfiles: Run lcitool-refresh to update Fedora and Alpine

[Thomas Huth]

Run "make lcitool-refresh" to update the dockerfiles with the recent
changes to the lcitool.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/docker/dockerfiles/alpine.docker             | 4 ++--
 tests/docker/dockerfiles/fedora-win64-cross.docker | 6 ++++--
 tests/docker/dockerfiles/fedora.docker             | 5 +++--
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker
index 554464f31e..b079a83fe2 100644
--- a/tests/docker/dockerfiles/alpine.docker
+++ b/tests/docker/dockerfiles/alpine.docker
@@ -1,10 +1,10 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool dockerfile --layers all alpine-318 qemu
+#  $ lcitool dockerfile --layers all alpine-319 qemu
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
-FROM docker.io/library/alpine:3.18
+FROM docker.io/library/alpine:3.19
 
 RUN apk update && \
     apk upgrade && \
diff --git a/tests/docker/dockerfiles/fedora-win64-cross.docker b/tests/docker/dockerfiles/fedora-win64-cross.docker
index 0f78711876..007e1574bd 100644
--- a/tests/docker/dockerfiles/fedora-win64-cross.docker
+++ b/tests/docker/dockerfiles/fedora-win64-cross.docker
@@ -1,10 +1,10 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool dockerfile --layers all --cross-arch mingw64 fedora-38 qemu,qemu-win-installer
+#  $ lcitool dockerfile --layers all --cross-arch mingw64 fedora-40 qemu,qemu-win-installer
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
-FROM registry.fedoraproject.org/fedora:38
+FROM registry.fedoraproject.org/fedora:40
 
 RUN dnf install -y nosync && \
     printf '#!/bin/sh\n\
@@ -51,6 +51,7 @@ exec "$@"\n' > /usr/bin/nosync && \
                python3-pip \
                python3-sphinx \
                python3-sphinx_rtd_theme \
+               python3-zombie-imp \
                sed \
                socat \
                sparse \
@@ -74,6 +75,7 @@ ENV NINJA "/usr/bin/ninja"
 ENV PYTHON "/usr/bin/python3"
 
 RUN nosync dnf install -y \
+               mingw-w64-tools \
                mingw32-nsis \
                mingw64-SDL2 \
                mingw64-SDL2_image \
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 098c894d10..44f239c088 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,10 +1,10 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool dockerfile --layers all fedora-38 qemu
+#  $ lcitool dockerfile --layers all fedora-40 qemu
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
-FROM registry.fedoraproject.org/fedora:38
+FROM registry.fedoraproject.org/fedora:40
 
 RUN dnf install -y nosync && \
     printf '#!/bin/sh\n\
@@ -110,6 +110,7 @@ exec "$@"\n' > /usr/bin/nosync && \
                python3-pip \
                python3-sphinx \
                python3-sphinx_rtd_theme \
+               python3-zombie-imp \
                rdma-core-devel \
                sed \
                snappy-devel \
-- 
2.45.1

Re: [PATCH 5/5] tests/docker/dockerfiles: Run lcitool-refresh to update Fedora and Alpine

[Daniel P. Berrangé]

On Sat, Jun 01, 2024 at 09:05:43AM +0200, Thomas Huth wrote:
> Run "make lcitool-refresh" to update the dockerfiles with the recent
> changes to the lcitool.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/docker/dockerfiles/alpine.docker             | 4 ++--
>  tests/docker/dockerfiles/fedora-win64-cross.docker | 6 ++++--
>  tests/docker/dockerfiles/fedora.docker             | 5 +++--
>  3 files changed, 9 insertions(+), 6 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|