[PATCH 0/5] maintainer updates for 9.0 (docker, plugin tests, deprecation, elf, semihosting)

[PATCH 0/5] maintainer updates for 9.0 (docker, plugin tests, deprecation, elf, semihosting)

[Alex Bennée]

A fairly random collection of fixes in this tree. I've still got a
report of openbsd rebuilding which is confusing me but I didn't want
to hold up getting eyes on real fixes.

The plugins register support may get added to the PR if it gets any
review.

The following need review:

  Revert "hw/elf_ops: Ignore loadable segments with zero size"
  docs: mark CRIS support as deprecated

Alex.

Alex Bennée (2):
  docs: mark CRIS support as deprecated
  Revert "hw/elf_ops: Ignore loadable segments with zero size"

Fabiano Rosas (1):
  tests/docker: Add sqlite3 module to openSUSE Leap container

Paolo Bonzini (2):
  configure: run plugin TCG tests again
  kconfig: use "select" to enable semihosting

 docs/about/deprecated.rst                     |  8 ++
 configure                                     |  3 +
 configs/devices/m68k-softmmu/default.mak      |  2 -
 configs/devices/mips-softmmu/common.mak       |  3 -
 configs/devices/nios2-softmmu/default.mak     |  2 -
 configs/devices/riscv32-softmmu/default.mak   |  2 -
 configs/devices/riscv64-softmmu/default.mak   |  2 -
 configs/devices/xtensa-softmmu/default.mak    |  2 -
 include/hw/elf_ops.h                          | 75 +++++++++----------
 target/m68k/Kconfig                           |  1 +
 target/mips/Kconfig                           |  1 +
 target/nios2/Kconfig                          |  1 +
 target/riscv/Kconfig                          |  2 +
 target/xtensa/Kconfig                         |  1 +
 tests/docker/dockerfiles/opensuse-leap.docker |  1 +
 tests/lcitool/mappings.yml                    |  4 +
 tests/lcitool/projects/qemu.yml               |  1 +
 17 files changed, 59 insertions(+), 52 deletions(-)

-- 
2.39.2

[PATCH 1/5] tests/docker: Add sqlite3 module to openSUSE Leap container

[Alex Bennée]

From: Fabiano Rosas <farosas@suse.de>

Avocado needs sqlite3:

  Failed to load plugin from module "avocado.plugins.journal":
  ImportError("Module 'sqlite3' is not installed.
  Use: sudo zypper install python311 to install it")

>From 'zypper info python311':
  "This package supplies rich command line features provided by
  readline, and sqlite3 support for the interpreter core, thus forming
  a so called "extended" runtime."

Include the appropriate package in the lcitool mappings which will
guarantee the dockerfile gets properly updated when lcitool is
run. Also include the updated dockerfile.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Suggested-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240117164227.32143-1-farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/opensuse-leap.docker | 1 +
 tests/lcitool/mappings.yml                    | 4 ++++
 tests/lcitool/projects/qemu.yml               | 1 +
 3 files changed, 6 insertions(+)

diff --git a/tests/docker/dockerfiles/opensuse-leap.docker b/tests/docker/dockerfiles/opensuse-leap.docker
index dc0e36ce488..cf753383a45 100644
--- a/tests/docker/dockerfiles/opensuse-leap.docker
+++ b/tests/docker/dockerfiles/opensuse-leap.docker
@@ -90,6 +90,7 @@ RUN zypper update -y && \
            pcre-devel-static \
            pipewire-devel \
            pkgconfig \
+           python311 \
            python311-base \
            python311-pip \
            python311-setuptools \
diff --git a/tests/lcitool/mappings.yml b/tests/lcitool/mappings.yml
index 0b908882f1d..407c03301bf 100644
--- a/tests/lcitool/mappings.yml
+++ b/tests/lcitool/mappings.yml
@@ -59,6 +59,10 @@ mappings:
     CentOSStream8:
     OpenSUSELeap15:
 
+  python3-sqlite3:
+    CentOSStream8: python38
+    OpenSUSELeap15: python311
+
   python3-tomli:
     # test using tomllib
     apk:
diff --git a/tests/lcitool/projects/qemu.yml b/tests/lcitool/projects/qemu.yml
index 82092c9f175..149b15de57b 100644
--- a/tests/lcitool/projects/qemu.yml
+++ b/tests/lcitool/projects/qemu.yml
@@ -97,6 +97,7 @@ packages:
  - python3-pip
  - python3-sphinx
  - python3-sphinx-rtd-theme
+ - python3-sqlite3
  - python3-tomli
  - python3-venv
  - rpm2cpio
-- 
2.39.2

[PATCH 2/5] docs: mark CRIS support as deprecated

[Alex Bennée]

This might be premature but while streamling the avocado tests I
realised the only tests we have are "check-tcg" ones. The aging
fedora-criss-cross image works well enough for developers but can't be
used in CI as we need supported build platforms to build QEMU.

Does this mean the writing is on the wall for this architecture?

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Rabin Vincent <rabinv@axis.com>
Cc: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <20230925144854.1872513-5-alex.bennee@linaro.org>
---
 docs/about/deprecated.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index d4492b94604..82922476d72 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -183,6 +183,14 @@ Nios II CPU (since 8.2)
 The Nios II architecture is orphan. The ``nios2`` guest CPU support is
 deprecated and will be removed in a future version of QEMU.
 
+CRIS CPU architecture (since 9.0)
+'''''''''''''''''''''''''''''''''
+
+The CRIS architecture was pulled from Linux in 4.17 and the compiler
+is no longer packaged in any distro making it harder to run the
+``check-tcg`` tests. Unless we can improve the testing situation there
+is a chance the code will bitrot without anyone noticing.
+
 
 System emulator machines
 ------------------------
-- 
2.39.2

[PATCH 3/5] configure: run plugin TCG tests again

[Alex Bennée]

From: Paolo Bonzini <pbonzini@redhat.com>

Commit 39fb3cfc28b ("configure: clean up plugin option handling", 2023-10-18)
dropped the CONFIG_PLUGIN line from tests/tcg/config-host.mak, due to confusion
caused by the shadowing of $config_host_mak.  However, TCG tests were still
expecting it.  Oops.

Put it back, in the meanwhile the shadowing is gone so it's clear that it goes
in the tests/tcg configuration.

Cc:  <alex.bennee@linaro.org>
Fixes: 39fb3cfc28b ("configure: clean up plugin option handling", 2023-10-18)
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20240124115332.612162-1-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/configure b/configure
index 3d8e24ae011..ff058d6c486 100755
--- a/configure
+++ b/configure
@@ -1644,6 +1644,9 @@ fi
 mkdir -p tests/tcg
 echo "# Automatically generated by configure - do not modify" > tests/tcg/$config_host_mak
 echo "SRC_PATH=$source_path" >> tests/tcg/$config_host_mak
+if test "$plugins" = "yes" ; then
+    echo "CONFIG_PLUGIN=y" >> tests/tcg/$config_host_mak
+fi
 
 tcg_tests_targets=
 for target in $target_list; do
-- 
2.39.2

[PATCH 4/5] Revert "hw/elf_ops: Ignore loadable segments with zero size"

[Alex Bennée]

This regressed qemu-system-xtensa:

    TEST    test_load_store on xtensa
  qemu-system-xtensa: Some ROM regions are overlapping
  These ROM regions might have been loaded by direct user request or by default.
  They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory.
  Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses.

  The following two regions overlap (in the memory address space):
    test_load_store ELF program header segment 1 (addresses 0x0000000000001000 - 0x0000000000001f26)
    test_load_store ELF program header segment 2 (addresses 0x0000000000001ab8 - 0x0000000000001ab8)
  make[1]: *** [Makefile:187: run-test_load_store] Error 1

This reverts commit 62570f1434160d356311e1c217537e24a4ac85cd.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 include/hw/elf_ops.h | 75 +++++++++++++++++++++-----------------------
 1 file changed, 36 insertions(+), 39 deletions(-)

diff --git a/include/hw/elf_ops.h b/include/hw/elf_ops.h
index 3e966ddd5a1..9c35d1b9da6 100644
--- a/include/hw/elf_ops.h
+++ b/include/hw/elf_ops.h
@@ -427,16 +427,6 @@ static ssize_t glue(load_elf, SZ)(const char *name, int fd,
             file_size = ph->p_filesz; /* Size of the allocated data */
             data_offset = ph->p_offset; /* Offset where the data is located */
 
-            /*
-             * Some ELF files really do have segments of zero size;
-             * just ignore them rather than trying to set the wrong addr,
-             * or create empty ROM blobs, because the zero-length blob can
-             * falsely trigger the overlapping-ROM-blobs check.
-             */
-            if (mem_size == 0) {
-                continue;
-            }
-
             if (file_size > 0) {
                 if (g_mapped_file_get_length(mapped_file) <
                     file_size + data_offset) {
@@ -540,38 +530,45 @@ static ssize_t glue(load_elf, SZ)(const char *name, int fd,
                 *pentry = ehdr.e_entry - ph->p_vaddr + ph->p_paddr;
             }
 
-            if (load_rom) {
-                g_autofree char *label =
-                    g_strdup_printf("%s ELF program header segment %d",
-                                    name, i);
-
-                /*
-                 * rom_add_elf_program() takes its own reference to
-                 * 'mapped_file'.
-                 */
-                rom_add_elf_program(label, mapped_file, data, file_size,
-                                    mem_size, addr, as);
-            } else {
-                MemTxResult res;
-
-                res = address_space_write(as ? as : &address_space_memory,
-                                          addr, MEMTXATTRS_UNSPECIFIED,
-                                          data, file_size);
-                if (res != MEMTX_OK) {
-                    goto fail;
-                }
-                /*
-                 * We need to zero'ify the space that is not copied
-                 * from file
-                 */
-                if (file_size < mem_size) {
-                    res = address_space_set(as ? as : &address_space_memory,
-                                            addr + file_size, 0,
-                                            mem_size - file_size,
-                                            MEMTXATTRS_UNSPECIFIED);
+            /* Some ELF files really do have segments of zero size;
+             * just ignore them rather than trying to create empty
+             * ROM blobs, because the zero-length blob can falsely
+             * trigger the overlapping-ROM-blobs check.
+             */
+            if (mem_size != 0) {
+                if (load_rom) {
+                    g_autofree char *label =
+                        g_strdup_printf("%s ELF program header segment %d",
+                                        name, i);
+
+                    /*
+                     * rom_add_elf_program() takes its own reference to
+                     * 'mapped_file'.
+                     */
+                    rom_add_elf_program(label, mapped_file, data, file_size,
+                                        mem_size, addr, as);
+                } else {
+                    MemTxResult res;
+
+                    res = address_space_write(as ? as : &address_space_memory,
+                                              addr, MEMTXATTRS_UNSPECIFIED,
+                                              data, file_size);
                     if (res != MEMTX_OK) {
                         goto fail;
                     }
+                    /*
+                     * We need to zero'ify the space that is not copied
+                     * from file
+                     */
+                    if (file_size < mem_size) {
+                        res = address_space_set(as ? as : &address_space_memory,
+                                                addr + file_size, 0,
+                                                mem_size - file_size,
+                                                MEMTXATTRS_UNSPECIFIED);
+                        if (res != MEMTX_OK) {
+                            goto fail;
+                        }
+                    }
                 }
             }
 
-- 
2.39.2

[PATCH 5/5] kconfig: use "select" to enable semihosting

[Alex Bennée]

From: Paolo Bonzini <pbonzini@redhat.com>

Just like all other dependencies, these can be expressed in Kconfig
files rather than in the default configurations.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20240129115809.1039924-1-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configs/devices/m68k-softmmu/default.mak    | 2 --
 configs/devices/mips-softmmu/common.mak     | 3 ---
 configs/devices/nios2-softmmu/default.mak   | 2 --
 configs/devices/riscv32-softmmu/default.mak | 2 --
 configs/devices/riscv64-softmmu/default.mak | 2 --
 configs/devices/xtensa-softmmu/default.mak  | 2 --
 target/m68k/Kconfig                         | 1 +
 target/mips/Kconfig                         | 1 +
 target/nios2/Kconfig                        | 1 +
 target/riscv/Kconfig                        | 2 ++
 target/xtensa/Kconfig                       | 1 +
 11 files changed, 6 insertions(+), 13 deletions(-)

diff --git a/configs/devices/m68k-softmmu/default.mak b/configs/devices/m68k-softmmu/default.mak
index 7f8619e4278..8dcaa28ed38 100644
--- a/configs/devices/m68k-softmmu/default.mak
+++ b/configs/devices/m68k-softmmu/default.mak
@@ -1,7 +1,5 @@
 # Default configuration for m68k-softmmu
 
-CONFIG_SEMIHOSTING=y
-
 # Boards:
 #
 CONFIG_AN5206=y
diff --git a/configs/devices/mips-softmmu/common.mak b/configs/devices/mips-softmmu/common.mak
index 7da99327a77..1a853841b27 100644
--- a/configs/devices/mips-softmmu/common.mak
+++ b/configs/devices/mips-softmmu/common.mak
@@ -1,8 +1,5 @@
 # Common mips*-softmmu CONFIG defines
 
-# CONFIG_SEMIHOSTING is always required on this architecture
-CONFIG_SEMIHOSTING=y
-
 CONFIG_ISA_BUS=y
 CONFIG_PCI=y
 CONFIG_PCI_DEVICES=y
diff --git a/configs/devices/nios2-softmmu/default.mak b/configs/devices/nios2-softmmu/default.mak
index 1bc4082ea99..e130d024e62 100644
--- a/configs/devices/nios2-softmmu/default.mak
+++ b/configs/devices/nios2-softmmu/default.mak
@@ -1,7 +1,5 @@
 # Default configuration for nios2-softmmu
 
-CONFIG_SEMIHOSTING=y
-
 # Boards:
 #
 CONFIG_NIOS2_10M50=y
diff --git a/configs/devices/riscv32-softmmu/default.mak b/configs/devices/riscv32-softmmu/default.mak
index d847bd5692e..94a236c9c25 100644
--- a/configs/devices/riscv32-softmmu/default.mak
+++ b/configs/devices/riscv32-softmmu/default.mak
@@ -3,8 +3,6 @@
 # Uncomment the following lines to disable these optional devices:
 #
 #CONFIG_PCI_DEVICES=n
-CONFIG_SEMIHOSTING=y
-CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
 
 # Boards:
 #
diff --git a/configs/devices/riscv64-softmmu/default.mak b/configs/devices/riscv64-softmmu/default.mak
index bc69301fa4a..3f680594484 100644
--- a/configs/devices/riscv64-softmmu/default.mak
+++ b/configs/devices/riscv64-softmmu/default.mak
@@ -3,8 +3,6 @@
 # Uncomment the following lines to disable these optional devices:
 #
 #CONFIG_PCI_DEVICES=n
-CONFIG_SEMIHOSTING=y
-CONFIG_ARM_COMPATIBLE_SEMIHOSTING=y
 
 # Boards:
 #
diff --git a/configs/devices/xtensa-softmmu/default.mak b/configs/devices/xtensa-softmmu/default.mak
index 4fe1bf00c94..49e4c9da88c 100644
--- a/configs/devices/xtensa-softmmu/default.mak
+++ b/configs/devices/xtensa-softmmu/default.mak
@@ -1,7 +1,5 @@
 # Default configuration for Xtensa
 
-CONFIG_SEMIHOSTING=y
-
 # Boards:
 #
 CONFIG_XTENSA_SIM=y
diff --git a/target/m68k/Kconfig b/target/m68k/Kconfig
index 23debad519a..9eae71486ff 100644
--- a/target/m68k/Kconfig
+++ b/target/m68k/Kconfig
@@ -1,2 +1,3 @@
 config M68K
     bool
+    select SEMIHOSTING
diff --git a/target/mips/Kconfig b/target/mips/Kconfig
index 6adf1453548..eb19c94c7d4 100644
--- a/target/mips/Kconfig
+++ b/target/mips/Kconfig
@@ -1,5 +1,6 @@
 config MIPS
     bool
+    select SEMIHOSTING
 
 config MIPS64
     bool
diff --git a/target/nios2/Kconfig b/target/nios2/Kconfig
index 1529ab8950d..c65550c861a 100644
--- a/target/nios2/Kconfig
+++ b/target/nios2/Kconfig
@@ -1,2 +1,3 @@
 config NIOS2
     bool
+    select SEMIHOSTING
diff --git a/target/riscv/Kconfig b/target/riscv/Kconfig
index b9e5932f13f..adb7de3f37d 100644
--- a/target/riscv/Kconfig
+++ b/target/riscv/Kconfig
@@ -1,5 +1,7 @@
 config RISCV32
     bool
+    select ARM_COMPATIBLE_SEMIHOSTING # for do_common_semihosting()
 
 config RISCV64
     bool
+    select ARM_COMPATIBLE_SEMIHOSTING # for do_common_semihosting()
diff --git a/target/xtensa/Kconfig b/target/xtensa/Kconfig
index a3c8dc7f6d7..5e46049262d 100644
--- a/target/xtensa/Kconfig
+++ b/target/xtensa/Kconfig
@@ -1,2 +1,3 @@
 config XTENSA
     bool
+    select SEMIHOSTING
-- 
2.39.2

Re: [PATCH 5/5] kconfig: use "select" to enable semihosting

[Thomas Huth]

On 01/02/2024 13.28, Alex Bennée wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
> 
> Just like all other dependencies, these can be expressed in Kconfig
> files rather than in the default configurations.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Acked-by: Alistair Francis <alistair.francis@wdc.com>
> Message-Id: <20240129115809.1039924-1-pbonzini@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   configs/devices/m68k-softmmu/default.mak    | 2 --
>   configs/devices/mips-softmmu/common.mak     | 3 ---
>   configs/devices/nios2-softmmu/default.mak   | 2 --
>   configs/devices/riscv32-softmmu/default.mak | 2 --
>   configs/devices/riscv64-softmmu/default.mak | 2 --
>   configs/devices/xtensa-softmmu/default.mak  | 2 --
>   target/m68k/Kconfig                         | 1 +
>   target/mips/Kconfig                         | 1 +
>   target/nios2/Kconfig                        | 1 +
>   target/riscv/Kconfig                        | 2 ++
>   target/xtensa/Kconfig                       | 1 +
>   11 files changed, 6 insertions(+), 13 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 2/5] docs: mark CRIS support as deprecated

[Thomas Huth]

On 01/02/2024 13.28, Alex Bennée wrote:
> This might be premature but while streamling the avocado tests I

s/streamling/streamlining/ ?

> realised the only tests we have are "check-tcg" ones. The aging
> fedora-criss-cross image works well enough for developers but can't be

s/criss/cris/

> used in CI as we need supported build platforms to build QEMU.
> 
> Does this mean the writing is on the wall for this architecture?
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Rabin Vincent <rabinv@axis.com>
> Cc: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
> Message-Id: <20230925144854.1872513-5-alex.bennee@linaro.org>
> ---
>   docs/about/deprecated.rst | 8 ++++++++
>   1 file changed, 8 insertions(+)
> 
> diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
> index d4492b94604..82922476d72 100644
> --- a/docs/about/deprecated.rst
> +++ b/docs/about/deprecated.rst
> @@ -183,6 +183,14 @@ Nios II CPU (since 8.2)
>   The Nios II architecture is orphan. The ``nios2`` guest CPU support is
>   deprecated and will be removed in a future version of QEMU.
>   
> +CRIS CPU architecture (since 9.0)
> +'''''''''''''''''''''''''''''''''
> +
> +The CRIS architecture was pulled from Linux in 4.17 and the compiler
> +is no longer packaged in any distro making it harder to run the
> +``check-tcg`` tests. Unless we can improve the testing situation there
> +is a chance the code will bitrot without anyone noticing.

With the typos fixed:
Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 4/5] Revert "hw/elf_ops: Ignore loadable segments with zero size"

[Thomas Huth]

On 01/02/2024 13.28, Alex Bennée wrote:
> This regressed qemu-system-xtensa:
> 
>      TEST    test_load_store on xtensa
>    qemu-system-xtensa: Some ROM regions are overlapping
>    These ROM regions might have been loaded by direct user request or by default.
>    They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory.
>    Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses.
> 
>    The following two regions overlap (in the memory address space):
>      test_load_store ELF program header segment 1 (addresses 0x0000000000001000 - 0x0000000000001f26)
>      test_load_store ELF program header segment 2 (addresses 0x0000000000001ab8 - 0x0000000000001ab8)
>    make[1]: *** [Makefile:187: run-test_load_store] Error 1
> 
> This reverts commit 62570f1434160d356311e1c217537e24a4ac85cd.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   include/hw/elf_ops.h | 75 +++++++++++++++++++++-----------------------
>   1 file changed, 36 insertions(+), 39 deletions(-)

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 4/5] Revert "hw/elf_ops: Ignore loadable segments with zero size"

[Peter Maydell]

On Thu, 1 Feb 2024 at 12:30, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> This regressed qemu-system-xtensa:
>
>     TEST    test_load_store on xtensa
>   qemu-system-xtensa: Some ROM regions are overlapping
>   These ROM regions might have been loaded by direct user request or by default.
>   They could be BIOS/firmware images, a guest kernel, initrd or some other file loaded into guest memory.
>   Check whether you intended to load all this guest code, and whether it has been built to load to the correct addresses.
>
>   The following two regions overlap (in the memory address space):
>     test_load_store ELF program header segment 1 (addresses 0x0000000000001000 - 0x0000000000001f26)
>     test_load_store ELF program header segment 2 (addresses 0x0000000000001ab8 - 0x0000000000001ab8)

Hmm -- this second segment is zero length, so why did we create
a ROM blob for it? The commit being reverted here looks like it
ought to be expanding the set of things for which we say
"zero size, ignore entirely"...

Anyway, revert given we have a regression is the first thing
to do if there's not an immediately obvious fix.

-- PMM