[PULL 00/21] (Mostly) build system patches for 2023-06-06

[PULL 00/21] (Mostly) build system patches for 2023-06-06

[Paolo Bonzini]

The following changes since commit 369081c4558e7e940fa36ce59bf17b2e390f55d3:

  Merge tag 'pull-tcg-20230605' of https://gitlab.com/rth7680/qemu into staging (2023-06-05 13:16:56 -0700)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 6f3ae23b29ad5831902e3ecdc7e443bbbf295bde:

  configure: remove --with-git-submodules= (2023-06-06 16:30:01 +0200)

----------------------------------------------------------------
* finish atomics revamp
* meson.build tweaks
* revert avocado update
* always upgrade/downgrade locally installed Python packages
* switch from submodules to subprojects
* remove --with-git= option
* rename --enable-pypi to --enable-download, control submodules and subprojects too

----------------------------------------------------------------
Michal Privoznik (1):
      meson: Avoid implicit declaration of absent functions

Paolo Bonzini (15):
      atomics: eliminate mb_read/mb_set
      scripts: remove dead file
      Revert "tests/requirements.txt: bump up avocado-framework version to 101.0"
      tests: Use separate virtual environment for avocado
      mkvenv: always pass locally-installed packages to pip
      configure: remove --with-git= option
      configure: rename --enable-pypi to --enable-download, control subprojects too
      git-submodule: allow partial update of .git-submodule-status
      build: log submodule update from git-submodule.sh
      meson: subprojects: replace submodules with wrap files
      configure: move SLOF submodule handling to pc-bios/s390-ccw
      pc-bios/s390-ccw: always build network bootloader
      meson: subprojects: replace berkeley-{soft,test}float-3 with wraps
      build: remove git submodule handling from main makefile
      configure: remove --with-git-submodules=

Philippe Mathieu-Daudé (1):
      scsi/qemu-pr-helper: Drop support for 'old' libmultipath API

Thomas Huth (4):
      meson.build: Group the UI entries in a separate summary section
      meson.build: Group the network backend entries in a separate summary section
      meson.build: Group the audio backend entries in a separate summary section
      meson.build: Use -Wno-undef only for SDL2 versions that need it

 .gitignore                                         |   2 -
 .gitlab-ci.d/buildtest-template.yml                |   4 +-
 .gitlab-ci.d/buildtest.yml                         |   6 +-
 .gitmodules                                        |  15 -
 Makefile                                           |  12 -
 accel/tcg/cpu-exec.c                               |   2 +-
 accel/tcg/tcg-accel-ops-mttcg.c                    |   2 +-
 accel/tcg/tcg-accel-ops-rr.c                       |   4 +-
 configure                                          | 169 ++-----
 docs/devel/acpi-bits.rst                           |   6 +-
 docs/devel/atomics.rst                             |  27 +-
 docs/devel/testing.rst                             |  14 +-
 include/qemu/atomic.h                              |  17 +-
 meson.build                                        | 134 +++--
 monitor/qmp.c                                      |   2 +-
 pc-bios/s390-ccw/Makefile                          |  16 +-
 python/scripts/mkvenv.py                           |  76 ++-
 scripts/archive-source.sh                          |  27 +-
 scripts/ci/org.centos/stream/8/x86_64/configure    |   3 -
 scripts/ci/org.centos/stream/8/x86_64/test-avocado |   4 +-
 scripts/device-crash-test                          |   2 +-
 scripts/git-submodule.sh                           |  69 +--
 scripts/make-release                               |   5 +
 scripts/test-driver.py                             |  35 --
 scsi/qemu-pr-helper.c                              |   4 -
 softmmu/cpus.c                                     |   2 +-
 softmmu/physmem.c                                  |   2 +-
 subprojects/.gitignore                             |   8 +
 subprojects/berkeley-softfloat-3.wrap              |   5 +
 subprojects/berkeley-testfloat-3.wrap              |   5 +
 subprojects/dtc                                    |   1 -
 subprojects/dtc.wrap                               |   4 +
 subprojects/keycodemapdb                           |   1 -
 subprojects/keycodemapdb.wrap                      |   4 +
 subprojects/libvfio-user                           |   1 -
 subprojects/libvfio-user.wrap                      |   4 +
 .../packagefiles/berkeley-softfloat-3/meson.build  | 339 +++++++++++++
 .../berkeley-softfloat-3/meson_options.txt         |   1 +
 .../packagefiles/berkeley-testfloat-3/meson.build  | 220 +++++++++
 .../berkeley-testfloat-3/meson_options.txt         |   1 +
 target/arm/hvf/hvf.c                               |   2 +-
 tests/Makefile.include                             |  32 +-
 tests/fp/meson.build                               | 541 +--------------------
 tests/requirements.txt                             |   9 +-
 tests/unit/test-aio-multithread.c                  |   2 +-
 tests/vm/Makefile.include                          |   2 +-
 util/qemu-coroutine-lock.c                         |   4 +-
 47 files changed, 900 insertions(+), 947 deletions(-)
 delete mode 100644 scripts/test-driver.py
 create mode 100644 subprojects/.gitignore
 create mode 100644 subprojects/berkeley-softfloat-3.wrap
 create mode 100644 subprojects/berkeley-testfloat-3.wrap
 delete mode 160000 subprojects/dtc
 create mode 100644 subprojects/dtc.wrap
 delete mode 160000 subprojects/keycodemapdb
 create mode 100644 subprojects/keycodemapdb.wrap
 delete mode 160000 subprojects/libvfio-user
 create mode 100644 subprojects/libvfio-user.wrap
 create mode 100644 subprojects/packagefiles/berkeley-softfloat-3/meson.build
 create mode 100644 subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt
 create mode 100644 subprojects/packagefiles/berkeley-testfloat-3/meson.build
 create mode 100644 subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt
-- 
2.40.1

[PULL 01/21] meson: Avoid implicit declaration of absent functions

[Paolo Bonzini]

From: Michal Privoznik <mprivozn@redhat.com>

While detecting a presence of a function via 'cc.links()'
gives desired result (i.e. detects whether function is present),
it also produces a warning on systems where the function is not
present (into meson-log.txt), e.g.:

  qemu.git/build/meson-private/tmph74x3p38/testfile.c:2:34: \
  warning: implicit declaration of function 'malloc_trim' [-Wimplicit-function-declaration]

And some distributions (e.g. Gentoo) parse the meson log and
consider these erroneous because it can lead to feature
misdetection (see [1]).

We can check whether given function exists via
'cc.has_function()' or whether STATX_* macros exist via
'cc.has_header_symbol()'.

1: https://wiki.gentoo.org/wiki/Modern_C_porting
Resolves: https://bugs.gentoo.org/898810
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Message-Id: <8e02776d18595a1c575c90a189ff65f1785f76ca.1685442612.git.mprivozn@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 26 +++++---------------------
 1 file changed, 5 insertions(+), 21 deletions(-)

diff --git a/meson.build b/meson.build
index a61d3e9b06a..969a84f2a44 100644
--- a/meson.build
+++ b/meson.build
@@ -1797,8 +1797,7 @@ malloc = []
 if get_option('malloc') == 'system'
   has_malloc_trim = \
     get_option('malloc_trim').allowed() and \
-    cc.links('''#include <malloc.h>
-                int main(void) { malloc_trim(0); return 0; }''')
+    cc.has_function('malloc_trim', prefix: '#include <malloc.h>')
 else
   has_malloc_trim = false
   malloc = cc.find_library(get_option('malloc'), required: true)
@@ -1811,34 +1810,19 @@ if not has_malloc_trim and get_option('malloc_trim').enabled()
   endif
 endif
 
-# Check whether the glibc provides statx()
-
 gnu_source_prefix = '''
   #ifndef _GNU_SOURCE
   #define _GNU_SOURCE
   #endif
 '''
-statx_test = gnu_source_prefix + '''
-  #include <sys/stat.h>
-  int main(void) {
-    struct statx statxbuf;
-    statx(0, "", 0, STATX_BASIC_STATS, &statxbuf);
-    return 0;
-  }'''
 
-has_statx = cc.links(statx_test)
+# Check whether the glibc provides STATX_BASIC_STATS
+
+has_statx = cc.has_header_symbol('sys/stat.h', 'STATX_BASIC_STATS', prefix: gnu_source_prefix)
 
 # Check whether statx() provides mount ID information
 
-statx_mnt_id_test = gnu_source_prefix + '''
-  #include <sys/stat.h>
-  int main(void) {
-    struct statx statxbuf;
-    statx(0, "", 0, STATX_BASIC_STATS | STATX_MNT_ID, &statxbuf);
-    return statxbuf.stx_mnt_id;
-  }'''
-
-has_statx_mnt_id = cc.links(statx_mnt_id_test)
+has_statx_mnt_id = cc.has_header_symbol('sys/stat.h', 'STATX_MNT_ID', prefix: gnu_source_prefix)
 
 have_vhost_user_blk_server = get_option('vhost_user_blk_server') \
   .require(targetos == 'linux',
-- 
2.40.1

[PULL 02/21] atomics: eliminate mb_read/mb_set

[Paolo Bonzini]

qatomic_mb_read and qatomic_mb_set were the very first atomic primitives
introduced for QEMU; their semantics are unclear and they provide a false
sense of safety.

The last use of qatomic_mb_read() has been removed, so delete it.
qatomic_mb_set() instead can survive as an optimized
qatomic_set()+smp_mb(), similar to Linux's smp_store_mb(), but
rename it to qatomic_set_mb() to match the order of the two
operations.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 accel/tcg/cpu-exec.c              |  2 +-
 accel/tcg/tcg-accel-ops-mttcg.c   |  2 +-
 accel/tcg/tcg-accel-ops-rr.c      |  4 ++--
 docs/devel/atomics.rst            | 27 ++++-----------------------
 include/qemu/atomic.h             | 17 +++++------------
 monitor/qmp.c                     |  2 +-
 softmmu/cpus.c                    |  2 +-
 softmmu/physmem.c                 |  2 +-
 target/arm/hvf/hvf.c              |  2 +-
 tests/unit/test-aio-multithread.c |  2 +-
 util/qemu-coroutine-lock.c        |  4 ++--
 11 files changed, 20 insertions(+), 46 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 1cf4f1fa227..42086525d7a 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -774,7 +774,7 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
      * Ensure zeroing happens before reading cpu->exit_request or
      * cpu->interrupt_request (see also smp_wmb in cpu_exit())
      */
-    qatomic_mb_set(&cpu_neg(cpu)->icount_decr.u16.high, 0);
+    qatomic_set_mb(&cpu_neg(cpu)->icount_decr.u16.high, 0);
 
     if (unlikely(qatomic_read(&cpu->interrupt_request))) {
         int interrupt_request;
diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index 5d72c9b1bd3..b320ff00373 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -119,7 +119,7 @@ static void *mttcg_cpu_thread_fn(void *arg)
             }
         }
 
-        qatomic_mb_set(&cpu->exit_request, 0);
+        qatomic_set_mb(&cpu->exit_request, 0);
         qemu_wait_io_event(cpu);
     } while (!cpu->unplug || cpu_can_run(cpu));
 
diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c
index 70b9b89073c..23e4d0f4527 100644
--- a/accel/tcg/tcg-accel-ops-rr.c
+++ b/accel/tcg/tcg-accel-ops-rr.c
@@ -244,7 +244,7 @@ static void *rr_cpu_thread_fn(void *arg)
 
         while (cpu && cpu_work_list_empty(cpu) && !cpu->exit_request) {
             /* Store rr_current_cpu before evaluating cpu_can_run().  */
-            qatomic_mb_set(&rr_current_cpu, cpu);
+            qatomic_set_mb(&rr_current_cpu, cpu);
 
             current_cpu = cpu;
 
@@ -287,7 +287,7 @@ static void *rr_cpu_thread_fn(void *arg)
         qatomic_set(&rr_current_cpu, NULL);
 
         if (cpu && cpu->exit_request) {
-            qatomic_mb_set(&cpu->exit_request, 0);
+            qatomic_set_mb(&cpu->exit_request, 0);
         }
 
         if (icount_enabled() && all_cpu_threads_idle()) {
diff --git a/docs/devel/atomics.rst b/docs/devel/atomics.rst
index 248076375bf..ff9b5ee30c8 100644
--- a/docs/devel/atomics.rst
+++ b/docs/devel/atomics.rst
@@ -102,28 +102,10 @@ Similar operations return the new value of ``*ptr``::
     typeof(*ptr) qatomic_or_fetch(ptr, val)
     typeof(*ptr) qatomic_xor_fetch(ptr, val)
 
-``qemu/atomic.h`` also provides loads and stores that cannot be reordered
-with each other::
+``qemu/atomic.h`` also provides an optimized shortcut for
+``qatomic_set`` followed by ``smp_mb``::
 
-    typeof(*ptr) qatomic_mb_read(ptr)
-    void         qatomic_mb_set(ptr, val)
-
-However these do not provide sequential consistency and, in particular,
-they do not participate in the total ordering enforced by
-sequentially-consistent operations.  For this reason they are deprecated.
-They should instead be replaced with any of the following (ordered from
-easiest to hardest):
-
-- accesses inside a mutex or spinlock
-
-- lightweight synchronization primitives such as ``QemuEvent``
-
-- RCU operations (``qatomic_rcu_read``, ``qatomic_rcu_set``) when publishing
-  or accessing a new version of a data structure
-
-- other atomic accesses: ``qatomic_read`` and ``qatomic_load_acquire`` for
-  loads, ``qatomic_set`` and ``qatomic_store_release`` for stores, ``smp_mb``
-  to forbid reordering subsequent loads before a store.
+    void         qatomic_set_mb(ptr, val)
 
 
 Weak atomic access and manual memory barriers
@@ -523,8 +505,7 @@ and memory barriers, and the equivalents in QEMU:
       | ::                             |
       |                                |
       |   a = qatomic_read(&x);        |
-      |   qatomic_set(&x, a + 2);      |
-      |   smp_mb();                    |
+      |   qatomic_set_mb(&x, a + 2);   |
       |   b = qatomic_read(&y);        |
       +--------------------------------+
 
diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index f85834ee8b2..d95612f7a08 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -259,24 +259,17 @@
 # define smp_mb__after_rmw() smp_mb()
 #endif
 
-/* qatomic_mb_read/set semantics map Java volatile variables. They are
- * less expensive on some platforms (notably POWER) than fully
- * sequentially consistent operations.
- *
- * As long as they are used as paired operations they are safe to
- * use. See docs/devel/atomics.rst for more discussion.
+/*
+ * On some architectures, qatomic_set_mb is more efficient than a store
+ * plus a fence.
  */
 
-#define qatomic_mb_read(ptr)                             \
-    qatomic_load_acquire(ptr)
-
 #if !defined(QEMU_SANITIZE_THREAD) && \
     (defined(__i386__) || defined(__x86_64__) || defined(__s390x__))
-/* This is more efficient than a store plus a fence.  */
-# define qatomic_mb_set(ptr, i) \
+# define qatomic_set_mb(ptr, i) \
     ({ (void)qatomic_xchg(ptr, i); smp_mb__after_rmw(); })
 #else
-# define qatomic_mb_set(ptr, i) \
+# define qatomic_set_mb(ptr, i) \
    ({ qatomic_store_release(ptr, i); smp_mb(); })
 #endif
 
diff --git a/monitor/qmp.c b/monitor/qmp.c
index c8e0156974d..6eee450fe40 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -246,7 +246,7 @@ static QMPRequest *monitor_qmp_dispatcher_pop_any(void)
          *
          * Clear qmp_dispatcher_co_busy before reading request.
          */
-        qatomic_mb_set(&qmp_dispatcher_co_busy, false);
+        qatomic_set_mb(&qmp_dispatcher_co_busy, false);
 
         WITH_QEMU_LOCK_GUARD(&monitor_lock) {
             QMPRequest *req_obj;
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 9cbc8172b5f..fed20ffb5dd 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -405,7 +405,7 @@ static void qemu_cpu_stop(CPUState *cpu, bool exit)
 
 void qemu_wait_io_event_common(CPUState *cpu)
 {
-    qatomic_mb_set(&cpu->thread_kicked, false);
+    qatomic_set_mb(&cpu->thread_kicked, false);
     if (cpu->stop) {
         qemu_cpu_stop(cpu, false);
     }
diff --git a/softmmu/physmem.c b/softmmu/physmem.c
index 9d7e172260f..588d0d166b9 100644
--- a/softmmu/physmem.c
+++ b/softmmu/physmem.c
@@ -3132,7 +3132,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
     bounce.buffer = NULL;
     memory_region_unref(bounce.mr);
     /* Clear in_use before reading map_client_list.  */
-    qatomic_mb_set(&bounce.in_use, false);
+    qatomic_set_mb(&bounce.in_use, false);
     cpu_notify_map_clients();
 }
 
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index ad65603445e..5900dc788f4 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1229,7 +1229,7 @@ static void hvf_wait_for_ipi(CPUState *cpu, struct timespec *ts)
      * Use pselect to sleep so that other threads can IPI us while we're
      * sleeping.
      */
-    qatomic_mb_set(&cpu->thread_kicked, false);
+    qatomic_set_mb(&cpu->thread_kicked, false);
     qemu_mutex_unlock_iothread();
     pselect(0, 0, 0, 0, ts, &cpu->hvf->unblock_ipi_mask);
     qemu_mutex_lock_iothread();
diff --git a/tests/unit/test-aio-multithread.c b/tests/unit/test-aio-multithread.c
index 80c5d4e2e6e..08d4570ccb1 100644
--- a/tests/unit/test-aio-multithread.c
+++ b/tests/unit/test-aio-multithread.c
@@ -154,7 +154,7 @@ static coroutine_fn void test_multi_co_schedule_entry(void *opaque)
         n = g_test_rand_int_range(0, NUM_CONTEXTS);
         schedule_next(n);
 
-        qatomic_mb_set(&to_schedule[id], qemu_coroutine_self());
+        qatomic_set_mb(&to_schedule[id], qemu_coroutine_self());
         /* finish_cb can run here.  */
         qemu_coroutine_yield();
         g_assert(to_schedule[id] == NULL);
diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c
index 84a50a9e911..2534435388f 100644
--- a/util/qemu-coroutine-lock.c
+++ b/util/qemu-coroutine-lock.c
@@ -202,7 +202,7 @@ static void coroutine_fn qemu_co_mutex_lock_slowpath(AioContext *ctx,
     push_waiter(mutex, &w);
 
     /*
-     * Add waiter before reading mutex->handoff.  Pairs with qatomic_mb_set
+     * Add waiter before reading mutex->handoff.  Pairs with qatomic_set_mb
      * in qemu_co_mutex_unlock.
      */
     smp_mb__after_rmw();
@@ -310,7 +310,7 @@ void coroutine_fn qemu_co_mutex_unlock(CoMutex *mutex)
 
         our_handoff = mutex->sequence;
         /* Set handoff before checking for waiters.  */
-        qatomic_mb_set(&mutex->handoff, our_handoff);
+        qatomic_set_mb(&mutex->handoff, our_handoff);
         if (!has_waiters(mutex)) {
             /* The concurrent lock has not added itself yet, so it
              * will be able to pick our handoff.
-- 
2.40.1

[PULL 03/21] scripts: remove dead file

[Paolo Bonzini]

scripts/test-driver.py was used when "make check" was already using meson
introspection data, but it did not execute "meson test".  It is dead since
commit 3d2f73ef75e ("build: use "meson test" as the test harness", 2021-12-23).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/test-driver.py | 35 -----------------------------------
 1 file changed, 35 deletions(-)
 delete mode 100644 scripts/test-driver.py

diff --git a/scripts/test-driver.py b/scripts/test-driver.py
deleted file mode 100644
index eef74b29a8f..00000000000
--- a/scripts/test-driver.py
+++ /dev/null
@@ -1,35 +0,0 @@
-#! /usr/bin/env python3
-
-# Wrapper for tests that hides the output if they succeed.
-# Used by "make check"
-#
-# Copyright (C) 2020 Red Hat, Inc.
-#
-# Author: Paolo Bonzini <pbonzini@redhat.com>
-
-import subprocess
-import sys
-import os
-import argparse
-
-parser = argparse.ArgumentParser(description='Test driver for QEMU')
-parser.add_argument('-C', metavar='DIR', dest='dir', default='.',
-                    help='change to DIR before doing anything else')
-parser.add_argument('-v', '--verbose', dest='verbose', action='store_true',
-                    help='be more verbose')
-parser.add_argument('test_args', nargs=argparse.REMAINDER)
-
-args = parser.parse_args()
-os.chdir(args.dir)
-
-test_args = args.test_args
-if test_args[0] == '--':
-    test_args = test_args[1:]
-
-if args.verbose:
-    result = subprocess.run(test_args, stdout=None, stderr=None)
-else:
-    result = subprocess.run(test_args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
-    if result.returncode:
-        sys.stdout.buffer.write(result.stdout)
-sys.exit(result.returncode)
-- 
2.40.1

[PULL 04/21] meson.build: Group the UI entries in a separate summary section

[Paolo Bonzini]

From: Thomas Huth <thuth@redhat.com>

Let's make it easier for the users to spot UI-related entries in
the summary of the meson output.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230602171832.533739-2-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 38 ++++++++++++++++++++++----------------
 1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/meson.build b/meson.build
index 969a84f2a44..cfacd662edb 100644
--- a/meson.build
+++ b/meson.build
@@ -4227,32 +4227,43 @@ summary_info += {'rng-none':          get_option('rng_none')}
 summary_info += {'Linux keyring':     have_keyring}
 summary(summary_info, bool_yn: true, section: 'Crypto')
 
-# Libraries
+# UI
 summary_info = {}
 if targetos == 'darwin'
   summary_info += {'Cocoa support':           cocoa}
-  summary_info += {'vmnet.framework support': vmnet}
 endif
 summary_info += {'SDL support':       sdl}
 summary_info += {'SDL image support': sdl_image}
 summary_info += {'GTK support':       gtk}
 summary_info += {'pixman':            pixman}
 summary_info += {'VTE support':       vte}
-summary_info += {'slirp support':     slirp}
-summary_info += {'libtasn1':          tasn1}
-summary_info += {'PAM':               pam}
-summary_info += {'iconv support':     iconv}
-summary_info += {'curses support':    curses}
-summary_info += {'virgl support':     virgl}
-summary_info += {'blkio support':     blkio}
-summary_info += {'curl support':      curl}
-summary_info += {'Multipath support': mpathpersist}
 summary_info += {'PNG support':       png}
 summary_info += {'VNC support':       vnc}
 if vnc.found()
   summary_info += {'VNC SASL support':  sasl}
   summary_info += {'VNC JPEG support':  jpeg}
 endif
+summary_info += {'spice protocol support': spice_protocol}
+if spice_protocol.found()
+  summary_info += {'  spice server support': spice}
+endif
+summary_info += {'curses support':    curses}
+summary_info += {'brlapi support':    brlapi}
+summary(summary_info, bool_yn: true, section: 'User interface')
+
+# Libraries
+summary_info = {}
+if targetos == 'darwin'
+  summary_info += {'vmnet.framework support': vmnet}
+endif
+summary_info += {'slirp support':     slirp}
+summary_info += {'libtasn1':          tasn1}
+summary_info += {'PAM':               pam}
+summary_info += {'iconv support':     iconv}
+summary_info += {'virgl support':     virgl}
+summary_info += {'blkio support':     blkio}
+summary_info += {'curl support':      curl}
+summary_info += {'Multipath support': mpathpersist}
 if targetos not in ['darwin', 'haiku', 'windows']
   summary_info += {'OSS support':     oss}
   summary_info += {'sndio support':   sndio}
@@ -4267,7 +4278,6 @@ if targetos == 'linux'
 endif
 summary_info += {'Pipewire support':   pipewire}
 summary_info += {'JACK support':      jack}
-summary_info += {'brlapi support':    brlapi}
 summary_info += {'vde support':       vde}
 summary_info += {'netmap support':    have_netmap}
 summary_info += {'l2tpv3 support':    have_l2tpv3}
@@ -4279,10 +4289,6 @@ summary_info += {'PVRDMA support':    have_pvrdma}
 summary_info += {'fdt support':       fdt_opt == 'disabled' ? false : fdt_opt}
 summary_info += {'libcap-ng support': libcap_ng}
 summary_info += {'bpf support':       libbpf}
-summary_info += {'spice protocol support': spice_protocol}
-if spice_protocol.found()
-  summary_info += {'  spice server support': spice}
-endif
 summary_info += {'rbd support':       rbd}
 summary_info += {'smartcard support': cacard}
 summary_info += {'U2F support':       u2f}
-- 
2.40.1

[PULL 05/21] meson.build: Group the network backend entries in a separate summary section

[Paolo Bonzini]

From: Thomas Huth <thuth@redhat.com>

Let's make it easier for the users to spot network-related entries
in the summary of the meson output.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230602171832.533739-3-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/meson.build b/meson.build
index cfacd662edb..b0adf5b4617 100644
--- a/meson.build
+++ b/meson.build
@@ -4251,12 +4251,19 @@ summary_info += {'curses support':    curses}
 summary_info += {'brlapi support':    brlapi}
 summary(summary_info, bool_yn: true, section: 'User interface')
 
-# Libraries
+# Network backends
 summary_info = {}
 if targetos == 'darwin'
   summary_info += {'vmnet.framework support': vmnet}
 endif
 summary_info += {'slirp support':     slirp}
+summary_info += {'vde support':       vde}
+summary_info += {'netmap support':    have_netmap}
+summary_info += {'l2tpv3 support':    have_l2tpv3}
+summary(summary_info, bool_yn: true, section: 'Network backends')
+
+# Libraries
+summary_info = {}
 summary_info += {'libtasn1':          tasn1}
 summary_info += {'PAM':               pam}
 summary_info += {'iconv support':     iconv}
@@ -4278,9 +4285,6 @@ if targetos == 'linux'
 endif
 summary_info += {'Pipewire support':   pipewire}
 summary_info += {'JACK support':      jack}
-summary_info += {'vde support':       vde}
-summary_info += {'netmap support':    have_netmap}
-summary_info += {'l2tpv3 support':    have_l2tpv3}
 summary_info += {'Linux AIO support': libaio}
 summary_info += {'Linux io_uring support': linux_io_uring}
 summary_info += {'ATTR/XATTR support': libattr}
-- 
2.40.1

[PULL 06/21] meson.build: Group the audio backend entries in a separate summary section

[Paolo Bonzini]

From: Thomas Huth <thuth@redhat.com>

Let's make it easier for the users to spot audio-related entries
in the summary of the meson output.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230602171832.533739-4-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/meson.build b/meson.build
index b0adf5b4617..97b32e9beb5 100644
--- a/meson.build
+++ b/meson.build
@@ -4251,6 +4251,24 @@ summary_info += {'curses support':    curses}
 summary_info += {'brlapi support':    brlapi}
 summary(summary_info, bool_yn: true, section: 'User interface')
 
+# Audio backends
+summary_info = {}
+if targetos not in ['darwin', 'haiku', 'windows']
+  summary_info += {'OSS support':     oss}
+  summary_info += {'sndio support':   sndio}
+elif targetos == 'darwin'
+  summary_info += {'CoreAudio support': coreaudio}
+elif targetos == 'windows'
+  summary_info += {'DirectSound support': dsound}
+endif
+if targetos == 'linux'
+  summary_info += {'ALSA support':    alsa}
+  summary_info += {'PulseAudio support': pulse}
+endif
+summary_info += {'Pipewire support':  pipewire}
+summary_info += {'JACK support':      jack}
+summary(summary_info, bool_yn: true, section: 'Audio backends')
+
 # Network backends
 summary_info = {}
 if targetos == 'darwin'
@@ -4271,20 +4289,6 @@ summary_info += {'virgl support':     virgl}
 summary_info += {'blkio support':     blkio}
 summary_info += {'curl support':      curl}
 summary_info += {'Multipath support': mpathpersist}
-if targetos not in ['darwin', 'haiku', 'windows']
-  summary_info += {'OSS support':     oss}
-  summary_info += {'sndio support':   sndio}
-elif targetos == 'darwin'
-  summary_info += {'CoreAudio support': coreaudio}
-elif targetos == 'windows'
-  summary_info += {'DirectSound support': dsound}
-endif
-if targetos == 'linux'
-  summary_info += {'ALSA support':    alsa}
-  summary_info += {'PulseAudio support': pulse}
-endif
-summary_info += {'Pipewire support':   pipewire}
-summary_info += {'JACK support':      jack}
 summary_info += {'Linux AIO support': libaio}
 summary_info += {'Linux io_uring support': linux_io_uring}
 summary_info += {'ATTR/XATTR support': libattr}
-- 
2.40.1

[PULL 07/21] meson.build: Use -Wno-undef only for SDL2 versions that need it

[Paolo Bonzini]

From: Thomas Huth <thuth@redhat.com>

There is no need to disable this useful compiler warning for
all versions of the SDL. Unfortunately, various versions are
buggy (beside SDL 2.0.8, the version 2.26.0 and 2.26.1 are
broken, too, see https://github.com/libsdl-org/SDL/issues/6619 ),
but we can use a simple compiler check to see whether we need
the -Wno-undef or not.

This also enables the printing of the version number with
good versions of the SDL in the summary of the meson output
again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230605114523.282987-1-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meson.build b/meson.build
index 97b32e9beb5..6bbeffe5711 100644
--- a/meson.build
+++ b/meson.build
@@ -1273,10 +1273,16 @@ if not get_option('sdl').auto() or have_system
   sdl_image = not_found
 endif
 if sdl.found()
-  # work around 2.0.8 bug
-  sdl = declare_dependency(compile_args: '-Wno-undef',
-                           dependencies: sdl,
-                           version: sdl.version())
+  # Some versions of SDL have problems with -Wundef
+  if not cc.compiles('''
+                     #include <SDL.h>
+                     #include <SDL_syswm.h>
+                     int main(int argc, char *argv[]) { return 0; }
+                     ''', dependencies: sdl, args: '-Werror=undef')
+    sdl = declare_dependency(compile_args: '-Wno-undef',
+                             dependencies: sdl,
+                             version: sdl.version())
+  endif
   sdl_image = dependency('SDL2_image', required: get_option('sdl_image'),
                          method: 'pkg-config')
 else
-- 
2.40.1

[PULL 08/21] scsi/qemu-pr-helper: Drop support for 'old' libmultipath API

[Paolo Bonzini]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

Commit 1b0578f5c4 ("qemu-pr-helper: Fix build on CentOS 7") added
code to probe for 'old' libmultipath API on CentOS 7. However since
merge commit 8c345b3e6a (June 2021) we don't build/test CentOS 7 as
it felt out of our list of supported distrib versions. Therefore we
can safely remove the 'old' API check (mostly reverting commit
1b0578f5c4, except the code got converted to meson in commit
6ec0e15d95 "meson: move libmpathpersist test").

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230605174146.87440-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 meson.build           | 19 ++-----------------
 scsi/qemu-pr-helper.c |  4 ----
 2 files changed, 2 insertions(+), 21 deletions(-)

diff --git a/meson.build b/meson.build
index 6bbeffe5711..6767a0d3faa 100644
--- a/meson.build
+++ b/meson.build
@@ -1092,9 +1092,8 @@ endif
 
 mpathlibs = [libudev]
 mpathpersist = not_found
-mpathpersist_new_api = false
 if targetos == 'linux' and have_tools and get_option('mpath').allowed()
-  mpath_test_source_new = '''
+  mpath_test_source = '''
     #include <libudev.h>
     #include <mpath_persist.h>
     unsigned mpath_mx_alloc_len = 1024;
@@ -1111,16 +1110,6 @@ if targetos == 'linux' and have_tools and get_option('mpath').allowed()
         multipath_conf = mpath_lib_init();
         return 0;
     }'''
-  mpath_test_source_old = '''
-      #include <libudev.h>
-      #include <mpath_persist.h>
-      unsigned mpath_mx_alloc_len = 1024;
-      int logsink;
-      int main(void) {
-          struct udev *udev = udev_new();
-          mpath_lib_init(udev);
-          return 0;
-      }'''
   libmpathpersist = cc.find_library('mpathpersist',
                                     required: get_option('mpath'))
   if libmpathpersist.found()
@@ -1139,10 +1128,7 @@ if targetos == 'linux' and have_tools and get_option('mpath').allowed()
     endforeach
     if mpathlibs.length() == 0
       msg = 'Dependencies missing for libmpathpersist'
-    elif cc.links(mpath_test_source_new, dependencies: mpathlibs)
-      mpathpersist = declare_dependency(dependencies: mpathlibs)
-      mpathpersist_new_api = true
-    elif cc.links(mpath_test_source_old, dependencies: mpathlibs)
+    elif cc.links(mpath_test_source, dependencies: mpathlibs)
       mpathpersist = declare_dependency(dependencies: mpathlibs)
     else
       msg = 'Cannot detect libmpathpersist API'
@@ -2094,7 +2080,6 @@ config_host_data.set('CONFIG_GCOV', get_option('b_coverage'))
 config_host_data.set('CONFIG_LIBUDEV', libudev.found())
 config_host_data.set('CONFIG_LZO', lzo.found())
 config_host_data.set('CONFIG_MPATH', mpathpersist.found())
-config_host_data.set('CONFIG_MPATH_NEW_API', mpathpersist_new_api)
 config_host_data.set('CONFIG_BLKIO', blkio.found())
 if blkio.found()
   config_host_data.set('CONFIG_BLKIO_VHOST_VDPA_FD',
diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c
index a857e80c038..ae44a816e17 100644
--- a/scsi/qemu-pr-helper.c
+++ b/scsi/qemu-pr-helper.c
@@ -280,11 +280,7 @@ void put_multipath_config(struct config *conf)
 static void multipath_pr_init(void)
 {
     udev = udev_new();
-#ifdef CONFIG_MPATH_NEW_API
     multipath_conf = mpath_lib_init();
-#else
-    mpath_lib_init(udev);
-#endif
 }
 
 static int is_mpath(int fd)
-- 
2.40.1

[PULL 09/21] Revert "tests/requirements.txt: bump up avocado-framework version to 101.0"

[Paolo Bonzini]

This reverts commit ec5ffa0056389c3c10ea2de1e78366f66f4e5abc.

Bumping avocado to version 101 has two issues.  First, there are problems
where Avocado is not logging of command lines or terminal output, and not
collecting Python logs outside the avocado namespace.

Second, the recent changes to Python handling mean that there is a single
virtual environment for all the build, instead of a separate one for testing.
Requiring a too-new version of avocado causes conflicts with any avocado
plugins installed on the host:

   $ make check-venv
   make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
     GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
     VENVPIP install -e /home/berrange/src/virt/qemu/python/
     VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt
   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

To avoid this issue, tests/requirements.txt should use a ">=" constraint
and the version of Avocado should be limited to what distros provide
in the system packages.  Only Fedora has Avocado, and more specifically
version 92.0 (though 98.0 is also available as a module).  As a first
step, this patch reverts the introduction of a too-new Avocado.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/Makefile.include | 18 +++++++-----------
 tests/requirements.txt |  2 +-
 2 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 0184ef22373..8294a44816c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -136,18 +136,14 @@ get-vm-image-fedora-31-%: check-venv
 # download all vm images, according to defined targets
 get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
 
-JOBS_OPTION=$(lastword -j1 $(filter-out -j, $(filter -j%,$(MAKEFLAGS))))
-
 check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
-	$(call quiet-command, 							\
-            $(PYTHON) -m avocado 						\
-            --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) 	\
-            $(if $(AVOCADO_TAGS),, 						\
-			--filter-by-tags-include-empty 				\
-			--filter-by-tags-include-empty-key) 			\
-		--max-parallel-tasks $(JOBS_OPTION:-j%=%) 			\
-            $(AVOCADO_CMDLINE_TAGS) 						\
-            $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), 			\
+	$(call quiet-command, \
+            $(PYTHON) -m avocado \
+            --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
+            $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
+			--filter-by-tags-include-empty-key) \
+            $(AVOCADO_CMDLINE_TAGS) \
+            $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
             "AVOCADO", "tests/avocado")
 
 check-acceptance-deprecated-warning:
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 0e008b9aec3..07e713ef5ac 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -5,5 +5,5 @@
 # Note that qemu.git/python/ is implicitly installed to this venv when
 # 'make check-venv' is run, and will persist until configure is run
 # again.
-avocado-framework==101.0
+avocado-framework==88.1
 pycdlib==1.11.0
-- 
2.40.1

[PULL 10/21] tests: Use separate virtual environment for avocado

[Paolo Bonzini]

This reverts commits eea2d141179 ("Makefile: remove $(TESTS_PYTHON)",
2023-05-26) and 9c6692db550 ("tests: Use configure-provided pyvenv for
tests", 2023-05-18).

Right now, there is a conflict between wanting a ">=" constraint when
using a distro-provided package and wanting a "==" constraint when
installing Avocado from PyPI; this would provide the best of both worlds
in terms of resiliency for both distros that have required packages and
distros that don't.

The conflict is visible also for meson, where we would like to install
the latest 0.63.x version but also accept a distro 1.1.x version.
But it is worse for avocado, for two reasons:

1) we cannot use an "==" constraint to install avocado if the venv
includes a system avocado.  The distro will package plugins that have
"==" constraints on the version that is included in the distro, and, using
"pip install avocado==88.1" on a venv that includes system packages will
result in this error:

   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

2) we cannot use ">=" either if the venv does _not_ include a system
avocado, because that would result in the installation of v101.0 which
is the one we've just reverted.

So the idea is to encode the dependencies as an (acceptable, locked)
tuple, like this hypothetical TOML that would be committed inside
python/ and used by mkvenv.py:

  [meson]
  meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" }

  [docs]
  # 6.0 drops support for Python 3.7
  sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" }
  sphinx_rtd_theme = { minimum = "0.5" }

  [avocado]
  avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" }

Once this is implemented, it would also be possible to install avocado in
pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora
and CentOS Stream (the only distros where it's available).  But until
this is implemented, keep avocado in a separate venv.  There is still the
benefit of using a single python for meson custom_targets and for sphinx.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/buildtest.yml                       |  6 +++---
 docs/devel/acpi-bits.rst                         |  6 +++---
 docs/devel/testing.rst                           | 14 +++++++-------
 .../ci/org.centos/stream/8/x86_64/test-avocado   |  4 ++--
 scripts/device-crash-test                        |  2 +-
 tests/Makefile.include                           | 16 +++++++++-------
 tests/requirements.txt                           |  7 ++-----
 tests/vm/Makefile.include                        |  2 +-
 8 files changed, 28 insertions(+), 29 deletions(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 0f1be14cb62..1922caf5363 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -103,7 +103,7 @@ crash-test-debian:
   script:
     - cd build
     - make NINJA=":" check-venv
-    - pyvenv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
+    - tests/venv/bin/python3 scripts/device-crash-test -q --tcg-only ./qemu-system-i386
 
 build-system-fedora:
   extends:
@@ -146,8 +146,8 @@ crash-test-fedora:
   script:
     - cd build
     - make NINJA=":" check-venv
-    - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
-    - pyvenv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
+    - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-ppc
+    - tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
 
 build-system-centos:
   extends:
diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst
index 9677b0098f4..22e2580200c 100644
--- a/docs/devel/acpi-bits.rst
+++ b/docs/devel/acpi-bits.rst
@@ -61,19 +61,19 @@ Under ``tests/avocado/`` as the root we have:
    ::
 
      $ make check-venv (needed only the first time to create the venv)
-     $ ./pyvenv/bin/avocado run -t acpi tests/avocado
+     $ ./tests/venv/bin/avocado run -t acpi tests/avocado
 
    The above will run all acpi avocado tests including this one.
    In order to run the individual tests, perform the following:
    ::
 
-     $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
+     $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py --tap -
 
    The above will produce output in tap format. You can omit "--tap -" in the
    end and it will produce output like the following:
    ::
 
-      $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
+      $ ./tests/venv/bin/avocado run tests/avocado/acpi-bits.py
       Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
       JOB ID     : eab225724da7b64c012c65705dc2fa14ab1defef
       JOB LOG    : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 2cafec41782..203facb417b 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -888,9 +888,9 @@ You can run the avocado tests simply by executing:
 
   make check-avocado
 
-This involves the automatic installation, from PyPI, of all the
-necessary avocado-framework dependencies into the QEMU venv within the
-build tree (at ``./pyvenv``). Test results are also saved within the
+This involves the automatic creation of Python virtual environment
+within the build tree (at ``tests/venv``) which will have all the
+right dependencies, and will save tests results also within the
 build tree (at ``tests/results``).
 
 Note: the build environment must be using a Python 3 stack, and have
@@ -947,7 +947,7 @@ may be invoked by running:
 
  .. code::
 
-  pyvenv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
+  tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
 
 Note that if ``make check-avocado`` was not executed before, it is
 possible to create the Python virtual environment with the dependencies
@@ -962,20 +962,20 @@ a test file. To run tests from a single file within the build tree, use:
 
  .. code::
 
-  pyvenv/bin/avocado run tests/avocado/$TESTFILE
+  tests/venv/bin/avocado run tests/avocado/$TESTFILE
 
 To run a single test within a test file, use:
 
  .. code::
 
-  pyvenv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
+  tests/venv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
 
 Valid test names are visible in the output from any previous execution
 of Avocado or ``make check-avocado``, and can also be queried using:
 
  .. code::
 
-  pyvenv/bin/avocado list tests/avocado
+  tests/venv/bin/avocado list tests/avocado
 
 Manual Installation
 ~~~~~~~~~~~~~~~~~~~
diff --git a/scripts/ci/org.centos/stream/8/x86_64/test-avocado b/scripts/ci/org.centos/stream/8/x86_64/test-avocado
index 73e7a1a3126..e0443fc8ae8 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/test-avocado
+++ b/scripts/ci/org.centos/stream/8/x86_64/test-avocado
@@ -4,7 +4,7 @@
 # KVM and x86_64, or tests that are generic enough to be valid for all
 # targets. Such a test list can be generated with:
 #
-# ./pyvenv/bin/avocado list --filter-by-tags-include-empty \
+# ./tests/venv/bin/avocado list --filter-by-tags-include-empty \
 #   --filter-by-tags-include-empty-key -t accel:kvm,arch:x86_64 \
 #   tests/avocado/
 #
@@ -22,7 +22,7 @@
 #   - tests/avocado/virtio_check_params.py:VirtioMaxSegSettingsCheck.test_machine_types
 #
 make get-vm-images
-./pyvenv/bin/avocado run \
+./tests/venv/bin/avocado run \
     --job-results-dir=tests/results/ \
     tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_kvm \
     tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm \
diff --git a/scripts/device-crash-test b/scripts/device-crash-test
index 353aa575d7b..b74d887331d 100755
--- a/scripts/device-crash-test
+++ b/scripts/device-crash-test
@@ -43,7 +43,7 @@ except ModuleNotFoundError as exc:
     print(f"Module '{exc.name}' not found.")
     print("  Try 'make check-venv' from your build directory,")
     print("  and then one way to run this script is like so:")
-    print(f'  > $builddir/pyvenv/bin/python3 "{path}"')
+    print(f'  > $builddir/tests/venv/bin/python3 "{path}"')
     sys.exit(1)
 
 logger = logging.getLogger('device-crash-test')
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8294a44816c..9422ddaece5 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -89,9 +89,10 @@ distclean-tcg: $(DISTCLEAN_TCG_TARGET_RULES)
 # Build up our target list from the filtered list of ninja targets
 TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 
-TESTS_VENV_TOKEN=$(BUILD_DIR)/pyvenv/tests.group
+TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
 TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
 TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
+TESTS_PYTHON=$(TESTS_VENV_DIR)/bin/python3
 ifndef AVOCADO_TESTS
 	AVOCADO_TESTS=tests/avocado
 endif
@@ -107,10 +108,11 @@ else
 endif
 
 quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
-    $(PYTHON) -m pip -q --disable-pip-version-check $1, \
+    $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \
     "VENVPIP","$1")
 
-$(TESTS_VENV_TOKEN): $(TESTS_VENV_REQ)
+$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
+	$(call quiet-command, $(PYTHON) -m venv $@, VENV, $@)
 	$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
 	$(call quiet-venv-pip,install -r $(TESTS_VENV_REQ))
 	$(call quiet-command, touch $@)
@@ -119,7 +121,7 @@ $(TESTS_RESULTS_DIR):
 	$(call quiet-command, mkdir -p $@, \
             MKDIR, $@)
 
-check-venv: $(TESTS_VENV_TOKEN)
+check-venv: $(TESTS_VENV_DIR)
 
 FEDORA_31_ARCHES_TARGETS=$(patsubst %-softmmu,%, $(filter %-softmmu,$(TARGETS)))
 FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_TARGETS))
@@ -129,7 +131,7 @@ FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
 # download one specific Fedora 31 image
 get-vm-image-fedora-31-%: check-venv
 	$(call quiet-command, \
-             $(PYTHON) -m avocado vmimage get \
+             $(TESTS_PYTHON) -m avocado vmimage get \
              --distro=fedora --distro-version=31 --arch=$*, \
 	"AVOCADO", "Downloading avocado tests VM image for $*")
 
@@ -138,7 +140,7 @@ get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOW
 
 check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
 	$(call quiet-command, \
-            $(PYTHON) -m avocado \
+            $(TESTS_PYTHON) -m avocado \
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
             $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
 			--filter-by-tags-include-empty-key) \
@@ -161,7 +163,7 @@ check:
 check-build: run-ninja
 
 check-clean:
-	rm -rf $(TESTS_RESULTS_DIR)
+	rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR)
 
 clean: check-clean clean-tcg
 distclean: distclean-tcg
diff --git a/tests/requirements.txt b/tests/requirements.txt
index 07e713ef5ac..0ba561b6bdf 100644
--- a/tests/requirements.txt
+++ b/tests/requirements.txt
@@ -1,9 +1,6 @@
 # Add Python module requirements, one per line, to be installed
-# in the qemu build_dir/pyvenv Python virtual environment. For more info,
+# in the tests/venv Python virtual environment. For more info,
 # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-#
-# Note that qemu.git/python/ is implicitly installed to this venv when
-# 'make check-venv' is run, and will persist until configure is run
-# again.
+# Note that qemu.git/python/ is always implicitly installed.
 avocado-framework==88.1
 pycdlib==1.11.0
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index f0f5d32fb0f..c2a8ca1c175 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -5,7 +5,7 @@ ifeq ($(realpath $(SRC_PATH)),$(realpath .))
 VM_PYTHON = PYTHONPATH=$(SRC_PATH)/python /usr/bin/env python3
 VM_VENV =
 else
-VM_PYTHON = $(PYTHON)
+VM_PYTHON = $(TESTS_PYTHON)
 VM_VENV = check-venv
 endif
 
-- 
2.40.1

[PULL 11/21] mkvenv: always pass locally-installed packages to pip

[Paolo Bonzini]

Let pip decide whether a new version should be installed or the current
one is okay.  This ensures that the virtual environment is updated
(either upgraded or downgraded) whenever a new version of a package is
requested.

The hardest part here is figuring out if a package is installed in
the venv (which also has to be done twice to account for the presence
of either setuptools in Python <3.8, or importlib in Python >=3.8).

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Cc: John Snow <jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 python/scripts/mkvenv.py | 76 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 74 insertions(+), 2 deletions(-)

diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py
index 3a9aef46a51..a47f1eaf5d6 100644
--- a/python/scripts/mkvenv.py
+++ b/python/scripts/mkvenv.py
@@ -553,6 +553,74 @@ def pkgname_from_depspec(dep_spec: str) -> str:
     return match.group(0)
 
 
+def _get_path_importlib(package: str) -> Optional[str]:
+    # pylint: disable=import-outside-toplevel
+    # pylint: disable=no-name-in-module
+    # pylint: disable=import-error
+    try:
+        # First preference: Python 3.8+ stdlib
+        from importlib.metadata import (  # type: ignore
+            PackageNotFoundError,
+            distribution,
+        )
+    except ImportError as exc:
+        logger.debug("%s", str(exc))
+        # Second preference: Commonly available PyPI backport
+        from importlib_metadata import (  # type: ignore
+            PackageNotFoundError,
+            distribution,
+        )
+
+    try:
+        return str(distribution(package).locate_file("."))
+    except PackageNotFoundError:
+        return None
+
+
+def _get_path_pkg_resources(package: str) -> Optional[str]:
+    # pylint: disable=import-outside-toplevel
+    # Bundled with setuptools; has a good chance of being available.
+    import pkg_resources
+
+    try:
+        return str(pkg_resources.get_distribution(package).location)
+    except pkg_resources.DistributionNotFound:
+        return None
+
+
+def _get_path(package: str) -> Optional[str]:
+    try:
+        return _get_path_importlib(package)
+    except ImportError as exc:
+        logger.debug("%s", str(exc))
+
+    try:
+        return _get_path_pkg_resources(package)
+    except ImportError as exc:
+        logger.debug("%s", str(exc))
+        raise Ouch(
+            "Neither importlib.metadata nor pkg_resources found. "
+            "Use Python 3.8+, or install importlib-metadata or setuptools."
+        ) from exc
+
+
+def _path_is_prefix(prefix: Optional[str], path: str) -> bool:
+    try:
+        return (
+            prefix is not None and os.path.commonpath([prefix, path]) == prefix
+        )
+    except ValueError:
+        return False
+
+
+def _is_system_package(package: str) -> bool:
+    path = _get_path(package)
+    return path is not None and not (
+        _path_is_prefix(sysconfig.get_path("purelib"), path)
+        or _path_is_prefix(sysconfig.get_path("platlib"), path)
+    )
+
+
 def _get_version_importlib(package: str) -> Optional[str]:
     # pylint: disable=import-outside-toplevel
     # pylint: disable=no-name-in-module
@@ -741,8 +809,12 @@ def _do_ensure(
     for spec in dep_specs:
         matcher = distlib.version.LegacyMatcher(spec)
         ver = _get_version(matcher.name)
-        if ver is None or not matcher.match(
-            distlib.version.LegacyVersion(ver)
+        if (
+            ver is None
+            # Always pass installed package to pip, so that they can be
+            # updated if the requested version changes
+            or not _is_system_package(matcher.name)
+            or not matcher.match(distlib.version.LegacyVersion(ver))
         ):
             absent.append(spec)
         else:
-- 
2.40.1

[PULL 12/21] configure: remove --with-git= option

[Paolo Bonzini]

The scenario for which --with-git= was introduced was to use a SOCKS proxy
such as tsocks.  However, this was back in 2017 when QEMU's submodules
used the git:// protocol, and it is not as important when using the
"smart HTTP" backend; for example, neither "meson subprojects download"
nor scripts/checkpatch.pl obey the GIT environment variable.

So remove the knob, but test for the presence of git in the configure and
git-submodule.sh scripts, and suggest using --with-git-submodules=validate
+ a manual invocation of git-submodule.sh when git does not work.  Hopefully
in the future the GIT environment variable will be supported by Meson.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile                                      |  2 +-
 configure                                     |  8 ++------
 meson.build                                   |  1 -
 .../ci/org.centos/stream/8/x86_64/configure   |  2 --
 scripts/git-submodule.sh                      | 19 ++++++++++---------
 5 files changed, 13 insertions(+), 19 deletions(-)

diff --git a/Makefile b/Makefile
index 08fb6a3b058..8005f1cc53e 100644
--- a/Makefile
+++ b/Makefile
@@ -53,7 +53,7 @@ Makefile: .git-submodule-status
 git-submodule-update:
 ifneq ($(GIT_SUBMODULES_ACTION),ignore)
 	$(call quiet-command, \
-		(GIT="$(GIT)" "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)), \
+		(GIT=git "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)), \
 		"GIT","$(GIT_SUBMODULES)")
 endif
 
diff --git a/configure b/configure
index d674a966731..92bacc756c0 100755
--- a/configure
+++ b/configure
@@ -751,8 +751,6 @@ for opt do
   ;;
   --enable-fdt=*) fdt="$optarg"
   ;;
-  --with-git=*) git="$optarg"
-  ;;
   --with-git-submodules=*)
       git_submodules_action="$optarg"
   ;;
@@ -791,7 +789,7 @@ fi
 
 case $git_submodules_action in
     update|validate)
-        if test ! -e "$source_path/.git"; then
+        if test ! -e "$source_path/.git" || ! has git; then
             echo "ERROR: cannot $git_submodules_action git submodules without .git"
             exit 1
         fi
@@ -892,7 +890,6 @@ Advanced options (experts only):
   --python=PYTHON          use specified python [$python]
   --ninja=NINJA            use specified ninja [$ninja]
   --smbd=SMBD              use specified smbd [$smbd]
-  --with-git=GIT           use specified git [$git]
   --with-git-submodules=update   update git submodules (default if .git dir exists)
   --with-git-submodules=validate fail if git submodules are not up to date
   --with-git-submodules=ignore   do not update or check git submodules (default if no .git dir)
@@ -1699,7 +1696,7 @@ fi
 #######################################
 # generate config-host.mak
 
-if ! (GIT="$git" "$source_path/scripts/git-submodule.sh" "$git_submodules_action" "$git_submodules"); then
+if ! (GIT=git "$source_path/scripts/git-submodule.sh" "$git_submodules_action" "$git_submodules"); then
     exit 1
 fi
 
@@ -1709,7 +1706,6 @@ echo "# Automatically generated by configure - do not modify" > $config_host_mak
 echo >> $config_host_mak
 
 echo all: >> $config_host_mak
-echo "GIT=$git" >> $config_host_mak
 echo "GIT_SUBMODULES=$git_submodules" >> $config_host_mak
 echo "GIT_SUBMODULES_ACTION=$git_submodules_action" >> $config_host_mak
 
diff --git a/meson.build b/meson.build
index 6767a0d3faa..4302c019237 100644
--- a/meson.build
+++ b/meson.build
@@ -4011,7 +4011,6 @@ summary(summary_info, bool_yn: true, section: 'Directories')
 
 # Host binaries
 summary_info = {}
-summary_info += {'git':               config_host['GIT']}
 summary_info += {'python':            '@0@ (version: @1@)'.format(python.full_path(), python.language_version())}
 summary_info += {'sphinx-build':      sphinx_build}
 if config_host.has_key('HAVE_GDB_BIN')
diff --git a/scripts/ci/org.centos/stream/8/x86_64/configure b/scripts/ci/org.centos/stream/8/x86_64/configure
index 6e8983f39cd..de76510978f 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/configure
+++ b/scripts/ci/org.centos/stream/8/x86_64/configure
@@ -29,14 +29,12 @@
 --extra-cflags="-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" \
 --with-suffix="qemu-kvm" \
 --firmwarepath=/usr/share/qemu-firmware \
---with-git=meson \
 --with-git-submodules=update \
 --target-list="x86_64-softmmu" \
 --block-drv-rw-whitelist="qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle,gluster" \
 --audio-drv-list="" \
 --block-drv-ro-whitelist="vmdk,vhdx,vpc,https,ssh" \
 --with-coroutine=ucontext \
---with-git=git \
 --tls-priority=@QEMU,SYSTEM \
 --disable-attr \
 --disable-auth-pam \
diff --git a/scripts/git-submodule.sh b/scripts/git-submodule.sh
index 7be41f59483..0ce1efc44e5 100755
--- a/scripts/git-submodule.sh
+++ b/scripts/git-submodule.sh
@@ -12,7 +12,7 @@ maybe_modules="$@"
 # if --with-git-submodules=ignore, do nothing
 test "$command" = "ignore" && exit 0
 
-test -z "$GIT" && GIT=git
+test -z "$GIT" && GIT=$(command -v git)
 
 cd "$(dirname "$0")/.."
 
@@ -21,19 +21,14 @@ update_error() {
     echo
     echo "Unable to automatically checkout GIT submodules '$modules'."
     echo "If you require use of an alternative GIT binary (for example to"
-    echo "enable use of a transparent proxy), then please specify it by"
-    echo "running configure by with the '--with-git' argument. e.g."
-    echo
-    echo " $ ./configure --with-git='tsocks git'"
-    echo
-    echo "Alternatively you may disable automatic GIT submodule checkout"
-    echo "with:"
+    echo "enable use of a transparent proxy), please disable automatic"
+    echo "GIT submodule checkout with:"
     echo
     echo " $ ./configure --with-git-submodules=validate"
     echo
     echo "and then manually update submodules prior to running make, with:"
     echo
-    echo " $ scripts/git-submodule.sh update $modules"
+    echo " $ GIT='tsocks git' scripts/git-submodule.sh update $modules"
     echo
     exit 1
 }
@@ -57,6 +52,12 @@ then
     exit 1
 fi
 
+if test -n "$maybe_modules" && test -z "$GIT"
+then
+    echo "$0: unexpectedly called with submodules but git binary not found"
+    exit 1
+fi
+
 modules=""
 for m in $maybe_modules
 do
-- 
2.40.1

[PULL 13/21] configure: rename --enable-pypi to --enable-download, control subprojects too

[Paolo Bonzini]

The behavior of --{enable,disable}-pypi is similar to that of
-Dwrapmode={default,nodownload} respectively.  In particular,
in both cases a feature needs to be explicitly enabled for the
dependency to be downloaded.

So, use a single option to control both cases.  Now, --enable-slirp
will trigger cloning and building of libslirp if the .pc file
is not found on the machine.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore             |  2 --
 configure              | 17 +++++++----------
 meson.build            | 12 ++++++++----
 subprojects/.gitignore |  3 +++
 4 files changed, 18 insertions(+), 16 deletions(-)
 create mode 100644 subprojects/.gitignore

diff --git a/.gitignore b/.gitignore
index 1ea59f48192..61fa39967b5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -20,5 +20,3 @@ GTAGS
 *.swp
 *.patch
 *.gcov
-
-/subprojects/slirp
diff --git a/configure b/configure
index 92bacc756c0..52274212310 100755
--- a/configure
+++ b/configure
@@ -266,7 +266,7 @@ bsd_user=""
 plugins="$default_feature"
 ninja=""
 python=
-pypi="enabled"
+download="enabled"
 bindir="bin"
 skip_meson=no
 vfio_user_server="disabled"
@@ -754,9 +754,9 @@ for opt do
   --with-git-submodules=*)
       git_submodules_action="$optarg"
   ;;
-  --disable-pypi) pypi="disabled"
+  --disable-download) download="disabled"
   ;;
-  --enable-pypi) pypi="enabled"
+  --enable-download) download="enabled"
   ;;
   --enable-plugins) if test "$mingw32" = "yes"; then
                         error_exit "TCG plugins not currently supported on Windows platforms"
@@ -960,7 +960,7 @@ python="$(command -v "$python")"
 # - venv is allowed to use system packages;
 # - all setup can be performed offline;
 # - missing packages may be fetched from PyPI,
-#   unless --disable-pypi is passed.
+#   unless --disable-download is passed.
 # - pip is not installed into the venv when possible,
 #   but ensurepip is called as a fallback when necessary.
 
@@ -977,7 +977,7 @@ python="$python -B"
 mkvenv="$python ${source_path}/python/scripts/mkvenv.py"
 
 mkvenv_flags=""
-if test "$pypi" = "enabled" ; then
+if test "$download" = "enabled" ; then
     mkvenv_flags="--online"
 fi
 
@@ -1000,7 +1000,7 @@ meson="$(cd pyvenv/bin; pwd)/meson"
 # Conditionally ensure Sphinx is installed.
 
 mkvenv_flags=""
-if test "$pypi" = "enabled" -a "$docs" = "enabled" ; then
+if test "$download" = "enabled" -a "$docs" = "enabled" ; then
     mkvenv_flags="--online"
 fi
 
@@ -1940,11 +1940,8 @@ if test "$skip_meson" = no; then
 
   rm -rf meson-private meson-info meson-logs
 
-  # Prevent meson from automatically downloading wrapped subprojects when missing.
-  # You can use 'meson subprojects download' before running configure.
-  meson_option_add "--wrap-mode=nodownload"
-
   # Built-in options
+  test "$download" = "disabled" && meson_option_add "--wrap-mode=nodownload"
   test "$bindir" != "bin" && meson_option_add "-Dbindir=$bindir"
   test "$default_feature" = no && meson_option_add -Dauto_features=disabled
   test "$static" = yes && meson_option_add -Dprefer_static=true
diff --git a/meson.build b/meson.build
index 4302c019237..f8af4b02e44 100644
--- a/meson.build
+++ b/meson.build
@@ -3985,8 +3985,15 @@ endif
 # Configuration summary #
 #########################
 
-# Directories
+# Build environment
 summary_info = {}
+summary_info += {'Build directory':   meson.current_build_dir()}
+summary_info += {'Source path':       meson.current_source_dir()}
+summary_info += {'GIT submodules':    config_host['GIT_SUBMODULES']}
+summary_info += {'Download dependencies': get_option('wrap_mode') != 'nodownload'}
+summary(summary_info, bool_yn: true, section: 'Build environment')
+
+# Directories
 summary_info += {'Install prefix':    get_option('prefix')}
 summary_info += {'BIOS directory':    qemu_datadir}
 pathsep = targetos == 'windows' ? ';' : ':'
@@ -4004,9 +4011,6 @@ else
   summary_info += {'local state directory': 'queried at runtime'}
 endif
 summary_info += {'Doc directory':     get_option('prefix') / get_option('docdir')}
-summary_info += {'Build directory':   meson.current_build_dir()}
-summary_info += {'Source path':       meson.current_source_dir()}
-summary_info += {'GIT submodules':    config_host['GIT_SUBMODULES']}
 summary(summary_info, bool_yn: true, section: 'Directories')
 
 # Host binaries
diff --git a/subprojects/.gitignore b/subprojects/.gitignore
new file mode 100644
index 00000000000..7560ebb0b1a
--- /dev/null
+++ b/subprojects/.gitignore
@@ -0,0 +1,3 @@
+/packagecache
+
+/slirp
-- 
2.40.1

[PULL 14/21] git-submodule: allow partial update of .git-submodule-status

[Paolo Bonzini]

Allow a specific subdirectory to run git-submodule.sh with only a
subset of submodules, without removing the others from the
.git-submodule-status file.

This also allows scripts/git-submodule.sh to be more lenient:
validating an empty set of submodules is not a mistake.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/git-submodule.sh | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/scripts/git-submodule.sh b/scripts/git-submodule.sh
index 0ce1efc44e5..b7d8f05352c 100755
--- a/scripts/git-submodule.sh
+++ b/scripts/git-submodule.sh
@@ -72,12 +72,8 @@ done
 
 case "$command" in
 status|validate)
-    if test -z "$maybe_modules"
-    then
-         test -s ${substat} && validate_error "$command" || exit 0
-    fi
-
     test -f "$substat" || validate_error "$command"
+    test -z "$maybe_modules" && exit 0
     for module in $modules; do
         CURSTATUS=$($GIT submodule status $module)
         OLDSTATUS=$(cat $substat | grep $module)
@@ -88,17 +84,23 @@ status|validate)
     exit 0
     ;;
 update)
-    if test -z "$maybe_modules"
-    then
-        test -e $substat || touch $substat
-        exit 0
-    fi
+    test -e $substat || touch $substat
+    test -z "$maybe_modules" && exit 0
 
     $GIT submodule update --init $modules 1>/dev/null
     test $? -ne 0 && update_error "failed to update modules"
 
-    $GIT submodule status $modules > "${substat}"
-    test $? -ne 0 && update_error "failed to save git submodule status" >&2
+    (while read -r; do
+        for module in $modules; do
+            case $REPLY in
+                *" $module "*) continue 2 ;;
+            esac
+        done
+        printf '%s\n' "$REPLY"
+    done
+    $GIT submodule status $modules
+    test $? -ne 0 && update_error "failed to save git submodule status" >&2) < $substat > $substat.new
+    mv -f $substat.new $substat
     ;;
 esac
 
-- 
2.40.1

[PULL 15/21] build: log submodule update from git-submodule.sh

[Paolo Bonzini]

Print exactly which submodules have been updated, by reusing the logic of
"git-submodule.sh validate" after executing "git submodule update --init'.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile                 |  4 +---
 scripts/git-submodule.sh | 16 +++++++++++-----
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/Makefile b/Makefile
index 8005f1cc53e..d68196acb9e 100644
--- a/Makefile
+++ b/Makefile
@@ -52,9 +52,7 @@ Makefile: .git-submodule-status
 .PHONY: git-submodule-update
 git-submodule-update:
 ifneq ($(GIT_SUBMODULES_ACTION),ignore)
-	$(call quiet-command, \
-		(GIT=git "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)), \
-		"GIT","$(GIT_SUBMODULES)")
+	$(quiet-@)GIT=git "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)
 endif
 
 # 0. ensure the build tree is okay
diff --git a/scripts/git-submodule.sh b/scripts/git-submodule.sh
index b7d8f05352c..38b55c90e11 100755
--- a/scripts/git-submodule.sh
+++ b/scripts/git-submodule.sh
@@ -46,6 +46,13 @@ validate_error() {
     exit 1
 }
 
+check_updated() {
+    local CURSTATUS OLDSTATUS
+    CURSTATUS=$($GIT submodule status $module)
+    OLDSTATUS=$(grep $module $substat)
+    test "$CURSTATUS" = "$OLDSTATUS"
+}
+
 if test -n "$maybe_modules" && ! test -e ".git"
 then
     echo "$0: unexpectedly called with submodules but no git checkout exists"
@@ -75,11 +82,7 @@ status|validate)
     test -f "$substat" || validate_error "$command"
     test -z "$maybe_modules" && exit 0
     for module in $modules; do
-        CURSTATUS=$($GIT submodule status $module)
-        OLDSTATUS=$(cat $substat | grep $module)
-        if test "$CURSTATUS" != "$OLDSTATUS"; then
-            validate_error "$command"
-        fi
+        check_updated $module || validate_error "$command"
     done
     exit 0
     ;;
@@ -89,6 +92,9 @@ update)
 
     $GIT submodule update --init $modules 1>/dev/null
     test $? -ne 0 && update_error "failed to update modules"
+    for module in $modules; do
+        check_updated $module || echo Updated "$module"
+    done
 
     (while read -r; do
         for module in $modules; do
-- 
2.40.1

[PULL 16/21] meson: subprojects: replace submodules with wrap files

[Paolo Bonzini]

Compared to submodules, .wrap files have several advantages:

* option parsing and downloading is delegated to meson

* the commit is stored in a text file instead of a magic entry in the
  git tree object

* we could stop shipping external dependencies that are only used as a
  fallback, but not break compilation on platforms that lack them.
  For example it may make sense to download dtc at build time, controlled
  by --enable-download, even when building from a tarball.  Right now,
  this patch does the opposite: make-release treats dtc like libvfio-user
  (which is not stable API and therefore hasn't found its way into any
  distros) and keycodemap (which is a copylib, for better or worse).

dependency() can fall back to a wrap automatically.  However, this
is only possible for libraries that come with a .pc file, and this
is not very common for libfdt even though the upstream project in
principle provides it; it also removes the control that we provide with
--enable-fdt={system,internal}.  Therefore, the logic to pick system
vs. internal libfdt is left untouched.

--enable-fdt=git is removed; it was already a synonym for
--enable-fdt=internal.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml |  1 +
 .gitmodules                         |  9 ----
 configure                           | 77 ++++++++++++-----------------
 meson.build                         | 15 +-----
 scripts/archive-source.sh           | 11 ++++-
 scripts/make-release                |  5 ++
 subprojects/.gitignore              |  3 ++
 subprojects/dtc                     |  1 -
 subprojects/dtc.wrap                |  4 ++
 subprojects/keycodemapdb            |  1 -
 subprojects/keycodemapdb.wrap       |  4 ++
 subprojects/libvfio-user            |  1 -
 subprojects/libvfio-user.wrap       |  4 ++
 13 files changed, 63 insertions(+), 73 deletions(-)
 delete mode 160000 subprojects/dtc
 create mode 100644 subprojects/dtc.wrap
 delete mode 160000 subprojects/keycodemapdb
 create mode 100644 subprojects/keycodemapdb.wrap
 delete mode 160000 subprojects/libvfio-user
 create mode 100644 subprojects/libvfio-user.wrap

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index c9f2e737c09..3c997d7265b 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -44,6 +44,7 @@
   script:
     - scripts/git-submodule.sh update
         $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
+    - meson subprojects download $(cd build/subprojects && echo *)
     - cd build
     - find . -type f -exec touch {} +
     # Avoid recompiling by hiding ninja with NINJA=":"
diff --git a/.gitmodules b/.gitmodules
index f8b2ddf3877..a934c85e4e4 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -13,9 +13,6 @@
 [submodule "roms/qemu-palcode"]
 	path = roms/qemu-palcode
 	url = https://gitlab.com/qemu-project/qemu-palcode.git
-[submodule "subprojects/dtc"]
-	path = subprojects/dtc
-	url = https://gitlab.com/qemu-project/dtc.git
 [submodule "roms/u-boot"]
 	path = roms/u-boot
 	url = https://gitlab.com/qemu-project/u-boot.git
@@ -25,9 +22,6 @@
 [submodule "roms/QemuMacDrivers"]
 	path = roms/QemuMacDrivers
 	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
-[submodule "subprojects/keycodemapdb"]
-	path = subprojects/keycodemapdb
-	url = https://gitlab.com/qemu-project/keycodemapdb.git
 [submodule "roms/seabios-hppa"]
 	path = roms/seabios-hppa
 	url = https://gitlab.com/qemu-project/seabios-hppa.git
@@ -55,6 +49,3 @@
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
 	url = https://gitlab.com/libvirt/libvirt-ci.git
-[submodule "subprojects/libvfio-user"]
-	path = subprojects/libvfio-user
-	url = https://gitlab.com/qemu-project/libvfio-user.git
diff --git a/configure b/configure
index 52274212310..45d43ddbd97 100755
--- a/configure
+++ b/configure
@@ -253,7 +253,7 @@ else
     git_submodules_action="ignore"
 fi
 
-git_submodules="subprojects/keycodemapdb"
+git_submodules=""
 git="git"
 debug_tcg="no"
 docs="auto"
@@ -269,7 +269,6 @@ python=
 download="enabled"
 bindir="bin"
 skip_meson=no
-vfio_user_server="disabled"
 use_containers="yes"
 gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb")
 gdb_arches=""
@@ -281,16 +280,13 @@ unset target_list_exclude
 
 # The following Meson options are handled manually (still they
 # are included in the automatically generated help message)
-
-# 1. Track which submodules are needed
-fdt="auto"
-
-# 2. Automatically enable/disable other options
+# because they automatically enable/disable other options
 tcg="auto"
 cfi="false"
 
-# 3. Need to check for -static-pie before Meson runs.  Also,
-# Meson has PIE as a boolean rather than enabled/disabled/auto.
+# Meson has PIE as a boolean rather than enabled/disabled/auto,
+# and we also need to check for -static-pie before Meson runs
+# which requires knowing whether --static is enabled.
 pie=""
 static="no"
 
@@ -743,14 +739,6 @@ for opt do
   ;;
   --disable-cfi) cfi="false"
   ;;
-  --disable-fdt) fdt="disabled"
-  ;;
-  --enable-fdt) fdt="enabled"
-  ;;
-  --enable-fdt=git) fdt="internal"
-  ;;
-  --enable-fdt=*) fdt="$optarg"
-  ;;
   --with-git-submodules=*)
       git_submodules_action="$optarg"
   ;;
@@ -772,10 +760,6 @@ for opt do
   ;;
   --gdb=*) gdb_bin="$optarg"
   ;;
-  --enable-vfio-user-server) vfio_user_server="enabled"
-  ;;
-  --disable-vfio-user-server) vfio_user_server="disabled"
-  ;;
   # everything else has the same name in configure and meson
   --*) meson_option_parse "$opt" "$optarg"
   ;;
@@ -787,6 +771,32 @@ if test "$plugins" = "yes" -a "$tcg" = "disabled"; then
     error_exit "Can't enable plugins on non-TCG builds"
 fi
 
+if ! test -f "$source_path/subprojects/keycodemapdb/README" \
+    && test -f "$download" = disabled
+then
+    echo
+    echo "ERROR: missing subprojects"
+    echo
+    if test -e "$source_path/.git"; then
+        echo "--disable-download specified but subprojects were not"
+        echo 'checked out.  Please invoke "meson subprojects download"'
+        echo "before configuring QEMU, or remove --disable-download"
+        echo "from the command line."
+    else
+        echo "This is not a GIT checkout but subproject content appears to"
+        echo "be missing. Do not use 'git archive' or GitHub download links"
+        echo "to acquire QEMU source archives. Non-GIT builds are only"
+        echo "supported with source archives linked from:"
+        echo
+        echo "  https://www.qemu.org/download/#source"
+        echo
+        echo "Developers working with GIT can use scripts/archive-source.sh"
+        echo "if they need to create valid source archives."
+    fi
+    echo
+    exit 1
+fi
+
 case $git_submodules_action in
     update|validate)
         if test ! -e "$source_path/.git" || ! has git; then
@@ -795,7 +805,7 @@ case $git_submodules_action in
         fi
     ;;
     ignore)
-        if ! test -f "$source_path/subprojects/keycodemapdb/README"
+        if ! test -f "$source_path/tests/fp/berkeley-testfloat-3/README.md"
         then
             echo
             echo "ERROR: missing GIT submodules"
@@ -1164,16 +1174,6 @@ EOF
   fi
 fi
 
-##########################################
-# fdt probe
-
-case "$fdt" in
-  auto | enabled | internal)
-    # Simpler to always update submodule, even if not needed.
-    git_submodules="${git_submodules} subprojects/dtc"
-    ;;
-esac
-
 ########################################
 # check if ccache is interfering with
 # semantic analysis of macros
@@ -1609,17 +1609,6 @@ write_target_makefile() {
   fi
 }
 
-##########################################
-# check for vfio_user_server
-
-case "$vfio_user_server" in
-  enabled )
-    if test "$git_submodules_action" != "ignore"; then
-      git_submodules="${git_submodules} subprojects/libvfio-user"
-    fi
-    ;;
-esac
-
 #######################################
 # cross-compiled firmware targets
 
@@ -1951,12 +1940,10 @@ if test "$skip_meson" = no; then
   # QEMU options
   test "$cfi" != false && meson_option_add "-Dcfi=$cfi"
   test "$docs" != auto && meson_option_add "-Ddocs=$docs"
-  test "$fdt" != auto && meson_option_add "-Dfdt=$fdt"
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
   test "$qemu_suffix" != qemu && meson_option_add "-Dqemu_suffix=$qemu_suffix"
   test "$smbd" != '' && meson_option_add "-Dsmbd=$smbd"
   test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
-  test "$vfio_user_server" != auto && meson_option_add "-Dvfio_user_server=$vfio_user_server"
   run_meson() {
     NINJA=$ninja $meson setup --prefix "$prefix" "$@" $cross_arg "$PWD" "$source_path"
   }
diff --git a/meson.build b/meson.build
index f8af4b02e44..74fb147fde3 100644
--- a/meson.build
+++ b/meson.build
@@ -3031,14 +3031,7 @@ endif
 
 libvfio_user_dep = not_found
 if have_system and vfio_user_server_allowed
-  have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/meson.build')
-
-  if not have_internal
-    error('libvfio-user source not found - please pull git submodule')
-  endif
-
-  libvfio_user_proj = subproject('libvfio-user')
-
+  libvfio_user_proj = subproject('libvfio-user', required: true)
   libvfio_user_dep = libvfio_user_proj.get_variable('libvfio_user_dep')
 endif
 
@@ -3066,12 +3059,6 @@ if fdt_required.length() > 0 or fdt_opt == 'enabled'
   endif
   if not fdt.found()
     assert(fdt_opt == 'internal')
-    have_internal = fs.exists(meson.current_source_dir() / 'subprojects/dtc/meson.build')
-
-    if not have_internal
-      error('libfdt source not found - please pull git submodule')
-    endif
-
     libfdt_proj = subproject('dtc', required: true,
                              default_options: ['tools=false',  'yaml=disabled',
                                                'python=disabled', 'default_library=static'])
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index dba5ae05b67..b99cb66e412 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -26,8 +26,8 @@ sub_file="${sub_tdir}/submodule.tar"
 # independent of what the developer currently has initialized
 # in their checkout, because the build environment is completely
 # different to the host OS.
-submodules="subprojects/dtc subprojects/keycodemapdb"
-submodules="$submodules tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3"
+subprojects="dtc keycodemapdb libvfio-user"
+submodules="tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3"
 sub_deinit=""
 
 function cleanup() {
@@ -70,4 +70,11 @@ for sm in $submodules; do
     tar --concatenate --file "$tar_file" "$sub_file"
     test $? -ne 0 && error "failed append submodule $sm to $tar_file"
 done
+
+for sp in $subprojects; do
+    meson subprojects download $sp
+    test $? -ne 0 && error "failed to download subproject $sp"
+    tar --append --file "$tar_file" --exclude=.git subprojects/$sp
+    test $? -ne 0 && error "failed to append subproject $sp to $tar_file"
+done
 exit 0
diff --git a/scripts/make-release b/scripts/make-release
index 44a9d86a04a..0604e61b814 100755
--- a/scripts/make-release
+++ b/scripts/make-release
@@ -16,6 +16,9 @@ if [ $# -ne 2 ]; then
     exit 0
 fi
 
+# Only include wraps that are invoked with subproject()
+SUBPROJECTS="dtc libvfio-user keycodemapdb"
+
 src="$1"
 version="$2"
 destination=qemu-${version}
@@ -26,6 +29,8 @@ git clone --single-branch -b "v${version}" -c advice.detachedHead=false \
 pushd ${destination}
 
 git submodule update --init --single-branch
+meson subprojects download $SUBPROJECTS
+
 (cd roms/seabios && git describe --tags --long --dirty > .version)
 (cd roms/skiboot && ./make_version.sh > .version)
 # Fetch edk2 submodule's submodules, since it won't have access to them via
diff --git a/subprojects/.gitignore b/subprojects/.gitignore
index 7560ebb0b1a..935e9a72a8f 100644
--- a/subprojects/.gitignore
+++ b/subprojects/.gitignore
@@ -1,3 +1,6 @@
 /packagecache
 
+/dtc
+/keycodemapdb
+/libvfio-user
 /slirp
diff --git a/subprojects/dtc b/subprojects/dtc
deleted file mode 160000
index b6910bec116..00000000000
--- a/subprojects/dtc
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit b6910bec11614980a21e46fbccc35934b671bd81
diff --git a/subprojects/dtc.wrap b/subprojects/dtc.wrap
new file mode 100644
index 00000000000..d1bc9174e91
--- /dev/null
+++ b/subprojects/dtc.wrap
@@ -0,0 +1,4 @@
+[wrap-git]
+url = https://gitlab.com/qemu-project/dtc.git
+revision = b6910bec11614980a21e46fbccc35934b671bd81
+depth = 1
diff --git a/subprojects/keycodemapdb b/subprojects/keycodemapdb
deleted file mode 160000
index f5772a62ec5..00000000000
--- a/subprojects/keycodemapdb
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit f5772a62ec52591ff6870b7e8ef32482371f22c6
diff --git a/subprojects/keycodemapdb.wrap b/subprojects/keycodemapdb.wrap
new file mode 100644
index 00000000000..dda7b0e5716
--- /dev/null
+++ b/subprojects/keycodemapdb.wrap
@@ -0,0 +1,4 @@
+[wrap-git]
+url = https://gitlab.com/qemu-project/keycodemapdb.git
+revision = f5772a62ec52591ff6870b7e8ef32482371f22c6
+depth = 1
diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
deleted file mode 160000
index 0b28d205572..00000000000
--- a/subprojects/libvfio-user
+++ /dev/null
@@ -1 +0,0 @@
-Subproject commit 0b28d205572c80b568a1003db2c8f37ca333e4d7
diff --git a/subprojects/libvfio-user.wrap b/subprojects/libvfio-user.wrap
new file mode 100644
index 00000000000..416955ca451
--- /dev/null
+++ b/subprojects/libvfio-user.wrap
@@ -0,0 +1,4 @@
+[wrap-git]
+url = https://gitlab.com/qemu-project/libvfio-user.git
+revision = 0b28d205572c80b568a1003db2c8f37ca333e4d7
+depth = 1
-- 
2.40.1

[PULL 17/21] configure: move SLOF submodule handling to pc-bios/s390-ccw

[Paolo Bonzini]

Move the handling of the roms/SLOF submodule out of the main Makefile,
since we are going to remove submodules from the build process of QEMU.

Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml |  2 +-
 configure                           |  7 +++----
 pc-bios/s390-ccw/Makefile           | 11 +++++++++++
 3 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 3c997d7265b..d01d504ec5f 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -43,7 +43,7 @@
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
     - scripts/git-submodule.sh update
-        $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
+        roms/SLOF $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
     - meson subprojects download $(cd build/subprojects && echo *)
     - cd build
     - find . -type f -exec touch {} +
diff --git a/configure b/configure
index 45d43ddbd97..de3904fb59d 100755
--- a/configure
+++ b/configure
@@ -1662,7 +1662,8 @@ fi
 
 # Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
 # (which is the lowest architecture level that Clang supports)
-if have_target s390x-softmmu && probe_target_compiler s390x-softmmu; then
+if have_target s390x-softmmu && probe_target_compiler s390x-softmmu && \
+    GIT=git "$source_path/scripts/git-submodule.sh" "$git_submodules_action" roms/SLOF >> config.log 2>&1; then
   write_c_skeleton
   do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
   has_z900=$?
@@ -1675,10 +1676,8 @@ if have_target s390x-softmmu && probe_target_compiler s390x-softmmu; then
     config_mak=pc-bios/s390-ccw/config-host.mak
     echo "# Automatically generated by configure - do not modify" > $config_mak
     echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
+    echo "GIT_SUBMODULES_ACTION=$git_submodules_action" >> $config_mak
     write_target_makefile >> $config_mak
-    # SLOF is required for building the s390-ccw firmware on s390x,
-    # since it is using the libnet code from SLOF for network booting.
-    git_submodules="${git_submodules} roms/SLOF"
   fi
 fi
 
diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 10e8f5cb633..9c5276f8ade 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -6,6 +6,8 @@ include config-host.mak
 CFLAGS = -O2 -g
 MAKEFLAGS += -rR
 
+GIT_SUBMODULES = roms/SLOF
+
 NULL :=
 SPACE := $(NULL) #
 TARGET_PREFIX := $(patsubst %/,%:$(SPACE),$(TARGET_DIR))
@@ -80,3 +82,12 @@ clean:
 
 distclean:
 	rm -f config-cc.mak
+
+.PHONY: git-submodule-update
+$(SRC_PATH)/../../.git-submodule-status: git-submodule-update config-host.mak
+Makefile: $(SRC_PATH)/../../.git-submodule-status
+
+git-submodule-update:
+ifneq ($(GIT_SUBMODULES_ACTION),ignore)
+	$(quiet-@)GIT=git "$(SRC_PATH)/../../scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)
+endif
-- 
2.40.1

[PULL 18/21] pc-bios/s390-ccw: always build network bootloader

[Paolo Bonzini]

In the beginning, the network bootloader was considered experimental and
thus optional, but it is well established nowadays and configure always
checks for roms/SLOF before compiling pc-bios/s390-ccw.

Therefore, it makes sense to always build it together with the other
part of the s390-ccw bios.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 pc-bios/s390-ccw/Makefile | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 9c5276f8ade..2e8cc015aa0 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -67,12 +67,7 @@ s390-ccw.img: s390-ccw.elf
 
 $(OBJECTS): Makefile
 
-ifneq ($(wildcard $(SRC_PATH)/../../roms/SLOF/lib/libnet),)
 include $(SRC_PATH)/netboot.mak
-else
-s390-netboot.img:
-	@echo "s390-netboot.img not built since roms/SLOF/ is not available."
-endif
 
 ALL_OBJS = $(sort $(OBJECTS) $(NETOBJS) $(LIBCOBJS) $(LIBNETOBJS))
 -include $(ALL_OBJS:%.o=%.d)
-- 
2.40.1

[PULL 19/21] meson: subprojects: replace berkeley-{soft, test}float-3 with wraps

[Paolo Bonzini]

Unlike other subprojects, these require an overlay directory to include
meson rules to build the libraries.  The rules are basically lifted
from tests/fp/meson.build, with a few changes to create platform.h
and publish a dependency.

The build defines are passed through a subproject option, and posted
back to users of the library via the dependency's compile_args.

The only remaining user of GIT_SUBMODULES and GIT_SUBMODULES_ACTION
is roms/SLOF, which is used to build pc-bios/s390-ccw.  All other
roms submodules are only present to satisfy the license on pre-built
firmware blobs.

Best reviewed with --color-moved.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitmodules                                   |   6 -
 configure                                     |  27 -
 scripts/archive-source.sh                     |  22 +-
 scripts/make-release                          |   2 +-
 subprojects/.gitignore                        |   2 +
 subprojects/berkeley-softfloat-3.wrap         |   5 +
 subprojects/berkeley-testfloat-3.wrap         |   5 +
 .../berkeley-softfloat-3/meson.build          | 339 +++++++++++
 .../berkeley-softfloat-3/meson_options.txt    |   1 +
 .../berkeley-testfloat-3/meson.build          | 220 +++++++
 .../berkeley-testfloat-3/meson_options.txt    |   1 +
 tests/fp/meson.build                          | 541 +-----------------
 12 files changed, 590 insertions(+), 581 deletions(-)
 create mode 100644 subprojects/berkeley-softfloat-3.wrap
 create mode 100644 subprojects/berkeley-testfloat-3.wrap
 create mode 100644 subprojects/packagefiles/berkeley-softfloat-3/meson.build
 create mode 100644 subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt
 create mode 100644 subprojects/packagefiles/berkeley-testfloat-3/meson.build
 create mode 100644 subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt

diff --git a/.gitmodules b/.gitmodules
index a934c85e4e4..73cae4cd4da 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -28,12 +28,6 @@
 [submodule "roms/u-boot-sam460ex"]
 	path = roms/u-boot-sam460ex
 	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
-[submodule "tests/fp/berkeley-testfloat-3"]
-	path = tests/fp/berkeley-testfloat-3
-	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
-[submodule "tests/fp/berkeley-softfloat-3"]
-	path = tests/fp/berkeley-softfloat-3
-	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
 [submodule "roms/edk2"]
 	path = roms/edk2
 	url = https://gitlab.com/qemu-project/edk2.git
diff --git a/configure b/configure
index de3904fb59d..4dad32938df 100755
--- a/configure
+++ b/configure
@@ -805,28 +805,6 @@ case $git_submodules_action in
         fi
     ;;
     ignore)
-        if ! test -f "$source_path/tests/fp/berkeley-testfloat-3/README.md"
-        then
-            echo
-            echo "ERROR: missing GIT submodules"
-            echo
-            if test -e "$source_path/.git"; then
-                echo "--with-git-submodules=ignore specified but submodules were not"
-                echo "checked out.  Please initialize and update submodules."
-            else
-                echo "This is not a GIT checkout but module content appears to"
-                echo "be missing. Do not use 'git archive' or GitHub download links"
-                echo "to acquire QEMU source archives. Non-GIT builds are only"
-                echo "supported with source archives linked from:"
-                echo
-                echo "  https://www.qemu.org/download/#source"
-                echo
-                echo "Developers working with GIT can use scripts/archive-source.sh"
-                echo "if they need to create valid source archives."
-            fi
-            echo
-            exit 1
-        fi
     ;;
     *)
         echo "ERROR: invalid --with-git-submodules= value '$git_submodules_action'"
@@ -1142,11 +1120,6 @@ if test "$tcg" = "auto"; then
   fi
 fi
 
-if test "$tcg" = "enabled"; then
-    git_submodules="$git_submodules tests/fp/berkeley-testfloat-3"
-    git_submodules="$git_submodules tests/fp/berkeley-softfloat-3"
-fi
-
 ##########################################
 # big/little endian test
 cat > $TMPC << EOF
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index b99cb66e412..48996304910 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -26,8 +26,7 @@ sub_file="${sub_tdir}/submodule.tar"
 # independent of what the developer currently has initialized
 # in their checkout, because the build environment is completely
 # different to the host OS.
-subprojects="dtc keycodemapdb libvfio-user"
-submodules="tests/fp/berkeley-softfloat-3 tests/fp/berkeley-testfloat-3"
+subprojects="dtc keycodemapdb libvfio-user berkeley-softfloat-3 berkeley-testfloat-3"
 sub_deinit=""
 
 function cleanup() {
@@ -51,25 +50,6 @@ function tree_ish() {
 
 git archive --format tar "$(tree_ish)" > "$tar_file"
 test $? -ne 0 && error "failed to archive qemu"
-for sm in $submodules; do
-    status="$(git submodule status "$sm")"
-    smhash="${status#[ +-]}"
-    smhash="${smhash%% *}"
-    case "$status" in
-        -*)
-            sub_deinit="$sub_deinit $sm"
-            git submodule update --init "$sm"
-            test $? -ne 0 && error "failed to update submodule $sm"
-            ;;
-        +*)
-            echo "WARNING: submodule $sm is out of sync"
-            ;;
-    esac
-    (cd $sm; git archive --format tar --prefix "$sm/" $(tree_ish)) > "$sub_file"
-    test $? -ne 0 && error "failed to archive submodule $sm ($smhash)"
-    tar --concatenate --file "$tar_file" "$sub_file"
-    test $? -ne 0 && error "failed append submodule $sm to $tar_file"
-done
 
 for sp in $subprojects; do
     meson subprojects download $sp
diff --git a/scripts/make-release b/scripts/make-release
index 0604e61b814..c5db87b3f91 100755
--- a/scripts/make-release
+++ b/scripts/make-release
@@ -17,7 +17,7 @@ if [ $# -ne 2 ]; then
 fi
 
 # Only include wraps that are invoked with subproject()
-SUBPROJECTS="dtc libvfio-user keycodemapdb"
+SUBPROJECTS="dtc libvfio-user keycodemapdb berkeley-softfloat-3 berkeley-testfloat-3"
 
 src="$1"
 version="$2"
diff --git a/subprojects/.gitignore b/subprojects/.gitignore
index 935e9a72a8f..adca0266be6 100644
--- a/subprojects/.gitignore
+++ b/subprojects/.gitignore
@@ -1,5 +1,7 @@
 /packagecache
 
+/berkeley-softfloat-3
+/berkeley-testfloat-3
 /dtc
 /keycodemapdb
 /libvfio-user
diff --git a/subprojects/berkeley-softfloat-3.wrap b/subprojects/berkeley-softfloat-3.wrap
new file mode 100644
index 00000000000..a8fd87740b1
--- /dev/null
+++ b/subprojects/berkeley-softfloat-3.wrap
@@ -0,0 +1,5 @@
+[wrap-git]
+url = https://gitlab.com/qemu-project/berkeley-softfloat-3
+revision = b64af41c3276f97f0e181920400ee056b9c88037
+patch_directory = berkeley-softfloat-3
+depth = 1
diff --git a/subprojects/berkeley-testfloat-3.wrap b/subprojects/berkeley-testfloat-3.wrap
new file mode 100644
index 00000000000..6ad80a37b2a
--- /dev/null
+++ b/subprojects/berkeley-testfloat-3.wrap
@@ -0,0 +1,5 @@
+[wrap-git]
+url = https://gitlab.com/qemu-project/berkeley-testfloat-3
+revision = 40619cbb3bf32872df8c53cc457039229428a263
+patch_directory = berkeley-testfloat-3
+depth = 1
diff --git a/subprojects/packagefiles/berkeley-softfloat-3/meson.build b/subprojects/packagefiles/berkeley-softfloat-3/meson.build
new file mode 100644
index 00000000000..4ce964b8389
--- /dev/null
+++ b/subprojects/packagefiles/berkeley-softfloat-3/meson.build
@@ -0,0 +1,339 @@
+project('berkeley-softfloat-3',  'c',
+        default_options: ['warning_level=1', 'c_std=gnu99'])
+
+fpcflags = get_option('defines')
+
+platform_data = configuration_data()
+platform_data.set('INLINE', 'static inline')
+platform_data.set('LITTLEENDIAN', host_machine.endian() == 'little')
+configure_file(output: 'platform.h', configuration: platform_data)
+
+sfdir = 'source'
+sfspedir = sfdir / '8086-SSE'
+sfinc = include_directories('.', sfdir / 'include', sfspedir)
+
+add_project_arguments([
+  '-Wno-implicit-fallthrough',
+  '-Wno-missing-prototypes',
+  '-Wno-redundant-decls',
+  '-Wno-return-type',
+  '-Wno-error',
+], native: false, language: 'c')
+
+libsoftfloat = static_library(
+  'softfloat',
+  files(
+    # primitives
+    sfdir / 's_eq128.c',
+    sfdir / 's_le128.c',
+    sfdir / 's_lt128.c',
+    sfdir / 's_shortShiftLeft128.c',
+    sfdir / 's_shortShiftRight128.c',
+    sfdir / 's_shortShiftRightJam64.c',
+    sfdir / 's_shortShiftRightJam64Extra.c',
+    sfdir / 's_shortShiftRightJam128.c',
+    sfdir / 's_shortShiftRightJam128Extra.c',
+    sfdir / 's_shiftRightJam32.c',
+    sfdir / 's_shiftRightJam64.c',
+    sfdir / 's_shiftRightJam64Extra.c',
+    sfdir / 's_shiftRightJam128.c',
+    sfdir / 's_shiftRightJam128Extra.c',
+    sfdir / 's_shiftRightJam256M.c',
+    sfdir / 's_countLeadingZeros8.c',
+    sfdir / 's_countLeadingZeros16.c',
+    sfdir / 's_countLeadingZeros32.c',
+    sfdir / 's_countLeadingZeros64.c',
+    sfdir / 's_add128.c',
+    sfdir / 's_add256M.c',
+    sfdir / 's_sub128.c',
+    sfdir / 's_sub256M.c',
+    sfdir / 's_mul64ByShifted32To128.c',
+    sfdir / 's_mul64To128.c',
+    sfdir / 's_mul128By32.c',
+    sfdir / 's_mul128To256M.c',
+    sfdir / 's_approxRecip_1Ks.c',
+    sfdir / 's_approxRecip32_1.c',
+    sfdir / 's_approxRecipSqrt_1Ks.c',
+    sfdir / 's_approxRecipSqrt32_1.c',
+    # others
+    sfdir / 's_roundToUI32.c',
+    sfdir / 's_roundToUI64.c',
+    sfdir / 's_roundToI32.c',
+    sfdir / 's_roundToI64.c',
+    sfdir / 's_normSubnormalF16Sig.c',
+    sfdir / 's_roundPackToF16.c',
+    sfdir / 's_normRoundPackToF16.c',
+    sfdir / 's_addMagsF16.c',
+    sfdir / 's_subMagsF16.c',
+    sfdir / 's_mulAddF16.c',
+    sfdir / 's_normSubnormalF32Sig.c',
+    sfdir / 's_roundPackToF32.c',
+    sfdir / 's_normRoundPackToF32.c',
+    sfdir / 's_addMagsF32.c',
+    sfdir / 's_subMagsF32.c',
+    sfdir / 's_mulAddF32.c',
+    sfdir / 's_normSubnormalF64Sig.c',
+    sfdir / 's_roundPackToF64.c',
+    sfdir / 's_normRoundPackToF64.c',
+    sfdir / 's_addMagsF64.c',
+    sfdir / 's_subMagsF64.c',
+    sfdir / 's_mulAddF64.c',
+    sfdir / 's_normSubnormalExtF80Sig.c',
+    sfdir / 's_roundPackToExtF80.c',
+    sfdir / 's_normRoundPackToExtF80.c',
+    sfdir / 's_addMagsExtF80.c',
+    sfdir / 's_subMagsExtF80.c',
+    sfdir / 's_normSubnormalF128Sig.c',
+    sfdir / 's_roundPackToF128.c',
+    sfdir / 's_normRoundPackToF128.c',
+    sfdir / 's_addMagsF128.c',
+    sfdir / 's_subMagsF128.c',
+    sfdir / 's_mulAddF128.c',
+    sfdir / 'softfloat_state.c',
+    sfdir / 'ui32_to_f16.c',
+    sfdir / 'ui32_to_f32.c',
+    sfdir / 'ui32_to_f64.c',
+    sfdir / 'ui32_to_extF80.c',
+    sfdir / 'ui32_to_extF80M.c',
+    sfdir / 'ui32_to_f128.c',
+    sfdir / 'ui32_to_f128M.c',
+    sfdir / 'ui64_to_f16.c',
+    sfdir / 'ui64_to_f32.c',
+    sfdir / 'ui64_to_f64.c',
+    sfdir / 'ui64_to_extF80.c',
+    sfdir / 'ui64_to_extF80M.c',
+    sfdir / 'ui64_to_f128.c',
+    sfdir / 'ui64_to_f128M.c',
+    sfdir / 'i32_to_f16.c',
+    sfdir / 'i32_to_f32.c',
+    sfdir / 'i32_to_f64.c',
+    sfdir / 'i32_to_extF80.c',
+    sfdir / 'i32_to_extF80M.c',
+    sfdir / 'i32_to_f128.c',
+    sfdir / 'i32_to_f128M.c',
+    sfdir / 'i64_to_f16.c',
+    sfdir / 'i64_to_f32.c',
+    sfdir / 'i64_to_f64.c',
+    sfdir / 'i64_to_extF80.c',
+    sfdir / 'i64_to_extF80M.c',
+    sfdir / 'i64_to_f128.c',
+    sfdir / 'i64_to_f128M.c',
+    sfdir / 'f16_to_ui32.c',
+    sfdir / 'f16_to_ui64.c',
+    sfdir / 'f16_to_i32.c',
+    sfdir / 'f16_to_i64.c',
+    sfdir / 'f16_to_ui32_r_minMag.c',
+    sfdir / 'f16_to_ui64_r_minMag.c',
+    sfdir / 'f16_to_i32_r_minMag.c',
+    sfdir / 'f16_to_i64_r_minMag.c',
+    sfdir / 'f16_to_f32.c',
+    sfdir / 'f16_to_f64.c',
+    sfdir / 'f16_to_extF80.c',
+    sfdir / 'f16_to_extF80M.c',
+    sfdir / 'f16_to_f128.c',
+    sfdir / 'f16_to_f128M.c',
+    sfdir / 'f16_roundToInt.c',
+    sfdir / 'f16_add.c',
+    sfdir / 'f16_sub.c',
+    sfdir / 'f16_mul.c',
+    sfdir / 'f16_mulAdd.c',
+    sfdir / 'f16_div.c',
+    sfdir / 'f16_rem.c',
+    sfdir / 'f16_sqrt.c',
+    sfdir / 'f16_eq.c',
+    sfdir / 'f16_le.c',
+    sfdir / 'f16_lt.c',
+    sfdir / 'f16_eq_signaling.c',
+    sfdir / 'f16_le_quiet.c',
+    sfdir / 'f16_lt_quiet.c',
+    sfdir / 'f16_isSignalingNaN.c',
+    sfdir / 'f32_to_ui32.c',
+    sfdir / 'f32_to_ui64.c',
+    sfdir / 'f32_to_i32.c',
+    sfdir / 'f32_to_i64.c',
+    sfdir / 'f32_to_ui32_r_minMag.c',
+    sfdir / 'f32_to_ui64_r_minMag.c',
+    sfdir / 'f32_to_i32_r_minMag.c',
+    sfdir / 'f32_to_i64_r_minMag.c',
+    sfdir / 'f32_to_f16.c',
+    sfdir / 'f32_to_f64.c',
+    sfdir / 'f32_to_extF80.c',
+    sfdir / 'f32_to_extF80M.c',
+    sfdir / 'f32_to_f128.c',
+    sfdir / 'f32_to_f128M.c',
+    sfdir / 'f32_roundToInt.c',
+    sfdir / 'f32_add.c',
+    sfdir / 'f32_sub.c',
+    sfdir / 'f32_mul.c',
+    sfdir / 'f32_mulAdd.c',
+    sfdir / 'f32_div.c',
+    sfdir / 'f32_rem.c',
+    sfdir / 'f32_sqrt.c',
+    sfdir / 'f32_eq.c',
+    sfdir / 'f32_le.c',
+    sfdir / 'f32_lt.c',
+    sfdir / 'f32_eq_signaling.c',
+    sfdir / 'f32_le_quiet.c',
+    sfdir / 'f32_lt_quiet.c',
+    sfdir / 'f32_isSignalingNaN.c',
+    sfdir / 'f64_to_ui32.c',
+    sfdir / 'f64_to_ui64.c',
+    sfdir / 'f64_to_i32.c',
+    sfdir / 'f64_to_i64.c',
+    sfdir / 'f64_to_ui32_r_minMag.c',
+    sfdir / 'f64_to_ui64_r_minMag.c',
+    sfdir / 'f64_to_i32_r_minMag.c',
+    sfdir / 'f64_to_i64_r_minMag.c',
+    sfdir / 'f64_to_f16.c',
+    sfdir / 'f64_to_f32.c',
+    sfdir / 'f64_to_extF80.c',
+    sfdir / 'f64_to_extF80M.c',
+    sfdir / 'f64_to_f128.c',
+    sfdir / 'f64_to_f128M.c',
+    sfdir / 'f64_roundToInt.c',
+    sfdir / 'f64_add.c',
+    sfdir / 'f64_sub.c',
+    sfdir / 'f64_mul.c',
+    sfdir / 'f64_mulAdd.c',
+    sfdir / 'f64_div.c',
+    sfdir / 'f64_rem.c',
+    sfdir / 'f64_sqrt.c',
+    sfdir / 'f64_eq.c',
+    sfdir / 'f64_le.c',
+    sfdir / 'f64_lt.c',
+    sfdir / 'f64_eq_signaling.c',
+    sfdir / 'f64_le_quiet.c',
+    sfdir / 'f64_lt_quiet.c',
+    sfdir / 'f64_isSignalingNaN.c',
+    sfdir / 'extF80_to_ui32.c',
+    sfdir / 'extF80_to_ui64.c',
+    sfdir / 'extF80_to_i32.c',
+    sfdir / 'extF80_to_i64.c',
+    sfdir / 'extF80_to_ui32_r_minMag.c',
+    sfdir / 'extF80_to_ui64_r_minMag.c',
+    sfdir / 'extF80_to_i32_r_minMag.c',
+    sfdir / 'extF80_to_i64_r_minMag.c',
+    sfdir / 'extF80_to_f16.c',
+    sfdir / 'extF80_to_f32.c',
+    sfdir / 'extF80_to_f64.c',
+    sfdir / 'extF80_to_f128.c',
+    sfdir / 'extF80_roundToInt.c',
+    sfdir / 'extF80_add.c',
+    sfdir / 'extF80_sub.c',
+    sfdir / 'extF80_mul.c',
+    sfdir / 'extF80_div.c',
+    sfdir / 'extF80_rem.c',
+    sfdir / 'extF80_sqrt.c',
+    sfdir / 'extF80_eq.c',
+    sfdir / 'extF80_le.c',
+    sfdir / 'extF80_lt.c',
+    sfdir / 'extF80_eq_signaling.c',
+    sfdir / 'extF80_le_quiet.c',
+    sfdir / 'extF80_lt_quiet.c',
+    sfdir / 'extF80_isSignalingNaN.c',
+    sfdir / 'extF80M_to_ui32.c',
+    sfdir / 'extF80M_to_ui64.c',
+    sfdir / 'extF80M_to_i32.c',
+    sfdir / 'extF80M_to_i64.c',
+    sfdir / 'extF80M_to_ui32_r_minMag.c',
+    sfdir / 'extF80M_to_ui64_r_minMag.c',
+    sfdir / 'extF80M_to_i32_r_minMag.c',
+    sfdir / 'extF80M_to_i64_r_minMag.c',
+    sfdir / 'extF80M_to_f16.c',
+    sfdir / 'extF80M_to_f32.c',
+    sfdir / 'extF80M_to_f64.c',
+    sfdir / 'extF80M_to_f128M.c',
+    sfdir / 'extF80M_roundToInt.c',
+    sfdir / 'extF80M_add.c',
+    sfdir / 'extF80M_sub.c',
+    sfdir / 'extF80M_mul.c',
+    sfdir / 'extF80M_div.c',
+    sfdir / 'extF80M_rem.c',
+    sfdir / 'extF80M_sqrt.c',
+    sfdir / 'extF80M_eq.c',
+    sfdir / 'extF80M_le.c',
+    sfdir / 'extF80M_lt.c',
+    sfdir / 'extF80M_eq_signaling.c',
+    sfdir / 'extF80M_le_quiet.c',
+    sfdir / 'extF80M_lt_quiet.c',
+    sfdir / 'f128_to_ui32.c',
+    sfdir / 'f128_to_ui64.c',
+    sfdir / 'f128_to_i32.c',
+    sfdir / 'f128_to_i64.c',
+    sfdir / 'f128_to_ui32_r_minMag.c',
+    sfdir / 'f128_to_ui64_r_minMag.c',
+    sfdir / 'f128_to_i32_r_minMag.c',
+    sfdir / 'f128_to_i64_r_minMag.c',
+    sfdir / 'f128_to_f16.c',
+    sfdir / 'f128_to_f32.c',
+    sfdir / 'f128_to_extF80.c',
+    sfdir / 'f128_to_f64.c',
+    sfdir / 'f128_roundToInt.c',
+    sfdir / 'f128_add.c',
+    sfdir / 'f128_sub.c',
+    sfdir / 'f128_mul.c',
+    sfdir / 'f128_mulAdd.c',
+    sfdir / 'f128_div.c',
+    sfdir / 'f128_rem.c',
+    sfdir / 'f128_sqrt.c',
+    sfdir / 'f128_eq.c',
+    sfdir / 'f128_le.c',
+    sfdir / 'f128_lt.c',
+    sfdir / 'f128_eq_signaling.c',
+    sfdir / 'f128_le_quiet.c',
+    sfdir / 'f128_lt_quiet.c',
+    sfdir / 'f128_isSignalingNaN.c',
+    sfdir / 'f128M_to_ui32.c',
+    sfdir / 'f128M_to_ui64.c',
+    sfdir / 'f128M_to_i32.c',
+    sfdir / 'f128M_to_i64.c',
+    sfdir / 'f128M_to_ui32_r_minMag.c',
+    sfdir / 'f128M_to_ui64_r_minMag.c',
+    sfdir / 'f128M_to_i32_r_minMag.c',
+    sfdir / 'f128M_to_i64_r_minMag.c',
+    sfdir / 'f128M_to_f16.c',
+    sfdir / 'f128M_to_f32.c',
+    sfdir / 'f128M_to_extF80M.c',
+    sfdir / 'f128M_to_f64.c',
+    sfdir / 'f128M_roundToInt.c',
+    sfdir / 'f128M_add.c',
+    sfdir / 'f128M_sub.c',
+    sfdir / 'f128M_mul.c',
+    sfdir / 'f128M_mulAdd.c',
+    sfdir / 'f128M_div.c',
+    sfdir / 'f128M_rem.c',
+    sfdir / 'f128M_sqrt.c',
+    sfdir / 'f128M_eq.c',
+    sfdir / 'f128M_le.c',
+    sfdir / 'f128M_lt.c',
+    sfdir / 'f128M_eq_signaling.c',
+    sfdir / 'f128M_le_quiet.c',
+    sfdir / 'f128M_lt_quiet.c',
+    # spe
+    sfspedir / 'softfloat_raiseFlags.c',
+    sfspedir / 's_f16UIToCommonNaN.c',
+    sfspedir / 's_commonNaNToF16UI.c',
+    sfspedir / 's_propagateNaNF16UI.c',
+    sfspedir / 's_f32UIToCommonNaN.c',
+    sfspedir / 's_commonNaNToF32UI.c',
+    sfspedir / 's_propagateNaNF32UI.c',
+    sfspedir / 's_f64UIToCommonNaN.c',
+    sfspedir / 's_commonNaNToF64UI.c',
+    sfspedir / 's_propagateNaNF64UI.c',
+    sfspedir / 'extF80M_isSignalingNaN.c',
+    sfspedir / 's_extF80UIToCommonNaN.c',
+    sfspedir / 's_commonNaNToExtF80UI.c',
+    sfspedir / 's_propagateNaNExtF80UI.c',
+    sfspedir / 'f128M_isSignalingNaN.c',
+    sfspedir / 's_f128UIToCommonNaN.c',
+    sfspedir / 's_commonNaNToF128UI.c',
+    sfspedir / 's_propagateNaNF128UI.c',
+  ),
+  include_directories: sfinc,
+  c_args: fpcflags,
+)
+
+libsoftfloat_dep = declare_dependency(
+    link_with: libsoftfloat,
+    include_directories: sfinc,
+    compile_args: fpcflags)
diff --git a/subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt b/subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt
new file mode 100644
index 00000000000..868ae57e805
--- /dev/null
+++ b/subprojects/packagefiles/berkeley-softfloat-3/meson_options.txt
@@ -0,0 +1 @@
+option('defines', type : 'array', value : [])
diff --git a/subprojects/packagefiles/berkeley-testfloat-3/meson.build b/subprojects/packagefiles/berkeley-testfloat-3/meson.build
new file mode 100644
index 00000000000..a41673d616b
--- /dev/null
+++ b/subprojects/packagefiles/berkeley-testfloat-3/meson.build
@@ -0,0 +1,220 @@
+project('berkeley-testfloat-3',  'c',
+        default_options: ['warning_level=1', 'c_std=gnu99'])
+
+fpcflags = get_option('defines')
+
+platform_data = configuration_data()
+platform_data.set('INLINE', 'static inline')
+platform_data.set('LITTLEENDIAN', host_machine.endian() == 'little')
+configure_file(output: 'platform.h', configuration: platform_data)
+
+tfdir = 'source'
+tfinc = include_directories('.', tfdir)
+
+add_project_arguments(
+  [
+    '-Wno-implicit-fallthrough',
+    '-Wno-strict-prototypes',
+    '-Wno-unknown-pragmas',
+    '-Wno-uninitialized',
+    '-Wno-missing-prototypes',
+    '-Wno-return-type',
+    '-Wno-unused-function',
+    '-Wno-missing-format-attribute',
+    '-Wno-error',
+  ] + meson.get_compiler('c').get_supported_arguments('-Wno-ignored-pragmas'),
+  native: false, language: 'c')
+
+tfgencases = [
+  tfdir / 'genCases_ui32.c',
+  tfdir / 'genCases_ui64.c',
+  tfdir / 'genCases_i32.c',
+  tfdir / 'genCases_i64.c',
+  tfdir / 'genCases_f16.c',
+  tfdir / 'genCases_f32.c',
+  tfdir / 'genCases_f64.c',
+  tfdir / 'genCases_extF80.c',
+  tfdir / 'genCases_f128.c',
+]
+
+tfwritecase = [
+  tfdir / 'writeCase_a_ui32.c',
+  tfdir / 'writeCase_a_ui64.c',
+  tfdir / 'writeCase_a_f16.c',
+  tfdir / 'writeCase_ab_f16.c',
+  tfdir / 'writeCase_abc_f16.c',
+  tfdir / 'writeCase_a_f32.c',
+  tfdir / 'writeCase_ab_f32.c',
+  tfdir / 'writeCase_abc_f32.c',
+  tfdir / 'writeCase_a_f64.c',
+  tfdir / 'writeCase_ab_f64.c',
+  tfdir / 'writeCase_abc_f64.c',
+  tfdir / 'writeCase_a_extF80M.c',
+  tfdir / 'writeCase_ab_extF80M.c',
+  tfdir / 'writeCase_a_f128M.c',
+  tfdir / 'writeCase_ab_f128M.c',
+  tfdir / 'writeCase_abc_f128M.c',
+  tfdir / 'writeCase_z_bool.c',
+  tfdir / 'writeCase_z_ui32.c',
+  tfdir / 'writeCase_z_ui64.c',
+  tfdir / 'writeCase_z_f16.c',
+  tfdir / 'writeCase_z_f32.c',
+  tfdir / 'writeCase_z_f64.c',
+  tfdir / 'writeCase_z_extF80M.c',
+  tfdir / 'writeCase_z_f128M.c',
+]
+
+tftest = [
+  tfdir / 'test_a_ui32_z_f16.c',
+  tfdir / 'test_a_ui32_z_f32.c',
+  tfdir / 'test_a_ui32_z_f64.c',
+  tfdir / 'test_a_ui32_z_extF80.c',
+  tfdir / 'test_a_ui32_z_f128.c',
+  tfdir / 'test_a_ui64_z_f16.c',
+  tfdir / 'test_a_ui64_z_f32.c',
+  tfdir / 'test_a_ui64_z_f64.c',
+  tfdir / 'test_a_ui64_z_extF80.c',
+  tfdir / 'test_a_ui64_z_f128.c',
+  tfdir / 'test_a_i32_z_f16.c',
+  tfdir / 'test_a_i32_z_f32.c',
+  tfdir / 'test_a_i32_z_f64.c',
+  tfdir / 'test_a_i32_z_extF80.c',
+  tfdir / 'test_a_i32_z_f128.c',
+  tfdir / 'test_a_i64_z_f16.c',
+  tfdir / 'test_a_i64_z_f32.c',
+  tfdir / 'test_a_i64_z_f64.c',
+  tfdir / 'test_a_i64_z_extF80.c',
+  tfdir / 'test_a_i64_z_f128.c',
+  tfdir / 'test_a_f16_z_ui32_rx.c',
+  tfdir / 'test_a_f16_z_ui64_rx.c',
+  tfdir / 'test_a_f16_z_i32_rx.c',
+  tfdir / 'test_a_f16_z_i64_rx.c',
+  tfdir / 'test_a_f16_z_ui32_x.c',
+  tfdir / 'test_a_f16_z_ui64_x.c',
+  tfdir / 'test_a_f16_z_i32_x.c',
+  tfdir / 'test_a_f16_z_i64_x.c',
+  tfdir / 'test_a_f16_z_f32.c',
+  tfdir / 'test_a_f16_z_f64.c',
+  tfdir / 'test_a_f16_z_extF80.c',
+  tfdir / 'test_a_f16_z_f128.c',
+  tfdir / 'test_az_f16.c',
+  tfdir / 'test_az_f16_rx.c',
+  tfdir / 'test_abz_f16.c',
+  tfdir / 'test_abcz_f16.c',
+  tfdir / 'test_ab_f16_z_bool.c',
+  tfdir / 'test_a_f32_z_ui32_rx.c',
+  tfdir / 'test_a_f32_z_ui64_rx.c',
+  tfdir / 'test_a_f32_z_i32_rx.c',
+  tfdir / 'test_a_f32_z_i64_rx.c',
+  tfdir / 'test_a_f32_z_ui32_x.c',
+  tfdir / 'test_a_f32_z_ui64_x.c',
+  tfdir / 'test_a_f32_z_i32_x.c',
+  tfdir / 'test_a_f32_z_i64_x.c',
+  tfdir / 'test_a_f32_z_f16.c',
+  tfdir / 'test_a_f32_z_f64.c',
+  tfdir / 'test_a_f32_z_extF80.c',
+  tfdir / 'test_a_f32_z_f128.c',
+  tfdir / 'test_az_f32.c',
+  tfdir / 'test_az_f32_rx.c',
+  tfdir / 'test_abz_f32.c',
+  tfdir / 'test_abcz_f32.c',
+  tfdir / 'test_ab_f32_z_bool.c',
+  tfdir / 'test_a_f64_z_ui32_rx.c',
+  tfdir / 'test_a_f64_z_ui64_rx.c',
+  tfdir / 'test_a_f64_z_i32_rx.c',
+  tfdir / 'test_a_f64_z_i64_rx.c',
+  tfdir / 'test_a_f64_z_ui32_x.c',
+  tfdir / 'test_a_f64_z_ui64_x.c',
+  tfdir / 'test_a_f64_z_i32_x.c',
+  tfdir / 'test_a_f64_z_i64_x.c',
+  tfdir / 'test_a_f64_z_f16.c',
+  tfdir / 'test_a_f64_z_f32.c',
+  tfdir / 'test_a_f64_z_extF80.c',
+  tfdir / 'test_a_f64_z_f128.c',
+  tfdir / 'test_az_f64.c',
+  tfdir / 'test_az_f64_rx.c',
+  tfdir / 'test_abz_f64.c',
+  tfdir / 'test_abcz_f64.c',
+  tfdir / 'test_ab_f64_z_bool.c',
+  tfdir / 'test_a_extF80_z_ui32_rx.c',
+  tfdir / 'test_a_extF80_z_ui64_rx.c',
+  tfdir / 'test_a_extF80_z_i32_rx.c',
+  tfdir / 'test_a_extF80_z_i64_rx.c',
+  tfdir / 'test_a_extF80_z_ui32_x.c',
+  tfdir / 'test_a_extF80_z_ui64_x.c',
+  tfdir / 'test_a_extF80_z_i32_x.c',
+  tfdir / 'test_a_extF80_z_i64_x.c',
+  tfdir / 'test_a_extF80_z_f16.c',
+  tfdir / 'test_a_extF80_z_f32.c',
+  tfdir / 'test_a_extF80_z_f64.c',
+  tfdir / 'test_a_extF80_z_f128.c',
+  tfdir / 'test_az_extF80.c',
+  tfdir / 'test_az_extF80_rx.c',
+  tfdir / 'test_abz_extF80.c',
+  tfdir / 'test_ab_extF80_z_bool.c',
+  tfdir / 'test_a_f128_z_ui32_rx.c',
+  tfdir / 'test_a_f128_z_ui64_rx.c',
+  tfdir / 'test_a_f128_z_i32_rx.c',
+  tfdir / 'test_a_f128_z_i64_rx.c',
+  tfdir / 'test_a_f128_z_ui32_x.c',
+  tfdir / 'test_a_f128_z_ui64_x.c',
+  tfdir / 'test_a_f128_z_i32_x.c',
+  tfdir / 'test_a_f128_z_i64_x.c',
+  tfdir / 'test_a_f128_z_f16.c',
+  tfdir / 'test_a_f128_z_f32.c',
+  tfdir / 'test_a_f128_z_f64.c',
+  tfdir / 'test_a_f128_z_extF80.c',
+  tfdir / 'test_az_f128.c',
+  tfdir / 'test_az_f128_rx.c',
+  tfdir / 'test_abz_f128.c',
+  tfdir / 'test_abcz_f128.c',
+  tfdir / 'test_ab_f128_z_bool.c',
+]
+
+libsoftfloat_proj = subproject('berkeley-softfloat-3', required: true)
+libsoftfloat = libsoftfloat_proj.get_variable('libsoftfloat_dep')
+
+libtestfloat = static_library(
+  'testfloat',
+  files(
+    tfdir / 'uint128_inline.c',
+    tfdir / 'uint128.c',
+    tfdir / 'fail.c',
+    tfdir / 'functions_common.c',
+    tfdir / 'functionInfos.c',
+    tfdir / 'standardFunctionInfos.c',
+    tfdir / 'random.c',
+    tfdir / 'genCases_common.c',
+    tfgencases,
+    tfdir / 'genCases_writeTestsTotal.c',
+    tfdir / 'verCases_inline.c',
+    tfdir / 'verCases_common.c',
+    tfdir / 'verCases_writeFunctionName.c',
+    tfdir / 'readHex.c',
+    tfdir / 'writeHex.c',
+    tfwritecase,
+    tfdir / 'testLoops_common.c',
+    tftest,
+  ),
+  dependencies: libsoftfloat.partial_dependency(includes: true, compile_args: true),
+  c_args: fpcflags,
+)
+
+libtestfloat_dep = declare_dependency(
+    link_with: libtestfloat,
+    dependencies: libsoftfloat,
+    include_directories: tfinc,
+    compile_args: fpcflags)
+
+libslowfloat = static_library(
+  'slowfloat',
+  tfdir / 'slowfloat.c',
+  dependencies: libsoftfloat.partial_dependency(includes: true, compile_args: true),
+  c_args: fpcflags,
+)
+
+libslowfloat_dep = declare_dependency(
+    link_with: libslowfloat,
+    dependencies: libsoftfloat,
+    include_directories: tfinc,
+    compile_args: fpcflags)
diff --git a/subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt b/subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt
new file mode 100644
index 00000000000..868ae57e805
--- /dev/null
+++ b/subprojects/packagefiles/berkeley-testfloat-3/meson_options.txt
@@ -0,0 +1 @@
+option('defines', type : 'array', value : [])
diff --git a/tests/fp/meson.build b/tests/fp/meson.build
index f9ca6a93b49..cbc17392d67 100644
--- a/tests/fp/meson.build
+++ b/tests/fp/meson.build
@@ -7,13 +7,15 @@ if targetos == 'windows'
   subdir_done()
 endif
 
-fpcflags = [
+sfcflags = [
   # softfloat defines
   '-DSOFTFLOAT_ROUND_ODD',
   '-DINLINE_LEVEL=5',
   '-DSOFTFLOAT_FAST_DIV32TO16',
   '-DSOFTFLOAT_FAST_DIV64TO32',
   '-DSOFTFLOAT_FAST_INT64',
+]
+tfcflags = [
   # testfloat defines
   '-DFLOAT16',
   '-DFLOAT64',
@@ -23,523 +25,16 @@ fpcflags = [
   '-DLONG_DOUBLE_IS_EXTFLOAT80',
 ]
 
-sfdir = 'berkeley-softfloat-3/source'
-sfspedir = sfdir / '8086-SSE'
-tfdir = 'berkeley-testfloat-3/source'
+libsoftfloat_proj = subproject('berkeley-softfloat-3', required: true,
+    default_options: 'defines=' + ','.join(sfcflags))
+libsoftfloat = libsoftfloat_proj.get_variable('libsoftfloat_dep')
 
-sfinc = include_directories(sfdir / 'include', sfspedir)
+libtestfloat_proj = subproject('berkeley-testfloat-3', required: true,
+    default_options: 'defines=' + ','.join(tfcflags))
+libtestfloat = libtestfloat_proj.get_variable('libtestfloat_dep')
+libslowfloat = libtestfloat_proj.get_variable('libslowfloat_dep')
 
-tfcflags = [
-  '-Wno-implicit-fallthrough',
-  '-Wno-strict-prototypes',
-  '-Wno-unknown-pragmas',
-  '-Wno-uninitialized',
-  '-Wno-missing-prototypes',
-  '-Wno-return-type',
-  '-Wno-unused-function',
-  '-Wno-missing-format-attribute',
-  '-Wno-error',
-]
-
-if cc.get_id() == 'clang'
-  # Clang does not support '#pragma STDC FENV_ACCESS'
-  tfcflags += [ '-Wno-ignored-pragmas' ]
-endif
-
-tfgencases = [
-  tfdir / 'genCases_ui32.c',
-  tfdir / 'genCases_ui64.c',
-  tfdir / 'genCases_i32.c',
-  tfdir / 'genCases_i64.c',
-  tfdir / 'genCases_f16.c',
-  tfdir / 'genCases_f32.c',
-  tfdir / 'genCases_f64.c',
-  tfdir / 'genCases_extF80.c',
-  tfdir / 'genCases_f128.c',
-]
-
-tfwritecase = [
-  tfdir / 'writeCase_a_ui32.c',
-  tfdir / 'writeCase_a_ui64.c',
-  tfdir / 'writeCase_a_f16.c',
-  tfdir / 'writeCase_ab_f16.c',
-  tfdir / 'writeCase_abc_f16.c',
-  tfdir / 'writeCase_a_f32.c',
-  tfdir / 'writeCase_ab_f32.c',
-  tfdir / 'writeCase_abc_f32.c',
-  tfdir / 'writeCase_a_f64.c',
-  tfdir / 'writeCase_ab_f64.c',
-  tfdir / 'writeCase_abc_f64.c',
-  tfdir / 'writeCase_a_extF80M.c',
-  tfdir / 'writeCase_ab_extF80M.c',
-  tfdir / 'writeCase_a_f128M.c',
-  tfdir / 'writeCase_ab_f128M.c',
-  tfdir / 'writeCase_abc_f128M.c',
-  tfdir / 'writeCase_z_bool.c',
-  tfdir / 'writeCase_z_ui32.c',
-  tfdir / 'writeCase_z_ui64.c',
-  tfdir / 'writeCase_z_f16.c',
-  tfdir / 'writeCase_z_f32.c',
-  tfdir / 'writeCase_z_f64.c',
-  tfdir / 'writeCase_z_extF80M.c',
-  tfdir / 'writeCase_z_f128M.c',
-]
-
-tftest = [
-  tfdir / 'test_a_ui32_z_f16.c',
-  tfdir / 'test_a_ui32_z_f32.c',
-  tfdir / 'test_a_ui32_z_f64.c',
-  tfdir / 'test_a_ui32_z_extF80.c',
-  tfdir / 'test_a_ui32_z_f128.c',
-  tfdir / 'test_a_ui64_z_f16.c',
-  tfdir / 'test_a_ui64_z_f32.c',
-  tfdir / 'test_a_ui64_z_f64.c',
-  tfdir / 'test_a_ui64_z_extF80.c',
-  tfdir / 'test_a_ui64_z_f128.c',
-  tfdir / 'test_a_i32_z_f16.c',
-  tfdir / 'test_a_i32_z_f32.c',
-  tfdir / 'test_a_i32_z_f64.c',
-  tfdir / 'test_a_i32_z_extF80.c',
-  tfdir / 'test_a_i32_z_f128.c',
-  tfdir / 'test_a_i64_z_f16.c',
-  tfdir / 'test_a_i64_z_f32.c',
-  tfdir / 'test_a_i64_z_f64.c',
-  tfdir / 'test_a_i64_z_extF80.c',
-  tfdir / 'test_a_i64_z_f128.c',
-  tfdir / 'test_a_f16_z_ui32_rx.c',
-  tfdir / 'test_a_f16_z_ui64_rx.c',
-  tfdir / 'test_a_f16_z_i32_rx.c',
-  tfdir / 'test_a_f16_z_i64_rx.c',
-  tfdir / 'test_a_f16_z_ui32_x.c',
-  tfdir / 'test_a_f16_z_ui64_x.c',
-  tfdir / 'test_a_f16_z_i32_x.c',
-  tfdir / 'test_a_f16_z_i64_x.c',
-  tfdir / 'test_a_f16_z_f32.c',
-  tfdir / 'test_a_f16_z_f64.c',
-  tfdir / 'test_a_f16_z_extF80.c',
-  tfdir / 'test_a_f16_z_f128.c',
-  tfdir / 'test_az_f16.c',
-  tfdir / 'test_az_f16_rx.c',
-  tfdir / 'test_abz_f16.c',
-  tfdir / 'test_abcz_f16.c',
-  tfdir / 'test_ab_f16_z_bool.c',
-  tfdir / 'test_a_f32_z_ui32_rx.c',
-  tfdir / 'test_a_f32_z_ui64_rx.c',
-  tfdir / 'test_a_f32_z_i32_rx.c',
-  tfdir / 'test_a_f32_z_i64_rx.c',
-  tfdir / 'test_a_f32_z_ui32_x.c',
-  tfdir / 'test_a_f32_z_ui64_x.c',
-  tfdir / 'test_a_f32_z_i32_x.c',
-  tfdir / 'test_a_f32_z_i64_x.c',
-  tfdir / 'test_a_f32_z_f16.c',
-  tfdir / 'test_a_f32_z_f64.c',
-  tfdir / 'test_a_f32_z_extF80.c',
-  tfdir / 'test_a_f32_z_f128.c',
-  tfdir / 'test_az_f32.c',
-  tfdir / 'test_az_f32_rx.c',
-  tfdir / 'test_abz_f32.c',
-  tfdir / 'test_abcz_f32.c',
-  tfdir / 'test_ab_f32_z_bool.c',
-  tfdir / 'test_a_f64_z_ui32_rx.c',
-  tfdir / 'test_a_f64_z_ui64_rx.c',
-  tfdir / 'test_a_f64_z_i32_rx.c',
-  tfdir / 'test_a_f64_z_i64_rx.c',
-  tfdir / 'test_a_f64_z_ui32_x.c',
-  tfdir / 'test_a_f64_z_ui64_x.c',
-  tfdir / 'test_a_f64_z_i32_x.c',
-  tfdir / 'test_a_f64_z_i64_x.c',
-  tfdir / 'test_a_f64_z_f16.c',
-  tfdir / 'test_a_f64_z_f32.c',
-  tfdir / 'test_a_f64_z_extF80.c',
-  tfdir / 'test_a_f64_z_f128.c',
-  tfdir / 'test_az_f64.c',
-  tfdir / 'test_az_f64_rx.c',
-  tfdir / 'test_abz_f64.c',
-  tfdir / 'test_abcz_f64.c',
-  tfdir / 'test_ab_f64_z_bool.c',
-  tfdir / 'test_a_extF80_z_ui32_rx.c',
-  tfdir / 'test_a_extF80_z_ui64_rx.c',
-  tfdir / 'test_a_extF80_z_i32_rx.c',
-  tfdir / 'test_a_extF80_z_i64_rx.c',
-  tfdir / 'test_a_extF80_z_ui32_x.c',
-  tfdir / 'test_a_extF80_z_ui64_x.c',
-  tfdir / 'test_a_extF80_z_i32_x.c',
-  tfdir / 'test_a_extF80_z_i64_x.c',
-  tfdir / 'test_a_extF80_z_f16.c',
-  tfdir / 'test_a_extF80_z_f32.c',
-  tfdir / 'test_a_extF80_z_f64.c',
-  tfdir / 'test_a_extF80_z_f128.c',
-  tfdir / 'test_az_extF80.c',
-  tfdir / 'test_az_extF80_rx.c',
-  tfdir / 'test_abz_extF80.c',
-  tfdir / 'test_ab_extF80_z_bool.c',
-  tfdir / 'test_a_f128_z_ui32_rx.c',
-  tfdir / 'test_a_f128_z_ui64_rx.c',
-  tfdir / 'test_a_f128_z_i32_rx.c',
-  tfdir / 'test_a_f128_z_i64_rx.c',
-  tfdir / 'test_a_f128_z_ui32_x.c',
-  tfdir / 'test_a_f128_z_ui64_x.c',
-  tfdir / 'test_a_f128_z_i32_x.c',
-  tfdir / 'test_a_f128_z_i64_x.c',
-  tfdir / 'test_a_f128_z_f16.c',
-  tfdir / 'test_a_f128_z_f32.c',
-  tfdir / 'test_a_f128_z_f64.c',
-  tfdir / 'test_a_f128_z_extF80.c',
-  tfdir / 'test_az_f128.c',
-  tfdir / 'test_az_f128_rx.c',
-  tfdir / 'test_abz_f128.c',
-  tfdir / 'test_abcz_f128.c',
-  tfdir / 'test_ab_f128_z_bool.c',
-]
-
-libtestfloat = static_library(
-  'testfloat',
-  files(
-    tfdir / 'uint128_inline.c',
-    tfdir / 'uint128.c',
-    tfdir / 'fail.c',
-    tfdir / 'functions_common.c',
-    tfdir / 'functionInfos.c',
-    tfdir / 'standardFunctionInfos.c',
-    tfdir / 'random.c',
-    tfdir / 'genCases_common.c',
-    tfgencases,
-    tfdir / 'genCases_writeTestsTotal.c',
-    tfdir / 'verCases_inline.c',
-    tfdir / 'verCases_common.c',
-    tfdir / 'verCases_writeFunctionName.c',
-    tfdir / 'readHex.c',
-    tfdir / 'writeHex.c',
-    tfwritecase,
-    tfdir / 'testLoops_common.c',
-    tftest,
-  ),
-  include_directories: sfinc,
-  c_args: tfcflags + fpcflags,
-)
-
-sfcflags = [
-  '-Wno-implicit-fallthrough',
-  '-Wno-missing-prototypes',
-  '-Wno-redundant-decls',
-  '-Wno-return-type',
-  '-Wno-error',
-]
-
-libsoftfloat = static_library(
-  'softfloat',
-  files(
-    # primitives
-    sfdir / 's_eq128.c',
-    sfdir / 's_le128.c',
-    sfdir / 's_lt128.c',
-    sfdir / 's_shortShiftLeft128.c',
-    sfdir / 's_shortShiftRight128.c',
-    sfdir / 's_shortShiftRightJam64.c',
-    sfdir / 's_shortShiftRightJam64Extra.c',
-    sfdir / 's_shortShiftRightJam128.c',
-    sfdir / 's_shortShiftRightJam128Extra.c',
-    sfdir / 's_shiftRightJam32.c',
-    sfdir / 's_shiftRightJam64.c',
-    sfdir / 's_shiftRightJam64Extra.c',
-    sfdir / 's_shiftRightJam128.c',
-    sfdir / 's_shiftRightJam128Extra.c',
-    sfdir / 's_shiftRightJam256M.c',
-    sfdir / 's_countLeadingZeros8.c',
-    sfdir / 's_countLeadingZeros16.c',
-    sfdir / 's_countLeadingZeros32.c',
-    sfdir / 's_countLeadingZeros64.c',
-    sfdir / 's_add128.c',
-    sfdir / 's_add256M.c',
-    sfdir / 's_sub128.c',
-    sfdir / 's_sub256M.c',
-    sfdir / 's_mul64ByShifted32To128.c',
-    sfdir / 's_mul64To128.c',
-    sfdir / 's_mul128By32.c',
-    sfdir / 's_mul128To256M.c',
-    sfdir / 's_approxRecip_1Ks.c',
-    sfdir / 's_approxRecip32_1.c',
-    sfdir / 's_approxRecipSqrt_1Ks.c',
-    sfdir / 's_approxRecipSqrt32_1.c',
-    # others
-    sfdir / 's_roundToUI32.c',
-    sfdir / 's_roundToUI64.c',
-    sfdir / 's_roundToI32.c',
-    sfdir / 's_roundToI64.c',
-    sfdir / 's_normSubnormalF16Sig.c',
-    sfdir / 's_roundPackToF16.c',
-    sfdir / 's_normRoundPackToF16.c',
-    sfdir / 's_addMagsF16.c',
-    sfdir / 's_subMagsF16.c',
-    sfdir / 's_mulAddF16.c',
-    sfdir / 's_normSubnormalF32Sig.c',
-    sfdir / 's_roundPackToF32.c',
-    sfdir / 's_normRoundPackToF32.c',
-    sfdir / 's_addMagsF32.c',
-    sfdir / 's_subMagsF32.c',
-    sfdir / 's_mulAddF32.c',
-    sfdir / 's_normSubnormalF64Sig.c',
-    sfdir / 's_roundPackToF64.c',
-    sfdir / 's_normRoundPackToF64.c',
-    sfdir / 's_addMagsF64.c',
-    sfdir / 's_subMagsF64.c',
-    sfdir / 's_mulAddF64.c',
-    sfdir / 's_normSubnormalExtF80Sig.c',
-    sfdir / 's_roundPackToExtF80.c',
-    sfdir / 's_normRoundPackToExtF80.c',
-    sfdir / 's_addMagsExtF80.c',
-    sfdir / 's_subMagsExtF80.c',
-    sfdir / 's_normSubnormalF128Sig.c',
-    sfdir / 's_roundPackToF128.c',
-    sfdir / 's_normRoundPackToF128.c',
-    sfdir / 's_addMagsF128.c',
-    sfdir / 's_subMagsF128.c',
-    sfdir / 's_mulAddF128.c',
-    sfdir / 'softfloat_state.c',
-    sfdir / 'ui32_to_f16.c',
-    sfdir / 'ui32_to_f32.c',
-    sfdir / 'ui32_to_f64.c',
-    sfdir / 'ui32_to_extF80.c',
-    sfdir / 'ui32_to_extF80M.c',
-    sfdir / 'ui32_to_f128.c',
-    sfdir / 'ui32_to_f128M.c',
-    sfdir / 'ui64_to_f16.c',
-    sfdir / 'ui64_to_f32.c',
-    sfdir / 'ui64_to_f64.c',
-    sfdir / 'ui64_to_extF80.c',
-    sfdir / 'ui64_to_extF80M.c',
-    sfdir / 'ui64_to_f128.c',
-    sfdir / 'ui64_to_f128M.c',
-    sfdir / 'i32_to_f16.c',
-    sfdir / 'i32_to_f32.c',
-    sfdir / 'i32_to_f64.c',
-    sfdir / 'i32_to_extF80.c',
-    sfdir / 'i32_to_extF80M.c',
-    sfdir / 'i32_to_f128.c',
-    sfdir / 'i32_to_f128M.c',
-    sfdir / 'i64_to_f16.c',
-    sfdir / 'i64_to_f32.c',
-    sfdir / 'i64_to_f64.c',
-    sfdir / 'i64_to_extF80.c',
-    sfdir / 'i64_to_extF80M.c',
-    sfdir / 'i64_to_f128.c',
-    sfdir / 'i64_to_f128M.c',
-    sfdir / 'f16_to_ui32.c',
-    sfdir / 'f16_to_ui64.c',
-    sfdir / 'f16_to_i32.c',
-    sfdir / 'f16_to_i64.c',
-    sfdir / 'f16_to_ui32_r_minMag.c',
-    sfdir / 'f16_to_ui64_r_minMag.c',
-    sfdir / 'f16_to_i32_r_minMag.c',
-    sfdir / 'f16_to_i64_r_minMag.c',
-    sfdir / 'f16_to_f32.c',
-    sfdir / 'f16_to_f64.c',
-    sfdir / 'f16_to_extF80.c',
-    sfdir / 'f16_to_extF80M.c',
-    sfdir / 'f16_to_f128.c',
-    sfdir / 'f16_to_f128M.c',
-    sfdir / 'f16_roundToInt.c',
-    sfdir / 'f16_add.c',
-    sfdir / 'f16_sub.c',
-    sfdir / 'f16_mul.c',
-    sfdir / 'f16_mulAdd.c',
-    sfdir / 'f16_div.c',
-    sfdir / 'f16_rem.c',
-    sfdir / 'f16_sqrt.c',
-    sfdir / 'f16_eq.c',
-    sfdir / 'f16_le.c',
-    sfdir / 'f16_lt.c',
-    sfdir / 'f16_eq_signaling.c',
-    sfdir / 'f16_le_quiet.c',
-    sfdir / 'f16_lt_quiet.c',
-    sfdir / 'f16_isSignalingNaN.c',
-    sfdir / 'f32_to_ui32.c',
-    sfdir / 'f32_to_ui64.c',
-    sfdir / 'f32_to_i32.c',
-    sfdir / 'f32_to_i64.c',
-    sfdir / 'f32_to_ui32_r_minMag.c',
-    sfdir / 'f32_to_ui64_r_minMag.c',
-    sfdir / 'f32_to_i32_r_minMag.c',
-    sfdir / 'f32_to_i64_r_minMag.c',
-    sfdir / 'f32_to_f16.c',
-    sfdir / 'f32_to_f64.c',
-    sfdir / 'f32_to_extF80.c',
-    sfdir / 'f32_to_extF80M.c',
-    sfdir / 'f32_to_f128.c',
-    sfdir / 'f32_to_f128M.c',
-    sfdir / 'f32_roundToInt.c',
-    sfdir / 'f32_add.c',
-    sfdir / 'f32_sub.c',
-    sfdir / 'f32_mul.c',
-    sfdir / 'f32_mulAdd.c',
-    sfdir / 'f32_div.c',
-    sfdir / 'f32_rem.c',
-    sfdir / 'f32_sqrt.c',
-    sfdir / 'f32_eq.c',
-    sfdir / 'f32_le.c',
-    sfdir / 'f32_lt.c',
-    sfdir / 'f32_eq_signaling.c',
-    sfdir / 'f32_le_quiet.c',
-    sfdir / 'f32_lt_quiet.c',
-    sfdir / 'f32_isSignalingNaN.c',
-    sfdir / 'f64_to_ui32.c',
-    sfdir / 'f64_to_ui64.c',
-    sfdir / 'f64_to_i32.c',
-    sfdir / 'f64_to_i64.c',
-    sfdir / 'f64_to_ui32_r_minMag.c',
-    sfdir / 'f64_to_ui64_r_minMag.c',
-    sfdir / 'f64_to_i32_r_minMag.c',
-    sfdir / 'f64_to_i64_r_minMag.c',
-    sfdir / 'f64_to_f16.c',
-    sfdir / 'f64_to_f32.c',
-    sfdir / 'f64_to_extF80.c',
-    sfdir / 'f64_to_extF80M.c',
-    sfdir / 'f64_to_f128.c',
-    sfdir / 'f64_to_f128M.c',
-    sfdir / 'f64_roundToInt.c',
-    sfdir / 'f64_add.c',
-    sfdir / 'f64_sub.c',
-    sfdir / 'f64_mul.c',
-    sfdir / 'f64_mulAdd.c',
-    sfdir / 'f64_div.c',
-    sfdir / 'f64_rem.c',
-    sfdir / 'f64_sqrt.c',
-    sfdir / 'f64_eq.c',
-    sfdir / 'f64_le.c',
-    sfdir / 'f64_lt.c',
-    sfdir / 'f64_eq_signaling.c',
-    sfdir / 'f64_le_quiet.c',
-    sfdir / 'f64_lt_quiet.c',
-    sfdir / 'f64_isSignalingNaN.c',
-    sfdir / 'extF80_to_ui32.c',
-    sfdir / 'extF80_to_ui64.c',
-    sfdir / 'extF80_to_i32.c',
-    sfdir / 'extF80_to_i64.c',
-    sfdir / 'extF80_to_ui32_r_minMag.c',
-    sfdir / 'extF80_to_ui64_r_minMag.c',
-    sfdir / 'extF80_to_i32_r_minMag.c',
-    sfdir / 'extF80_to_i64_r_minMag.c',
-    sfdir / 'extF80_to_f16.c',
-    sfdir / 'extF80_to_f32.c',
-    sfdir / 'extF80_to_f64.c',
-    sfdir / 'extF80_to_f128.c',
-    sfdir / 'extF80_roundToInt.c',
-    sfdir / 'extF80_add.c',
-    sfdir / 'extF80_sub.c',
-    sfdir / 'extF80_mul.c',
-    sfdir / 'extF80_div.c',
-    sfdir / 'extF80_rem.c',
-    sfdir / 'extF80_sqrt.c',
-    sfdir / 'extF80_eq.c',
-    sfdir / 'extF80_le.c',
-    sfdir / 'extF80_lt.c',
-    sfdir / 'extF80_eq_signaling.c',
-    sfdir / 'extF80_le_quiet.c',
-    sfdir / 'extF80_lt_quiet.c',
-    sfdir / 'extF80_isSignalingNaN.c',
-    sfdir / 'extF80M_to_ui32.c',
-    sfdir / 'extF80M_to_ui64.c',
-    sfdir / 'extF80M_to_i32.c',
-    sfdir / 'extF80M_to_i64.c',
-    sfdir / 'extF80M_to_ui32_r_minMag.c',
-    sfdir / 'extF80M_to_ui64_r_minMag.c',
-    sfdir / 'extF80M_to_i32_r_minMag.c',
-    sfdir / 'extF80M_to_i64_r_minMag.c',
-    sfdir / 'extF80M_to_f16.c',
-    sfdir / 'extF80M_to_f32.c',
-    sfdir / 'extF80M_to_f64.c',
-    sfdir / 'extF80M_to_f128M.c',
-    sfdir / 'extF80M_roundToInt.c',
-    sfdir / 'extF80M_add.c',
-    sfdir / 'extF80M_sub.c',
-    sfdir / 'extF80M_mul.c',
-    sfdir / 'extF80M_div.c',
-    sfdir / 'extF80M_rem.c',
-    sfdir / 'extF80M_sqrt.c',
-    sfdir / 'extF80M_eq.c',
-    sfdir / 'extF80M_le.c',
-    sfdir / 'extF80M_lt.c',
-    sfdir / 'extF80M_eq_signaling.c',
-    sfdir / 'extF80M_le_quiet.c',
-    sfdir / 'extF80M_lt_quiet.c',
-    sfdir / 'f128_to_ui32.c',
-    sfdir / 'f128_to_ui64.c',
-    sfdir / 'f128_to_i32.c',
-    sfdir / 'f128_to_i64.c',
-    sfdir / 'f128_to_ui32_r_minMag.c',
-    sfdir / 'f128_to_ui64_r_minMag.c',
-    sfdir / 'f128_to_i32_r_minMag.c',
-    sfdir / 'f128_to_i64_r_minMag.c',
-    sfdir / 'f128_to_f16.c',
-    sfdir / 'f128_to_f32.c',
-    sfdir / 'f128_to_extF80.c',
-    sfdir / 'f128_to_f64.c',
-    sfdir / 'f128_roundToInt.c',
-    sfdir / 'f128_add.c',
-    sfdir / 'f128_sub.c',
-    sfdir / 'f128_mul.c',
-    sfdir / 'f128_mulAdd.c',
-    sfdir / 'f128_div.c',
-    sfdir / 'f128_rem.c',
-    sfdir / 'f128_sqrt.c',
-    sfdir / 'f128_eq.c',
-    sfdir / 'f128_le.c',
-    sfdir / 'f128_lt.c',
-    sfdir / 'f128_eq_signaling.c',
-    sfdir / 'f128_le_quiet.c',
-    sfdir / 'f128_lt_quiet.c',
-    sfdir / 'f128_isSignalingNaN.c',
-    sfdir / 'f128M_to_ui32.c',
-    sfdir / 'f128M_to_ui64.c',
-    sfdir / 'f128M_to_i32.c',
-    sfdir / 'f128M_to_i64.c',
-    sfdir / 'f128M_to_ui32_r_minMag.c',
-    sfdir / 'f128M_to_ui64_r_minMag.c',
-    sfdir / 'f128M_to_i32_r_minMag.c',
-    sfdir / 'f128M_to_i64_r_minMag.c',
-    sfdir / 'f128M_to_f16.c',
-    sfdir / 'f128M_to_f32.c',
-    sfdir / 'f128M_to_extF80M.c',
-    sfdir / 'f128M_to_f64.c',
-    sfdir / 'f128M_roundToInt.c',
-    sfdir / 'f128M_add.c',
-    sfdir / 'f128M_sub.c',
-    sfdir / 'f128M_mul.c',
-    sfdir / 'f128M_mulAdd.c',
-    sfdir / 'f128M_div.c',
-    sfdir / 'f128M_rem.c',
-    sfdir / 'f128M_sqrt.c',
-    sfdir / 'f128M_eq.c',
-    sfdir / 'f128M_le.c',
-    sfdir / 'f128M_lt.c',
-    sfdir / 'f128M_eq_signaling.c',
-    sfdir / 'f128M_le_quiet.c',
-    sfdir / 'f128M_lt_quiet.c',
-    # spe
-    sfspedir / 'softfloat_raiseFlags.c',
-    sfspedir / 's_f16UIToCommonNaN.c',
-    sfspedir / 's_commonNaNToF16UI.c',
-    sfspedir / 's_propagateNaNF16UI.c',
-    sfspedir / 's_f32UIToCommonNaN.c',
-    sfspedir / 's_commonNaNToF32UI.c',
-    sfspedir / 's_propagateNaNF32UI.c',
-    sfspedir / 's_f64UIToCommonNaN.c',
-    sfspedir / 's_commonNaNToF64UI.c',
-    sfspedir / 's_propagateNaNF64UI.c',
-    sfspedir / 'extF80M_isSignalingNaN.c',
-    sfspedir / 's_extF80UIToCommonNaN.c',
-    sfspedir / 's_commonNaNToExtF80UI.c',
-    sfspedir / 's_propagateNaNExtF80UI.c',
-    sfspedir / 'f128M_isSignalingNaN.c',
-    sfspedir / 's_f128UIToCommonNaN.c',
-    sfspedir / 's_commonNaNToF128UI.c',
-    sfspedir / 's_propagateNaNF128UI.c',
-  ),
-  include_directories: sfinc,
-  c_args: sfcflags + fpcflags,
-)
-
-fpcflags += [
+fpcflags = [
   # work around TARGET_* poisoning
   '-DHW_POISON_H',
   # define a target to match testfloat's implementation-defined choices, such as
@@ -551,10 +46,8 @@ fpcflags += [
 
 fptest = executable(
   'fp-test',
-  ['fp-test.c', tfdir / 'slowfloat.c', '../../fpu/softfloat.c'],
-  link_with: [libtestfloat, libsoftfloat],
-  dependencies: [qemuutil],
-  include_directories: [sfinc, include_directories(tfdir)],
+  ['fp-test.c', '../../fpu/softfloat.c'],
+  dependencies: [qemuutil, libsoftfloat, libtestfloat, libslowfloat],
   c_args: fpcflags,
 )
 softfloat_conv_tests = {
@@ -636,18 +129,14 @@ test('fp-test-mulAdd', fptest,
 executable(
   'fp-bench',
   ['fp-bench.c', '../../fpu/softfloat.c'],
-  link_with: [libtestfloat, libsoftfloat],
-  dependencies: [qemuutil],
-  include_directories: [sfinc, include_directories(tfdir)],
+  dependencies: [qemuutil, libtestfloat, libsoftfloat],
   c_args: fpcflags,
 )
 
 fptestlog2 = executable(
   'fp-test-log2',
   ['fp-test-log2.c', '../../fpu/softfloat.c'],
-  link_with: [libsoftfloat],
-  dependencies: [qemuutil],
-  include_directories: [sfinc],
+  dependencies: [qemuutil, libsoftfloat],
   c_args: fpcflags,
 )
 test('fp-test-log2', fptestlog2,
-- 
2.40.1

[PULL 20/21] build: remove git submodule handling from main makefile

[Paolo Bonzini]

The only remaining user of submodules at build time is roms/SLOF,
which is handled in pc-bios/s390-ccw/Makefile.  Remove the relevant
code from the main makefile.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml |  3 +--
 Makefile                            | 10 ----------
 configure                           |  7 -------
 meson.build                         |  1 -
 4 files changed, 1 insertion(+), 20 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index d01d504ec5f..76ff1dfcb6f 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -42,8 +42,7 @@
   stage: test
   image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
   script:
-    - scripts/git-submodule.sh update
-        roms/SLOF $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
+    - scripts/git-submodule.sh update roms/SLOF
     - meson subprojects download $(cd build/subprojects && echo *)
     - cd build
     - find . -type f -exec touch {} +
diff --git a/Makefile b/Makefile
index d68196acb9e..b22bf6fba12 100644
--- a/Makefile
+++ b/Makefile
@@ -45,16 +45,6 @@ include config-host.mak
 include Makefile.prereqs
 Makefile.prereqs: config-host.mak
 
-git-submodule-update:
-.git-submodule-status: git-submodule-update config-host.mak
-Makefile: .git-submodule-status
-
-.PHONY: git-submodule-update
-git-submodule-update:
-ifneq ($(GIT_SUBMODULES_ACTION),ignore)
-	$(quiet-@)GIT=git "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)
-endif
-
 # 0. ensure the build tree is okay
 
 # Check that we're not trying to do an out-of-tree build from
diff --git a/configure b/configure
index 4dad32938df..bc0660f5a32 100755
--- a/configure
+++ b/configure
@@ -253,7 +253,6 @@ else
     git_submodules_action="ignore"
 fi
 
-git_submodules=""
 git="git"
 debug_tcg="no"
 docs="auto"
@@ -1657,18 +1656,12 @@ fi
 #######################################
 # generate config-host.mak
 
-if ! (GIT=git "$source_path/scripts/git-submodule.sh" "$git_submodules_action" "$git_submodules"); then
-    exit 1
-fi
-
 config_host_mak="config-host.mak"
 
 echo "# Automatically generated by configure - do not modify" > $config_host_mak
 echo >> $config_host_mak
 
 echo all: >> $config_host_mak
-echo "GIT_SUBMODULES=$git_submodules" >> $config_host_mak
-echo "GIT_SUBMODULES_ACTION=$git_submodules_action" >> $config_host_mak
 
 if test "$debug_tcg" = "yes" ; then
   echo "CONFIG_DEBUG_TCG=y" >> $config_host_mak
diff --git a/meson.build b/meson.build
index 74fb147fde3..6cd29c11b05 100644
--- a/meson.build
+++ b/meson.build
@@ -3976,7 +3976,6 @@ endif
 summary_info = {}
 summary_info += {'Build directory':   meson.current_build_dir()}
 summary_info += {'Source path':       meson.current_source_dir()}
-summary_info += {'GIT submodules':    config_host['GIT_SUBMODULES']}
 summary_info += {'Download dependencies': get_option('wrap_mode') != 'nodownload'}
 summary(summary_info, bool_yn: true, section: 'Build environment')
 
-- 
2.40.1

[PULL 21/21] configure: remove --with-git-submodules=

[Paolo Bonzini]

Reuse --enable/--disable-download to control git submodules as well.
Adjust the error messages of git-submodule.sh to refer to the new
option.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure                                     | 40 +++++--------------
 .../ci/org.centos/stream/8/x86_64/configure   |  1 -
 scripts/git-submodule.sh                      |  8 ++--
 3 files changed, 12 insertions(+), 37 deletions(-)

diff --git a/configure b/configure
index bc0660f5a32..8765b88e12f 100755
--- a/configure
+++ b/configure
@@ -246,13 +246,7 @@ for opt do
 done
 
 
-if test -e "$source_path/.git"
-then
-    git_submodules_action="update"
-else
-    git_submodules_action="ignore"
-fi
-
+git_submodules_action="update"
 git="git"
 debug_tcg="no"
 docs="auto"
@@ -738,12 +732,9 @@ for opt do
   ;;
   --disable-cfi) cfi="false"
   ;;
-  --with-git-submodules=*)
-      git_submodules_action="$optarg"
+  --disable-download) download="disabled"; git_submodules_action=validate;
   ;;
-  --disable-download) download="disabled"
-  ;;
-  --enable-download) download="enabled"
+  --enable-download) download="enabled"; git_submodules_action=update;
   ;;
   --enable-plugins) if test "$mingw32" = "yes"; then
                         error_exit "TCG plugins not currently supported on Windows platforms"
@@ -765,6 +756,11 @@ for opt do
   esac
 done
 
+if ! test -e "$source_path/.git"
+then
+    git_submodules_action="ignore"
+fi
+
 # test for any invalid configuration combinations
 if test "$plugins" = "yes" -a "$tcg" = "disabled"; then
     error_exit "Can't enable plugins on non-TCG builds"
@@ -796,21 +792,6 @@ then
     exit 1
 fi
 
-case $git_submodules_action in
-    update|validate)
-        if test ! -e "$source_path/.git" || ! has git; then
-            echo "ERROR: cannot $git_submodules_action git submodules without .git"
-            exit 1
-        fi
-    ;;
-    ignore)
-    ;;
-    *)
-        echo "ERROR: invalid --with-git-submodules= value '$git_submodules_action'"
-        exit 1
-    ;;
-esac
-
 default_target_list=""
 mak_wilds=""
 
@@ -877,9 +858,6 @@ Advanced options (experts only):
   --python=PYTHON          use specified python [$python]
   --ninja=NINJA            use specified ninja [$ninja]
   --smbd=SMBD              use specified smbd [$smbd]
-  --with-git-submodules=update   update git submodules (default if .git dir exists)
-  --with-git-submodules=validate fail if git submodules are not up to date
-  --with-git-submodules=ignore   do not update or check git submodules (default if no .git dir)
   --static                 enable static build [$static]
   --bindir=PATH            install binaries in PATH
   --with-suffix=SUFFIX     suffix for QEMU data inside datadir/libdir/sysconfdir/docdir [$qemu_suffix]
@@ -1024,7 +1002,7 @@ fi
 # Consult white-list to determine whether to enable werror
 # by default.  Only enable by default for git builds
 if test -z "$werror" ; then
-    if test "$git_submodules_action" != "ignore" && \
+    if test -e "$source_path/.git" && \
         { test "$linux" = "yes" || test "$mingw32" = "yes"; }; then
         werror="yes"
     else
diff --git a/scripts/ci/org.centos/stream/8/x86_64/configure b/scripts/ci/org.centos/stream/8/x86_64/configure
index de76510978f..d02b09a4b9b 100755
--- a/scripts/ci/org.centos/stream/8/x86_64/configure
+++ b/scripts/ci/org.centos/stream/8/x86_64/configure
@@ -29,7 +29,6 @@
 --extra-cflags="-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" \
 --with-suffix="qemu-kvm" \
 --firmwarepath=/usr/share/qemu-firmware \
---with-git-submodules=update \
 --target-list="x86_64-softmmu" \
 --block-drv-rw-whitelist="qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle,gluster" \
 --audio-drv-list="" \
diff --git a/scripts/git-submodule.sh b/scripts/git-submodule.sh
index 38b55c90e11..11fad2137cd 100755
--- a/scripts/git-submodule.sh
+++ b/scripts/git-submodule.sh
@@ -9,7 +9,7 @@ command=$1
 shift
 maybe_modules="$@"
 
-# if --with-git-submodules=ignore, do nothing
+# if not running in a git checkout, do nothing
 test "$command" = "ignore" && exit 0
 
 test -z "$GIT" && GIT=$(command -v git)
@@ -24,7 +24,7 @@ update_error() {
     echo "enable use of a transparent proxy), please disable automatic"
     echo "GIT submodule checkout with:"
     echo
-    echo " $ ./configure --with-git-submodules=validate"
+    echo " $ ./configure --disable-download"
     echo
     echo "and then manually update submodules prior to running make, with:"
     echo
@@ -39,9 +39,7 @@ validate_error() {
         echo "configured for validate only. Please run"
         echo "  scripts/git-submodule.sh update $maybe_modules"
         echo "from the source directory or call configure with"
-        echo "  --with-git-submodules=update"
-        echo "To disable GIT submodules validation, use"
-        echo "  --with-git-submodules=ignore"
+        echo "  --enable-download"
     fi
     exit 1
 }
-- 
2.40.1

Re: [PULL 00/21] (Mostly) build system patches for 2023-06-06

[Richard Henderson]

On 6/6/23 07:30, Paolo Bonzini wrote:
> The following changes since commit 369081c4558e7e940fa36ce59bf17b2e390f55d3:
> 
>    Merge tag 'pull-tcg-20230605' ofhttps://gitlab.com/rth7680/qemu  into staging (2023-06-05 13:16:56 -0700)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/bonzini/qemu.git  tags/for-upstream
> 
> for you to fetch changes up to 6f3ae23b29ad5831902e3ecdc7e443bbbf295bde:
> 
>    configure: remove --with-git-submodules= (2023-06-06 16:30:01 +0200)
> 
> ----------------------------------------------------------------
> * finish atomics revamp
> * meson.build tweaks
> * revert avocado update
> * always upgrade/downgrade locally installed Python packages
> * switch from submodules to subprojects
> * remove --with-git= option
> * rename --enable-pypi to --enable-download, control submodules and subprojects too

Applied, thanks.  Please update https://wiki.qemu.org/ChangeLog/8.1 as appropriate.


r~

Re: [PULL 16/21] meson: subprojects: replace submodules with wrap files

[Richard Henderson]

On 6/6/23 07:31, Paolo Bonzini wrote:
> +if ! test -f "$source_path/subprojects/keycodemapdb/README" \
> +    && test -f "$download" = disabled

test -f x = y

is some sort of paste-o.  Remove the -f, I'm sure.


r~

Re: [PULL 14/21] git-submodule: allow partial update of .git-submodule-status

[Peter Maydell]

On Tue, 6 Jun 2023 at 15:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> Allow a specific subdirectory to run git-submodule.sh with only a
> subset of submodules, without removing the others from the
> .git-submodule-status file.
>
> This also allows scripts/git-submodule.sh to be more lenient:
> validating an empty set of submodules is not a mistake.
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

I've noticed that when doing a build sometimes this
script now produces an error:

make: Entering directory
'/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/clang'
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/pc-bios/s390-ccw/../../scripts/git-submodule.sh:
106: read: arg count
[1/154] Generating qemu-version.h with a custom command (wrapped by
meson to capture output)
[etc]


> -    $GIT submodule status $modules > "${substat}"
> -    test $? -ne 0 && update_error "failed to save git submodule status" >&2
> +    (while read -r; do

This is because "read" without a variable name argument is
a non-POSIX extension. I think the fix to this is to say
"read -r REPLY" rather than omitting the variable name and
assuming it will default to REPLY.

> +        for module in $modules; do
> +            case $REPLY in
> +                *" $module "*) continue 2 ;;
> +            esac
> +        done
> +        printf '%s\n' "$REPLY"
> +    done

I'll send a patch shortly.

thanks
-- PMM