[PULL 0/6] Biuld system and CI changes for 2023-11-10

[PULL 0/6] Biuld system and CI changes for 2023-11-10

[Paolo Bonzini]

The following changes since commit ad6ef0a42e314a8c6ac6c96d5f6e607a1e5644b5:

  Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging (2023-11-09 08:26:01 +0800)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 1d802d050caeff83add1054bee1fc9f98e59a3f2:

  .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma) (2023-11-10 10:39:05 +0100)

----------------------------------------------------------------
* document what configure does with virtual environments
* Bump known good meson version to v1.2.3
* Upgrade macOS to 13 (Ventura) and Add manual testing of macOS 14 (Sonoma)
* use simple assertions instead of Coverity models

----------------------------------------------------------------
Paolo Bonzini (2):
      tests: respect --enable/--disable-download for Avocado
      docs: document what configure does with virtual environments

Philippe Mathieu-Daudé (3):
      buildsys: Bump known good meson version to v1.2.3
      .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
      .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)

Vladimir Sementsov-Ogievskiy (1):
      coverity: physmem: use simple assertions instead of modelling

 .gitlab-ci.d/cirrus.yml                            |  22 +++++-
 .../cirrus/{macos-12.vars => macos-13.vars}        |   2 +-
 .gitlab-ci.d/cirrus/macos-14.vars                  |  16 ++++
 configure                                          |   9 ++-
 docs/devel/build-system.rst                        |  88 ++++++++++++++++++++-
 python/scripts/vendor.py                           |   4 +-
 python/wheels/meson-0.63.3-py3-none-any.whl        | Bin 926526 -> 0 bytes
 python/wheels/meson-1.2.3-py3-none-any.whl         | Bin 0 -> 964928 bytes
 pythondeps.toml                                    |   5 +-
 scripts/coverity-scan/model.c                      |  88 ---------------------
 system/physmem.c                                   |  22 ++++++
 tests/Makefile.include                             |   2 +-
 tests/lcitool/libvirt-ci                           |   2 +-
 tests/lcitool/refresh                              |   3 +-
 14 files changed, 157 insertions(+), 106 deletions(-)
 rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
 create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
 delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
 create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
-- 
2.41.0

[PULL 1/6] coverity: physmem: use simple assertions instead of modelling

[Paolo Bonzini]

From: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

Unfortunately Coverity doesn't follow the logic aroung "len" and "l"
variables in stacks finishing with flatview_{read,write}_continue() and
generate a lot of OVERRUN false-positives. When small buffer (2 or 4
bytes) is passed to mem read/write path, Coverity assumes the worst
case of sz=8 in stn_he_p()/ldn_he_p() (defined in
include/qemu/bswap.h), and reports buffer overrun.

To silence these false-positives we have model functions, which hide
real logic from Coverity.

However, it turned out that these new two assertions are enough to
quiet Coverity.

Assertions are better than hiding the logic, so let's drop the
modelling and move to assertions for memory r/w call stacks.

After patch, the sequence

 cov-make-library --output-file /tmp/master.xmldb \
    scripts/coverity-scan/model.c
 cov-build --dir ~/covtmp/master make -j9
 cov-analyze --user-model-file /tmp/master.xmldb \
    --dir ~/covtmp/master --all --strip-path "$(pwd)
 cov-format-errors --dir ~/covtmp/master \
    --html-output ~/covtmp/master_html_report

Generate for me the same big set of CIDs excepept for 6 disappeared (so
it becomes even better).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: David Hildenbrand <david@redhat.com>
Message-ID: <20231005140326.332830-1-vsementsov@yandex-team.ru>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/coverity-scan/model.c | 88 -----------------------------------
 system/physmem.c              | 22 +++++++++
 2 files changed, 22 insertions(+), 88 deletions(-)

diff --git a/scripts/coverity-scan/model.c b/scripts/coverity-scan/model.c
index 686d1a30089..a064d840849 100644
--- a/scripts/coverity-scan/model.c
+++ b/scripts/coverity-scan/model.c
@@ -42,94 +42,6 @@ typedef _Bool bool;
 
 typedef struct va_list_str *va_list;
 
-/* exec.c */
-
-typedef struct AddressSpace AddressSpace;
-typedef struct MemoryRegionCache MemoryRegionCache;
-typedef uint64_t hwaddr;
-typedef uint32_t MemTxResult;
-typedef struct MemTxAttrs {} MemTxAttrs;
-
-static void __bufwrite(uint8_t *buf, ssize_t len)
-{
-    int first, last;
-    __coverity_negative_sink__(len);
-    if (len == 0) return;
-    buf[0] = first;
-    buf[len-1] = last;
-    __coverity_writeall__(buf);
-}
-
-static void __bufread(uint8_t *buf, ssize_t len)
-{
-    __coverity_negative_sink__(len);
-    if (len == 0) return;
-    int first = buf[0];
-    int last = buf[len-1];
-}
-
-MemTxResult address_space_read_cached(MemoryRegionCache *cache, hwaddr addr,
-                                      MemTxAttrs attrs,
-                                      void *buf, int len)
-{
-    MemTxResult result;
-    // TODO: investigate impact of treating reads as producing
-    // tainted data, with __coverity_tainted_data_argument__(buf).
-    __bufwrite(buf, len);
-    return result;
-}
-
-MemTxResult address_space_write_cached(MemoryRegionCache *cache, hwaddr addr,
-                                MemTxAttrs attrs,
-                                const void *buf, int len)
-{
-    MemTxResult result;
-    __bufread(buf, len);
-    return result;
-}
-
-MemTxResult address_space_rw_cached(MemoryRegionCache *cache, hwaddr addr,
-                                    MemTxAttrs attrs,
-                                    void *buf, int len, bool is_write)
-{
-    if (is_write) {
-        return address_space_write_cached(cache, addr, attrs, buf, len);
-    } else {
-        return address_space_read_cached(cache, addr, attrs, buf, len);
-    }
-}
-
-MemTxResult address_space_read(AddressSpace *as, hwaddr addr,
-                               MemTxAttrs attrs,
-                               void *buf, int len)
-{
-    MemTxResult result;
-    // TODO: investigate impact of treating reads as producing
-    // tainted data, with __coverity_tainted_data_argument__(buf).
-    __bufwrite(buf, len);
-    return result;
-}
-
-MemTxResult address_space_write(AddressSpace *as, hwaddr addr,
-                                MemTxAttrs attrs,
-                                const void *buf, int len)
-{
-    MemTxResult result;
-    __bufread(buf, len);
-    return result;
-}
-
-MemTxResult address_space_rw(AddressSpace *as, hwaddr addr,
-                             MemTxAttrs attrs,
-                             void *buf, int len, bool is_write)
-{
-    if (is_write) {
-        return address_space_write(as, addr, attrs, buf, len);
-    } else {
-        return address_space_read(as, addr, attrs, buf, len);
-    }
-}
-
 /* Tainting */
 
 typedef struct {} name2keysym_t;
diff --git a/system/physmem.c b/system/physmem.c
index fc2b0fee018..a63853a7bc9 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2699,6 +2699,17 @@ static MemTxResult flatview_write_continue(FlatView *fv, hwaddr addr,
             l = memory_access_size(mr, l, addr1);
             /* XXX: could force current_cpu to NULL to avoid
                potential bugs */
+
+            /*
+             * Assure Coverity (and ourselves) that we are not going to OVERRUN
+             * the buffer by following ldn_he_p().
+             */
+#ifdef QEMU_STATIC_ANALYSIS
+            assert((l == 1 && len >= 1) ||
+                   (l == 2 && len >= 2) ||
+                   (l == 4 && len >= 4) ||
+                   (l == 8 && len >= 8));
+#endif
             val = ldn_he_p(buf, l);
             result |= memory_region_dispatch_write(mr, addr1, val,
                                                    size_memop(l), attrs);
@@ -2769,6 +2780,17 @@ MemTxResult flatview_read_continue(FlatView *fv, hwaddr addr,
             l = memory_access_size(mr, l, addr1);
             result |= memory_region_dispatch_read(mr, addr1, &val,
                                                   size_memop(l), attrs);
+
+            /*
+             * Assure Coverity (and ourselves) that we are not going to OVERRUN
+             * the buffer by following stn_he_p().
+             */
+#ifdef QEMU_STATIC_ANALYSIS
+            assert((l == 1 && len >= 1) ||
+                   (l == 2 && len >= 2) ||
+                   (l == 4 && len >= 4) ||
+                   (l == 8 && len >= 8));
+#endif
             stn_he_p(buf, l, val);
         } else {
             /* RAM case */
-- 
2.41.0

[PULL 2/6] tests: respect --enable/--disable-download for Avocado

[Paolo Bonzini]

Pass the content of $mkvenv_flags (which is either "--online"
or empty) down to tests/Makefile.include.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure              | 9 +++++----
 tests/Makefile.include | 2 +-
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index abcb199aa87..2da3c278db6 100755
--- a/configure
+++ b/configure
@@ -968,14 +968,14 @@ meson="$(cd pyvenv/bin; pwd)/meson"
 
 # Conditionally ensure Sphinx is installed.
 
-mkvenv_flags=""
-if test "$download" = "enabled" -a "$docs" = "enabled" ; then
-    mkvenv_flags="--online"
+mkvenv_online_flag=""
+if test "$download" = "enabled" ; then
+    mkvenv_online_flag=" --online"
 fi
 
 if test "$docs" != "disabled" ; then
     if ! $mkvenv ensuregroup \
-         $mkvenv_flags \
+         $(test "$docs" = "enabled" && echo "$mkvenv_online_flag") \
          ${source_path}/pythondeps.toml docs;
     then
         if test "$docs" = "enabled" ; then
@@ -1631,6 +1631,7 @@ if test "$container" != no; then
 fi
 echo "SUBDIRS=$subdirs" >> $config_host_mak
 echo "PYTHON=$python" >> $config_host_mak
+echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
 echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
 echo "MESON=$meson" >> $config_host_mak
 echo "NINJA=$ninja" >> $config_host_mak
diff --git a/tests/Makefile.include b/tests/Makefile.include
index dab1989a071..c9d1674bd07 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -111,7 +111,7 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
 
 $(TESTS_VENV_TOKEN): $(SRC_PATH)/pythondeps.toml
 	$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
-	$(PYTHON) python/scripts/mkvenv.py ensuregroup --online $< avocado
+	$(MKVENV_ENSUREGROUP) $< avocado
 	$(call quiet-command, touch $@)
 
 $(TESTS_RESULTS_DIR):
-- 
2.41.0

[PULL 3/6] docs: document what configure does with virtual environments

[Paolo Bonzini]

Given the recent confusion around how QEMU detects the system
Meson installation, and/or decides to install its own, it is
time to fill in the "Python virtual environments and the QEMU
build system" section of the documentation.

As a curiosity, a first and partial draft of the text was generated
by an LLM[1].  It required quite a bit of editing and probably did not
save much time, but some expressions do remain in the finished text.

[1] https://chat.openai.com/share/42c1500d-71c1-480b-bab9-7ccc2c155365

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/devel/build-system.rst | 88 +++++++++++++++++++++++++++++++++++--
 pythondeps.toml             |  3 +-
 2 files changed, 87 insertions(+), 4 deletions(-)

diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 21f78da7d1d..43d6005881e 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -122,10 +122,78 @@ functioning.  These are performed using a few more helper functions:
    indicated by $TMPC.
 
 
-Python virtual environments and the QEMU build system
------------------------------------------------------
+Python virtual environments and the build process
+-------------------------------------------------
+
+An important step in ``configure`` is to create a Python virtual
+environment (venv) during the configuration phase.  The Python interpreter
+comes from the ``--python`` command line option, the ``$PYTHON`` variable
+from the environment, or the system PATH, in this order.  The venv resides
+in the ``pyvenv`` directory in the build tree, and provides consistency
+in how the build process runs Python code.
+
+At this stage, ``configure`` also queries the chosen Python interpreter
+about QEMU's build dependencies.  Note that the build process does  *not*
+look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
+likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
+This avoids a potential mismatch, where Meson and Sphinx binaries on the
+PATH might operate in a different Python environment than the one chosen
+by the user during the build process.  On the other hand, it introduces
+a potential source of confusion where the user installs a dependency but
+``configure`` is not able to find it.  When this happens, the dependency
+was installed in the ``site-packages`` directory of another interpreter,
+or with the wrong ``pip`` program.
+
+If a package is available for the chosen interpreter, ``configure``
+prepares a small script that invokes it from the venv itself[#distlib]_.
+If not, ``configure`` can also optionally install dependencies in the
+virtual environment with ``pip``, either from wheels in ``python/wheels``
+or by downloading the package with PyPI.  Downloading can be disabled with
+``--disable-download``; and anyway, it only happens when a ``configure``
+option (currently, only ``--enable-docs``) is explicitly enabled but
+the dependencies are not present[#pip]_.
+
+.. [#distlib] The scripts are created based on the package's metadata,
+              specifically the ``console_script`` entry points.  This is the
+              same mechanism that ``pip`` uses when installing a package.
+              Currently, in all cases it would be possible to use ``python -m``
+              instead of an entry point script, which makes this approach a
+              bit overkill.  On the other hand, creating the scripts is
+              future proof and it makes the contents of the ``pyvenv/bin``
+              directory more informative.  Portability is also not an issue,
+              because the Python Packaging Authority provides a package
+              ``distlib.scripts`` to perform this task.
+
+.. [#pip] ``pip`` might also be used when running ``make check-avocado``
+           if downloading is enabled, to ensure that Avocado is
+           available.
+
+The required versions of the packages are stored in a configuration file
+``pythondeps.toml``.  The format is custom to QEMU, but it is documented
+at the top of the file itself and it should be easy to understand.  The
+requirements should make it possible to use the version that is packaged
+that is provided by supported distros.
+
+When dependencies are downloaded, instead, ``configure`` uses a "known
+good" version that is also listed in ``pythondeps.toml``.  In this
+scenario, ``pythondeps.toml`` behaves like the "lock file" used by
+``cargo``, ``poetry`` or other dependency management systems.
+
+
+Bundled Python packages
+-----------------------
+
+Python packages that are **mandatory** dependencies to build QEMU,
+but are not available in all supported distros, are bundled with the
+QEMU sources.  Currently this includes Meson (outdated in CentOS 8
+and derivatives, Ubuntu 20.04 and 22.04, and openSUSE Leap) and tomli
+(absent in Ubuntu 20.04).
+
+If you need to update these, please do so by modifying and rerunning
+``python/scripts/vendor.py``.  This script embeds the sha256 hash of
+package sources and checks it.  The pypi.org web site provides an easy
+way to retrieve the sha256 hash of the sources.
 
-TBD
 
 Stage 2: Meson
 ==============
@@ -376,6 +444,15 @@ This is needed to obey the --python= option passed to the configure
 script, which may point to something other than the first python3
 binary on the path.
 
+By the time Meson runs, Python dependencies are available in the virtual
+environment and should be invoked through the scripts that ``configure``
+places under ``pyvenv``.  One way to do so is as follows, using Meson's
+``find_program`` function::
+
+  sphinx_build = find_program(
+       fs.parent(python.full_path()) / 'sphinx-build',
+       required: get_option('docs'))
+
 
 Stage 3: Make
 =============
@@ -434,6 +511,11 @@ number of dynamically created files listed later.
   executables.  Build rules for various subdirectories are included in
   other meson.build files spread throughout the QEMU source tree.
 
+``python/scripts/mkvenv.py``
+  A wrapper for the Python ``venv`` and ``distlib.scripts`` packages.
+  It handles creating the virtual environment, creating scripts in
+  ``pyvenv/bin``, and calling ``pip`` to install dependencies.
+
 ``tests/Makefile.include``
   Rules for external test harnesses. These include the TCG tests
   and the Avocado-based integration tests.
diff --git a/pythondeps.toml b/pythondeps.toml
index 0a35ebcf9f0..4beadfd96f5 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -10,7 +10,8 @@
 # - accepted: accepted versions when using a system package
 # - installed: fixed version to install in the virtual environment
 #              if a system package is not found; if not specified,
-#              the minimum and maximum
+#              defaults to the same as "accepted" or, if also missing,
+#              to the newest version available on PyPI.
 # - canary: if specified, use this program name to present more
 #           precise error diagnostics to the user.  For example,
 #           'sphinx-build' can be used as a bellwether for the
-- 
2.41.0

[PULL 4/6] buildsys: Bump known good meson version to v1.2.3

[Paolo Bonzini]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

We need meson v1.2.3 to build QEMU on macOS Sonoma.  It
also builds fine all our CI jobs (as tested by also bumping
"accepted" in pythondeps.toml), so let's use it as our
"good enough" packaged wheel.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1939
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 python/scripts/vendor.py                    |   4 ++--
 python/wheels/meson-0.63.3-py3-none-any.whl | Bin 926526 -> 0 bytes
 python/wheels/meson-1.2.3-py3-none-any.whl  | Bin 0 -> 964928 bytes
 pythondeps.toml                             |   2 +-
 4 files changed, 3 insertions(+), 3 deletions(-)
 delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
 create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl

diff --git a/python/scripts/vendor.py b/python/scripts/vendor.py
index 76274871170..1038b14ae0c 100755
--- a/python/scripts/vendor.py
+++ b/python/scripts/vendor.py
@@ -41,8 +41,8 @@ def main() -> int:
     parser.parse_args()
 
     packages = {
-        "meson==0.63.3":
-        "d677b809c4895dcbaac9bf6c43703fcb3609a4b24c6057c78f828590049cf43a",
+        "meson==1.2.3":
+        "4533a43c34548edd1f63a276a42690fce15bde9409bcf20c4b8fa3d7e4d7cac1",
 
         "tomli==2.0.1":
         "939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc",
diff --git a/python/wheels/meson-0.63.3-py3-none-any.whl b/python/wheels/meson-0.63.3-py3-none-any.whl
deleted file mode 100644
index 8a191e3a200eba6783297676729ec85ceb96f89d..0000000000000000000000000000000000000000
Binary files a/python/wheels/meson-0.63.3-py3-none-any.whl and /dev/null differ
diff --git a/python/wheels/meson-1.2.3-py3-none-any.whl b/python/wheels/meson-1.2.3-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..a8b84e5f114ad3fbec8ae56008426a90bccfc168
GIT binary patch
Binary files /dev/null and b/python/wheels/meson-1.2.3-py3-none-any.whl differ
diff --git a/pythondeps.toml b/pythondeps.toml
index 4beadfd96f5..0e884159993 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -19,7 +19,7 @@
 
 [meson]
 # The install key should match the version in python/wheels/
-meson = { accepted = ">=0.63.0", installed = "0.63.3", canary = "meson" }
+meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" }
 
 [docs]
 sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
-- 
2.41.0

[PULL 5/6] .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)

[Paolo Bonzini]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

macOS 14 "Sonoma" was released on September 2023 [1].

According to QEMU's support policy, we stop supporting the
previous major release two years after the the new major
release has been published. Replace the macOS 12 (Monterey)
testing by macOS 13 (Ventura, released on October 2022, [2]).

Refresh the generated files by running:

  $ make lcitool-refresh

[1] https://www.apple.com/newsroom/2023/09/macos-sonoma-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20231108162022.76189-1-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/cirrus.yml                              | 6 +++---
 .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} | 2 +-
 tests/lcitool/refresh                                | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)
 rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index e7f1f83c2c4..07dc6edae1c 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -59,13 +59,13 @@ x64-freebsd-13-build:
     INSTALL_COMMAND: pkg install -y
     TEST_TARGETS: check
 
-aarch64-macos-12-base-build:
+aarch64-macos-13-base-build:
   extends: .cirrus_build_job
   variables:
-    NAME: macos-12
+    NAME: macos-13
     CIRRUS_VM_INSTANCE_TYPE: macos_instance
     CIRRUS_VM_IMAGE_SELECTOR: image
-    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-monterey-base:latest
+    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-ventura-base:latest
     CIRRUS_VM_CPUS: 12
     CIRRUS_VM_RAM: 24G
     UPDATE_COMMAND: brew update
diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-13.vars
similarity index 95%
rename from .gitlab-ci.d/cirrus/macos-12.vars
rename to .gitlab-ci.d/cirrus/macos-13.vars
index 5f3fb346d1d..534f0299560 100644
--- a/.gitlab-ci.d/cirrus/macos-12.vars
+++ b/.gitlab-ci.d/cirrus/macos-13.vars
@@ -1,6 +1,6 @@
 # THIS FILE WAS AUTO-GENERATED
 #
-#  $ lcitool variables macos-12 qemu
+#  $ lcitool variables macos-13 qemu
 #
 # https://gitlab.com/libvirt/libvirt-ci
 
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 92e7d30982e..2259f131b48 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -196,7 +196,7 @@ try:
     # Cirrus packages lists for GitLab
     #
     generate_cirrus("freebsd-13")
-    generate_cirrus("macos-12")
+    generate_cirrus("macos-13")
 
     #
     # VM packages lists
-- 
2.41.0

[PULL 6/6] .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)

[Paolo Bonzini]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

Upgrade libvirt-ci so it covers macOS 14. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sonoma release. Refresh the
lci-tool generated files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-3-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitlab-ci.d/cirrus.yml           | 16 ++++++++++++++++
 .gitlab-ci.d/cirrus/macos-14.vars | 16 ++++++++++++++++
 tests/lcitool/libvirt-ci          |  2 +-
 tests/lcitool/refresh             |  1 +
 4 files changed, 34 insertions(+), 1 deletion(-)
 create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 07dc6edae1c..64f2e25afab 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -74,6 +74,22 @@ aarch64-macos-13-base-build:
     PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
     TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
 
+aarch64-macos-14-base-build:
+  extends: .cirrus_build_job
+  variables:
+    NAME: macos-14
+    CIRRUS_VM_INSTANCE_TYPE: macos_instance
+    CIRRUS_VM_IMAGE_SELECTOR: image
+    CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-sonoma-base:latest
+    CIRRUS_VM_CPUS: 12
+    CIRRUS_VM_RAM: 24G
+    UPDATE_COMMAND: brew update
+    INSTALL_COMMAND: brew install
+    PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
+    PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
+    TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
+    QEMU_JOB_OPTIONAL: 1
+
 
 # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
 .cirrus_kvm_job:
diff --git a/.gitlab-ci.d/cirrus/macos-14.vars b/.gitlab-ci.d/cirrus/macos-14.vars
new file mode 100644
index 00000000000..43070f4a265
--- /dev/null
+++ b/.gitlab-ci.d/cirrus/macos-14.vars
@@ -0,0 +1,16 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+#  $ lcitool variables macos-14 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
+CCACHE='/opt/homebrew/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
+MAKE='/opt/homebrew/bin/gmake'
+NINJA='/opt/homebrew/bin/ninja'
+PACKAGING_COMMAND='brew'
+PIP3='/opt/homebrew/bin/pip3'
+PKGS='bash bc bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo json-c libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 xorriso zlib zstd'
+PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme tomli'
+PYTHON='/opt/homebrew/bin/python3'
diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
index 36bc517161c..77c800186f3 160000
--- a/tests/lcitool/libvirt-ci
+++ b/tests/lcitool/libvirt-ci
@@ -1 +1 @@
-Subproject commit 36bc517161c45ead20224d47f2dc4fa428af6724
+Subproject commit 77c800186f34b21be7660750577cc5582a914deb
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 2259f131b48..e11d0ba381f 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -197,6 +197,7 @@ try:
     #
     generate_cirrus("freebsd-13")
     generate_cirrus("macos-13")
+    generate_cirrus("macos-14")
 
     #
     # VM packages lists
-- 
2.41.0

Re: [PULL 0/6] Biuld system and CI changes for 2023-11-10

[Stefan Hajnoczi]

On Sat, 11 Nov 2023 at 23:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit ad6ef0a42e314a8c6ac6c96d5f6e607a1e5644b5:
>
>   Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging (2023-11-09 08:26:01 +0800)
>
> are available in the Git repository at:
>
>   https://gitlab.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 1d802d050caeff83add1054bee1fc9f98e59a3f2:
>
>   .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma) (2023-11-10 10:39:05 +0100)
>
> ----------------------------------------------------------------
> * document what configure does with virtual environments
> * Bump known good meson version to v1.2.3
> * Upgrade macOS to 13 (Ventura) and Add manual testing of macOS 14 (Sonoma)
> * use simple assertions instead of Coverity models

I'm not sure which commit causes this, but there is an msys failure:

"gcc" "-m64" "-mcx16" -o tests/plugin/libbb.dll
plugins/qemu_plugin_api.lib tests/plugin/libbb.dll.p/bb.c.obj
tests/plugin/libbb.dll.p/.._.._contrib_plugins_win32_linker.c.obj
"-Wl,--allow-shlib-undefined" "-shared" "-Wl,--start-group"
"-Wl,--out-implib=tests/plugin/libbb.dll.a" "-fstack-protector-strong"
"-Wl,--no-seh" "-Wl,--nxcompat" "-Wl,--dynamicbase"
"-Wl,--high-entropy-va" "-Wl,--warn-common"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libglib-2.0.dll.a"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libintl.dll.a"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libgmodule-2.0.dll.a"
"-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32"
"-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32" "-Wl,--end-group"
C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
tests/plugin/libbb.dll.p/bb.c.obj:bb.c:(.text+0x219): undefined
reference to `__imp_qemu_plugin_outs'

Here is the failing output with this PR applied:
https://gitlab.com/qemu-project/qemu/-/jobs/5516759417

Here is the master branch's passing output:
https://gitlab.com/qemu-project/qemu/-/jobs/5513282154

Please take a look. Thanks!

Stefan

>
> ----------------------------------------------------------------
> Paolo Bonzini (2):
>       tests: respect --enable/--disable-download for Avocado
>       docs: document what configure does with virtual environments
>
> Philippe Mathieu-Daudé (3):
>       buildsys: Bump known good meson version to v1.2.3
>       .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
>       .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
>
> Vladimir Sementsov-Ogievskiy (1):
>       coverity: physmem: use simple assertions instead of modelling
>
>  .gitlab-ci.d/cirrus.yml                            |  22 +++++-
>  .../cirrus/{macos-12.vars => macos-13.vars}        |   2 +-
>  .gitlab-ci.d/cirrus/macos-14.vars                  |  16 ++++
>  configure                                          |   9 ++-
>  docs/devel/build-system.rst                        |  88 ++++++++++++++++++++-
>  python/scripts/vendor.py                           |   4 +-
>  python/wheels/meson-0.63.3-py3-none-any.whl        | Bin 926526 -> 0 bytes
>  python/wheels/meson-1.2.3-py3-none-any.whl         | Bin 0 -> 964928 bytes
>  pythondeps.toml                                    |   5 +-
>  scripts/coverity-scan/model.c                      |  88 ---------------------
>  system/physmem.c                                   |  22 ++++++
>  tests/Makefile.include                             |   2 +-
>  tests/lcitool/libvirt-ci                           |   2 +-
>  tests/lcitool/refresh                              |   3 +-
>  14 files changed, 157 insertions(+), 106 deletions(-)
>  rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
>  create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
>  delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
>  create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
> --
> 2.41.0
>
>

Re: [PULL 0/6] Biuld system and CI changes for 2023-11-10

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 3453 bytes --]

Il dom 12 nov 2023, 13:06 Stefan Hajnoczi <stefanha@gmail.com> ha scritto:

> On Sat, 11 Nov 2023 at 23:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> > The following changes since commit
> ad6ef0a42e314a8c6ac6c96d5f6e607a1e5644b5:
> >
> >   Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into
> staging (2023-11-09 08:26:01 +0800)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/bonzini/qemu.git tags/for-upstream
> >
> > for you to fetch changes up to 1d802d050caeff83add1054bee1fc9f98e59a3f2:
> >
> >   .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
> (2023-11-10 10:39:05 +0100)
> >
> > ----------------------------------------------------------------
> > * document what configure does with virtual environments
> > * Bump known good meson version to v1.2.3
> > * Upgrade macOS to 13 (Ventura) and Add manual testing of macOS 14
> (Sonoma)
> > * use simple assertions instead of Coverity models
>
> I'm not sure which commit causes this, but there is an msys failure:
>
> Here is the failing output with this PR applied:
> https://gitlab.com/qemu-project/qemu/-/jobs/5516759417
>
> Here is the master branch's passing output:
> https://gitlab.com/qemu-project/qemu/-/jobs/5513282154
>
> Please take a look. Thanks!
>

Looks like the series causes issue #1972 to appear more prominently. I will
resubmit once it's fixed (a patch is already available).

Paolo


> Stefan
>
> >
> > ----------------------------------------------------------------
> > Paolo Bonzini (2):
> >       tests: respect --enable/--disable-download for Avocado
> >       docs: document what configure does with virtual environments
> >
> > Philippe Mathieu-Daudé (3):
> >       buildsys: Bump known good meson version to v1.2.3
> >       .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
> >       .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
> >
> > Vladimir Sementsov-Ogievskiy (1):
> >       coverity: physmem: use simple assertions instead of modelling
> >
> >  .gitlab-ci.d/cirrus.yml                            |  22 +++++-
> >  .../cirrus/{macos-12.vars => macos-13.vars}        |   2 +-
> >  .gitlab-ci.d/cirrus/macos-14.vars                  |  16 ++++
> >  configure                                          |   9 ++-
> >  docs/devel/build-system.rst                        |  88
> ++++++++++++++++++++-
> >  python/scripts/vendor.py                           |   4 +-
> >  python/wheels/meson-0.63.3-py3-none-any.whl        | Bin 926526 -> 0
> bytes
> >  python/wheels/meson-1.2.3-py3-none-any.whl         | Bin 0 -> 964928
> bytes
> >  pythondeps.toml                                    |   5 +-
> >  scripts/coverity-scan/model.c                      |  88
> ---------------------
> >  system/physmem.c                                   |  22 ++++++
> >  tests/Makefile.include                             |   2 +-
> >  tests/lcitool/libvirt-ci                           |   2 +-
> >  tests/lcitool/refresh                              |   3 +-
> >  14 files changed, 157 insertions(+), 106 deletions(-)
> >  rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
> >  create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
> >  delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
> >  create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
> > --
> > 2.41.0
> >
> >
>
>

[-- Attachment #2: Type: text/html, Size: 5050 bytes --]