* [PULL 1/6] coverity: physmem: use simple assertions instead of modelling
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-11 15:29 ` [PULL 2/6] tests: respect --enable/--disable-download for Avocado Paolo Bonzini
` (5 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Vladimir Sementsov-Ogievskiy, David Hildenbrand
From: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Unfortunately Coverity doesn't follow the logic aroung "len" and "l"
variables in stacks finishing with flatview_{read,write}_continue() and
generate a lot of OVERRUN false-positives. When small buffer (2 or 4
bytes) is passed to mem read/write path, Coverity assumes the worst
case of sz=8 in stn_he_p()/ldn_he_p() (defined in
include/qemu/bswap.h), and reports buffer overrun.
To silence these false-positives we have model functions, which hide
real logic from Coverity.
However, it turned out that these new two assertions are enough to
quiet Coverity.
Assertions are better than hiding the logic, so let's drop the
modelling and move to assertions for memory r/w call stacks.
After patch, the sequence
cov-make-library --output-file /tmp/master.xmldb \
scripts/coverity-scan/model.c
cov-build --dir ~/covtmp/master make -j9
cov-analyze --user-model-file /tmp/master.xmldb \
--dir ~/covtmp/master --all --strip-path "$(pwd)
cov-format-errors --dir ~/covtmp/master \
--html-output ~/covtmp/master_html_report
Generate for me the same big set of CIDs excepept for 6 disappeared (so
it becomes even better).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: David Hildenbrand <david@redhat.com>
Message-ID: <20231005140326.332830-1-vsementsov@yandex-team.ru>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
scripts/coverity-scan/model.c | 88 -----------------------------------
system/physmem.c | 22 +++++++++
2 files changed, 22 insertions(+), 88 deletions(-)
diff --git a/scripts/coverity-scan/model.c b/scripts/coverity-scan/model.c
index 686d1a30089..a064d840849 100644
--- a/scripts/coverity-scan/model.c
+++ b/scripts/coverity-scan/model.c
@@ -42,94 +42,6 @@ typedef _Bool bool;
typedef struct va_list_str *va_list;
-/* exec.c */
-
-typedef struct AddressSpace AddressSpace;
-typedef struct MemoryRegionCache MemoryRegionCache;
-typedef uint64_t hwaddr;
-typedef uint32_t MemTxResult;
-typedef struct MemTxAttrs {} MemTxAttrs;
-
-static void __bufwrite(uint8_t *buf, ssize_t len)
-{
- int first, last;
- __coverity_negative_sink__(len);
- if (len == 0) return;
- buf[0] = first;
- buf[len-1] = last;
- __coverity_writeall__(buf);
-}
-
-static void __bufread(uint8_t *buf, ssize_t len)
-{
- __coverity_negative_sink__(len);
- if (len == 0) return;
- int first = buf[0];
- int last = buf[len-1];
-}
-
-MemTxResult address_space_read_cached(MemoryRegionCache *cache, hwaddr addr,
- MemTxAttrs attrs,
- void *buf, int len)
-{
- MemTxResult result;
- // TODO: investigate impact of treating reads as producing
- // tainted data, with __coverity_tainted_data_argument__(buf).
- __bufwrite(buf, len);
- return result;
-}
-
-MemTxResult address_space_write_cached(MemoryRegionCache *cache, hwaddr addr,
- MemTxAttrs attrs,
- const void *buf, int len)
-{
- MemTxResult result;
- __bufread(buf, len);
- return result;
-}
-
-MemTxResult address_space_rw_cached(MemoryRegionCache *cache, hwaddr addr,
- MemTxAttrs attrs,
- void *buf, int len, bool is_write)
-{
- if (is_write) {
- return address_space_write_cached(cache, addr, attrs, buf, len);
- } else {
- return address_space_read_cached(cache, addr, attrs, buf, len);
- }
-}
-
-MemTxResult address_space_read(AddressSpace *as, hwaddr addr,
- MemTxAttrs attrs,
- void *buf, int len)
-{
- MemTxResult result;
- // TODO: investigate impact of treating reads as producing
- // tainted data, with __coverity_tainted_data_argument__(buf).
- __bufwrite(buf, len);
- return result;
-}
-
-MemTxResult address_space_write(AddressSpace *as, hwaddr addr,
- MemTxAttrs attrs,
- const void *buf, int len)
-{
- MemTxResult result;
- __bufread(buf, len);
- return result;
-}
-
-MemTxResult address_space_rw(AddressSpace *as, hwaddr addr,
- MemTxAttrs attrs,
- void *buf, int len, bool is_write)
-{
- if (is_write) {
- return address_space_write(as, addr, attrs, buf, len);
- } else {
- return address_space_read(as, addr, attrs, buf, len);
- }
-}
-
/* Tainting */
typedef struct {} name2keysym_t;
diff --git a/system/physmem.c b/system/physmem.c
index fc2b0fee018..a63853a7bc9 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2699,6 +2699,17 @@ static MemTxResult flatview_write_continue(FlatView *fv, hwaddr addr,
l = memory_access_size(mr, l, addr1);
/* XXX: could force current_cpu to NULL to avoid
potential bugs */
+
+ /*
+ * Assure Coverity (and ourselves) that we are not going to OVERRUN
+ * the buffer by following ldn_he_p().
+ */
+#ifdef QEMU_STATIC_ANALYSIS
+ assert((l == 1 && len >= 1) ||
+ (l == 2 && len >= 2) ||
+ (l == 4 && len >= 4) ||
+ (l == 8 && len >= 8));
+#endif
val = ldn_he_p(buf, l);
result |= memory_region_dispatch_write(mr, addr1, val,
size_memop(l), attrs);
@@ -2769,6 +2780,17 @@ MemTxResult flatview_read_continue(FlatView *fv, hwaddr addr,
l = memory_access_size(mr, l, addr1);
result |= memory_region_dispatch_read(mr, addr1, &val,
size_memop(l), attrs);
+
+ /*
+ * Assure Coverity (and ourselves) that we are not going to OVERRUN
+ * the buffer by following stn_he_p().
+ */
+#ifdef QEMU_STATIC_ANALYSIS
+ assert((l == 1 && len >= 1) ||
+ (l == 2 && len >= 2) ||
+ (l == 4 && len >= 4) ||
+ (l == 8 && len >= 8));
+#endif
stn_he_p(buf, l, val);
} else {
/* RAM case */
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PULL 2/6] tests: respect --enable/--disable-download for Avocado
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
2023-11-11 15:29 ` [PULL 1/6] coverity: physmem: use simple assertions instead of modelling Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-11 15:29 ` [PULL 3/6] docs: document what configure does with virtual environments Paolo Bonzini
` (4 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel
Pass the content of $mkvenv_flags (which is either "--online"
or empty) down to tests/Makefile.include.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
configure | 9 +++++----
tests/Makefile.include | 2 +-
2 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/configure b/configure
index abcb199aa87..2da3c278db6 100755
--- a/configure
+++ b/configure
@@ -968,14 +968,14 @@ meson="$(cd pyvenv/bin; pwd)/meson"
# Conditionally ensure Sphinx is installed.
-mkvenv_flags=""
-if test "$download" = "enabled" -a "$docs" = "enabled" ; then
- mkvenv_flags="--online"
+mkvenv_online_flag=""
+if test "$download" = "enabled" ; then
+ mkvenv_online_flag=" --online"
fi
if test "$docs" != "disabled" ; then
if ! $mkvenv ensuregroup \
- $mkvenv_flags \
+ $(test "$docs" = "enabled" && echo "$mkvenv_online_flag") \
${source_path}/pythondeps.toml docs;
then
if test "$docs" = "enabled" ; then
@@ -1631,6 +1631,7 @@ if test "$container" != no; then
fi
echo "SUBDIRS=$subdirs" >> $config_host_mak
echo "PYTHON=$python" >> $config_host_mak
+echo "MKVENV_ENSUREGROUP=$mkvenv ensuregroup $mkvenv_online_flag" >> $config_host_mak
echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
echo "MESON=$meson" >> $config_host_mak
echo "NINJA=$ninja" >> $config_host_mak
diff --git a/tests/Makefile.include b/tests/Makefile.include
index dab1989a071..c9d1674bd07 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -111,7 +111,7 @@ quiet-venv-pip = $(quiet-@)$(call quiet-command-run, \
$(TESTS_VENV_TOKEN): $(SRC_PATH)/pythondeps.toml
$(call quiet-venv-pip,install -e "$(SRC_PATH)/python/")
- $(PYTHON) python/scripts/mkvenv.py ensuregroup --online $< avocado
+ $(MKVENV_ENSUREGROUP) $< avocado
$(call quiet-command, touch $@)
$(TESTS_RESULTS_DIR):
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PULL 3/6] docs: document what configure does with virtual environments
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
2023-11-11 15:29 ` [PULL 1/6] coverity: physmem: use simple assertions instead of modelling Paolo Bonzini
2023-11-11 15:29 ` [PULL 2/6] tests: respect --enable/--disable-download for Avocado Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-11 15:29 ` [PULL 4/6] buildsys: Bump known good meson version to v1.2.3 Paolo Bonzini
` (3 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel
Given the recent confusion around how QEMU detects the system
Meson installation, and/or decides to install its own, it is
time to fill in the "Python virtual environments and the QEMU
build system" section of the documentation.
As a curiosity, a first and partial draft of the text was generated
by an LLM[1]. It required quite a bit of editing and probably did not
save much time, but some expressions do remain in the finished text.
[1] https://chat.openai.com/share/42c1500d-71c1-480b-bab9-7ccc2c155365
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
docs/devel/build-system.rst | 88 +++++++++++++++++++++++++++++++++++--
pythondeps.toml | 3 +-
2 files changed, 87 insertions(+), 4 deletions(-)
diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 21f78da7d1d..43d6005881e 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -122,10 +122,78 @@ functioning. These are performed using a few more helper functions:
indicated by $TMPC.
-Python virtual environments and the QEMU build system
------------------------------------------------------
+Python virtual environments and the build process
+-------------------------------------------------
+
+An important step in ``configure`` is to create a Python virtual
+environment (venv) during the configuration phase. The Python interpreter
+comes from the ``--python`` command line option, the ``$PYTHON`` variable
+from the environment, or the system PATH, in this order. The venv resides
+in the ``pyvenv`` directory in the build tree, and provides consistency
+in how the build process runs Python code.
+
+At this stage, ``configure`` also queries the chosen Python interpreter
+about QEMU's build dependencies. Note that the build process does *not*
+look for ``meson``, ``sphinx-build`` or ``avocado`` binaries in the PATH;
+likewise, there are no options such as ``--meson`` or ``--sphinx-build``.
+This avoids a potential mismatch, where Meson and Sphinx binaries on the
+PATH might operate in a different Python environment than the one chosen
+by the user during the build process. On the other hand, it introduces
+a potential source of confusion where the user installs a dependency but
+``configure`` is not able to find it. When this happens, the dependency
+was installed in the ``site-packages`` directory of another interpreter,
+or with the wrong ``pip`` program.
+
+If a package is available for the chosen interpreter, ``configure``
+prepares a small script that invokes it from the venv itself[#distlib]_.
+If not, ``configure`` can also optionally install dependencies in the
+virtual environment with ``pip``, either from wheels in ``python/wheels``
+or by downloading the package with PyPI. Downloading can be disabled with
+``--disable-download``; and anyway, it only happens when a ``configure``
+option (currently, only ``--enable-docs``) is explicitly enabled but
+the dependencies are not present[#pip]_.
+
+.. [#distlib] The scripts are created based on the package's metadata,
+ specifically the ``console_script`` entry points. This is the
+ same mechanism that ``pip`` uses when installing a package.
+ Currently, in all cases it would be possible to use ``python -m``
+ instead of an entry point script, which makes this approach a
+ bit overkill. On the other hand, creating the scripts is
+ future proof and it makes the contents of the ``pyvenv/bin``
+ directory more informative. Portability is also not an issue,
+ because the Python Packaging Authority provides a package
+ ``distlib.scripts`` to perform this task.
+
+.. [#pip] ``pip`` might also be used when running ``make check-avocado``
+ if downloading is enabled, to ensure that Avocado is
+ available.
+
+The required versions of the packages are stored in a configuration file
+``pythondeps.toml``. The format is custom to QEMU, but it is documented
+at the top of the file itself and it should be easy to understand. The
+requirements should make it possible to use the version that is packaged
+that is provided by supported distros.
+
+When dependencies are downloaded, instead, ``configure`` uses a "known
+good" version that is also listed in ``pythondeps.toml``. In this
+scenario, ``pythondeps.toml`` behaves like the "lock file" used by
+``cargo``, ``poetry`` or other dependency management systems.
+
+
+Bundled Python packages
+-----------------------
+
+Python packages that are **mandatory** dependencies to build QEMU,
+but are not available in all supported distros, are bundled with the
+QEMU sources. Currently this includes Meson (outdated in CentOS 8
+and derivatives, Ubuntu 20.04 and 22.04, and openSUSE Leap) and tomli
+(absent in Ubuntu 20.04).
+
+If you need to update these, please do so by modifying and rerunning
+``python/scripts/vendor.py``. This script embeds the sha256 hash of
+package sources and checks it. The pypi.org web site provides an easy
+way to retrieve the sha256 hash of the sources.
-TBD
Stage 2: Meson
==============
@@ -376,6 +444,15 @@ This is needed to obey the --python= option passed to the configure
script, which may point to something other than the first python3
binary on the path.
+By the time Meson runs, Python dependencies are available in the virtual
+environment and should be invoked through the scripts that ``configure``
+places under ``pyvenv``. One way to do so is as follows, using Meson's
+``find_program`` function::
+
+ sphinx_build = find_program(
+ fs.parent(python.full_path()) / 'sphinx-build',
+ required: get_option('docs'))
+
Stage 3: Make
=============
@@ -434,6 +511,11 @@ number of dynamically created files listed later.
executables. Build rules for various subdirectories are included in
other meson.build files spread throughout the QEMU source tree.
+``python/scripts/mkvenv.py``
+ A wrapper for the Python ``venv`` and ``distlib.scripts`` packages.
+ It handles creating the virtual environment, creating scripts in
+ ``pyvenv/bin``, and calling ``pip`` to install dependencies.
+
``tests/Makefile.include``
Rules for external test harnesses. These include the TCG tests
and the Avocado-based integration tests.
diff --git a/pythondeps.toml b/pythondeps.toml
index 0a35ebcf9f0..4beadfd96f5 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -10,7 +10,8 @@
# - accepted: accepted versions when using a system package
# - installed: fixed version to install in the virtual environment
# if a system package is not found; if not specified,
-# the minimum and maximum
+# defaults to the same as "accepted" or, if also missing,
+# to the newest version available on PyPI.
# - canary: if specified, use this program name to present more
# precise error diagnostics to the user. For example,
# 'sphinx-build' can be used as a bellwether for the
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PULL 4/6] buildsys: Bump known good meson version to v1.2.3
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
` (2 preceding siblings ...)
2023-11-11 15:29 ` [PULL 3/6] docs: document what configure does with virtual environments Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-11 15:29 ` [PULL 5/6] .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura) Paolo Bonzini
` (2 subsequent siblings)
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@linaro.org>
We need meson v1.2.3 to build QEMU on macOS Sonoma. It
also builds fine all our CI jobs (as tested by also bumping
"accepted" in pythondeps.toml), so let's use it as our
"good enough" packaged wheel.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1939
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-2-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
python/scripts/vendor.py | 4 ++--
python/wheels/meson-0.63.3-py3-none-any.whl | Bin 926526 -> 0 bytes
python/wheels/meson-1.2.3-py3-none-any.whl | Bin 0 -> 964928 bytes
pythondeps.toml | 2 +-
4 files changed, 3 insertions(+), 3 deletions(-)
delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
diff --git a/python/scripts/vendor.py b/python/scripts/vendor.py
index 76274871170..1038b14ae0c 100755
--- a/python/scripts/vendor.py
+++ b/python/scripts/vendor.py
@@ -41,8 +41,8 @@ def main() -> int:
parser.parse_args()
packages = {
- "meson==0.63.3":
- "d677b809c4895dcbaac9bf6c43703fcb3609a4b24c6057c78f828590049cf43a",
+ "meson==1.2.3":
+ "4533a43c34548edd1f63a276a42690fce15bde9409bcf20c4b8fa3d7e4d7cac1",
"tomli==2.0.1":
"939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc",
diff --git a/python/wheels/meson-0.63.3-py3-none-any.whl b/python/wheels/meson-0.63.3-py3-none-any.whl
deleted file mode 100644
index 8a191e3a200eba6783297676729ec85ceb96f89d..0000000000000000000000000000000000000000
Binary files a/python/wheels/meson-0.63.3-py3-none-any.whl and /dev/null differ
diff --git a/python/wheels/meson-1.2.3-py3-none-any.whl b/python/wheels/meson-1.2.3-py3-none-any.whl
new file mode 100644
index 0000000000000000000000000000000000000000..a8b84e5f114ad3fbec8ae56008426a90bccfc168
GIT binary patch
Binary files /dev/null and b/python/wheels/meson-1.2.3-py3-none-any.whl differ
diff --git a/pythondeps.toml b/pythondeps.toml
index 4beadfd96f5..0e884159993 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -19,7 +19,7 @@
[meson]
# The install key should match the version in python/wheels/
-meson = { accepted = ">=0.63.0", installed = "0.63.3", canary = "meson" }
+meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" }
[docs]
sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PULL 5/6] .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
` (3 preceding siblings ...)
2023-11-11 15:29 ` [PULL 4/6] buildsys: Bump known good meson version to v1.2.3 Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-11 15:29 ` [PULL 6/6] .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma) Paolo Bonzini
2023-11-12 18:06 ` [PULL 0/6] Biuld system and CI changes for 2023-11-10 Stefan Hajnoczi
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé, Daniel P . Berrangé
From: Philippe Mathieu-Daudé <philmd@linaro.org>
macOS 14 "Sonoma" was released on September 2023 [1].
According to QEMU's support policy, we stop supporting the
previous major release two years after the the new major
release has been published. Replace the macOS 12 (Monterey)
testing by macOS 13 (Ventura, released on October 2022, [2]).
Refresh the generated files by running:
$ make lcitool-refresh
[1] https://www.apple.com/newsroom/2023/09/macos-sonoma-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20231108162022.76189-1-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
.gitlab-ci.d/cirrus.yml | 6 +++---
.gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} | 2 +-
tests/lcitool/refresh | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index e7f1f83c2c4..07dc6edae1c 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -59,13 +59,13 @@ x64-freebsd-13-build:
INSTALL_COMMAND: pkg install -y
TEST_TARGETS: check
-aarch64-macos-12-base-build:
+aarch64-macos-13-base-build:
extends: .cirrus_build_job
variables:
- NAME: macos-12
+ NAME: macos-13
CIRRUS_VM_INSTANCE_TYPE: macos_instance
CIRRUS_VM_IMAGE_SELECTOR: image
- CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-monterey-base:latest
+ CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-ventura-base:latest
CIRRUS_VM_CPUS: 12
CIRRUS_VM_RAM: 24G
UPDATE_COMMAND: brew update
diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-13.vars
similarity index 95%
rename from .gitlab-ci.d/cirrus/macos-12.vars
rename to .gitlab-ci.d/cirrus/macos-13.vars
index 5f3fb346d1d..534f0299560 100644
--- a/.gitlab-ci.d/cirrus/macos-12.vars
+++ b/.gitlab-ci.d/cirrus/macos-13.vars
@@ -1,6 +1,6 @@
# THIS FILE WAS AUTO-GENERATED
#
-# $ lcitool variables macos-12 qemu
+# $ lcitool variables macos-13 qemu
#
# https://gitlab.com/libvirt/libvirt-ci
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 92e7d30982e..2259f131b48 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -196,7 +196,7 @@ try:
# Cirrus packages lists for GitLab
#
generate_cirrus("freebsd-13")
- generate_cirrus("macos-12")
+ generate_cirrus("macos-13")
#
# VM packages lists
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* [PULL 6/6] .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
` (4 preceding siblings ...)
2023-11-11 15:29 ` [PULL 5/6] .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura) Paolo Bonzini
@ 2023-11-11 15:29 ` Paolo Bonzini
2023-11-12 18:06 ` [PULL 0/6] Biuld system and CI changes for 2023-11-10 Stefan Hajnoczi
6 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-11 15:29 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@linaro.org>
Upgrade libvirt-ci so it covers macOS 14. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sonoma release. Refresh the
lci-tool generated files.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-3-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
.gitlab-ci.d/cirrus.yml | 16 ++++++++++++++++
.gitlab-ci.d/cirrus/macos-14.vars | 16 ++++++++++++++++
tests/lcitool/libvirt-ci | 2 +-
tests/lcitool/refresh | 1 +
4 files changed, 34 insertions(+), 1 deletion(-)
create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 07dc6edae1c..64f2e25afab 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -74,6 +74,22 @@ aarch64-macos-13-base-build:
PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
+aarch64-macos-14-base-build:
+ extends: .cirrus_build_job
+ variables:
+ NAME: macos-14
+ CIRRUS_VM_INSTANCE_TYPE: macos_instance
+ CIRRUS_VM_IMAGE_SELECTOR: image
+ CIRRUS_VM_IMAGE_NAME: ghcr.io/cirruslabs/macos-sonoma-base:latest
+ CIRRUS_VM_CPUS: 12
+ CIRRUS_VM_RAM: 24G
+ UPDATE_COMMAND: brew update
+ INSTALL_COMMAND: brew install
+ PATH_EXTRA: /opt/homebrew/ccache/libexec:/opt/homebrew/gettext/bin
+ PKG_CONFIG_PATH: /opt/homebrew/curl/lib/pkgconfig:/opt/homebrew/ncurses/lib/pkgconfig:/opt/homebrew/readline/lib/pkgconfig
+ TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
+ QEMU_JOB_OPTIONAL: 1
+
# The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
.cirrus_kvm_job:
diff --git a/.gitlab-ci.d/cirrus/macos-14.vars b/.gitlab-ci.d/cirrus/macos-14.vars
new file mode 100644
index 00000000000..43070f4a265
--- /dev/null
+++ b/.gitlab-ci.d/cirrus/macos-14.vars
@@ -0,0 +1,16 @@
+# THIS FILE WAS AUTO-GENERATED
+#
+# $ lcitool variables macos-14 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
+CCACHE='/opt/homebrew/bin/ccache'
+CPAN_PKGS=''
+CROSS_PKGS=''
+MAKE='/opt/homebrew/bin/gmake'
+NINJA='/opt/homebrew/bin/ninja'
+PACKAGING_COMMAND='brew'
+PIP3='/opt/homebrew/bin/pip3'
+PKGS='bash bc bison bzip2 capstone ccache cmocka ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo json-c libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 xorriso zlib zstd'
+PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme tomli'
+PYTHON='/opt/homebrew/bin/python3'
diff --git a/tests/lcitool/libvirt-ci b/tests/lcitool/libvirt-ci
index 36bc517161c..77c800186f3 160000
--- a/tests/lcitool/libvirt-ci
+++ b/tests/lcitool/libvirt-ci
@@ -1 +1 @@
-Subproject commit 36bc517161c45ead20224d47f2dc4fa428af6724
+Subproject commit 77c800186f34b21be7660750577cc5582a914deb
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 2259f131b48..e11d0ba381f 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -197,6 +197,7 @@ try:
#
generate_cirrus("freebsd-13")
generate_cirrus("macos-13")
+ generate_cirrus("macos-14")
#
# VM packages lists
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PULL 0/6] Biuld system and CI changes for 2023-11-10
2023-11-11 15:29 [PULL 0/6] Biuld system and CI changes for 2023-11-10 Paolo Bonzini
` (5 preceding siblings ...)
2023-11-11 15:29 ` [PULL 6/6] .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma) Paolo Bonzini
@ 2023-11-12 18:06 ` Stefan Hajnoczi
2023-11-14 13:54 ` Paolo Bonzini
6 siblings, 1 reply; 9+ messages in thread
From: Stefan Hajnoczi @ 2023-11-12 18:06 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: qemu-devel
On Sat, 11 Nov 2023 at 23:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit ad6ef0a42e314a8c6ac6c96d5f6e607a1e5644b5:
>
> Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging (2023-11-09 08:26:01 +0800)
>
> are available in the Git repository at:
>
> https://gitlab.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 1d802d050caeff83add1054bee1fc9f98e59a3f2:
>
> .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma) (2023-11-10 10:39:05 +0100)
>
> ----------------------------------------------------------------
> * document what configure does with virtual environments
> * Bump known good meson version to v1.2.3
> * Upgrade macOS to 13 (Ventura) and Add manual testing of macOS 14 (Sonoma)
> * use simple assertions instead of Coverity models
I'm not sure which commit causes this, but there is an msys failure:
"gcc" "-m64" "-mcx16" -o tests/plugin/libbb.dll
plugins/qemu_plugin_api.lib tests/plugin/libbb.dll.p/bb.c.obj
tests/plugin/libbb.dll.p/.._.._contrib_plugins_win32_linker.c.obj
"-Wl,--allow-shlib-undefined" "-shared" "-Wl,--start-group"
"-Wl,--out-implib=tests/plugin/libbb.dll.a" "-fstack-protector-strong"
"-Wl,--no-seh" "-Wl,--nxcompat" "-Wl,--dynamicbase"
"-Wl,--high-entropy-va" "-Wl,--warn-common"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libglib-2.0.dll.a"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libintl.dll.a"
"C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/lib/libgmodule-2.0.dll.a"
"-lkernel32" "-luser32" "-lgdi32" "-lwinspool" "-lshell32" "-lole32"
"-loleaut32" "-luuid" "-lcomdlg32" "-ladvapi32" "-Wl,--end-group"
C:/GitLab-Runner/builds/qemu-project/qemu/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
tests/plugin/libbb.dll.p/bb.c.obj:bb.c:(.text+0x219): undefined
reference to `__imp_qemu_plugin_outs'
Here is the failing output with this PR applied:
https://gitlab.com/qemu-project/qemu/-/jobs/5516759417
Here is the master branch's passing output:
https://gitlab.com/qemu-project/qemu/-/jobs/5513282154
Please take a look. Thanks!
Stefan
>
> ----------------------------------------------------------------
> Paolo Bonzini (2):
> tests: respect --enable/--disable-download for Avocado
> docs: document what configure does with virtual environments
>
> Philippe Mathieu-Daudé (3):
> buildsys: Bump known good meson version to v1.2.3
> .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
> .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
>
> Vladimir Sementsov-Ogievskiy (1):
> coverity: physmem: use simple assertions instead of modelling
>
> .gitlab-ci.d/cirrus.yml | 22 +++++-
> .../cirrus/{macos-12.vars => macos-13.vars} | 2 +-
> .gitlab-ci.d/cirrus/macos-14.vars | 16 ++++
> configure | 9 ++-
> docs/devel/build-system.rst | 88 ++++++++++++++++++++-
> python/scripts/vendor.py | 4 +-
> python/wheels/meson-0.63.3-py3-none-any.whl | Bin 926526 -> 0 bytes
> python/wheels/meson-1.2.3-py3-none-any.whl | Bin 0 -> 964928 bytes
> pythondeps.toml | 5 +-
> scripts/coverity-scan/model.c | 88 ---------------------
> system/physmem.c | 22 ++++++
> tests/Makefile.include | 2 +-
> tests/lcitool/libvirt-ci | 2 +-
> tests/lcitool/refresh | 3 +-
> 14 files changed, 157 insertions(+), 106 deletions(-)
> rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
> create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
> delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
> create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
> --
> 2.41.0
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [PULL 0/6] Biuld system and CI changes for 2023-11-10
2023-11-12 18:06 ` [PULL 0/6] Biuld system and CI changes for 2023-11-10 Stefan Hajnoczi
@ 2023-11-14 13:54 ` Paolo Bonzini
0 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2023-11-14 13:54 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: qemu-devel
[-- Attachment #1: Type: text/plain, Size: 3453 bytes --]
Il dom 12 nov 2023, 13:06 Stefan Hajnoczi <stefanha@gmail.com> ha scritto:
> On Sat, 11 Nov 2023 at 23:30, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> > The following changes since commit
> ad6ef0a42e314a8c6ac6c96d5f6e607a1e5644b5:
> >
> > Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into
> staging (2023-11-09 08:26:01 +0800)
> >
> > are available in the Git repository at:
> >
> > https://gitlab.com/bonzini/qemu.git tags/for-upstream
> >
> > for you to fetch changes up to 1d802d050caeff83add1054bee1fc9f98e59a3f2:
> >
> > .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
> (2023-11-10 10:39:05 +0100)
> >
> > ----------------------------------------------------------------
> > * document what configure does with virtual environments
> > * Bump known good meson version to v1.2.3
> > * Upgrade macOS to 13 (Ventura) and Add manual testing of macOS 14
> (Sonoma)
> > * use simple assertions instead of Coverity models
>
> I'm not sure which commit causes this, but there is an msys failure:
>
> Here is the failing output with this PR applied:
> https://gitlab.com/qemu-project/qemu/-/jobs/5516759417
>
> Here is the master branch's passing output:
> https://gitlab.com/qemu-project/qemu/-/jobs/5513282154
>
> Please take a look. Thanks!
>
Looks like the series causes issue #1972 to appear more prominently. I will
resubmit once it's fixed (a patch is already available).
Paolo
> Stefan
>
> >
> > ----------------------------------------------------------------
> > Paolo Bonzini (2):
> > tests: respect --enable/--disable-download for Avocado
> > docs: document what configure does with virtual environments
> >
> > Philippe Mathieu-Daudé (3):
> > buildsys: Bump known good meson version to v1.2.3
> > .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
> > .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
> >
> > Vladimir Sementsov-Ogievskiy (1):
> > coverity: physmem: use simple assertions instead of modelling
> >
> > .gitlab-ci.d/cirrus.yml | 22 +++++-
> > .../cirrus/{macos-12.vars => macos-13.vars} | 2 +-
> > .gitlab-ci.d/cirrus/macos-14.vars | 16 ++++
> > configure | 9 ++-
> > docs/devel/build-system.rst | 88
> ++++++++++++++++++++-
> > python/scripts/vendor.py | 4 +-
> > python/wheels/meson-0.63.3-py3-none-any.whl | Bin 926526 -> 0
> bytes
> > python/wheels/meson-1.2.3-py3-none-any.whl | Bin 0 -> 964928
> bytes
> > pythondeps.toml | 5 +-
> > scripts/coverity-scan/model.c | 88
> ---------------------
> > system/physmem.c | 22 ++++++
> > tests/Makefile.include | 2 +-
> > tests/lcitool/libvirt-ci | 2 +-
> > tests/lcitool/refresh | 3 +-
> > 14 files changed, 157 insertions(+), 106 deletions(-)
> > rename .gitlab-ci.d/cirrus/{macos-12.vars => macos-13.vars} (95%)
> > create mode 100644 .gitlab-ci.d/cirrus/macos-14.vars
> > delete mode 100644 python/wheels/meson-0.63.3-py3-none-any.whl
> > create mode 100644 python/wheels/meson-1.2.3-py3-none-any.whl
> > --
> > 2.41.0
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 5050 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread