[PATCH v2 0/5] Shorten the runtime of some gitlab-CI shared runner jobs

[PATCH v2 0/5] Shorten the runtime of some gitlab-CI shared runner jobs

[Thomas Huth]

We're currently facing the problem that the gitlab-CI jobs for the
shared runners take too much of the limited CI minutes on gitlab.com.
Here are now some patches that optimize some of the jobs a little bit
to take less runtime.

v2:
- Dropped the patches that have already been merged
- Rework the sigaltstack patch according to Daniel's rewview comments
- Add Alex' patch as replacement for the gcov-gprof patch in v1
- Add patch to compile with --enable-fdt=system by default

Alex Bennée (1):
  build: deprecate --enable-gprof builds and remove from CI

Thomas Huth (4):
  gitlab-ci.d/buildtest: Remove aarch64-softmmu from the
    build-system-ubuntu job
  gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack
    job
  .gitlab-ci.d/buildtest-template: Simplify the configure step
  gitlab-ci.d: Build with --enable-fdt=system by default

 docs/about/deprecated.rst            | 14 +++++++++
 meson.build                          |  7 ++++-
 .gitlab-ci.d/buildtest-template.yml  | 10 +++----
 .gitlab-ci.d/buildtest.yml           | 43 ++++++----------------------
 .gitlab-ci.d/crossbuild-template.yml |  5 ++--
 .gitlab-ci.d/crossbuilds.yml         |  4 ++-
 .gitlab-ci.d/windows.yml             |  7 +++--
 meson_options.txt                    |  3 +-
 8 files changed, 46 insertions(+), 47 deletions(-)

-- 
2.31.1

[PATCH v2 1/5] build: deprecate --enable-gprof builds and remove from CI

[Thomas Huth]

From: Alex Bennée <alex.bennee@linaro.org>

As gprof relies on instrumentation you rarely get useful data compared
to a real optimised build. Lets deprecate the build option and
simplify the CI configuration as a result.

Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1338
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230131094224.861621-1-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 docs/about/deprecated.rst  | 14 ++++++++++++++
 meson.build                |  7 ++++++-
 .gitlab-ci.d/buildtest.yml | 19 ++++---------------
 meson_options.txt          |  3 ++-
 4 files changed, 26 insertions(+), 17 deletions(-)

diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
index da2e6fe63d..9317046177 100644
--- a/docs/about/deprecated.rst
+++ b/docs/about/deprecated.rst
@@ -20,6 +20,20 @@ they were first deprecated in the 2.10.0 release.
 What follows is a list of all features currently marked as
 deprecated.
 
+Build options
+-------------
+
+``gprof`` builds (since 8.0)
+''''''''''''''''''''''''''''
+
+The ``--enable-gprof`` configure setting relies on compiler
+instrumentation to gather its data which can distort the generated
+profile. As other non-instrumenting tools are available that give a
+more holistic view of the system with non-instrumented binaries we are
+deprecating the build option and no longer defend it in CI. The
+``--enable-gcov`` build option remains for analysis test case
+coverage.
+
 System emulator command line arguments
 --------------------------------------
 
diff --git a/meson.build b/meson.build
index 4ba3bf3431..bc60bf3c4c 100644
--- a/meson.build
+++ b/meson.build
@@ -3784,7 +3784,12 @@ summary_info += {'mutex debugging':   get_option('debug_mutex')}
 summary_info += {'memory allocator':  get_option('malloc')}
 summary_info += {'avx2 optimization': config_host_data.get('CONFIG_AVX2_OPT')}
 summary_info += {'avx512f optimization': config_host_data.get('CONFIG_AVX512F_OPT')}
-summary_info += {'gprof enabled':     get_option('gprof')}
+if get_option('gprof')
+  gprof_info = 'YES (deprecated)'
+else
+  gprof_info = get_option('gprof')
+endif
+summary_info += {'gprof':             gprof_info}
 summary_info += {'gcov':              get_option('b_coverage')}
 summary_info += {'thread sanitizer':  config_host.has_key('CONFIG_TSAN')}
 summary_info += {'CFI support':       get_option('cfi')}
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 0aa149a352..8f332fc36f 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -467,27 +467,16 @@ tsan-build:
     TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user
     MAKE_CHECK_ARGS: bench V=1
 
-# gprof/gcov are GCC features
-build-gprof-gcov:
+# gcov is a GCC features
+gcov:
   extends: .native_build_job_template
   needs:
     job: amd64-ubuntu2004-container
+  timeout: 80m
   variables:
     IMAGE: ubuntu2004
-    CONFIGURE_ARGS: --enable-gprof --enable-gcov
+    CONFIGURE_ARGS: --enable-gcov
     TARGETS: aarch64-softmmu ppc64-softmmu s390x-softmmu x86_64-softmmu
-  artifacts:
-    expire_in: 1 days
-    paths:
-      - build
-
-check-gprof-gcov:
-  extends: .native_test_job_template
-  needs:
-    - job: build-gprof-gcov
-      artifacts: true
-  variables:
-    IMAGE: ubuntu2004
     MAKE_CHECK_ARGS: check
   after_script:
     - cd build
diff --git a/meson_options.txt b/meson_options.txt
index 559a571b6b..53459c15fc 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -316,7 +316,8 @@ option('debug_stack_usage', type: 'boolean', value: false,
 option('qom_cast_debug', type: 'boolean', value: false,
        description: 'cast debugging support')
 option('gprof', type: 'boolean', value: false,
-       description: 'QEMU profiling with gprof')
+       description: 'QEMU profiling with gprof',
+       deprecated: true)
 option('profiler', type: 'boolean', value: false,
        description: 'profiler support')
 option('slirp_smbd', type : 'feature', value : 'auto',
-- 
2.31.1

Re: [PATCH v2 1/5] build: deprecate --enable-gprof builds and remove from CI

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 4900 bytes --]

On Tue, 2023-02-07 at 21:14 +0100, Thomas Huth wrote:
> From: Alex Bennée <alex.bennee@linaro.org>
> 
> As gprof relies on instrumentation you rarely get useful data compared
> to a real optimised build. Lets deprecate the build option and
> simplify the CI configuration as a result.
> 
> Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1338
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20230131094224.861621-1-alex.bennee@linaro.org>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  docs/about/deprecated.rst  | 14 ++++++++++++++
>  meson.build                |  7 ++++++-
>  .gitlab-ci.d/buildtest.yml | 19 ++++---------------
>  meson_options.txt          |  3 ++-
>  4 files changed, 26 insertions(+), 17 deletions(-)
> 
> diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst
> index da2e6fe63d..9317046177 100644
> --- a/docs/about/deprecated.rst
> +++ b/docs/about/deprecated.rst
> @@ -20,6 +20,20 @@ they were first deprecated in the 2.10.0 release.
>  What follows is a list of all features currently marked as
>  deprecated.
>  
> +Build options
> +-------------
> +
> +``gprof`` builds (since 8.0)
> +''''''''''''''''''''''''''''
> +
> +The ``--enable-gprof`` configure setting relies on compiler
> +instrumentation to gather its data which can distort the generated
> +profile. As other non-instrumenting tools are available that give a
> +more holistic view of the system with non-instrumented binaries we are
> +deprecating the build option and no longer defend it in CI. The

I'm not sure I understand the verb 'defend' in that sentence?

> +``--enable-gcov`` build option remains for analysis test case
> +coverage.
> +
>  System emulator command line arguments
>  --------------------------------------
>  
> diff --git a/meson.build b/meson.build
> index 4ba3bf3431..bc60bf3c4c 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -3784,7 +3784,12 @@ summary_info += {'mutex debugging':   get_option('debug_mutex')}
>  summary_info += {'memory allocator':  get_option('malloc')}
>  summary_info += {'avx2 optimization': config_host_data.get('CONFIG_AVX2_OPT')}
>  summary_info += {'avx512f optimization': config_host_data.get('CONFIG_AVX512F_OPT')}
> -summary_info += {'gprof enabled':     get_option('gprof')}
> +if get_option('gprof')
> +  gprof_info = 'YES (deprecated)'
> +else
> +  gprof_info = get_option('gprof')
> +endif
> +summary_info += {'gprof':             gprof_info}
>  summary_info += {'gcov':              get_option('b_coverage')}
>  summary_info += {'thread sanitizer':  config_host.has_key('CONFIG_TSAN')}
>  summary_info += {'CFI support':       get_option('cfi')}
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index 0aa149a352..8f332fc36f 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -467,27 +467,16 @@ tsan-build:
>      TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user
>      MAKE_CHECK_ARGS: bench V=1
>  
> -# gprof/gcov are GCC features
> -build-gprof-gcov:
> +# gcov is a GCC features

Nit: "is a GCC feature". Other than that, FWIW,

Reviewed-by: David Woodhouse <dwmw2@infradead.org>


> +gcov:
>    extends: .native_build_job_template
>    needs:
>      job: amd64-ubuntu2004-container
> +  timeout: 80m
>    variables:
>      IMAGE: ubuntu2004
> -    CONFIGURE_ARGS: --enable-gprof --enable-gcov
> +    CONFIGURE_ARGS: --enable-gcov
>      TARGETS: aarch64-softmmu ppc64-softmmu s390x-softmmu x86_64-softmmu
> -  artifacts:
> -    expire_in: 1 days
> -    paths:
> -      - build
> -
> -check-gprof-gcov:
> -  extends: .native_test_job_template
> -  needs:
> -    - job: build-gprof-gcov
> -      artifacts: true
> -  variables:
> -    IMAGE: ubuntu2004
>      MAKE_CHECK_ARGS: check
>    after_script:
>      - cd build
> diff --git a/meson_options.txt b/meson_options.txt
> index 559a571b6b..53459c15fc 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -316,7 +316,8 @@ option('debug_stack_usage', type: 'boolean', value: false,
>  option('qom_cast_debug', type: 'boolean', value: false,
>         description: 'cast debugging support')
>  option('gprof', type: 'boolean', value: false,
> -       description: 'QEMU profiling with gprof')
> +       description: 'QEMU profiling with gprof',
> +       deprecated: true)
>  option('profiler', type: 'boolean', value: false,
>         description: 'profiler support')
>  option('slirp_smbd', type : 'feature', value : 'auto',


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[PATCH v2 2/5] gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu job

[Thomas Huth]

aarch64-softmmu is also checked on the same version of Ubuntu in the
gcov job, so it is redundant to check again in the normal ubuntu job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 8f332fc36f..8fff961b44 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -42,7 +42,7 @@ build-system-ubuntu:
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-capstone
-    TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
+    TARGETS: alpha-softmmu cris-softmmu hppa-softmmu
       microblazeel-softmmu mips64el-softmmu
     MAKE_CHECK_ARGS: check-build
   artifacts:
-- 
2.31.1

Re: [PATCH v2 2/5] gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu job

[Daniel P. Berrangé]

On Tue, Feb 07, 2023 at 09:14:44PM +0100, Thomas Huth wrote:
> aarch64-softmmu is also checked on the same version of Ubuntu in the
> gcov job, so it is redundant to check again in the normal ubuntu job.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci.d/buildtest.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 2/5] gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu job

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

On Tue, 2023-02-07 at 21:14 +0100, Thomas Huth wrote:
> aarch64-softmmu is also checked on the same version of Ubuntu in the
> gcov job, so it is redundant to check again in the normal ubuntu job.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: David Woodhouse <dwmw2@infradead.org>


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[PATCH v2 3/5] gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack job

[Thomas Huth]

We can get rid of the build-coroutine-sigaltstack job by moving
the configure flags that should be tested here to other jobs:
Move --with-coroutine=sigaltstack to the build-system-debian job
(where the coroutines should get some more test coverage with
"make check-block", too) and --enable-trace-backends=ftrace to
the cross-s390x-kvm-only job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest.yml   | 13 +------------
 .gitlab-ci.d/crossbuilds.yml |  2 +-
 2 files changed, 2 insertions(+), 13 deletions(-)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 8fff961b44..8697c61072 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -74,6 +74,7 @@ build-system-debian:
     job: amd64-debian-container
   variables:
     IMAGE: debian-amd64
+    CONFIGURE_ARGS: --with-coroutine=sigaltstack
     TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
       riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -534,18 +535,6 @@ build-tci:
     - QTEST_QEMU_BINARY="./qemu-system-s390x" ./tests/qtest/pxe-test -m slow
     - make check-tcg
 
-# Alternate coroutines implementations are only really of interest to KVM users
-# However we can't test against KVM on Gitlab-CI so we can only run unit tests
-build-coroutine-sigaltstack:
-  extends: .native_build_job_template
-  needs:
-    job: amd64-ubuntu2004-container
-  variables:
-    IMAGE: ubuntu2004
-    CONFIGURE_ARGS: --with-coroutine=sigaltstack --disable-tcg
-                    --enable-trace-backends=ftrace
-    MAKE_CHECK_ARGS: check-unit
-
 # Check our reduced build configurations
 build-without-defaults:
   extends: .native_build_job_template
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 74d6259b90..57637c5127 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -159,7 +159,7 @@ cross-s390x-kvm-only:
     job: s390x-debian-cross-container
   variables:
     IMAGE: debian-s390x-cross
-    EXTRA_CONFIGURE_OPTS: --disable-tcg
+    EXTRA_CONFIGURE_OPTS: --disable-tcg --enable-trace-backends=ftrace
 
 cross-mips64el-kvm-only:
   extends: .cross_accel_build_job
-- 
2.31.1

Re: [PATCH v2 3/5] gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack job

[Daniel P. Berrangé]

On Tue, Feb 07, 2023 at 09:14:45PM +0100, Thomas Huth wrote:
> We can get rid of the build-coroutine-sigaltstack job by moving
> the configure flags that should be tested here to other jobs:
> Move --with-coroutine=sigaltstack to the build-system-debian job
> (where the coroutines should get some more test coverage with
> "make check-block", too) and --enable-trace-backends=ftrace to
> the cross-s390x-kvm-only job.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci.d/buildtest.yml   | 13 +------------
>  .gitlab-ci.d/crossbuilds.yml |  2 +-
>  2 files changed, 2 insertions(+), 13 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 3/5] gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack job

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

On Tue, 2023-02-07 at 21:14 +0100, Thomas Huth wrote:
> We can get rid of the build-coroutine-sigaltstack job by moving
> the configure flags that should be tested here to other jobs:
> Move --with-coroutine=sigaltstack to the build-system-debian job
> (where the coroutines should get some more test coverage with
> "make check-block", too) and --enable-trace-backends=ftrace to
> the cross-s390x-kvm-only job.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: David Woodhouse <dwmw2@infradead.org>

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[PATCH v2 4/5] .gitlab-ci.d/buildtest-template: Simplify the configure step

[Thomas Huth]

It's easier to use ${TARGETS:+--target-list="$TARGETS"} to add
a --target-list parameter depending on whether the TARGETS variable
is set or not.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 73ecfabb8d..4a922d9c33 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -11,12 +11,10 @@
       fi
     - mkdir build
     - cd build
-    - if test -n "$TARGETS";
-      then
-        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ;
-      else
-        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ;
-      fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - ../configure --enable-werror --disable-docs
+          ${LD_JOBS:+--meson=git} ${TARGETS:+--target-list="$TARGETS"}
+          $CONFIGURE_ARGS ||
+      { cat config.log meson-logs/meson-log.txt && exit 1; }
     - if test -n "$LD_JOBS";
       then
         ../meson/meson.py configure . -Dbackend_max_links="$LD_JOBS" ;
-- 
2.31.1

Re: [PATCH v2 4/5] .gitlab-ci.d/buildtest-template: Simplify the configure step

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

On Tue, 2023-02-07 at 21:14 +0100, Thomas Huth wrote:
> It's easier to use ${TARGETS:+--target-list="$TARGETS"} to add
> a --target-list parameter depending on whether the TARGETS variable
> is set or not.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: David Woodhouse <dwmw2@infradead.org>


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

[PATCH v2 5/5] gitlab-ci.d: Build with --enable-fdt=system by default

[Thomas Huth]

By using --enable-fdt=system we can make sure that the configure
script does not try to check out the "dtc" submodule. This should
help to safe some precious CI minutes in the long run.

While we're at it, also drop some now-redundant --enable-slirp
and --enable-capstone statements. These used to have the "=system"
suffix in the past, too, which has been dropped when the their
corresponding submodules had been removed. Since these features
are auto-enabled anyway now (since the containers have the right
libraries installed), we do not need the explicit --enable-...
statements anymore.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  | 2 +-
 .gitlab-ci.d/buildtest.yml           | 9 +++------
 .gitlab-ci.d/crossbuild-template.yml | 5 +++--
 .gitlab-ci.d/crossbuilds.yml         | 2 ++
 .gitlab-ci.d/windows.yml             | 7 +++++--
 5 files changed, 14 insertions(+), 11 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 4a922d9c33..cb96b55c3f 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -11,7 +11,7 @@
       fi
     - mkdir build
     - cd build
-    - ../configure --enable-werror --disable-docs
+    - ../configure --enable-werror --disable-docs --enable-fdt=system
           ${LD_JOBS:+--meson=git} ${TARGETS:+--target-list="$TARGETS"}
           $CONFIGURE_ARGS ||
       { cat config.log meson-logs/meson-log.txt && exit 1; }
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 8697c61072..d903c42798 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -41,7 +41,7 @@ build-system-ubuntu:
     job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
-    CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-capstone
+    CONFIGURE_ARGS: --enable-docs
     TARGETS: alpha-softmmu cris-softmmu hppa-softmmu
       microblazeel-softmmu mips64el-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -120,7 +120,6 @@ build-system-fedora:
   variables:
     IMAGE: fedora
     CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs
-             --enable-fdt=system --enable-slirp --enable-capstone
     TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu
       xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -166,9 +165,8 @@ build-system-centos:
     job: amd64-centos8-container
   variables:
     IMAGE: centos8
-    CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system
+    CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-vfio-user-server
       --enable-modules --enable-trace-backends=dtrace --enable-docs
-      --enable-vfio-user-server
     TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
       x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -201,7 +199,6 @@ build-system-opensuse:
     job: amd64-opensuse-leap-container
   variables:
     IMAGE: opensuse-leap
-    CONFIGURE_ARGS: --enable-fdt=system
     TARGETS: s390x-softmmu x86_64-softmmu aarch64-softmmu
     MAKE_CHECK_ARGS: check-build
   artifacts:
@@ -464,7 +461,7 @@ tsan-build:
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-tsan --cc=clang-10 --cxx=clang++-10
-          --enable-trace-backends=ust --enable-fdt=system --disable-slirp
+          --enable-trace-backends=ust --disable-slirp
     TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user
     MAKE_CHECK_ARGS: bench V=1
 
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 6d709628f1..d07989e3b0 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -6,8 +6,9 @@
   script:
     - mkdir build
     - cd build
-    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
-        --disable-user --target-list-exclude="arm-softmmu cris-softmmu
+    - ../configure --enable-werror --disable-docs --enable-fdt=system
+        --disable-user $QEMU_CONFIGURE_OPTS $EXTRA_CONFIGURE_OPTS
+        --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
           mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
           sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 57637c5127..101416080c 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -175,6 +175,7 @@ cross-win32-system:
     job: win32-fedora-cross-container
   variables:
     IMAGE: fedora-win32-cross
+    EXTRA_CONFIGURE_OPTS: --enable-fdt=internal
     CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu m68k-softmmu
                         microblazeel-softmmu mips64el-softmmu nios2-softmmu
   artifacts:
@@ -187,6 +188,7 @@ cross-win64-system:
     job: win64-fedora-cross-container
   variables:
     IMAGE: fedora-win64-cross
+    EXTRA_CONFIGURE_OPTS: --enable-fdt=internal
     CROSS_SKIP_TARGETS: alpha-softmmu avr-softmmu hppa-softmmu
                         m68k-softmmu microblazeel-softmmu nios2-softmmu
                         or1k-softmmu rx-softmmu sh4eb-softmmu sparc64-softmmu
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index cf445b77f6..87235e43b4 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -38,6 +38,7 @@ msys2-64bit:
       mingw-w64-x86_64-capstone
       mingw-w64-x86_64-curl
       mingw-w64-x86_64-cyrus-sasl
+      mingw-w64-x86_64-dtc
       mingw-w64-x86_64-gcc
       mingw-w64-x86_64-glib2
       mingw-w64-x86_64-gnutls
@@ -71,7 +72,7 @@ msys2-64bit:
   # for the msys2 64-bit job, due to the build could not complete within
   # the project timeout.
   - ..\msys64\usr\bin\bash -lc '../configure --target-list=x86_64-softmmu
-      --without-default-devices'
+      --without-default-devices --enable-fdt=system'
   - ..\msys64\usr\bin\bash -lc 'make'
   # qTests don't run successfully with "--without-default-devices",
   # so let's exclude the qtests from CI for now.
@@ -86,6 +87,7 @@ msys2-32bit:
       mingw-w64-i686-capstone
       mingw-w64-i686-curl
       mingw-w64-i686-cyrus-sasl
+      mingw-w64-i686-dtc
       mingw-w64-i686-gcc
       mingw-w64-i686-glib2
       mingw-w64-i686-gnutls
@@ -113,7 +115,8 @@ msys2-32bit:
   - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - mkdir output
   - cd output
-  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu'
+  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu
+                                --enable-fdt=system'
   - ..\msys64\usr\bin\bash -lc 'make'
   - ..\msys64\usr\bin\bash -lc 'make check MTESTARGS=\"--no-suite qtest\" ||
                                 { cat meson-logs/testlog.txt; exit 1; }'
-- 
2.31.1

Re: [PATCH v2 5/5] gitlab-ci.d: Build with --enable-fdt=system by default

[Daniel P. Berrangé]

On Tue, Feb 07, 2023 at 09:14:47PM +0100, Thomas Huth wrote:
> By using --enable-fdt=system we can make sure that the configure
> script does not try to check out the "dtc" submodule. This should
> help to safe some precious CI minutes in the long run.

If our containers have the system dtc installed, I'm pretty
surprised that configure is choosing to use dtc submodule.
I thought we won't touch the submodule at all if system dtc
was sufficiently new.

IOW, do we have a logic bug in configure making it incorrectly
use dtc submodules ?

> While we're at it, also drop some now-redundant --enable-slirp
> and --enable-capstone statements. These used to have the "=system"
> suffix in the past, too, which has been dropped when the their
> corresponding submodules had been removed. Since these features
> are auto-enabled anyway now (since the containers have the right
> libraries installed), we do not need the explicit --enable-...
> statements anymore.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci.d/buildtest-template.yml  | 2 +-
>  .gitlab-ci.d/buildtest.yml           | 9 +++------
>  .gitlab-ci.d/crossbuild-template.yml | 5 +++--
>  .gitlab-ci.d/crossbuilds.yml         | 2 ++
>  .gitlab-ci.d/windows.yml             | 7 +++++--
>  5 files changed, 14 insertions(+), 11 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


since it at least does what it claims, even if it the configure
logic is questionable.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 5/5] gitlab-ci.d: Build with --enable-fdt=system by default

[Thomas Huth]

On 23/02/2023 15.02, Daniel P. Berrangé wrote:
> On Tue, Feb 07, 2023 at 09:14:47PM +0100, Thomas Huth wrote:
>> By using --enable-fdt=system we can make sure that the configure
>> script does not try to check out the "dtc" submodule. This should
>> help to safe some precious CI minutes in the long run.
> 
> If our containers have the system dtc installed, I'm pretty
> surprised that configure is choosing to use dtc submodule.
> I thought we won't touch the submodule at all if system dtc
> was sufficiently new.
> 
> IOW, do we have a logic bug in configure making it incorrectly
> use dtc submodules ?

Yes, it sounds weird at the first glance, but it's really this way (look for 
the "Simpler to always update submodule, even if not needed" comment in the 
configure script): The problem is that the initial submodule handling is 
done in configure already, so you have to know the needed submodules there 
already. But the check for usability of libfdt is only done in meson.build, 
so you already need to have the submodule available there in case the 
system's libfdt is not usable.

It could maybe cleaned up somehow, but OTOH, I'm still hoping that we can 
rid of the dtc submodule in the near future, so it's maybe not worth the 
effort to spend too much time with this right now.

  Thomas