[PATCH 0/2] gitlab: improve debuggability of jobs

[PATCH 0/2] gitlab: improve debuggability of jobs

[Daniel P. Berrangé]

This ensures that the main configure/make commands
run have their args echoed into the build logs.

Daniel P. Berrangé (2):
  gitlab: remove redundant setting of PKG_CONFIG_PATH
  gitlab: show configure/make args before running

 .gitlab-ci.d/buildtest-template.yml  |  4 ++++
 .gitlab-ci.d/crossbuild-template.yml | 24 ++++++++++++++++++------
 2 files changed, 22 insertions(+), 6 deletions(-)

-- 
2.37.2

[PATCH 1/2] gitlab: remove redundant setting of PKG_CONFIG_PATH

[Daniel P. Berrangé]

The PKG_CONFIG_PATH variable is not defined in GitLab CI
envs and even if it was, we don't need to set it to its
existing value.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/crossbuild-template.yml | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 5e8892fd49..6d709628f1 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -6,8 +6,7 @@
   script:
     - mkdir build
     - cd build
-    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
-      ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
           mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
@@ -32,8 +31,7 @@
   script:
     - mkdir build
     - cd build
-    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
-      ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
 
@@ -44,8 +42,7 @@
   script:
     - mkdir build
     - cd build
-    - PKG_CONFIG_PATH=$PKG_CONFIG_PATH
-      ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+    - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-system --target-list-exclude="aarch64_be-linux-user
           alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user
           nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
-- 
2.37.2

[PATCH 2/2] gitlab: show configure/make args before running

[Daniel P. Berrangé]

When debugging failed jobs it is helpful to see the
full configure/make args used, without having to search
the gitlab config file to figure it out.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  |  4 ++++
 .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++
 2 files changed, 19 insertions(+)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 73ecfabb8d..0af980c71d 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -13,8 +13,10 @@
     - cd build
     - if test -n "$TARGETS";
       then
+        echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list=\"$TARGETS\"" ;
         ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ;
       else
+        echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS" ;
         ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ;
       fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
     - if test -n "$LD_JOBS";
@@ -24,6 +26,7 @@
     - make -j"$JOBS"
     - if test -n "$MAKE_CHECK_ARGS";
       then
+        echo "make -j\"$JOBS\" $MAKE_CHECK_ARGS" ;
         make -j"$JOBS" $MAKE_CHECK_ARGS ;
       fi
 
@@ -37,6 +40,7 @@
     - cd build
     - find . -type f -exec touch {} +
     # Avoid recompiling by hiding ninja with NINJA=":"
+    - echo "make NINJA=\":\" $MAKE_CHECK_ARGS"
     - make NINJA=":" $MAKE_CHECK_ARGS
 
 .native_test_job_template:
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 6d709628f1..d217dcaae0 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -6,11 +6,17 @@
   script:
     - mkdir build
     - cd build
+    - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+        --disable-user --target-list-exclude=\"arm-softmmu cris-softmmu
+          i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
+          mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
+          sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS\""
     - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-user --target-list-exclude="arm-softmmu cris-softmmu
           i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu
           mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
           sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS"
+    - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
     - if grep -q "EXESUF=.exe" config-host.mak;
       then make installer;
@@ -31,8 +37,11 @@
   script:
     - mkdir build
     - cd build
+    - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+        --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS"
     - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS
+    - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
 
 .cross_user_build_job:
@@ -42,9 +51,15 @@
   script:
     - mkdir build
     - cd build
+    - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
+        --disable-system --target-list-exclude=\"aarch64_be-linux-user
+          alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user
+          nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
+          xtensa-linux-user $CROSS_SKIP_TARGETS\""
     - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS
         --disable-system --target-list-exclude="aarch64_be-linux-user
           alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user
           nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
           xtensa-linux-user $CROSS_SKIP_TARGETS"
+    - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS"
     - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
-- 
2.37.2

Re: [PATCH 1/2] gitlab: remove redundant setting of PKG_CONFIG_PATH

[Stefan Hajnoczi]

On Thu, 3 Nov 2022 at 13:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> The PKG_CONFIG_PATH variable is not defined in GitLab CI
> envs and even if it was, we don't need to set it to its
> existing value.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/crossbuild-template.yml | 9 +++------
>  1 file changed, 3 insertions(+), 6 deletions(-)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Re: [PATCH 2/2] gitlab: show configure/make args before running

[Stefan Hajnoczi]

On Thu, 3 Nov 2022 at 13:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> When debugging failed jobs it is helpful to see the
> full configure/make args used, without having to search
> the gitlab config file to figure it out.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/buildtest-template.yml  |  4 ++++
>  .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++
>  2 files changed, 19 insertions(+)

On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure
how to get escaping right though.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

Re: [PATCH 2/2] gitlab: show configure/make args before running

[Richard Henderson]

On 11/4/22 06:06, Stefan Hajnoczi wrote:
> On Thu, 3 Nov 2022 at 13:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>
>> When debugging failed jobs it is helpful to see the
>> full configure/make args used, without having to search
>> the gitlab config file to figure it out.
>>
>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>> ---
>>   .gitlab-ci.d/buildtest-template.yml  |  4 ++++
>>   .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++
>>   2 files changed, 19 insertions(+)
> 
> On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure
> how to get escaping right though.

set -x && ../configure ... ?


r~

Re: [PATCH 2/2] gitlab: show configure/make args before running

[Daniel P. Berrangé]

On Fri, Nov 04, 2022 at 08:31:31AM +1100, Richard Henderson wrote:
> On 11/4/22 06:06, Stefan Hajnoczi wrote:
> > On Thu, 3 Nov 2022 at 13:30, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > 
> > > When debugging failed jobs it is helpful to see the
> > > full configure/make args used, without having to search
> > > the gitlab config file to figure it out.
> > > 
> > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > > ---
> > >   .gitlab-ci.d/buildtest-template.yml  |  4 ++++
> > >   .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++
> > >   2 files changed, 19 insertions(+)
> > 
> > On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure
> > how to get escaping right though.
> 
> set -x && ../configure ... ?

I tried adding 'set -x' to the script: blocks too. It turns out that
gitlab has an  echo '$CMD' for each line in the yaml script: section

IOW, with my patch here we get a log that looks like

$ mkdir build
$ cd build
$ if test -n "$TARGETS"; then echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list=\"$TARGETS\"" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
../configure --enable-werror --disable-docs  --disable-gcrypt --enable-nettle --enable-docs --enable-fdt=system --enable-slirp --enable-capstone --target-list="tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu"


and with adding 'set -x' we get


++ echo '$ mkdir build'
$ mkdir build
++ mkdir build
++ echo '$ cd build'
$ cd build
++ cd build
++ echo '$ if test -n "$TARGETS"; then ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }'
$ if test -n "$TARGETS"; then ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
++ test -n 'tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu'
++ ../configure --enable-werror --disable-docs --disable-gcrypt --enable-nettle --enable-docs --enable-fdt=system --enable-slirp --enable-capstone '--target-list=tricore-softmmu microblaze-softmmu mips-softmmu xtensa-sof


Doing 'set -x && configure' only marginally helps, as the 'set -x' still
takes effect for all commands that come after configure too.


The other option is to not do either 'echo' not 'set -x', and instead
do this:

  printenv | grep -v -E "(CI_|GITLAB)"

so that the build logs has a dump of any env vars set. So when seeing

./configure $CONFIGURE_ARGS

we have a record of what $CONFIGURE_ARGS was set to.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 0/2] gitlab: improve debuggability of jobs

[Paolo Bonzini]

Queued patch 1 while the second is figured out, thanks.

Paolo