[PATCH 0/4] gitlab: misc artifact / log handling fixes

[PATCH 0/4] gitlab: misc artifact / log handling fixes

[Daniel P. Berrangé]

Daniel P. Berrangé (4):
  gitlab: replace avocado results files with meson results files
  gitlab: always include entire of meson-logs directory
  gitlab: include all junit XML files from meson
  gitlab: prevent duplicated meson log artifacts in test jobs

 .gitlab-ci.d/buildtest-template.yml  | 16 +++++++++++-----
 .gitlab-ci.d/buildtest.yml           |  4 ++--
 .gitlab-ci.d/crossbuild-template.yml |  4 ++--
 .gitlab-ci.d/custom-runners.yml      |  2 +-
 .gitlab-ci.d/windows.yml             |  4 ++--
 5 files changed, 18 insertions(+), 12 deletions(-)

-- 
2.50.1

[PATCH 1/4] gitlab: replace avocado results files with meson results files

[Daniel P. Berrangé]

The 'results.xml' file and 'test-results' directory were both outputs
of the avovcado test runner. Since we're now using meson with the new
functional test framework, we must reference meson results files as the
CI artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 038c3c9540..4672298214 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -104,11 +104,10 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/tests/results/latest/results.xml
-      - build/tests/results/latest/test-results
+      - build/meson-logs/testlog.txt
       - build/tests/functional/*/*/*.log
     reports:
-      junit: build/tests/results/latest/results.xml
+      junit: build/meson-logs/testlog.junit.xml
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
-- 
2.50.1

Re: [PATCH 1/4] gitlab: replace avocado results files with meson results files

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> The 'results.xml' file and 'test-results' directory were both outputs
> of the avovcado test runner. Since we're now using meson with the new
> functional test framework, we must reference meson results files as the
> CI artifacts.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

[PATCH 2/4] gitlab: always include entire of meson-logs directory

[Daniel P. Berrangé]

There are files besides testlog.txt that may be useful as published
CI artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  | 4 ++--
 .gitlab-ci.d/buildtest.yml           | 2 +-
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/windows.yml             | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 4672298214..4cc5b20790 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -88,7 +88,7 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
 
@@ -104,7 +104,7 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
       - build/tests/functional/*/*/*.log
     reports:
       junit: build/meson-logs/testlog.junit.xml
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index d888a60063..778289267f 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -613,7 +613,7 @@ gcov:
     when: always
     expire_in: 2 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
       coverage_report:
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 303943f818..7e70376cfc 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -128,6 +128,6 @@
     when: always
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: build/meson-logs/testlog.junit.xml
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index beac39e5bd..f14e9ca134 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -24,7 +24,7 @@ msys2-64bit:
     name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
     expire_in: 7 days
     paths:
-      - build/meson-logs/testlog.txt
+      - build/meson-logs
     reports:
       junit: "build/meson-logs/testlog.junit.xml"
   before_script:
-- 
2.50.1

Re: [PATCH 2/4] gitlab: always include entire of meson-logs directory

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> There are files besides testlog.txt that may be useful as published
> CI artifacts.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

[PATCH 3/4] gitlab: include all junit XML files from meson

[Daniel P. Berrangé]

The junit XML file produced by meson does not always have the
name 'testlog.junit.xml' - in the case of 'make check-functional'
there is a 'testlog-thorough.junit.xml' file too.

Improve CI debugging robustness by capturing all junit files that
meson produces.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml  | 4 ++--
 .gitlab-ci.d/buildtest.yml           | 2 +-
 .gitlab-ci.d/crossbuild-template.yml | 2 +-
 .gitlab-ci.d/custom-runners.yml      | 2 +-
 .gitlab-ci.d/windows.yml             | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 4cc5b20790..308490a35a 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -90,7 +90,7 @@
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
 
 .functional_test_job_template:
   extends: .common_test_job_template
@@ -107,7 +107,7 @@
       - build/meson-logs
       - build/tests/functional/*/*/*.log
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 778289267f..83c2867295 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -615,7 +615,7 @@ gcov:
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
       coverage_report:
         coverage_format: cobertura
         path: build/coverage.xml
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 7e70376cfc..58136d06e4 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -130,4 +130,4 @@
     paths:
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
index 1aa3c60efe..2d493f70f7 100644
--- a/.gitlab-ci.d/custom-runners.yml
+++ b/.gitlab-ci.d/custom-runners.yml
@@ -26,7 +26,7 @@
       - build/build.ninja
       - build/meson-logs
     reports:
-      junit: build/meson-logs/testlog.junit.xml
+      junit: build/meson-logs/*.junit.xml
 
 include:
   - local: '/.gitlab-ci.d/custom-runners/ubuntu-22.04-s390x.yml'
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index f14e9ca134..1e6a01bd9a 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -26,7 +26,7 @@ msys2-64bit:
     paths:
       - build/meson-logs
     reports:
-      junit: "build/meson-logs/testlog.junit.xml"
+      junit: build/meson-logs/*.junit.xml
   before_script:
   - Write-Output "Acquiring msys2.exe installer at $(Get-Date -Format u)"
   - If ( !(Test-Path -Path msys64\var\cache ) ) {
-- 
2.50.1

Re: [PATCH 3/4] gitlab: include all junit XML files from meson

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> The junit XML file produced by meson does not always have the
> name 'testlog.junit.xml' - in the case of 'make check-functional'
> there is a 'testlog-thorough.junit.xml' file too.
>
> Improve CI debugging robustness by capturing all junit files that
> meson produces.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

[PATCH 4/4] gitlab: prevent duplicated meson log artifacts in test jobs

[Daniel P. Berrangé]

The build jobs will populate build/meson-logs/ with various files
that are added as artifacts.

The test jobs preserve the state of the build jobs, so we must
delete any pre-existing logs to prevent confusion from duplicate
artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index 308490a35a..d866cb12bb 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -83,6 +83,10 @@
 
 .native_test_job_template:
   extends: .common_test_job_template
+  before_script:
+    # Prevent logs from the build job that run earlier
+    # from being duplicated in the test job artifacts
+    - rm -f build/meson-logs/*
   artifacts:
     name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
     when: always
@@ -111,6 +115,9 @@
   before_script:
     - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
     - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
+    # Prevent logs from the build job that run earlier
+    # from being duplicated in the test job artifacts
+    - rm -f build/meson-logs/*
   after_script:
     - cd build
     - du -chs ${CI_PROJECT_DIR}/*-cache
-- 
2.50.1

Re: [PATCH 4/4] gitlab: prevent duplicated meson log artifacts in test jobs

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> The build jobs will populate build/meson-logs/ with various files
> that are added as artifacts.
>
> The test jobs preserve the state of the build jobs, so we must
> delete any pre-existing logs to prevent confusion from duplicate
> artifacts.

Will this cause us to rebuild needlessly?

>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  .gitlab-ci.d/buildtest-template.yml | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> index 308490a35a..d866cb12bb 100644
> --- a/.gitlab-ci.d/buildtest-template.yml
> +++ b/.gitlab-ci.d/buildtest-template.yml
> @@ -83,6 +83,10 @@
>  
>  .native_test_job_template:
>    extends: .common_test_job_template
> +  before_script:
> +    # Prevent logs from the build job that run earlier
> +    # from being duplicated in the test job artifacts
> +    - rm -f build/meson-logs/*
>    artifacts:
>      name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
>      when: always
> @@ -111,6 +115,9 @@
>    before_script:
>      - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
>      - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
> +    # Prevent logs from the build job that run earlier
> +    # from being duplicated in the test job artifacts
> +    - rm -f build/meson-logs/*
>    after_script:
>      - cd build
>      - du -chs ${CI_PROJECT_DIR}/*-cache

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 4/4] gitlab: prevent duplicated meson log artifacts in test jobs

[Daniel P. Berrangé]

On Mon, Sep 08, 2025 at 10:21:38PM +0100, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > The build jobs will populate build/meson-logs/ with various files
> > that are added as artifacts.
> >
> > The test jobs preserve the state of the build jobs, so we must
> > delete any pre-existing logs to prevent confusion from duplicate
> > artifacts.
> 
> Will this cause us to rebuild needlessly?

These files aren't involved in any rebuild / stale checks &
the test pipeline I ran show no difference in what was done.

> 
> >
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >  .gitlab-ci.d/buildtest-template.yml | 7 +++++++
> >  1 file changed, 7 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> > index 308490a35a..d866cb12bb 100644
> > --- a/.gitlab-ci.d/buildtest-template.yml
> > +++ b/.gitlab-ci.d/buildtest-template.yml
> > @@ -83,6 +83,10 @@
> >  
> >  .native_test_job_template:
> >    extends: .common_test_job_template
> > +  before_script:
> > +    # Prevent logs from the build job that run earlier
> > +    # from being duplicated in the test job artifacts
> > +    - rm -f build/meson-logs/*
> >    artifacts:
> >      name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
> >      when: always
> > @@ -111,6 +115,9 @@
> >    before_script:
> >      - export QEMU_TEST_ALLOW_UNTRUSTED_CODE=1
> >      - export QEMU_TEST_CACHE_DIR=${CI_PROJECT_DIR}/functional-cache
> > +    # Prevent logs from the build job that run earlier
> > +    # from being duplicated in the test job artifacts
> > +    - rm -f build/meson-logs/*
> >    after_script:
> >      - cd build
> >      - du -chs ${CI_PROJECT_DIR}/*-cache
> 
> -- 
> Alex Bennée
> Virtualisation Tech Lead @ Linaro
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 4/4] gitlab: prevent duplicated meson log artifacts in test jobs

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Mon, Sep 08, 2025 at 10:21:38PM +0100, Alex Bennée wrote:
>> Daniel P. Berrangé <berrange@redhat.com> writes:
>> 
>> > The build jobs will populate build/meson-logs/ with various files
>> > that are added as artifacts.
>> >
>> > The test jobs preserve the state of the build jobs, so we must
>> > delete any pre-existing logs to prevent confusion from duplicate
>> > artifacts.
>> 
>> Will this cause us to rebuild needlessly?
>
> These files aren't involved in any rebuild / stale checks &
> the test pipeline I ran show no difference in what was done.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro