From: "Alex Bennée" <alex.bennee@linaro.org>
To: qemu-devel@nongnu.org
Cc: peter.maydell@linaro.org, "Alex Bennée" <alex.bennee@linaro.org>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>,
"Thomas Huth" <thuth@redhat.com>,
"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
"Beraldo Leal" <bleal@redhat.com>
Subject: [PULL v2 22/24] gitlab: move the majority of artefact handling to a template
Date: Wed, 1 Mar 2023 15:16:02 +0000 [thread overview]
Message-ID: <20230301151604.1948813-23-alex.bennee@linaro.org> (raw)
In-Reply-To: <20230301151604.1948813-1-alex.bennee@linaro.org>
To avoid lots of copy and paste lets deal with artefacts in a
template. This way we can filter out most of the pre-binary object and
library files we no longer need as we have the final binaries.
build-system-alpine also saved .git-submodule-status so for simplicity
we bring that into the template as well.
As an example the build-system-ubuntu artefacts before this patch
where around 1.3 GB, after dropping the object files it comes to 970
MB.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230228190653.1602033-23-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index cb96b55c3f..a6cfe9be97 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -25,6 +25,22 @@
make -j"$JOBS" $MAKE_CHECK_ARGS ;
fi
+# We jump some hoops in common_test_job_template to avoid
+# rebuilding all the object files we skip in the artifacts
+.native_build_artifact_template:
+ artifacts:
+ expire_in: 2 days
+ paths:
+ - build
+ - .git-submodule-status
+ exclude:
+ - build/**/*.p
+ - build/**/*.a.p
+ - build/**/*.fa.p
+ - build/**/*.c.o
+ - build/**/*.c.o.d
+ - build/**/*.fa
+
.common_test_job_template:
extends: .base_job_template
stage: test
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 43f9e4a81d..44b8275299 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -2,7 +2,9 @@ include:
- local: '/.gitlab-ci.d/buildtest-template.yml'
build-system-alpine:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
- job: amd64-alpine-container
variables:
@@ -11,11 +13,6 @@ build-system-alpine:
microblazeel-softmmu mips64el-softmmu
MAKE_CHECK_ARGS: check-build
CONFIGURE_ARGS: --enable-docs --enable-trace-backends=log,simple,syslog
- artifacts:
- expire_in: 2 days
- paths:
- - .git-submodule-status
- - build
check-system-alpine:
extends: .native_test_job_template
@@ -36,7 +33,9 @@ avocado-system-alpine:
MAKE_CHECK_ARGS: check-avocado
build-system-ubuntu:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-ubuntu2204-container
variables:
@@ -45,10 +44,6 @@ build-system-ubuntu:
TARGETS: alpha-softmmu cris-softmmu hppa-softmmu
microblazeel-softmmu mips64el-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-ubuntu:
extends: .native_test_job_template
@@ -69,7 +64,9 @@ avocado-system-ubuntu:
MAKE_CHECK_ARGS: check-avocado
build-system-debian:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-debian-container
variables:
@@ -78,10 +75,6 @@ build-system-debian:
TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-debian:
extends: .native_test_job_template
@@ -114,7 +107,9 @@ crash-test-debian:
- tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-i386
build-system-fedora:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-fedora-container
variables:
@@ -123,10 +118,6 @@ build-system-fedora:
TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu
xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-fedora:
extends: .native_test_job_template
@@ -160,7 +151,9 @@ crash-test-fedora:
- tests/venv/bin/python3 scripts/device-crash-test -q ./qemu-system-riscv32
build-system-centos:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-centos8-container
variables:
@@ -170,10 +163,6 @@ build-system-centos:
TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-centos:
extends: .native_test_job_template
@@ -194,17 +183,15 @@ avocado-system-centos:
MAKE_CHECK_ARGS: check-avocado
build-system-opensuse:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-opensuse-leap-container
variables:
IMAGE: opensuse-leap
TARGETS: s390x-softmmu x86_64-softmmu aarch64-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-opensuse:
extends: .native_test_job_template
@@ -339,7 +326,9 @@ clang-user:
# Split in three sets of build/check/avocado to limit the execution time of each
# job
build-cfi-aarch64:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
- job: amd64-fedora-container
variables:
@@ -355,10 +344,6 @@ build-cfi-aarch64:
# skipped until the situation has been solved.
QEMU_JOB_SKIPPED: 1
timeout: 90m
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-cfi-aarch64:
extends: .native_test_job_template
@@ -379,7 +364,9 @@ avocado-cfi-aarch64:
MAKE_CHECK_ARGS: check-avocado
build-cfi-ppc64-s390x:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
- job: amd64-fedora-container
variables:
@@ -395,10 +382,6 @@ build-cfi-ppc64-s390x:
# skipped until the situation has been solved.
QEMU_JOB_SKIPPED: 1
timeout: 80m
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-cfi-ppc64-s390x:
extends: .native_test_job_template
@@ -419,7 +402,9 @@ avocado-cfi-ppc64-s390x:
MAKE_CHECK_ARGS: check-avocado
build-cfi-x86_64:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
- job: amd64-fedora-container
variables:
@@ -431,10 +416,6 @@ build-cfi-x86_64:
TARGETS: x86_64-softmmu
MAKE_CHECK_ARGS: check-build
timeout: 70m
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-cfi-x86_64:
extends: .native_test_job_template
@@ -564,7 +545,9 @@ build-libvhost-user:
# No targets are built here, just tools, docs, and unit tests. This
# also feeds into the eventual documentation deployment steps later
build-tools-and-docs-debian:
- extends: .native_build_job_template
+ extends:
+ - .native_build_job_template
+ - .native_build_artifact_template
needs:
job: amd64-debian-container
# when running on 'master' we use pre-existing container
@@ -574,10 +557,6 @@ build-tools-and-docs-debian:
MAKE_CHECK_ARGS: check-unit ctags TAGS cscope
CONFIGURE_ARGS: --disable-system --disable-user --enable-docs --enable-tools
QEMU_JOB_PUBLISH: 1
- artifacts:
- expire_in: 2 days
- paths:
- - build
# Prepare for GitLab pages deployment. Anything copied into the
# "public" directory will be deployed to $USER.gitlab.io/$PROJECT
--
2.39.2
next prev parent reply other threads:[~2023-03-01 15:26 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-01 15:15 [PULL v2 00/24] testing updates (gitlab, cirrus, docker, avocado, windows) Alex Bennée
2023-03-01 15:15 ` [PULL v2 01/24] tests: don't run socat tests on MacOS as well Alex Bennée
2023-03-01 15:15 ` [PULL v2 02/24] tests: add socat dependency for tests Alex Bennée
2023-03-01 15:15 ` [PULL v2 03/24] tests: be a bit more strict cleaning up fifos Alex Bennée
2023-03-01 15:15 ` [PULL v2 04/24] tests: make fp-test less chatty when running from test suite Alex Bennée
2023-03-01 15:15 ` [PULL v2 05/24] gitlab-ci: Use artifacts instead of dumping logs in the Cirrus-CI jobs Alex Bennée
2023-03-01 15:15 ` [PULL v2 06/24] gitlab: extend custom runners with base_job_template Alex Bennée
2023-03-01 15:15 ` [PULL v2 07/24] tests: don't run benchmarks for the tsan build Alex Bennée
2023-03-01 15:15 ` [PULL v2 08/24] testing: update ubuntu2004 to ubuntu2204 Alex Bennée
2023-03-01 15:15 ` [PULL v2 09/24] tests: skip the nios2 replay_kernel test Alex Bennée
2023-03-01 15:15 ` [PULL v2 10/24] tests: add tuxrun baseline test to avocado Alex Bennée
2023-03-01 15:15 ` [PULL v2 11/24] tests/docker: Use binaries for debian-tricore-cross Alex Bennée
2023-03-01 15:15 ` [PULL v2 12/24] tests: ensure we export job results for some cross builds Alex Bennée
2023-03-01 15:15 ` [PULL v2 13/24] cirrus.yml: Improve the windows_msys2_task Alex Bennée
2023-03-01 15:15 ` [PULL v2 14/24] tests/dockerfiles: unify debian-toolchain references Alex Bennée
2023-03-01 15:15 ` [PULL v2 15/24] gitlab: Use plain docker in container-template.yml Alex Bennée
2023-03-01 15:15 ` [PULL v2 16/24] tests: Ensure TAP version is printed before other messages Alex Bennée
2023-03-01 15:15 ` [PULL v2 17/24] configure: expose the direct container command Alex Bennée
2023-03-01 15:15 ` [PULL v2 18/24] tests/lcitool: append user setting stanza to dockerfiles Alex Bennée
2023-03-01 15:15 ` [PULL v2 19/24] tests/docker: add USER stanzas to non-lci images Alex Bennée
2023-03-01 15:16 ` [PULL v2 20/24] tests/docker: use direct RUNC call to build containers Alex Bennée
2023-03-01 15:16 ` [PULL v2 21/24] tests/docker: use direct RUNC call to run test jobs Alex Bennée
2023-03-01 15:16 ` Alex Bennée [this message]
2023-03-01 15:16 ` [PULL v2 23/24] tests/docker: add zstdtools to the images Alex Bennée
2023-03-01 15:16 ` [PULL v2 24/24] tests/avocado: disable BootLinuxPPC64 test in CI Alex Bennée
2023-03-01 17:39 ` [PULL v2 00/24] testing updates (gitlab, cirrus, docker, avocado, windows) Peter Maydell
2023-03-01 18:17 ` Alex Bennée
2023-03-01 19:19 ` Peter Maydell
2023-03-01 19:47 ` Alex Bennée
2023-03-02 10:56 ` Peter Maydell
2023-03-02 11:15 ` Philippe Mathieu-Daudé
2023-03-02 11:19 ` Thomas Huth
2023-03-02 11:21 ` Peter Maydell
2023-03-02 11:34 ` Alex Bennée
2023-03-02 11:24 ` Alex Bennée
2023-03-02 12:00 ` Daniel P. Berrangé
2023-03-02 12:37 ` Philippe Mathieu-Daudé
2023-03-02 11:30 ` Alex Bennée
2023-03-02 11:34 ` Peter Maydell
2023-03-02 12:03 ` Alex Bennée
2023-03-02 16:02 ` Peter Maydell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230301151604.1948813-23-alex.bennee@linaro.org \
--to=alex.bennee@linaro.org \
--cc=bleal@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=thuth@redhat.com \
--cc=wainersm@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).