From: "Alex Bennée" <alex.bennee@linaro.org>
To: cota@braap.org, famz@redhat.com, berrange@redhat.com,
f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com,
aurelien@aurel32.net, agraf@suse.de, peter.maydell@linaro.org
Cc: qemu-devel@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>
Subject: [Qemu-devel] [PATCH v7 48/54] tests: add top-level make dependency for docker builds
Date: Fri, 15 Jun 2018 20:46:59 +0100 [thread overview]
Message-ID: <20180615194705.28019-49-alex.bennee@linaro.org> (raw)
In-Reply-To: <20180615194705.28019-1-alex.bennee@linaro.org>
One problem with satisfying your docker dependencies in a sub-make it
you might end up trying to satisfy the dependency multiple times. This
is especially a problem with debian-sid based cross compilers and CI
setups. We solve this by doing a docker build pass at the top level
before any sub-makes are called.
We still need to satisfy dependencies in the Makefile.target call so
people can run tests from individual target directories. We introduce
a new Makefile.probe which gets called for each PROBE_TARGET and
allows us to build up the list. It does require multiply including
config-target.mak which shouldn't cause any issues as it shouldn't
define anything that clashes with config-host.mak. However we undefine
a few key variables each time around.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
v7
- remove $(info) spam
- just add the explicit docker dependency to each build target
- include build-tcg-prereqs for CI tasks
---
tests/Makefile.include | 11 +++++++++++
tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++
2 files changed, 42 insertions(+)
create mode 100644 tests/tcg/Makefile.probe
diff --git a/tests/Makefile.include b/tests/Makefile.include
index c8faa01a33..2dd0700a88 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -938,6 +938,14 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS))
CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS))
RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS))
+# Probe for the Docker Builds needed for each build
+DOCKER_BUILD_TARGETS:=
+$(foreach PROBE_TARGET,$(TARGET_LIST), \
+ $(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) \
+ $(if $(DOCKER_PREREQ), \
+ $(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ)) \
+ $(eval build-tcg-tests-$(PROBE_TARGET): $(DOCKER_PREREQ))))
+
build-tcg-tests-%:
$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,)
@@ -947,6 +955,9 @@ run-tcg-tests-%: build-tcg-tests-%
clean-tcg-tests-%:
$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" clean-guest-tests,)
+.PHONY: build-tcg-prereqs
+build-tcg-prereqs: $(DOCKER_BUILD_TARGETS)
+
.PHONY: build-tcg
build-tcg: $(BUILD_TCG_TARGET_RULES)
diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe
new file mode 100644
index 0000000000..15c0412657
--- /dev/null
+++ b/tests/tcg/Makefile.probe
@@ -0,0 +1,31 @@
+# -*- Mode: makefile -*-
+#
+# TCG Compiler Probe
+#
+# This Makefile fragement is included multiple times in the main make
+# script to probe for available compilers. This is used to build up a
+# selection of required docker targets before we invoke a sub-make for
+# each target.
+
+# First we need the target makefile which tells us the target architecture
+-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak
+
+# Then we load up the target architecture makefiles which tell us
+# about the compilers
+CROSS_CC_GUEST:=
+DOCKER_IMAGE:=
+DOCKER_PREREQ:=
+
+-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include
+-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include
+
+ifndef CROSS_CC_GUEST
+ifneq ($(DOCKER_IMAGE),)
+DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE)
+endif
+endif
+
+# Clean-up
+# undefine TARGET_NAME
+# undefine TARGET_BASE_ARCH
+# undefine TARGET_ABI_DIR
--
2.17.1
next prev parent reply other threads:[~2018-06-15 19:55 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-15 19:46 [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 01/54] configure: add support for --cross-cc-FOO Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 02/54] configure: move i386_cc to cross_cc_i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 03/54] configure: allow user to specify --cross-cc-cflags-foo= Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 04/54] configure: set cross_cc_FOO for host compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 05/54] docker: Add "cc" subcommand Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 06/54] docker: extend "cc" command to accept compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 07/54] docker: allow "cc" command to run in user context Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 08/54] docker: Makefile.include introduce DOCKER_SCRIPT Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 09/54] tests/tcg: move architecture independent tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 10/54] tests/tcg/multiarch: Build fix for linux-test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 11/54] tests/tcg/multiarch: enable additional linux-test tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 12/54] tests/tcg/multiarch: don't hard code paths/ports for linux-test Alex Bennée
2018-06-16 19:46 ` Thomas Huth
2018-06-18 10:56 ` Alex Bennée
2018-06-18 11:08 ` Daniel P. Berrangé
2018-06-18 11:09 ` Thomas Huth
2018-06-18 11:14 ` Daniel P. Berrangé
2018-06-18 12:04 ` Alex Bennée
2018-06-18 12:40 ` Thomas Huth
2018-06-18 11:08 ` Thomas Huth
2018-06-16 22:24 ` Philippe Mathieu-Daudé
2018-06-17 9:18 ` Alex Bennée
2018-06-18 15:18 ` Philippe Mathieu-Daudé
2018-06-18 15:23 ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 13/54] tests/tcg/multiarch: move most output to stdout Alex Bennée
2018-06-16 19:48 ` Thomas Huth
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 14/54] tests/tcg: move i386 specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 15/54] tests/tcg: enable building for i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 16/54] tests/tcg/i386: Build fix for hello-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 17/54] tests/tcg/i386: fix test-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 18/54] tests/tcg/i386: add runner for test-i386-fprem Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 19/54] tests/tcg/x86_64: add Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 20/54] tests/tcg/i386/test-i386: use modern vector_size attributes Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 21/54] tests/tcg/i386/test-i386: fix printf format Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 22/54] tests/tcg: move ARM specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 23/54] tests/tcg: enable building for ARM Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 24/54] tests/tcg/arm: fix up test-arm-iwmmxt test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 25/54] tests/tcg: enable building for AArch64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 26/54] tests/tcg/arm: add fcvt test cases for AArch32/64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 27/54] tests/tcg: move MIPS specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 28/54] tests/tcg: enable building for MIPS Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 29/54] tests/docker/Makefile.include: fix mipsel-cross dependancy Alex Bennée
2018-06-16 22:27 ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 30/54] tests/tcg/mips: include common mips hello-mips Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 31/54] tests/tcg: enable building for s390x Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 32/54] tests/tcg: enable building for ppc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 33/54] tests/tcg: enable building for Alpha Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 34/54] tests/tcg/alpha: add Alpha specific tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 35/54] tests/tcg: enable building for HPPA Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 36/54] tests/tcg: enable building for m68k Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 37/54] tests/tcg: enable building for sh4 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 38/54] tests/tcg: enable building for sparc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 39/54] tests/tcg: enable building for mips64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 40/54] tests/tcg: enable building for RISCV64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 41/54] docker: move debian-powerpc-cross to sid based build Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 42/54] tests/tcg: enable building for PowerPC Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 43/54] tests/tcg/Makefile: update to be called from Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 44/54] Makefile.target: add (clean-/build-)guest-tests targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 45/54] tests/Makefile.include: add [build|clean|check]-tcg targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 46/54] tests/tcg: add run, diff, and skip helper macros Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 47/54] tests/tcg: override runners for broken tests Alex Bennée
2018-06-15 19:46 ` Alex Bennée [this message]
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 49/54] docker: docker.py don't conflate checksums for extra_files Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 50/54] docker: docker.py add check sub-command Alex Bennée
2018-06-18 15:06 ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 51/54] tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1 Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 52/54] docker: docker.py adding age check command Alex Bennée
2018-06-18 15:07 ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 53/54] tests/docker/Makefile.include: only force SID to NOCACHE if old Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 54/54] .travis.yml: add check-tcg test Alex Bennée
2018-06-15 22:29 ` [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! no-reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180615194705.28019-49-alex.bennee@linaro.org \
--to=alex.bennee@linaro.org \
--cc=agraf@suse.de \
--cc=aurelien@aurel32.net \
--cc=balrogg@gmail.com \
--cc=berrange@redhat.com \
--cc=cota@braap.org \
--cc=f4bug@amsat.org \
--cc=famz@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=richard.henderson@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).