qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Alex Bennée" <alex.bennee@linaro.org>
To: cota@braap.org, famz@redhat.com, berrange@redhat.com,
	f4bug@amsat.org, richard.henderson@linaro.org, balrogg@gmail.com,
	aurelien@aurel32.net, agraf@suse.de, peter.maydell@linaro.org
Cc: qemu-devel@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>
Subject: [Qemu-devel] [PATCH v7 48/54] tests: add top-level make dependency for docker builds
Date: Fri, 15 Jun 2018 20:46:59 +0100	[thread overview]
Message-ID: <20180615194705.28019-49-alex.bennee@linaro.org> (raw)
In-Reply-To: <20180615194705.28019-1-alex.bennee@linaro.org>

One problem with satisfying your docker dependencies in a sub-make it
you might end up trying to satisfy the dependency multiple times. This
is especially a problem with debian-sid based cross compilers and CI
setups. We solve this by doing a docker build pass at the top level
before any sub-makes are called.

We still need to satisfy dependencies in the Makefile.target call so
people can run tests from individual target directories. We introduce
a new Makefile.probe which gets called for each PROBE_TARGET and
allows us to build up the list. It does require multiply including
config-target.mak which shouldn't cause any issues as it shouldn't
define anything that clashes with config-host.mak. However we undefine
a few key variables each time around.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v7
  - remove $(info) spam
  - just add the explicit docker dependency to each build target
  - include build-tcg-prereqs for CI tasks
---
 tests/Makefile.include   | 11 +++++++++++
 tests/tcg/Makefile.probe | 31 +++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+)
 create mode 100644 tests/tcg/Makefile.probe

diff --git a/tests/Makefile.include b/tests/Makefile.include
index c8faa01a33..2dd0700a88 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -938,6 +938,14 @@ BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(LINUX_USER_TARGETS))
 CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(LINUX_USER_TARGETS))
 RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(LINUX_USER_TARGETS))
 
+# Probe for the Docker Builds needed for each build
+DOCKER_BUILD_TARGETS:=
+$(foreach PROBE_TARGET,$(TARGET_LIST), 				\
+	$(eval -include $(SRC_PATH)/tests/tcg/Makefile.probe) 	\
+	$(if $(DOCKER_PREREQ), 					\
+		$(eval DOCKER_BUILD_TARGETS+=$(DOCKER_PREREQ))  \
+		$(eval build-tcg-tests-$(PROBE_TARGET): $(DOCKER_PREREQ))))
+
 build-tcg-tests-%:
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" guest-tests,)
 
@@ -947,6 +955,9 @@ run-tcg-tests-%: build-tcg-tests-%
 clean-tcg-tests-%:
 	$(call quiet-command,$(MAKE) $(SUBDIR_MAKEFLAGS) -C $* V="$(V)" TARGET_DIR="$*/" clean-guest-tests,)
 
+.PHONY: build-tcg-prereqs
+build-tcg-prereqs: $(DOCKER_BUILD_TARGETS)
+
 .PHONY: build-tcg
 build-tcg: $(BUILD_TCG_TARGET_RULES)
 
diff --git a/tests/tcg/Makefile.probe b/tests/tcg/Makefile.probe
new file mode 100644
index 0000000000..15c0412657
--- /dev/null
+++ b/tests/tcg/Makefile.probe
@@ -0,0 +1,31 @@
+# -*- Mode: makefile -*-
+#
+# TCG Compiler Probe
+#
+# This Makefile fragement is included multiple times in the main make
+# script to probe for available compilers. This is used to build up a
+# selection of required docker targets before we invoke a sub-make for
+# each target.
+
+# First we need the target makefile which tells us the target architecture
+-include $(BUILD_DIR)/$(PROBE_TARGET)/config-target.mak
+
+# Then we load up the target architecture makefiles which tell us
+# about the compilers
+CROSS_CC_GUEST:=
+DOCKER_IMAGE:=
+DOCKER_PREREQ:=
+
+-include $(SRC_PATH)/tests/tcg/$(TARGET_BASE_ARCH)/Makefile.include
+-include $(SRC_PATH)/tests/tcg/$(TARGET_NAME)/Makefile.include
+
+ifndef CROSS_CC_GUEST
+ifneq ($(DOCKER_IMAGE),)
+DOCKER_PREREQ:=docker-image-$(DOCKER_IMAGE)
+endif
+endif
+
+# Clean-up
+# undefine TARGET_NAME
+# undefine TARGET_BASE_ARCH
+# undefine TARGET_ABI_DIR
-- 
2.17.1

  parent reply	other threads:[~2018-06-15 19:55 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-15 19:46 [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 01/54] configure: add support for --cross-cc-FOO Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 02/54] configure: move i386_cc to cross_cc_i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 03/54] configure: allow user to specify --cross-cc-cflags-foo= Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 04/54] configure: set cross_cc_FOO for host compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 05/54] docker: Add "cc" subcommand Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 06/54] docker: extend "cc" command to accept compiler Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 07/54] docker: allow "cc" command to run in user context Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 08/54] docker: Makefile.include introduce DOCKER_SCRIPT Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 09/54] tests/tcg: move architecture independent tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 10/54] tests/tcg/multiarch: Build fix for linux-test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 11/54] tests/tcg/multiarch: enable additional linux-test tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 12/54] tests/tcg/multiarch: don't hard code paths/ports for linux-test Alex Bennée
2018-06-16 19:46   ` Thomas Huth
2018-06-18 10:56     ` Alex Bennée
2018-06-18 11:08       ` Daniel P. Berrangé
2018-06-18 11:09         ` Thomas Huth
2018-06-18 11:14           ` Daniel P. Berrangé
2018-06-18 12:04         ` Alex Bennée
2018-06-18 12:40           ` Thomas Huth
2018-06-18 11:08       ` Thomas Huth
2018-06-16 22:24   ` Philippe Mathieu-Daudé
2018-06-17  9:18     ` Alex Bennée
2018-06-18 15:18       ` Philippe Mathieu-Daudé
2018-06-18 15:23         ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 13/54] tests/tcg/multiarch: move most output to stdout Alex Bennée
2018-06-16 19:48   ` Thomas Huth
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 14/54] tests/tcg: move i386 specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 15/54] tests/tcg: enable building for i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 16/54] tests/tcg/i386: Build fix for hello-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 17/54] tests/tcg/i386: fix test-i386 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 18/54] tests/tcg/i386: add runner for test-i386-fprem Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 19/54] tests/tcg/x86_64: add Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 20/54] tests/tcg/i386/test-i386: use modern vector_size attributes Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 21/54] tests/tcg/i386/test-i386: fix printf format Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 22/54] tests/tcg: move ARM specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 23/54] tests/tcg: enable building for ARM Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 24/54] tests/tcg/arm: fix up test-arm-iwmmxt test Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 25/54] tests/tcg: enable building for AArch64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 26/54] tests/tcg/arm: add fcvt test cases for AArch32/64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 27/54] tests/tcg: move MIPS specific tests into subdir Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 28/54] tests/tcg: enable building for MIPS Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 29/54] tests/docker/Makefile.include: fix mipsel-cross dependancy Alex Bennée
2018-06-16 22:27   ` Philippe Mathieu-Daudé
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 30/54] tests/tcg/mips: include common mips hello-mips Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 31/54] tests/tcg: enable building for s390x Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 32/54] tests/tcg: enable building for ppc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 33/54] tests/tcg: enable building for Alpha Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 34/54] tests/tcg/alpha: add Alpha specific tests Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 35/54] tests/tcg: enable building for HPPA Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 36/54] tests/tcg: enable building for m68k Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 37/54] tests/tcg: enable building for sh4 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 38/54] tests/tcg: enable building for sparc64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 39/54] tests/tcg: enable building for mips64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 40/54] tests/tcg: enable building for RISCV64 Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 41/54] docker: move debian-powerpc-cross to sid based build Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 42/54] tests/tcg: enable building for PowerPC Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 43/54] tests/tcg/Makefile: update to be called from Makefile.target Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 44/54] Makefile.target: add (clean-/build-)guest-tests targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 45/54] tests/Makefile.include: add [build|clean|check]-tcg targets Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 46/54] tests/tcg: add run, diff, and skip helper macros Alex Bennée
2018-06-15 19:46 ` [Qemu-devel] [PATCH v7 47/54] tests/tcg: override runners for broken tests Alex Bennée
2018-06-15 19:46 ` Alex Bennée [this message]
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 49/54] docker: docker.py don't conflate checksums for extra_files Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 50/54] docker: docker.py add check sub-command Alex Bennée
2018-06-18 15:06   ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 51/54] tests/Makefile: call sub-makes with SKIP_DOCKER_BUILD=1 Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 52/54] docker: docker.py adding age check command Alex Bennée
2018-06-18 15:07   ` Philippe Mathieu-Daudé
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 53/54] tests/docker/Makefile.include: only force SID to NOCACHE if old Alex Bennée
2018-06-15 19:47 ` [Qemu-devel] [PATCH v7 54/54] .travis.yml: add check-tcg test Alex Bennée
2018-06-15 22:29 ` [Qemu-devel] [PATCH v7 00/54] fix building of tests/tcg - last chance to review! no-reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180615194705.28019-49-alex.bennee@linaro.org \
    --to=alex.bennee@linaro.org \
    --cc=agraf@suse.de \
    --cc=aurelien@aurel32.net \
    --cc=balrogg@gmail.com \
    --cc=berrange@redhat.com \
    --cc=cota@braap.org \
    --cc=f4bug@amsat.org \
    --cc=famz@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=richard.henderson@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).