[Qemu-devel] [PATCH v3 0/5] generic docker run patches

[Qemu-devel] [PATCH v3 0/5] generic docker run patches

[Alex Bennée]

Hi Fam,

I've re-based the series for the generic run target. The aim being to
allow a developer to run tests against any generic docker target even
if it is not in the list of approved targets:

  make docker-run TEST=test-quick IMAGE=debian:arm64 \
    EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1

Since last post:
  - fix squashing of TARGET_LIST
  - add new patch to remove verification (could be squashed)

Alex Bennée (5):
  tests/docker: add travis dockerfile
  tests/docker: test-build script
  tests/docker: make test-mingw honour TARGET_LIST
  tests/docker/Makefile.include: add a generic docker-run target
  tests/docker/Makefile.include: remove verification targets

 tests/docker/Makefile.include          | 61 +++++++++++++++++++++-------------
 tests/docker/dockerfiles/travis.docker |  6 ++++
 tests/docker/test-build                | 20 +++++++++++
 tests/docker/test-mingw                |  3 +-
 4 files changed, 66 insertions(+), 24 deletions(-)
 create mode 100644 tests/docker/dockerfiles/travis.docker
 create mode 100755 tests/docker/test-build

-- 
2.9.3

[Qemu-devel] [PATCH v3 1/5] tests/docker: add travis dockerfile

[Alex Bennée]

This target grabs the latest Travis containers from their repository at
quay.io and then installs QEMU's build dependencies. With this it is
possible to run on broadly the same setup as they have on travis-ci.org.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/travis.docker | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 tests/docker/dockerfiles/travis.docker

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
new file mode 100644
index 0000000..e4983ae
--- /dev/null
+++ b/tests/docker/dockerfiles/travis.docker
@@ -0,0 +1,6 @@
+FROM quay.io/travisci/travis-ruby
+RUN apt-get update
+RUN apt-get -y build-dep qemu
+RUN apt-get -y build-dep device-tree-compiler
+RUN apt-get -y install python2.7 dh-autoreconf
+ENV FEATURES pyyaml
-- 
2.9.3

[Qemu-devel] [PATCH v3 2/5] tests/docker: test-build script

[Alex Bennée]

Much like test-quick but only builds. This is useful for some of the
build targets like ThreadSanitizer that don't yet pass "make check".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v3
  - fix comments for script
  - merge with new build_dir
  - flatten default target list
---
 tests/docker/test-build | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100755 tests/docker/test-build

diff --git a/tests/docker/test-build b/tests/docker/test-build
new file mode 100755
index 0000000..031a7d9
--- /dev/null
+++ b/tests/docker/test-build
@@ -0,0 +1,20 @@
+#!/bin/bash -e
+#
+# Quick compile test without the make check step of test-quick.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
-- 
2.9.3

[Qemu-devel] [PATCH v3 3/5] tests/docker: make test-mingw honour TARGET_LIST

[Alex Bennée]

The other builders honour this variable, so should the mingw build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-mingw | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
index 3396876..2adadcb 100755
--- a/tests/docker/test-mingw
+++ b/tests/docker/test-mingw
@@ -16,9 +16,10 @@
 requires mingw dtc
 
 cd "$BUILD_DIR"
+DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
 
 for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
-    TARGET_LIST=x86_64-softmmu,aarch64-softmmu \
+    TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
         build_qemu --cross-prefix=$prefix \
         --enable-trace-backends=simple \
         --enable-debug \
-- 
2.9.3

[Qemu-devel] [PATCH v3 4/5] tests/docker/Makefile.include: add a generic docker-run target

[Alex Bennée]

This re-factors the docker makefile to include a docker-run target which
can be controlled entirely from environment variables specified on the
make command line. This allows us to run against any given docker image
we may have in our repository, for example:

    make docker-run TEST="test-quick" IMAGE="debian:arm64" \
         EXECUTABLE=./aarch64-linux-user/qemu-aarch64

The existing docker-foo@bar targets still work but the inline
verification has been shunted into other target prerequisites before a
sub-make is invoked for the docker-run target.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
NB: I dropped the awk magic that verifies the image exists before
running. I couldn't get the thing to work in my shell so wasn't quite
sure what it was doing.

v2
 - fix spelling on arbitrary
 - document docker-run in help
 - add note on docker.py update
 - reduce noise on verifying other images
 - revert back to using filter
 - fix merge conflict
---
 tests/docker/Makefile.include | 89 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 66 insertions(+), 23 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index b44daab..ba7c984 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,6 +78,7 @@ docker:
 	@echo '                         "IMAGE" is one of the listed container name."'
 	@echo '    docker-image:        Build all images.'
 	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
+	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
 	@echo
 	@echo 'Available container images:'
 	@echo '    $(DOCKER_IMAGES)'
@@ -101,31 +102,73 @@ docker:
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
-docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
-docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%: docker-qemu-src
+# This rule if for directly running against an arbitrary docker target.
+# It is called by the expanded docker targets (e.g. make
+# docker-test-foo@bar) which will do additional verification.
+#
+# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
+#
+docker-run: docker-qemu-src
 	@mkdir -p "$(DOCKER_CCACHE_DIR)"
-	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
-		then echo "Invalid target"; exit 1; \
+	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
+		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
+	fi
+	$(if $(EXECUTABLE),						\
+		$(call quiet-command,					\
+			$(SRC_PATH)/tests/docker/docker.py update 	\
+			$(IMAGE) $(EXECUTABLE),				\
+			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
+	$(call quiet-command,						\
+		$(SRC_PATH)/tests/docker/docker.py run 			\
+			-t 						\
+			$(if $V,,--rm) 					\
+			$(if $(DEBUG),-i,--net=none) 			\
+			-e TARGET_LIST=$(TARGET_LIST) 			\
+			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
+			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
+			-e SHOW_ENV=$(SHOW_ENV) 			\
+			-e CCACHE_DIR=/var/tmp/ccache 			\
+			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
+			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
+			$(IMAGE) 					\
+			/var/tmp/qemu/run 				\
+			$(TEST), "  RUN $(TEST) in ${IMAGE}")
+
+#
+# Verification targets
+#
+# These targets help verify the test (CMD) and docker tag (IMAGE) are
+# part of the built in set of tests and images. You can still call the
+# docker-run target directly for testsing against arbitary images.
+#
+
+docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
+docker-verify-image-%:
+	@if test -z "$(IMAGE)";			 	\
+		then echo "Invalid image"; exit 1; 	\
 	fi
-	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
-		$(call quiet-command,\
-			if $(SRC_PATH)/tests/docker/docker.py images | \
-				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
-				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
-				-t \
-				$(if $(DEBUG),-i,--net=none) \
-				-e TARGET_LIST=$(TARGET_LIST) \
-				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
-				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
-				-e CCACHE_DIR=/var/tmp/ccache \
-				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
-				qemu:$(IMAGE) \
-				/var/tmp/qemu/run \
-				$(CMD); \
-			fi \
-			,"RUN","$(CMD) in $(IMAGE)")))
+	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
+		echo "$(IMAGE) is not a known image"; exit 1)	\
+
+docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
+docker-verify-test-%:
+	@if test -z "$(CMD)"; 				\
+		then echo "Invalid test"; exit 1; 	\
+	fi
+	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
+		echo "$(CMD) is not a known test"; exit 1)
+
+# Run targets
+#
+# This will take a target such as docker-test-foo@bar and verify that:
+#  - the test test-foo is a known test
+#  - the image bar is a known image
+#
+# It will then call the docker-run
+docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
+docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
+docker-run-%: docker-verify-image-% docker-verify-test-%
+	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
 
 docker-clean:
 	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
-- 
2.9.3

[Qemu-devel] [PATCH v3 5/5] tests/docker/Makefile.include: remove verification targets

[Alex Bennée]

As it turns out there are a bunch of verification targets we don't hit
because of other matching patterns in rules.mk. Drop the verification
steps all together.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v3
  - new for v3, could be squashed with previous commit
---
 tests/docker/Makefile.include | 34 +++-------------------------------
 1 file changed, 3 insertions(+), 31 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ba7c984..925f711 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -134,40 +134,12 @@ docker-run: docker-qemu-src
 			/var/tmp/qemu/run 				\
 			$(TEST), "  RUN $(TEST) in ${IMAGE}")
 
+# Run targets:
 #
-# Verification targets
-#
-# These targets help verify the test (CMD) and docker tag (IMAGE) are
-# part of the built in set of tests and images. You can still call the
-# docker-run target directly for testsing against arbitary images.
-#
-
-docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
-docker-verify-image-%:
-	@if test -z "$(IMAGE)";			 	\
-		then echo "Invalid image"; exit 1; 	\
-	fi
-	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
-		echo "$(IMAGE) is not a known image"; exit 1)	\
-
-docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
-docker-verify-test-%:
-	@if test -z "$(CMD)"; 				\
-		then echo "Invalid test"; exit 1; 	\
-	fi
-	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
-		echo "$(CMD) is not a known test"; exit 1)
-
-# Run targets
-#
-# This will take a target such as docker-test-foo@bar and verify that:
-#  - the test test-foo is a known test
-#  - the image bar is a known image
-#
-# It will then call the docker-run
+# Of the form docker-TEST-FOO@IMAGE-BAR which will then be expanded into a call to "make docker-run"
 docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
 docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%: docker-verify-image-% docker-verify-test-%
+docker-run-%:
 	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
 
 docker-clean:
-- 
2.9.3

Re: [Qemu-devel] [PATCH v3 0/5] generic docker run patches

[Fam Zheng]

On Tue, 10/11 17:16, Alex Bennée wrote:
> Hi Fam,
> 
> I've re-based the series for the generic run target. The aim being to
> allow a developer to run tests against any generic docker target even
> if it is not in the list of approved targets:
> 
>   make docker-run TEST=test-quick IMAGE=debian:arm64 \
>     EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1
> 
> Since last post:
>   - fix squashing of TARGET_LIST
>   - add new patch to remove verification (could be squashed)

Queued for next pull, thanks!

Fam

> 
> Alex Bennée (5):
>   tests/docker: add travis dockerfile
>   tests/docker: test-build script
>   tests/docker: make test-mingw honour TARGET_LIST
>   tests/docker/Makefile.include: add a generic docker-run target
>   tests/docker/Makefile.include: remove verification targets
> 
>  tests/docker/Makefile.include          | 61 +++++++++++++++++++++-------------
>  tests/docker/dockerfiles/travis.docker |  6 ++++
>  tests/docker/test-build                | 20 +++++++++++
>  tests/docker/test-mingw                |  3 +-
>  4 files changed, 66 insertions(+), 24 deletions(-)
>  create mode 100644 tests/docker/dockerfiles/travis.docker
>  create mode 100755 tests/docker/test-build
> 
> -- 
> 2.9.3
>