[Qemu-devel] [PATCH v2 0/4] generic docker run patches

[Qemu-devel] [PATCH v2 0/4] generic docker run patches

[Alex Bennée]

Hi Fam,

I've re-based the series for the generic run target. The aim being to
allow a developer to run tests against any generic docker target even
if it is not in the list of approved targets:

  make docker-run TEST=test-quick IMAGE=debian:arm64 \
    EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1

The existing auto targets still work but have been converted to use
the generic runner once they have done the validation.

Since last post:
  - review comments on the main patch
  - added a travis docker file
  - minor tweak to the test-mingw script

Alex Bennée (4):
  tests/docker: add travis dockerfile
  tests/docker: test-build script
  tests/docker: make test-mingw honour TARGET_LIST
  tests/docker/Makefile.include: add a generic docker-run target

 tests/docker/Makefile.include          | 89 +++++++++++++++++++++++++---------
 tests/docker/dockerfiles/travis.docker |  6 +++
 tests/docker/test-build                | 18 +++++++
 tests/docker/test-mingw                |  4 +-
 4 files changed, 93 insertions(+), 24 deletions(-)
 create mode 100644 tests/docker/dockerfiles/travis.docker
 create mode 100755 tests/docker/test-build

-- 
2.9.3

[Qemu-devel] [PATCH v2 1/4] tests/docker: add travis dockerfile

[Alex Bennée]

This target grabs the latest Travis containers from their repository at
quay.io and then installs QEMU's build dependencies. With this it is
possible to run on broadly the same setup as they have on travis-ci.org.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/travis.docker | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 tests/docker/dockerfiles/travis.docker

diff --git a/tests/docker/dockerfiles/travis.docker b/tests/docker/dockerfiles/travis.docker
new file mode 100644
index 0000000..e4983ae
--- /dev/null
+++ b/tests/docker/dockerfiles/travis.docker
@@ -0,0 +1,6 @@
+FROM quay.io/travisci/travis-ruby
+RUN apt-get update
+RUN apt-get -y build-dep qemu
+RUN apt-get -y build-dep device-tree-compiler
+RUN apt-get -y install python2.7 dh-autoreconf
+ENV FEATURES pyyaml
-- 
2.9.3

[Qemu-devel] [PATCH v2 2/4] tests/docker: test-build script

[Alex Bennée]

Much like test-quick but only builds. This is useful for some of the
build targets like ThreadSanitizer that don't yet pass "make check".

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-build | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100755 tests/docker/test-build

diff --git a/tests/docker/test-build b/tests/docker/test-build
new file mode 100755
index 0000000..d237ead
--- /dev/null
+++ b/tests/docker/test-build
@@ -0,0 +1,18 @@
+#!/bin/bash -e
+#
+# Quick compiling test that everyone already does. But why not automate it?
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng <famz@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2 2/4] tests/docker: test-build script

[Fam Zheng]

On Mon, 10/03 17:32, Alex Bennée wrote:
> Much like test-quick but only builds. This is useful for some of the
> build targets like ThreadSanitizer that don't yet pass "make check".
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/test-build | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>  create mode 100755 tests/docker/test-build
> 
> diff --git a/tests/docker/test-build b/tests/docker/test-build
> new file mode 100755
> index 0000000..d237ead
> --- /dev/null
> +++ b/tests/docker/test-build
> @@ -0,0 +1,18 @@
> +#!/bin/bash -e
> +#
> +# Quick compiling test that everyone already does. But why not automate it?
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng <famz@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"

Like I said in v1, better flatten this to "x86_64-softmmu,aarch64-softmmu" as it
is in test-quick now.

Fam

> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> +build_qemu
> -- 
> 2.9.3
> 

Re: [Qemu-devel] [PATCH v2 2/4] tests/docker: test-build script

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Mon, 10/03 17:32, Alex Bennée wrote:
>> Much like test-quick but only builds. This is useful for some of the
>> build targets like ThreadSanitizer that don't yet pass "make check".
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  tests/docker/test-build | 18 ++++++++++++++++++
>>  1 file changed, 18 insertions(+)
>>  create mode 100755 tests/docker/test-build
>>
>> diff --git a/tests/docker/test-build b/tests/docker/test-build
>> new file mode 100755
>> index 0000000..d237ead
>> --- /dev/null
>> +++ b/tests/docker/test-build
>> @@ -0,0 +1,18 @@
>> +#!/bin/bash -e
>> +#
>> +# Quick compiling test that everyone already does. But why not automate it?
>> +#
>> +# Copyright (c) 2016 Red Hat Inc.
>> +#
>> +# Authors:
>> +#  Fam Zheng <famz@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2
>> +# or (at your option) any later version. See the COPYING file in
>> +# the top-level directory.
>> +
>> +. common.rc
>> +
>> +DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
>
> Like I said in v1, better flatten this to "x86_64-softmmu,aarch64-softmmu" as it
> is in test-quick now.

oops, sorry missed that. Will fix.

>
> Fam
>
>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>> +build_qemu
>> --
>> 2.9.3
>>


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 2/4] tests/docker: test-build script

[Fam Zheng]

On Tue, 10/11 09:49, Alex Bennée wrote:
> 
> Fam Zheng <famz@redhat.com> writes:
> 
> > On Mon, 10/03 17:32, Alex Bennée wrote:
> >> Much like test-quick but only builds. This is useful for some of the
> >> build targets like ThreadSanitizer that don't yet pass "make check".
> >>
> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >> ---
> >>  tests/docker/test-build | 18 ++++++++++++++++++
> >>  1 file changed, 18 insertions(+)
> >>  create mode 100755 tests/docker/test-build
> >>
> >> diff --git a/tests/docker/test-build b/tests/docker/test-build
> >> new file mode 100755
> >> index 0000000..d237ead
> >> --- /dev/null
> >> +++ b/tests/docker/test-build
> >> @@ -0,0 +1,18 @@
> >> +#!/bin/bash -e
> >> +#
> >> +# Quick compiling test that everyone already does. But why not automate it?
> >> +#
> >> +# Copyright (c) 2016 Red Hat Inc.
> >> +#
> >> +# Authors:
> >> +#  Fam Zheng <famz@redhat.com>
> >> +#
> >> +# This work is licensed under the terms of the GNU GPL, version 2
> >> +# or (at your option) any later version. See the COPYING file in
> >> +# the top-level directory.
> >> +
> >> +. common.rc
> >> +
> >> +DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
> >
> > Like I said in v1, better flatten this to "x86_64-softmmu,aarch64-softmmu" as it
> > is in test-quick now.
> 
> oops, sorry missed that. Will fix.

While we are at it, how about copying from test-full to cover more targets?

Fam

> 
> >
> > Fam
> >
> >> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> >> +build_qemu
> >> --
> >> 2.9.3
> >>
> 
> 
> --
> Alex Bennée

[Qemu-devel] [PATCH v2 3/4] tests/docker: make test-mingw honour TARGET_LIST

[Alex Bennée]

The other builders honour this variable, so should the mingw build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-mingw | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
index c03757a..51e9c0b 100755
--- a/tests/docker/test-mingw
+++ b/tests/docker/test-mingw
@@ -15,8 +15,10 @@
 
 requires mingw dtc
 
+DEF_TARGET_LIST="x86_64-softmmu,aarch64-softmmu"
+
 for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
-    TARGET_LIST=x86_64-softmmu,aarch64-softmmu \
+    TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
         build_qemu --cross-prefix=$prefix \
         --enable-trace-backends=simple \
         --enable-debug \
-- 
2.9.3

[Qemu-devel] [PATCH v2 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Alex Bennée]

This re-factors the docker makefile to include a docker-run target which
can be controlled entirely from environment variables specified on the
make command line. This allows us to run against any given docker image
we may have in our repository, for example:

    make docker-run TEST="test-quick" IMAGE="debian:arm64" \
         EXECUTABLE=./aarch64-linux-user/qemu-aarch64

The existing docker-foo@bar targets still work but the inline
verification has been shunted into other target prerequisites before a
sub-make is invoked for the docker-run target.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
NB: I dropped the awk magic that verifies the image exists before
running. I couldn't get the thing to work in my shell so wasn't quite
sure what it was doing.

v2
 - fix spelling on arbitrary
 - document docker-run in help
 - add note on docker.py update
 - reduce noise on verifying other images
 - revert back to using filter
 - fix merge conflict
---
 tests/docker/Makefile.include | 89 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 66 insertions(+), 23 deletions(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 2fcc3c6..87735d8 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -78,6 +78,7 @@ docker:
 	@echo '                         "IMAGE" is one of the listed container name."'
 	@echo '    docker-image:        Build all images.'
 	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
+	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
 	@echo
 	@echo 'Available container images:'
 	@echo '    $(DOCKER_IMAGES)'
@@ -101,31 +102,73 @@ docker:
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 
-docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
-docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
-docker-run-%: docker-qemu-src
+# This rule if for directly running against an arbitrary docker target.
+# It is called by the expanded docker targets (e.g. make
+# docker-test-foo@bar) which will do additional verification.
+#
+# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
+#
+docker-run: docker-qemu-src
 	@mkdir -p "$(DOCKER_CCACHE_DIR)"
-	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
-		then echo "Invalid target"; exit 1; \
+	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
+		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
+	fi
+	$(if $(EXECUTABLE),						\
+		$(call quiet-command,					\
+			$(SRC_PATH)/tests/docker/docker.py update 	\
+			$(IMAGE) $(EXECUTABLE),				\
+			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
+	$(call quiet-command,						\
+		$(SRC_PATH)/tests/docker/docker.py run 			\
+			-t 						\
+			$(if $V,,--rm) 					\
+			$(if $(DEBUG),-i,--net=none) 			\
+			-e TARGET_LIST=$(TARGET_LIST) 			\
+			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
+			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
+			-e SHOW_ENV=$(SHOW_ENV) 			\
+			-e CCACHE_DIR=/var/tmp/ccache 			\
+			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
+			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
+			$(IMAGE) 					\
+			/var/tmp/qemu/run 				\
+			$(TEST), "  RUN $(TEST) in ${IMAGE}")
+
+#
+# Verification targets
+#
+# These targets help verify the test (CMD) and docker tag (IMAGE) are
+# part of the built in set of tests and images. You can still call the
+# docker-run target directly for testsing against arbitary images.
+#
+
+docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
+docker-verify-image-%:
+	@if test -z "$(IMAGE)";			 	\
+		then echo "Invalid image"; exit 1; 	\
 	fi
-	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
-		$(call quiet-command,\
-			if $(SRC_PATH)/tests/docker/docker.py images | \
-				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
-				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
-				-t \
-				$(if $(DEBUG),-i,--net=none) \
-				-e TARGET_LIST=$(TARGET_LIST) \
-				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
-				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
-				-e CCACHE_DIR=/var/tmp/ccache \
-				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
-				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
-				qemu:$(IMAGE) \
-				/var/tmp/qemu/run \
-				$(CMD); \
-			fi \
-			, "  RUN $(CMD) in $(IMAGE)")))
+	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
+		echo "$(IMAGE) is not a known image"; exit 1)	\
+
+docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
+docker-verify-test-%:
+	@if test -z "$(CMD)"; 				\
+		then echo "Invalid test"; exit 1; 	\
+	fi
+	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
+		echo "$(CMD) is not a known test"; exit 1)
+
+# Run targets
+#
+# This will take a target such as docker-test-foo@bar and verify that:
+#  - the test test-foo is a known test
+#  - the image bar is a known image
+#
+# It will then call the docker-run
+docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
+docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
+docker-run-%: docker-verify-image-% docker-verify-test-%
+	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
 
 docker-clean:
 	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
-- 
2.9.3

Re: [Qemu-devel] [PATCH v2 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Fam Zheng]

On Mon, 10/03 17:32, Alex Bennée wrote:
> This re-factors the docker makefile to include a docker-run target which
> can be controlled entirely from environment variables specified on the
> make command line. This allows us to run against any given docker image
> we may have in our repository, for example:
> 
>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> 
> The existing docker-foo@bar targets still work but the inline
> verification has been shunted into other target prerequisites before a
> sub-make is invoked for the docker-run target.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Actually I now realize we never use the verification for invalid tests and
images because of other implicit/pattern rules in rules.mak:

    $ make docker-test-quick@bar
      LD -r docker-test-quick@bar.mo
    /usr/bin/ld: no input files
    collect2: error: ld returned 1 exit status
    rules.mak:99: recipe for target 'docker-test-quick@bar.mo' failed
    make: *** [docker-test-quick@bar.mo] Error 1
    $ make docker-test-quick@bar^C
    $ make docker-test-foo@fedora
      LD -r docker-test-foo@fedora.mo
    /usr/bin/ld: no input files
    collect2: error: ld returned 1 exit status
    rules.mak:99: recipe for target 'docker-test-foo@fedora.mo' failed
    make: *** [docker-test-foo@fedora.mo] Error 1

Maybe we can simply get rid of that altogether? What do you think?

Fam

> 
> ---
> NB: I dropped the awk magic that verifies the image exists before
> running. I couldn't get the thing to work in my shell so wasn't quite
> sure what it was doing.
> 
> v2
>  - fix spelling on arbitrary
>  - document docker-run in help
>  - add note on docker.py update
>  - reduce noise on verifying other images
>  - revert back to using filter
>  - fix merge conflict
> ---
>  tests/docker/Makefile.include | 89 ++++++++++++++++++++++++++++++++-----------
>  1 file changed, 66 insertions(+), 23 deletions(-)
> 
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index 2fcc3c6..87735d8 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -78,6 +78,7 @@ docker:
>  	@echo '                         "IMAGE" is one of the listed container name."'
>  	@echo '    docker-image:        Build all images.'
>  	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
> +	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
>  	@echo
>  	@echo 'Available container images:'
>  	@echo '    $(DOCKER_IMAGES)'
> @@ -101,31 +102,73 @@ docker:
>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>  
> -docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> -docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> -docker-run-%: docker-qemu-src
> +# This rule if for directly running against an arbitrary docker target.
> +# It is called by the expanded docker targets (e.g. make
> +# docker-test-foo@bar) which will do additional verification.
> +#
> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> +#
> +docker-run: docker-qemu-src
>  	@mkdir -p "$(DOCKER_CCACHE_DIR)"
> -	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> -		then echo "Invalid target"; exit 1; \
> +	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
> +		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
> +	fi
> +	$(if $(EXECUTABLE),						\
> +		$(call quiet-command,					\
> +			$(SRC_PATH)/tests/docker/docker.py update 	\
> +			$(IMAGE) $(EXECUTABLE),				\
> +			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
> +	$(call quiet-command,						\
> +		$(SRC_PATH)/tests/docker/docker.py run 			\
> +			-t 						\
> +			$(if $V,,--rm) 					\
> +			$(if $(DEBUG),-i,--net=none) 			\
> +			-e TARGET_LIST=$(TARGET_LIST) 			\
> +			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
> +			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
> +			-e SHOW_ENV=$(SHOW_ENV) 			\
> +			-e CCACHE_DIR=/var/tmp/ccache 			\
> +			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> +			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
> +			$(IMAGE) 					\
> +			/var/tmp/qemu/run 				\
> +			$(TEST), "  RUN $(TEST) in ${IMAGE}")
> +
> +#
> +# Verification targets
> +#
> +# These targets help verify the test (CMD) and docker tag (IMAGE) are
> +# part of the built in set of tests and images. You can still call the
> +# docker-run target directly for testsing against arbitary images.
> +#
> +
> +docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
> +docker-verify-image-%:
> +	@if test -z "$(IMAGE)";			 	\
> +		then echo "Invalid image"; exit 1; 	\
>  	fi
> -	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> -		$(call quiet-command,\
> -			if $(SRC_PATH)/tests/docker/docker.py images | \
> -				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
> -				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
> -				-t \
> -				$(if $(DEBUG),-i,--net=none) \
> -				-e TARGET_LIST=$(TARGET_LIST) \
> -				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
> -				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
> -				-e CCACHE_DIR=/var/tmp/ccache \
> -				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> -				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
> -				qemu:$(IMAGE) \
> -				/var/tmp/qemu/run \
> -				$(CMD); \
> -			fi \
> -			, "  RUN $(CMD) in $(IMAGE)")))
> +	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
> +		echo "$(IMAGE) is not a known image"; exit 1)	\
> +
> +docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
> +docker-verify-test-%:
> +	@if test -z "$(CMD)"; 				\
> +		then echo "Invalid test"; exit 1; 	\
> +	fi
> +	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
> +		echo "$(CMD) is not a known test"; exit 1)
> +
> +# Run targets
> +#
> +# This will take a target such as docker-test-foo@bar and verify that:
> +#  - the test test-foo is a known test
> +#  - the image bar is a known image
> +#
> +# It will then call the docker-run
> +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> +docker-run-%: docker-verify-image-% docker-verify-test-%
> +	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
>  
>  docker-clean:
>  	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
> -- 
> 2.9.3
> 

Re: [Qemu-devel] [PATCH v2 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Alex Bennée]

Fam Zheng <famz@redhat.com> writes:

> On Mon, 10/03 17:32, Alex Bennée wrote:
>> This re-factors the docker makefile to include a docker-run target which
>> can be controlled entirely from environment variables specified on the
>> make command line. This allows us to run against any given docker image
>> we may have in our repository, for example:
>>
>>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
>>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
>>
>> The existing docker-foo@bar targets still work but the inline
>> verification has been shunted into other target prerequisites before a
>> sub-make is invoked for the docker-run target.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>
> Actually I now realize we never use the verification for invalid tests and
> images because of other implicit/pattern rules in rules.mak:
>
>     $ make docker-test-quick@bar
>       LD -r docker-test-quick@bar.mo
>     /usr/bin/ld: no input files
>     collect2: error: ld returned 1 exit status
>     rules.mak:99: recipe for target 'docker-test-quick@bar.mo' failed
>     make: *** [docker-test-quick@bar.mo] Error 1
>     $ make docker-test-quick@bar^C
>     $ make docker-test-foo@fedora
>       LD -r docker-test-foo@fedora.mo
>     /usr/bin/ld: no input files
>     collect2: error: ld returned 1 exit status
>     rules.mak:99: recipe for target 'docker-test-foo@fedora.mo' failed
>     make: *** [docker-test-foo@fedora.mo] Error 1
>
> Maybe we can simply get rid of that altogether? What do you think?

Yeah it looks like it. I'll include a patch to drop them which you can
squash into the other one if you want.

>
> Fam
>
>>
>> ---
>> NB: I dropped the awk magic that verifies the image exists before
>> running. I couldn't get the thing to work in my shell so wasn't quite
>> sure what it was doing.
>>
>> v2
>>  - fix spelling on arbitrary
>>  - document docker-run in help
>>  - add note on docker.py update
>>  - reduce noise on verifying other images
>>  - revert back to using filter
>>  - fix merge conflict
>> ---
>>  tests/docker/Makefile.include | 89 ++++++++++++++++++++++++++++++++-----------
>>  1 file changed, 66 insertions(+), 23 deletions(-)
>>
>> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
>> index 2fcc3c6..87735d8 100644
>> --- a/tests/docker/Makefile.include
>> +++ b/tests/docker/Makefile.include
>> @@ -78,6 +78,7 @@ docker:
>>  	@echo '                         "IMAGE" is one of the listed container name."'
>>  	@echo '    docker-image:        Build all images.'
>>  	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
>> +	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
>>  	@echo
>>  	@echo 'Available container images:'
>>  	@echo '    $(DOCKER_IMAGES)'
>> @@ -101,31 +102,73 @@ docker:
>>  	@echo '    NOCACHE=1            Ignore cache when build images.'
>>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
>>
>> -docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
>> -docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
>> -docker-run-%: docker-qemu-src
>> +# This rule if for directly running against an arbitrary docker target.
>> +# It is called by the expanded docker targets (e.g. make
>> +# docker-test-foo@bar) which will do additional verification.
>> +#
>> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
>> +#
>> +docker-run: docker-qemu-src
>>  	@mkdir -p "$(DOCKER_CCACHE_DIR)"
>> -	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
>> -		then echo "Invalid target"; exit 1; \
>> +	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
>> +		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
>> +	fi
>> +	$(if $(EXECUTABLE),						\
>> +		$(call quiet-command,					\
>> +			$(SRC_PATH)/tests/docker/docker.py update 	\
>> +			$(IMAGE) $(EXECUTABLE),				\
>> +			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
>> +	$(call quiet-command,						\
>> +		$(SRC_PATH)/tests/docker/docker.py run 			\
>> +			-t 						\
>> +			$(if $V,,--rm) 					\
>> +			$(if $(DEBUG),-i,--net=none) 			\
>> +			-e TARGET_LIST=$(TARGET_LIST) 			\
>> +			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
>> +			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
>> +			-e SHOW_ENV=$(SHOW_ENV) 			\
>> +			-e CCACHE_DIR=/var/tmp/ccache 			\
>> +			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
>> +			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
>> +			$(IMAGE) 					\
>> +			/var/tmp/qemu/run 				\
>> +			$(TEST), "  RUN $(TEST) in ${IMAGE}")
>> +
>> +#
>> +# Verification targets
>> +#
>> +# These targets help verify the test (CMD) and docker tag (IMAGE) are
>> +# part of the built in set of tests and images. You can still call the
>> +# docker-run target directly for testsing against arbitary images.
>> +#
>> +
>> +docker-verify-image-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-verify-image-\([^@]*\)@\(.*\)/\2/')
>> +docker-verify-image-%:
>> +	@if test -z "$(IMAGE)";			 	\
>> +		then echo "Invalid image"; exit 1; 	\
>>  	fi
>> -	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
>> -		$(call quiet-command,\
>> -			if $(SRC_PATH)/tests/docker/docker.py images | \
>> -				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
>> -				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
>> -				-t \
>> -				$(if $(DEBUG),-i,--net=none) \
>> -				-e TARGET_LIST=$(TARGET_LIST) \
>> -				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
>> -				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
>> -				-e CCACHE_DIR=/var/tmp/ccache \
>> -				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
>> -				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
>> -				qemu:$(IMAGE) \
>> -				/var/tmp/qemu/run \
>> -				$(CMD); \
>> -			fi \
>> -			, "  RUN $(CMD) in $(IMAGE)")))
>> +	$(if $(filter $(IMAGE), $(DOCKER_IMAGES)), ,		\
>> +		echo "$(IMAGE) is not a known image"; exit 1)	\
>> +
>> +docker-verify-test-%: CMD = $(shell echo '$@' | sed -e 's/docker-verify-test-\([^@]*\)@\(.*\)/\1/')
>> +docker-verify-test-%:
>> +	@if test -z "$(CMD)"; 				\
>> +		then echo "Invalid test"; exit 1; 	\
>> +	fi
>> +	$(if $(filter $(CMD), $(DOCKER_TESTS)), ,	\
>> +		echo "$(CMD) is not a known test"; exit 1)
>> +
>> +# Run targets
>> +#
>> +# This will take a target such as docker-test-foo@bar and verify that:
>> +#  - the test test-foo is a known test
>> +#  - the image bar is a known image
>> +#
>> +# It will then call the docker-run
>> +docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
>> +docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
>> +docker-run-%: docker-verify-image-% docker-verify-test-%
>> +	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
>>
>>  docker-clean:
>>  	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
>> --
>> 2.9.3
>>


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 0/4] generic docker run patches

[Alex Bennée]

Alex Bennée <alex.bennee@linaro.org> writes:

> Hi Fam,
>
> I've re-based the series for the generic run target. The aim being to
> allow a developer to run tests against any generic docker target even
> if it is not in the list of approved targets:

Ping?

>
>   make docker-run TEST=test-quick IMAGE=debian:arm64 \
>     EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1
>
> The existing auto targets still work but have been converted to use
> the generic runner once they have done the validation.
>
> Since last post:
>   - review comments on the main patch
>   - added a travis docker file
>   - minor tweak to the test-mingw script
>
> Alex Bennée (4):
>   tests/docker: add travis dockerfile
>   tests/docker: test-build script
>   tests/docker: make test-mingw honour TARGET_LIST
>   tests/docker/Makefile.include: add a generic docker-run target
>
>  tests/docker/Makefile.include          | 89 +++++++++++++++++++++++++---------
>  tests/docker/dockerfiles/travis.docker |  6 +++
>  tests/docker/test-build                | 18 +++++++
>  tests/docker/test-mingw                |  4 +-
>  4 files changed, 93 insertions(+), 24 deletions(-)
>  create mode 100644 tests/docker/dockerfiles/travis.docker
>  create mode 100755 tests/docker/test-build


--
Alex Bennée

Re: [Qemu-devel] [PATCH v2 0/4] generic docker run patches

[Fam Zheng]

On Tue, 10/11 07:48, Alex Bennée wrote:
> 
> Alex Bennée <alex.bennee@linaro.org> writes:
> 
> > Hi Fam,
> >
> > I've re-based the series for the generic run target. The aim being to
> > allow a developer to run tests against any generic docker target even
> > if it is not in the list of approved targets:
> 
> Ping?

Sorry, still in the middle of processing my inbox after a long holiday (1st -
7th Oct), will take a look at this today.

Fam

> 
> >
> >   make docker-run TEST=test-quick IMAGE=debian:arm64 \
> >     EXECUTABLE=./aarch64-linux-user/qemu-aarch64 J=9 SHOW_ENV=1
> >
> > The existing auto targets still work but have been converted to use
> > the generic runner once they have done the validation.
> >
> > Since last post:
> >   - review comments on the main patch
> >   - added a travis docker file
> >   - minor tweak to the test-mingw script
> >
> > Alex Bennée (4):
> >   tests/docker: add travis dockerfile
> >   tests/docker: test-build script
> >   tests/docker: make test-mingw honour TARGET_LIST
> >   tests/docker/Makefile.include: add a generic docker-run target
> >
> >  tests/docker/Makefile.include          | 89 +++++++++++++++++++++++++---------
> >  tests/docker/dockerfiles/travis.docker |  6 +++
> >  tests/docker/test-build                | 18 +++++++
> >  tests/docker/test-mingw                |  4 +-
> >  4 files changed, 93 insertions(+), 24 deletions(-)
> >  create mode 100644 tests/docker/dockerfiles/travis.docker
> >  create mode 100755 tests/docker/test-build
> 
> 
> --
> Alex Bennée