Re: [Qemu-devel] [PULL 4/4] tests/docker/Makefile.include: add a generic docker-run target

[Fam Zheng <famz@redhat.com>]

On Sat, 10/15 15:48, Alex Bennée wrote:
> 
> Paolo Bonzini <pbonzini@redhat.com> writes:
> 
> > On 14/10/2016 17:29, Fam Zheng wrote:
> >> From: Alex Bennée <alex.bennee@linaro.org>
> >>
> >> This re-factors the docker makefile to include a docker-run target which
> >> can be controlled entirely from environment variables specified on the
> >> make command line. This allows us to run against any given docker image
> >> we may have in our repository, for example:
> >>
> >>     make docker-run TEST="test-quick" IMAGE="debian:arm64" \
> >>          EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> >>
> >> The existing docker-foo@bar targets still work but the inline
> >> verification has been dropped because we already don't hit that due to
> >> other pattern rules in rules.mak.
> >>
> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >>
> >> Message-Id: <20161011161625.9070-5-alex.bennee@linaro.org>
> >> Message-Id: <20161011161625.9070-6-alex.bennee@linaro.org>
> >> [Squash in the verification removal patch. - Fam]
> >> Signed-off-by: Fam Zheng <famz@redhat.com>
> >> ---
> >>  tests/docker/Makefile.include | 61 +++++++++++++++++++++++++++----------------
> >>  1 file changed, 38 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> >> index b44daab..925f711 100644
> >> --- a/tests/docker/Makefile.include
> >> +++ b/tests/docker/Makefile.include
> >> @@ -78,6 +78,7 @@ docker:
> >>  	@echo '                         "IMAGE" is one of the listed container name."'
> >>  	@echo '    docker-image:        Build all images.'
> >>  	@echo '    docker-image-IMAGE:  Build image "IMAGE".'
> >> +	@echo '    docker-run:          For manually running a "TEST" with "IMAGE"'
> >>  	@echo
> >>  	@echo 'Available container images:'
> >>  	@echo '    $(DOCKER_IMAGES)'
> >> @@ -101,31 +102,45 @@ docker:
> >>  	@echo '    NOCACHE=1            Ignore cache when build images.'
> >>  	@echo '    EXECUTABLE=<path>    Include executable in image.'
> >>
> >> +# This rule if for directly running against an arbitrary docker target.
> >> +# It is called by the expanded docker targets (e.g. make
> >> +# docker-test-foo@bar) which will do additional verification.
> >> +#
> >> +# For example: make docker-run TEST="test-quick" IMAGE="debian:arm64" EXECUTABLE=./aarch64-linux-user/qemu-aarch64
> >> +#
> >> +docker-run: docker-qemu-src
> >> +	@mkdir -p "$(DOCKER_CCACHE_DIR)"
> >> +	@if test -z "$(IMAGE)" || test -z "$(TEST)"; \
> >> +		then echo "Invalid target $(IMAGE)/$(TEST)"; exit 1; \
> >> +	fi
> >> +	$(if $(EXECUTABLE),						\
> >> +		$(call quiet-command,					\
> >> +			$(SRC_PATH)/tests/docker/docker.py update 	\
> >> +			$(IMAGE) $(EXECUTABLE),				\
> >> +			"  COPYING $(EXECUTABLE) to $(IMAGE)"))
> >> +	$(call quiet-command,						\
> >> +		$(SRC_PATH)/tests/docker/docker.py run 			\
> >> +			-t 						\
> >> +			$(if $V,,--rm) 					\
> >> +			$(if $(DEBUG),-i,--net=none) 			\
> >> +			-e TARGET_LIST=$(TARGET_LIST) 			\
> >> +			-e EXTRA_CONFIGURE_OPTS="$(EXTRA_CONFIGURE_OPTS)" \
> >> +			-e V=$V -e J=$J -e DEBUG=$(DEBUG)		\
> >> +			-e SHOW_ENV=$(SHOW_ENV) 			\
> >> +			-e CCACHE_DIR=/var/tmp/ccache 			\
> >> +			-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> >> +			-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z 	\
> >> +			$(IMAGE) 					\
> >> +			/var/tmp/qemu/run 				\
> >> +			$(TEST), "  RUN $(TEST) in ${IMAGE}")
> >> +
> >> +# Run targets:
> >> +#
> >> +# Of the form docker-TEST-FOO@IMAGE-BAR which will then be expanded into a call to "make docker-run"
> >>  docker-run-%: CMD = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\1/')
> >>  docker-run-%: IMAGE = $(shell echo '$@' | sed -e 's/docker-run-\([^@]*\)@\(.*\)/\2/')
> >> -docker-run-%: docker-qemu-src
> >> -	@mkdir -p "$(DOCKER_CCACHE_DIR)"
> >> -	@if test -z "$(IMAGE)" || test -z "$(CMD)"; \
> >> -		then echo "Invalid target"; exit 1; \
> >> -	fi
> >> -	$(if $(filter $(TESTS),$(CMD)),$(if $(filter $(IMAGES),$(IMAGE)), \
> >> -		$(call quiet-command,\
> >> -			if $(SRC_PATH)/tests/docker/docker.py images | \
> >> -				awk '$$1=="qemu" && $$2=="$(IMAGE)"{found=1} END{exit(!found)}'; then \
> >> -				$(SRC_PATH)/tests/docker/docker.py run $(if $V,,--rm) \
> >> -				-t \
> >> -				$(if $(DEBUG),-i,--net=none) \
> >> -				-e TARGET_LIST=$(TARGET_LIST) \
> >> -				-e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) \
> >> -				-e V=$V -e J=$J -e DEBUG=$(DEBUG) -e SHOW_ENV=$(SHOW_ENV)\
> >> -				-e CCACHE_DIR=/var/tmp/ccache \
> >> -				-v $$(readlink -e $(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
> >> -				-v $(DOCKER_CCACHE_DIR):/var/tmp/ccache:z \
> >> -				qemu:$(IMAGE) \
> >> -				/var/tmp/qemu/run \
> >> -				$(CMD); \
> >> -			fi \
> >> -			,"RUN","$(CMD) in $(IMAGE)")))
> >> +docker-run-%:
> >> +	@make docker-run TEST=$(CMD) IMAGE=qemu:$(IMAGE)
> >
> > This "make" should be $(MAKE), so that some parameters are handled
> > correctly (e.g. -n/--dry-run) and especially so that parallel make works
> > correctly.

Parallism should terminate here because docker-run is a leaf target. But you are
right about parameters.

> 
> Fam,
> 
> Do you want me to re-spin and re-send or are you going to fix up in your
> pull request?

I'll fix up in my tree and send pull request again.

Fam

> 
> >
> > Thanks,
> >
> > Paolo
> >
> >>  docker-clean:
> >>  	$(call quiet-command, $(SRC_PATH)/tests/docker/docker.py clean)
> >>
> 
> 
> --
> Alex Bennée