* [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel
@ 2016-09-29 8:11 Fam Zheng
2016-09-30 13:25 ` Marc-André Lureau
0 siblings, 1 reply; 3+ messages in thread
From: Fam Zheng @ 2016-09-29 8:11 UTC (permalink / raw)
To: qemu-devel; +Cc: berrange, jsnow, pbonzini
Previously all qtest cases in a category, such as check-qtest-y, are
executed in a single long gtester command. This patch separates each
test into its own make target to allow better parallism, unless gcov is
configured.
Slightly reorganize the gcov enabled case too, but the execution
sequence is kept.
This saves >50% of the time (see below), which means a lot for patchew
automatic testing given how 'make check' takes part in testings.
On my machine:
before:
$ make docker-test-quick@fedora J=8 &>/dev/null && \
time make docker-test-quick@fedora J=8 >/dev/null;
real 1m47.090s
user 0m1.599s
sys 0m0.258s
after:
real 0m54.067s
user 0m1.447s
sys 0m0.253s
Signed-off-by: Fam Zheng <famz@redhat.com>
---
v2: Fix gcov case by not changing it. [Daniel]
---
tests/Makefile.include | 48 ++++++++++++++++++++++++++++++++++++------------
1 file changed, 36 insertions(+), 12 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 8162f6f..575030a 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -729,27 +729,51 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
# gtester tests, possibly with verbose output
.PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
-$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
- $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
- $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
+
+run-tests = $(call quiet-command,\
+ $(if $(QTEST_TARGET), \
+ QTEST_QEMU_BINARY=$(QTEST_TARGET)-softmmu/qemu-system-$(QTEST_TARGET)) \
QTEST_QEMU_IMG=qemu-img$(EXESUF) \
MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + 1))} \
- gtester $(GTESTER_OPTIONS) -m=$(SPEED) $(check-qtest-$*-y) $(check-qtest-generic-y),"GTESTER $@")
- $(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y) $(gcov-files-generic-y); do \
+ gtester $1 $(GTESTER_OPTIONS) -m=$(SPEED),"GTESTER $2")
+
+ifneq ($(CONFIG_GCOV),)
+
+check-qtest-%: QTEST_TARGET=$*
+$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y)
+ @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
+ $(call run-tests, $(check-qtest-$*-y), $@)
+ @for f in $(gcov-files-$*-y) $(gcov-files-generic-y); do \
echo Gcov report for $$f:;\
$(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \
- done,)
+ done
.PHONY: $(patsubst %, check-%, $(check-unit-y))
$(patsubst %, check-%, $(check-unit-y)): check-%: %
- $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda,)
- $(call quiet-command, \
- MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + 1))} \
- gtester $(GTESTER_OPTIONS) -m=$(SPEED) $*,"GTESTER $*")
- $(if $(CONFIG_GCOV),@for f in $(gcov-files-$(subst tests/,,$*)-y) $(gcov-files-generic-y); do \
+ @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
+ $(call run-tests, $*, $*)
+ @for f in $(gcov-files-$(subst tests/,,$*)-y) $(gcov-files-generic-y); do \
echo Gcov report for $$f:;\
$(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \
- done,)
+ done
+
+else
+
+run-test-%: tests/%
+ $(call run-tests, $<, $<)
+
+$(foreach target, $(QTEST_TARGETS), \
+ $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \
+ $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \
+ $(check-qtest-y) \
+ $(check-qtest-$(target)-y) \
+ $(check-qtest-generic-y))) \
+)
+$(patsubst %, check-%, $(check-unit-y)): check-tests/%: run-test-%
+
+endif
+
+.PHONY: $(patsubst %, check-%, $(check-unit-y))
# gtester tests with XML output
--
2.7.4
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel
2016-09-29 8:11 [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel Fam Zheng
@ 2016-09-30 13:25 ` Marc-André Lureau
2016-10-12 1:17 ` Fam Zheng
0 siblings, 1 reply; 3+ messages in thread
From: Marc-André Lureau @ 2016-09-30 13:25 UTC (permalink / raw)
To: Fam Zheng, qemu-devel; +Cc: jsnow, pbonzini
Hi
On Thu, Sep 29, 2016 at 12:13 PM Fam Zheng <famz@redhat.com> wrote:
> Previously all qtest cases in a category, such as check-qtest-y, are
> executed in a single long gtester command. This patch separates each
> test into its own make target to allow better parallism, unless gcov is
> configured.
>
> Slightly reorganize the gcov enabled case too, but the execution
> sequence is kept.
>
> This saves >50% of the time (see below), which means a lot for patchew
> automatic testing given how 'make check' takes part in testings.
>
> On my machine:
>
> before:
>
> $ make docker-test-quick@fedora J=8 &>/dev/null && \
> time make docker-test-quick@fedora J=8 >/dev/null;
>
> real 1m47.090s
> user 0m1.599s
> sys 0m0.258s
>
> after:
>
> real 0m54.067s
> user 0m1.447s
> sys 0m0.253s
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
>
> ---
>
> v2: Fix gcov case by not changing it. [Daniel]
> ---
> tests/Makefile.include | 48
> ++++++++++++++++++++++++++++++++++++------------
> 1 file changed, 36 insertions(+), 12 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 8162f6f..575030a 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -729,27 +729,51 @@ GCOV_OPTIONS = -n $(if $(V),-f,)
> # gtester tests, possibly with verbose output
>
> .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS))
> -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%:
> $(check-qtest-y)
> - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda
> */*/*/*.gcda,)
> - $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \
> +
> +run-tests = $(call quiet-command,\
> + $(if $(QTEST_TARGET), \
> +
> QTEST_QEMU_BINARY=$(QTEST_TARGET)-softmmu/qemu-system-$(QTEST_TARGET)) \
> QTEST_QEMU_IMG=qemu-img$(EXESUF) \
> MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 +
> 1))} \
> - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $(check-qtest-$*-y)
> $(check-qtest-generic-y),"GTESTER $@")
> - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y)
> $(gcov-files-generic-y); do \
> + gtester $1 $(GTESTER_OPTIONS) -m=$(SPEED),"GTESTER $2")
> +
> +ifneq ($(CONFIG_GCOV),)
> +
> +check-qtest-%: QTEST_TARGET=$*
> +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%:
> $(check-qtest-y)
> + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
> + $(call run-tests, $(check-qtest-$*-y), $@)
> + @for f in $(gcov-files-$*-y) $(gcov-files-generic-y); do \
> echo Gcov report for $$f:;\
> $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \
> - done,)
> + done
>
> .PHONY: $(patsubst %, check-%, $(check-unit-y))
> $(patsubst %, check-%, $(check-unit-y)): check-%: %
> - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda
> */*/*/*.gcda,)
> - $(call quiet-command, \
> - MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 +
> 1))} \
> - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $*,"GTESTER $*")
> - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$(subst tests/,,$*)-y)
> $(gcov-files-generic-y); do \
> + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
> + $(call run-tests, $*, $*)
> + @for f in $(gcov-files-$(subst tests/,,$*)-y)
> $(gcov-files-generic-y); do \
> echo Gcov report for $$f:;\
> $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \
> - done,)
> + done
> +
> +else
> +
> +run-test-%: tests/%
> + $(call run-tests, $<, $<)
> +
> +$(foreach target, $(QTEST_TARGETS), \
> + $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \
> + $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \
> + $(check-qtest-y) \
> + $(check-qtest-$(target)-y) \
> + $(check-qtest-generic-y))) \
> +)
> +$(patsubst %, check-%, $(check-unit-y)): check-tests/%: run-test-%
> +
> +endif
> +
> +.PHONY: $(patsubst %, check-%, $(check-unit-y))
>
>
it's quite difficult to read, but it looks correct to me. Something I wish
we had that we can perhaps add as well here (add dep on -softmmu build):
diff --git a/tests/Makefile.include b/tests/Makefile.include
index f5491c7..8e5dc72 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -743,6 +743,7 @@ run-tests = $(call quiet-command,\
ifneq ($(CONFIG_GCOV),)
check-qtest-%: QTEST_TARGET=$*
+check-qtest-%: subdir-$*-softmmu
$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%:
$(check-qtest-y)
@rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda
$(call run-tests, $(check-qtest-$*-y), $@)
@@ -767,6 +768,7 @@ run-test-%: tests/%
$(foreach target, $(QTEST_TARGETS), \
$(eval check-qtest-$(target): QTEST_TARGET := $(target)) \
+ $(eval check-qtest-$(target): subdir-$(target)-softmmu) \
$(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \
$(check-qtest-y) \
$(check-qtest-$(target)-y) \
I can post a seperate patch later though,
Similarly, qga test:
-tests/test-qga: tests/test-qga.o $(qtest-obj-y)
+tests/test-qga: tests/test-qga.o $(qtest-obj-y) qemu-ga$(EXESUF)
In any case
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> # gtester tests with XML output
>
> --
> 2.7.4
>
>
> --
Marc-André Lureau
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel
2016-09-30 13:25 ` Marc-André Lureau
@ 2016-10-12 1:17 ` Fam Zheng
0 siblings, 0 replies; 3+ messages in thread
From: Fam Zheng @ 2016-10-12 1:17 UTC (permalink / raw)
To: pbonzini; +Cc: Marc-André Lureau, qemu-devel, jsnow
On Fri, 09/30 13:25, Marc-André Lureau wrote:
> In any case
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Paolo, are you happy with taking this patch?
Fam
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-10-12 1:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-09-29 8:11 [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel Fam Zheng
2016-09-30 13:25 ` Marc-André Lureau
2016-10-12 1:17 ` Fam Zheng
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).