* [PATCH 1/9] perf build: Set parallel making options build-test
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-19 13:34 ` [tip:perf/urgent] " tip-bot for Wang Nan
2016-01-13 12:17 ` [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test Wang Nan
` (8 subsequent siblings)
9 siblings, 1 reply; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan,
Arnaldo Carvalho de Melo, Namhyung Kim
'make build-test' is painful because of time consuming. In a full test,
all test cases are built twice with tools/perf/Makefile and
tools/perf/Makefile.perf. 'Makefile' automatically computes parallel
options for make, but 'Makefile.perf' not, so all test cases is built
with one job. It is very slow.
This patch adds '-j' options to Makefile.perf testing. It computes
parallel building options like what tools/perf/Makefile does, and pass
'-j' option to Makefile.perf test.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/make | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index df38dec..c0ee679 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
# no target specified, trigger the whole suite
all:
@echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
- @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf
+ @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
else
# run only specific test over 'Makefile'
%:
@@ -14,6 +14,15 @@ endif
else
PERF := .
+PARALLEL_OPT=
+ifeq ($(SET_PARALLEL),1)
+ cores := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
+ ifeq ($(cores),0)
+ cores := 1
+ endif
+ PARALLEL_OPT="-j$(cores)"
+endif
+
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
LC_COLLATE=C
@@ -252,7 +261,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
$(run):
$(call clean)
@TMP_DEST=$$(mktemp -d); \
- cmd="cd $(PERF) && make -f $(MK) DESTDIR=$$TMP_DEST $($@)"; \
+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1; \
echo " test: $(call test,$@)" >> $@ 2>&1; \
@@ -263,7 +272,7 @@ $(run_O):
$(call clean)
@TMP_O=$$(mktemp -d); \
TMP_DEST=$$(mktemp -d); \
- cmd="cd $(PERF) && make -f $(MK) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1 && \
echo " test: $(call test_O,$@)" >> $@ 2>&1; \
@@ -277,15 +286,15 @@ tarpkg:
rm -f $@
make_kernelsrc:
- @echo "- make -C <kernelsrc> tools/perf"
+ @echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
$(call clean); \
- (make -C ../.. tools/perf) > $@ 2>&1 && \
+ (make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
test -x perf && rm -f $@ || (cat $@ ; false)
make_kernelsrc_tools:
- @echo "- make -C <kernelsrc>/tools perf"
+ @echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
$(call clean); \
- (make -C ../../tools perf) > $@ 2>&1 && \
+ (make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
test -x perf && rm -f $@ || (cat $@ ; false)
all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [tip:perf/urgent] perf build: Set parallel making options build-test
2016-01-13 12:17 ` [PATCH 1/9] perf build: Set parallel making options build-test Wang Nan
@ 2016-01-19 13:34 ` tip-bot for Wang Nan
0 siblings, 0 replies; 29+ messages in thread
From: tip-bot for Wang Nan @ 2016-01-19 13:34 UTC (permalink / raw)
To: linux-tip-commits
Cc: mingo, acme, namhyung, wangnan0, jolsa, tglx, lizefan, hpa,
linux-kernel
Commit-ID: 7be43dfb1e617b87bf2d936d82c026be39b43910
Gitweb: http://git.kernel.org/tip/7be43dfb1e617b87bf2d936d82c026be39b43910
Author: Wang Nan <wangnan0@huawei.com>
AuthorDate: Wed, 13 Jan 2016 12:17:14 +0000
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Fri, 15 Jan 2016 16:31:59 -0300
perf build: Set parallel making options build-test
'make build-test' is painful because of time consuming. In a full test,
all test cases are built twice with tools/perf/Makefile and
tools/perf/Makefile.perf. 'Makefile' automatically computes parallel
options for make, but 'Makefile.perf' not, so all test cases is built
with one job. It is very slow.
This patch adds '-j' options to Makefile.perf testing. It computes
parallel building options like what tools/perf/Makefile does, and pass
'-j' option to Makefile.perf test.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1452687442-6186-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/tests/make | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index df38dec..c0ee679 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
# no target specified, trigger the whole suite
all:
@echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
- @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf
+ @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
else
# run only specific test over 'Makefile'
%:
@@ -14,6 +14,15 @@ endif
else
PERF := .
+PARALLEL_OPT=
+ifeq ($(SET_PARALLEL),1)
+ cores := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
+ ifeq ($(cores),0)
+ cores := 1
+ endif
+ PARALLEL_OPT="-j$(cores)"
+endif
+
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
LC_COLLATE=C
@@ -252,7 +261,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
$(run):
$(call clean)
@TMP_DEST=$$(mktemp -d); \
- cmd="cd $(PERF) && make -f $(MK) DESTDIR=$$TMP_DEST $($@)"; \
+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1; \
echo " test: $(call test,$@)" >> $@ 2>&1; \
@@ -263,7 +272,7 @@ $(run_O):
$(call clean)
@TMP_O=$$(mktemp -d); \
TMP_DEST=$$(mktemp -d); \
- cmd="cd $(PERF) && make -f $(MK) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1 && \
echo " test: $(call test_O,$@)" >> $@ 2>&1; \
@@ -277,15 +286,15 @@ tarpkg:
rm -f $@
make_kernelsrc:
- @echo "- make -C <kernelsrc> tools/perf"
+ @echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
$(call clean); \
- (make -C ../.. tools/perf) > $@ 2>&1 && \
+ (make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
test -x perf && rm -f $@ || (cat $@ ; false)
make_kernelsrc_tools:
- @echo "- make -C <kernelsrc>/tools perf"
+ @echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
$(call clean); \
- (make -C ../../tools perf) > $@ 2>&1 && \
+ (make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
test -x perf && rm -f $@ || (cat $@ ; false)
all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
2016-01-13 12:17 ` [PATCH 1/9] perf build: Set parallel making options build-test Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-14 9:32 ` Jiri Olsa
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
` (7 subsequent siblings)
9 siblings, 1 reply; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan,
Arnaldo Carvalho de Melo, Jiri Olsa, Namhyung Kim
Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
option when it is passed through cmdline only, because of code in
tools/scripts/Makefile.include:
ifneq ($(O),)
ifeq ($(origin O), command line)
...
ABSOLUTE_O := $(shell cd $(O) ; pwd)
OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
endif
endif
This patch passes 'O' to Makefile.perf through cmdline explicitly
to make it follow O variable during build-test.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/make | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index c0ee679..14d7b8d 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
# no target specified, trigger the whole suite
all:
@echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
- @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
+ @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
else
# run only specific test over 'Makefile'
%:
@@ -13,6 +13,14 @@ else
endif
else
PERF := .
+O_OPT :=
+
+ifneq ($(O),)
+ FULL_O := $(shell readlink -f $(O) || echo $(O))
+ ifeq ($(SET_O),1)
+ O_OPT := 'O=$(FULL_O)'
+ endif
+endif
PARALLEL_OPT=
ifeq ($(SET_PARALLEL),1)
@@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
$(run):
$(call clean)
@TMP_DEST=$$(mktemp -d); \
- cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
echo "- $@: $$cmd" && echo $$cmd > $@ && \
( eval $$cmd ) >> $@ 2>&1; \
echo " test: $(call test,$@)" >> $@ 2>&1; \
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-13 12:17 ` [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test Wang Nan
@ 2016-01-14 9:32 ` Jiri Olsa
2016-01-14 9:56 ` Wangnan (F)
0 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 9:32 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:15PM +0000, Wang Nan wrote:
> Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
> option when it is passed through cmdline only, because of code in
> tools/scripts/Makefile.include:
>
> ifneq ($(O),)
> ifeq ($(origin O), command line)
> ...
> ABSOLUTE_O := $(shell cd $(O) ; pwd)
> OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
> endif
> endif
>
> This patch passes 'O' to Makefile.perf through cmdline explicitly
> to make it follow O variable during build-test.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
> tools/perf/tests/make | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> index c0ee679..14d7b8d 100644
> --- a/tools/perf/tests/make
> +++ b/tools/perf/tests/make
> @@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
> # no target specified, trigger the whole suite
> all:
> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
> - @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
> + @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
> else
> # run only specific test over 'Makefile'
> %:
> @@ -13,6 +13,14 @@ else
> endif
> else
> PERF := .
> +O_OPT :=
> +
> +ifneq ($(O),)
> + FULL_O := $(shell readlink -f $(O) || echo $(O))
> + ifeq ($(SET_O),1)
> + O_OPT := 'O=$(FULL_O)'
> + endif
> +endif
>
> PARALLEL_OPT=
> ifeq ($(SET_PARALLEL),1)
> @@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
> $(run):
> $(call clean)
> @TMP_DEST=$$(mktemp -d); \
> - cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> + cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
hum, but this set is to test make without O=... so why would you set it?
run_O is the target for O=... tests
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-14 9:32 ` Jiri Olsa
@ 2016-01-14 9:56 ` Wangnan (F)
2016-01-14 14:28 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 29+ messages in thread
From: Wangnan (F) @ 2016-01-14 9:56 UTC (permalink / raw)
To: Jiri Olsa
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On 2016/1/14 17:32, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 12:17:15PM +0000, Wang Nan wrote:
>> Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
>> option when it is passed through cmdline only, because of code in
>> tools/scripts/Makefile.include:
>>
>> ifneq ($(O),)
>> ifeq ($(origin O), command line)
>> ...
>> ABSOLUTE_O := $(shell cd $(O) ; pwd)
>> OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
>> endif
>> endif
>>
>> This patch passes 'O' to Makefile.perf through cmdline explicitly
>> to make it follow O variable during build-test.
>>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> ---
>> tools/perf/tests/make | 12 ++++++++++--
>> 1 file changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
>> index c0ee679..14d7b8d 100644
>> --- a/tools/perf/tests/make
>> +++ b/tools/perf/tests/make
>> @@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
>> # no target specified, trigger the whole suite
>> all:
>> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
>> - @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
>> + @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
>> else
>> # run only specific test over 'Makefile'
>> %:
>> @@ -13,6 +13,14 @@ else
>> endif
>> else
>> PERF := .
>> +O_OPT :=
>> +
>> +ifneq ($(O),)
>> + FULL_O := $(shell readlink -f $(O) || echo $(O))
>> + ifeq ($(SET_O),1)
>> + O_OPT := 'O=$(FULL_O)'
>> + endif
>> +endif
>>
>> PARALLEL_OPT=
>> ifeq ($(SET_PARALLEL),1)
>> @@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
>> $(run):
>> $(call clean)
>> @TMP_DEST=$$(mktemp -d); \
>> - cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
>> + cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> hum, but this set is to test make without O=... so why would you set it?
> run_O is the target for O=... tests
I have strong motivation to avoid polluting source directory.
I have different platforms and sometime I want to build them
(cross-compile) parallelly. I use yocto for this. Yocto appends 'O'
automatically. This is good for building, but when I use the
framework for build-test I have to manually adjust the generated
scripts (and it would be regenerated again in next run) to make it
work, and also I'm unable to test them in parallel.
For people who don't care about this, $(O_OPT) is empty so he/she
still tests without 'O=...' case.
Thank you.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-14 9:56 ` Wangnan (F)
@ 2016-01-14 14:28 ` Arnaldo Carvalho de Melo
2016-01-14 14:37 ` Jiri Olsa
0 siblings, 1 reply; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2016-01-14 14:28 UTC (permalink / raw)
To: Wangnan (F)
Cc: Jiri Olsa, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Namhyung Kim
Em Thu, Jan 14, 2016 at 05:56:06PM +0800, Wangnan (F) escreveu:
>
>
> On 2016/1/14 17:32, Jiri Olsa wrote:
> >On Wed, Jan 13, 2016 at 12:17:15PM +0000, Wang Nan wrote:
> >>Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
> >>option when it is passed through cmdline only, because of code in
> >>tools/scripts/Makefile.include:
> >>
> >> ifneq ($(O),)
> >> ifeq ($(origin O), command line)
> >> ...
> >> ABSOLUTE_O := $(shell cd $(O) ; pwd)
> >> OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
> >> endif
> >> endif
> >>
> >>This patch passes 'O' to Makefile.perf through cmdline explicitly
> >>to make it follow O variable during build-test.
> >>
> >>Signed-off-by: Wang Nan <wangnan0@huawei.com>
> >>Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> >>Cc: Jiri Olsa <jolsa@kernel.org>
> >>Cc: Namhyung Kim <namhyung@kernel.org>
> >>---
> >> tools/perf/tests/make | 12 ++++++++++--
> >> 1 file changed, 10 insertions(+), 2 deletions(-)
> >>
> >>diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> >>index c0ee679..14d7b8d 100644
> >>--- a/tools/perf/tests/make
> >>+++ b/tools/perf/tests/make
> >>@@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
> >> # no target specified, trigger the whole suite
> >> all:
> >> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
> >>- @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
> >>+ @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
> >> else
> >> # run only specific test over 'Makefile'
> >> %:
> >>@@ -13,6 +13,14 @@ else
> >> endif
> >> else
> >> PERF := .
> >>+O_OPT :=
> >>+
> >>+ifneq ($(O),)
> >>+ FULL_O := $(shell readlink -f $(O) || echo $(O))
> >>+ ifeq ($(SET_O),1)
> >>+ O_OPT := 'O=$(FULL_O)'
> >>+ endif
> >>+endif
> >> PARALLEL_OPT=
> >> ifeq ($(SET_PARALLEL),1)
> >>@@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
> >> $(run):
> >> $(call clean)
> >> @TMP_DEST=$$(mktemp -d); \
> >>- cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> >>+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> >hum, but this set is to test make without O=... so why would you set it?
> >run_O is the target for O=... tests
>
> I have strong motivation to avoid polluting source directory.
yeah!
> I have different platforms and sometime I want to build them
> (cross-compile) parallelly. I use yocto for this. Yocto appends 'O'
> automatically. This is good for building, but when I use the
> framework for build-test I have to manually adjust the generated
> scripts (and it would be regenerated again in next run) to make it
> work, and also I'm unable to test them in parallel.
>
> For people who don't care about this, $(O_OPT) is empty so he/she
> still tests without 'O=...' case.
>
> Thank you.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-14 14:28 ` Arnaldo Carvalho de Melo
@ 2016-01-14 14:37 ` Jiri Olsa
2016-01-14 14:42 ` Arnaldo Carvalho de Melo
2016-01-14 15:20 ` pi3orama
0 siblings, 2 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 14:37 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Wangnan (F), linux-kernel, pi3orama, lizefan, Jiri Olsa,
Namhyung Kim
On Thu, Jan 14, 2016 at 11:28:23AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jan 14, 2016 at 05:56:06PM +0800, Wangnan (F) escreveu:
> >
> >
> > On 2016/1/14 17:32, Jiri Olsa wrote:
> > >On Wed, Jan 13, 2016 at 12:17:15PM +0000, Wang Nan wrote:
> > >>Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
> > >>option when it is passed through cmdline only, because of code in
> > >>tools/scripts/Makefile.include:
> > >>
> > >> ifneq ($(O),)
> > >> ifeq ($(origin O), command line)
> > >> ...
> > >> ABSOLUTE_O := $(shell cd $(O) ; pwd)
> > >> OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
> > >> endif
> > >> endif
> > >>
> > >>This patch passes 'O' to Makefile.perf through cmdline explicitly
> > >>to make it follow O variable during build-test.
> > >>
> > >>Signed-off-by: Wang Nan <wangnan0@huawei.com>
> > >>Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> > >>Cc: Jiri Olsa <jolsa@kernel.org>
> > >>Cc: Namhyung Kim <namhyung@kernel.org>
> > >>---
> > >> tools/perf/tests/make | 12 ++++++++++--
> > >> 1 file changed, 10 insertions(+), 2 deletions(-)
> > >>
> > >>diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> > >>index c0ee679..14d7b8d 100644
> > >>--- a/tools/perf/tests/make
> > >>+++ b/tools/perf/tests/make
> > >>@@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
> > >> # no target specified, trigger the whole suite
> > >> all:
> > >> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
> > >>- @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
> > >>+ @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
> > >> else
> > >> # run only specific test over 'Makefile'
> > >> %:
> > >>@@ -13,6 +13,14 @@ else
> > >> endif
> > >> else
> > >> PERF := .
> > >>+O_OPT :=
> > >>+
> > >>+ifneq ($(O),)
> > >>+ FULL_O := $(shell readlink -f $(O) || echo $(O))
> > >>+ ifeq ($(SET_O),1)
> > >>+ O_OPT := 'O=$(FULL_O)'
> > >>+ endif
> > >>+endif
> > >> PARALLEL_OPT=
> > >> ifeq ($(SET_PARALLEL),1)
> > >>@@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
> > >> $(run):
> > >> $(call clean)
> > >> @TMP_DEST=$$(mktemp -d); \
> > >>- cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> > >>+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> > >hum, but this set is to test make without O=... so why would you set it?
> > >run_O is the target for O=... tests
> >
> > I have strong motivation to avoid polluting source directory.
>
> yeah!
heh, I knew you'd be excited ;-) but does that actually mean
that build-test will never run tests in source directory?
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-14 14:37 ` Jiri Olsa
@ 2016-01-14 14:42 ` Arnaldo Carvalho de Melo
2016-01-14 15:20 ` pi3orama
1 sibling, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2016-01-14 14:42 UTC (permalink / raw)
To: Jiri Olsa
Cc: Wangnan (F), linux-kernel, pi3orama, lizefan, Jiri Olsa,
Namhyung Kim
Em Thu, Jan 14, 2016 at 03:37:28PM +0100, Jiri Olsa escreveu:
> On Thu, Jan 14, 2016 at 11:28:23AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Jan 14, 2016 at 05:56:06PM +0800, Wangnan (F) escreveu:
> > > >>+ cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
> > > >hum, but this set is to test make without O=... so why would you set it?
> > > >run_O is the target for O=... tests
> > >
> > > I have strong motivation to avoid polluting source directory.
> >
> > yeah!
>
> heh, I knew you'd be excited ;-) but does that actually mean
> that build-test will never run tests in source directory?
Humm, I think this is something to be supported, but source code
repository polution is really bad and a pet peeve I have with
build-test, so the fact that Wang is working to remove that limitation
is indeed exciting :-)
- Arnaldo
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test
2016-01-14 14:37 ` Jiri Olsa
2016-01-14 14:42 ` Arnaldo Carvalho de Melo
@ 2016-01-14 15:20 ` pi3orama
1 sibling, 0 replies; 29+ messages in thread
From: pi3orama @ 2016-01-14 15:20 UTC (permalink / raw)
To: Jiri Olsa
Cc: Arnaldo Carvalho de Melo, Wangnan (F), linux-kernel, lizefan,
Jiri Olsa, Namhyung Kim
发自我的 iPhone
> 在 2016年1月14日，下午10:37，Jiri Olsa <jolsa@redhat.com> 写道：
>
>> On Thu, Jan 14, 2016 at 11:28:23AM -0300, Arnaldo Carvalho de Melo wrote:
>> Em Thu, Jan 14, 2016 at 05:56:06PM +0800, Wangnan (F) escreveu:
>>>
>>>
>>>> On 2016/1/14 17:32, Jiri Olsa wrote:
>>>>> On Wed, Jan 13, 2016 at 12:17:15PM +0000, Wang Nan wrote:
>>>>> Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
>>>>> option when it is passed through cmdline only, because of code in
>>>>> tools/scripts/Makefile.include:
>>>>>
>>>>> ifneq ($(O),)
>>>>> ifeq ($(origin O), command line)
>>>>> ...
>>>>> ABSOLUTE_O := $(shell cd $(O) ; pwd)
>>>>> OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
>>>>> endif
>>>>> endif
>>>>>
>>>>> This patch passes 'O' to Makefile.perf through cmdline explicitly
>>>>> to make it follow O variable during build-test.
>>>>>
>>>>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>>>>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>>>>> Cc: Jiri Olsa <jolsa@kernel.org>
>>>>> Cc: Namhyung Kim <namhyung@kernel.org>
>>>>> ---
>>>>> tools/perf/tests/make | 12 ++++++++++--
>>>>> 1 file changed, 10 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
>>>>> index c0ee679..14d7b8d 100644
>>>>> --- a/tools/perf/tests/make
>>>>> +++ b/tools/perf/tests/make
>>>>> @@ -5,7 +5,7 @@ ifeq ($(MAKECMDGOALS),)
>>>>> # no target specified, trigger the whole suite
>>>>> all:
>>>>> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
>>>>> - @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
>>>>> + @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
>>>>> else
>>>>> # run only specific test over 'Makefile'
>>>>> %:
>>>>> @@ -13,6 +13,14 @@ else
>>>>> endif
>>>>> else
>>>>> PERF := .
>>>>> +O_OPT :=
>>>>> +
>>>>> +ifneq ($(O),)
>>>>> + FULL_O := $(shell readlink -f $(O) || echo $(O))
>>>>> + ifeq ($(SET_O),1)
>>>>> + O_OPT := 'O=$(FULL_O)'
>>>>> + endif
>>>>> +endif
>>>>> PARALLEL_OPT=
>>>>> ifeq ($(SET_PARALLEL),1)
>>>>> @@ -261,7 +269,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
>>>>> $(run):
>>>>> $(call clean)
>>>>> @TMP_DEST=$$(mktemp -d); \
>>>>> - cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
>>>>> + cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
>>>> hum, but this set is to test make without O=... so why would you set it?
>>>> run_O is the target for O=... tests
>>>
>>> I have strong motivation to avoid polluting source directory.
>>
>> yeah!
>
> heh, I knew you'd be excited ;-) but does that actually mean
> that build-test will never run tests in source directory?
>
build-test should be able to run at source
directory, and I try to not passing extra
O option to make in this situation. This is the
reason why I choose to use O_OPT instead
of directly adding O=xxx to make command.
I tested all cases (with and without -O),
in and out of source directory (for v1. For v2
I remove some test cases). It should work.
Thank you.
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
2016-01-13 12:17 ` [PATCH 1/9] perf build: Set parallel making options build-test Wang Nan
2016-01-13 12:17 ` [PATCH 2/9] perf build: Pass O option to Makefile.perf in build-test Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-14 9:56 ` Jiri Olsa
` (3 more replies)
2016-01-13 12:17 ` [PATCH 4/9] perf build: Pass O option to kernel makefile " Wang Nan
` (6 subsequent siblings)
9 siblings, 4 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan,
Arnaldo Carvalho de Melo, Jiri Olsa, Namhyung Kim
If an 'O' is passed to 'make build-test', many 'test -x' and 'test -f'
will fail because perf resides in a different directory. Fix this by
computing PERF_OUT according to 'O' and test correct output files.
For make_kernelsrc and make_kernelsrc_tools, set KBUILD_OUTPUT_DIR
instead because the path is different from others ($(O)/perf vs
$(O)/tools/perf).
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/make | 23 +++++++++++++++--------
1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 14d7b8d..1e59ce8 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -13,10 +13,12 @@ else
endif
else
PERF := .
+PERF_OUT := $(PERF)
O_OPT :=
ifneq ($(O),)
FULL_O := $(shell readlink -f $(O) || echo $(O))
+ PERF_OUT := $(FULL_O)
ifeq ($(SET_O),1)
O_OPT := 'O=$(FULL_O)'
endif
@@ -173,11 +175,11 @@ test_make_doc := $(test_ok)
test_make_help_O := $(test_ok)
test_make_doc_O := $(test_ok)
-test_make_python_perf_so := test -f $(PERF)/python/perf.so
+test_make_python_perf_so := test -f $(PERF_OUT)/python/perf.so
-test_make_perf_o := test -f $(PERF)/perf.o
-test_make_util_map_o := test -f $(PERF)/util/map.o
-test_make_util_pmu_bison_o := test -f $(PERF)/util/pmu-bison.o
+test_make_perf_o := test -f $(PERF_OUT)/perf.o
+test_make_util_map_o := test -f $(PERF_OUT)/util/map.o
+test_make_util_pmu_bison_o := test -f $(PERF_OUT)/util/pmu-bison.o
define test_dest_files
for file in $(1); do \
@@ -244,7 +246,7 @@ test_make_perf_o_O := test -f $$TMP_O/perf.o
test_make_util_map_o_O := test -f $$TMP_O/util/map.o
test_make_util_pmu_bison_o_O := test -f $$TMP_O/util/pmu-bison.o
-test_default = test -x $(PERF)/perf
+test_default = test -x $(PERF_OUT)/perf
test = $(if $(test_$1),$(test_$1),$(test_default))
test_default_O = test -x $$TMP_O/perf
@@ -264,7 +266,7 @@ endif
MAKEFLAGS := --no-print-directory
-clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
+clean := @(cd $(PERF); make -s -f $(MK) O=$(PERF_OUT) clean >/dev/null; make -s -f $(MK) clean >/dev/null)
$(run):
$(call clean)
@@ -293,17 +295,22 @@ tarpkg:
( eval $$cmd ) >> $@ 2>&1 && \
rm -f $@
+KBUILD_OUTPUT_DIR := ../..
+ifneq ($(O),)
+ KBUILD_OUTPUT_DIR := $(O)
+endif
+
make_kernelsrc:
@echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
$(call clean); \
(make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
- test -x perf && rm -f $@ || (cat $@ ; false)
+ test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
make_kernelsrc_tools:
@echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
$(call clean); \
(make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
- test -x perf && rm -f $@ || (cat $@ ; false)
+ test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
@echo OK
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
@ 2016-01-14 9:56 ` Jiri Olsa
2016-01-14 9:57 ` Jiri Olsa
` (2 subsequent siblings)
3 siblings, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 9:56 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
> If an 'O' is passed to 'make build-test', many 'test -x' and 'test -f'
> will fail because perf resides in a different directory. Fix this by
> computing PERF_OUT according to 'O' and test correct output files.
> For make_kernelsrc and make_kernelsrc_tools, set KBUILD_OUTPUT_DIR
> instead because the path is different from others ($(O)/perf vs
> $(O)/tools/perf).
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
> tools/perf/tests/make | 23 +++++++++++++++--------
> 1 file changed, 15 insertions(+), 8 deletions(-)
>
> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> index 14d7b8d..1e59ce8 100644
> --- a/tools/perf/tests/make
> +++ b/tools/perf/tests/make
> @@ -13,10 +13,12 @@ else
> endif
> else
> PERF := .
> +PERF_OUT := $(PERF)
> O_OPT :=
given that we use _O suffix everywhere in here, would PERF_O suit better?
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
2016-01-14 9:56 ` Jiri Olsa
@ 2016-01-14 9:57 ` Jiri Olsa
2016-01-14 9:59 ` Wangnan (F)
2016-01-14 10:03 ` Jiri Olsa
2016-01-14 10:04 ` Jiri Olsa
3 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 9:57 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
SNIP
> +test_make_python_perf_so := test -f $(PERF_OUT)/python/perf.so
>
> -test_make_perf_o := test -f $(PERF)/perf.o
> -test_make_util_map_o := test -f $(PERF)/util/map.o
> -test_make_util_pmu_bison_o := test -f $(PERF)/util/pmu-bison.o
> +test_make_perf_o := test -f $(PERF_OUT)/perf.o
> +test_make_util_map_o := test -f $(PERF_OUT)/util/map.o
> +test_make_util_pmu_bison_o := test -f $(PERF_OUT)/util/pmu-bison.o
>
> define test_dest_files
> for file in $(1); do \
> @@ -244,7 +246,7 @@ test_make_perf_o_O := test -f $$TMP_O/perf.o
> test_make_util_map_o_O := test -f $$TMP_O/util/map.o
> test_make_util_pmu_bison_o_O := test -f $$TMP_O/util/pmu-bison.o
>
> -test_default = test -x $(PERF)/perf
> +test_default = test -x $(PERF_OUT)/perf
> test = $(if $(test_$1),$(test_$1),$(test_default))
>
> test_default_O = test -x $$TMP_O/perf
> @@ -264,7 +266,7 @@ endif
>
> MAKEFLAGS := --no-print-directory
>
> -clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
> +clean := @(cd $(PERF); make -s -f $(MK) O=$(PERF_OUT) clean >/dev/null; make -s -f $(MK) clean >/dev/null)
what's the second make clean for?
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-14 9:57 ` Jiri Olsa
@ 2016-01-14 9:59 ` Wangnan (F)
0 siblings, 0 replies; 29+ messages in thread
From: Wangnan (F) @ 2016-01-14 9:59 UTC (permalink / raw)
To: Jiri Olsa
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On 2016/1/14 17:57, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
>
> SNIP
>
>> +test_make_python_perf_so := test -f $(PERF_OUT)/python/perf.so
>>
>> -test_make_perf_o := test -f $(PERF)/perf.o
>> -test_make_util_map_o := test -f $(PERF)/util/map.o
>> -test_make_util_pmu_bison_o := test -f $(PERF)/util/pmu-bison.o
>> +test_make_perf_o := test -f $(PERF_OUT)/perf.o
>> +test_make_util_map_o := test -f $(PERF_OUT)/util/map.o
>> +test_make_util_pmu_bison_o := test -f $(PERF_OUT)/util/pmu-bison.o
>>
>> define test_dest_files
>> for file in $(1); do \
>> @@ -244,7 +246,7 @@ test_make_perf_o_O := test -f $$TMP_O/perf.o
>> test_make_util_map_o_O := test -f $$TMP_O/util/map.o
>> test_make_util_pmu_bison_o_O := test -f $$TMP_O/util/pmu-bison.o
>>
>> -test_default = test -x $(PERF)/perf
>> +test_default = test -x $(PERF_OUT)/perf
>> test = $(if $(test_$1),$(test_$1),$(test_default))
>>
>> test_default_O = test -x $$TMP_O/perf
>> @@ -264,7 +266,7 @@ endif
>>
>> MAKEFLAGS := --no-print-directory
>>
>> -clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
>> +clean := @(cd $(PERF); make -s -f $(MK) O=$(PERF_OUT) clean >/dev/null; make -s -f $(MK) clean >/dev/null)
> what's the second make clean for?
I want to ensure the source directory is also clean.
Looks like I forget 'O' would be passed to the second make also...
> thanks,
> jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
2016-01-14 9:56 ` Jiri Olsa
2016-01-14 9:57 ` Jiri Olsa
@ 2016-01-14 10:03 ` Jiri Olsa
2016-01-14 10:04 ` Jiri Olsa
3 siblings, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 10:03 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
> If an 'O' is passed to 'make build-test', many 'test -x' and 'test -f'
> will fail because perf resides in a different directory. Fix this by
> computing PERF_OUT according to 'O' and test correct output files.
> For make_kernelsrc and make_kernelsrc_tools, set KBUILD_OUTPUT_DIR
> instead because the path is different from others ($(O)/perf vs
> $(O)/tools/perf).
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
> tools/perf/tests/make | 23 +++++++++++++++--------
> 1 file changed, 15 insertions(+), 8 deletions(-)
>
> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> index 14d7b8d..1e59ce8 100644
> --- a/tools/perf/tests/make
> +++ b/tools/perf/tests/make
> @@ -13,10 +13,12 @@ else
> endif
> else
> PERF := .
> +PERF_OUT := $(PERF)
> O_OPT :=
>
> ifneq ($(O),)
> FULL_O := $(shell readlink -f $(O) || echo $(O))
> + PERF_OUT := $(FULL_O)
> ifeq ($(SET_O),1)
> O_OPT := 'O=$(FULL_O)'
> endif
> @@ -173,11 +175,11 @@ test_make_doc := $(test_ok)
> test_make_help_O := $(test_ok)
> test_make_doc_O := $(test_ok)
>
> -test_make_python_perf_so := test -f $(PERF)/python/perf.so
> +test_make_python_perf_so := test -f $(PERF_OUT)/python/perf.so
>
> -test_make_perf_o := test -f $(PERF)/perf.o
> -test_make_util_map_o := test -f $(PERF)/util/map.o
> -test_make_util_pmu_bison_o := test -f $(PERF)/util/pmu-bison.o
> +test_make_perf_o := test -f $(PERF_OUT)/perf.o
> +test_make_util_map_o := test -f $(PERF_OUT)/util/map.o
> +test_make_util_pmu_bison_o := test -f $(PERF_OUT)/util/pmu-bison.o
>
> define test_dest_files
> for file in $(1); do \
> @@ -244,7 +246,7 @@ test_make_perf_o_O := test -f $$TMP_O/perf.o
> test_make_util_map_o_O := test -f $$TMP_O/util/map.o
> test_make_util_pmu_bison_o_O := test -f $$TMP_O/util/pmu-bison.o
>
> -test_default = test -x $(PERF)/perf
> +test_default = test -x $(PERF_OUT)/perf
> test = $(if $(test_$1),$(test_$1),$(test_default))
>
> test_default_O = test -x $$TMP_O/perf
> @@ -264,7 +266,7 @@ endif
>
> MAKEFLAGS := --no-print-directory
>
> -clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
> +clean := @(cd $(PERF); make -s -f $(MK) O=$(PERF_OUT) clean >/dev/null; make -s -f $(MK) clean >/dev/null)
>
> $(run):
> $(call clean)
> @@ -293,17 +295,22 @@ tarpkg:
> ( eval $$cmd ) >> $@ 2>&1 && \
> rm -f $@
>
> +KBUILD_OUTPUT_DIR := ../..
> +ifneq ($(O),)
> + KBUILD_OUTPUT_DIR := $(O)
> +endif
hum, why kbuild? BUILD_O, KERNELSRC_O ... ?
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
` (2 preceding siblings ...)
2016-01-14 10:03 ` Jiri Olsa
@ 2016-01-14 10:04 ` Jiri Olsa
2016-01-14 10:14 ` Wangnan (F)
3 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 10:04 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
SNIP
> ( eval $$cmd ) >> $@ 2>&1 && \
> rm -f $@
>
> +KBUILD_OUTPUT_DIR := ../..
> +ifneq ($(O),)
> + KBUILD_OUTPUT_DIR := $(O)
> +endif
> +
> make_kernelsrc:
> @echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
> $(call clean); \
> (make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
> - test -x perf && rm -f $@ || (cat $@ ; false)
> + test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
do you miss one extra /perf in here? like it should be
test -x $(KBUILD_OUTPUT_DIR)/tools/perf/perf
>
> make_kernelsrc_tools:
> @echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
> $(call clean); \
> (make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
> - test -x perf && rm -f $@ || (cat $@ ; false)
> + test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
ditto
thanks,
jirka
>
> all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
> @echo OK
> --
> 1.8.3.4
>
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH 3/9] perf build: Test correct path of perf in build-test
2016-01-14 10:04 ` Jiri Olsa
@ 2016-01-14 10:14 ` Wangnan (F)
0 siblings, 0 replies; 29+ messages in thread
From: Wangnan (F) @ 2016-01-14 10:14 UTC (permalink / raw)
To: Jiri Olsa
Cc: acme, linux-kernel, pi3orama, lizefan, Arnaldo Carvalho de Melo,
Jiri Olsa, Namhyung Kim
On 2016/1/14 18:04, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 12:17:16PM +0000, Wang Nan wrote:
>
> SNIP
>
>> ( eval $$cmd ) >> $@ 2>&1 && \
>> rm -f $@
>>
>> +KBUILD_OUTPUT_DIR := ../..
>> +ifneq ($(O),)
>> + KBUILD_OUTPUT_DIR := $(O)
>> +endif
>> +
>> make_kernelsrc:
>> @echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
>> $(call clean); \
>> (make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
>> - test -x perf && rm -f $@ || (cat $@ ; false)
>> + test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
> do you miss one extra /perf in here? like it should be
>
> test -x $(KBUILD_OUTPUT_DIR)/tools/perf/perf
Right. Will fix.
Thank you.
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 4/9] perf build: Pass O option to kernel makefile in build-test
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (2 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 3/9] perf build: Test correct path of perf " Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-13 12:17 ` [PATCH 5/9] perf build: Add feature-dump target Wang Nan
` (5 subsequent siblings)
9 siblings, 0 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan,
Arnaldo Carvalho de Melo, Jiri Olsa, Namhyung Kim
Kernel makefile only follow 'O' option passed from command line
explicitely. In build-test with 'O' option set, kernel makefile
contaminate kernel source directory. Build test also fail if we
don't create output directory manually.
K_O_OPT is added and passed to kernel makefile if 'O' is passed
to build-test.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/make | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 1e59ce8..336a6a6 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -22,6 +22,7 @@ ifneq ($(O),)
ifeq ($(SET_O),1)
O_OPT := 'O=$(FULL_O)'
endif
+ K_O_OPT := 'O=$(FULL_O)'
endif
PARALLEL_OPT=
@@ -301,15 +302,15 @@ ifneq ($(O),)
endif
make_kernelsrc:
- @echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
+ @echo "- make -C <kernelsrc> $(PARALLEL_OPT) $(K_O_OPT) tools/perf"
$(call clean); \
- (make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
+ (make -C ../.. $(PARALLEL_OPT) $(K_O_OPT) tools/perf) > $@ 2>&1 && \
test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
make_kernelsrc_tools:
- @echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
+ @echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) $(K_O_OPT) perf"
$(call clean); \
- (make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
+ (make -C ../../tools $(PARALLEL_OPT) $(K_O_OPT) perf) > $@ 2>&1 && \
test -x $(KBUILD_OUTPUT_DIR)/tools/perf && rm -f $@ || (cat $@ ; false)
all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 5/9] perf build: Add feature-dump target
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (3 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 4/9] perf build: Pass O option to kernel makefile " Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-13 12:17 ` [PATCH 6/9] perf build: Introduce FEATURES_DUMP make variable Wang Nan
` (4 subsequent siblings)
9 siblings, 0 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa; +Cc: linux-kernel, pi3orama, lizefan, Jiri Olsa, Wang Nan
From: Jiri Olsa <jolsa@kernel.org>
To provide FEATURE-DUMP into $(FEATURE_DUMP_COPY)
if defined, with no further action.
Get feature dump of the current build:
$ make feature-dump
BUILD: Doing 'make -j4' parallel build
Auto-detecting system features:
... dwarf: [ on ]
FEATURE-DUMP file available in FEATURE-DUMP
Get feature dump static build into /tmp/fd file:
$ make feature-dump FEATURE_DUMP_COPY=/tmp/fd LDFLAGS=-static
BUILD: Doing 'make -j4' parallel build
Auto-detecting system features:
... dwarf: [ OFF ]
SNIP
FEATURE-DUMP file copied into /tmp/fd
Suggested-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-xzqhfxw3euqmls3cve0ruuol@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Wang Nan <wangnan0@huawei.com>
---
tools/perf/Makefile.perf | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 0a22407..f758a72 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -611,6 +611,17 @@ clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean
$(python-clean)
#
+# To provide FEATURE-DUMP into $(FEATURE_DUMP_COPY)
+# file if defined, with no further action.
+feature-dump:
+ifdef FEATURE_DUMP_COPY
+ @cp $(OUTPUT)FEATURE-DUMP $(FEATURE_DUMP_COPY)
+ @echo "FEATURE-DUMP file copied into $(FEATURE_DUMP_COPY)"
+else
+ @echo "FEATURE-DUMP file available in $(OUTPUT)FEATURE-DUMP"
+endif
+
+#
# Trick: if ../../.git does not exist - we are building out of tree for example,
# then force version regeneration:
#
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 6/9] perf build: Introduce FEATURES_DUMP make variable
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (4 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 5/9] perf build: Add feature-dump target Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-13 12:17 ` [PATCH 7/9] tools build: Allow subprojects select all feature checkers Wang Nan
` (3 subsequent siblings)
9 siblings, 0 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa; +Cc: linux-kernel, pi3orama, lizefan, Jiri Olsa
From: Jiri Olsa <jolsa@kernel.org>
Introducing FEATURES_DUMP make variable to provide features
detection dump file and bypass the feature detection.
The intention is to use this during build tests to skip
repeated features detection, like:
Get feature dump static build into /tmp/fd file:
$ make feature-dump FEATURE_DUMP_COPY=/tmp/fd LDFLAGS=-static
BUILD: Doing 'make -j4' parallel build
Auto-detecting system features:
... dwarf: [ OFF ]
SNIP
FEATURE-DUMP file copied into /tmp/fd
Use /tmp/fd to build perf:
$ make FEATURES_DUMP=/tmp/fd LDFLAGS=-static
$ file perf
perf: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for ...
Suggested-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-fhb47m6t18txuwrzu33is2bo@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/Makefile.perf | 14 +++++++++++++-
tools/perf/config/Makefile | 4 ++++
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index f758a72..5d34815 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -77,6 +77,9 @@ include config/utilities.mak
# Define NO_AUXTRACE if you do not want AUX area tracing support
#
# Define NO_LIBBPF if you do not want BPF support
+#
+# Define FEATURES_DUMP to provide features detection dump file
+# and bypass the feature detection
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
@@ -166,6 +169,15 @@ ifeq ($(config),1)
include config/Makefile
endif
+# The FEATURE_DUMP_EXPORT holds location of the actual
+# FEATURE_DUMP file to be used to bypass feature detection
+# (for bpf or any other subproject)
+ifeq ($(FEATURES_DUMP),)
+FEATURE_DUMP_EXPORT := $(realpath $(OUTPUT)FEATURE-DUMP)
+else
+FEATURE_DUMP_EXPORT := $(FEATURES_DUMP)
+endif
+
export prefix bindir sharedir sysconfdir DESTDIR
# sparse is architecture-neutral, which means that we need to tell it
@@ -436,7 +448,7 @@ $(LIBAPI)-clean:
$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
$(LIBBPF): fixdep FORCE
- $(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a FEATURES_DUMP=$(realpath $(OUTPUT)FEATURE-DUMP)
+ $(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a FEATURES_DUMP=$(FEATURE_DUMP_EXPORT)
$(LIBBPF)-clean:
$(call QUIET_CLEAN, libbpf)
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index e5959c1..511141b 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -181,7 +181,11 @@ LDFLAGS += -Wl,-z,noexecstack
EXTLIBS = -lpthread -lrt -lm -ldl
+ifeq ($(FEATURES_DUMP),)
include $(srctree)/tools/build/Makefile.feature
+else
+include $(FEATURES_DUMP)
+endif
ifeq ($(feature-stackprotector-all), 1)
CFLAGS += -fstack-protector-all
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 7/9] tools build: Allow subprojects select all feature checkers
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (5 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 6/9] perf build: Introduce FEATURES_DUMP make variable Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-13 12:17 ` [PATCH 8/9] perf build: Select all feature checkers for feature-dump Wang Nan
` (2 subsequent siblings)
9 siblings, 0 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
Put feature checkers not in original FEATURE_TESTS to a new list
and allow subproject select all feature checkers by setting
FEATURE_TESTS to 'all'.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/build/Makefile.feature | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 02db3cd..674c47d 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -27,7 +27,7 @@ endef
# the rule that uses them - an example for that is the 'bionic'
# feature check. ]
#
-FEATURE_TESTS ?= \
+FEATURE_TESTS_BASIC := \
backtrace \
dwarf \
fortify-source \
@@ -56,6 +56,25 @@ FEATURE_TESTS ?= \
get_cpuid \
bpf
+# FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
+# of all feature tests
+FEATURE_TESTS_EXTRA := \
+ bionic \
+ compile-32 \
+ compile-x32 \
+ cplus-demangle \
+ hello \
+ libbabeltrace \
+ liberty \
+ liberty-z \
+ libunwind-debug-frame
+
+FEATURE_TESTS ?= $(FEATURE_TESTS_BASIC)
+
+ifeq ($(FEATURE_TESTS),all)
+ FEATURE_TESTS := $(FEATURE_TESTS_BASIC) $(FEATURE_TESTS_EXTRA)
+endif
+
FEATURE_DISPLAY ?= \
dwarf \
glibc \
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 8/9] perf build: Select all feature checkers for feature-dump
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (6 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 7/9] tools build: Allow subprojects select all feature checkers Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-14 10:18 ` Jiri Olsa
2016-01-13 12:17 ` [PATCH 9/9] perf build: Use feature dump file for build-test Wang Nan
2016-01-14 1:54 ` [PATCH 0/9] perf build: Make build-test faster Wangnan (F)
9 siblings, 1 reply; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
Set FEATURE_TESTS to 'all' so all possible feature checkers are'
executed. Without this setting the output feature dump file miss
some feature, for example, liberity. Select all checker so we won't
use a incomplete feature dump file.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/Makefile.perf | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 5d34815..a199fc4 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -166,6 +166,11 @@ endif
endif
ifeq ($(config),1)
+ifdef MAKECMDGOALS
+ifeq ($(filter feature-dump,$(MAKECMDGOALS)),feature-dump)
+FEATURE_TESTS := all
+endif
+endif
include config/Makefile
endif
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH 8/9] perf build: Select all feature checkers for feature-dump
2016-01-13 12:17 ` [PATCH 8/9] perf build: Select all feature checkers for feature-dump Wang Nan
@ 2016-01-14 10:18 ` Jiri Olsa
0 siblings, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 10:18 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:21PM +0000, Wang Nan wrote:
> Set FEATURE_TESTS to 'all' so all possible feature checkers are'
> executed. Without this setting the output feature dump file miss
> some feature, for example, liberity. Select all checker so we won't
> use a incomplete feature dump file.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
> tools/perf/Makefile.perf | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 5d34815..a199fc4 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -166,6 +166,11 @@ endif
> endif
>
> ifeq ($(config),1)
> +ifdef MAKECMDGOALS
> +ifeq ($(filter feature-dump,$(MAKECMDGOALS)),feature-dump)
> +FEATURE_TESTS := all
> +endif
> +endif
please put the changelog comment in here as well
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 9/9] perf build: Use feature dump file for build-test
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (7 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 8/9] perf build: Select all feature checkers for feature-dump Wang Nan
@ 2016-01-13 12:17 ` Wang Nan
2016-01-14 10:19 ` Jiri Olsa
2016-01-14 10:23 ` Jiri Olsa
2016-01-14 1:54 ` [PATCH 0/9] perf build: Make build-test faster Wangnan (F)
9 siblings, 2 replies; 29+ messages in thread
From: Wang Nan @ 2016-01-13 12:17 UTC (permalink / raw)
To: acme, jolsa
Cc: linux-kernel, pi3orama, lizefan, Wang Nan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
To prevent feature check run too many times, this patch utilizes
previous introduced feature-dump make target and FEATURES_DUMP
variable, makes sure the feature checkers run only once when doing
build-test for normal test cases.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
---
tools/perf/tests/make | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 336a6a6..10493ab 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -15,6 +15,7 @@ else
PERF := .
PERF_OUT := $(PERF)
O_OPT :=
+FULL_O := $(shell readlink -f $(PERF_OUT) || echo $(PERF_OUT))
ifneq ($(O),)
FULL_O := $(shell readlink -f $(O) || echo $(O))
@@ -319,5 +320,29 @@ all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
out: $(run_O)
@echo OK
+FEATURES_DUMP_FILE := $(FULL_O)/BUILD_TEST_FEATURE_DUMP
+FEATURES_DUMP_FILE_STATIC := $(FULL_O)/BUILD_TEST_FEATURE_DUMP_STATIC
+
+$(FEATURES_DUMP_FILE):
+ $(call clean)
+ @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) feature-dump"; \
+ echo "- $@: $$cmd" && echo $$cmd && \
+ ( eval $$cmd ) 2>&1
+
+$(FEATURES_DUMP_FILE_STATIC):
+ $(call clean)
+ @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) LDFLAGS='-static' feature-dump"; \
+ echo "- $@: $$cmd" && echo $$cmd && \
+ ( eval $$cmd ) 2>&1
+
+$(foreach t,$(run) $(run_O),$(eval \
+ $(t): $(if $(findstring make_static,$(t)),\
+ $(FEATURES_DUMP_FILE_STATIC),\
+ $(FEATURES_DUMP_FILE))))
+
+$(foreach t,$(run),$(if $(findstring make_static,$(t)),\
+ $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE_STATIC)),\
+ $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE))))
+
.PHONY: all $(run) $(run_O) tarpkg clean make_kernelsrc make_kernelsrc_tools
endif # ifndef MK
--
1.8.3.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH 9/9] perf build: Use feature dump file for build-test
2016-01-13 12:17 ` [PATCH 9/9] perf build: Use feature dump file for build-test Wang Nan
@ 2016-01-14 10:19 ` Jiri Olsa
2016-01-14 10:24 ` Wangnan (F)
2016-01-14 10:23 ` Jiri Olsa
1 sibling, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 10:19 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:22PM +0000, Wang Nan wrote:
SNIP
> +$(FEATURES_DUMP_FILE_STATIC):
> + $(call clean)
> + @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) LDFLAGS='-static' feature-dump"; \
> + echo "- $@: $$cmd" && echo $$cmd && \
> + ( eval $$cmd ) 2>&1
> +
> +$(foreach t,$(run) $(run_O),$(eval \
> + $(t): $(if $(findstring make_static,$(t)),\
> + $(FEATURES_DUMP_FILE_STATIC),\
> + $(FEATURES_DUMP_FILE))))
> +
> +$(foreach t,$(run),$(if $(findstring make_static,$(t)),\
> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE_STATIC)),\
> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE))))
> +
the BUILD_TEST_FEATURE_DUMP rebuild is not silent:
[jolsa@krava perf]$ rm BUILD_TEST_FEATURE_DUMP
[jolsa@krava perf]$ make -f tests/make
Testing Makefile
- /home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP: cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
BUILD: Doing 'make -j4' parallel build
Auto-detecting system features:
... dwarf: [ on ]
... glibc: [ on ]
... gtk2: [ on ]
... libaudit: [ on ]
... libbfd: [ on ]
... libelf: [ on ]
... libnuma: [ on ]
... numa_num_possible_cpus: [ on ]
... libperl: [ on ]
... libpython: [ on ]
... libslang: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ on ]
... zlib: [ on ]
... lzma: [ on ]
... get_cpuid: [ on ]
... bpf: [ on ]
...
also should 'make clean' remove BUILD_TEST_FEATURE_DUMP?
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 9/9] perf build: Use feature dump file for build-test
2016-01-14 10:19 ` Jiri Olsa
@ 2016-01-14 10:24 ` Wangnan (F)
2016-01-14 12:44 ` Jiri Olsa
0 siblings, 1 reply; 29+ messages in thread
From: Wangnan (F) @ 2016-01-14 10:24 UTC (permalink / raw)
To: Jiri Olsa
Cc: acme, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
On 2016/1/14 18:19, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 12:17:22PM +0000, Wang Nan wrote:
>
> SNIP
>
>> +$(FEATURES_DUMP_FILE_STATIC):
>> + $(call clean)
>> + @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) LDFLAGS='-static' feature-dump"; \
>> + echo "- $@: $$cmd" && echo $$cmd && \
>> + ( eval $$cmd ) 2>&1
>> +
>> +$(foreach t,$(run) $(run_O),$(eval \
>> + $(t): $(if $(findstring make_static,$(t)),\
>> + $(FEATURES_DUMP_FILE_STATIC),\
>> + $(FEATURES_DUMP_FILE))))
>> +
>> +$(foreach t,$(run),$(if $(findstring make_static,$(t)),\
>> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE_STATIC)),\
>> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE))))
>> +
> the BUILD_TEST_FEATURE_DUMP rebuild is not silent:
>
> [jolsa@krava perf]$ rm BUILD_TEST_FEATURE_DUMP
> [jolsa@krava perf]$ make -f tests/make
> Testing Makefile
> - /home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP: cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
> cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
> BUILD: Doing 'make -j4' parallel build
>
> Auto-detecting system features:
> ... dwarf: [ on ]
> ... glibc: [ on ]
> ... gtk2: [ on ]
> ... libaudit: [ on ]
> ... libbfd: [ on ]
> ... libelf: [ on ]
> ... libnuma: [ on ]
> ... numa_num_possible_cpus: [ on ]
> ... libperl: [ on ]
> ... libpython: [ on ]
> ... libslang: [ on ]
> ... libunwind: [ on ]
> ... libdw-dwarf-unwind: [ on ]
> ... zlib: [ on ]
> ... lzma: [ on ]
> ... get_cpuid: [ on ]
> ... bpf: [ on ]
>
> ...
>
>
> also should 'make clean' remove BUILD_TEST_FEATURE_DUMP?
If a simple 'make clean' removes these two feature dump file then
they are unable to reused by multiple test cases. During build-test
'make clean' is called many times.
What about force removing these two files here:
ifeq ($(MAKECMDGOALS),)
# no target specified, trigger the whole suite
all:
@rm -f $(FEATURES_DUMP_FILE) $(FEATURES_DUMP_FILE_STATIC) <---
*HERE*
@echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make
MK=Makefile.perf SET_PARALLEL=1 SET_O=1
else
# run only specific test over 'Makefile'
%:
@rm -f $(FEATURES_DUMP_FILE) $(FEATURES_DUMP_FILE_STATIC) <---
*and HERE*
@echo "Testing Makefile"; $(MAKE) -sf tests/make
MK=Makefile $@
endif
> thanks,
> jirka
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH 9/9] perf build: Use feature dump file for build-test
2016-01-14 10:24 ` Wangnan (F)
@ 2016-01-14 12:44 ` Jiri Olsa
0 siblings, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 12:44 UTC (permalink / raw)
To: Wangnan (F)
Cc: acme, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
On Thu, Jan 14, 2016 at 06:24:00PM +0800, Wangnan (F) wrote:
>
>
> On 2016/1/14 18:19, Jiri Olsa wrote:
> >On Wed, Jan 13, 2016 at 12:17:22PM +0000, Wang Nan wrote:
> >
> >SNIP
> >
> >>+$(FEATURES_DUMP_FILE_STATIC):
> >>+ $(call clean)
> >>+ @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) LDFLAGS='-static' feature-dump"; \
> >>+ echo "- $@: $$cmd" && echo $$cmd && \
> >>+ ( eval $$cmd ) 2>&1
> >>+
> >>+$(foreach t,$(run) $(run_O),$(eval \
> >>+ $(t): $(if $(findstring make_static,$(t)),\
> >>+ $(FEATURES_DUMP_FILE_STATIC),\
> >>+ $(FEATURES_DUMP_FILE))))
> >>+
> >>+$(foreach t,$(run),$(if $(findstring make_static,$(t)),\
> >>+ $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE_STATIC)),\
> >>+ $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE))))
> >>+
> >the BUILD_TEST_FEATURE_DUMP rebuild is not silent:
> >
> >[jolsa@krava perf]$ rm BUILD_TEST_FEATURE_DUMP
> >[jolsa@krava perf]$ make -f tests/make
> >Testing Makefile
> >- /home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP: cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
> >cd . && make FEATURE_DUMP_COPY=/home/jolsa/kernel/linux-perf/tools/perf/BUILD_TEST_FEATURE_DUMP feature-dump
> > BUILD: Doing 'make -j4' parallel build
> >
> >Auto-detecting system features:
> >... dwarf: [ on ]
> >... glibc: [ on ]
> >... gtk2: [ on ]
> >... libaudit: [ on ]
> >... libbfd: [ on ]
> >... libelf: [ on ]
> >... libnuma: [ on ]
> >... numa_num_possible_cpus: [ on ]
> >... libperl: [ on ]
> >... libpython: [ on ]
> >... libslang: [ on ]
> >... libunwind: [ on ]
> >... libdw-dwarf-unwind: [ on ]
> >... zlib: [ on ]
> >... lzma: [ on ]
> >... get_cpuid: [ on ]
> >... bpf: [ on ]
> >
> >...
> >
> >
> >also should 'make clean' remove BUILD_TEST_FEATURE_DUMP?
>
> If a simple 'make clean' removes these two feature dump file then
> they are unable to reused by multiple test cases. During build-test
> 'make clean' is called many times.
>
> What about force removing these two files here:
>
> ifeq ($(MAKECMDGOALS),)
> # no target specified, trigger the whole suite
> all:
> @rm -f $(FEATURES_DUMP_FILE) $(FEATURES_DUMP_FILE_STATIC) <---
> *HERE*
> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile
> @echo "Testing Makefile.perf"; $(MAKE) -sf tests/make
> MK=Makefile.perf SET_PARALLEL=1 SET_O=1
> else
> # run only specific test over 'Makefile'
> %:
> @rm -f $(FEATURES_DUMP_FILE) $(FEATURES_DUMP_FILE_STATIC) <--- *and
> HERE*
> @echo "Testing Makefile"; $(MAKE) -sf tests/make MK=Makefile $@
> endif
wouldn't they be rebuilt right away?
I think it'd be ok just to remove them right after the test
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 9/9] perf build: Use feature dump file for build-test
2016-01-13 12:17 ` [PATCH 9/9] perf build: Use feature dump file for build-test Wang Nan
2016-01-14 10:19 ` Jiri Olsa
@ 2016-01-14 10:23 ` Jiri Olsa
1 sibling, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2016-01-14 10:23 UTC (permalink / raw)
To: Wang Nan
Cc: acme, linux-kernel, pi3orama, lizefan, Jiri Olsa,
Arnaldo Carvalho de Melo, Namhyung Kim
On Wed, Jan 13, 2016 at 12:17:22PM +0000, Wang Nan wrote:
> To prevent feature check run too many times, this patch utilizes
> previous introduced feature-dump make target and FEATURES_DUMP
> variable, makes sure the feature checkers run only once when doing
> build-test for normal test cases.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
> tools/perf/tests/make | 25 +++++++++++++++++++++++++
> 1 file changed, 25 insertions(+)
>
> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> index 336a6a6..10493ab 100644
> --- a/tools/perf/tests/make
> +++ b/tools/perf/tests/make
> @@ -15,6 +15,7 @@ else
> PERF := .
> PERF_OUT := $(PERF)
> O_OPT :=
> +FULL_O := $(shell readlink -f $(PERF_OUT) || echo $(PERF_OUT))
>
> ifneq ($(O),)
> FULL_O := $(shell readlink -f $(O) || echo $(O))
> @@ -319,5 +320,29 @@ all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
> out: $(run_O)
> @echo OK
>
> +FEATURES_DUMP_FILE := $(FULL_O)/BUILD_TEST_FEATURE_DUMP
> +FEATURES_DUMP_FILE_STATIC := $(FULL_O)/BUILD_TEST_FEATURE_DUMP_STATIC
> +
> +$(FEATURES_DUMP_FILE):
> + $(call clean)
> + @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) feature-dump"; \
> + echo "- $@: $$cmd" && echo $$cmd && \
> + ( eval $$cmd ) 2>&1
> +
> +$(FEATURES_DUMP_FILE_STATIC):
> + $(call clean)
> + @cmd="cd $(PERF) && make FEATURE_DUMP_COPY=$@ $(O_OPT) LDFLAGS='-static' feature-dump"; \
> + echo "- $@: $$cmd" && echo $$cmd && \
> + ( eval $$cmd ) 2>&1
> +
> +$(foreach t,$(run) $(run_O),$(eval \
> + $(t): $(if $(findstring make_static,$(t)),\
> + $(FEATURES_DUMP_FILE_STATIC),\
> + $(FEATURES_DUMP_FILE))))
could you please put comment in here.. it's not too obvious ;-)
like saying it's just adding proper feature dump dependency
for run/run_O targets.. or something like that
> +
> +$(foreach t,$(run),$(if $(findstring make_static,$(t)),\
> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE_STATIC)),\
> + $(eval $(t) := $($(t)) FEATURES_DUMP=$(FEATURES_DUMP_FILE))))
> +
ditto
thanks,
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 0/9] perf build: Make build-test faster
2016-01-13 12:17 [PATCH 0/9] perf build: Make build-test faster Wang Nan
` (8 preceding siblings ...)
2016-01-13 12:17 ` [PATCH 9/9] perf build: Use feature dump file for build-test Wang Nan
@ 2016-01-14 1:54 ` Wangnan (F)
9 siblings, 0 replies; 29+ messages in thread
From: Wangnan (F) @ 2016-01-14 1:54 UTC (permalink / raw)
To: acme, jolsa; +Cc: linux-kernel, pi3orama, lizefan
Hi, Jiri and Arnaldo,
You can find this series on git-tree:
git://git.kernel.org/pub/scm/linux/kernel/git/pi3orama/linux.git
perf/build-test
Other patches I sent these days can be found in perf/ebpf and
perf/overwrite.
Thank you.
On 2016/1/13 20:17, Wang Nan wrote:
> Utilize Jiri's two RFCs to make build-test faster.
>
> Before his RFC on a 24 core machine:
> real 21m16.579s
> user 176m12.989s
> sys 20m54.950s
>
> After his RFC:
> real 16m0.619s
> user 172m53.896s
> sys 19m22.777s
>
> Speedup: 24.75%.
>
> Jiri Olsa (2):
> perf build: Add feature-dump target
> perf build: Introduce FEATURES_DUMP make variable
>
> Wang Nan (7):
> perf build: Set parallel making options build-test
> perf build: Pass O option to Makefile.perf in build-test
> perf build: Test correct path of perf in build-test
> perf build: Pass O option to kernel makefile in build-test
> tools build: Allow subprojects select all feature checkers
> perf build: Select all feature checkers for feature-dump
> perf build: Use feature dump file for build-test
>
> tools/build/Makefile.feature | 21 +++++++++++-
> tools/perf/Makefile.perf | 30 ++++++++++++++++-
> tools/perf/config/Makefile | 4 +++
> tools/perf/tests/make | 80 +++++++++++++++++++++++++++++++++++---------
> 4 files changed, 118 insertions(+), 17 deletions(-)
>
^ permalink raw reply [flat|nested] 29+ messages in thread