From: Ingo Molnar <mingo@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Namhyung Kim <namhyung@kernel.org>,
David Ahern <dsahern@gmail.com>, Jiri Olsa <jolsa@redhat.com>
Subject: [RFC PATCH 00/50] tools/perf: Speed up the build system
Date: Mon, 7 Oct 2013 13:55:53 +0200 [thread overview]
Message-ID: <1381147003-2574-1-git-send-email-mingo@kernel.org> (raw)
This series improves various aspects of the tools/perf build system.
The biggest change is the rewrite of the feature detection framework and a
conversion of all feature testcases to the new method. The new feature
detection machinery relies on three new methods:
- split-out, independent testcases that allow "make -j" parallel builds
[ the old build system built the 20+ testcases serially, greatly
slowing down the build. ]
- dependency auto-generation via gcc's -MD feature, which allows the
'caching' of testcases.
- a 'test-all' testcase that contains all the other testcases and allows
a fast check for the typical 'all dependencies are installed' case.
The series also includes a number of fixes and convenience features:
- auto-detect the number of CPUs on the system and add the -j<cpus>
option automatically
- Fix 'make O=/some/dir/ perf.o' type of rules to work in the $(OUTPUT)
directory
- Improve documentation builds/install to not require feature-detection
- speed up linking, eliminate spurious rebuilds
Altogether build speed on my testsystem improved by a factor of 2.1x:
[BEFORE]
$ cd tools/perf
$ time make -j12 install
...
real 0m27.522s
user 1m59.478s
sys 0m9.714s
[AFTER]
$ cd tools/perf
$ time make install
...
real 0m12.658s
user 1m39.507s
sys 0m7.706s
Re-build speed, which is important to perf developers, increased 5.0x:
[BEFORE]
$ touch perf.c
$ time make
...
real 0m9.368s
user 0m8.166s
sys 0m1.015s
[AFTER]
$ touch perf.c
$ time make
BUILD: Doing 'make -j12' parallel build
CC perf.o
LINK perf
real 0m1.857s
user 0m1.522s
sys 0m0.295s
Empty build speed of 'make' in an already built tree - often done by
developers to see whether they are testing the latest binary, increased
more than 50-fold:
[BEFORE]
$ time make
...
real 0m9.105s
user 0m7.975s
sys 0m0.949s
[AFTER]
$ time make
$ time make
BUILD: Doing 'make -j12' parallel build
real 0m0.172s
user 0m0.119s
sys 0m0.029s
Thanks,
Ingo
=============>
Ingo Molnar (50):
tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT
tools/perf/build: Add feature check core code
tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically
tools/perf/build: Split out feature check: 'libnuma'
tools/perf/build: Split out feature check: 'stackprotector-all'
tools/perf/build: Split out feature check: 'stackprotector'
tools/perf/build: Split out feature check: 'volatile-register-var'
tools/perf/build: Split out feature check: 'fortify-source'
tools/perf/build: Split out feature check: 'bionic'
tools/perf/build: Clean up the libelf logic in config/Makefile
tools/perf/build: Split out feature check: 'libelf'
tools/perf/build: Split out feature check: 'glibc'
tools/perf/build: Split out feature check: 'dwarf'
tools/perf/build: Clean up the mmap logic in config/Makefile
tools/perf/build: Split out feature check: 'libelf-mmap'
tools/perf/build: Split out feature check: 'libelf-getphdrnum'
tools/perf/build: Clean up the libunwind logic in config/Makefile
tools/perf/build: Split out feature check: 'libunwind'
tools/perf/build: Split out feature check: 'libaudit'
tools/perf/build: Split out feature check: 'libslang'
tools/perf/build: Split out feature check: 'gtk2'
tools/perf/build: Split out feature check: 'gtk2-infobar'
tools/perf/build: Split out feature check: 'libperl'
tools/perf/build: Split out feature check: 'libpython'
tools/perf/build: Split out feature check: 'libpython-version'
tools/perf/build: Split out feature check: 'libbfd'
tools/perf/build: Split out feature check: 'on-exit'
tools/perf/build: Split out feature check: 'backtrace'
tools/perf: Clean up util/include/linux/compiler.h
tools/perf: Turn strlcpy() into a __weak function
tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target
tools/perf/build: Speed up git-version test on re-make
tools/perf/build: Speed up the final link
tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al
tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile
tools/perf/build: Speed up auto-detection
tools/perf/build: Improve printout-of auto-detected features
tools/perf/build: Automatically build in parallel, based on number of CPUs in the system
tools/perf/build: Flip Makefile.parallel and Makefile.perf
tools/perf/build: Standardize the various messages output by parallel make
tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle'
tools/perf/build: Remove unused config/feature-tests.mak
tools/perf/build: Clean up various testcases
tools/perf/build: Collapse the test-all.c testcase
tools/perf/build: Pass through all targets to Makefile.perf
tools/perf/build: Make sure autodep feature binaries honor the O= setting
tools/perf/build: Exclude MAKEFLAGS from nested invocation
tools/perf/build: Fix non-canonical directory names in O=
tools/perf/build: Fix O=/some/dir perf.o type of targets
tools/perf/build: Harmonize the style of the feature testcases
tools/perf/Documentation/Makefile | 19 +-
tools/perf/Makefile | 840 ++---------------------------
tools/perf/Makefile.perf | 855 ++++++++++++++++++++++++++++++
tools/perf/arch/x86/include/perf_regs.h | 6 +-
tools/perf/arch/x86/util/unwind.c | 4 +-
tools/perf/bench/mem-memcpy-arch.h | 2 +-
tools/perf/bench/mem-memcpy.c | 2 +-
tools/perf/bench/mem-memset-arch.h | 2 +-
tools/perf/bench/mem-memset.c | 2 +-
tools/perf/builtin-bench.c | 4 +-
tools/perf/builtin-inject.c | 2 +-
tools/perf/builtin-probe.c | 14 +-
tools/perf/builtin-record.c | 12 +-
tools/perf/config/Makefile | 326 +++++++-----
tools/perf/config/feature-checks/Makefile | 148 ++++++
tools/perf/config/feature-checks/test-all.c | 106 ++++
tools/perf/config/feature-checks/test-backtrace.c | 13 +
tools/perf/config/feature-checks/test-bionic.c | 6 +
tools/perf/config/feature-checks/test-cplus-demangle.c | 14 +
tools/perf/config/feature-checks/test-dwarf.c | 10 +
tools/perf/config/feature-checks/test-fortify-source.c | 6 +
tools/perf/config/feature-checks/test-glibc.c | 8 +
tools/perf/config/feature-checks/test-gtk2-infobar.c | 11 +
tools/perf/config/feature-checks/test-gtk2.c | 10 +
tools/perf/config/feature-checks/test-hello.c | 6 +
tools/perf/config/feature-checks/test-libaudit.c | 10 +
tools/perf/config/feature-checks/test-libbfd.c | 15 +
tools/perf/config/feature-checks/test-libelf-getphdrnum.c | 8 +
tools/perf/config/feature-checks/test-libelf-mmap.c | 8 +
tools/perf/config/feature-checks/test-libelf.c | 8 +
tools/perf/config/feature-checks/test-libnuma.c | 9 +
tools/perf/config/feature-checks/test-libperl.c | 9 +
tools/perf/config/feature-checks/test-libpython-version.c | 10 +
tools/perf/config/feature-checks/test-libpython.c | 8 +
tools/perf/config/feature-checks/test-libslang.c | 6 +
tools/perf/config/feature-checks/test-libunwind.c | 27 +
tools/perf/config/feature-checks/test-on-exit.c | 15 +
tools/perf/config/feature-checks/test-stackprotector-all.c | 6 +
tools/perf/config/feature-checks/test-stackprotector.c | 6 +
tools/perf/config/feature-checks/test-volatile-register-var.c | 6 +
tools/perf/config/feature-tests.mak | 246 ---------
tools/perf/config/utilities.mak | 14 -
tools/perf/perf.c | 4 +-
tools/perf/ui/gtk/browser.c | 2 +-
tools/perf/ui/gtk/gtk.h | 4 +-
tools/perf/ui/gtk/util.c | 4 +-
tools/perf/ui/ui.h | 4 +-
tools/perf/util/annotate.h | 4 +-
tools/perf/util/cache.h | 3 +-
tools/perf/util/generate-cmdlist.sh | 4 +-
tools/perf/util/hist.h | 4 +-
tools/perf/util/include/dwarf-regs.h | 2 +-
tools/perf/util/include/linux/compiler.h | 19 +-
tools/perf/util/map.c | 2 +-
tools/perf/util/path.c | 10 +-
tools/perf/util/perf_regs.h | 4 +-
tools/perf/util/probe-event.c | 4 +-
tools/perf/util/probe-finder.h | 4 +-
tools/perf/util/symbol-elf.c | 2 +-
tools/perf/util/symbol.h | 8 +-
tools/perf/util/unwind.h | 4 +-
tools/perf/util/util.c | 4 +-
62 files changed, 1678 insertions(+), 1257 deletions(-)
create mode 100644 tools/perf/Makefile.perf
create mode 100644 tools/perf/config/feature-checks/Makefile
create mode 100644 tools/perf/config/feature-checks/test-all.c
create mode 100644 tools/perf/config/feature-checks/test-backtrace.c
create mode 100644 tools/perf/config/feature-checks/test-bionic.c
create mode 100644 tools/perf/config/feature-checks/test-cplus-demangle.c
create mode 100644 tools/perf/config/feature-checks/test-dwarf.c
create mode 100644 tools/perf/config/feature-checks/test-fortify-source.c
create mode 100644 tools/perf/config/feature-checks/test-glibc.c
create mode 100644 tools/perf/config/feature-checks/test-gtk2-infobar.c
create mode 100644 tools/perf/config/feature-checks/test-gtk2.c
create mode 100644 tools/perf/config/feature-checks/test-hello.c
create mode 100644 tools/perf/config/feature-checks/test-libaudit.c
create mode 100644 tools/perf/config/feature-checks/test-libbfd.c
create mode 100644 tools/perf/config/feature-checks/test-libelf-getphdrnum.c
create mode 100644 tools/perf/config/feature-checks/test-libelf-mmap.c
create mode 100644 tools/perf/config/feature-checks/test-libelf.c
create mode 100644 tools/perf/config/feature-checks/test-libnuma.c
create mode 100644 tools/perf/config/feature-checks/test-libperl.c
create mode 100644 tools/perf/config/feature-checks/test-libpython-version.c
create mode 100644 tools/perf/config/feature-checks/test-libpython.c
create mode 100644 tools/perf/config/feature-checks/test-libslang.c
create mode 100644 tools/perf/config/feature-checks/test-libunwind.c
create mode 100644 tools/perf/config/feature-checks/test-on-exit.c
create mode 100644 tools/perf/config/feature-checks/test-stackprotector-all.c
create mode 100644 tools/perf/config/feature-checks/test-stackprotector.c
create mode 100644 tools/perf/config/feature-checks/test-volatile-register-var.c
delete mode 100644 tools/perf/config/feature-tests.mak
--
1.7.11.7
next reply other threads:[~2013-10-07 11:57 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-07 11:55 Ingo Molnar [this message]
2013-10-07 11:55 ` [PATCH 01/50] tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT Ingo Molnar
2013-10-07 11:55 ` [PATCH 02/50] tools/perf/build: Add feature check core code Ingo Molnar
2013-10-07 11:55 ` [PATCH 03/50] tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically Ingo Molnar
2013-10-07 11:55 ` [PATCH 04/50] tools/perf/build: Split out feature check: 'libnuma' Ingo Molnar
2013-10-07 11:55 ` [PATCH 05/50] tools/perf/build: Split out feature check: 'stackprotector-all' Ingo Molnar
2013-10-07 11:55 ` [PATCH 06/50] tools/perf/build: Split out feature check: 'stackprotector' Ingo Molnar
2013-10-07 11:56 ` [PATCH 07/50] tools/perf/build: Split out feature check: 'volatile-register-var' Ingo Molnar
2013-10-07 11:56 ` [PATCH 08/50] tools/perf/build: Split out feature check: 'fortify-source' Ingo Molnar
2013-10-07 11:56 ` [PATCH 09/50] tools/perf/build: Split out feature check: 'bionic' Ingo Molnar
2013-10-07 11:56 ` [PATCH 10/50] tools/perf/build: Clean up the libelf logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 11/50] tools/perf/build: Split out feature check: 'libelf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 12/50] tools/perf/build: Split out feature check: 'glibc' Ingo Molnar
2013-10-07 11:56 ` [PATCH 13/50] tools/perf/build: Split out feature check: 'dwarf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 14/50] tools/perf/build: Clean up the mmap logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 15/50] tools/perf/build: Split out feature check: 'libelf-mmap' Ingo Molnar
2013-10-07 11:56 ` [PATCH 16/50] tools/perf/build: Split out feature check: 'libelf-getphdrnum' Ingo Molnar
2013-10-07 11:56 ` [PATCH 17/50] tools/perf/build: Clean up the libunwind logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 18/50] tools/perf/build: Split out feature check: 'libunwind' Ingo Molnar
2013-10-07 11:56 ` [PATCH 19/50] tools/perf/build: Split out feature check: 'libaudit' Ingo Molnar
2013-10-07 21:00 ` Jiri Olsa
2013-10-08 8:19 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 20/50] tools/perf/build: Split out feature check: 'libslang' Ingo Molnar
2013-10-07 14:52 ` David Ahern
2013-10-08 8:06 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 21/50] tools/perf/build: Split out feature check: 'gtk2' Ingo Molnar
2013-10-07 11:56 ` [PATCH 22/50] tools/perf/build: Split out feature check: 'gtk2-infobar' Ingo Molnar
2013-10-07 11:56 ` [PATCH 23/50] tools/perf/build: Split out feature check: 'libperl' Ingo Molnar
2013-10-07 11:56 ` [PATCH 24/50] tools/perf/build: Split out feature check: 'libpython' Ingo Molnar
2013-10-07 11:56 ` [PATCH 25/50] tools/perf/build: Split out feature check: 'libpython-version' Ingo Molnar
2013-10-07 11:56 ` [PATCH 26/50] tools/perf/build: Split out feature check: 'libbfd' Ingo Molnar
2013-10-07 15:11 ` David Ahern
2013-10-07 21:18 ` Jiri Olsa
2013-10-08 8:15 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 27/50] tools/perf/build: Split out feature check: 'on-exit' Ingo Molnar
2013-10-07 11:56 ` [PATCH 28/50] tools/perf/build: Split out feature check: 'backtrace' Ingo Molnar
2013-10-07 11:56 ` [PATCH 29/50] tools/perf: Clean up util/include/linux/compiler.h Ingo Molnar
2013-10-07 11:56 ` [PATCH 30/50] tools/perf: Turn strlcpy() into a __weak function Ingo Molnar
2013-10-07 11:56 ` [PATCH 31/50] tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target Ingo Molnar
2013-10-07 21:27 ` Jiri Olsa
2013-10-08 8:23 ` Ingo Molnar
2013-10-07 21:59 ` Jiri Olsa
2013-10-07 11:56 ` [PATCH 32/50] tools/perf/build: Speed up git-version test on re-make Ingo Molnar
2013-10-07 11:56 ` [PATCH 33/50] tools/perf/build: Speed up the final link Ingo Molnar
2013-10-07 11:56 ` [PATCH 34/50] tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al Ingo Molnar
2013-10-07 11:56 ` [PATCH 35/50] tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 36/50] tools/perf/build: Speed up auto-detection Ingo Molnar
2013-10-07 21:57 ` Jiri Olsa
2013-10-08 8:36 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 37/50] tools/perf/build: Improve printout-of auto-detected features Ingo Molnar
2013-10-07 22:12 ` Jiri Olsa
2013-10-08 8:46 ` Ingo Molnar
2013-10-08 15:51 ` Jiri Olsa
2013-10-09 6:42 ` [PATCH 53/52] tools/perf/build: Clean up feature_print_code() Ingo Molnar
2013-10-09 8:38 ` Jiri Olsa
2013-10-15 5:20 ` [tip:perf/core] " tip-bot for Jiri Olsa
2013-10-07 11:56 ` [PATCH 38/50] tools/perf/build: Automatically build in parallel, based on number of CPUs in the system Ingo Molnar
2013-10-07 11:56 ` [PATCH 39/50] tools/perf/build: Flip Makefile.parallel and Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 40/50] tools/perf/build: Standardize the various messages output by parallel make Ingo Molnar
2013-10-07 11:56 ` [PATCH 41/50] tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle' Ingo Molnar
2013-10-07 11:56 ` [PATCH 42/50] tools/perf/build: Remove unused config/feature-tests.mak Ingo Molnar
2013-10-07 11:56 ` [PATCH 43/50] tools/perf/build: Clean up various testcases Ingo Molnar
2013-10-07 11:56 ` [PATCH 44/50] tools/perf/build: Collapse the test-all.c testcase Ingo Molnar
2013-10-07 22:25 ` Jiri Olsa
2013-10-08 8:47 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 45/50] tools/perf/build: Pass through all targets to Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 46/50] tools/perf/build: Make sure autodep feature binaries honor the O= setting Ingo Molnar
2013-10-07 11:56 ` [PATCH 47/50] tools/perf/build: Exclude MAKEFLAGS from nested invocation Ingo Molnar
2013-10-07 11:56 ` [PATCH 48/50] tools/perf/build: Fix non-canonical directory names in O= Ingo Molnar
2013-10-07 11:56 ` [PATCH 49/50] tools/perf/build: Fix O=/some/dir perf.o type of targets Ingo Molnar
2013-10-07 22:33 ` Jiri Olsa
2013-10-08 8:49 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 50/50] tools/perf/build: Harmonize the style of the feature testcases Ingo Molnar
2013-10-07 13:52 ` [RFC PATCH 00/50] tools/perf: Speed up the build system David Ahern
2013-10-07 14:11 ` Ingo Molnar
2013-10-07 15:35 ` David Ahern
2013-10-07 15:51 ` [PATCH] tools/perf/build: Pass through LDFLAGS to feature tests Ingo Molnar
2013-10-07 16:07 ` David Ahern
2013-10-15 5:20 ` [tip:perf/core] " tip-bot for Ingo Molnar
2013-10-07 16:41 ` [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-08 1:41 ` David Ahern
2013-10-08 12:48 ` Ingo Molnar
2013-10-07 20:15 ` Andi Kleen
2013-10-08 9:02 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1381147003-2574-1-git-send-email-mingo@kernel.org \
--to=mingo@kernel.org \
--cc=a.p.zijlstra@chello.nl \
--cc=acme@redhat.com \
--cc=dsahern@gmail.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=namhyung@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).