From: Ingo Molnar <mingo@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Namhyung Kim <namhyung@kernel.org>,
David Ahern <dsahern@gmail.com>, Jiri Olsa <jolsa@redhat.com>
Subject: [RFC PATCH 00/50] tools/perf: Speed up the build system
Date: Mon, 7 Oct 2013 13:55:53 +0200 [thread overview]
Message-ID: <1381147003-2574-1-git-send-email-mingo@kernel.org> (raw)
This series improves various aspects of the tools/perf build system.
The biggest change is the rewrite of the feature detection framework and a
conversion of all feature testcases to the new method. The new feature
detection machinery relies on three new methods:
- split-out, independent testcases that allow "make -j" parallel builds
[ the old build system built the 20+ testcases serially, greatly
slowing down the build. ]
- dependency auto-generation via gcc's -MD feature, which allows the
'caching' of testcases.
- a 'test-all' testcase that contains all the other testcases and allows
a fast check for the typical 'all dependencies are installed' case.
The series also includes a number of fixes and convenience features:
- auto-detect the number of CPUs on the system and add the -j<cpus>
option automatically
- Fix 'make O=/some/dir/ perf.o' type of rules to work in the $(OUTPUT)
directory
- Improve documentation builds/install to not require feature-detection
- speed up linking, eliminate spurious rebuilds
Altogether build speed on my testsystem improved by a factor of 2.1x:
[BEFORE]
$ cd tools/perf
$ time make -j12 install
...
real 0m27.522s
user 1m59.478s
sys 0m9.714s
[AFTER]
$ cd tools/perf
$ time make install
...
real 0m12.658s
user 1m39.507s
sys 0m7.706s
Re-build speed, which is important to perf developers, increased 5.0x:
[BEFORE]
$ touch perf.c
$ time make
...
real 0m9.368s
user 0m8.166s
sys 0m1.015s
[AFTER]
$ touch perf.c
$ time make
BUILD: Doing 'make -j12' parallel build
CC perf.o
LINK perf
real 0m1.857s
user 0m1.522s
sys 0m0.295s
Empty build speed of 'make' in an already built tree - often done by
developers to see whether they are testing the latest binary, increased
more than 50-fold:
[BEFORE]
$ time make
...
real 0m9.105s
user 0m7.975s
sys 0m0.949s
[AFTER]
$ time make
$ time make
BUILD: Doing 'make -j12' parallel build
real 0m0.172s
user 0m0.119s
sys 0m0.029s
Thanks,
Ingo
=============>
Ingo Molnar (50):
tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT
tools/perf/build: Add feature check core code
tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically
tools/perf/build: Split out feature check: 'libnuma'
tools/perf/build: Split out feature check: 'stackprotector-all'
tools/perf/build: Split out feature check: 'stackprotector'
tools/perf/build: Split out feature check: 'volatile-register-var'
tools/perf/build: Split out feature check: 'fortify-source'
tools/perf/build: Split out feature check: 'bionic'
tools/perf/build: Clean up the libelf logic in config/Makefile
tools/perf/build: Split out feature check: 'libelf'
tools/perf/build: Split out feature check: 'glibc'
tools/perf/build: Split out feature check: 'dwarf'
tools/perf/build: Clean up the mmap logic in config/Makefile
tools/perf/build: Split out feature check: 'libelf-mmap'
tools/perf/build: Split out feature check: 'libelf-getphdrnum'
tools/perf/build: Clean up the libunwind logic in config/Makefile
tools/perf/build: Split out feature check: 'libunwind'
tools/perf/build: Split out feature check: 'libaudit'
tools/perf/build: Split out feature check: 'libslang'
tools/perf/build: Split out feature check: 'gtk2'
tools/perf/build: Split out feature check: 'gtk2-infobar'
tools/perf/build: Split out feature check: 'libperl'
tools/perf/build: Split out feature check: 'libpython'
tools/perf/build: Split out feature check: 'libpython-version'
tools/perf/build: Split out feature check: 'libbfd'
tools/perf/build: Split out feature check: 'on-exit'
tools/perf/build: Split out feature check: 'backtrace'
tools/perf: Clean up util/include/linux/compiler.h
tools/perf: Turn strlcpy() into a __weak function
tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target
tools/perf/build: Speed up git-version test on re-make
tools/perf/build: Speed up the final link
tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al
tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile
tools/perf/build: Speed up auto-detection
tools/perf/build: Improve printout-of auto-detected features
tools/perf/build: Automatically build in parallel, based on number of CPUs in the system
tools/perf/build: Flip Makefile.parallel and Makefile.perf
tools/perf/build: Standardize the various messages output by parallel make
tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle'
tools/perf/build: Remove unused config/feature-tests.mak
tools/perf/build: Clean up various testcases
tools/perf/build: Collapse the test-all.c testcase
tools/perf/build: Pass through all targets to Makefile.perf
tools/perf/build: Make sure autodep feature binaries honor the O= setting
tools/perf/build: Exclude MAKEFLAGS from nested invocation
tools/perf/build: Fix non-canonical directory names in O=
tools/perf/build: Fix O=/some/dir perf.o type of targets
tools/perf/build: Harmonize the style of the feature testcases
tools/perf/Documentation/Makefile | 19 +-
tools/perf/Makefile | 840 ++---------------------------
tools/perf/Makefile.perf | 855 ++++++++++++++++++++++++++++++
tools/perf/arch/x86/include/perf_regs.h | 6 +-
tools/perf/arch/x86/util/unwind.c | 4 +-
tools/perf/bench/mem-memcpy-arch.h | 2 +-
tools/perf/bench/mem-memcpy.c | 2 +-
tools/perf/bench/mem-memset-arch.h | 2 +-
tools/perf/bench/mem-memset.c | 2 +-
tools/perf/builtin-bench.c | 4 +-
tools/perf/builtin-inject.c | 2 +-
tools/perf/builtin-probe.c | 14 +-
tools/perf/builtin-record.c | 12 +-
tools/perf/config/Makefile | 326 +++++++-----
tools/perf/config/feature-checks/Makefile | 148 ++++++
tools/perf/config/feature-checks/test-all.c | 106 ++++
tools/perf/config/feature-checks/test-backtrace.c | 13 +
tools/perf/config/feature-checks/test-bionic.c | 6 +
tools/perf/config/feature-checks/test-cplus-demangle.c | 14 +
tools/perf/config/feature-checks/test-dwarf.c | 10 +
tools/perf/config/feature-checks/test-fortify-source.c | 6 +
tools/perf/config/feature-checks/test-glibc.c | 8 +
tools/perf/config/feature-checks/test-gtk2-infobar.c | 11 +
tools/perf/config/feature-checks/test-gtk2.c | 10 +
tools/perf/config/feature-checks/test-hello.c | 6 +
tools/perf/config/feature-checks/test-libaudit.c | 10 +
tools/perf/config/feature-checks/test-libbfd.c | 15 +
tools/perf/config/feature-checks/test-libelf-getphdrnum.c | 8 +
tools/perf/config/feature-checks/test-libelf-mmap.c | 8 +
tools/perf/config/feature-checks/test-libelf.c | 8 +
tools/perf/config/feature-checks/test-libnuma.c | 9 +
tools/perf/config/feature-checks/test-libperl.c | 9 +
tools/perf/config/feature-checks/test-libpython-version.c | 10 +
tools/perf/config/feature-checks/test-libpython.c | 8 +
tools/perf/config/feature-checks/test-libslang.c | 6 +
tools/perf/config/feature-checks/test-libunwind.c | 27 +
tools/perf/config/feature-checks/test-on-exit.c | 15 +
tools/perf/config/feature-checks/test-stackprotector-all.c | 6 +
tools/perf/config/feature-checks/test-stackprotector.c | 6 +
tools/perf/config/feature-checks/test-volatile-register-var.c | 6 +
tools/perf/config/feature-tests.mak | 246 ---------
tools/perf/config/utilities.mak | 14 -
tools/perf/perf.c | 4 +-
tools/perf/ui/gtk/browser.c | 2 +-
tools/perf/ui/gtk/gtk.h | 4 +-
tools/perf/ui/gtk/util.c | 4 +-
tools/perf/ui/ui.h | 4 +-
tools/perf/util/annotate.h | 4 +-
tools/perf/util/cache.h | 3 +-
tools/perf/util/generate-cmdlist.sh | 4 +-
tools/perf/util/hist.h | 4 +-
tools/perf/util/include/dwarf-regs.h | 2 +-
tools/perf/util/include/linux/compiler.h | 19 +-
tools/perf/util/map.c | 2 +-
tools/perf/util/path.c | 10 +-
tools/perf/util/perf_regs.h | 4 +-
tools/perf/util/probe-event.c | 4 +-
tools/perf/util/probe-finder.h | 4 +-
tools/perf/util/symbol-elf.c | 2 +-
tools/perf/util/symbol.h | 8 +-
tools/perf/util/unwind.h | 4 +-
tools/perf/util/util.c | 4 +-
62 files changed, 1678 insertions(+), 1257 deletions(-)
create mode 100644 tools/perf/Makefile.perf
create mode 100644 tools/perf/config/feature-checks/Makefile
create mode 100644 tools/perf/config/feature-checks/test-all.c
create mode 100644 tools/perf/config/feature-checks/test-backtrace.c
create mode 100644 tools/perf/config/feature-checks/test-bionic.c
create mode 100644 tools/perf/config/feature-checks/test-cplus-demangle.c
create mode 100644 tools/perf/config/feature-checks/test-dwarf.c
create mode 100644 tools/perf/config/feature-checks/test-fortify-source.c
create mode 100644 tools/perf/config/feature-checks/test-glibc.c
create mode 100644 tools/perf/config/feature-checks/test-gtk2-infobar.c
create mode 100644 tools/perf/config/feature-checks/test-gtk2.c
create mode 100644 tools/perf/config/feature-checks/test-hello.c
create mode 100644 tools/perf/config/feature-checks/test-libaudit.c
create mode 100644 tools/perf/config/feature-checks/test-libbfd.c
create mode 100644 tools/perf/config/feature-checks/test-libelf-getphdrnum.c
create mode 100644 tools/perf/config/feature-checks/test-libelf-mmap.c
create mode 100644 tools/perf/config/feature-checks/test-libelf.c
create mode 100644 tools/perf/config/feature-checks/test-libnuma.c
create mode 100644 tools/perf/config/feature-checks/test-libperl.c
create mode 100644 tools/perf/config/feature-checks/test-libpython-version.c
create mode 100644 tools/perf/config/feature-checks/test-libpython.c
create mode 100644 tools/perf/config/feature-checks/test-libslang.c
create mode 100644 tools/perf/config/feature-checks/test-libunwind.c
create mode 100644 tools/perf/config/feature-checks/test-on-exit.c
create mode 100644 tools/perf/config/feature-checks/test-stackprotector-all.c
create mode 100644 tools/perf/config/feature-checks/test-stackprotector.c
create mode 100644 tools/perf/config/feature-checks/test-volatile-register-var.c
delete mode 100644 tools/perf/config/feature-tests.mak
--
1.7.11.7
next reply other threads:[~2013-10-07 11:57 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-07 11:55 Ingo Molnar [this message]
2013-10-07 11:55 ` [PATCH 01/50] tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT Ingo Molnar
2013-10-07 11:55 ` [PATCH 02/50] tools/perf/build: Add feature check core code Ingo Molnar
2013-10-07 11:55 ` [PATCH 03/50] tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically Ingo Molnar
2013-10-07 11:55 ` [PATCH 04/50] tools/perf/build: Split out feature check: 'libnuma' Ingo Molnar
2013-10-07 11:55 ` [PATCH 05/50] tools/perf/build: Split out feature check: 'stackprotector-all' Ingo Molnar
2013-10-07 11:55 ` [PATCH 06/50] tools/perf/build: Split out feature check: 'stackprotector' Ingo Molnar
2013-10-07 11:56 ` [PATCH 07/50] tools/perf/build: Split out feature check: 'volatile-register-var' Ingo Molnar
2013-10-07 11:56 ` [PATCH 08/50] tools/perf/build: Split out feature check: 'fortify-source' Ingo Molnar
2013-10-07 11:56 ` [PATCH 09/50] tools/perf/build: Split out feature check: 'bionic' Ingo Molnar
2013-10-07 11:56 ` [PATCH 10/50] tools/perf/build: Clean up the libelf logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 11/50] tools/perf/build: Split out feature check: 'libelf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 12/50] tools/perf/build: Split out feature check: 'glibc' Ingo Molnar
2013-10-07 11:56 ` [PATCH 13/50] tools/perf/build: Split out feature check: 'dwarf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 14/50] tools/perf/build: Clean up the mmap logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 15/50] tools/perf/build: Split out feature check: 'libelf-mmap' Ingo Molnar
2013-10-07 11:56 ` [PATCH 16/50] tools/perf/build: Split out feature check: 'libelf-getphdrnum' Ingo Molnar
2013-10-07 11:56 ` [PATCH 17/50] tools/perf/build: Clean up the libunwind logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 18/50] tools/perf/build: Split out feature check: 'libunwind' Ingo Molnar
2013-10-07 11:56 ` [PATCH 19/50] tools/perf/build: Split out feature check: 'libaudit' Ingo Molnar
2013-10-07 21:00 ` Jiri Olsa
2013-10-08 8:19 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 20/50] tools/perf/build: Split out feature check: 'libslang' Ingo Molnar
2013-10-07 14:52 ` David Ahern
2013-10-08 8:06 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 21/50] tools/perf/build: Split out feature check: 'gtk2' Ingo Molnar
2013-10-07 11:56 ` [PATCH 22/50] tools/perf/build: Split out feature check: 'gtk2-infobar' Ingo Molnar
2013-10-07 11:56 ` [PATCH 23/50] tools/perf/build: Split out feature check: 'libperl' Ingo Molnar
2013-10-07 11:56 ` [PATCH 24/50] tools/perf/build: Split out feature check: 'libpython' Ingo Molnar
2013-10-07 11:56 ` [PATCH 25/50] tools/perf/build: Split out feature check: 'libpython-version' Ingo Molnar
2013-10-07 11:56 ` [PATCH 26/50] tools/perf/build: Split out feature check: 'libbfd' Ingo Molnar
2013-10-07 15:11 ` David Ahern
2013-10-07 21:18 ` Jiri Olsa
2013-10-08 8:15 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 27/50] tools/perf/build: Split out feature check: 'on-exit' Ingo Molnar
2013-10-07 11:56 ` [PATCH 28/50] tools/perf/build: Split out feature check: 'backtrace' Ingo Molnar
2013-10-07 11:56 ` [PATCH 29/50] tools/perf: Clean up util/include/linux/compiler.h Ingo Molnar
2013-10-07 11:56 ` [PATCH 30/50] tools/perf: Turn strlcpy() into a __weak function Ingo Molnar
2013-10-07 11:56 ` [PATCH 31/50] tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target Ingo Molnar
2013-10-07 21:27 ` Jiri Olsa
2013-10-08 8:23 ` Ingo Molnar
2013-10-07 21:59 ` Jiri Olsa
2013-10-07 11:56 ` [PATCH 32/50] tools/perf/build: Speed up git-version test on re-make Ingo Molnar
2013-10-07 11:56 ` [PATCH 33/50] tools/perf/build: Speed up the final link Ingo Molnar
2013-10-07 11:56 ` [PATCH 34/50] tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al Ingo Molnar
2013-10-07 11:56 ` [PATCH 35/50] tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 36/50] tools/perf/build: Speed up auto-detection Ingo Molnar
2013-10-07 21:57 ` Jiri Olsa
2013-10-08 8:36 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 37/50] tools/perf/build: Improve printout-of auto-detected features Ingo Molnar
2013-10-07 22:12 ` Jiri Olsa
2013-10-08 8:46 ` Ingo Molnar
2013-10-08 15:51 ` Jiri Olsa
2013-10-09 6:42 ` [PATCH 53/52] tools/perf/build: Clean up feature_print_code() Ingo Molnar
2013-10-09 8:38 ` Jiri Olsa
2013-10-15 5:20 ` [tip:perf/core] " tip-bot for Jiri Olsa
2013-10-07 11:56 ` [PATCH 38/50] tools/perf/build: Automatically build in parallel, based on number of CPUs in the system Ingo Molnar
2013-10-07 11:56 ` [PATCH 39/50] tools/perf/build: Flip Makefile.parallel and Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 40/50] tools/perf/build: Standardize the various messages output by parallel make Ingo Molnar
2013-10-07 11:56 ` [PATCH 41/50] tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle' Ingo Molnar
2013-10-07 11:56 ` [PATCH 42/50] tools/perf/build: Remove unused config/feature-tests.mak Ingo Molnar
2013-10-07 11:56 ` [PATCH 43/50] tools/perf/build: Clean up various testcases Ingo Molnar
2013-10-07 11:56 ` [PATCH 44/50] tools/perf/build: Collapse the test-all.c testcase Ingo Molnar
2013-10-07 22:25 ` Jiri Olsa
2013-10-08 8:47 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 45/50] tools/perf/build: Pass through all targets to Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 46/50] tools/perf/build: Make sure autodep feature binaries honor the O= setting Ingo Molnar
2013-10-07 11:56 ` [PATCH 47/50] tools/perf/build: Exclude MAKEFLAGS from nested invocation Ingo Molnar
2013-10-07 11:56 ` [PATCH 48/50] tools/perf/build: Fix non-canonical directory names in O= Ingo Molnar
2013-10-07 11:56 ` [PATCH 49/50] tools/perf/build: Fix O=/some/dir perf.o type of targets Ingo Molnar
2013-10-07 22:33 ` Jiri Olsa
2013-10-08 8:49 ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 50/50] tools/perf/build: Harmonize the style of the feature testcases Ingo Molnar
2013-10-07 13:52 ` [RFC PATCH 00/50] tools/perf: Speed up the build system David Ahern
2013-10-07 14:11 ` Ingo Molnar
2013-10-07 15:35 ` David Ahern
2013-10-07 15:51 ` [PATCH] tools/perf/build: Pass through LDFLAGS to feature tests Ingo Molnar
2013-10-07 16:07 ` David Ahern
2013-10-15 5:20 ` [tip:perf/core] " tip-bot for Ingo Molnar
2013-10-07 16:41 ` [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-08 1:41 ` David Ahern
2013-10-08 12:48 ` Ingo Molnar
2013-10-07 20:15 ` Andi Kleen
2013-10-08 9:02 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1381147003-2574-1-git-send-email-mingo@kernel.org \
--to=mingo@kernel.org \
--cc=a.p.zijlstra@chello.nl \
--cc=acme@redhat.com \
--cc=dsahern@gmail.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=namhyung@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.