linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ingo Molnar <mingo@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Namhyung Kim <namhyung@kernel.org>,
	David Ahern <dsahern@gmail.com>, Jiri Olsa <jolsa@redhat.com>
Subject: [PATCH 34/50] tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al
Date: Mon,  7 Oct 2013 13:56:27 +0200	[thread overview]
Message-ID: <1381147003-2574-35-git-send-email-mingo@kernel.org> (raw)
In-Reply-To: <1381147003-2574-1-git-send-email-mingo@kernel.org>

Linus reported the following perf build system bug:

  'Another annoyance during that make was that "make install" seems to
   want to re-make the thing I just built. That's absolutely horrible, [...]'

The thing that got re-built were 'only' the (numerous) feature checks,
not the whole project - but still it was mighty annoying as the feature
checks took 9+ seconds even on reasonably fast boxes.

Even with the autodep patches where feature detection is much faster
it wastes resources, wastes screen real estate and confuses users if
we execute feature detection twice.

There were two sources for these unnecessary re-builds of the feature
checks:

 - Unnecessary nested invocations of $(MAKE), apparently to be able
   to do conditional compilation dependent on documentation tools
   presence. Use straight dependencies instead, with no nesting.

 - A direct invocation of $(MAKE) to rebuild the PERF-VERSION-FILE.
   This is apparently done to be able to include it into the
   Makefile:

    -include $(OUTPUT)PERF-VERSION-FILE

   but that's entirely pointless for two reasons: 1) the version file
   gets regenerated by the initial build pass anyway, 2) including it
   is futile, given its contents:

    #define PERF_VERSION "3.12.rc3.g8510c7"

   'make' will interpret that as a comment line...

   So just remove this part of the doc-generation logic.

With these things fixed a 'make install' now rebuilds only what is needed.

A repeated 'make install' on an already built tree is super fast now,
it finishes in under 0.3 seconds:

  #
  #  After the patch:
  #

  $ time make install

  ...

  real    0m0.280s
  user    0m0.162s
  sys     0m0.054s

Prior all the autodep changes and prior this fix, a repeat 'make install'
took 24.1 seconds (!) on the same system:

  #
  #  Before the patches:
  #

  $ time make install

  ...

  real    0m24.109s
  user    0m21.171s
  sys     0m2.449s

Which almost entirely was caused by fixable build system fat.
We are now literally ~86 times faster.

A fresh rebuild and install now takes just 11.4 seconds:

  #
  #  After the patch:
  #

  $ make clean
  $ time make -j16 install

  ...

  real    0m11.457s
  user    1m43.411s
  sys     0m7.610s

Without the patches it took 27.8 seconds:

  #
  #  Before the patches:
  #

  $ make clean
  $ time make -j16 install

  ...

  real    0m27.801s
  user    1m59.242s
  sys     0m9.749s

So even in the complete rebuild case we are now ~2.5 times faster.

Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/n/tip-x4qjnxjGrgxpribq8sdakfTp@git.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 tools/perf/Documentation/Makefile | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)

diff --git a/tools/perf/Documentation/Makefile b/tools/perf/Documentation/Makefile
index 5a37a7c..eaa477f 100644
--- a/tools/perf/Documentation/Makefile
+++ b/tools/perf/Documentation/Makefile
@@ -192,13 +192,14 @@ do-install-man: man
 
 install-man: check-man-tools man
 
-try-install-man:
 ifdef missing_tools
-	$(warning Please install $(missing_tools) to have the man pages installed)
+  DO_INSTALL_MAN = $(warning Please install $(missing_tools) to have the man pages installed)
 else
-	$(MAKE) do-install-man
+  DO_INSTALL_MAN = do-install-man
 endif
 
+try-install-man: $(DO_INSTALL_MAN)
+
 install-info: info
 	$(INSTALL) -d -m 755 $(DESTDIR)$(infodir)
 	$(INSTALL) -m 644 $(OUTPUT)perf.info $(OUTPUT)perfman.info $(DESTDIR)$(infodir)
@@ -216,14 +217,6 @@ install-pdf: pdf
 #install-html: html
 #	'$(SHELL_PATH_SQ)' ./install-webdoc.sh $(DESTDIR)$(htmldir)
 
-ifneq ($(MAKECMDGOALS),clean)
-ifneq ($(MAKECMDGOALS),tags)
-$(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
-	$(QUIET_SUBDIR0)../ $(QUIET_SUBDIR1) $(OUTPUT)PERF-VERSION-FILE
-
--include $(OUTPUT)PERF-VERSION-FILE
-endif
-endif
 
 #
 # Determine "include::" file references in asciidoc files.
@@ -342,5 +335,3 @@ $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
 
 #quick-install-html:
 #	'$(SHELL_PATH_SQ)' ./install-doc-quick.sh $(HTML_REF) $(DESTDIR)$(htmldir)
-
-.PHONY: .FORCE-PERF-VERSION-FILE
-- 
1.7.11.7


  parent reply	other threads:[~2013-10-07 12:02 UTC|newest]

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-10-07 11:55 [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-07 11:55 ` [PATCH 01/50] tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT Ingo Molnar
2013-10-07 11:55 ` [PATCH 02/50] tools/perf/build: Add feature check core code Ingo Molnar
2013-10-07 11:55 ` [PATCH 03/50] tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically Ingo Molnar
2013-10-07 11:55 ` [PATCH 04/50] tools/perf/build: Split out feature check: 'libnuma' Ingo Molnar
2013-10-07 11:55 ` [PATCH 05/50] tools/perf/build: Split out feature check: 'stackprotector-all' Ingo Molnar
2013-10-07 11:55 ` [PATCH 06/50] tools/perf/build: Split out feature check: 'stackprotector' Ingo Molnar
2013-10-07 11:56 ` [PATCH 07/50] tools/perf/build: Split out feature check: 'volatile-register-var' Ingo Molnar
2013-10-07 11:56 ` [PATCH 08/50] tools/perf/build: Split out feature check: 'fortify-source' Ingo Molnar
2013-10-07 11:56 ` [PATCH 09/50] tools/perf/build: Split out feature check: 'bionic' Ingo Molnar
2013-10-07 11:56 ` [PATCH 10/50] tools/perf/build: Clean up the libelf logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 11/50] tools/perf/build: Split out feature check: 'libelf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 12/50] tools/perf/build: Split out feature check: 'glibc' Ingo Molnar
2013-10-07 11:56 ` [PATCH 13/50] tools/perf/build: Split out feature check: 'dwarf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 14/50] tools/perf/build: Clean up the mmap logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 15/50] tools/perf/build: Split out feature check: 'libelf-mmap' Ingo Molnar
2013-10-07 11:56 ` [PATCH 16/50] tools/perf/build: Split out feature check: 'libelf-getphdrnum' Ingo Molnar
2013-10-07 11:56 ` [PATCH 17/50] tools/perf/build: Clean up the libunwind logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 18/50] tools/perf/build: Split out feature check: 'libunwind' Ingo Molnar
2013-10-07 11:56 ` [PATCH 19/50] tools/perf/build: Split out feature check: 'libaudit' Ingo Molnar
2013-10-07 21:00   ` Jiri Olsa
2013-10-08  8:19     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 20/50] tools/perf/build: Split out feature check: 'libslang' Ingo Molnar
2013-10-07 14:52   ` David Ahern
2013-10-08  8:06     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 21/50] tools/perf/build: Split out feature check: 'gtk2' Ingo Molnar
2013-10-07 11:56 ` [PATCH 22/50] tools/perf/build: Split out feature check: 'gtk2-infobar' Ingo Molnar
2013-10-07 11:56 ` [PATCH 23/50] tools/perf/build: Split out feature check: 'libperl' Ingo Molnar
2013-10-07 11:56 ` [PATCH 24/50] tools/perf/build: Split out feature check: 'libpython' Ingo Molnar
2013-10-07 11:56 ` [PATCH 25/50] tools/perf/build: Split out feature check: 'libpython-version' Ingo Molnar
2013-10-07 11:56 ` [PATCH 26/50] tools/perf/build: Split out feature check: 'libbfd' Ingo Molnar
2013-10-07 15:11   ` David Ahern
2013-10-07 21:18     ` Jiri Olsa
2013-10-08  8:15       ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 27/50] tools/perf/build: Split out feature check: 'on-exit' Ingo Molnar
2013-10-07 11:56 ` [PATCH 28/50] tools/perf/build: Split out feature check: 'backtrace' Ingo Molnar
2013-10-07 11:56 ` [PATCH 29/50] tools/perf: Clean up util/include/linux/compiler.h Ingo Molnar
2013-10-07 11:56 ` [PATCH 30/50] tools/perf: Turn strlcpy() into a __weak function Ingo Molnar
2013-10-07 11:56 ` [PATCH 31/50] tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target Ingo Molnar
2013-10-07 21:27   ` Jiri Olsa
2013-10-08  8:23     ` Ingo Molnar
2013-10-07 21:59   ` Jiri Olsa
2013-10-07 11:56 ` [PATCH 32/50] tools/perf/build: Speed up git-version test on re-make Ingo Molnar
2013-10-07 11:56 ` [PATCH 33/50] tools/perf/build: Speed up the final link Ingo Molnar
2013-10-07 11:56 ` Ingo Molnar [this message]
2013-10-07 11:56 ` [PATCH 35/50] tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 36/50] tools/perf/build: Speed up auto-detection Ingo Molnar
2013-10-07 21:57   ` Jiri Olsa
2013-10-08  8:36     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 37/50] tools/perf/build: Improve printout-of auto-detected features Ingo Molnar
2013-10-07 22:12   ` Jiri Olsa
2013-10-08  8:46     ` Ingo Molnar
2013-10-08 15:51       ` Jiri Olsa
2013-10-09  6:42         ` [PATCH 53/52] tools/perf/build: Clean up feature_print_code() Ingo Molnar
2013-10-09  8:38           ` Jiri Olsa
2013-10-15  5:20         ` [tip:perf/core] " tip-bot for Jiri Olsa
2013-10-07 11:56 ` [PATCH 38/50] tools/perf/build: Automatically build in parallel, based on number of CPUs in the system Ingo Molnar
2013-10-07 11:56 ` [PATCH 39/50] tools/perf/build: Flip Makefile.parallel and Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 40/50] tools/perf/build: Standardize the various messages output by parallel make Ingo Molnar
2013-10-07 11:56 ` [PATCH 41/50] tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle' Ingo Molnar
2013-10-07 11:56 ` [PATCH 42/50] tools/perf/build: Remove unused config/feature-tests.mak Ingo Molnar
2013-10-07 11:56 ` [PATCH 43/50] tools/perf/build: Clean up various testcases Ingo Molnar
2013-10-07 11:56 ` [PATCH 44/50] tools/perf/build: Collapse the test-all.c testcase Ingo Molnar
2013-10-07 22:25   ` Jiri Olsa
2013-10-08  8:47     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 45/50] tools/perf/build: Pass through all targets to Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 46/50] tools/perf/build: Make sure autodep feature binaries honor the O= setting Ingo Molnar
2013-10-07 11:56 ` [PATCH 47/50] tools/perf/build: Exclude MAKEFLAGS from nested invocation Ingo Molnar
2013-10-07 11:56 ` [PATCH 48/50] tools/perf/build: Fix non-canonical directory names in O= Ingo Molnar
2013-10-07 11:56 ` [PATCH 49/50] tools/perf/build: Fix O=/some/dir perf.o type of targets Ingo Molnar
2013-10-07 22:33   ` Jiri Olsa
2013-10-08  8:49     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 50/50] tools/perf/build: Harmonize the style of the feature testcases Ingo Molnar
2013-10-07 13:52 ` [RFC PATCH 00/50] tools/perf: Speed up the build system David Ahern
2013-10-07 14:11   ` Ingo Molnar
2013-10-07 15:35     ` David Ahern
2013-10-07 15:51       ` [PATCH] tools/perf/build: Pass through LDFLAGS to feature tests Ingo Molnar
2013-10-07 16:07         ` David Ahern
2013-10-15  5:20         ` [tip:perf/core] " tip-bot for Ingo Molnar
2013-10-07 16:41       ` [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-08  1:41         ` David Ahern
2013-10-08 12:48           ` Ingo Molnar
2013-10-07 20:15 ` Andi Kleen
2013-10-08  9:02   ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1381147003-2574-35-git-send-email-mingo@kernel.org \
    --to=mingo@kernel.org \
    --cc=a.p.zijlstra@chello.nl \
    --cc=acme@redhat.com \
    --cc=dsahern@gmail.com \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=namhyung@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).