public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* KBuild 2.5 Impressions
@ 2002-05-30  9:45 Daniel Phillips
  2002-05-30 12:50 ` Martin Dalecki
                   ` (3 more replies)
  0 siblings, 4 replies; 48+ messages in thread
From: Daniel Phillips @ 2002-05-30  9:45 UTC (permalink / raw)
  To: Linus Torvalds, linux-kernel; +Cc: Keith Owens

I wanted to know how well kbuild 2.5 really works, so I got the patches
from kbuild.sourceforge.net and gave them a test drive, comparing to
old kbuild.  First I did an out-of-the-box build, then I did 'touch
fs/ext2/inode.c' and rebuilt.  Results are as follows:

                         old kbuild       kbuild 2.5     Speedup

   First time build:     7 min 8 sec     5 min 55 sec     17%
   Incremental build:     25.94 sec       15.31 sec       41% :-)

The test system is a 2 x 1 GHz server with scsi raid.  Note that I only
used one processor for the tests, as I normally do: in my experience,
with old kbuild I spend more time picking up the broken pices than
I save in compile time.

For good measure, I did a make mrproper and rebuild with -j3.  That
completed in 3 minutes, 25 seconds.  Very impressive, and what's more,
running that make a second time completes in 12 seconds, as opposed to
old make, which tends to erroneously rebuild a large number of files,
and sometimes does not complete the job correctly.

I have only one point on the curve, but it confirms what Keith has
been saying.  The incremental build speed is especially important to
me.  If I were working all the time on 2.5 - sadly, I'm not - I would
without question be using kbuild 2.5 simply by reason of the fast
incremental builds.

Miscellaneous Notes
-------------------

Along the way, old kbuild did the usual wrong things:

  - In the incremental build, 6 files rebuilt that should not have been

  - Once, when I interrupted the make dep, subsequent make deps would
    no longer work, forcing me to do make mrproper and start again.

  - Way too much output to the screen

I took a quick look into some of the kbuild 2.5 Makefile.in files, just
to get a feeling for what they look like, and they look fine to me.  For
example, fs/ext2 has acquired a Makefile.in file, with contents that
seem easily understandible.  The syntax of these files is explained in
the documentation, which is provided by the patch set, in:

   Documentation/kbuild/kbuild-2.5.txt

Reading through it, I found out about the NO_MAKEFILE_GEN, an option
that promises to make a fast build system even faster for interative
development, the type I always do.  There is also a lot of informative
design documentation.  Amazingly, there is even postscript
documentation.  Full marks for documentation here.

Note to Keith: this file needs to be in some obvious place on the
sourceforge site.

Both the old and new build systems failed the rename test: rename the
root of the source tree and it will no longer build.  The kbuild 2.5
documentation explains:

  Unlike kbuild 2.4, renaming a source or object tree does not force a
  complete rebuild.  After renaming any tree and updating the KBUILD_SRCTREE
  and KBUILD_OBJTREE variables, just run make.  Unless you have changed any
  files, nothing should be rebuilt, however the make step must be run to save
  the new tree names ready for the install step.

but this did not work for me.  However, I do not doubt that it will
soon.

Reading through the patches, I noticed there is some useful looking
help text added for the makefile operations, but it wasn't immediately
obvious how to get it.  Reading the patch is, of course, one way.

There is no Python anywhere to be seen in kbuild 2.5, for those who
worry about that.  It is coded in C, about 10,000 lines it seems.
It has a simple built in database which I suppose accounts for some
of that.  For what it does, it seems quite reasonable.  There is a
bison-generated grammar, presumeably to parse the Makefile.in
language.  This is the proper way do do things - much more reliable
and maintainable than a hand-coded parser, not to mention more
efficient.  Most of the rest of the patch consists of Makefile.in
files.  I can really see why this is a big job for Keith to maintain
by himself: the maintainers really need to be involved.  I doubt that
it will be hard, and by appearances, there is no need for any
particular arch to step up to kbuild 2.5 immediately, since the old
build system still works.  In any event, the 1386-specific part of
the patch is only 700 or so lines long.

The output that kbuild 2.5 generates is a vast improvment over old
make.  It's enough to see the progress of the make, while not
obscuring the warnings.  This by itself should help in cleaning up
the tree.

Note to anyone who wants to try this themselves: with the kbuild 2.5
patches applied, nothing changes (and the old build system is used)
unless you add '-f Makefile-2.5' to the make command.  It does not
appear to be necessary to supply a bzImage target, and in fact,
Makefile-2.5 doesn't recognize it.  That's basically all you have to
know.

Detailed Timings
----------------

Old kbuild, out of the box build:

424.18user 29.65system 7:07.94elapsed 106%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1552796major+1856222minor)pagefaults 0swaps

touch fs/ext2/inode.c incremental build:

24.08user 2.04system 0:25.94elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (114294major+80954minor)pagefaults 0swaps

kbuild 2.5, out of the box build:

341.29user 18.71system 5:54.87elapsed 101%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1080599major+1427293minor)pagefaults 0swaps

touch fs/ext2/inode.c incremental build:

13.75user 1.76system 0:15.31elapsed 101%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (122776major+59338minor)pagefaults 0swaps

-- 
Daniel

^ permalink raw reply	[flat|nested] 48+ messages in thread
* Re: KBuild 2.5 Impressions
@ 2002-06-03  2:49 Dan Kegel
  2002-06-03  3:28 ` Linus Torvalds
  0 siblings, 1 reply; 48+ messages in thread
From: Dan Kegel @ 2002-06-03  2:49 UTC (permalink / raw)
  To: linux-kernel@vger.kernel.org, Daniel Phillips

Daniel Phillips wrote:
> Well, actually a lot of the work done by Kai is simply importing
> portions of Keith's work that break out easily, which is purely
> duplication of effort, since such work is already in progress.  In
> fact it creates more work, because then we have to go parse Kai's
> patches and find out what he submitted, then see if it gets applied
> so we can mark it 'applied' in the list.  This is a real waste of
> time, and did I mention, it's divisive?

Linus sees Kai as being the most promising fellow to
integrate kbuild2.5 right now (see
http://marc.theaimsgroup.com/?l=linux-kernel&m=102307114005894&w=2 )
and Kai is willing to take it on in just the way Linus wants.
It's probably worth giving Kai and Linus the benefit of the doubt for a while,
even if it does mean having to rejigger the kbuild-2.5 patch
each time Linux accepts one of Kai's patches.

I personally am anxious to see kbuild-2.5 make it into the kernel,
but I also feel it can only benefit from a strong review of
the sort that comes about during gradual evolution
of the kernel build process towards the techniques used by kbuild-2.5.

Thanks to everyone, Keith, Daniel, Thunder, and Kai, who are
working (together or not!) on moving the kernel build process
into the modern era.
- Dan

^ permalink raw reply	[flat|nested] 48+ messages in thread

end of thread, other threads:[~2002-06-03  7:43 UTC | newest]

Thread overview: 48+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2002-05-30  9:45 KBuild 2.5 Impressions Daniel Phillips
2002-05-30 12:50 ` Martin Dalecki
2002-05-30 14:08   ` Kenneth Johansson
2002-05-30 13:28 ` Paul P Komkoff Jr
2002-05-30 21:55 ` Ion Badulescu
2002-05-30 22:29   ` Thunder from the hill
2002-05-30 23:19   ` Daniel Phillips
2002-05-31  0:09     ` David Lang
2002-05-31  0:29       ` Daniel Phillips
2002-05-31  0:32       ` Larry McVoy
2002-06-03  7:21         ` Rusty Russell
2002-05-31  0:13     ` Kenneth Johansson
2002-05-31  0:47       ` Daniel Phillips
2002-05-31  1:20         ` Skip Ford
2002-05-31  1:24           ` Daniel Phillips
2002-05-31 13:48             ` Tomas Szepe
2002-05-31 14:28               ` Nicolas Pitre
2002-05-31 15:21                 ` Thunder from the hill
     [not found]           ` <Pine.LNX.4.44.0205302129120.29405-100000@hawkeye.luckynet.adm>
2002-05-31  4:01             ` Skip Ford
2002-05-31  4:46               ` KBuild 2.5 Migration Daniel Phillips
2002-05-31  5:12                 ` Skip Ford
2002-05-31  0:48       ` KBuild 2.5 Impressions Nicolas Pitre
2002-05-31  3:29       ` Hua Zhong
2002-05-31  3:39         ` Daniel Phillips
2002-05-31  3:55         ` Thunder from the hill
2002-06-02  4:03     ` Ion Badulescu
2002-06-02  6:51       ` Daniel Phillips
2002-06-02  7:58       ` Thunder from the hill
2002-06-02 14:03         ` Sam Ravnborg
2002-06-02 14:38           ` Daniel Phillips
2002-06-02 14:56             ` Sam Ravnborg
2002-06-02 15:05               ` Thunder from the hill
2002-06-02 15:16               ` Daniel Phillips
2002-06-02 18:26                 ` Sam Ravnborg
2002-06-02 18:39                   ` Thunder from the hill
2002-06-02 18:57                     ` Ion Badulescu
2002-06-02 19:15                       ` Thunder from the hill
2002-06-03  2:04                 ` Kai Germaschewski
2002-06-02 15:47           ` Thunder from the hill
2002-06-02 11:21 ` Peter Osterlund
2002-06-02 11:37   ` Thunder from the hill
2002-06-02 11:53     ` Peter Osterlund
2002-06-02 12:03       ` Thunder from the hill
2002-06-02 12:09         ` Thunder from the hill
2002-06-02 12:51         ` Peter Osterlund
2002-06-02 14:00           ` Daniel Phillips
  -- strict thread matches above, loose matches on Subject: below --
2002-06-03  2:49 Dan Kegel
2002-06-03  3:28 ` Linus Torvalds

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox