[RFC/PATCH] improve interaction with ccache

[RFC/PATCH] improve interaction with ccache

[Kai Germaschewski]

As various people pointed out, ccache is a great win for people compiling 
a lot of kernels. (For info on ccache, see ccache.samba.org)

Unfortunately, when a kernel tree is moved around / cloned / symlinked, 
the compiler cache doesn't have many hits, since some header files use 
BUG(), which in turn uses __FILE__, which currently includes the absolute 
path to the header file.

The appended patch solves this problem by including the headers a relative 
path which will not change when the tree is moved.

I'd like to get comments if
o the patch breaks anything for someone and
o it's considered useful enough for inclusion

--Kai


# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.553   -> 1.554  
#	          Rules.make	1.21    -> 1.22   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/05/19	kai@tp1.ruhr-uni-bochum.de	1.554
# kbuild: Improve interaction with ccache
# 
# Use relativ paths on the command line, thus __FILE__
# in an include file looks like "../../include/linux/foo.h" instead of
# "/home/kai/kernel/v2.5/linux-2.5.make-work/include/linux/foo.h".
# 
# The latter is unnecessarily long and in particular changes when
# you move your source tree around, thus causing cache misses with
# ccache.
# --------------------------------------------
#
diff -Nru a/Rules.make b/Rules.make
--- a/Rules.make	Sun May 19 18:50:41 2002
+++ b/Rules.make	Sun May 19 18:50:41 2002
@@ -49,13 +49,28 @@
 MOD_SUB_DIRS	:= $(sort $(subdir-m) $(both-m))
 ALL_SUB_DIRS	:= $(sort $(subdir-y) $(subdir-m) $(subdir-n) $(subdir-))
 
+empty :=
+space := $(empty) $(empty)
+
+ifeq ($(TOPDIR),$(CURDIR))
+  RELDIR := ./
+  TOPDIR_REL := ./
+else
+  RELDIR := $(subst $(TOPDIR)/,,$(CURDIR))
+  TOPDIR_REL := $(subst $(space),,$(foreach d,$(subst /, ,$(RELDIR)),../))
+endif
+
 #
 # Common rules
 #
 
 # export_flags will be set to -DEXPORT_SYMBOL for objects in $(export-objs)
 
-c_flags = $(CFLAGS) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) $(export_flags)
+c_flags = $(subst $(TOPDIR)/,$(TOPDIR_REL), \
+	          $(CFLAGS) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) \
+	  	  -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) \
+	  	  $(export_flags) \
+	   )
 
 cmd_cc_s_c = $(CC) $(c_flags) -S $< -o $@
 
@@ -78,7 +93,9 @@
 # the Makefiles to these standard rules.  -- rmk, mec
 ifdef USE_STANDARD_AS_RULE
 
-a_flags = $(AFLAGS) $(EXTRA_AFLAGS) $(AFLAGS_$(*F).o)
+a_flags = $(subst $(TOPDIR)/,$(TOPDIR_REL), \
+	          $(AFLAGS) $(EXTRA_AFLAGS) $(AFLAGS_$(*F).o) \
+	   )
 
 cmd_as_s_S = $(CPP) $(a_flags) $< > $@
 
@@ -375,5 +392,5 @@
 if_changed = $(if $(strip $? \
 		          $(filter-out $($(1)),$(cmd_$@))\
 			  $(filter-out $(cmd_$@),$($(1)))),\
-	       @echo $($(1)); $($(1)); echo 'cmd_$@ := $($(1))' > .$@.cmd)
+	       @echo '$($(1))'; $($(1)); echo 'cmd_$@ := $($(1))' > .$@.cmd)
 

Re: [RFC/PATCH] improve interaction with ccache

[Keith Owens]

On Sun, 19 May 2002 19:11:41 -0500 (CDT), 
Kai Germaschewski <kai-germaschewski@uiowa.edu> wrote:
>As various people pointed out, ccache is a great win for people compiling 
>a lot of kernels. (For info on ccache, see ccache.samba.org)
>
>Unfortunately, when a kernel tree is moved around / cloned / symlinked, 
>the compiler cache doesn't have many hits, since some header files use 
>BUG(), which in turn uses __FILE__, which currently includes the absolute 
>path to the header file.
>
>The appended patch solves this problem by including the headers a relative 
>path which will not change when the tree is moved.

You are fixing the symptom, not the cause.  The symptom is too many
compiles, people are using ccache to attempt to fix the symptom.  The
cause is a kernel build system that forces people to make clean or
mrproper between builds instead of reusing existing objects.

Fix the cause, not the symptom.

You will find that relative include paths completely stuff up
builds with separate source and object trees.  It will also mess up
people who compile add on code outside the kernel tree, CURDIR is not
related to TOPDIR.

All of these problems are fixed in kbuild 2.5, correctly.

Re: [RFC/PATCH] improve interaction with ccache

[Albert D. Cahalan]

Keith Owens writes:
> Kai Germaschewski <kai-germaschewski@uiowa.edu> wrote:

>> As various people pointed out, ccache is a great win for people compiling 
>> a lot of kernels. (For info on ccache, see ccache.samba.org)
...
> You are fixing the symptom, not the cause.  The symptom is too many
> compiles, people are using ccache to attempt to fix the symptom.  The
> cause is a kernel build system that forces people to make clean or
> mrproper between builds instead of reusing existing objects.
>
> Fix the cause, not the symptom.

Cause: gcc is slow
Symptom: builds are slow
Fix: make gcc fast

That fix won't happen, so we cache the results.
We have two ways to do this:

a. use "make", relying solely on timestamps
b. use "ccache", which uses an md5 checksum AFAIK

With ccache, one could even get rid of make.
It's redundant; just use a shell script. :-)

Re: [RFC/PATCH] improve interaction with ccache

[Kai Germaschewski]

On Mon, 20 May 2002, Keith Owens wrote:

> You are fixing the symptom, not the cause.  The symptom is too many
> compiles, people are using ccache to attempt to fix the symptom.  The
> cause is a kernel build system that forces people to make clean or
> mrproper between builds instead of reusing existing objects.

Well, I basically never do make clean or make mrproper (unless I'm playing 
with the build system itself, there it's of course necessary for testing). 

However, I do have a lot of clones of the bk trees around, used to work on 
different patches. And of course only few files differ between these 
trees, so using ccache is a big win when doing compiles in the various 
trees.

> You will find that relative include paths completely stuff up
> builds with separate source and object trees.  It will also mess up
> people who compile add on code outside the kernel tree, CURDIR is not
> related to TOPDIR.

Well, the current kbuild doesn't do separate source/object, so that's not 
an issue. If people compile out of tree things using Rules.make, you're 
right, I need to make sure to stick to the absolute path there.

--Kai