[PATCH] Avoid recompilation of unrelated files

[PATCH] Avoid recompilation of unrelated files

[Bean]

[-- Attachment #1: Type: text/plain, Size: 1395 bytes --]

A problem of the grub2 make system is that even the smallest change of source
code can trigger a huge recompilation. Try this:

touch commands/cmp.c
make

The ideal situation is that only files related to cmp module is recompiled. 
But unfortunately, this is not the case. The problem lies in file dependence.

cmp.mod: pre-cmp.o mod-cmp.o

pre-cmp.o: cmp_mod-commands_cmp.o

cmp_mod-commands_cmp.o: cmp.c

mod-cmp.o: mod-cmp.c

mod-cmp.c:  moddep.lst genmodsrc.sh

def-cmp.lst: pre-cmp.o

und-cmp.lst: pre-cmp.o

moddep.lst: def-cmp.lst und-cmp.lst

cmp.c => cmp_mod-commands_cmp.o => pre-cmp.o => def-cmp.lst and und-cmp.lst
=> moddep.lst

When cmp.c changes, moddep.lst changes. But every mod-*.c depends on 
moddep.lst, they all need to be recreated. This will lead to a huge
recompilation process.

def-cmp.lst and und-cmp.lst contains the exported symbol and unresolved symbol
of the module. Their content rarely change. So we can update them only when 
it's necessary, not every time we recompile the module.

The solution:

Merge these three rules:

pre-cmp.o: cmp_mod-commands_cmp.o

def-cmp.lst: pre-cmp.o

und-cmp.lst: pre-cmp.o

The output of def-cmp.lst is saved as tmp_pre-cmp.lst. When tmp_pre-cmp.lst
and the old pre-cmp.lst is different, move tmp_pre-cmp.lst to pre-cmp.lst, 
otherwise, just delete tmp_pre-cmp.lst. und-cmp.lst can be handled in exactly
the same way.

-- 
Bean

[-- Attachment #2: grub2-genmk.diff --]
[-- Type: text/plain, Size: 1497 bytes --]

	* genmk.rb: Avoid recompilation of unrelated files.


Index: genmk.rb
===================================================================
RCS file: /sources/grub/grub2/genmk.rb,v
retrieving revision 1.27
diff -u -r1.27 genmk.rb
--- genmk.rb	22 Sep 2006 00:27:38 -0000	1.27
+++ genmk.rb	26 Jun 2007 18:06:28 -0000
@@ -120,6 +120,13 @@
 #{pre_obj}: $(#{prefix}_DEPENDENCIES) #{objs_str}
 	-rm -f $@
 	$(TARGET_CC) $(#{prefix}_LDFLAGS) $(TARGET_LDFLAGS) -Wl,-r,-d -o $@ #{objs_str}
+ifneq ($(#{prefix}_EXPORTS),no)
+	$(NM) -g --defined-only -P -p $< | sed 's/^\\([^ ]*\\).*/\\1 #{mod_name}/' > tmp_#{defsym}
+	if cmp -s tmp_#{defsym} #{defsym} ; then rm tmp_#{defsym} ; else mv tmp_#{defsym} #{defsym} ; fi 
+endif
+	echo '#{mod_name}' > tmp_#{undsym}
+	$(NM) -u -P -p $< | cut -f1 -d' ' >> tmp_#{undsym}
+	if cmp -s tmp_#{undsym} #{undsym} ; then rm tmp_#{undsym} ; else mv tmp_#{undsym} #{undsym} ; fi
 
 #{mod_obj}: #{mod_src}
 	$(TARGET_CC) $(TARGET_CPPFLAGS) $(TARGET_CFLAGS) $(#{prefix}_CFLAGS) -c -o $@ $<
@@ -127,15 +134,6 @@
 #{mod_src}: moddep.lst genmodsrc.sh
 	sh $(srcdir)/genmodsrc.sh '#{mod_name}' $< > $@ || (rm -f $@; exit 1)
 
-ifneq ($(#{prefix}_EXPORTS),no)
-#{defsym}: #{pre_obj}
-	$(NM) -g --defined-only -P -p $< | sed 's/^\\([^ ]*\\).*/\\1 #{mod_name}/' > $@
-endif
-
-#{undsym}: #{pre_obj}
-	echo '#{mod_name}' > $@
-	$(NM) -u -P -p $< | cut -f1 -d' ' >> $@
-
 " + objs.collect_with_index do |obj, i|
       src = sources[i]
       fake_obj = File.basename(src).suffix('o')

Re: [PATCH] Avoid recompilation of unrelated files

[Bean]

On Wed, Jun 27, 2007 at 02:52:12AM +0800, Bean wrote:
> A problem of the grub2 make system is that even the smallest change of source
> code can trigger a huge recompilation. Try this:
> 
> touch commands/cmp.c
> make
> 
> The ideal situation is that only files related to cmp module is recompiled. 
> But unfortunately, this is not the case. The problem lies in file dependence.
> 
> cmp.mod: pre-cmp.o mod-cmp.o
> 
> pre-cmp.o: cmp_mod-commands_cmp.o
> 
> cmp_mod-commands_cmp.o: cmp.c
> 
> mod-cmp.o: mod-cmp.c
> 
> mod-cmp.c:  moddep.lst genmodsrc.sh
> 
> def-cmp.lst: pre-cmp.o
> 
> und-cmp.lst: pre-cmp.o
> 
> moddep.lst: def-cmp.lst und-cmp.lst
> 
> cmp.c => cmp_mod-commands_cmp.o => pre-cmp.o => def-cmp.lst and und-cmp.lst
> => moddep.lst
> 
> When cmp.c changes, moddep.lst changes. But every mod-*.c depends on 
> moddep.lst, they all need to be recreated. This will lead to a huge
> recompilation process.
> 
> def-cmp.lst and und-cmp.lst contains the exported symbol and unresolved symbol
> of the module. Their content rarely change. So we can update them only when 
> it's necessary, not every time we recompile the module.
> 
> The solution:
> 
> Merge these three rules:
> 
> pre-cmp.o: cmp_mod-commands_cmp.o
> 
> def-cmp.lst: pre-cmp.o
> 
> und-cmp.lst: pre-cmp.o
> 
> The output of def-cmp.lst is saved as tmp_pre-cmp.lst. When tmp_pre-cmp.lst
> and the old pre-cmp.lst is different, move tmp_pre-cmp.lst to pre-cmp.lst, 
> otherwise, just delete tmp_pre-cmp.lst. und-cmp.lst can be handled in exactly
> the same way.

I'm sorry, the previous patch doesn't work properly. It's still possible to
implement this feature using a flag variable. If any of the def-*.lst or
und-*.lst changes, we set the variable. When it's time to rebuild moddep.lst,
the flag variable is checked. If it's not set, it means nothing has changed
since last build, we can just touch moddep.lst instead of rebuilding it. The
same process can be applied to mod-*.c files as well. However, this method is
not very elegant, I'm not posting it here any more.

-- 
Bean