[PATCH 0/4] kbuild: Move vmlinux link out of top-level Makefile

[PATCH 0/4] kbuild: Move vmlinux link out of top-level Makefile

[Sam Ravnborg]

The following patchset moves all the magic used
to deal with the final link of vmlinux from the
top-level Makefile to a shell script.

The main motivation was to convert the almost
unreadable mess in the top-level Makefile to
an easy to read/maintain shell script.

The final link stages are serialized
anyway - so there were nothing gained by using
the Makefile logic to handle this.

What we gain:
- A readable final link of vmlinux
- Less junk in the top-level Makefile

What we loose by the conversion:
- We no longer rebuild if options to ld changes.
- We no logner rebuild if one of the .tmp_kallsyms
  files are deleted.
- We no longer rebuild if vmlinux.o is deleted

None of the above cases are hit by a typical
kernel developer so the drawbacks are acceptable.

I had to implement special handling of um in the
link-vmlinux script. It is not pretty by any means,
but I could not come up with something better.
My previous attempts to use ld for um linking has failed,
so this kludge was introduced.

There is no outstanding issues to my knowledge,
and the patchset is ready to be applied.
I assume this should go in via the kbuild#misc tree.

	Sam

 Documentation/kbuild/kbuild.txt |   15 +++
 Makefile                        |  213 ++------------------------------------
 arch/sparc/Makefile             |   11 --
 arch/sparc/boot/Makefile        |   14 ++-
 arch/um/Makefile                |   11 +--
 scripts/link-vmlinux.sh         |  204 +++++++++++++++++++++++++++++++++++++
 6 files changed, 242 insertions(+), 226 deletions(-)

Sam Ravnborg (4):
      kbuild: drop unused KBUILD_VMLINUX_OBJS from top-level Makefile
      kbuild: refactor final link of sparc32
      kbuild: link of vmlinux moved to a script
      kbuild: document KBUILD_LDS, KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN

[PATCH 0/4] kbuild: Move vmlinux link out of top-level Makefile

[Sam Ravnborg]

The following patchset moves all the magic used
to deal with the final link of vmlinux from the
top-level Makefile to a shell script.

The main motivation was to convert the almost
unreadable mess in the top-level Makefile to
an easy to read/maintain shell script.

The final link stages are serialized
anyway - so there were nothing gained by using
the Makefile logic to handle this.

What we gain:
- A readable final link of vmlinux
- Less junk in the top-level Makefile

What we loose by the conversion:
- We no longer rebuild if options to ld changes.
- We no logner rebuild if one of the .tmp_kallsyms
  files are deleted.
- We no longer rebuild if vmlinux.o is deleted

None of the above cases are hit by a typical
kernel developer so the drawbacks are acceptable.

I had to implement special handling of um in the
link-vmlinux script. It is not pretty by any means,
but I could not come up with something better.
My previous attempts to use ld for um linking has failed,
so this kludge was introduced.

There is no outstanding issues to my knowledge,
and the patchset is ready to be applied.
I assume this should go in via the kbuild#misc tree.

	Sam

 Documentation/kbuild/kbuild.txt |   15 +++
 Makefile                        |  213 ++------------------------------------
 arch/sparc/Makefile             |   11 --
 arch/sparc/boot/Makefile        |   14 ++-
 arch/um/Makefile                |   11 +--
 scripts/link-vmlinux.sh         |  204 +++++++++++++++++++++++++++++++++++++
 6 files changed, 242 insertions(+), 226 deletions(-)

Sam Ravnborg (4):
      kbuild: drop unused KBUILD_VMLINUX_OBJS from top-level Makefile
      kbuild: refactor final link of sparc32
      kbuild: link of vmlinux moved to a script
      kbuild: document KBUILD_LDS, KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN

[PATCH 1/4] kbuild: drop unused KBUILD_VMLINUX_OBJS from top-level Makefile

[Sam Ravnborg]

From a5fa412754aef31d590f805049fd8e25857b1802 Mon Sep 17 00:00:00 2001
From: Sam Ravnborg <sam@ravnborg.org>
Date: Mon, 23 Apr 2012 18:52:39 +0200
Subject: [PATCH 1/4] kbuild: drop unused KBUILD_VMLINUX_OBJS from top-level
 Makefile

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
---
 Makefile |    1 -
 1 file changed, 1 deletion(-)

diff --git a/Makefile b/Makefile
index f6578f4..287aee4 100644
--- a/Makefile
+++ b/Makefile
@@ -758,7 +758,6 @@ vmlinux-init := $(head-y) $(init-y)
 vmlinux-main := $(core-y) $(libs-y) $(drivers-y) $(net-y)
 vmlinux-all  := $(vmlinux-init) $(vmlinux-main)
 vmlinux-lds  := arch/$(SRCARCH)/kernel/vmlinux.lds
-export KBUILD_VMLINUX_OBJS := $(vmlinux-all)
 
 # Rule to link vmlinux - also used during CONFIG_KALLSYMS
 # May be overridden by arch/$(ARCH)/Makefile
-- 
1.7.10

[PATCH 2/4] kbuild: refactor final link of sparc32

[Sam Ravnborg]

From 8716c7f473ec65c57811bb21095354b813b48898 Mon Sep 17 00:00:00 2001
From: Sam Ravnborg <sam@ravnborg.org>
Date: Mon, 23 Apr 2012 20:29:07 +0200
Subject: [PATCH 2/4] kbuild: refactor final link of sparc32

sparc32 uses an additional final link to support btfix.
Introduce a new set of exported variables in the top-level Makefile
to make the extra linking step simpler.

sparc32 have hardcoded knowledge of kallsyms support. This fix
include support for EXTRA_KALLSYM_PASS=1.
The ugly part is that it is hardcoded in the arch/sparc/boot
Makefile.

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: "David S. Miller" <davem@davemloft.net>
---
 Makefile                 |    5 +++++
 arch/sparc/Makefile      |   11 -----------
 arch/sparc/boot/Makefile |   14 +++++++++-----
 3 files changed, 14 insertions(+), 16 deletions(-)

diff --git a/Makefile b/Makefile
index 287aee4..00bf5b1 100644
--- a/Makefile
+++ b/Makefile
@@ -727,6 +727,11 @@ libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
 libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
 libs-y		:= $(libs-y1) $(libs-y2)
 
+# externally visible symbols
+export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
+export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y)
+export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
+
 # Build vmlinux
 # ---------------------------------------------------------------------------
 # vmlinux is built from the objects selected by $(vmlinux-init) and
diff --git a/arch/sparc/Makefile b/arch/sparc/Makefile
index eddcfb3..3195f77 100644
--- a/arch/sparc/Makefile
+++ b/arch/sparc/Makefile
@@ -72,17 +72,6 @@ libs-y                 += arch/sparc/lib/
 
 drivers-$(CONFIG_OPROFILE)	+= arch/sparc/oprofile/
 
-# Export what is needed by arch/sparc/boot/Makefile
-export VMLINUX_INIT VMLINUX_MAIN
-VMLINUX_INIT := $(head-y) $(init-y)
-VMLINUX_MAIN := $(core-y) kernel/ mm/ fs/ ipc/ security/ crypto/ block/
-VMLINUX_MAIN += $(patsubst %/, %/lib.a, $(libs-y)) $(libs-y)
-VMLINUX_MAIN += $(drivers-y) $(net-y)
-
-ifdef CONFIG_KALLSYMS
-export kallsyms.o := .tmp_kallsyms2.o
-endif
-
 boot := arch/sparc/boot
 
 # Default target
diff --git a/arch/sparc/boot/Makefile b/arch/sparc/boot/Makefile
index d56d199..78f1994 100644
--- a/arch/sparc/boot/Makefile
+++ b/arch/sparc/boot/Makefile
@@ -39,11 +39,15 @@ define rule_image
 	echo 'cmd_$@ := $(cmd_image)' > $(@D)/.$(@F).cmd
 endef
 
-BTOBJS := $(patsubst %/, %/built-in.o, $(VMLINUX_INIT))
-BTLIBS := $(patsubst %/, %/built-in.o, $(VMLINUX_MAIN))
-LDFLAGS_image := -T arch/sparc/kernel/vmlinux.lds $(BTOBJS) \
-                  --start-group $(BTLIBS) --end-group \
-                  $(kallsyms.o) $(obj)/btfix.o
+# Support for kallsyms
+kallsyms-$(CONFIG_KALLSYMS) := .tmp_kallsyms2.o
+ifdef KALLSYMS_EXTRA_PASS
+        kallsyms-$(CONFIG_KALLSYMS) := .tmp_kallsyms3.o
+endif
+
+LDFLAGS_image := -T $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT)           \
+                  --start-group $(KBUILD_VMLINUX_MAIN) --end-group \
+                  $(kallsyms-y) $(obj)/btfix.o
 
 # Link the final image including btfixup'ed symbols.
 # This is a replacement for the link done in the top-level Makefile.
-- 
1.7.10

[PATCH 3/4] kbuild: link of vmlinux moved to a script

[Sam Ravnborg]

From 988727cf128a7439b28ddbbee11ee068d007cb00 Mon Sep 17 00:00:00 2001
From: Sam Ravnborg <sam@ravnborg.org>
Date: Mon, 23 Apr 2012 22:55:04 +0200
Subject: [PATCH 3/4] kbuild: link of vmlinux moved to a script

Move the final link of vmlinux to a script to improve
readability and maintainability of the code.
The Makefil fragments used to link vmlinux has over the
years seen far too much changes and the logic was very
hard to follow.

As the process by nature is serialized there was
nothing gained including this in the Makefile.

"um" has special link requirments - and the
only was to handle this was to hard-code the linking
of "um" in the script.
This was better than trying to modularize it only for the
benefit of "um" anyway.

The shell script has been improved after input from:
Arnaud Lacombe <lacombar@gmail.com>

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: Arnaud Lacombe <lacombar@gmail.com>
Cc: Richard Weinberger <richard@nod.at>
---
 Makefile                |  211 ++---------------------------------------------
 arch/um/Makefile        |   11 +--
 scripts/link-vmlinux.sh |  204 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 215 insertions(+), 211 deletions(-)
 create mode 100644 scripts/link-vmlinux.sh

diff --git a/Makefile b/Makefile
index 00bf5b1..9cbf8bd 100644
--- a/Makefile
+++ b/Makefile
@@ -341,7 +341,6 @@ AWK		= awk
 GENKSYMS	= scripts/genksyms/genksyms
 INSTALLKERNEL  := installkernel
 DEPMOD		= /sbin/depmod
-KALLSYMS	= scripts/kallsyms
 PERL		= perl
 CHECK		= sparse
 
@@ -727,191 +726,18 @@ libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
 libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
 libs-y		:= $(libs-y1) $(libs-y2)
 
-# externally visible symbols
+# Externally visible symbols (used by link-vmlinux.sh)
 export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
 export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y)
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
+export LDFLAGS_vmlinux
 
-# Build vmlinux
-# ---------------------------------------------------------------------------
-# vmlinux is built from the objects selected by $(vmlinux-init) and
-# $(vmlinux-main). Most are built-in.o files from top-level directories
-# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
-# Ordering when linking is important, and $(vmlinux-init) must be first.
-#
-# vmlinux
-#   ^
-#   |
-#   +-< $(vmlinux-init)
-#   |   +--< init/version.o + more
-#   |
-#   +--< $(vmlinux-main)
-#   |    +--< driver/built-in.o mm/built-in.o + more
-#   |
-#   +-< kallsyms.o (see description in CONFIG_KALLSYMS section)
-#
-# vmlinux version (uname -v) cannot be updated during normal
-# descending-into-subdirs phase since we do not yet know if we need to
-# update vmlinux.
-# Therefore this step is delayed until just before final link of vmlinux -
-# except in the kallsyms case where it is done just before adding the
-# symbols to the kernel.
-#
-# System.map is generated to document addresses of all kernel symbols
-
-vmlinux-init := $(head-y) $(init-y)
-vmlinux-main := $(core-y) $(libs-y) $(drivers-y) $(net-y)
-vmlinux-all  := $(vmlinux-init) $(vmlinux-main)
-vmlinux-lds  := arch/$(SRCARCH)/kernel/vmlinux.lds
-
-# Rule to link vmlinux - also used during CONFIG_KALLSYMS
-# May be overridden by arch/$(ARCH)/Makefile
-quiet_cmd_vmlinux__ ?= LD      $@
-      cmd_vmlinux__ ?= $(LD) $(LDFLAGS) $(LDFLAGS_vmlinux) -o $@ \
-      -T $(vmlinux-lds) $(vmlinux-init)                          \
-      --start-group $(vmlinux-main) --end-group                  \
-      $(filter-out $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) vmlinux.o FORCE ,$^)
-
-# Generate new vmlinux version
-quiet_cmd_vmlinux_version = GEN     .version
-      cmd_vmlinux_version = set -e;                     \
-	if [ ! -r .version ]; then			\
-	  rm -f .version;				\
-	  echo 1 >.version;				\
-	else						\
-	  mv .version .old_version;			\
-	  expr 0$$(cat .old_version) + 1 >.version;	\
-	fi;						\
-	$(MAKE) $(build)=init
-
-# Generate System.map
-quiet_cmd_sysmap = SYSMAP
-      cmd_sysmap = $(CONFIG_SHELL) $(srctree)/scripts/mksysmap
-
-# Link of vmlinux
-# If CONFIG_KALLSYMS is set .version is already updated
-# Generate System.map and verify that the content is consistent
-# Use + in front of the vmlinux_version rule to silent warning with make -j2
-# First command is ':' to allow us to use + in front of the rule
-define rule_vmlinux__
-	:
-	$(if $(CONFIG_KALLSYMS),,+$(call cmd,vmlinux_version))
-
-	$(call cmd,vmlinux__)
-	$(Q)echo 'cmd_$@ := $(cmd_vmlinux__)' > $(@D)/.$(@F).cmd
-
-	$(Q)$(if $($(quiet)cmd_sysmap),                                      \
-	  echo '  $($(quiet)cmd_sysmap)  System.map' &&)                     \
-	$(cmd_sysmap) $@ System.map;                                         \
-	if [ $$? -ne 0 ]; then                                               \
-		rm -f $@;                                                    \
-		/bin/false;                                                  \
-	fi;
-	$(verify_kallsyms)
-endef
-
-
-ifdef CONFIG_KALLSYMS
-# Generate section listing all symbols and add it into vmlinux $(kallsyms.o)
-# It's a three stage process:
-# o .tmp_vmlinux1 has all symbols and sections, but __kallsyms is
-#   empty
-#   Running kallsyms on that gives us .tmp_kallsyms1.o with
-#   the right size - vmlinux version (uname -v) is updated during this step
-# o .tmp_vmlinux2 now has a __kallsyms section of the right size,
-#   but due to the added section, some addresses have shifted.
-#   From here, we generate a correct .tmp_kallsyms2.o
-# o The correct .tmp_kallsyms2.o is linked into the final vmlinux.
-# o Verify that the System.map from vmlinux matches the map from
-#   .tmp_vmlinux2, just in case we did not generate kallsyms correctly.
-# o If 'make KALLSYMS_EXTRA_PASS=1" was used, do an extra pass using
-#   .tmp_vmlinux3 and .tmp_kallsyms3.o.  This is only meant as a
-#   temporary bypass to allow the kernel to be built while the
-#   maintainers work out what went wrong with kallsyms.
-
-last_kallsyms := 2
-
-ifdef KALLSYMS_EXTRA_PASS
-ifneq ($(KALLSYMS_EXTRA_PASS),0)
-last_kallsyms := 3
-endif
-endif
-
-kallsyms.o := .tmp_kallsyms$(last_kallsyms).o
-
-define verify_kallsyms
-	$(Q)$(if $($(quiet)cmd_sysmap),                                      \
-	  echo '  $($(quiet)cmd_sysmap)  .tmp_System.map' &&)                \
-	  $(cmd_sysmap) .tmp_vmlinux$(last_kallsyms) .tmp_System.map
-	$(Q)cmp -s System.map .tmp_System.map ||                             \
-		(echo Inconsistent kallsyms data;                            \
-		 echo This is a bug - please report about it;                \
-		 echo Try "make KALLSYMS_EXTRA_PASS=1" as a workaround;      \
-		 rm .tmp_kallsyms* ; /bin/false )
-endef
-
-# Update vmlinux version before link
-# Use + in front of this rule to silent warning about make -j1
-# First command is ':' to allow us to use + in front of this rule
-cmd_ksym_ld = $(cmd_vmlinux__)
-define rule_ksym_ld
-	: 
-	+$(call cmd,vmlinux_version)
-	$(call cmd,vmlinux__)
-	$(Q)echo 'cmd_$@ := $(cmd_vmlinux__)' > $(@D)/.$(@F).cmd
-endef
-
-# Generate .S file with all kernel symbols
-quiet_cmd_kallsyms = KSYM    $@
-      cmd_kallsyms = $(NM) -n $< | $(KALLSYMS) \
-                     $(if $(CONFIG_KALLSYMS_ALL),--all-symbols) > $@
-
-.tmp_kallsyms1.o .tmp_kallsyms2.o .tmp_kallsyms3.o: %.o: %.S scripts FORCE
-	$(call if_changed_dep,as_o_S)
-
-.tmp_kallsyms%.S: .tmp_vmlinux% $(KALLSYMS)
-	$(call cmd,kallsyms)
+vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN)
 
-# .tmp_vmlinux1 must be complete except kallsyms, so update vmlinux version
-.tmp_vmlinux1: $(vmlinux-lds) $(vmlinux-all) FORCE
-	$(call if_changed_rule,ksym_ld)
-
-.tmp_vmlinux2: $(vmlinux-lds) $(vmlinux-all) .tmp_kallsyms1.o FORCE
-	$(call if_changed,vmlinux__)
-
-.tmp_vmlinux3: $(vmlinux-lds) $(vmlinux-all) .tmp_kallsyms2.o FORCE
-	$(call if_changed,vmlinux__)
-
-# Needs to visit scripts/ before $(KALLSYMS) can be used.
-$(KALLSYMS): scripts ;
-
-# Generate some data for debugging strange kallsyms problems
-debug_kallsyms: .tmp_map$(last_kallsyms)
-
-.tmp_map%: .tmp_vmlinux% FORCE
-	($(OBJDUMP) -h $< | $(AWK) '/^ +[0-9]/{print $$4 " 0 " $$2}'; $(NM) $<) | sort > $@
-
-.tmp_map3: .tmp_map2
-
-.tmp_map2: .tmp_map1
-
-endif # ifdef CONFIG_KALLSYMS
-
-# Do modpost on a prelinked vmlinux. The finally linked vmlinux has
-# relevant sections renamed as per the linker script.
-quiet_cmd_vmlinux-modpost = LD      $@
-      cmd_vmlinux-modpost = $(LD) $(LDFLAGS) -r -o $@                          \
-	 $(vmlinux-init) --start-group $(vmlinux-main) --end-group             \
-	 $(filter-out $(vmlinux-init) $(vmlinux-main) FORCE ,$^)
-define rule_vmlinux-modpost
-	:
-	+$(call cmd,vmlinux-modpost)
-	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost $@
-	$(Q)echo 'cmd_$@ := $(cmd_vmlinux-modpost)' > $(dot-target).cmd
-endef
-
-# vmlinux image - including updated kernel symbols
-vmlinux: $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) vmlinux.o $(kallsyms.o) FORCE
+# Final link of vmlinux
+# Include targets which we want to
+# execute if the rest of the kernel build went well.
+vmlinux: scripts/link-vmlinux.sh $(vmlinux-deps)
 ifdef CONFIG_HEADERS_CHECK
 	$(Q)$(MAKE) -f $(srctree)/Makefile headers_check
 endif
@@ -921,22 +747,11 @@ endif
 ifdef CONFIG_BUILD_DOCSRC
 	$(Q)$(MAKE) $(build)=Documentation
 endif
-	$(call vmlinux-modpost)
-	$(call if_changed_rule,vmlinux__)
-	$(Q)rm -f .old_version
-
-# build vmlinux.o first to catch section mismatch errors early
-ifdef CONFIG_KALLSYMS
-.tmp_vmlinux1: vmlinux.o
-endif
-
-modpost-init := $(filter-out init/built-in.o, $(vmlinux-init))
-vmlinux.o: $(modpost-init) $(vmlinux-main) FORCE
-	$(call if_changed_rule,vmlinux-modpost)
+	+$(Q)$(CONFIG_SHELL) $<
 
 # The actual objects are generated when descending, 
 # make sure no implicit rule kicks in
-$(sort $(vmlinux-init) $(vmlinux-main)) $(vmlinux-lds): $(vmlinux-dirs) ;
+$(sort $(vmlinux-deps)): $(vmlinux-dirs) ;
 
 # Handle descending into subdirectories listed in $(vmlinux-dirs)
 # Preset locale variables to speed up the build process. Limit locale
@@ -1540,14 +1355,6 @@ quiet_cmd_depmod = DEPMOD  $(KERNELRELEASE)
 cmd_crmodverdir = $(Q)mkdir -p $(MODVERDIR) \
                   $(if $(KBUILD_MODULES),; rm -f $(MODVERDIR)/*)
 
-a_flags = -Wp,-MD,$(depfile) $(KBUILD_AFLAGS) $(AFLAGS_KERNEL) \
-	  $(KBUILD_AFLAGS_KERNEL)                              \
-	  $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(KBUILD_CPPFLAGS) \
-	  $(modkern_aflags) $(EXTRA_AFLAGS) $(AFLAGS_$(basetarget).o)
-
-quiet_cmd_as_o_S = AS      $@
-cmd_as_o_S       = $(CC) $(a_flags) -c -o $@ $<
-
 # read all saved command lines
 
 targets := $(wildcard $(sort $(targets)))
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 55c0661..0970910 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -121,15 +121,8 @@ LINK_WRAPS = -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 
 LD_FLAGS_CMDLINE = $(foreach opt,$(LDFLAGS),-Wl,$(opt))
 
-CFLAGS_vmlinux := $(LINK-y) $(LINK_WRAPS) $(LD_FLAGS_CMDLINE)
-define cmd_vmlinux__
-	$(CC) $(CFLAGS_vmlinux) -o $@ \
-	-Wl,-T,$(vmlinux-lds) $(vmlinux-init) \
-	-Wl,--start-group $(vmlinux-main) -Wl,--end-group \
-	-lutil \
-	$(filter-out $(vmlinux-lds) $(vmlinux-init) $(vmlinux-main) vmlinux.o \
-	FORCE ,$^) ; rm -f linux
-endef
+# Used by link-vmlinux.sh which has special support for um link
+export CFLAGS_vmlinux := $(LINK-y) $(LINK_WRAPS) $(LD_FLAGS_CMDLINE)
 
 # When cleaning we don't include .config, so we don't include
 # TT or skas makefiles and don't clean skas_ptregs.h.
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
new file mode 100644
index 0000000..afd0b96
--- /dev/null
+++ b/scripts/link-vmlinux.sh
@@ -0,0 +1,204 @@
+#!/bin/sh
+#
+# link vmlinux
+#
+# vmlinux is linked from the objects selected by $(KBUILD_VMLINUX_INIT) and
+# $(KBUILD_VMLINUX_MAIN). Most are built-in.o files from top-level directories
+# in the kernel tree, others are specified in arch/$(ARCH)/Makefile.
+# Ordering when linking is important, and $(KBUILD_VMLINUX_INIT) must be first.
+#
+# vmlinux
+#   ^
+#   |
+#   +-< $(KBUILD_VMLINUX_INIT)
+#   |   +--< init/version.o + more
+#   |
+#   +--< $(KBUILD_VMLINUX_MAIN)
+#   |    +--< drivers/built-in.o mm/built-in.o + more
+#   |
+#   +-< ${kallsymso} (see description in KALLSYMS section)
+#
+# vmlinux version (uname -v) cannot be updated during normal
+# descending-into-subdirs phase since we do not yet know if we need to
+# update vmlinux.
+# Therefore this step is delayed until just before final link of vmlinux.
+#
+# System.map is generated to document addresses of all kernel symbols
+
+# We need access to CONFIG_ symbols
+source ./.config
+
+# Error out on error
+set -e
+
+# Nice output in kbuild format
+# Will be supressed by "make -s"
+info()
+{
+	if [ "${quiet}" != "silent_" ]; then
+		printf "  %-7s %s\n" ${1} ${2}
+	fi
+}
+
+# Link of vmlinux.o used for section mismatch analysis
+# ${1} output file
+modpost_link()
+{
+	${LD} ${LDFLAGS} -r -o ${1} ${KBUILD_VMLINUX_INIT}                   \
+		--start-group ${KBUILD_VMLINUX_MAIN} --end-group
+}
+
+# Link of vmlinux
+# ${1} - optional extra .o files
+# ${2} - output file
+vmlinux_link()
+{
+	local lds="${objtree}/${KBUILD_LDS}"
+
+	if [ "${SRCARCH}" != "um" ]; then
+		${LD} ${LDFLAGS} ${LDFLAGS_vmlinux} -o ${2}                  \
+			-T ${lds} ${KBUILD_VMLINUX_INIT}                     \
+			--start-group ${KBUILD_VMLINUX_MAIN} --end-group ${1}
+	else
+		${CC} ${CFLAGS_vmlinux} -o ${2}                              \
+			-Wl,-T,${lds} ${KBUILD_VMLINUX_INIT}                 \
+			-Wl,--start-group                                    \
+				 ${KBUILD_VMLINUX_MAIN}                      \
+			-Wl,--end-group                                      \
+			-lutil ${1}
+		rm -f linux
+	fi
+}
+
+
+# Create ${2} .o file with all symbols from the ${1} object file
+kallsyms()
+{
+	info KSYM ${2}
+	local kallsymopt;
+
+	if [ -n "${CONFIG_KALLSYMS_ALL}" ]; then
+		kallsymopt=--all-symbols
+	fi
+
+	local aflags="${KBUILD_AFLAGS} ${NOSTDINC_FLAGS}                     \
+		      ${LINUXINCLUDE} ${KBUILD_CPPFLAGS}"
+
+	${NM} -n ${1} | \
+		scripts/kallsyms ${kallsymopt} | \
+		${CC} ${aflags} -c -o ${2} -x assembler-with-cpp -
+}
+
+# Create map file with all symbols from ${1}
+# See mksymap for additional details
+mksysmap()
+{
+	${CONFIG_SHELL} "${srctree}/scripts/mksysmap" ${1} ${2}
+}
+
+# Delete output files in case of error
+trap cleanup SIGHUP SIGINT SIGQUIT SIGTERM ERR
+cleanup()
+{
+	rm -f vmlinux.o
+	rm -f .old_version
+	rm -f .tmp_vmlinux*
+	rm -f .tmp_kallsyms*
+	rm -f vmlinux
+	rm -f .tmp_System.map
+	rm -f System.map
+}
+
+#
+#
+# Use "make V=1" to debug this script
+case "${KBUILD_VERBOSE}" in
+*1*)
+	set -x
+	;;
+esac
+
+#link vmlinux.o
+info LD vmlinux.o
+modpost_link vmlinux.o
+
+# modpost vmlinux.o to check for section mismatches
+${MAKE} -f "${srctree}/scripts/Makefile.modpost" vmlinux.o
+
+# Update version
+info GEN .version
+if [ ! -r .version ]; then
+	rm -f .version;
+	echo 1 >.version;
+else
+	mv .version .old_version;
+	expr 0$(cat .old_version) + 1 >.version;
+fi;
+
+# final build of init/
+${MAKE} -f "${srctree}/scripts/Makefile.build" obj=init
+
+kallsymso=""
+kallsyms_vmlinux=""
+if [ -n "${CONFIG_KALLSYMS}" ]; then
+
+	# kallsyms support
+	# Generate section listing all symbols and add it into vmlinux
+	# It's a three step process:
+	# 1)  Link .tmp_vmlinux1 so it has all symbols and sections,
+	#     but __kallsyms is empty.
+	#     Running kallsyms on that gives us .tmp_kallsyms1.o with
+	#     the right size
+	# 2)  Link .tmp_vmlinux2 so it now has a __kallsyms section of
+	#     the right size, but due to the added section, some
+	#     addresses have shifted.
+	#     From here, we generate a correct .tmp_kallsyms2.o
+	# 2a) We may use an extra pass as this has been necessary to
+	#     woraround some alignment related bugs.
+	#     KALLSYMS_EXTRA_PASS=1 is used to trigger this.
+	# 3)  The correct ${kallsymso} is linked into the final vmlinux.
+	#
+	# a)  Verify that the System.map from vmlinux matches the map from
+	#     ${kallsymso}.
+
+	kallsymso=.tmp_kallsyms2.o
+	kallsyms_vmlinux=.tmp_vmlinux2
+
+	# step 1
+	vmlinux_link "" .tmp_vmlinux1
+	kallsyms .tmp_vmlinux1 .tmp_kallsyms1.o
+
+	# step 2
+	vmlinux_link .tmp_kallsyms1.o .tmp_vmlinux2
+	kallsyms .tmp_vmlinux2 .tmp_kallsyms2.o
+
+	# step 2a
+	if [ -n "${KALLSYMS_EXTRA_PASS}" ]; then
+		kallsymso=.tmp_kallsyms3.o
+		kallsyms_vmlinux=.tmp_vmlinux2
+
+		vmlinux_link .tmp_kallsyms2.o .tmp_vmlinux3
+
+		kallsyms .tmp_vmlinux3 .tmp_kallsyms3.o
+	fi
+fi
+
+vmlinux_link "${kallsymso}" vmlinux
+
+info SYSMAP System.map
+mksysmap vmlinux System.map
+
+# step a (see comment above)
+if [ -n "${CONFIG_KALLSYMS}" ]; then
+	mksysmap ${kallsyms_vmlinux} .tmp_System.map
+
+	if [ $(cmp -s System.map .tmp_System.map) ]; then
+		echo Inconsistent kallsyms data
+		echo echo Try "make KALLSYMS_EXTRA_PASS=1" as a workaround
+		cleanup
+		exit 1
+	fi
+fi
+
+# We made a new kernel - delete old version file
+rm -f .old_version
-- 
1.7.10

[PATCH 4/4] kbuild: document KBUILD_LDS, KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN

[Sam Ravnborg]

From 8e5298ec2fd86f90ce55503b02ccb67c30677611 Mon Sep 17 00:00:00 2001
From: Sam Ravnborg <sam@ravnborg.org>
Date: Mon, 23 Apr 2012 23:06:25 +0200
Subject: [PATCH 4/4] kbuild: document KBUILD_LDS, KBUILD_VMLINUX_INIT and
 KBUILD_VMLINUX_MAIN

Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
---
 Documentation/kbuild/kbuild.txt |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
index 68e32bb..47038a2 100644
--- a/Documentation/kbuild/kbuild.txt
+++ b/Documentation/kbuild/kbuild.txt
@@ -214,3 +214,18 @@ KBUILD_BUILD_USER, KBUILD_BUILD_HOST
 These two variables allow to override the user@host string displayed during
 boot and in /proc/version. The default value is the output of the commands
 whoami and host, respectively.
+
+KBUILD_LDS
+--------------------------------------------------
+The linker script with full path. Assigned by the top-level Makefile.
+
+KBUILD_VMLINUX_INIT
+--------------------------------------------------
+All objects files for the init (first) part of vmlinux.
+Files specified with KBUILD_VMLINUX_INIT are linked first.
+
+KBUILD_VMLINUX_MAIN
+--------------------------------------------------
+All objects files for the main part of vmlinux.
+KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN together specify
+all the object files used to link vmlinux.
-- 
1.7.10

Re: [PATCH 0/4] kbuild: Move vmlinux link out of top-level Makefile

[Andi Kleen]

Sam Ravnborg <sam@ravnborg.org> writes:
>
> What we loose by the conversion:
> - We no longer rebuild if options to ld changes.

I think that's a problem. Can that be fixed somehow?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: [PATCH 3/4] kbuild: link of vmlinux moved to a script

[Nick Bowler]

Hi Sam,

Without actually running it, a few comments on the shell script itself.

On 2012-04-24 21:44 +0200, Sam Ravnborg wrote:
> +++ b/scripts/link-vmlinux.sh
[...]
> +# We need access to CONFIG_ symbols
> +source ./.config

source is non-standard and not widely supported.  In particular, this
will not work on systems using dash for /bin/sh.  In this instance,
simply replacing source with the portable dot builtin will be exactly
equivalent, as in:

   . ./.config

> +# Error out on error
> +set -e

While a valiant effort, using set -e to handle errors is just asking
for surprises down the road.  Ignoring the fact that "set -e" has been
historically underspecified (and there is thus some variety in how
different shells handle it), consider the following:

  set -e
  foo() {
    false
    echo oops
  }
  foo || echo foo failed

Because the _call_ to foo appears on the left side of ||, the commands
_within_ foo are not subject to the effects of set -e.  In bash, the
above script therefore prints "oops", and does not normally print "foo
failed".

> +# Delete output files in case of error
> +trap cleanup SIGHUP SIGINT SIGQUIT SIGTERM ERR

This is another ksh-ism, which will again fail on systems with /bin/sh
being dash.  More portable:

  trap cleanup HUP INT QUIT TERM
  trap '(exit $?) || cleanup' EXIT

However, the Autoconf shell portability manual documents some confusion
regarding what $? should be in an exit trap when it is entered by using
the exit command:

  "Bash considers exit to be the last command, while Zsh and Solaris
  /bin/sh consider that when the trap is run it is still in the exit,
  hence it is the previous exit status that the trap receives".

The manual subsequently recommends that "exit 42" therefore be written
as "(exit 42); exit 42".  Fortunately, in this script, the only call to
exit immediately follows an explicit call to cleanup, so it won't
actually matter whether or not the exit trap calls cleanup again.

Nevertheless, I cannot reproduce the described trap behaviour with
current versions of Zsh, so maybe all this information is out of date.
That being said ...

> +cleanup()
> +{
> +	rm -f vmlinux.o
> +	rm -f .old_version
> +	rm -f .tmp_vmlinux*
> +	rm -f .tmp_kallsyms*
> +	rm -f vmlinux
> +	rm -f .tmp_System.map
> +	rm -f System.map
> +}

... this whole ad-hoc cleanup mechanism looks prone to failure.
Basically, if something goes really wrong and files get left behind, a
subsequent "make" is going to see up-to-date files and proceed with
garbage.  A more robust approach is, for filenames used in the Makefile,
to output to "dummy" files (e.g., write to vmlinux.tmp).  Only after
everything was successful (either at the very end of the script or in
the makefile rule which calls it), rename the dummy files to their
actual filename.

That way, the cleanup becomes "best effort" and, from a build
correctness point of view, won't matter if it misses removing
files for whatever reason.

[...]
> +# step a (see comment above)
> +if [ -n "${CONFIG_KALLSYMS}" ]; then
> +	mksysmap ${kallsyms_vmlinux} .tmp_System.map
> +
> +	if [ $(cmp -s System.map .tmp_System.map) ]; then

This test is wrong: it is passing the output of cmp to the [ builtin.
Aside from not being properly quoted (so the output of cmp is subject to
word splitting, which will make [ unhappy if it actually happens),
you've asked cmp to produce no output by giving it the -s option so
this test will always be false.

Presumably this should be:

        if ! cmp -s System.map .tmp_System.map; then

which actually tests the exit status of cmp instead of its output, and
executes the branch if cmp failed.  (Aside: some older shells don't
support the "if ! cmd; then stuff; fi" pattern, so you sometimes see it
written as: "if cmd; then :; else stuff; fi".  We probably don't care
too much about such shells here).

> +		echo Inconsistent kallsyms data
> +		echo echo Try "make KALLSYMS_EXTRA_PASS=1" as a workaround
> +		cleanup
> +		exit 1
> +	fi
> +fi

Cheers,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: [PATCH 3/4] kbuild: link of vmlinux moved to a script

[Sam Ravnborg]

On Wed, Apr 25, 2012 at 10:24:52AM -0400, Nick Bowler wrote:
> Hi Sam,
> 
> Without actually running it, a few comments on the shell script itself.

Thanks for a detailed review.
I will address it all or let you know if I fail to address some part of it.

	Sam