[RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[Sam Ravnborg]

The approach today is that all files that ends up in vmlinux are
linked directory by directory using "ld -r".

The following patch moves away from all the intermediate links
and do one huge link of all the .o files used for the final vmlinux.

The trick used is that for each directory a linker script is
generated that name all the file to be linked - and for
sub-directories this is just the linker script for that directory.

The file continue to be named "built-in.o" but it is really a linker script.

A limitation using linker scripts like this is that this introduces
a maximum depth of 10 levels (ld limitation when using INPUT).
And the final link will for a kernel likely involves > 1000 .o files.

For now this is an experimental path - and I have not even tried
to boot the kernel (as this is not simple in my current setup).

Feedback from the binutils people would be great if this is
actually a good way to get rid of the use of "ld -r" in the kernel.
Copied a few binutils people to hopefully get some feedback.

A typical file looks like this:

$cat kernel/built-in.o
INPUT(kernel/fork.o kernel/exec_domain.o ... kernel/locking/built-in.o ...)

kernel/locking/built-in.o is another linker script that again reference some .o files.

The background for this patch was that the LTO patch-kit that Andi Kleen is
working on referenced a thread in binutils mailing list that complained
about the use of "ld -r" in the kernel build and that LTO support for
"ld -r" builds would likely not be accepted in binutils.

I had woked on the following some years ago and recreated the patch.
I recall the idea is from someone else - but have long lost memory of
from where I saw it.

	Sam


diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index a48abea..b6ca48a 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -249,14 +249,6 @@ config DEBUG_SECTION_MISMATCH
 	    the analysis would not catch the illegal reference.
 	    This option tells gcc to inline less (but it does result in
 	    a larger kernel).
-	  - Run the section mismatch analysis for each module/built-in.o file.
-	    When we run the section mismatch analysis on vmlinux.o, we
-	    lose valueble information about where the mismatch was
-	    introduced.
-	    Running the analysis for each module/built-in.o file
-	    tells where the mismatch happens much closer to the
-	    source. The drawback is that the same mismatch is
-	    reported at least twice.
 	  - Enable verbose reporting from modpost in order to help resolve
 	    the section mismatches that are reported.
 
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index d5d859c..a568276 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -158,11 +158,6 @@ ifneq ($(KBUILD_CHECKSRC),0)
   endif
 endif
 
-# Do section mismatch analysis for each module/built-in.o
-ifdef CONFIG_DEBUG_SECTION_MISMATCH
-  cmd_secanalysis = ; scripts/mod/modpost $@
-endif
-
 # Compile C sources (.c)
 # ---------------------------------------------------------------------------
 
@@ -378,10 +373,9 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
 ifdef builtin-target
 quiet_cmd_link_o_target = LD      $@
 # If the list of objects to link is empty, just create an empty built-in.o
-cmd_link_o_target = $(if $(strip $(obj-y)),\
-		      $(LD) $(ld_flags) -r -o $@ $(filter $(obj-y), $^) \
-		      $(cmd_secanalysis),\
-		      rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@)
+cmd_link_o_target = $(if $(filter $(obj-y), $^),                   \
+			echo INPUT\($(filter $(obj-y), $^)\) > $@, \
+			echo "/* empty */" > $@)
 
 $(builtin-target): $(obj-y) FORCE
 	$(call if_changed,link_o_target)
@@ -428,10 +422,10 @@ $($(subst $(obj)/,,$(@:.o=-objs)))    \
 $($(subst $(obj)/,,$(@:.o=-y)))), $^)
  
 quiet_cmd_link_multi-y = LD      $@
-cmd_link_multi-y = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps) $(cmd_secanalysis)
+cmd_link_multi-y = echo INPUT\($(link_multi_deps)\) > $@
 
 quiet_cmd_link_multi-m = LD [M]  $@
-cmd_link_multi-m = $(cmd_link_multi-y)
+cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(link_multi_deps)
 
 # We would rather have a list of rules like
 # 	foo.o: $(foo-objs)

Re: [RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[Alan Modra]

On Sun, Feb 16, 2014 at 09:37:24PM +0100, Sam Ravnborg wrote:
> The trick used is that for each directory a linker script is
> generated that name all the file to be linked - and for
> sub-directories this is just the linker script for that directory.
> 
> The file continue to be named "built-in.o" but it is really a linker script.

You'll need binutils newer than 2012-06-25
https://sourceware.org/ml/binutils/2012-06/msg00201.html

> I recall the idea is from someone else - but have long lost memory of
> from where I saw it.

I think it might have been my idea originally, implemented by Stephen
Rothwell.

Another possibility is to use thin archives to package up the objects.
Thin archives are like normal archives except they just contain paths
to the objects, not the object contents.  Paths are flattened when
adding one thin archive to another.  I think Stephen may have tried
that idea too.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[Sam Ravnborg]

On Wed, Feb 19, 2014 at 01:28:56AM +1030, Alan Modra wrote:
> On Sun, Feb 16, 2014 at 09:37:24PM +0100, Sam Ravnborg wrote:
> > The trick used is that for each directory a linker script is
> > generated that name all the file to be linked - and for
> > sub-directories this is just the linker script for that directory.
> > 
> > The file continue to be named "built-in.o" but it is really a linker script.
> 
> You'll need binutils newer than 2012-06-25
> https://sourceware.org/ml/binutils/2012-06/msg00201.html
> 
> > I recall the idea is from someone else - but have long lost memory of
> > from where I saw it.
> 
> I think it might have been my idea originally, implemented by Stephen
> Rothwell.
Sounds right - added in cc:

> 
> Another possibility is to use thin archives to package up the objects.
> Thin archives are like normal archives except they just contain paths
> to the objects, not the object contents.  Paths are flattened when
> adding one thin archive to another.  I think Stephen may have tried
> that idea too.
I just coded this in a similar simple fashion.
It broke building vdso - needs to look into that and will get back when I may
understand why my link suddenly broke.

Thanks for the feedback!

	Sam

Re: [RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[Alan Modra]

On Tue, Feb 18, 2014 at 10:48:01PM +0100, Sam Ravnborg wrote:
> > Another possibility is to use thin archives to package up the objects.
> > Thin archives are like normal archives except they just contain paths
> > to the objects, not the object contents.  Paths are flattened when
> > adding one thin archive to another.  I think Stephen may have tried
> > that idea too.
> I just coded this in a similar simple fashion.
> It broke building vdso - needs to look into that and will get back when I may
> understand why my link suddenly broke.

Missing --whole-archive?

-- 
Alan Modra
Australia Development Lab, IBM

Re: [RFC] kbuild: drop use of "ld -r" for intermediate links of vmlinux

[Sam Ravnborg]

On Wed, Feb 19, 2014 at 11:31:00AM +1030, Alan Modra wrote:
> On Tue, Feb 18, 2014 at 10:48:01PM +0100, Sam Ravnborg wrote:
> > > Another possibility is to use thin archives to package up the objects.
> > > Thin archives are like normal archives except they just contain paths
> > > to the objects, not the object contents.  Paths are flattened when
> > > adding one thin archive to another.  I think Stephen may have tried
> > > that idea too.
> > I just coded this in a similar simple fashion.
> > It broke building vdso - needs to look into that and will get back when I may
> > understand why my link suddenly broke.
> 
> Missing --whole-archive?

No - at least something else to begin with.
A special liker script is used when building the vdsp files for x86 - and the
linker script is ignored as such when embedded in a .o file with ar it seems.
This is not a big suprise.

The makefile magic used for the vdso part of the kernel needs a bit more coffee
than what is good for me for now...
Will look at it again later.

Thanks for the feedback.

	Sam