[Buildroot] [PATCH] Allow Buildroot to update toolchain

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

Sometimes it is useful to self contain a toolchain.  To do this, one
must somehow build in the extra libraries needed to make the toolchain
complete.  This patch adjusts Buildroot to, by configuration, modify the
external toolchain while building additional libraries.

Signed-off-by: Andy Kennedy <andy.kennedy@adtran.com>
---
diff -Naur a/package/Makefile.in b/package/Makefile.in
--- a/package/Makefile.in	2013-09-17 06:42:07.000000000 -0500
+++ b/package/Makefile.in	2013-09-26 09:28:26.000000000 -0500
@@ -77,7 +77,9 @@
 endif
 
 STAGING_SUBDIR = usr/$(GNU_TARGET_NAME)/sysroot
+ifneq ($(BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN),y)
 STAGING_DIR    = $(HOST_DIR)/$(STAGING_SUBDIR)
+endif
 
 TARGET_OPTIMIZATION:=$(call qstrip,$(BR2_TARGET_OPTIMIZATION))
 
@@ -161,6 +163,11 @@
 TARGET_CC_NOCCACHE  := $(TARGET_CC)
 TARGET_CXX_NOCCACHE := $(TARGET_CXX)
 
+ifeq ($(BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN),y)
+TARGET_CROSS=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PATH))/bin/$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PREFIX))-
+STAGING_DIR    := $(shell $(TARGET_CC) --print-sysroot)
+endif
+
 ifeq ($(BR2_CCACHE),y)
 TARGET_CC  := $(CCACHE) $(TARGET_CC)
 TARGET_CXX := $(CCACHE) $(TARGET_CXX)
diff -Naur a/toolchain/helpers.mk b/toolchain/helpers.mk
--- a/toolchain/helpers.mk	2013-09-17 06:42:07.000000000 -0500
+++ b/toolchain/helpers.mk	2013-09-26 11:16:53.000000000 -0500
@@ -81,6 +81,10 @@
  \
 	echo -n
 
+ifeq ($(BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN),y)
+	copy_toolchain_lib_root := echo "Not doing copy toolchain" ; exit ; $(copy_toolchain_lib_root)
+endif
+
 #
 # Copy the full external toolchain sysroot directory to the staging
 # dir. The operation of this function is rendered a little bit
@@ -160,6 +164,9 @@
 	fi ; \
 	find $(STAGING_DIR) -type d | xargs chmod 755
 
+ifeq ($(BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN),y)
+	copy_toolchain_sysroot := echo "Not doing a copy" ; exit ; $(copy_toolchain_sysroot)
+endif
 #
 # Check the availability of a particular glibc feature. This function
 # is used to check toolchain options that are always supported by
diff -Naur a/toolchain/toolchain-external/Config.in b/toolchain/toolchain-external/Config.in
--- a/toolchain/toolchain-external/Config.in	2013-09-17 06:42:07.000000000 -0500
+++ b/toolchain/toolchain-external/Config.in	2013-09-25 16:33:48.000000000 -0500
@@ -948,6 +948,15 @@
 	  here, separated by spaces. They will be copied to the
 	  target's /lib directory.
 
+config BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN
+	bool "Copy Libraries from Buildroot into toolchain sysroot"
+	help
+	  Select this option if you want to have Buildroot copy
+	  the generated libraries you have selected into the
+	  External toolchain.  This is useful if you want to make
+	  a fully loaded toolchain (with no external sysroot) for
+	  building standalone applications outside of Buildroot.
+
 endif # BR2_TOOLCHAIN_EXTERNAL_CUSTOM
 
 config BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[Peter Korsgaard]

>>>>> "ANDY" == ANDY KENNEDY <ANDY.KENNEDY@adtran.com> writes:

 ANDY> Sometimes it is useful to self contain a toolchain.  To do this, one
 ANDY> must somehow build in the extra libraries needed to make the toolchain
 ANDY> complete.  This patch adjusts Buildroot to, by configuration, modify the
 ANDY> external toolchain while building additional libraries.

I'm not sure I understand the use case here. How is it different that
setting BR2_HOST_DIR to something, do a build and then use the toolchain
in BR2_HOST_DIR/usr/bin outside buildroot?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

> -----Original Message-----
> From: Peter Korsgaard [mailto:jacmet at gmail.com] On Behalf Of Peter Korsgaard
> Sent: Thursday, September 26, 2013 3:28 PM
> To: ANDY KENNEDY
> Cc: 'buildroot at busybox.net'
> Subject: Re: [PATCH] Allow Buildroot to update toolchain
> 
> >>>>> "ANDY" == ANDY KENNEDY <ANDY.KENNEDY@adtran.com> writes:
> 
>  ANDY> Sometimes it is useful to self contain a toolchain.  To do this, one
>  ANDY> must somehow build in the extra libraries needed to make the toolchain
>  ANDY> complete.  This patch adjusts Buildroot to, by configuration, modify the
>  ANDY> external toolchain while building additional libraries.
> 
> I'm not sure I understand the use case here. How is it different that
> setting BR2_HOST_DIR to something, do a build and then use the toolchain
> in BR2_HOST_DIR/usr/bin outside buildroot?

Oh, my bad.  I mean for when one uses an external toolchain.  For example:

I build a toolchain using Crosstool-NG.  I have a few libraries in there, but
to fully use this as a cross compiler I need things like ncurses, pcap, lttng,
etc.  In this case, I would either need to have some sort of SYSROOT configured.
It is not always desirable to have those two items disjoined.  So, I dump my
additional libraries into the toolchain.

I've been carrying this patch along for a long time now (3 years +), but I
thought maybe someone else would like the ability to do so as well.

I'm sure I haven't done the description in the Config.in file justice.  I
toyed with the idea of make some comment that shows up if this is selected
to warn of what the person was actually doing, but I thought better of it.

Another concern I have is that, if one builds using an external toolchain and
does a 'make allyesconfig' then this option would also be selected, thereby
polluting their original toolchain.

I dunno, if you don't like this, I'll continue to drag it along outside of
BR.

Andy

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[Thomas Petazzoni]

Dear ANDY KENNEDY,

On Thu, 26 Sep 2013 20:34:40 +0000, ANDY KENNEDY wrote:

> Oh, my bad.  I mean for when one uses an external toolchain.  For
> example:
> 
> I build a toolchain using Crosstool-NG.  I have a few libraries in
> there, but to fully use this as a cross compiler I need things like
> ncurses, pcap, lttng, etc.  In this case, I would either need to have
> some sort of SYSROOT configured. It is not always desirable to have
> those two items disjoined.  So, I dump my additional libraries into
> the toolchain.

I still don't understand. What you're explaining is exactly what
Buildroot is doing today. When you use an external toolchain, Buildroot
copies the toolchain sysroot (which contains just the C library) in
output/staging and then progressively "improves" this sysroot by adding
all the libraries that have been cross-compiled for your target.

If what you need is then the ability to deliver this toolchain to
application developers who will not be using Buildroot, but need a
toolchain with all the libraries, then the proper way to do this is, as
Peter suggested, to do BR2_HOST_DIR=/opt/some-location/, then do your
Buildroot build, tarball /opt/some-location/ and deliver that as the
toolchain to be used by your application developers.

Or maybe I misunderstanding the use case?

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

> I still don't understand. What you're explaining is exactly what
> Buildroot is doing today. When you use an external toolchain, Buildroot
> copies the toolchain sysroot (which contains just the C library) in
> output/staging and then progressively "improves" this sysroot by adding
> all the libraries that have been cross-compiled for your target.
> 
> If what you need is then the ability to deliver this toolchain to
> application developers who will not be using Buildroot, but need a
> toolchain with all the libraries, then the proper way to do this is, as
> Peter suggested, to do BR2_HOST_DIR=/opt/some-location/, then do your
> Buildroot build, tarball /opt/some-location/ and deliver that as the
> toolchain to be used by your application developers.

Oh, really?  I totally missed that then.  I remember when that came
out (like last year sometime, right?) but I didn't really look into
it that much.

Well, cool!  I guess I reinvented the wheel on this one, eh?

I guess I'll try your way then and see what happens ;).

If it works exactly as what I'm doing that's cool!

> 
> Or maybe I misunderstanding the use case?

Um, yeah, that's it.  It isn't that I'm goofy or anything ;).

Andy

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

> I still don't understand. What you're explaining is exactly what
> Buildroot is doing today. When you use an external toolchain, Buildroot
> copies the toolchain sysroot (which contains just the C library) in
> output/staging and then progressively "improves" this sysroot by adding
> all the libraries that have been cross-compiled for your target.
> 
> If what you need is then the ability to deliver this toolchain to
> application developers who will not be using Buildroot, but need a
> toolchain with all the libraries, then the proper way to do this is, as
> Peter suggested, to do BR2_HOST_DIR=/opt/some-location/, then do your
> Buildroot build, tarball /opt/some-location/ and deliver that as the
> toolchain to be used by your application developers.

Thomas/Peter/all:

The following blurb (from the help menu) doesn't really say what
you said above:

Directory to store all the binary files that are built for the host.
This includes the cross compilation toolchain when building the
internal buildroot toolchain.

What the above says to me is that aclocal, automake, bison, flex,
etc will end up in my toolchain.  I don't really want this.

Now, if using an external toolchain changes this statement to what
you said "then progressively "improves" this sysroot by adding
all the libraries that have been cross-compiled for your target."
without making a bunch of additional build tools in there (which
won't really be needed in the follow-on builds by developers not
using Buildroot), then we probably need to specify this.  Otherwise,
my patch ONLY updates the sysroot and does not dump a bunch of
library compile time junk into the toolchain (never to be used
again).

Andy

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

> > I still don't understand. What you're explaining is exactly what
> > Buildroot is doing today. When you use an external toolchain, Buildroot
> > copies the toolchain sysroot (which contains just the C library) in
> > output/staging and then progressively "improves" this sysroot by adding
> > all the libraries that have been cross-compiled for your target.
> >
> > If what you need is then the ability to deliver this toolchain to
> > application developers who will not be using Buildroot, but need a
> > toolchain with all the libraries, then the proper way to do this is, as
> > Peter suggested, to do BR2_HOST_DIR=/opt/some-location/, then do your
> > Buildroot build, tarball /opt/some-location/ and deliver that as the
> > toolchain to be used by your application developers.
> 
> Thomas/Peter/all:
> 
> The following blurb (from the help menu) doesn't really say what
> you said above:
> 
> Directory to store all the binary files that are built for the host.
> This includes the cross compilation toolchain when building the
> internal buildroot toolchain.
> 
> What the above says to me is that aclocal, automake, bison, flex,
> etc will end up in my toolchain.  I don't really want this.
> 
> Now, if using an external toolchain changes this statement to what
> you said "then progressively "improves" this sysroot by adding
> all the libraries that have been cross-compiled for your target."
> without making a bunch of additional build tools in there (which
> won't really be needed in the follow-on builds by developers not
> using Buildroot), then we probably need to specify this.  Otherwise,
> my patch ONLY updates the sysroot and does not dump a bunch of
> library compile time junk into the toolchain (never to be used
> again).
> 

Okay, I see the difference between my patch and what HOST_DIR does.
When using HOST_DIR, the normal location of output/host is redirected
to /opt/somedir then the toolchain, all existing libs, all buildroot
built libs get dumped into that location.

In my patch, I have made this assumption:  The user already has
a toolchain in /opt/somedir/mytoolchain and desires to keep that
location.  After looking at what the code does, I don't really think
that I can specify the SAME location for HOST_DIR as it would attempt
to copy files onto itself (which would then, presumably, error out).
In the case of using the SYSROOT as the staging directory, ONLY the
libraries get copied back to the toolchain's sysroot (and the really
nice thing about this is that the user never has to tell me where to
put the stuff, I asked the toolchain for the sysroot dir).

So, as I stated before, I'll keep dragging this patch around if you
don't want to include it into Buildroot, but I believe this may be
something that others would like to use.  But, I've been wrong
MANY times before ;).

Andy

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[Thomas Petazzoni]

Dear ANDY KENNEDY,

On Fri, 27 Sep 2013 15:41:36 +0000, ANDY KENNEDY wrote:

> Okay, I see the difference between my patch and what HOST_DIR does.
> When using HOST_DIR, the normal location of output/host is redirected
> to /opt/somedir then the toolchain, all existing libs, all buildroot
> built libs get dumped into that location.
> 
> In my patch, I have made this assumption:  The user already has
> a toolchain in /opt/somedir/mytoolchain and desires to keep that
> location.  After looking at what the code does, I don't really think
> that I can specify the SAME location for HOST_DIR as it would attempt
> to copy files onto itself (which would then, presumably, error out).
> In the case of using the SYSROOT as the staging directory, ONLY the
> libraries get copied back to the toolchain's sysroot (and the really
> nice thing about this is that the user never has to tell me where to
> put the stuff, I asked the toolchain for the sysroot dir).
> 
> So, as I stated before, I'll keep dragging this patch around if you
> don't want to include it into Buildroot, but I believe this may be
> something that others would like to use.  But, I've been wrong
> MANY times before ;).

I still don't quite understand your use case. If you don't use
BR2_HOST_DIR=/opt/someplace/ and keep telling your users to use their
original toolchain, they have to pass many options to the compiler and
linker to point to the right library locations. They also don't have
access to the right pkg-config that works in cross-compilation mode
that Buildroot has built.

I *really* believe you should have a deeper look at what it means to
specify a custom BR2_HOST_DIR, I think it's really what you should be
using.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

Offline, no need to clutter the list. . .
 
(Well, it was off-line, but Thomas refused to answer this without
it being posted to the list, so, sorry for the clutter).

> I still don't quite understand your use case. If you don't use
> BR2_HOST_DIR=/opt/someplace/ and keep telling your users to use their
> original toolchain, they have to pass many options to the compiler and
> linker to point to the right library locations. They also don't have
> access to the right pkg-config that works in cross-compilation mode
> that Buildroot has built.

Never have I had a problem using the libraries that I build into the
toolchain using this same procedure.  I have been doing this patch for
3+ years (I think it is actually going on 5 years now, with a couple of
revisions to the patch along the way).  I plug the libraries straight
into the EXISTING toolchain's sysroot.  Viola!  I have my original
toolchain location with a set of libraries the toolchain automagically
finds based on the fact that all these libs are sitting right in the
same location as the other libs (that originally were built by
crosstool-ng).

> 
> I *really* believe you should have a deeper look at what it means to
> specify a custom BR2_HOST_DIR, I think it's really what you should be
> using.

You assume too much.  I did try it.  I didn't like that additional
crap that was put in the host directory.  I don't need flex, bison,
yacc, make, automake, autoconf, etc in my toolchain.  I just don't need
that.  That takes up too much space on our servers when it is far better
to use the patch I sent.

Perhaps you should attempt my patch once (using it in the prescribed
method) to see the difference yourself.  What I did was to build the
original toolchain using Crosstool-NG, tar that up, then use that
tarball as the starting point.  After this, I built two versions of
the toolchain.  The first I started by using the unmodified 2013.08.1
Buildroot code.  After it was completed, I moved the two toolchains
to another directory (for safe keepings) untarred the toolchain again
and did another build (starting from a clean Buildroot directory,
then applying my patch) and build again using the same configuration
but setting HOST_DIR back to $(BUILD_DIR)/host and setting the new
configuration option (using menuconfig -- I'm lazy)
BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN to yes.

On the HOST_DIR side of the house, you end up with two directories
that contain most the same information, however, there are additional
binaries for the libs, header applications for <gcc family>, <binutils
family>, etc.  All in all, I ended up with (the combined two
directories) with about 283% the size of the original toolchain.

Using my method, I ended up with a directory with about 255% of
the original size.

/opt/toolchains# ls
myway/  orig/  powerpc-unknown-linux-gnuspe/  powerpc-unknown-linux-gnuspe-testing.tar  yourway/
/opt/toolchains# du -s yourway myway orig
1098948 yourway
991592  myway
387772  orig
/opt/toolchains# ls myway/ yourway/ orig/
myway/:
powerpc-unknown-linux-gnuspe-testing/

orig/:
powerpc-unknown-linux-gnuspe-testing/

yourway/:
buildroot-toolchain/  powerpc-unknown-linux-gnuspe-testing/

Now do you understand the difference between the two patches?

As far as a different use case, the only thing I can say is that I
build stuff into the original sysroot of the toolchain, such that the
toolchain KNOWS where the libs are, knows where the includes are, etc
without the aid of any additional support utilities.  Whether this
is beneficial to you or the rest of the world, I cannot say.  It is,
however, beneficial to me.

BTW, MAN!!! Ya'll really pound on me with every patch I push back
up, don't you!  If I wasn't so dedicated to furthering Linux I'd
probably abandon ship with this whole "push stuff back" idea.

Andy

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[Thomas Petazzoni]

Andy,

On Mon, 30 Sep 2013 15:40:23 +0000, ANDY KENNEDY wrote:
> Offline, no need to clutter the list. . .
>  
> (Well, it was off-line, but Thomas refused to answer this without
> it being posted to the list, so, sorry for the clutter).

Yes, I don't see much point in having private discussions about
Buildroot design decisions. The opinion of the entire community is
important.

> > I still don't quite understand your use case. If you don't use
> > BR2_HOST_DIR=/opt/someplace/ and keep telling your users to use their
> > original toolchain, they have to pass many options to the compiler and
> > linker to point to the right library locations. They also don't have
> > access to the right pkg-config that works in cross-compilation mode
> > that Buildroot has built.
> 
> Never have I had a problem using the libraries that I build into the
> toolchain using this same procedure.

That's because it works in your particular case, but it doesn't work in
the general case. For example, with your solution, try to build an
application which uses 'pkg-config' to discover libraries. It will
fail, because you don't have $(HOST_DIR)/usr/bin/pkg-config, which
Buildroot has specially compiled to make sure it points to the
cross-compiled libraries and headers.

>  I have been doing this patch for
> 3+ years (I think it is actually going on 5 years now, with a couple of
> revisions to the patch along the way).  I plug the libraries straight
> into the EXISTING toolchain's sysroot.  Viola!  I have my original
> toolchain location with a set of libraries the toolchain automagically
> finds based on the fact that all these libs are sitting right in the
> same location as the other libs (that originally were built by
> crosstool-ng).

Again, that all works when you consider a toolchain as being just the
cross-compiler (and its related linker, assembler, etc.) and the
headers/libraries.

Most build systems, including Buildroot, consider that what we should
provide to application developers is actually a SDK: it's not only a
toolchain, but also a set of tools compiled for the host that are
useful during the cross-compilation process of various components.
pkg-config was mentioned above, but even having a version of autoconf,
automake and libtool that is the same for all developers is a really
nice feature. I've worked for customers that were using Buildroot, and
it was definitely a clear feature that Buildroot provides its own
version of automake, autoconf, libtool, pkg-config and other host
tools, since it ensures all the application developers rely on the same
version of those tools.

> > I *really* believe you should have a deeper look at what it means to
> > specify a custom BR2_HOST_DIR, I think it's really what you should be
> > using.
> 
> You assume too much.  I did try it.  I didn't like that additional
> crap that was put in the host directory.  I don't need flex, bison,
> yacc, make, automake, autoconf, etc in my toolchain.  I just don't need
> that.  That takes up too much space on our servers when it is far better
> to use the patch I sent.

See the above explanation.

> Perhaps you should attempt my patch once (using it in the prescribed
> method) to see the difference yourself.  What I did was to build the
> original toolchain using Crosstool-NG, tar that up, then use that
> tarball as the starting point.  After this, I built two versions of
> the toolchain.  The first I started by using the unmodified 2013.08.1
> Buildroot code.  After it was completed, I moved the two toolchains
> to another directory (for safe keepings) untarred the toolchain again
> and did another build (starting from a clean Buildroot directory,
> then applying my patch) and build again using the same configuration
> but setting HOST_DIR back to $(BUILD_DIR)/host and setting the new
> configuration option (using menuconfig -- I'm lazy)
> BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN to yes.

I'm not sure to follow your procedure works. But clearly, a Buildroot
option that copies back the libraries into the external toolchain is
not acceptable: the external toolchain is not meant to be changed by
Buildroot. If you really want to do that, you can just take
output/host/usr/<tuple>/sysroot and use that to replace the original
toolchain sysroot.

> On the HOST_DIR side of the house, you end up with two directories
> that contain most the same information, however, there are additional
> binaries for the libs, header applications for <gcc family>, <binutils
> family>, etc.  All in all, I ended up with (the combined two
> directories) with about 283% the size of the original toolchain.
> 
> Using my method, I ended up with a directory with about 255% of
> the original size.
> 
> /opt/toolchains# ls
> myway/  orig/  powerpc-unknown-linux-gnuspe/  powerpc-unknown-linux-gnuspe-testing.tar  yourway/
> /opt/toolchains# du -s yourway myway orig
> 1098948 yourway
> 991592  myway

So you've saved 107 MB over 1 GB, that's about 10%. I don't see how it
matches the 283% and 255% numbers you've given above.

> 387772  orig
> /opt/toolchains# ls myway/ yourway/ orig/
> myway/:
> powerpc-unknown-linux-gnuspe-testing/
> 
> orig/:
> powerpc-unknown-linux-gnuspe-testing/
> 
> yourway/:
> buildroot-toolchain/  powerpc-unknown-linux-gnuspe-testing/
> 
> Now do you understand the difference between the two patches?

Which "two" patches ? You've submitted only one.

> As far as a different use case, the only thing I can say is that I
> build stuff into the original sysroot of the toolchain, such that the
> toolchain KNOWS where the libs are, knows where the includes are, etc
> without the aid of any additional support utilities.  Whether this
> is beneficial to you or the rest of the world, I cannot say.  It is,
> however, beneficial to me.
> 
> BTW, MAN!!! Ya'll really pound on me with every patch I push back
> up, don't you!  If I wasn't so dedicated to furthering Linux I'd
> probably abandon ship with this whole "push stuff back" idea.

I'm sorry if you feel this way. I believe the Buildroot community
accepts a huge number of patches from various contributors. See for
example the numbers I've shown at
http://free-electrons.com/blog/buildroot-community-activity/: the
number of contributors is increasing significantly. I'm reviewing the
patches from many other contributors, and their patches are being
merged.

However, you have to understand that touching to the core mechanisms of
Buildroot takes more time than getting package changes merged. This is
because we're paying close attention to keep Buildroot simple, and
avoid having multiple mechanisms to do the same thing, or mechanisms
that serve too specific use cases that we think shouldn't be handled by
Buildroot.

I personally believe that the existing Buildroot users like it because
it is quite simple to understand and easy to use. If we start to accept
all sorts of mechanisms without a good and sensible use case, a good
coherency with the other existing mechanisms, then we will loose this
simplicity, which is the core advantage of Buildroot.

Best regards,

Thomas Petazzoni
-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] Allow Buildroot to update toolchain

[ANDY KENNEDY]

> > Never have I had a problem using the libraries that I build into the
> > toolchain using this same procedure.
> 
> That's because it works in your particular case, but it doesn't work in
> the general case. For example, with your solution, try to build an
> application which uses 'pkg-config' to discover libraries. It will
> fail, because you don't have $(HOST_DIR)/usr/bin/pkg-config, which
> Buildroot has specially compiled to make sure it points to the
> cross-compiled libraries and headers.

Most of the time pkg-config is not needed as the code being built is
proprietary and the companies that are using these don't have knowledge
of how/when to use these utilities.

> > Perhaps you should attempt my patch once (using it in the prescribed
> > method) to see the difference yourself.  What I did was to build the
> > original toolchain using Crosstool-NG, tar that up, then use that
> > tarball as the starting point.  After this, I built two versions of
> > the toolchain.  The first I started by using the unmodified 2013.08.1
> > Buildroot code.  After it was completed, I moved the two toolchains
> > to another directory (for safe keepings) untarred the toolchain again
> > and did another build (starting from a clean Buildroot directory,
> > then applying my patch) and build again using the same configuration
> > but setting HOST_DIR back to $(BUILD_DIR)/host and setting the new
> > configuration option (using menuconfig -- I'm lazy)
> > BR2_TOOLCHAIN_COPY_LIBS_INTO_TOOLCHAIN to yes.
> 
> I'm not sure to follow your procedure works. But clearly, a Buildroot
> option that copies back the libraries into the external toolchain is
> not acceptable: the external toolchain is not meant to be changed by
> Buildroot. If you really want to do that, you can just take
> output/host/usr/<tuple>/sysroot and use that to replace the original
> toolchain sysroot.

Hmmm, I'm not sure that would be exactly what I was going for, but
I'd have to try it (and I don't really care at this point).  Other than
the size differences below, the nice to have is that the toolchain is
contained in one directory.  Whereas I could push the original toolchain
a level deep then locate the Buildroot host dir next to it in that dir,
this, I fear, would be confusing to those who consume the toolchains.  I
would expect the question of "which one am I supposed to use" to come
back to me.  Though, one could fix this by (as I do anyway) by providing
a Makefile in the toolchain directory which will generate an environment
shell script to "configure" the toolchain for a given shell.

But, anywho, it doesn't matter.

> > /opt/toolchains# du -s yourway myway orig
> > 1098948 yourway
> > 991592  myway
> > 387772  orig
> 
> So you've saved 107 MB over 1 GB, that's about 10%. I don't see how it
> matches the 283% and 255% numbers you've given above.

1098948/387772 = 2.8340055... * 100% = 283%
991592/387772 = 2.557152....  * 100% = 255% (I guess this should have
been 256% with the rounding -- I truncated, my bad).

> > Now do you understand the difference between the two patches?
> 
> Which "two" patches ? You've submitted only one.

The patch submitted about a year or two ago that moved around the
HOST_DIR stuff vs the one I submitted last week.

> However, you have to understand that touching to the core mechanisms of
> Buildroot takes more time than getting package changes merged. This is
> because we're paying close attention to keep Buildroot simple, and
> avoid having multiple mechanisms to do the same thing, or mechanisms
> that serve too specific use cases that we think shouldn't be handled by
> Buildroot.
> 
> I personally believe that the existing Buildroot users like it because
> it is quite simple to understand and easy to use. If we start to accept
> all sorts of mechanisms without a good and sensible use case, a good
> coherency with the other existing mechanisms, then we will loose this
> simplicity, which is the core advantage of Buildroot.

One cannot argue with the principles presented here.  The application of
such principles are where I have differing views.

As far as this patch goes, if all within the Buildroot community are
vehemently against such a patch, so be it.  I will not blow against
the wind.

::enter drama queen mode::
Defeated,
::exit drama queen mode::
Andy