[Buildroot] Generating external toolchains with Buildroot

[Buildroot] Generating external toolchains with Buildroot

[Thomas Petazzoni]

Hello,

In order to make the autobuilders toolchain available to everyone, I
had to generate tarballs of Buildroot toolchains. Basically, I just did
BR2_HOST_DIR=/opt/<some-toolchain-name> and built just the toolchain.
This has raised two problems.

Problem 1: everything is in usr/
================================

In the host directory, Buildroot installs everything under a usr/
subdirectory. So the toolchain binaries are available under
usr/bin/<foo>-gcc and al.

However, all other external toolchains (from Sourcery CodeBench,
Crosstool-NG, Blackfin, Linaro, etc.) all have the toolchain binaries
directly under bin/.

So the external toolchain code extracts the toolchain tarball into
host/opt/ext-toolchain/ and excepts to find the binaries in
host/opt/external-toolchain/bin/. Which is not the case with Buildroot
toolchain if you simply create a tarball of the host/ directory
contents.

For now, I've simply adjusted how I create the tarball so that the
tarball has everything without the usr/ sub-directory.

Question: should we be changing our host directory policy and use
--prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we
keep things as it is and leave with this little drawback ?

Problem 2: relocalability
=========================

The main problem though is the relocalability of the toolchain. We have
discussed in length the relocability of the Buildroot SDK (i.e, the
host/ directory) during previous Buildroot Developer meetings. For the
specific case of a SDK that contains only the toolchain, there is only
one problem I had: we build gmp, mpfr and gmp as shared libraries, and
the gcc binaries have absolute rpath in their binaries to reference the
host/usr/lib directory in which these shared libraries are installed.

We thought of solving this problem by using a relative rpath:
-Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for
binaries installed in host/usr/bin, but not for the cc1 and collect2
binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those
binaries would need a different relative rpath to work properly.

For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this
problem by modifying Buildroot so that it builds mpfr, gmp and mpc as
static libraries, so that the gcc binaries do not depend to any shared
libraries besides the C library. And this seems to work fine, though it
is not a completely satisfying solution.

Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and
-Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this
problem? Another solution?

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Generating external toolchains with Buildroot

[Arnout Vandecappelle]

  Underwhelming number of replies in this thread...

On 08/25/12 11:30, Thomas Petazzoni wrote:
> Hello,
>
> In order to make the autobuilders toolchain available to everyone, I
> had to generate tarballs of Buildroot toolchains. Basically, I just did
> BR2_HOST_DIR=/opt/<some-toolchain-name>  and built just the toolchain.
> This has raised two problems.
>
> Problem 1: everything is in usr/
> ================================
>
[snip]
>
> Question: should we be changing our host directory policy and use
> --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we
> keep things as it is and leave with this little drawback ?

  Yes!  I've always been annoyed with the redundant usr part, especially
since two or three packages install in $(HOST_DIR)/bin...

  It's a big change but can probably easily be automated with sed.


> Problem 2: relocalability
> =========================
>
> The main problem though is the relocalability of the toolchain. We have
> discussed in length the relocability of the Buildroot SDK (i.e, the
> host/ directory) during previous Buildroot Developer meetings. For the
> specific case of a SDK that contains only the toolchain, there is only
> one problem I had: we build gmp, mpfr and gmp as shared libraries, and
> the gcc binaries have absolute rpath in their binaries to reference the
> host/usr/lib directory in which these shared libraries are installed.
>
> We thought of solving this problem by using a relative rpath:
> -Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for
> binaries installed in host/usr/bin, but not for the cc1 and collect2
> binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those
> binaries would need a different relative rpath to work properly.
>
> For the toolchains that I've put online at
> http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this
> problem by modifying Buildroot so that it builds mpfr, gmp and mpc as
> static libraries, so that the gcc binaries do not depend to any shared
> libraries besides the C library. And this seems to work fine, though it
> is not a completely satisfying solution.

  I'm not sure if linking mpfr, gmp and mpc statically is so bad...  How
much larger do the binaries become with static linking?  If it's not much
larger, the startup of cc1 may actually become faster with static linking!

  Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M; statically-linked
cc1 from Sourcery's gcc-4.5.2 is 12M.

  I think crosstool-NG links statically as well.  Yann?


> Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and
> -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this
> problem? Another solution?

  Not very nice.  Perhaps a specific rpath for linking cc1 and friends?


  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Generating external toolchains with Buildroot

[Thomas Petazzoni]

Le Tue, 28 Aug 2012 09:36:40 +0200,
Arnout Vandecappelle <arnout@mind.be> a ?crit :

> > Problem 1: everything is in usr/
> > ================================
> >
> [snip]
> >
> > Question: should we be changing our host directory policy and use
> > --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or
> > should we keep things as it is and leave with this little drawback ?
> 
>   Yes!  I've always been annoyed with the redundant usr part,
> especially since two or three packages install in $(HOST_DIR)/bin...

Ah, really? Which ones?

>   It's a big change but can probably easily be automated with sed.

Well, most of the host packages are probably using the
host-autotools-package infrastructure, so those are easy to change:
only one place. Of course, all the host-generic-package would need
manual intervention.

Peter, what's your feeling about this?

> > For the toolchains that I've put online at
> > http://autobuild.buildroot.org/toolchains/tarballs/, I've solved
> > this problem by modifying Buildroot so that it builds mpfr, gmp and
> > mpc as static libraries, so that the gcc binaries do not depend to
> > any shared libraries besides the C library. And this seems to work
> > fine, though it is not a completely satisfying solution.
> 
>   I'm not sure if linking mpfr, gmp and mpc statically is so bad...

No, it's not so bad. It's just that from a "beauty" point of view, I
would have preferred to be able to link dynamically against them. Just
because we _should_ be able to do it :-)

> How much larger do the binaries become with static linking?  If it's
> not much larger, the startup of cc1 may actually become faster with
> static linking!
> 
>   Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M;
> statically-linked cc1 from Sourcery's gcc-4.5.2 is 12M.

Yes, the impact is not that significant, you're right.

>   I think crosstool-NG links statically as well.  Yann?

Yes, crosstool-NG links statically.

> > Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib
> > and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve
> > this problem? Another solution?
> 
>   Not very nice.  Perhaps a specific rpath for linking cc1 and
> friends?

That means diving into the internal gcc build process. You, crazy
guy? :-)

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Generating external toolchains with Buildroot

[Kevin Chadwick]

> For the toolchains that I've put online at
> http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this
> problem by modifying Buildroot so that it builds mpfr, gmp and mpc as
> static libraries, so that the gcc binaries do not depend to any shared
> libraries besides the C library. And this seems to work fine, though it
> is not a completely satisfying solution.

What's the draw back there?

-- 
_______________________________________________________________________

'Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface'

(Doug McIlroy)
_______________________________________________________________________