[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Bryce Schober]

What I mean by "SDK" is the ability to compile external programs using the
buildroot toolchain and libraries on any reasonably-compatible host. I'm
assuming that the toolchain itself is built properly static for its own
host independence (which in my case, it is). Ideally the toolchain in the
output/host directory would be capable of standing alone, when built
properly

I'm having two issues w/ archving the output/host dir for extraction and
usage in a different host directory. I'm using a custom-built crosstool-ng
built externally and independent of buildroot.

The external toolchain import results in three distinct problems that I can
see:
1. The symbolic links for binutils binaries in use absolute paths to link
to the external toolchain instead of copying them in.
2. The ext-toolchain-wrapper uses an absolute path for the sysroot
directory.
3. The ext-toolchain-wrapper seems to point to the old (import-from) bin
directory, not the path to which the toolchain has been copied.

I'm not particularly familiar with the relocatability of toolchains or
buildroot's method thereof, and so am not sure where to start with a
proposal, let alone a patch. If I could get some direction as to how best
to tackle this, I'd be grateful.

--
Bryce Schober
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120109/c3ac3fdc/attachment.html>

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Thomas De Schampheleire]

Hi Bryce,

On Mon, Jan 9, 2012 at 11:54 PM, Bryce Schober <bryce.schober@gmail.com> wrote:
> What I mean by "SDK" is the ability to compile external programs using the
> buildroot toolchain and libraries on any reasonably-compatible host. I'm
> assuming that the toolchain itself is built properly static for its own host
> independence (which in my case, it is).?Ideally the toolchain in the
> output/host directory would be capable of standing alone, when built
> properly
>
> I'm having two issues w/ archving the?output/host dir for extraction and
> usage in a different host directory. I'm using a custom-built crosstool-ng
> built externally and independent of buildroot.
>
> The external toolchain import results in three distinct problems that I can
> see:
> 1. The symbolic links for binutils binaries in use absolute paths to link to
> the external toolchain instead of copying them in.
> 2. The ext-toolchain-wrapper uses an absolute path for the sysroot
> directory.
> 3. The ext-toolchain-wrapper seems to point to the old (import-from) bin
> directory, not the path to which the toolchain has been copied.
>
> I'm not particularly familiar with the relocatability of toolchains or
> buildroot's method thereof, and so am not sure where to start with a
> proposal, let alone a patch. If I could get some direction as to how best to
> tackle this, I'd be grateful.

It is known that the toolchains generated by buildroot are not relocatable.
However, you can tell buildroot to create the toolchain with
crosstool-ng, and this one is relocatable. I suggest you give that a
try to see if it solves your problems.
I'm using buildroot in the same way and it works fine for me.

Best regards,
Thomas

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Bryce Schober]

As I said in my original post, I am using an externally generated and
configured (and statically linked) crosstool-ng toolchain. I believe that
it is the buildroot import thereof that is at fault. Again, in summary:
1. Buildroot generates absolute-pathed symbolic links to the external
binutils in output/host/usr/bin instead of copying them in.
2. Buildroot's ext-toolchain-wrapper uses an absolute path for its sysroot
configuration.
3. Buildroot's ext-toolchain-wrapper uses an absolute path for its binary
exec path.

I realize that these problems are triggered by my specification of:
BR2_TOOLCHAIN_EXTERNAL_PATH="$(TOPDIR)/../my-ext-toolchain-dir/"
But I can't specify merely "../my-ext-toolchain-dir/" without breaking
things either.

<><  <><  <><
Bryce Schober



On Tue, Jan 10, 2012 at 3:26 AM, Thomas De Schampheleire <
patrickdepinguin+buildroot@gmail.com> wrote:

> Hi Bryce,
>
> On Mon, Jan 9, 2012 at 11:54 PM, Bryce Schober <bryce.schober@gmail.com>
> wrote:
> > What I mean by "SDK" is the ability to compile external programs using
> the
> > buildroot toolchain and libraries on any reasonably-compatible host. I'm
> > assuming that the toolchain itself is built properly static for its own
> host
> > independence (which in my case, it is). Ideally the toolchain in the
> > output/host directory would be capable of standing alone, when built
> > properly
> >
> > I'm having two issues w/ archving the output/host dir for extraction and
> > usage in a different host directory. I'm using a custom-built
> crosstool-ng
> > built externally and independent of buildroot.
> >
> > The external toolchain import results in three distinct problems that I
> can
> > see:
> > 1. The symbolic links for binutils binaries in use absolute paths to
> link to
> > the external toolchain instead of copying them in.
> > 2. The ext-toolchain-wrapper uses an absolute path for the sysroot
> > directory.
> > 3. The ext-toolchain-wrapper seems to point to the old (import-from) bin
> > directory, not the path to which the toolchain has been copied.
> >
> > I'm not particularly familiar with the relocatability of toolchains or
> > buildroot's method thereof, and so am not sure where to start with a
> > proposal, let alone a patch. If I could get some direction as to how
> best to
> > tackle this, I'd be grateful.
>
> It is known that the toolchains generated by buildroot are not relocatable.
> However, you can tell buildroot to create the toolchain with
> crosstool-ng, and this one is relocatable. I suggest you give that a
> try to see if it solves your problems.
> I'm using buildroot in the same way and it works fine for me.
>
> Best regards,
> Thomas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120110/6148e478/attachment-0001.html>

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Thomas De Schampheleire]

On Tue, Jan 10, 2012 at 9:22 PM, Bryce Schober <bryce.schober@gmail.com> wrote:
> As I said in my original post, I am using an externally generated and
> configured (and statically linked) crosstool-ng toolchain.

I'm sorry, I read over that part.

> I believe that it
> is the buildroot import thereof that is at fault. Again, in summary:
> 1. Buildroot generates absolute-pathed symbolic links to the external
> binutils in output/host/usr/bin instead of copying them in.
> 2. Buildroot's ext-toolchain-wrapper uses an absolute path for its sysroot
> configuration.
> 3. Buildroot's ext-toolchain-wrapper uses?an?absolute path for its binary
> exec path.
>
> I realize that these problems are triggered by my specification of:
> BR2_TOOLCHAIN_EXTERNAL_PATH="$(TOPDIR)/../my-ext-toolchain-dir/"
> But I can't specify merely "../my-ext-toolchain-dir/" without breaking
> things either.
>

Have you tried using BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD instead? This
requires you to archive the crosstool-ng toolchain and specify it to
buildroot. Buildroot will then extract it and unpack. The binutils
will be copied in this case, and not symlinked.

This is not possibly in stock buildroot. I have submitted patches for
this a while ago, but they are not yet included. I think there was a
comment by Thomas that I still have to fix. In the mean time, try
using patches 1 and 2 of the following series:
http://lists.busybox.net/pipermail/buildroot/2011-August/044779.html

Best regards,
Thomas


>
> On Tue, Jan 10, 2012 at 3:26 AM, Thomas De Schampheleire
> <patrickdepinguin+buildroot@gmail.com> wrote:
>>
>> Hi Bryce,
>>
>> On Mon, Jan 9, 2012 at 11:54 PM, Bryce Schober <bryce.schober@gmail.com>
>> wrote:
>> > What I mean by "SDK" is the ability to compile external programs using
>> > the
>> > buildroot toolchain and libraries on any reasonably-compatible host. I'm
>> > assuming that the toolchain itself is built properly static for its own
>> > host
>> > independence (which in my case, it is).?Ideally the toolchain in the
>> > output/host directory would be capable of standing alone, when built
>> > properly
>> >
>> > I'm having two issues w/ archving the?output/host dir for extraction and
>> > usage in a different host directory. I'm using a custom-built
>> > crosstool-ng
>> > built externally and independent of buildroot.
>> >
>> > The external toolchain import results in three distinct problems that I
>> > can
>> > see:
>> > 1. The symbolic links for binutils binaries in use absolute paths to
>> > link to
>> > the external toolchain instead of copying them in.
>> > 2. The ext-toolchain-wrapper uses an absolute path for the sysroot
>> > directory.
>> > 3. The ext-toolchain-wrapper seems to point to the old (import-from) bin
>> > directory, not the path to which the toolchain has been copied.
>> >
>> > I'm not particularly familiar with the relocatability of toolchains or
>> > buildroot's method thereof, and so am not sure where to start with a
>> > proposal, let alone a patch. If I could get some direction as to how
>> > best to
>> > tackle this, I'd be grateful.
>>
>> It is known that the toolchains generated by buildroot are not
>> relocatable.
>> However, you can tell buildroot to create the toolchain with
>> crosstool-ng, and this one is relocatable. I suggest you give that a
>> try to see if it solves your problems.
>> I'm using buildroot in the same way and it works fine for me.
>>
>> Best regards,
>> Thomas
>
>

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Michael S. Zick]

On Wed January 11 2012, Thomas De Schampheleire wrote:
> On Tue, Jan 10, 2012 at 9:22 PM, Bryce Schober <bryce.schober@gmail.com> wrote:
> > As I said in my original post, I am using an externally generated and
> > configured (and statically linked) crosstool-ng toolchain.
> 
> I'm sorry, I read over that part.
> 
> > I believe that it
> > is the buildroot import thereof that is at fault. Again, in summary:
> > 1. Buildroot generates absolute-pathed symbolic links to the external
> > binutils in output/host/usr/bin instead of copying them in.
> > 2. Buildroot's ext-toolchain-wrapper uses an absolute path for its sysroot
> > configuration.
> > 3. Buildroot's ext-toolchain-wrapper uses?an?absolute path for its binary
> > exec path.
> >
> > I realize that these problems are triggered by my specification of:
> > BR2_TOOLCHAIN_EXTERNAL_PATH="$(TOPDIR)/../my-ext-toolchain-dir/"
> > But I can't specify merely "../my-ext-toolchain-dir/" without breaking
> > things either.
> >
> 
> Have you tried using BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD instead? This
> requires you to archive the crosstool-ng toolchain and specify it to
> buildroot. Buildroot will then extract it and unpack. The binutils
> will be copied in this case, and not symlinked.
> 
> This is not possibly in stock buildroot. I have submitted patches for
> this a while ago, but they are not yet included. I think there was a
> comment by Thomas that I still have to fix. In the mean time, try
> using patches 1 and 2 of the following series:
> http://lists.busybox.net/pipermail/buildroot/2011-August/044779.html
>

The best kind of 'bump' for things in the "Pending" box,
somebody needs it. 

Shall we invent a 'Needed-by: ... ' attribute for the pending patches?

Mike
> Best regards,
> Thomas
> 
> 
> >
> > On Tue, Jan 10, 2012 at 3:26 AM, Thomas De Schampheleire
> > <patrickdepinguin+buildroot@gmail.com> wrote:
> >>
> >> Hi Bryce,
> >>
> >> On Mon, Jan 9, 2012 at 11:54 PM, Bryce Schober <bryce.schober@gmail.com>
> >> wrote:
> >> > What I mean by "SDK" is the ability to compile external programs using
> >> > the
> >> > buildroot toolchain and libraries on any reasonably-compatible host. I'm
> >> > assuming that the toolchain itself is built properly static for its own
> >> > host
> >> > independence (which in my case, it is).?Ideally the toolchain in the
> >> > output/host directory would be capable of standing alone, when built
> >> > properly
> >> >
> >> > I'm having two issues w/ archving the?output/host dir for extraction and
> >> > usage in a different host directory. I'm using a custom-built
> >> > crosstool-ng
> >> > built externally and independent of buildroot.
> >> >
> >> > The external toolchain import results in three distinct problems that I
> >> > can
> >> > see:
> >> > 1. The symbolic links for binutils binaries in use absolute paths to
> >> > link to
> >> > the external toolchain instead of copying them in.
> >> > 2. The ext-toolchain-wrapper uses an absolute path for the sysroot
> >> > directory.
> >> > 3. The ext-toolchain-wrapper seems to point to the old (import-from) bin
> >> > directory, not the path to which the toolchain has been copied.
> >> >
> >> > I'm not particularly familiar with the relocatability of toolchains or
> >> > buildroot's method thereof, and so am not sure where to start with a
> >> > proposal, let alone a patch. If I could get some direction as to how
> >> > best to
> >> > tackle this, I'd be grateful.
> >>
> >> It is known that the toolchains generated by buildroot are not
> >> relocatable.
> >> However, you can tell buildroot to create the toolchain with
> >> crosstool-ng, and this one is relocatable. I suggest you give that a
> >> try to see if it solves your problems.
> >> I'm using buildroot in the same way and it works fine for me.
> >>
> >> Best regards,
> >> Thomas
> >
> >
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Bryce Schober]

I just tried that, and I get the same results. Both the binutils symbolic
links and ext-toolchain-wrapper still use absolute paths. These patches
don't touch the logic used to assemble a full sysroot with the compiler and
buildroot host outputs in the same place.

It seems that the current importing and wrapping of any type of external
toolchain would have to be fundamentally re-worked in order to be
relocatable. It seems that two things would need to change:
1. Any symbolic links created would have to have any absolute paths
converted to relative.
2. Any paths used in the ext-toolchain-wrapper would have to be relative,
and the wrapper itself would have to determine the path to itself in order
to correctly exec the programs it wraps.

<><  <><  <><
Bryce Schober



On Tue, Jan 10, 2012 at 11:38 PM, Thomas De Schampheleire <
patrickdepinguin+buildroot@gmail.com> wrote:

> On Tue, Jan 10, 2012 at 9:22 PM, Bryce Schober <bryce.schober@gmail.com>
> wrote:
> > As I said in my original post, I am using an externally generated and
> > configured (and statically linked) crosstool-ng toolchain.
>
> I'm sorry, I read over that part.
>
> > I believe that it
> > is the buildroot import thereof that is at fault. Again, in summary:
> > 1. Buildroot generates absolute-pathed symbolic links to the external
> > binutils in output/host/usr/bin instead of copying them in.
> > 2. Buildroot's ext-toolchain-wrapper uses an absolute path for its
> sysroot
> > configuration.
> > 3. Buildroot's ext-toolchain-wrapper uses an absolute path for its binary
> > exec path.
> >
> > I realize that these problems are triggered by my specification of:
> > BR2_TOOLCHAIN_EXTERNAL_PATH="$(TOPDIR)/../my-ext-toolchain-dir/"
> > But I can't specify merely "../my-ext-toolchain-dir/" without breaking
> > things either.
> >
>
> Have you tried using BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD instead? This
> requires you to archive the crosstool-ng toolchain and specify it to
> buildroot. Buildroot will then extract it and unpack. The binutils
> will be copied in this case, and not symlinked.
>
> This is not possibly in stock buildroot. I have submitted patches for
> this a while ago, but they are not yet included. I think there was a
> comment by Thomas that I still have to fix. In the mean time, try
> using patches 1 and 2 of the following series:
> http://lists.busybox.net/pipermail/buildroot/2011-August/044779.html
>
> Best regards,
> Thomas
>
>
> >
> > On Tue, Jan 10, 2012 at 3:26 AM, Thomas De Schampheleire
> > <patrickdepinguin+buildroot@gmail.com> wrote:
> >>
> >> Hi Bryce,
> >>
> >> On Mon, Jan 9, 2012 at 11:54 PM, Bryce Schober <bryce.schober@gmail.com
> >
> >> wrote:
> >> > What I mean by "SDK" is the ability to compile external programs using
> >> > the
> >> > buildroot toolchain and libraries on any reasonably-compatible host.
> I'm
> >> > assuming that the toolchain itself is built properly static for its
> own
> >> > host
> >> > independence (which in my case, it is). Ideally the toolchain in the
> >> > output/host directory would be capable of standing alone, when built
> >> > properly
> >> >
> >> > I'm having two issues w/ archving the output/host dir for extraction
> and
> >> > usage in a different host directory. I'm using a custom-built
> >> > crosstool-ng
> >> > built externally and independent of buildroot.
> >> >
> >> > The external toolchain import results in three distinct problems that
> I
> >> > can
> >> > see:
> >> > 1. The symbolic links for binutils binaries in use absolute paths to
> >> > link to
> >> > the external toolchain instead of copying them in.
> >> > 2. The ext-toolchain-wrapper uses an absolute path for the sysroot
> >> > directory.
> >> > 3. The ext-toolchain-wrapper seems to point to the old (import-from)
> bin
> >> > directory, not the path to which the toolchain has been copied.
> >> >
> >> > I'm not particularly familiar with the relocatability of toolchains or
> >> > buildroot's method thereof, and so am not sure where to start with a
> >> > proposal, let alone a patch. If I could get some direction as to how
> >> > best to
> >> > tackle this, I'd be grateful.
> >>
> >> It is known that the toolchains generated by buildroot are not
> >> relocatable.
> >> However, you can tell buildroot to create the toolchain with
> >> crosstool-ng, and this one is relocatable. I suggest you give that a
> >> try to see if it solves your problems.
> >> I'm using buildroot in the same way and it works fine for me.
> >>
> >> Best regards,
> >> Thomas
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20120111/7ac2b48f/attachment.html>

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Peter Korsgaard]

>>>>> "Bryce" == Bryce Schober <bryce.schober@gmail.com> writes:

Hi,

 Bryce> I just tried that, and I get the same results. Both the binutils
 Bryce> symbolic links and?ext-toolchain-wrapper?still use absolute
 Bryce> paths. These patches don't touch the logic used to assemble a
 Bryce> full sysroot with the compiler and buildroot host outputs in the
 Bryce> same place.

 Bryce> It seems that the current importing and wrapping of any type of
 Bryce> external toolchain would have to be fundamentally re-worked in
 Bryce> order to be relocatable.

Sorry, I haven't read the entire thread, but are you aware of the
discussions we had at the Buildroot developers day in October?

Thomas made a nice summary of it:

http://lists.busybox.net/pipermail/buildroot/2011-November/047229.html

 Bryce> It seems that two things would need to change:
 Bryce> 1. Any symbolic links created would have to have any absolute paths converted
 Bryce> to relative.
 Bryce> 2. Any paths used in the ext-toolchain-wrapper would have to be relative, and
 Bryce> the wrapper itself would have to determine the path to itself in order to
 Bryce> correctly exec the programs it wraps.

Other things need to be fixed as well (as listed in the summary):

 - Host tools
 - .la files / cmake / qmake configuraiton
 - pkg-config
 - ..

So it's a fairly major change.

-- 
Bye, Peter Korsgaard

[Buildroot] Can't use output/host/ as "SDK" with external toolchain

[Thomas Petazzoni]

Hello Bryce,

Le Mon, 9 Jan 2012 14:54:13 -0800,
Bryce Schober <bryce.schober@gmail.com> a ?crit :

> What I mean by "SDK" is the ability to compile external programs
> using the buildroot toolchain and libraries on any
> reasonably-compatible host. I'm assuming that the toolchain itself is
> built properly static for its own host independence (which in my
> case, it is). Ideally the toolchain in the output/host directory
> would be capable of standing alone, when built properly
> 
> I'm having two issues w/ archving the output/host dir for extraction
> and usage in a different host directory. I'm using a custom-built
> crosstool-ng built externally and independent of buildroot.

As Peter has mentioned, moving the "SDK" to a different directory is
currently not supported. It has been discussed during our last meeting
in Prague, we listed the different things that we think need to be
fixed, but the work hasn't been done yet.

The meeting report that Peter has mentioned lists the different issues
that need to be fixed. So if you want to help, feel free to start with
this list :)

Those items are also on my TODO-list, but that one is so long that I
can't guarantee when I'll be able to do it.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com