[Buildroot] external toolchain and sysroot

[Buildroot] external toolchain and sysroot

[Yury Luneff]

Hello!

I'm warning, I'm new to buildroot.

I have built my toolchain with crosstool-NG with sysroot option 
enabled and i've chosen uclibc.

I'm trying to build my rootfs with this toolchain. I guess, I have 
proper buildroot configuration, but as soon as package has it 
dependency somewhere in new sysroot, I have linker problems:

/home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/bin/arm-indigo-linux-uclibcgnueabi-gcc 
-Os -pipe -Os  --sysroot /home/yury/buildroot/output/staging/  -DAUTOCONF -DPOSIX -DUSG -c window.c

I see --sysroot here and everything is ok.

But in linking phase:
/home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/bin/arm-indigo-linux-uclibcgnueabi-gcc 
-DAUTOCONF -DPOSIX -DUSG -o em ansi.o basic.o bind.o buffer.o crypt.o display.o eval.o exec.o 
file.o fileio.o ibmpc.o input.o isearch.o line.o lock.o main.o pklock.o posix.o random.o 
region.o search.o spawn.o tcap.o termio.o vmsvt.o vt52.o window.o word.o -lncurses
/home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/lib/gcc/arm-indigo-linux-uclibcgnueabi/4.3.4/../..
/../../arm-indigo-linux-uclibcgnueabi/bin/ld: cannot find -lncurses

here --sysroot is omitted and thus I'm very sad :-)

I can't figure this out by myself. TARGET_LDFLAGS are good, 
TARGET_CFLAGS are too.

It is definetely not the problem of package as openvpn doesn't find 
openssl too and the corresponding .so files are in staging. Or it is 
a bug in both uemacs and openvpn which is unprobable. I know there 
were problems with openvpn and external toolchains but it was kinda at 
least a year ago.

Buildroot is both latest stable version and the current snapshot -- 
the same problem.

How should I find my way through?

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-25, Yury Luneff <bitterman@ya.ru> wrote:

[...]

> I'm trying to build my rootfs with this toolchain. I guess, I have 
> proper buildroot configuration, but as soon as package has it 
> dependency somewhere in new sysroot, I have linker problems:
>
> /home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/bin/arm-indigo-linux-uclibcgnueabi-gcc 
> -Os -pipe -Os  --sysroot /home/yury/buildroot/output/staging/  -DAUTOCONF -DPOSIX -DUSG -c window.c
>
> I see --sysroot here and everything is ok.
>
> But in linking phase:
> /home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/bin/arm-indigo-linux-uclibcgnueabi-gcc 
> -DAUTOCONF -DPOSIX -DUSG -o em ansi.o basic.o bind.o buffer.o crypt.o display.o eval.o exec.o 
> file.o fileio.o ibmpc.o input.o isearch.o line.o lock.o main.o pklock.o posix.o random.o 
> region.o search.o spawn.o tcap.o termio.o vmsvt.o vt52.o window.o word.o -lncurses
> /home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/lib/gcc/arm-indigo-linux-uclibcgnueabi/4.3.4/../..
> /../../arm-indigo-linux-uclibcgnueabi/bin/ld: cannot find -lncurses
>
> here --sysroot is omitted and thus I'm very sad :-)

I know how you feel.  That's exactly the same problem I had building
busybox two weeks ago: --sysroot was passed correctly in the compile
phase, but wasn't passed in the link phase.  Apparently the external
toolchains used by the devs don't need a --sysroot during linking.
Apparently our toolchains do.

> I can't figure this out by myself. TARGET_LDFLAGS are good, 
> TARGET_CFLAGS are too.

In the busybox .mk file, I found that TARGET_LDFLAGS wasn't being used
(it wasn't being passed to the "make" command).  I had to patch the
.mk file to pass the TARGET_LDFLAGS value to make.

> It is definetely not the problem of package as openvpn doesn't find 
> openssl too and the corresponding .so files are in staging. Or it is 
> a bug in both uemacs and openvpn which is unprobable.

I don't think it's unprobably at all.  When I recently updated to the
current stable release, I found that both busybox and mtd-utils failed
to build because of missing "--sysroot" options in either compile or
link commands.  That's two packages out of the three that I have
selected that weren't passing a "--sysroot" option to the toolchain as
they should.

> I know there were problems with openvpn and external toolchains but
> it was kinda at least a year ago.
>
> Buildroot is both latest stable version and the current snapshot -- 
> the same problem.
>
> How should I find my way through?

It sounds like the .mk files are missing something.  See the following
bugs for patches that show how I fixed missing --sysroot problems for
busybox and mtd-utils:

  https://bugs.busybox.net/show_bug.cgi?id=1321
  https://bugs.busybox.net/show_bug.cgi?id=1327

-- 
Grant Edwards               grant.b.edwards        Yow! GOOD-NIGHT, everybody
                                  at               ... Now I have to go
                              gmail.com            administer FIRST-AID to my
                                                   pet LEISURE SUIT!!

[Buildroot] external toolchain and sysroot

[Yury Luneff]

> In the busybox .mk file, I found that TARGET_LDFLAGS wasn't being used
> (it wasn't being passed to the "make" command).  I had to patch the
> .mk file to pass the TARGET_LDFLAGS value to make.

>> It is definetely not the problem of package as openvpn doesn't find 
>> openssl too and the corresponding .so files are in staging. Or it is 
>> a bug in both uemacs and openvpn which is unprobable.

as for uemacs, its link command disregarded even CFLAGS, so I had to 
patch the makefile of uemacs. So I did the uemacs (just for fun, 
really, it's old like heaven and hell :-). Same problem, I guess. 
Shouldn't it be fixed project-wide? Documentations says that CT-NG is 
ok, but reality says something different.

As for openvpn, I guess the precise commands are common for autotools 
packages, so it is not quite so easy to find the right place to 
strike.

> It sounds like the .mk files are missing something.  See the following
> bugs for patches that show how I fixed missing --sysroot problems for
> busybox and mtd-utils:

Busybox is ok :(

[Buildroot] external toolchain and sysroot

[Yury Luneff]

i've fixed uemacs makefile.

And the other problem was not with openvpn but with vsftpd which .mk did 
not include "${TARGET_LDFLAGS}" at all.

It seem to be a common problem in buildroot for now.

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-25, Yury Luneff <bitterman@ya.ru> wrote:
> i've fixed uemacs makefile.
>
> And the other problem was not with openvpn but with vsftpd which .mk did 
> not include "${TARGET_LDFLAGS}" at all.
>
> It seem to be a common problem in buildroot for now.

Based on my sample of three packages (two had the problem), I'm
guessing it's a very common problem.  It looks like the buildroot
developers use external toolchains that don't require a --sysroot
option.  I don't know why that is, since external toolchains get
copied into staging before they are used.  That implies that they
won't work without a --sysroot option...

-- 
Grant Edwards               grant.b.edwards        Yow! My polyvinyl cowboy
                                  at               wallet was made in Hong
                              gmail.com            Kong by Montgomery Clift!

[Buildroot] external toolchain and sysroot

[Thomas Petazzoni]

On Thu, 25 Mar 2010 18:08:49 +0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> Based on my sample of three packages (two had the problem), I'm
> guessing it's a very common problem.  It looks like the buildroot
> developers use external toolchains that don't require a --sysroot
> option. 

They do. What we do to support external toolchains is :

 * We copy the full sysroot of the toolchain into $(STAGING_DIR)
 * We copy just the C library and related libraries from the sysroot to
   the $(TARGET_DIR)
 * We pass --sysroot $(STAGING_DIR) to every gcc/ld invocation, so that
   gcc/ld find the libraries and header files correctly.

Actually, this mechanism *only* works with toolchain supporting
--sysroot. Toolchains *not* supporting --sysroot are rejected (see
toolchain/external-toolchain/ext-tool.mk).

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-25, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> On Thu, 25 Mar 2010 18:08:49 +0000 (UTC)
> Grant Edwards <grant.b.edwards@gmail.com> wrote:
>
>> Based on my sample of three packages (two had the problem), I'm
>> guessing it's a very common problem.  It looks like the buildroot
>> developers use external toolchains that don't require a --sysroot
>> option. 
>
> They do.
>
> What we do to support external toolchains is :
>
>  * We copy the full sysroot of the toolchain into $(STAGING_DIR)
>  * We copy just the C library and related libraries from the sysroot to
>    the $(TARGET_DIR)

Agreed.

>  * We pass --sysroot $(STAGING_DIR) to every gcc/ld invocation, so that
>    gcc/ld find the libraries and header files correctly.

Perhaps that's what's supposed to happen, but there appear to be a lot
of packages where it doesn't happen (at least for some of us).

> Actually, this mechanism *only* works with toolchain supporting
> --sysroot. Toolchains *not* supporting --sysroot are rejected (see
> toolchain/external-toolchain/ext-tool.mk).

True.

-- 
Grant

[Buildroot] external toolchain and sysroot

[Thomas Petazzoni]

On Fri, 26 Mar 2010 01:04:33 +0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> >  * We pass --sysroot $(STAGING_DIR) to every gcc/ld invocation, so that
> >    gcc/ld find the libraries and header files correctly.
> 
> Perhaps that's what's supposed to happen, but there appear to be a lot
> of packages where it doesn't happen (at least for some of us).

As I said in my other mail in this thread, yes, there are many packages
that are broken with regard to --sysroot. Packages that use the
Makefile.autotools.in infrastructure should all work, but for all the
other packages, they should be fixed one by one.

Cheers,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-26, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> On Fri, 26 Mar 2010 01:04:33 +0000 (UTC)
> Grant Edwards <grant.b.edwards@gmail.com> wrote:
>
>> >  * We pass --sysroot $(STAGING_DIR) to every gcc/ld invocation, so that
>> >    gcc/ld find the libraries and header files correctly.
>> 
>> Perhaps that's what's supposed to happen, but there appear to be a lot
>> of packages where it doesn't happen (at least for some of us).
>
> As I said in my other mail in this thread, yes, there are many packages
> that are broken with regard to --sysroot.

Sorry, I guess I got confused by the contradiction.  Stating that
--sysroot is passed to every gcc/ld invocation meant to me that you
believed there were no packages that are broken with regard to
--sysroot.

> Packages that use the Makefile.autotools.in infrastructure should all
> work, but for all the other packages, they should be fixed one by
> one.

OK, that's all I was trying to establish: that some/many packages are
broken when used with external toolchains and not every gcc/ld
invocation is being passed --sysroot.

What I'm still puzzled about out is why builds fail for some external
toolchains and not others.  I'm pretty sure that I wasn't the first
person to attempt to build 2010.02 busybox with an external toolchain,
yet nobody else seems to have had the link failure due to the missing
--sysroot.  OTOH, --sysroot was clearly not present in the link flags.

I can't find any gcc/binutils config options that should cause a
difference like that.

-- 
Grant

[Buildroot] external toolchain and sysroot

[Thomas Petazzoni]

Hello,

On Sat, 27 Mar 2010 01:10:16 +0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> Sorry, I guess I got confused by the contradiction.  Stating that
> --sysroot is passed to every gcc/ld invocation meant to me that you
> believed there were no packages that are broken with regard to
> --sysroot.

Oh, yes, sorry, my wording was maybe confusing (english is not my
native language). To make external toolchain work, --sysroot *must* be
passed to every gcc/ld invocation, but it's clearly not the case in
many non-autotools packages.

> What I'm still puzzled about out is why builds fail for some external
> toolchains and not others.  I'm pretty sure that I wasn't the first
> person to attempt to build 2010.02 busybox with an external toolchain,
> yet nobody else seems to have had the link failure due to the missing
> --sysroot.  OTOH, --sysroot was clearly not present in the link flags.

I'm also a bit puzzled by the problem you had on Busybox, since I'm
definitely able to build it with 2010.02, as it is, with a Crosstool-NG
external toolchain (but not with a Buildroot external toolchain). I'll
try to reproduce.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-27, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:

> Oh, yes, sorry, my wording was maybe confusing (english is not my
> native language). To make external toolchain work, --sysroot *must* be
> passed to every gcc/ld invocation, but it's clearly not the case in
> many non-autotools packages.
>
>> What I'm still puzzled about out is why builds fail for some external
>> toolchains and not others.  I'm pretty sure that I wasn't the first
>> person to attempt to build 2010.02 busybox with an external toolchain,
>> yet nobody else seems to have had the link failure due to the missing
>> --sysroot.  OTOH, --sysroot was clearly not present in the link flags.
>
> I'm also a bit puzzled by the problem you had on Busybox, since I'm
> definitely able to build it with 2010.02, as it is, with a Crosstool-NG
> external toolchain (but not with a Buildroot external toolchain). I'll
> try to reproduce.

The other odd thing is that _I_ didn't used to have that problem, yet
it seems obvious that --sysroot was being passed. When I was using a
git snapshot from sometime in November/December which used gcc 4.3,
both busybox and mtd-utils built fine.  The external toolchain in both
cases was built by buildroot using exactly the same shell-script. 
AFAICT, the .mk files didn't change with respect to the --sysroot
option.  The only significant thing that seems to have changed was the
gcc version. I haven't had a chance to dig out a copy of my gcc 4.3
toolchain and verify that the toolchain change caused the problem in
my case.

-- 
Grant

[Buildroot] external toolchain and sysroot

[Grant Edwards]

On 2010-03-25, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> On Thu, 25 Mar 2010 18:08:49 +0000 (UTC)
> Grant Edwards <grant.b.edwards@gmail.com> wrote:
>
>> Based on my sample of three packages (two had the problem), I'm
>> guessing it's a very common problem.  It looks like the buildroot
>> developers use external toolchains that don't require a --sysroot
>> option. 
>
> They do. What we do to support external toolchains is :

Recent testing has confirmed that relocated toolchains built by
crosstool-NG can find library files without a --sysroot option.
Toolchains built by buildroot can't.  This explains the spate of
"missing --sysroot" bugs that affect people using relocated buildroot
toolchains and not people using relocated crosstool-NG toolchains.

-- 
Grant Edwards               grant.b.edwards        Yow! You can't hurt me!!
                                  at               I have an ASSUMABLE
                              gmail.com            MORTGAGE!!

[Buildroot] external toolchain and sysroot

[Thomas Petazzoni]

Hello,

On Thu, 25 Mar 2010 16:14:19 +0000 (UTC)
Grant Edwards <grant.b.edwards@gmail.com> wrote:

> > But in linking phase:
> > /home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/bin/arm-indigo-linux-uclibcgnueabi-gcc 
> > -DAUTOCONF -DPOSIX -DUSG -o em ansi.o basic.o bind.o buffer.o crypt.o display.o eval.o exec.o 
> > file.o fileio.o ibmpc.o input.o isearch.o line.o lock.o main.o pklock.o posix.o random.o 
> > region.o search.o spawn.o tcap.o termio.o vmsvt.o vt52.o window.o word.o -lncurses
> > /home/yury/toolchain/arm-indigo-linux-uclibcgnueabi/lib/gcc/arm-indigo-linux-uclibcgnueabi/4.3.4/../..
> > /../../arm-indigo-linux-uclibcgnueabi/bin/ld: cannot find -lncurses
> >
> > here --sysroot is omitted and thus I'm very sad :-)

That's a problem of uemacs.mk, as you found out later.

In fact, many of the packages that do not use the Makefile.autotools.in
infrastructure are broken with regard to external toolchain support.
I've fixed some of the packages I tested, but I couldn't fix them all.
So it's our job, external toolchain users, to progressively fix all
packages so that they build nicely with external toolchains. Of course,
if you don't know how to fix a package, just report the bug, so we'll
know it's broken and we'll try to fix it.

> I know how you feel.  That's exactly the same problem I had building
> busybox two weeks ago: --sysroot was passed correctly in the compile
> phase, but wasn't passed in the link phase.  Apparently the external
> toolchains used by the devs don't need a --sysroot during linking.
> Apparently our toolchains do.

I don't think so. Just like Yury, I use toolchains generated by
Crosstool-NG (for i686, ARM glibc, ARM uClibc OABI, ARM uClibc EABI,
PowerPC, etc.). So I encounter the same problem, it's just that I never
tried to build uemacs or vsftpd.

> > I can't figure this out by myself. TARGET_LDFLAGS are good, 
> > TARGET_CFLAGS are too.
> 
> In the busybox .mk file, I found that TARGET_LDFLAGS wasn't being used
> (it wasn't being passed to the "make" command).  I had to patch the
> .mk file to pass the TARGET_LDFLAGS value to make.

I've seen your bug about this and I don't have the same problem with
Crosstool-NG toolchains. It's still on my TODO-list to try to reproduce
the problem you're having.

For the moment, I'm busy doing some more heavy development work for
Buildroot, I plan to do more bug-fixing related work during the last
month of the release cycle.

Regards, 

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com