[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Cam Hutchison]

check_glibc checks for a valid glibc in an external toolchain, but
assumes that the files indicating the presence of glibc (ld-linux*.so.*,
ld.so.* or ld64.so.*) are in a top-level directory of the sysroot.

When building a toolchain with buildroot and a merged /usr, /lib is
a symlink to usr/lib. This is copied from the target to the staging
directory, and then to the sysroot, and the ultimate location of the
required files is in /usr/lib in the sysroot.

check_glibc fails to find the necessary files in this layout due to it
using find(1) with -maxdepth 2 and not following symlinks. This causes
the error 'Incorrect selection of the C library' when trying to use a
buildroot toolchain as an external toolchain that was built with a
merged /usr.

Tell find(1) to follow symlinks so that it can find the required files
to determine glibc availability in the external toolchain.

Signed-off-by: Cam Hutchison <camh@xdna.net>
---
 toolchain/helpers.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index e9e36d2069..63ef6fb4b0 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -227,7 +227,7 @@ check_glibc_rpc_feature = \
 #
 check_glibc = \
 	SYSROOT_DIR="$(strip $1)"; \
-	if test `find $${SYSROOT_DIR}/ -maxdepth 2 -name 'ld-linux*.so.*' -o -name 'ld.so.*' -o -name 'ld64.so.*' | wc -l` -eq 0 ; then \
+	if test `find -L $${SYSROOT_DIR}/ -maxdepth 2 -name 'ld-linux*.so.*' -o -name 'ld.so.*' -o -name 'ld64.so.*' | wc -l` -eq 0 ; then \
 		echo "Incorrect selection of the C library"; \
 		exit -1; \
 	fi; \
-- 
2.11.0

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Thomas Petazzoni]

Hello,

This generally looks ok, to me, except one paragraph of explanation,
see below.

On Sun,  3 Sep 2017 22:08:35 +1000, Cam Hutchison wrote:
> check_glibc checks for a valid glibc in an external toolchain, but
> assumes that the files indicating the presence of glibc (ld-linux*.so.*,
> ld.so.* or ld64.so.*) are in a top-level directory of the sysroot.
> 
> When building a toolchain with buildroot and a merged /usr, /lib is
> a symlink to usr/lib. This is copied from the target to the staging
> directory, and then to the sysroot, and the ultimate location of the
> required files is in /usr/lib in the sysroot.

I don't understand this sentence. Nothing gets copied from target to
staging, and staging *is* the sysroot, so there's no copy. Could you
explain what you wanted to say here, we can perhaps find a better
phrasing.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Cam Hutchison]

On 3 September 2017 at 22:17, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
>
> Hello,
>
> This generally looks ok, to me, except one paragraph of explanation,
> see below.
>
> On Sun,  3 Sep 2017 22:08:35 +1000, Cam Hutchison wrote:
> > check_glibc checks for a valid glibc in an external toolchain, but
> > assumes that the files indicating the presence of glibc (ld-linux*.so.*,
> > ld.so.* or ld64.so.*) are in a top-level directory of the sysroot.
> >
> > When building a toolchain with buildroot and a merged /usr, /lib is
> > a symlink to usr/lib. This is copied from the target to the staging
> > directory, and then to the sysroot, and the ultimate location of the
> > required files is in /usr/lib in the sysroot.
>
> I don't understand this sentence. Nothing gets copied from target to
> staging, and staging *is* the sysroot, so there's no copy. Could you
> explain what you wanted to say here, we can perhaps find a better
> phrasing.


Ok, I wasn't sure on how staging became sysroot - I thought it was
copied.

I misinterpreted 175a96c4909104bde706fa0e1f9010af8b252caa
(package/skeleton-common: simplify staging install) to mean it was
copying from the target, but it was the skeleton, not the target.

Would you like me to respin this with an updated comment, or will
you take it and change it as appropriate?

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Thomas Petazzoni]

Hello,

On Sun, 3 Sep 2017 22:22:24 +1000, Cam Hutchison wrote:

> > > When building a toolchain with buildroot and a merged /usr, /lib is
> > > a symlink to usr/lib. This is copied from the target to the staging
> > > directory, and then to the sysroot, and the ultimate location of the
> > > required files is in /usr/lib in the sysroot.  
> >
> > I don't understand this sentence. Nothing gets copied from target to
> > staging, and staging *is* the sysroot, so there's no copy. Could you
> > explain what you wanted to say here, we can perhaps find a better
> > phrasing.  
> 
> Ok, I wasn't sure on how staging became sysroot - I thought it was
> copied.
> 
> I misinterpreted 175a96c4909104bde706fa0e1f9010af8b252caa
> (package/skeleton-common: simplify staging install) to mean it was
> copying from the target, but it was the skeleton, not the target.
> 
> Would you like me to respin this with an updated comment, or will
> you take it and change it as appropriate?

In fact, I would first like to understand what you meant :)

Maybe what you meant is that: when the Buildroot toolchain is built
with a merged /usr, lib is a symlink to usr/lib in the staging
directory. Therefore, the resulting toolchain has such a symlink in its
sysroot, and such a symlink is present in the staging directory when
the toolchain is re-used as an external toolchain. The consequence is
that the dynamic linker is not located in /lib, but in /usr/lib, even
though it is accessible from /lib from a symlink.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Cam Hutchison]

On 3 September 2017 at 22:25, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Sun, 3 Sep 2017 22:22:24 +1000, Cam Hutchison wrote:
>
>> > > When building a toolchain with buildroot and a merged /usr, /lib is
>> > > a symlink to usr/lib. This is copied from the target to the staging
>> > > directory, and then to the sysroot, and the ultimate location of the
>> > > required files is in /usr/lib in the sysroot.
>> >
>> > I don't understand this sentence. Nothing gets copied from target to
>> > staging, and staging *is* the sysroot, so there's no copy. Could you
>> > explain what you wanted to say here, we can perhaps find a better
>> > phrasing.
>>
>> Ok, I wasn't sure on how staging became sysroot - I thought it was
>> copied.
>>
>> I misinterpreted 175a96c4909104bde706fa0e1f9010af8b252caa
>> (package/skeleton-common: simplify staging install) to mean it was
>> copying from the target, but it was the skeleton, not the target.
>>
>> Would you like me to respin this with an updated comment, or will
>> you take it and change it as appropriate?
>
> In fact, I would first like to understand what you meant :)
>
> Maybe what you meant is that: when the Buildroot toolchain is built
> with a merged /usr, lib is a symlink to usr/lib in the staging
> directory. Therefore, the resulting toolchain has such a symlink in its
> sysroot, and such a symlink is present in the staging directory when
> the toolchain is re-used as an external toolchain. The consequence is
> that the dynamic linker is not located in /lib, but in /usr/lib, even
> though it is accessible from /lib from a symlink.

Yes, that's what I meant. The find command that I modified was not finding
the files it was looking for because it would not look in usr/lib (it has a
maxdepth of 2). But it would not look in lib because it was not following
symlinks. So check_glibc never found the files it was looking for and
produced an error.

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Yann E. MORIN]

Thomas, All,

On 2017-09-03 14:25 +0200, Thomas Petazzoni spake thusly:
> On Sun, 3 Sep 2017 22:22:24 +1000, Cam Hutchison wrote:
> > > > When building a toolchain with buildroot and a merged /usr, /lib is
> > > > a symlink to usr/lib. This is copied from the target to the staging
> > > > directory, and then to the sysroot, and the ultimate location of the
> > > > required files is in /usr/lib in the sysroot.  
[--SNIP--]
> In fact, I would first like to understand what you meant :)
> 
> Maybe what you meant is that: when the Buildroot toolchain is built
> with a merged /usr, lib is a symlink to usr/lib in the staging
> directory. Therefore, the resulting toolchain has such a symlink in its
> sysroot, and such a symlink is present in the staging directory when
> the toolchain is re-used as an external toolchain. The consequence is
> that the dynamic linker is not located in /lib, but in /usr/lib, even
> though it is accessible from /lib from a symlink.

This is exactly the problem that Cam is trying to solve here.

Cam created a toolchain with Buildroot and a merged /usr. He then
re-uses that as an external toolchain.

Cam, what about the fiollowing:

    toolchain: detect external glibc in merged /usr

    When using an external toolchain that was built with Buildroot and a
    merged /usr., the dynamic linker is actually in /usr/lib.

    But the check_glibc macro only limits the depth it is looking for
    the dynamic linker, anf misses it when it is in /usr/lib because it
    is too deep.

    WE could fix that in two ways: increase the depth in which we look
    for it, or follow symlinks. We choose the second solution.

    Signed-off-by: you
    Cc: Thomas P.
    Cc: Thomas DS.
    Cc: Me

What do you all think about this? ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/1] toolchain: detect external glibc in symlinked /lib

[Cam Hutchison]

On 3 September 2017 at 22:49, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Cam, what about the fiollowing:
>
>     toolchain: detect external glibc in merged /usr
>
>     When using an external toolchain that was built with Buildroot and a
>     merged /usr., the dynamic linker is actually in /usr/lib.
>
>     But the check_glibc macro only limits the depth it is looking for
>     the dynamic linker, anf misses it when it is in /usr/lib because it
>     is too deep.
>
>     WE could fix that in two ways: increase the depth in which we look
>     for it, or follow symlinks. We choose the second solution.
>
>     Signed-off-by: you
>     Cc: Thomas P.
>     Cc: Thomas DS.
>     Cc: Me
>
> What do you all think about this? ;-)

LGTM. I'll resend shortly.

Thanks,
Cam