[Buildroot] [PATCH 0/4] Solving the musl dynamic linker issues

[Buildroot] [PATCH 0/4] Solving the musl dynamic linker issues

[Thomas Petazzoni]

Hello,

We currently have two outstanding musl dynamic linker issues:

 1. We don't handle all musl dynamic linker names for all
    architectures. I proposed a patch solving this at
    https://patchwork.ozlabs.org/patch/780411/, where the approach was
    to build a dummy program and use readelf to find the dynamic
    linker name, and therefore the symlink to libc.so that we need to
    create.

 2. We don't support Crosstool-NG musl toolchains, because they put
    libc.so in /usr/lib and not /lib, and therefore our current
    creation logic that makes the dynamic linker point to /lib/libc.so
    incorrect. Ilya Kuzmich <ilya.kuzmich@gmail.com> proposed patches
    (https://patchwork.ozlabs.org/patch/763977/
    https://patchwork.ozlabs.org/patch/748974/) to solve this.

After discussion with Arnout, we realized that all our messy logic for
creating the musl dynamic linker symbolic link is useless. The correct
symbolic link already exists in the original toolchain sysroot, so we
should just copy it from the sysroot, just like we do for glibc/uClibc.

This series of patches implements just that, with the main patch being
PATCH 3/4.

The first patch just adds more external toolchain tests to our runtime
tests infrastructure, to validate that the changes in this series
don't break things.

The second patch prepares the copy_toolchain_lib_root for copying a
symlink in /lib that potentially points to a file in /usr/lib (which
it currently doesn't handle).

The third patch is the core of the solution, basically putting ld*.so*
unconditionally in TOOLCHAIN_EXTERNAL_LIBS.

The fourth patch is a minor refactoring.

Thanks,

Thomas

Thomas Petazzoni (4):
  support/testing: add tests for musl and uclibc toolchains
  toolchain/helpers.mk: re-evaluate DESTDIR in copy_toolchain_lib_root
  toolchain-external: copy ld*.so* for all C libraries
  toolchain-external: also put libgcc_s.so unconditionally in
    TOOLCHAIN_EXTERNAL_LIBS

 support/testing/tests/toolchain/test_external.py   | 75 ++++++++++++++++++++++
 toolchain/helpers.mk                               |  2 +-
 .../toolchain-external/pkg-toolchain-external.mk   | 36 ++---------
 3 files changed, 81 insertions(+), 32 deletions(-)

-- 
2.9.4

[Buildroot] [PATCH 1/4] support/testing: add tests for musl and uclibc toolchains

[Thomas Petazzoni]

These tests simply build a system with musl and uclibc toolchains, and
boot them under qemu. It allows to minimally validate that our support
for musl/uclibc external toolchains is working. We already had some
tests covering glibc toolchains, so we can now easily test that all
three C libraries are supported.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
This commit is part of the series, as I've written/used those tests to
validate that things are still working correctly with all of glibc,
uclibc and musl toolchains.
---
 support/testing/tests/toolchain/test_external.py | 75 ++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/support/testing/tests/toolchain/test_external.py b/support/testing/tests/toolchain/test_external.py
index 0b15d48..f975c56 100644
--- a/support/testing/tests/toolchain/test_external.py
+++ b/support/testing/tests/toolchain/test_external.py
@@ -154,3 +154,78 @@ BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y
                            kernel="builtin",
                            options=["-initrd", img])
         self.emulator.login()
+
+class TestExternalToolchainBuildrootMusl(TestExternalToolchain):
+    config = BASIC_CONFIG + \
+"""
+BR2_arm=y
+BR2_cortex_a9=y
+BR2_ARM_ENABLE_VFP=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-cortex-a9-musl-2017.05-444-g6c704ba.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_11=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
+"""
+    toolchain_prefix = "arm-linux"
+
+    def test_run(self):
+        TestExternalToolchain.common_check(self)
+        img = os.path.join(self.builddir, "images", "rootfs.cpio")
+        self.emulator.boot(arch="armv7",
+                           kernel="builtin",
+                           options=["-initrd", img])
+        self.emulator.login()
+
+class TestExternalToolchainCtngMusl(TestExternalToolchain):
+    config = BASIC_CONFIG + \
+"""
+BR2_arm=y
+BR2_cortex_a9=y
+BR2_ARM_ENABLE_VFP=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://prout.org/arm-ctng-linux-musleabihf.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="arm-ctng-linux-musleabihf"
+BR2_TOOLCHAIN_EXTERNAL_GCC_7=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
+"""
+    toolchain_prefix = "arm-ctng-linux-musleabihf"
+
+    def test_run(self):
+        TestExternalToolchain.common_check(self)
+        img = os.path.join(self.builddir, "images", "rootfs.cpio")
+        self.emulator.boot(arch="armv7",
+                           kernel="builtin",
+                           options=["-initrd", img])
+        self.emulator.login()
+
+class TestExternalToolchainBuildrootuClibc(TestExternalToolchain):
+    config = BASIC_CONFIG + \
+"""
+BR2_arm=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2017.05-444-g6c704ba.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
+"""
+    toolchain_prefix = "arm-linux"
+
+    def test_run(self):
+        TestExternalToolchain.common_check(self)
+        img = os.path.join(self.builddir, "images", "rootfs.cpio")
+        self.emulator.boot(arch="armv7",
+                           kernel="builtin",
+                           options=["-initrd", img])
+        self.emulator.login()
-- 
2.9.4

[Buildroot] [PATCH 1/4] support/testing: add tests for musl and uclibc toolchains

[Thomas Petazzoni]

Hello,

On Sun,  2 Jul 2017 15:41:04 +0200, Thomas Petazzoni wrote:

> +BR2_arm=y
> +BR2_cortex_a9=y
> +BR2_ARM_ENABLE_VFP=y
> +BR2_TOOLCHAIN_EXTERNAL=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> +BR2_TOOLCHAIN_EXTERNAL_URL="http://prout.org/arm-ctng-linux-musleabihf.tar.xz"

OK, this should have been
http://autobuild.buildroot.net/toolchains/tarballs/arm-ctng-linux-musleabihf.tar.xz.
Will fix for v2.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 1/4] support/testing: add tests for musl and uclibc toolchains

[Arnout Vandecappelle]

On 02-07-17 15:41, Thomas Petazzoni wrote:
> These tests simply build a system with musl and uclibc toolchains, and
> boot them under qemu. It allows to minimally validate that our support
> for musl/uclibc external toolchains is working. We already had some
> tests covering glibc toolchains, so we can now easily test that all
> three C libraries are supported.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

 When we have the in-tree config fragments for the autobuilder toolchains, these
should be used instead of hardcoding. For now however (after your fix):

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> ---
> This commit is part of the series, as I've written/used those tests to
> validate that things are still working correctly with all of glibc,
> uclibc and musl toolchains.
> ---
>  support/testing/tests/toolchain/test_external.py | 75 ++++++++++++++++++++++++
>  1 file changed, 75 insertions(+)
> 
> diff --git a/support/testing/tests/toolchain/test_external.py b/support/testing/tests/toolchain/test_external.py
> index 0b15d48..f975c56 100644
> --- a/support/testing/tests/toolchain/test_external.py
> +++ b/support/testing/tests/toolchain/test_external.py
> @@ -154,3 +154,78 @@ BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y
>                             kernel="builtin",
>                             options=["-initrd", img])
>          self.emulator.login()
> +
> +class TestExternalToolchainBuildrootMusl(TestExternalToolchain):
> +    config = BASIC_CONFIG + \
> +"""
> +BR2_arm=y
> +BR2_cortex_a9=y
> +BR2_ARM_ENABLE_VFP=y
> +BR2_TOOLCHAIN_EXTERNAL=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> +BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-cortex-a9-musl-2017.05-444-g6c704ba.tar.bz2"
> +BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
> +BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_11=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
> +BR2_TOOLCHAIN_EXTERNAL_CXX=y
> +"""
> +    toolchain_prefix = "arm-linux"
> +
> +    def test_run(self):
> +        TestExternalToolchain.common_check(self)
> +        img = os.path.join(self.builddir, "images", "rootfs.cpio")
> +        self.emulator.boot(arch="armv7",
> +                           kernel="builtin",
> +                           options=["-initrd", img])
> +        self.emulator.login()
> +
> +class TestExternalToolchainCtngMusl(TestExternalToolchain):
> +    config = BASIC_CONFIG + \
> +"""
> +BR2_arm=y
> +BR2_cortex_a9=y
> +BR2_ARM_ENABLE_VFP=y
> +BR2_TOOLCHAIN_EXTERNAL=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> +BR2_TOOLCHAIN_EXTERNAL_URL="http://prout.org/arm-ctng-linux-musleabihf.tar.xz"
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="arm-ctng-linux-musleabihf"
> +BR2_TOOLCHAIN_EXTERNAL_GCC_7=y
> +BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
> +BR2_TOOLCHAIN_EXTERNAL_CXX=y
> +"""
> +    toolchain_prefix = "arm-ctng-linux-musleabihf"
> +
> +    def test_run(self):
> +        TestExternalToolchain.common_check(self)
> +        img = os.path.join(self.builddir, "images", "rootfs.cpio")
> +        self.emulator.boot(arch="armv7",
> +                           kernel="builtin",
> +                           options=["-initrd", img])
> +        self.emulator.login()
> +
> +class TestExternalToolchainBuildrootuClibc(TestExternalToolchain):
> +    config = BASIC_CONFIG + \
> +"""
> +BR2_arm=y
> +BR2_TOOLCHAIN_EXTERNAL=y
> +BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
> +BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
> +BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2017.05-444-g6c704ba.tar.bz2"
> +BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
> +BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
> +BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
> +# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
> +BR2_TOOLCHAIN_EXTERNAL_CXX=y
> +"""
> +    toolchain_prefix = "arm-linux"
> +
> +    def test_run(self):
> +        TestExternalToolchain.common_check(self)
> +        img = os.path.join(self.builddir, "images", "rootfs.cpio")
> +        self.emulator.boot(arch="armv7",
> +                           kernel="builtin",
> +                           options=["-initrd", img])
> +        self.emulator.login()
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 2/4] toolchain/helpers.mk: re-evaluate DESTDIR in copy_toolchain_lib_root

[Thomas Petazzoni]

copy_toolchain_lib_root copies libraries from staging to target,
resolving and copying symbolic links along the way.

The most inner loop, a "while" loop, starts from an initial name, and
if it's a symbolic link, gets resolved to the target, and the loop
iterates until we reach a real file. However, the destination folder
where the symbolic link or real file is created is computed in DESTDIR
only once, before this loop starts. Therefore, this loop works fine
when all symbolic links in the chain, and the real file all belong to
the same directory. But it doesn't do the correct thing when the
symbolic link and/or real file are in different folder.

An example is Crosstool-NG musl toolchains, where the dynamic loader
is in /lib/ld-musl*.so but points to ../usr/lib/libc.so. With the
current logic, we copy /lib/ld-musl*.so to /lib, but we also copy
libc.so to /lib instead of the expected /usr/lib.

This currently doesn't cause any problem because the musl dynamic
linker is manually created by the TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
hook. However, this logic has a number of problems, so in a followup
commit, we are going to put the musl dynamic linker in
TOOLCHAIN_EXTERNAL_LIBS, which will cause it to be copied by
copy_toolchain_lib_root. But we obviously want the link and its target
to be copied to the right place, hence this fix.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/helpers.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
index 90834f4..fe2b4b9 100644
--- a/toolchain/helpers.mk
+++ b/toolchain/helpers.mk
@@ -13,10 +13,10 @@ copy_toolchain_lib_root = \
 \
 	LIBPATHS=`find $(STAGING_DIR)/ -name "$${LIBPATTERN}" 2>/dev/null` ; \
 	for LIBPATH in $${LIBPATHS} ; do \
-		DESTDIR=`echo $${LIBPATH} | sed "s,^$(STAGING_DIR)/,," | xargs dirname` ; \
 		mkdir -p $(TARGET_DIR)/$${DESTDIR}; \
 		while true ; do \
 			LIBNAME=`basename $${LIBPATH}`; \
+			DESTDIR=`echo $${LIBPATH} | sed "s,^$(STAGING_DIR)/,," | xargs dirname` ; \
 			rm -fr $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
 			if test -h $${LIBPATH} ; then \
 				cp -d $${LIBPATH} $(TARGET_DIR)/$${DESTDIR}/$${LIBNAME}; \
-- 
2.9.4

[Buildroot] [PATCH 3/4] toolchain-external: copy ld*.so* for all C libraries

[Thomas Petazzoni]

Currently, for the dynamic loader, we're copying ld*.so* for glibc and
uClibc, except for glibc/EABIhf where we are explicitly copying
ld-linux-armhf.so.*. For musl, we're not copying the dynamic linker
because it's simply a symbolic link to libc.so. However, the name of
the musl dynamic linker changes from one architecture to the other,
and we don't handle all cases.

Since handling the musl dynamic linker symlink creation is becoming
more and more annoying to maintain, this commit makes musl use the
same mechanism as glibc/uClibc: put the dynamic linker in
TOOLCHAIN_EXTERNAL_LIBS.

In addition, the special condition on glibc/EABIhf was added in
11ec38b6950cf3337b52fb97f27c2fd7c776c5c2 ("toolchain-external: fix
Linaro ARM toolchain support") because an old Linaro toolchain had two
dynamic loaders, and we wanted to copy only one. But 1/ this is old
and 2/ having the two dynamic linkers doesn't really matter.

So this commit simply unconditionally adds "ld*.so*" to
TOOLCHAIN_EXTERNAL_LIBS, regardless of the C library being chosen. It
re-uses the musl dynamic linker symlink from the sysroot, which makes
it always correct, and allows us to remove the
TOOLCHAIN_EXTERNAL_MUSL_LD_LINK hook, and all the related logic.

This commit therefore solves two problems with the musl dynamic linker
symbolic link creation logic:

 1 We support all architectures, without having to hardcode in
   Buildroot the mapping between the CPU architecture and the
   corresponding dynamic linker name. For example, our current logic
   was not handling the mips64+n32 ABI case, where the dynamic linker
   is named ld-musl-mipsn32el.so.1.

 2 We support Crosstool-NG musl toolchains, where the dynamic linker
   is in /lib, but libc.so is in /usr/lib.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
This commit therefore replaces:

 - https://patchwork.ozlabs.org/patch/780411/ (was another solution
   for solving problem 1 above)

 - https://patchwork.ozlabs.org/patch/763977/ and
   https://patchwork.ozlabs.org/patch/748974/ (was another solution
   for solving problem 2 above)
---
 .../toolchain-external/pkg-toolchain-external.mk   | 32 ++--------------------
 1 file changed, 3 insertions(+), 29 deletions(-)

diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
index 8460e37..c1c3900 100644
--- a/toolchain/toolchain-external/pkg-toolchain-external.mk
+++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
@@ -107,13 +107,11 @@ endif
 #
 # Definitions of the list of libraries that should be copied to the target.
 #
+
+TOOLCHAIN_EXTERNAL_LIBS += ld*.so*
+
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
 TOOLCHAIN_EXTERNAL_LIBS += libatomic.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
-ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_ARM_EABIHF),yy)
-TOOLCHAIN_EXTERNAL_LIBS += ld-linux-armhf.so.*
-else
-TOOLCHAIN_EXTERNAL_LIBS += ld*.so.*
-endif
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
 TOOLCHAIN_EXTERNAL_LIBS += libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
@@ -470,30 +468,6 @@ define TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT
 	fi
 endef
 
-# Various utility functions used by the external toolchain based on musl.
-
-# With the musl C library, the libc.so library directly plays the role
-# of the dynamic library loader. We just need to create a symbolic
-# link to libc.so with the appropriate name.
-ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL):$(BR2_STATIC_LIBS),y:)
-ifeq ($(BR2_i386),y)
-MUSL_ARCH = i386
-else ifeq ($(BR2_ARM_EABIHF),y)
-MUSL_ARCH = armhf
-else ifeq ($(BR2_mips):$(BR2_SOFT_FLOAT),y:y)
-MUSL_ARCH = mips-sf
-else ifeq ($(BR2_mipsel):$(BR2_SOFT_FLOAT),y:y)
-MUSL_ARCH = mipsel-sf
-else ifeq ($(BR2_sh),y)
-MUSL_ARCH = sh
-else
-MUSL_ARCH = $(ARCH)
-endif
-define TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
-	ln -sf libc.so $(TARGET_DIR)/lib/ld-musl-$(MUSL_ARCH).so.1
-endef
-endif
-
 # uClibc-ng dynamic loader is called ld-uClibc.so.1, but gcc is not
 # patched specifically for uClibc-ng, so it continues to generate
 # binaries that expect the dynamic loader to be named ld-uClibc.so.0,
-- 
2.9.4

[Buildroot] [PATCH 3/4] toolchain-external: copy ld*.so* for all C libraries

[Baruch Siach]

Hi Thomas,

On Sun, Jul 02, 2017 at 03:41:06PM +0200, Thomas Petazzoni wrote:
> -define TOOLCHAIN_EXTERNAL_MUSL_LD_LINK
> -	ln -sf libc.so $(TARGET_DIR)/lib/ld-musl-$(MUSL_ARCH).so.1
> -endef
> -endif

You should also remove the reference to TOOLCHAIN_EXTERNAL_MUSL_LD_LINK below.

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] [PATCH 3/4] toolchain-external: copy ld*.so* for all C libraries

[Thomas De Schampheleire]

2017-07-02 15:41 GMT+02:00 Thomas Petazzoni
<thomas.petazzoni@free-electrons.com>:
> Currently, for the dynamic loader, we're copying ld*.so* for glibc and
> uClibc, except for glibc/EABIhf where we are explicitly copying
> ld-linux-armhf.so.*. For musl, we're not copying the dynamic linker
> because it's simply a symbolic link to libc.so. However, the name of
> the musl dynamic linker changes from one architecture to the other,
> and we don't handle all cases.
>
> Since handling the musl dynamic linker symlink creation is becoming
> more and more annoying to maintain, this commit makes musl use the
> same mechanism as glibc/uClibc: put the dynamic linker in
> TOOLCHAIN_EXTERNAL_LIBS.
>
> In addition, the special condition on glibc/EABIhf was added in
> 11ec38b6950cf3337b52fb97f27c2fd7c776c5c2 ("toolchain-external: fix
> Linaro ARM toolchain support") because an old Linaro toolchain had two
> dynamic loaders, and we wanted to copy only one. But 1/ this is old
> and 2/ having the two dynamic linkers doesn't really matter.
>
> So this commit simply unconditionally adds "ld*.so*" to
> TOOLCHAIN_EXTERNAL_LIBS, regardless of the C library being chosen. It
> re-uses the musl dynamic linker symlink from the sysroot, which makes
> it always correct, and allows us to remove the
> TOOLCHAIN_EXTERNAL_MUSL_LD_LINK hook, and all the related logic.
>
> This commit therefore solves two problems with the musl dynamic linker
> symbolic link creation logic:
>
>  1 We support all architectures, without having to hardcode in
>    Buildroot the mapping between the CPU architecture and the
>    corresponding dynamic linker name. For example, our current logic
>    was not handling the mips64+n32 ABI case, where the dynamic linker
>    is named ld-musl-mipsn32el.so.1.
>
>  2 We support Crosstool-NG musl toolchains, where the dynamic linker
>    is in /lib, but libc.so is in /usr/lib.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
> This commit therefore replaces:
>
>  - https://patchwork.ozlabs.org/patch/780411/ (was another solution
>    for solving problem 1 above)
>
>  - https://patchwork.ozlabs.org/patch/763977/ and
>    https://patchwork.ozlabs.org/patch/748974/ (was another solution
>    for solving problem 2 above)
> ---
>  .../toolchain-external/pkg-toolchain-external.mk   | 32 ++--------------------
>  1 file changed, 3 insertions(+), 29 deletions(-)
>
> diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
> index 8460e37..c1c3900 100644
> --- a/toolchain/toolchain-external/pkg-toolchain-external.mk
> +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
> @@ -107,13 +107,11 @@ endif
>  #
>  # Definitions of the list of libraries that should be copied to the target.
>  #
> +
> +TOOLCHAIN_EXTERNAL_LIBS += ld*.so*

In copy_toolchain_sysroot, the patterns ld*.so and ld*.so.* are
treated separately. I think the only reason must be to avoid matching
ld*.something (I can't think of another reason at this moment).
Perhaps that is too far-fetched, and copy_toolchain_sysroot should be
updated, but in any case it makes sense to line up.

Best regards,
Thomas

[Buildroot] [PATCH 3/4] toolchain-external: copy ld*.so* for all C libraries

[Thomas Petazzoni]

Hello,

On Sun, 2 Jul 2017 21:21:46 +0200, Thomas De Schampheleire wrote:

> > diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
> > index 8460e37..c1c3900 100644
> > --- a/toolchain/toolchain-external/pkg-toolchain-external.mk
> > +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
> > @@ -107,13 +107,11 @@ endif
> >  #
> >  # Definitions of the list of libraries that should be copied to the target.
> >  #
> > +
> > +TOOLCHAIN_EXTERNAL_LIBS += ld*.so*  
> 
> In copy_toolchain_sysroot, the patterns ld*.so and ld*.so.* are
> treated separately. I think the only reason must be to avoid matching
> ld*.something (I can't think of another reason at this moment).
> Perhaps that is too far-fetched, and copy_toolchain_sysroot should be
> updated, but in any case it makes sense to line up.

I think it's a separate issue, because we're already using ld*.so*
today in pkg-toolchain-external.mk for TOOLCHAIN_EXTERNAL_LIBS, I'm
just moving it around.

However, I agree we should probably line up things. In which direction
do you think we should line up? I guess using ld*.so* in
copy_toolchain_sysroot should be fine.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 4/4] toolchain-external: also put libgcc_s.so unconditionally in TOOLCHAIN_EXTERNAL_LIBS

[Thomas Petazzoni]

libgcc_s.so is now added to TOOLCHAIN_EXTERNAL_LIBS for glibc/uclibc
in one place, and for musl in another place. Bottom line: it should be
in TOOLCHAIN_EXTERNAL_LIBS unconditionally.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 toolchain/toolchain-external/pkg-toolchain-external.mk | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk
index c1c3900..f3f0f48 100644
--- a/toolchain/toolchain-external/pkg-toolchain-external.mk
+++ b/toolchain/toolchain-external/pkg-toolchain-external.mk
@@ -108,10 +108,10 @@ endif
 # Definitions of the list of libraries that should be copied to the target.
 #
 
-TOOLCHAIN_EXTERNAL_LIBS += ld*.so*
+TOOLCHAIN_EXTERNAL_LIBS += ld*.so* libgcc_s.so.*
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
-TOOLCHAIN_EXTERNAL_LIBS += libatomic.so.* libc.so.* libcrypt.so.* libdl.so.* libgcc_s.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
+TOOLCHAIN_EXTERNAL_LIBS += libatomic.so.* libc.so.* libcrypt.so.* libdl.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.*
 ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y)
 TOOLCHAIN_EXTERNAL_LIBS += libpthread.so.*
 ifneq ($(BR2_PACKAGE_GDB)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),)
@@ -125,7 +125,7 @@ TOOLCHAIN_EXTERNAL_LIBS += libnss_files.so.* libnss_dns.so.* libmvec.so.* libanl
 endif
 
 ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y)
-TOOLCHAIN_EXTERNAL_LIBS += libc.so libgcc_s.so.*
+TOOLCHAIN_EXTERNAL_LIBS += libc.so
 endif
 
 ifeq ($(BR2_INSTALL_LIBSTDCPP),y)
-- 
2.9.4

[Buildroot] [PATCH 0/4] Solving the musl dynamic linker issues

[Thomas Petazzoni]

Hello,

On Sun,  2 Jul 2017 15:41:03 +0200, Thomas Petazzoni wrote:

> Thomas Petazzoni (4):
>   support/testing: add tests for musl and uclibc toolchains
>   toolchain/helpers.mk: re-evaluate DESTDIR in copy_toolchain_lib_root
>   toolchain-external: copy ld*.so* for all C libraries
>   toolchain-external: also put libgcc_s.so unconditionally in
>     TOOLCHAIN_EXTERNAL_LIBS

I've applied the series, and I've added a 5th patch that removes the
ld*.so handling in copy_toolchain_sysroot, keeping only ld*.so.* to be
consistent with what TOOLCHAIN_EXTERNAL_LIBS says. This (hopefully)
addresses Thomas DS comment :-)

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com