Re: [Buildroot] [PATCH] toolchain-external: Fix external toolchains when BR2_ROOTFS_MERGED_USR

[Charlie Jenkins <charlie@rivosinc.com>]

On Tue, Mar 25, 2025 at 10:47:12PM +0100, Arnout Vandecappelle wrote:
> 
> 
> On 11/03/2025 03:48, Charlie Jenkins wrote:
> > External toolchains like crosstool-ng place libraries at /lib.
> > When BR2_ROOTFS_MERGED_USR is enabled, these libraries were not being
> > copied into the target. This caused these toolchains to be unusable with
> > this option.
> > 
> > WHen BR2_ROOTFS_MERGED_USR is selected, copy the /lib and /lib64
> > directories into /usr/lib and /usr/lib64 respectively.
> 
>  This should already happen, because the lib -> usr/lib symlink should be
> created in staging dir before the toolchain sysroot is copied.
> 
>  But you reported in the other thread that the problem is actually with the
> symlink:
> 
> > $ ls -l ./output/host/riscv64-buildroot-linux-gnu/sysroot/usr/lib/libm.so
> > lrwxrwxrwx 1 charlie rvs 6 Mar  9 22:58 ./output/host/riscv64-buildroot-linux-gnu/sysroot/usr/lib/libm.so -> ../../
> 
>  So this looks liek the symlink creation is going wrong. That is done by the
> following code:
> 
> 	for link in $$(find $(STAGING_DIR) -type l); do \
> 		target=$$(readlink $${link}) ; \
> 		if [ "$${target}" == "$${target$(SHARP_SIGN)/}" ] ; then \
> 			continue ; \
> 		fi ; \
> 		relpath="$(call relpath_prefix,$${target$(SHARP_SIGN)/})" ; \
> 		echo "Fixing symlink $${link} from $${target} to
> $${relpath}$${target$(SHARP_SIGN)/}" ; \
> 		ln -sf $${relpath}$${target$(SHARP_SIGN)/} $${link} ; \
> 	done ; \
> 	relpath="$(call relpath_prefix,$${ARCH_LIB_DIR})" ; \
> 	if [ "$${relpath}" != "" ]; then \
> 		for i in $$(find -H $(STAGING_DIR)/$${ARCH_LIB_DIR}
> $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} -type l -xtype l); do \
> 			LINKTARGET=`readlink $$i` ; \
> 			NEWLINKTARGET=$${LINKTARGET\#$$relpath} ; \
> 			ln -sf $${NEWLINKTARGET} $$i ; \
> 			$(call simplify_symlink,$$i,$(STAGING_DIR)) ; \
> 		done ; \
> 	fi ; \
> 
>  So I guess there is a symlink from usr/lib/libm.so to ../../lib/libm.so in
> the ct-ng toolchain, and that this doesn't get handled correctly by the code
> above. But honestly that code is not xxactly readable...

Are you suggesting that this is a problem with crosstool-ng and not with
Buildroot?

> 
> > 
> > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
> > ---
> > I originally reported this issue here [1].
> > 
> > [1] https://lore.kernel.org/buildroot/Z86AGdrKHaUqgLVt@ghost/T/#t
> > ---
> >   toolchain/helpers.mk | 7 +++++++
> >   1 file changed, 7 insertions(+)
> > 
> > diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk
> > index f3fdaaec07c5a01688da685c4d0fd4e2cb357b95..b565da8a3a8f12567920f14dd3b6a18e548da6c7 100644
> > --- a/toolchain/helpers.mk
> > +++ b/toolchain/helpers.mk
> > @@ -55,6 +55,9 @@ copy_toolchain_lib_root = \
> >   # corresponding architecture variants), and we don't want to import
> >   # them.
> >   #
> > +# Furthermore, when BR2_ROOTFS_MERGED_USR is enabled, the 'lib*' directories
> > +# need to be copied over to usr/'lib*'.
> > +#
> >   # If ARCH_LIB_DIR is not a singular directory component, e.g.
> >   # 'lib32/octeon2', then symbolic links in ARCH_LIB_DIR and
> >   # usr/ARCH_LIB_DIR may be broken because Buildroot will flatten the
> > @@ -109,6 +112,10 @@ copy_toolchain_sysroot = \
> >   	ARCH_SUBDIR="$(strip $3)"; \
> >   	ARCH_LIB_DIR="$(strip $4)" ; \
> >   	SUPPORT_LIB_DIR="$(strip $5)" ; \
> > +	if [ "$(BR2_ROOTFS_MERGED_USR)" == "y" ]; then \
> > +		rsync -au --chmod=u=rwX,go=rX $${ARCH_SYSROOT_DIR}/lib $(STAGING_DIR)/usr/lib 1>&2 ; \
> > +		rsync -au --chmod=u=rwX,go=rX $${ARCH_SYSROOT_DIR}/$${ARCH_LIB_DIR} $(STAGING_DIR)/usr/$${ARCH_LIB_DIR} 1>&2 ; \
> 
>  It looks suspicious to me that you copy both lib and ARCH_LIB_DIR - only
> one of them should be valid (or, usually, one is a symlink to the other).
> Also, ARCH_LIB_DIR is already handled in the loop below (in a slightly
> different way, because thanks to the trailing /, the lib -> usr/lib symlink
> is preserved).

Without both, the symlinks are not preserved. Getting rid of the
ARCH_LIB_DIR statement here results in the symlink being garbage:

./host/riscv64-buildroot-linux-gnu/sysroot/usr/lib/libm.so -> ../../

- Charlie

> 
>  Regards,
>  Arnout
> 
> > +	fi ; \
> >   	for i in etc $${ARCH_LIB_DIR} sbin usr usr/$${ARCH_LIB_DIR}; do \
> >   		if [ ! -d $${ARCH_SYSROOT_DIR}/$$i ] ; then \
> >   			continue ; \
> > 
> > ---
> > base-commit: 4809690d42f9b395ccb76ee7583c199fdd8d42c5
> > change-id: 20250310-fix_external_toolchains-d1b8c4da0be8
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot