Re: [OE-core] [scarthgap] recipe's sysroot misconfiguration in multilib

[Joao Marcos Costa <joaomarcos.costa@bootlin.com>]

Hello,

On 3/26/25 20:26, Khem Raj via lists.openembedded.org wrote:
> On Wed, Mar 26, 2025 at 12:15 PM Joao Marcos Costa
> <joaomarcos.costa@bootlin.com> wrote:
>>
>> Hello, Richard
>>
>> Thanks for your reply
>>
>> On 3/26/25 19:55, Richard Purdie via lists.openembedded.org wrote:
>>> On Wed, 2025-03-26 at 19:47 +0100, Joao Marcos Costa wrote:
>>>> I have been trying to set up a multilib environment to generate
>>>> libraries built with Control Flow Integrity support. The gcc toolchain
>>>> is already patched to support this custom multilib ABI, and it is indeed
>>>> capable of generating binaries with the CFI flags.
>>>>
>>>> I am, however, facing this strange issue where a header is not found in
>>>> the multilib recipe-sysroot (i.e., libcfi-recipe-sysroot) of a given
>>>> package (e.g., zstd), and do_compile task fails:
>>>>
>> (...)
>>>>
>>>> Besides my TUNE_*, DEFAULTTUNE, BASE_LIB values set up, I also added
>>>> 'GCCMULTILIB = "--enable-multilib"' to gcc-cross.inc, to enable multilib
>>>> for the cross toolchain. Could I be missing something, maybe with
>>>> recipes-devtools/gcc/gcc-multilib-config.inc?
>>>
>>> I'm finding it odd that you have to add --enable-multilib since in
>>> general with gcc-cross, we use separate cross toolchains for each
>>> multilib. This kind of implies you're configuring the setup differently
>>> to how the system would expect.
>>
>> If I don't, my cross toolchain is configured with "--disable-multilib".
>> To confirm this, I ran gcc with "-v" from within the recipe's native
>> sysroot, and the output is:
>>
>> Target: riscv64-freedomusdkmllibcfi-linux
>> Configured with:
>> ../../../../../../work-shared/gcc-internal-13.2.0-r0/git/configure
>> --build=x86_64-linux --host=x86_64-linux
>> --target=riscv64-freedomusdkmllibcfi-linux --prefix=//host/usr
>> --exec_prefix=//host/usr
>> --bindir=//host/usr/bin/riscv64-freedomusdkmllibcfi-linux
>> --sbindir=//host/usr/bin/riscv64-freedomusdkmllibcfi-linux
>> --libexecdir=//host/usr/libexec/riscv64-freedomusdkmllibcfi-linux
>> --datadir=//host/usr/share --sysconfdir=//host/etc
>> --sharedstatedir=//host/com --localstatedir=//host/var
>> --libdir=//host/usr/lib/riscv64-freedomusdkmllibcfi-linux
>> --includedir=//host/usr/include --oldincludedir=//host/usr/include
>> --infodir=//host/usr/share/info --mandir=//host/usr/share/man
>> --disable-silent-rules --disable-dependency-tracking
>> --with-libtool-sysroot=//host --enable-clocale=generic --with-gnu-ld
>> --enable-shared --enable-languages=c,c++,fortran --enable-threads=posix
>> --disable-multilib --enable-default-pie --enable-c99 --enable-long-long
>> --enable-(...)
>>
>> As a consequence, the build fails even before it gets to libcfi-zstd:
>>
>> ...libcfi-glibc-internal/2.37/recipe-sysroot-native/usr/bin/riscv64-freedomusdkmllibcfi-linux/../../libexec/riscv64-freedomusdkmllibcfi-linux/gcc/riscv64-freedomusdkmllibcfi-linux/13.2.0/ld.bfd:
>> cannot find -lgcc: No such file or directory
>> | collect2: error: ld returned 1 exit status
>> | make[2]: *** [Makefile:1293:
> 
> OE's notion of multilib is at system level, which means it will need
> full runtimes for the chosen multilibs, GCC builds the compiler
> runtime and has its own
> directory naming structure to place them which may not be inline with
> what OE toolchain is doing. For hosted targets we do not build a
> toolchain with all
> possible multilibs for a given architecture which technically is
> possible with gcc. our mutlilibs are a small subset of that and that
> is what toolchain will recognise
> provided its placed in right locations in sysroot.
> 

Thanks, Khem.

After your reply and Richard's, I checked with a Poky build (arm64 + 
lib32 multilib), and doing the same test (i.e., gcc -v) I see both 
compilers (64-bit and 32-bit) were configured with "--disable-multilib". 
I will get rid of that GCCMULTILIB then.

Please tell me if I got your point correctly: each multilib ABI (lib32, 
lib64, libfoo, libbar) will have it's respective toolchain built, and as 
long as the toolchain's naming matches what OE expects in terms of 
sysroot structure, everything will work transparently.

In case I got you right, how can I tell what exactly OE expects of the 
sysroot locations and directory names? I ask this specially because in 
my arm64 + lib32 test, the target sysroots are:

lib32-zstd/1.5.6/lib32-recipe-sysroot/usr/lib
zstd/1.5.6/recipe-sysroot/usr/lib (with a symlink inside to lib64)
zstd/1.5.6/recipe-sysroot/usr/lib64

and to be fair, I was expecting the very opposite. Something like:
lib32-zstd/1.5.6/lib32-recipe-sysroot/usr/lib32
zstd/1.5.6/recipe-sysroot/usr/lib

In other words, the target architecture libdir would remain the same 
(i.e. usr/lib) and the multilib variant would be usr/lib32).

Thanks once more.

-- 
Best regards,
João Marcos Costa