Re: [OE-core] Current native SDK glibc compat

["Anatol Belski" <anbelski@linux.microsoft.com>]

On 2/24/2021 5:49 PM, Richard Purdie wrote:
> On Wed, 2021-02-24 at 13:56 +0100, Anatol Belski wrote:
>> On 2/24/2021 1:32 PM, Richard Purdie wrote:
>>> Hi,
>>>
>>> On Wed, 2021-02-24 at 12:40 +0100, Anatol Belski wrote:
>>>> the current master build seems to be broken with symbols unavailable
>>>> from the host glibc. The following is to see on the SDK built and
>>>> installed on the same host Ubuntu 18.04.5 having glibc 2.27:
>>>>
>>>> $ . /tmp/poky-sdk-master-00/environment-setup-core2-64-poky-linux
>>>>
>>>> $ ldd $(which $CC)
>>>> /tmp/poky-sdk-master-00/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc:
>>>> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
>>> We change the loader path inside our SDK binaries so you can't trust the
>>> output from ldd, it will find a different result to what you'd see
>>> when you run the binary.
>>>
>>> What issue are you seeing trying to run these?
>> Initially it was sighted here appearing when a binary is actually invoked:
>>
>> https://github.com/meta-rust/meta-rust/pull/313#issuecomment-782784056
>>
>> I went digging to see similar cases.
>>
>>
>> Regarding the loader path, are you referring to this?
>>
>> $ chrpath $(which $CC)
>> /tmp/poky-sdk-master-00/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc:
>> RPATH=$ORIGIN/../../lib
> No, I mean the dynamic loader pointer.
>
> $ tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative python3-native/python3.9 --print-interpreter
> [...]tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
>
> Above I'm showing that a native binary in the build (python3-native)
> has the interpreter (dynamic loader) set to our uninative ld.so.
> The SDK is similar.
>
>> As the binary where the issue was sighted has this
>>
>> $ chrpath $(which cargo)
>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/cargo:
>> RUNPATH=$ORIGIN/../lib
>>
>>
>> but then, the DSOs have no rpath set, eg.
>>
>> $ chrpath
>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcrypto.so.1.1
>> /tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/../lib/libcrypto.so.1.1:
>> no rpath or runpath tag found.
>>
>>
>> so it might lead to the interferrence with the host. Does it perhaps
>> need both $ORIGIN/../../lib and $ORIGIN/../lib if binaries are in /usr ?
> Our dynamic loader knows how to use the specific sysroot and then
> fall back to /usr and /lib.

Thanks a lot for explaining this.

Getting back to the initial case led to the question, i'm now able to 
check what is the correct dynamic loader:

$ tmp/sysroots-uninative/x86_64-linux/usr/bin/patchelf-uninative 
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/usr/bin/cargo 
--print-interpreter
/tmp/rust-sdk-deploy-18/sysroots/x86_64-pokysdk-linux/lib/ld-linux-x86-64.so.2

and also could use --print-needed which would be similar to "readelf -d 
", but it'd be still not possible to check things at the runtime? OFC 
knowing what i know now, it's possible to locate the DSO and check 
symbols like this:

$ x86_64-poky-linux-objdump -T 
./sysroots/x86_64-pokysdk-linux/lib/libc.so.6 | grep GLIBC_2.33

just seems the catch points are quite subtle :) Perhaps there could be a 
recommendation on a good way validating the binaries in the SDK HOST part?

Thanks

Anatol