[PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Ovidiu Panait]

Extend the functionality provided by commit [1] to the SDK as well. This way we
can make sure that nativesdk-binutils finds SDK libraries first rather than
host ones.

This is useful for example when trying to build the linux kernel using
nativesdk-gcc. This scenario currently fails because it tries to link to host
libraries rather than SDK host ones:

make x86_64_defconfig
make bzImage
...
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile:1101: recipe for target 'prepare-objtool' failed
make: *** [prepare-objtool] Error 1
....
/../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
...

[1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
---
 meta/classes/populate_sdk_base.bbclass       | 5 +++++
 meta/recipes-core/meta/buildtools-tarball.bb | 4 ----
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index ef0d8bef58..6a1ebc8c02 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -185,6 +185,11 @@ fakeroot create_sdk_files() {
 	# Escape special characters like '+' and '.' in the SDKPATH
 	escaped_sdkpath=$(echo ${SDKPATH} |sed -e "s:[\+\.]:\\\\\\\\\0:g")
 	sed -i -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" ${SDK_OUTPUT}/${SDKPATH}/relocate_sdk.py
+
+       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
+       echo '${SDKPATHNATIVE}${libdir}
+${SDKPATHNATIVE}${base_libdir}
+include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
 }
 
 python check_sdk_sysroots() {
diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index 977b6a5e1d..c49802eef8 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -76,10 +76,6 @@ create_sdk_files_append () {
 	echo 'export GIT_SSL_CAINFO="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
 	echo 'export OPENSSL_CONF="${SDKPATHNATIVE}${sysconfdir}/ssl/openssl.cnf"' >>$script
 
-	mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
-	echo '${SDKPATHNATIVE}${libdir}
-${SDKPATHNATIVE}${base_libdir}
-include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
 	if [ "${SDKMACHINE}" = "i686" ]; then
 		echo 'export NO32LIBS="0"' >>$script
 		echo 'echo "$BB_ENV_EXTRAWHITE" | grep -q "NO32LIBS"' >>$script
-- 
2.17.1

[PATCH 2/2] populate_sdk_base: create_sdk_files: Use _nativesdk variants for libdir and base_libdir

[Ovidiu Panait]

Use the _nativesdk variants for libdir and base_libdir to fix the resulting
ld.so.conf paths in the SDK for systems with libdir=/lib64.

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
---
 meta/classes/populate_sdk_base.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index 6a1ebc8c02..f85c3b9f62 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -187,8 +187,8 @@ fakeroot create_sdk_files() {
 	sed -i -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" ${SDK_OUTPUT}/${SDKPATH}/relocate_sdk.py
 
        mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
-       echo '${SDKPATHNATIVE}${libdir}
-${SDKPATHNATIVE}${base_libdir}
+       echo '${SDKPATHNATIVE}${libdir_nativesdk}
+${SDKPATHNATIVE}${base_libdir_nativesdk}
 include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
 }
 
-- 
2.17.1

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Steve Sakoman]

Should I consider these two patches for dunfell LTS?

Thanks!

Steve

On Tue, Apr 28, 2020 at 2:59 AM Ovidiu Panait
<ovidiu.panait@windriver.com> wrote:
>
> Extend the functionality provided by commit [1] to the SDK as well. This way we
> can make sure that nativesdk-binutils finds SDK libraries first rather than
> host ones.
>
> This is useful for example when trying to build the linux kernel using
> nativesdk-gcc. This scenario currently fails because it tries to link to host
> libraries rather than SDK host ones:
>
> make x86_64_defconfig
> make bzImage
> ...
> error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
> Makefile:1101: recipe for target 'prepare-objtool' failed
> make: *** [prepare-objtool] Error 1
> ....
> /../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
> ...
>
> [1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]
>
> Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
> ---
>  meta/classes/populate_sdk_base.bbclass       | 5 +++++
>  meta/recipes-core/meta/buildtools-tarball.bb | 4 ----
>  2 files changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
> index ef0d8bef58..6a1ebc8c02 100644
> --- a/meta/classes/populate_sdk_base.bbclass
> +++ b/meta/classes/populate_sdk_base.bbclass
> @@ -185,6 +185,11 @@ fakeroot create_sdk_files() {
>         # Escape special characters like '+' and '.' in the SDKPATH
>         escaped_sdkpath=$(echo ${SDKPATH} |sed -e "s:[\+\.]:\\\\\\\\\0:g")
>         sed -i -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" ${SDK_OUTPUT}/${SDKPATH}/relocate_sdk.py
> +
> +       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
> +       echo '${SDKPATHNATIVE}${libdir}
> +${SDKPATHNATIVE}${base_libdir}
> +include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
>  }
>
>  python check_sdk_sysroots() {
> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
> index 977b6a5e1d..c49802eef8 100644
> --- a/meta/recipes-core/meta/buildtools-tarball.bb
> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
> @@ -76,10 +76,6 @@ create_sdk_files_append () {
>         echo 'export GIT_SSL_CAINFO="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
>         echo 'export OPENSSL_CONF="${SDKPATHNATIVE}${sysconfdir}/ssl/openssl.cnf"' >>$script
>
> -       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
> -       echo '${SDKPATHNATIVE}${libdir}
> -${SDKPATHNATIVE}${base_libdir}
> -include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
>         if [ "${SDKMACHINE}" = "i686" ]; then
>                 echo 'export NO32LIBS="0"' >>$script
>                 echo 'echo "$BB_ENV_EXTRAWHITE" | grep -q "NO32LIBS"' >>$script
> --
> 2.17.1
>
> 

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Khem Raj]

On 5/3/20 3:51 PM, Steve Sakoman wrote:
> Should I consider these two patches for dunfell LTS?
> 

Yes I think they are fixing a problem and are safe.

> Thanks!
> 
> Steve
> 
> On Tue, Apr 28, 2020 at 2:59 AM Ovidiu Panait
> <ovidiu.panait@windriver.com> wrote:
>>
>> Extend the functionality provided by commit [1] to the SDK as well. This way we
>> can make sure that nativesdk-binutils finds SDK libraries first rather than
>> host ones.
>>
>> This is useful for example when trying to build the linux kernel using
>> nativesdk-gcc. This scenario currently fails because it tries to link to host
>> libraries rather than SDK host ones:
>>
>> make x86_64_defconfig
>> make bzImage
>> ...
>> error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
>> Makefile:1101: recipe for target 'prepare-objtool' failed
>> make: *** [prepare-objtool] Error 1
>> ....
>> /../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
>> ...
>>
>> [1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]
>>
>> Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
>> ---
>>  meta/classes/populate_sdk_base.bbclass       | 5 +++++
>>  meta/recipes-core/meta/buildtools-tarball.bb | 4 ----
>>  2 files changed, 5 insertions(+), 4 deletions(-)
>>
>> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
>> index ef0d8bef58..6a1ebc8c02 100644
>> --- a/meta/classes/populate_sdk_base.bbclass
>> +++ b/meta/classes/populate_sdk_base.bbclass
>> @@ -185,6 +185,11 @@ fakeroot create_sdk_files() {
>>         # Escape special characters like '+' and '.' in the SDKPATH
>>         escaped_sdkpath=$(echo ${SDKPATH} |sed -e "s:[\+\.]:\\\\\\\\\0:g")
>>         sed -i -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" ${SDK_OUTPUT}/${SDKPATH}/relocate_sdk.py
>> +
>> +       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
>> +       echo '${SDKPATHNATIVE}${libdir}
>> +${SDKPATHNATIVE}${base_libdir}
>> +include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
>>  }
>>
>>  python check_sdk_sysroots() {
>> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
>> index 977b6a5e1d..c49802eef8 100644
>> --- a/meta/recipes-core/meta/buildtools-tarball.bb
>> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
>> @@ -76,10 +76,6 @@ create_sdk_files_append () {
>>         echo 'export GIT_SSL_CAINFO="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
>>         echo 'export OPENSSL_CONF="${SDKPATHNATIVE}${sysconfdir}/ssl/openssl.cnf"' >>$script
>>
>> -       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
>> -       echo '${SDKPATHNATIVE}${libdir}
>> -${SDKPATHNATIVE}${base_libdir}
>> -include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
>>         if [ "${SDKMACHINE}" = "i686" ]; then
>>                 echo 'export NO32LIBS="0"' >>$script
>>                 echo 'echo "$BB_ENV_EXTRAWHITE" | grep -q "NO32LIBS"' >>$script
>> --
>> 2.17.1
>>
>>
>>
>> 

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Denys Dmytriyenko]

On Thu, May 07, 2020 at 12:44:15PM -0700, Khem Raj wrote:
> 
> 
> On 5/3/20 3:51 PM, Steve Sakoman wrote:
> > Should I consider these two patches for dunfell LTS?
> > 
> 
> Yes I think they are fixing a problem and are safe.

I don't think it is safe though!

The code has been moved to create_sdk_files() which is a POSTPROCESS command 
for SDK creation. What if SDK already provides own /etc/ld.so.conf? The code 
just simply overwrites it.

-- 
Denys


> > Thanks!
> > 
> > Steve
> > 
> > On Tue, Apr 28, 2020 at 2:59 AM Ovidiu Panait
> > <ovidiu.panait@windriver.com> wrote:
> >>
> >> Extend the functionality provided by commit [1] to the SDK as well. This way we
> >> can make sure that nativesdk-binutils finds SDK libraries first rather than
> >> host ones.
> >>
> >> This is useful for example when trying to build the linux kernel using
> >> nativesdk-gcc. This scenario currently fails because it tries to link to host
> >> libraries rather than SDK host ones:
> >>
> >> make x86_64_defconfig
> >> make bzImage
> >> ...
> >> error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
> >> Makefile:1101: recipe for target 'prepare-objtool' failed
> >> make: *** [prepare-objtool] Error 1
> >> ....
> >> /../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
> >> ...
> >>
> >> [1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]
> >>
> >> Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
> >> ---
> >>  meta/classes/populate_sdk_base.bbclass       | 5 +++++
> >>  meta/recipes-core/meta/buildtools-tarball.bb | 4 ----
> >>  2 files changed, 5 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
> >> index ef0d8bef58..6a1ebc8c02 100644
> >> --- a/meta/classes/populate_sdk_base.bbclass
> >> +++ b/meta/classes/populate_sdk_base.bbclass
> >> @@ -185,6 +185,11 @@ fakeroot create_sdk_files() {
> >>         # Escape special characters like '+' and '.' in the SDKPATH
> >>         escaped_sdkpath=$(echo ${SDKPATH} |sed -e "s:[\+\.]:\\\\\\\\\0:g")
> >>         sed -i -e "s:##DEFAULT_INSTALL_DIR##:$escaped_sdkpath:" ${SDK_OUTPUT}/${SDKPATH}/relocate_sdk.py
> >> +
> >> +       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
> >> +       echo '${SDKPATHNATIVE}${libdir}
> >> +${SDKPATHNATIVE}${base_libdir}
> >> +include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
> >>  }
> >>
> >>  python check_sdk_sysroots() {
> >> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
> >> index 977b6a5e1d..c49802eef8 100644
> >> --- a/meta/recipes-core/meta/buildtools-tarball.bb
> >> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
> >> @@ -76,10 +76,6 @@ create_sdk_files_append () {
> >>         echo 'export GIT_SSL_CAINFO="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
> >>         echo 'export OPENSSL_CONF="${SDKPATHNATIVE}${sysconfdir}/ssl/openssl.cnf"' >>$script
> >>
> >> -       mkdir -p ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/
> >> -       echo '${SDKPATHNATIVE}${libdir}
> >> -${SDKPATHNATIVE}${base_libdir}
> >> -include /etc/ld.so.conf' > ${SDK_OUTPUT}/${SDKPATHNATIVE}${sysconfdir}/ld.so.conf
> >>         if [ "${SDKMACHINE}" = "i686" ]; then
> >>                 echo 'export NO32LIBS="0"' >>$script
> >>                 echo 'echo "$BB_ENV_EXTRAWHITE" | grep -q "NO32LIBS"' >>$script
> >> --
> >> 2.17.1
> >>
> >>
> >>
> >> 

> 

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Khem Raj]

On 5/7/20 12:57 PM, Denys Dmytriyenko wrote:
> On Thu, May 07, 2020 at 12:44:15PM -0700, Khem Raj wrote:
>>
>>
>> On 5/3/20 3:51 PM, Steve Sakoman wrote:
>>> Should I consider these two patches for dunfell LTS?
>>>
>>
>> Yes I think they are fixing a problem and are safe.
> 
> I don't think it is safe though!
> 
> The code has been moved to create_sdk_files() which is a POSTPROCESS command 
> for SDK creation. What if SDK already provides own /etc/ld.so.conf? The code 
> just simply overwrites it.
> 

but are there such usecases ? looking closely if SDK is extending the
ld.so.conf then they should be appending to it instead of overwriting
and same is true for this patch for master.

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Denys Dmytriyenko]

On Thu, May 07, 2020 at 01:20:40PM -0700, Khem Raj wrote:
> 
> 
> On 5/7/20 12:57 PM, Denys Dmytriyenko wrote:
> > On Thu, May 07, 2020 at 12:44:15PM -0700, Khem Raj wrote:
> >>
> >>
> >> On 5/3/20 3:51 PM, Steve Sakoman wrote:
> >>> Should I consider these two patches for dunfell LTS?
> >>>
> >>
> >> Yes I think they are fixing a problem and are safe.
> > 
> > I don't think it is safe though!
> > 
> > The code has been moved to create_sdk_files() which is a POSTPROCESS command 
> > for SDK creation. What if SDK already provides own /etc/ld.so.conf? The code 
> > just simply overwrites it.
> > 
> 
> but are there such usecases ? looking closely if SDK is extending the
> ld.so.conf then they should be appending to it instead of overwriting
> and same is true for this patch for master.

Exactly my point - should probably append, not overwrite.
As of usecases - I can think of a non-standard nativesdk tool that adjusts 
/etc/ld.so.conf during install...

-- 
Denys

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Steve Sakoman]

On Thu, May 7, 2020 at 10:30 AM Denys Dmytriyenko <denis@denix.org> wrote:
>
> On Thu, May 07, 2020 at 01:20:40PM -0700, Khem Raj wrote:
> >
> >
> > On 5/7/20 12:57 PM, Denys Dmytriyenko wrote:
> > > On Thu, May 07, 2020 at 12:44:15PM -0700, Khem Raj wrote:
> > >>
> > >>
> > >> On 5/3/20 3:51 PM, Steve Sakoman wrote:
> > >>> Should I consider these two patches for dunfell LTS?
> > >>>
> > >>
> > >> Yes I think they are fixing a problem and are safe.
> > >
> > > I don't think it is safe though!
> > >
> > > The code has been moved to create_sdk_files() which is a POSTPROCESS command
> > > for SDK creation. What if SDK already provides own /etc/ld.so.conf? The code
> > > just simply overwrites it.
> > >
> >
> > but are there such usecases ? looking closely if SDK is extending the
> > ld.so.conf then they should be appending to it instead of overwriting
> > and same is true for this patch for master.
>
> Exactly my point - should probably append, not overwrite.
> As of usecases - I can think of a non-standard nativesdk tool that adjusts
> /etc/ld.so.conf during install...

I think you've convinced me to pass on this for dunfell (at least in
the short term)

If a patch is submitted that appends rather than overwrites I'll
consider the series again once it makes it into master.

If you submit such a patch please tag it with [dunfell] too.

Thanks for the comments!

Steve
> 

Re: [OE-core] [PATCH 1/2] populate_sdk_base: Add ld.so.conf for nativesdk-binutils

[Khem Raj]

On Thu, May 7, 2020 at 4:02 PM Steve Sakoman <sakoman@gmail.com> wrote:
>
> On Thu, May 7, 2020 at 10:30 AM Denys Dmytriyenko <denis@denix.org> wrote:
> >
> > On Thu, May 07, 2020 at 01:20:40PM -0700, Khem Raj wrote:
> > >
> > >
> > > On 5/7/20 12:57 PM, Denys Dmytriyenko wrote:
> > > > On Thu, May 07, 2020 at 12:44:15PM -0700, Khem Raj wrote:
> > > >>
> > > >>
> > > >> On 5/3/20 3:51 PM, Steve Sakoman wrote:
> > > >>> Should I consider these two patches for dunfell LTS?
> > > >>>
> > > >>
> > > >> Yes I think they are fixing a problem and are safe.
> > > >
> > > > I don't think it is safe though!
> > > >
> > > > The code has been moved to create_sdk_files() which is a POSTPROCESS command
> > > > for SDK creation. What if SDK already provides own /etc/ld.so.conf? The code
> > > > just simply overwrites it.
> > > >
> > >
> > > but are there such usecases ? looking closely if SDK is extending the
> > > ld.so.conf then they should be appending to it instead of overwriting
> > > and same is true for this patch for master.
> >
> > Exactly my point - should probably append, not overwrite.
> > As of usecases - I can think of a non-standard nativesdk tool that adjusts
> > /etc/ld.so.conf during install...
>
> I think you've convinced me to pass on this for dunfell (at least in
> the short term)
>
> If a patch is submitted that appends rather than overwrites I'll
> consider the series again once it makes it into master.
>

yes, I think that will be right thing to do although I think it fills
in a good gap in dunfell
but as it is might not work as expected.

> If you submit such a patch please tag it with [dunfell] too.
>
> Thanks for the comments!
>
> Steve
> >