[PATCH] binutils: Enable --enable-new-dtags

[PATCH] binutils: Enable --enable-new-dtags

[Khem Raj]

Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
DT_RUNPATH, this order ensures that injecting
malicious shared objects is way harder with DT_RUNPATH.

This is now default on major linux distributions already

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-devtools/binutils/binutils.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
index b2dbf241df..c69d29448f 100644
--- a/meta/recipes-devtools/binutils/binutils.inc
+++ b/meta/recipes-devtools/binutils/binutils.inc
@@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
                 --disable-werror \
                 --enable-deterministic-archives \
                 --enable-plugins \
+                --enable-new-dtags \
                 --disable-gdb \
                 --disable-gdbserver \
                 --disable-libdecnumber \
-- 
2.39.2

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Alexandre Belloni]

Could this be the cause of this?

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230223-79c9rmcw/packages/diff-html/

On 22/02/2023 22:58:16-0800, Khem Raj wrote:
> Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> DT_RUNPATH, this order ensures that injecting
> malicious shared objects is way harder with DT_RUNPATH.
> 
> This is now default on major linux distributions already
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/binutils/binutils.inc | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> index b2dbf241df..c69d29448f 100644
> --- a/meta/recipes-devtools/binutils/binutils.inc
> +++ b/meta/recipes-devtools/binutils/binutils.inc
> @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
>                  --disable-werror \
>                  --enable-deterministic-archives \
>                  --enable-plugins \
> +                --enable-new-dtags \
>                  --disable-gdb \
>                  --disable-gdbserver \
>                  --disable-libdecnumber \
> -- 
> 2.39.2
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177602): https://lists.openembedded.org/g/openembedded-core/message/177602
> Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Khem Raj]

On Thu, Feb 23, 2023 at 3:34 PM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> Could this be the cause of this?
>
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230223-79c9rmcw/packages/diff-html/

most likely yes. I will take a look. Lets work on merging the other
binutils and gdb upgrade patch meanwhile.

>
> On 22/02/2023 22:58:16-0800, Khem Raj wrote:
> > Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> > DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> > DT_RUNPATH, this order ensures that injecting
> > malicious shared objects is way harder with DT_RUNPATH.
> >
> > This is now default on major linux distributions already
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/binutils/binutils.inc | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > index b2dbf241df..c69d29448f 100644
> > --- a/meta/recipes-devtools/binutils/binutils.inc
> > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
> >                  --disable-werror \
> >                  --enable-deterministic-archives \
> >                  --enable-plugins \
> > +                --enable-new-dtags \
> >                  --disable-gdb \
> >                  --disable-gdbserver \
> >                  --disable-libdecnumber \
> > --
> > 2.39.2
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#177602): https://lists.openembedded.org/g/openembedded-core/message/177602
> > Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Khem Raj]

On Thu, Feb 23, 2023 at 5:56 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> On Thu, Feb 23, 2023 at 3:34 PM Alexandre Belloni
> <alexandre.belloni@bootlin.com> wrote:
> >
> > Could this be the cause of this?
> >
> > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230223-79c9rmcw/packages/diff-html/
>
> most likely yes. I will take a look. Lets work on merging the other
> binutils and gdb upgrade patch meanwhile.

I looked at the diffoscope outputs you posted and it is noting the
difference in EFL flags

changing from
 0x0000000000000018 (BIND_NOW)
to
 0x000000000000001e (FLAGS)              BIND_NOW

However, in my local build I see that when I enable --enable-new-dtags
 than I always get
 0x000000000000001e (FLAGS)              BIND_NOW


but when I check the glibc builds without the patch then I get flags
to be 0x0000000000000018

So I wonder if we are comparing previously built glibc ( using
binutils without this patch )
with glibc compiled with binutils using this patch.

>
> >
> > On 22/02/2023 22:58:16-0800, Khem Raj wrote:
> > > Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> > > DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> > > DT_RUNPATH, this order ensures that injecting
> > > malicious shared objects is way harder with DT_RUNPATH.
> > >
> > > This is now default on major linux distributions already
> > >
> > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > ---
> > >  meta/recipes-devtools/binutils/binutils.inc | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > > index b2dbf241df..c69d29448f 100644
> > > --- a/meta/recipes-devtools/binutils/binutils.inc
> > > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > > @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
> > >                  --disable-werror \
> > >                  --enable-deterministic-archives \
> > >                  --enable-plugins \
> > > +                --enable-new-dtags \
> > >                  --disable-gdb \
> > >                  --disable-gdbserver \
> > >                  --disable-libdecnumber \
> > > --
> > > 2.39.2
> > >
> >
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#177602): https://lists.openembedded.org/g/openembedded-core/message/177602
> > > Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
> >
> >
> > --
> > Alexandre Belloni, co-owner and COO, Bootlin
> > Embedded Linux and Kernel engineering
> > https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Sun, 2023-02-26 at 22:43 -0800, Khem Raj wrote:
> On Thu, Feb 23, 2023 at 5:56 PM Khem Raj <raj.khem@gmail.com> wrote:
> > 
> > On Thu, Feb 23, 2023 at 3:34 PM Alexandre Belloni
> > <alexandre.belloni@bootlin.com> wrote:
> > > 
> > > Could this be the cause of this?
> > > 
> > > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230223-79c9rmcw/packages/diff-html/
> > 
> > most likely yes. I will take a look. Lets work on merging the other
> > binutils and gdb upgrade patch meanwhile.
> 
> I looked at the diffoscope outputs you posted and it is noting the
> difference in EFL flags
> 
> changing from
>  0x0000000000000018 (BIND_NOW)
> to
>  0x000000000000001e (FLAGS)              BIND_NOW
> 
> However, in my local build I see that when I enable --enable-new-dtags
>  than I always get
>  0x000000000000001e (FLAGS)              BIND_NOW
> 
> 
> but when I check the glibc builds without the patch then I get flags
> to be 0x0000000000000018
> 
> So I wonder if we are comparing previously built glibc ( using
> binutils without this patch )
> with glibc compiled with binutils using this patch.

I think that is possible and there is some task hash dependency issue
in here somewhere :/

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Alexandre Belloni]

Hello Khem,

As discussed I gave it a go again and got this:

| /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
| linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
| /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
| linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
| collect2: error: ld returned 1 exit status
| make[2]: *** [Makefile:2149: gdb] Error 1
| make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
| make[1]: *** [Makefile:11122: all-gdb] Error 2
| make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
| make: *** [Makefile:1005: all] Error 2
| ERROR: oe_runmake failed

https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/6741/steps/14/logs/stdio

I already reported it on the gdb upgrade:

https://lists.openembedded.org/g/openembedded-core/topic/97152035#177576


On 22/02/2023 22:58:16-0800, Khem Raj wrote:
> Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> DT_RUNPATH, this order ensures that injecting
> malicious shared objects is way harder with DT_RUNPATH.
> 
> This is now default on major linux distributions already
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/binutils/binutils.inc | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> index b2dbf241df..c69d29448f 100644
> --- a/meta/recipes-devtools/binutils/binutils.inc
> +++ b/meta/recipes-devtools/binutils/binutils.inc
> @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
>                  --disable-werror \
>                  --enable-deterministic-archives \
>                  --enable-plugins \
> +                --enable-new-dtags \
>                  --disable-gdb \
>                  --disable-gdbserver \
>                  --disable-libdecnumber \
> -- 
> 2.39.2
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177602): https://lists.openembedded.org/g/openembedded-core/message/177602
> Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Khem Raj]

On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> Hello Khem,
>
> As discussed I gave it a go again and got this:
>
> | /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> | linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> | /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> | linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> | collect2: error: ld returned 1 exit status
> | make[2]: *** [Makefile:2149: gdb] Error 1
> | make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> | make[1]: *** [Makefile:11122: all-gdb] Error 2
> | make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> | make: *** [Makefile:1005: all] Error 2
> | ERROR: oe_runmake failed

Is this host running updated buildtools tarball after the binutils ld
search path fix ?

>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/6741/steps/14/logs/stdio
>
> I already reported it on the gdb upgrade:
>
> https://lists.openembedded.org/g/openembedded-core/topic/97152035#177576
>
>
> On 22/02/2023 22:58:16-0800, Khem Raj wrote:
> > Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> > DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> > DT_RUNPATH, this order ensures that injecting
> > malicious shared objects is way harder with DT_RUNPATH.
> >
> > This is now default on major linux distributions already
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/binutils/binutils.inc | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > index b2dbf241df..c69d29448f 100644
> > --- a/meta/recipes-devtools/binutils/binutils.inc
> > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
> >                  --disable-werror \
> >                  --enable-deterministic-archives \
> >                  --enable-plugins \
> > +                --enable-new-dtags \
> >                  --disable-gdb \
> >                  --disable-gdbserver \
> >                  --disable-libdecnumber \
> > --
> > 2.39.2
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#177602): https://lists.openembedded.org/g/openembedded-core/message/177602
> > Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> <alexandre.belloni@bootlin.com> wrote:
> > 
> > Hello Khem,
> > 
> > As discussed I gave it a go again and got this:
> > 
> > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > collect2: error: ld returned 1 exit status
> > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > make: *** [Makefile:1005: all] Error 2
> > > ERROR: oe_runmake failed
> 
> Is this host running updated buildtools tarball after the binutils ld
> search path fix ?

Yes, it should be.

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Alexandre Belloni]

On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > <alexandre.belloni@bootlin.com> wrote:
> > > 
> > > Hello Khem,
> > > 
> > > As discussed I gave it a go again and got this:
> > > 
> > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > collect2: error: ld returned 1 exit status
> > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > make: *** [Makefile:1005: all] Error 2
> > > > ERROR: oe_runmake failed
> > 
> > Is this host running updated buildtools tarball after the binutils ld
> > search path fix ?
> 
> Yes, it should be.

Khem, you are probably right that this is host related, this failed
again on ubuntu2004:

https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio

I'm wondering whether this reproduces on master and we have simply been
lucky.

-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Alexandre Belloni]

On 28/02/2023 23:45:05+0100, Alexandre Belloni wrote:
> On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> > On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > > <alexandre.belloni@bootlin.com> wrote:
> > > > 
> > > > Hello Khem,
> > > > 
> > > > As discussed I gave it a go again and got this:
> > > > 
> > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > collect2: error: ld returned 1 exit status
> > > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > > make: *** [Makefile:1005: all] Error 2
> > > > > ERROR: oe_runmake failed
> > > 
> > > Is this host running updated buildtools tarball after the binutils ld
> > > search path fix ?
> > 
> > Yes, it should be.
> 
> Khem, you are probably right that this is host related, this failed
> again on ubuntu2004:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio
> 
> I'm wondering whether this reproduces on master and we have simply been
> lucky.

This doesn't reproduce on master on ubuntu2004 so I guess it is really
cause by the binutils upgrade.

The other issue is still repro:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2472/steps/12/logs/stdio
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230228-nmn0h352/packages/diff-html/


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Alexandre Belloni]

On 01/03/2023 10:25:58+0100, Alexandre Belloni via lists.openembedded.org wrote:
> On 28/02/2023 23:45:05+0100, Alexandre Belloni wrote:
> > On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> > > On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > > > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > > > <alexandre.belloni@bootlin.com> wrote:
> > > > > 
> > > > > Hello Khem,
> > > > > 
> > > > > As discussed I gave it a go again and got this:
> > > > > 
> > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > collect2: error: ld returned 1 exit status
> > > > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > > > make: *** [Makefile:1005: all] Error 2
> > > > > > ERROR: oe_runmake failed
> > > > 
> > > > Is this host running updated buildtools tarball after the binutils ld
> > > > search path fix ?
> > > 
> > > Yes, it should be.
> > 
> > Khem, you are probably right that this is host related, this failed
> > again on ubuntu2004:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio
> > 
> > I'm wondering whether this reproduces on master and we have simply been
> > lucky.
> 
> This doesn't reproduce on master on ubuntu2004 so I guess it is really
> cause by the binutils upgrade.

I was wrong, it just reproduced on master, debian11-ty-3:

https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/6730/steps/15/logs/stdio

> 
> The other issue is still repro:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2472/steps/12/logs/stdio
> https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230228-nmn0h352/packages/diff-html/
> 
> 
> -- 
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177879): https://lists.openembedded.org/g/openembedded-core/message/177879
> Mute This Topic: https://lists.openembedded.org/mt/97178429/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Thu, 2023-03-02 at 12:41 +0100, Alexandre Belloni wrote:
> On 01/03/2023 10:25:58+0100, Alexandre Belloni via lists.openembedded.org wrote:
> > On 28/02/2023 23:45:05+0100, Alexandre Belloni wrote:
> > > On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> > > > On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > > > > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > > > > <alexandre.belloni@bootlin.com> wrote:
> > > > > > 
> > > > > > Hello Khem,
> > > > > > 
> > > > > > As discussed I gave it a go again and got this:
> > > > > > 
> > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > collect2: error: ld returned 1 exit status
> > > > > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > > > > make: *** [Makefile:1005: all] Error 2
> > > > > > > ERROR: oe_runmake failed
> > > > > 
> > > > > Is this host running updated buildtools tarball after the binutils ld
> > > > > search path fix ?
> > > > 
> > > > Yes, it should be.
> > > 
> > > Khem, you are probably right that this is host related, this failed
> > > again on ubuntu2004:
> > > 
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio
> > > 
> > > I'm wondering whether this reproduces on master and we have simply been
> > > lucky.
> > 
> > This doesn't reproduce on master on ubuntu2004 so I guess it is really
> > cause by the binutils upgrade.
> 
> I was wrong, it just reproduced on master, debian11-ty-3:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/6730/steps/15/logs/stdio
> 

I went digging and the gdb/config.log file shows:

configure:28568: checking for ELF support in BFD
configure:28588: ./libtool --quiet --mode=link gcc  -o conftest -I../../gdb-13.1/gdb/../include -I../bfd -I../../gdb-13.1/gdb/../bfd -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -O2 -pipe     -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -I/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -L../bfd -L../libiberty conftest.c -lbfd -liberty  -lncursesw -lm -ldl  >&5
/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_join@GLIBC_2.34'
/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_create@GLIBC_2.34'
collect2: error: ld returned 1 exit status

which is the same issue as we saw previously. 

The really big difference here is that debian11 doesn't have buildtools
so this is with the host's ld and libpthread.

I'm guessing this is a libzstd linked on a more recent system which
then doesn't work against this older host libc. The one in uninative
could help and would likely resolve things, but isn't involved at this
point in a build. Not sure what the right fix is here.

Cheers,

Richard



> 

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Thu, 2023-03-02 at 13:18 +0000, Richard Purdie via
lists.openembedded.org wrote:
> On Thu, 2023-03-02 at 12:41 +0100, Alexandre Belloni wrote:
> > On 01/03/2023 10:25:58+0100, Alexandre Belloni via lists.openembedded.org wrote:
> > > On 28/02/2023 23:45:05+0100, Alexandre Belloni wrote:
> > > > On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> > > > > On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > > > > > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > > > > > <alexandre.belloni@bootlin.com> wrote:
> > > > > > > 
> > > > > > > Hello Khem,
> > > > > > > 
> > > > > > > As discussed I gave it a go again and got this:
> > > > > > > 
> > > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > > > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > > > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > > collect2: error: ld returned 1 exit status
> > > > > > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > > > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > > > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > > > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > > > > > make: *** [Makefile:1005: all] Error 2
> > > > > > > > ERROR: oe_runmake failed
> > > > > > 
> > > > > > Is this host running updated buildtools tarball after the binutils ld
> > > > > > search path fix ?
> > > > > 
> > > > > Yes, it should be.
> > > > 
> > > > Khem, you are probably right that this is host related, this failed
> > > > again on ubuntu2004:
> > > > 
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio
> > > > 
> > > > I'm wondering whether this reproduces on master and we have simply been
> > > > lucky.
> > > 
> > > This doesn't reproduce on master on ubuntu2004 so I guess it is really
> > > cause by the binutils upgrade.
> > 
> > I was wrong, it just reproduced on master, debian11-ty-3:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/6730/steps/15/logs/stdio
> > 
> 
> I went digging and the gdb/config.log file shows:
> 
> configure:28568: checking for ELF support in BFD
> configure:28588: ./libtool --quiet --mode=link gcc  -o conftest -I../../gdb-13.1/gdb/../include -I../bfd -I../../gdb-13.1/gdb/../bfd -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -O2 -pipe     -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -I/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -L../bfd -L../libiberty conftest.c -lbfd -liberty  -lncursesw -lm -ldl  >&5
> /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_join@GLIBC_2.34'
> /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_create@GLIBC_2.34'
> collect2: error: ld returned 1 exit status
> 
> which is the same issue as we saw previously. 
> 
> The really big difference here is that debian11 doesn't have buildtools
> so this is with the host's ld and libpthread.
> 
> I'm guessing this is a libzstd linked on a more recent system which
> then doesn't work against this older host libc. The one in uninative
> could help and would likely resolve things, but isn't involved at this
> point in a build. Not sure what the right fix is here.

This bug is horrible.

I reran a forced compile on that build and it all worked. Thankfully I
saved off the build directory, before and after and was able to diff
them.

The difference is this in python3.pc in the recipe-sysroot-native:

Libs.private: -ldl 

vs:

Libs.private: -lpthread -ldl  -lutil

which sticks a -lpthread onto the linker command in the configure tests
within gdb so that it can link correctly to libbfd, which then
convinces its that it has elf symbols.

The underlying issue is that particular gdb test appears to strip our
build time LDFLAGS off for the purposes of the test. I'm guessing
something like:


Index: gdb-13.1/gdb/acinclude.m4
===================================================================
--- gdb-13.1.orig/gdb/acinclude.m4
+++ gdb-13.1/gdb/acinclude.m4
@@ -234,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [
   # points somewhere with bfd, with -I/foo/lib and -L/foo/lib.  We
   # always want our bfd.
   CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS"
-  LDFLAGS="-L../bfd -L../libiberty"
+  LDFLAGS="-L../bfd -L../libiberty $LDFLAGS"
   intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'`
   LIBS="-lbfd -liberty $intl $LIBS"
   CC="./libtool --quiet --mode=link $CC"

would help. There is some irony when you read the comment there. It
looks like it is trying and failing to link to libbfd in the gdb build
anyway rather than one from the host.

I'll have to step away for food and so on shortly so I wanted to brain
dump on what I found so far.

[python3.pc will vary in the native case depending on the host libc so
some libs will be added even if not later needed by uninative]

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Thu, 2023-03-02 at 17:54 +0000, Richard Purdie via
lists.openembedded.org wrote:
> On Thu, 2023-03-02 at 13:18 +0000, Richard Purdie via
> lists.openembedded.org wrote:
> > On Thu, 2023-03-02 at 12:41 +0100, Alexandre Belloni wrote:
> > > On 01/03/2023 10:25:58+0100, Alexandre Belloni via lists.openembedded.org wrote:
> > > > On 28/02/2023 23:45:05+0100, Alexandre Belloni wrote:
> > > > > On 28/02/2023 17:50:05+0000, Richard Purdie wrote:
> > > > > > On Tue, 2023-02-28 at 08:43 -0800, Khem Raj wrote:
> > > > > > > On Tue, Feb 28, 2023 at 8:18 AM Alexandre Belloni
> > > > > > > <alexandre.belloni@bootlin.com> wrote:
> > > > > > > > 
> > > > > > > > Hello Khem,
> > > > > > > > 
> > > > > > > > As discussed I gave it a go again and got this:
> > > > > > > > 
> > > > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_corefile_thread(thread_info*, linux_corefile_thread_data*)':
> > > > > > > > > linux-tdep.c:(.text+0x13ac): undefined reference to `gcore_elf_build_thread_register_notes(gdbarch*, thread_info*, gdb_signal, bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > > > /home/pokybuild/yocto-worker/qemuarm/build/build/tmp/hosttools/ld: linux-tdep.o: in function `linux_make_corefile_notes(gdbarch*, bfd*, int*)':
> > > > > > > > > linux-tdep.c:(.text+0x49d7): undefined reference to `gcore_elf_make_tdesc_note(bfd*, std::unique_ptr<char, gdb::xfree_deleter<char> >*, int*)'
> > > > > > > > > collect2: error: ld returned 1 exit status
> > > > > > > > > make[2]: *** [Makefile:2149: gdb] Error 1
> > > > > > > > > make[2]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi/gdb'
> > > > > > > > > make[1]: *** [Makefile:11122: all-gdb] Error 2
> > > > > > > > > make[1]: Leaving directory '/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/work/x86_64-linux/gdb-cross-arm/13.1-r0/build-arm-poky-linux-gnueabi'
> > > > > > > > > make: *** [Makefile:1005: all] Error 2
> > > > > > > > > ERROR: oe_runmake failed
> > > > > > > 
> > > > > > > Is this host running updated buildtools tarball after the binutils ld
> > > > > > > search path fix ?
> > > > > > 
> > > > > > Yes, it should be.
> > > > > 
> > > > > Khem, you are probably right that this is host related, this failed
> > > > > again on ubuntu2004:
> > > > > 
> > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6689/steps/14/logs/stdio
> > > > > 
> > > > > I'm wondering whether this reproduces on master and we have simply been
> > > > > lucky.
> > > > 
> > > > This doesn't reproduce on master on ubuntu2004 so I guess it is really
> > > > cause by the binutils upgrade.
> > > 
> > > I was wrong, it just reproduced on master, debian11-ty-3:
> > > 
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/6730/steps/15/logs/stdio
> > > 
> > 
> > I went digging and the gdb/config.log file shows:
> > 
> > configure:28568: checking for ELF support in BFD
> > configure:28588: ./libtool --quiet --mode=link gcc  -o conftest -I../../gdb-13.1/gdb/../include -I../bfd -I../../gdb-13.1/gdb/../bfd -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -O2 -pipe     -isystem/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -I/home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/include -L../bfd -L../libiberty conftest.c -lbfd -liberty  -lncursesw -lm -ldl  >&5
> > /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_join@GLIBC_2.34'
> > /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/hosttools/ld: /home/pokybuild/yocto-worker/qemuarm64/build/build/tmp/work/x86_64-linux/gdb-cross-aarch64/13.1-r0/recipe-sysroot-native/usr/lib/libzstd.so: undefined reference to `pthread_create@GLIBC_2.34'
> > collect2: error: ld returned 1 exit status
> > 
> > which is the same issue as we saw previously. 
> > 
> > The really big difference here is that debian11 doesn't have buildtools
> > so this is with the host's ld and libpthread.
> > 
> > I'm guessing this is a libzstd linked on a more recent system which
> > then doesn't work against this older host libc. The one in uninative
> > could help and would likely resolve things, but isn't involved at this
> > point in a build. Not sure what the right fix is here.
> 
> This bug is horrible.
> 
> I reran a forced compile on that build and it all worked. Thankfully I
> saved off the build directory, before and after and was able to diff
> them.
> 
> The difference is this in python3.pc in the recipe-sysroot-native:
> 
> Libs.private: -ldl 
> 
> vs:
> 
> Libs.private: -lpthread -ldl  -lutil
> 
> which sticks a -lpthread onto the linker command in the configure tests
> within gdb so that it can link correctly to libbfd, which then
> convinces its that it has elf symbols.
> 
> The underlying issue is that particular gdb test appears to strip our
> build time LDFLAGS off for the purposes of the test. I'm guessing
> something like:
> 
> 
> Index: gdb-13.1/gdb/acinclude.m4
> ===================================================================
> --- gdb-13.1.orig/gdb/acinclude.m4
> +++ gdb-13.1/gdb/acinclude.m4
> @@ -234,7 +234,7 @@ AC_DEFUN([GDB_AC_CHECK_BFD], [
>    # points somewhere with bfd, with -I/foo/lib and -L/foo/lib.  We
>    # always want our bfd.
>    CFLAGS="-I${srcdir}/../include -I../bfd -I${srcdir}/../bfd $CFLAGS"
> -  LDFLAGS="-L../bfd -L../libiberty"
> +  LDFLAGS="-L../bfd -L../libiberty $LDFLAGS"
>    intl=`echo $LIBINTL | sed 's,${top_builddir}/,,g'`
>    LIBS="-lbfd -liberty $intl $LIBS"
>    CC="./libtool --quiet --mode=link $CC"
> 
> would help. There is some irony when you read the comment there. It
> looks like it is trying and failing to link to libbfd in the gdb build
> anyway rather than one from the host.
> 
> I'll have to step away for food and so on shortly so I wanted to brain
> dump on what I found so far.
> 
> [python3.pc will vary in the native case depending on the host libc so
> some libs will be added even if not later needed by uninative]

I confirmed that:

a) to reproduce you need a broken libzstd which on the autobuilder, I'd
replaced the bad one with a good one. The python3 changes aren't the
issue, they just look suspicious
b) the patch above doesn't work since we don't autoreconf gdb
c) if you patch configure to add $LDFLAGS it does make the build work

The patch works since the LDFLAGS from uninative then get used which
unbreak things.

So we just need to work out the right patch and ideally discuss this
with upstream binutils.

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Wed, 2023-02-22 at 22:58 -0800, Khem Raj wrote:
> Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> DT_RUNPATH, this order ensures that injecting
> malicious shared objects is way harder with DT_RUNPATH.
> 
> This is now default on major linux distributions already
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/binutils/binutils.inc | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> index b2dbf241df..c69d29448f 100644
> --- a/meta/recipes-devtools/binutils/binutils.inc
> +++ b/meta/recipes-devtools/binutils/binutils.inc
> @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
>                  --disable-werror \
>                  --enable-deterministic-archives \
>                  --enable-plugins \
> +                --enable-new-dtags \
>                  --disable-gdb \
>                  --disable-gdbserver \
>                  --disable-libdecnumber \

FWIW I 100% agree we should look to enable this. 

Sadly, doing so exposes a bug where things that should rebuild don't.
That manifests as a failure in the reproducible test builds on the
autobuilder. I suspect a taskhash problem somewhere, maybe hash
equivalence, maybe somewhere else, hard to say without debugging it.

I've been trying to get to this to help. The first issue was to sort
the other gdb issue this appeared to trigger but was in fact unrelated
and an issue from the recent binutils/gdb version upgrade. I've
debugged the initial buildtools tarball bug and re-deployed buildtools
on the infrastructure. That fixed some of the manifestations but not
all, I then debugged the remaining ones, worked out the regression in
upstream gdb and sent a patch yesterday which was merged upstream to
fix it.

I am trying to get to helping with the problem this patch causes but
these things take a ton of time. Whilst I may get paid to work on the
project, I am one person and I'm getting pulled in a ridiculous number
of directions at once.

People should worry this triggers a reproducibility issue, it means
there is a bug somewhere.

I am hoping to get to this and to help try and debug it but the patch
cannot merge until we get to the bottom of the issue it triggers.

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Richard Purdie]

On Wed, 2023-03-08 at 09:33 +0000, Richard Purdie via
lists.openembedded.org wrote:
> On Wed, 2023-02-22 at 22:58 -0800, Khem Raj wrote:
> > Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> > DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> > DT_RUNPATH, this order ensures that injecting
> > malicious shared objects is way harder with DT_RUNPATH.
> > 
> > This is now default on major linux distributions already
> > 
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/binutils/binutils.inc | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > index b2dbf241df..c69d29448f 100644
> > --- a/meta/recipes-devtools/binutils/binutils.inc
> > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
> >                  --disable-werror \
> >                  --enable-deterministic-archives \
> >                  --enable-plugins \
> > +                --enable-new-dtags \
> >                  --disable-gdb \
> >                  --disable-gdbserver \
> >                  --disable-libdecnumber \
> 
> FWIW I 100% agree we should look to enable this. 
> 
> Sadly, doing so exposes a bug where things that should rebuild don't.
> That manifests as a failure in the reproducible test builds on the
> autobuilder. I suspect a taskhash problem somewhere, maybe hash
> equivalence, maybe somewhere else, hard to say without debugging it.
> 
> I've been trying to get to this to help. The first issue was to sort
> the other gdb issue this appeared to trigger but was in fact unrelated
> and an issue from the recent binutils/gdb version upgrade. I've
> debugged the initial buildtools tarball bug and re-deployed buildtools
> on the infrastructure. That fixed some of the manifestations but not
> all, I then debugged the remaining ones, worked out the regression in
> upstream gdb and sent a patch yesterday which was merged upstream to
> fix it.
> 
> I am trying to get to helping with the problem this patch causes but
> these things take a ton of time. Whilst I may get paid to work on the
> project, I am one person and I'm getting pulled in a ridiculous number
> of directions at once.
> 
> People should worry this triggers a reproducibility issue, it means
> there is a bug somewhere.
> 
> I am hoping to get to this and to help try and debug it but the patch
> cannot merge until we get to the bottom of the issue it triggers.

I did have a look and the issue is actually fairly simple, glibc is
missing a dependency directly on binutils. Most recipes have this
magically, the magic is disabled for glibc as it is part of toolchain
bootstrap and has to be manually curated.

Adding the missing dependency appears to resolve things locally so I'll
send out a patch and we'll have to run some autobuilder tests.

Cheers,

Richard

Re: [OE-core] [PATCH] binutils: Enable --enable-new-dtags

[Khem Raj]

Awesome, thanks!  I think any recipe which is using
INHIBIT_DEFAULT_DEPENDENCIES could
run into these sorts of errors.

On Wed, Mar 8, 2023 at 3:53 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2023-03-08 at 09:33 +0000, Richard Purdie via
> lists.openembedded.org wrote:
> > On Wed, 2023-02-22 at 22:58 -0800, Khem Raj wrote:
> > > Use DT_RUNPATH over DT_RPATH. If DT_RUNPATH is present, LD_LIBRARY_PATH is searched before
> > > DT_RUNPATH, Search order is DT_RPATH then LD_LIBRARY_PATH then
> > > DT_RUNPATH, this order ensures that injecting
> > > malicious shared objects is way harder with DT_RUNPATH.
> > >
> > > This is now default on major linux distributions already
> > >
> > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > ---
> > >  meta/recipes-devtools/binutils/binutils.inc | 1 +
> > >  1 file changed, 1 insertion(+)
> > >
> > > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > > index b2dbf241df..c69d29448f 100644
> > > --- a/meta/recipes-devtools/binutils/binutils.inc
> > > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > > @@ -96,6 +96,7 @@ EXTRA_OECONF = "--program-prefix=${TARGET_PREFIX} \
> > >                  --disable-werror \
> > >                  --enable-deterministic-archives \
> > >                  --enable-plugins \
> > > +                --enable-new-dtags \
> > >                  --disable-gdb \
> > >                  --disable-gdbserver \
> > >                  --disable-libdecnumber \
> >
> > FWIW I 100% agree we should look to enable this.
> >
> > Sadly, doing so exposes a bug where things that should rebuild don't.
> > That manifests as a failure in the reproducible test builds on the
> > autobuilder. I suspect a taskhash problem somewhere, maybe hash
> > equivalence, maybe somewhere else, hard to say without debugging it.
> >
> > I've been trying to get to this to help. The first issue was to sort
> > the other gdb issue this appeared to trigger but was in fact unrelated
> > and an issue from the recent binutils/gdb version upgrade. I've
> > debugged the initial buildtools tarball bug and re-deployed buildtools
> > on the infrastructure. That fixed some of the manifestations but not
> > all, I then debugged the remaining ones, worked out the regression in
> > upstream gdb and sent a patch yesterday which was merged upstream to
> > fix it.
> >
> > I am trying to get to helping with the problem this patch causes but
> > these things take a ton of time. Whilst I may get paid to work on the
> > project, I am one person and I'm getting pulled in a ridiculous number
> > of directions at once.
> >
> > People should worry this triggers a reproducibility issue, it means
> > there is a bug somewhere.
> >
> > I am hoping to get to this and to help try and debug it but the patch
> > cannot merge until we get to the bottom of the issue it triggers.
>
> I did have a look and the issue is actually fairly simple, glibc is
> missing a dependency directly on binutils. Most recipes have this
> magically, the magic is disabled for glibc as it is part of toolchain
> bootstrap and has to be manually curated.
>
> Adding the missing dependency appears to resolve things locally so I'll
> send out a patch and we'll have to run some autobuilder tests.
>
> Cheers,
>
> Richard