[PATCH 0/2] prelink changes

[PATCH 0/2] prelink changes

[Mark Hatle]

Re-enable prelink on ARM and PPC.  The detected failures are not related to
anything inside of prelink, the binaries failed to execute before prelinking.

The other patch fixes the IFUNC resolution problem inside of glibc itself.
The scope was being resolved depth first, not breadth first as indicated in
the ELF specification.

The following changes since commit d3e5c1d17f80b3dd7a6d85e07b5c0c516f09ae93:

  local.conf.sample: Disable ARM and PPC due to prelink test case failures (2016-08-19 10:23:31 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib mhatle/prelink
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=mhatle/prelink

Mark Hatle (2):
  glibc: Fix scope resolution in glibc to be breadth first.
  Revert "local.conf.sample: Disable ARM and PPC due to prelink test
    case failures"

 meta/conf/local.conf.sample                        |  9 ----
 .../glibc/glibc/0026-build_local_scope.patch       | 56 ++++++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.24.bb              |  1 +
 3 files changed, 57 insertions(+), 9 deletions(-)
 create mode 100644 meta/recipes-core/glibc/glibc/0026-build_local_scope.patch

-- 
2.5.5

[PATCH 1/2] glibc: Fix scope resolution in glibc to be breadth first.

[Mark Hatle]

The ELF specification indicates symbol resolution should be breadth first, not
depth first.

The dl-deps.c: dl_build_locale_scope function is processing in a depth first
mode.  This is causes certain symbols to be incorrectly reported when
LD_TRACE_PRELINKING=1 is enabled.

See glibc BZ #20488 for more information.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 .../glibc/glibc/0026-build_local_scope.patch       | 56 ++++++++++++++++++++++
 meta/recipes-core/glibc/glibc_2.24.bb              |  1 +
 2 files changed, 57 insertions(+)
 create mode 100644 meta/recipes-core/glibc/glibc/0026-build_local_scope.patch

diff --git a/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch b/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch
new file mode 100644
index 0000000..435a8c8
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch
@@ -0,0 +1,56 @@
+From 6e4ec5a3c5fe63b6458036f18d43124de4a7e724 Mon Sep 17 00:00:00 2001
+From: Mark Hatle <mark.hatle@windriver.com>
+Date: Thu, 18 Aug 2016 14:07:58 -0500
+Subject: [PATCH] elf/dl-deps.c: Make _dl_build_local_scope breadth first
+
+According to the ELF specification:
+
+When resolving symbolic references, the dynamic linker examines the symbol
+tables with a breadth-first search.
+
+This function was using a depth first search.  By doing so the conflict
+resolution reported to the prelinker (when LD_TRACE_PRELINKING=1 is set)
+was incorrect.  This caused problems when their were various circular
+dependencies between libraries.  The problem usually manifested itself by
+the wrong IFUNC being executed.
+
+[BZ# 20488]
+
+Upstream-Status: Pending
+
+Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
+---
+ elf/dl-deps.c | 14 ++++++++++----
+ 1 file changed, 10 insertions(+), 4 deletions(-)
+
+diff --git a/elf/dl-deps.c b/elf/dl-deps.c
+index 6a82987..fc37c87 100644
+--- a/elf/dl-deps.c
++++ b/elf/dl-deps.c
+@@ -73,13 +73,19 @@ _dl_build_local_scope (struct link_map **list, struct link_map *map)
+ {
+   struct link_map **p = list;
+   struct link_map **q;
++  struct link_map **r;
+ 
+   *p++ = map;
+   map->l_reserved = 1;
+-  if (map->l_initfini)
+-    for (q = map->l_initfini + 1; *q; ++q)
+-      if (! (*q)->l_reserved)
+-	p += _dl_build_local_scope (p, *q);
++
++  for (r = list; r < p; ++r)
++    if ((*r)->l_initfini)
++      for (q = (*r)->l_initfini + 1; *q; ++q)
++	if (! (*q)->l_reserved)
++	  {
++	    *p++ = *q;
++	    (*q)->l_reserved = 1;
++	  }
+   return p - list;
+ }
+ 
+-- 
+2.5.5
+
diff --git a/meta/recipes-core/glibc/glibc_2.24.bb b/meta/recipes-core/glibc/glibc_2.24.bb
index e446e7a..a3ac738 100644
--- a/meta/recipes-core/glibc/glibc_2.24.bb
+++ b/meta/recipes-core/glibc/glibc_2.24.bb
@@ -36,6 +36,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
            file://0023-eglibc-Install-PIC-archives.patch \
            file://0024-eglibc-Forward-port-cross-locale-generation-support.patch \
            file://0025-Define-DUMMY_LOCALE_T-if-not-defined.patch \
+           file://0026-build_local_scope.patch \
 "
 
 SRC_URI += "\
-- 
2.5.5

Re: [PATCH 1/2] glibc: Fix scope resolution in glibc to be breadth first.

[Mark Hatle]

On 8/19/16 8:44 PM, Mark Hatle wrote:
> The ELF specification indicates symbol resolution should be breadth first, not
> depth first.
> 
> The dl-deps.c: dl_build_locale_scope function is processing in a depth first
> mode.  This is causes certain symbols to be incorrectly reported when
> LD_TRACE_PRELINKING=1 is enabled.
> 
> See glibc BZ #20488 for more information.
> 
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
>  .../glibc/glibc/0026-build_local_scope.patch       | 56 ++++++++++++++++++++++
>  meta/recipes-core/glibc/glibc_2.24.bb              |  1 +
>  2 files changed, 57 insertions(+)
>  create mode 100644 meta/recipes-core/glibc/glibc/0026-build_local_scope.patch
> 
> diff --git a/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch b/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch
> new file mode 100644
> index 0000000..435a8c8
> --- /dev/null
> +++ b/meta/recipes-core/glibc/glibc/0026-build_local_scope.patch
> @@ -0,0 +1,56 @@
> +From 6e4ec5a3c5fe63b6458036f18d43124de4a7e724 Mon Sep 17 00:00:00 2001
> +From: Mark Hatle <mark.hatle@windriver.com>
> +Date: Thu, 18 Aug 2016 14:07:58 -0500
> +Subject: [PATCH] elf/dl-deps.c: Make _dl_build_local_scope breadth first
> +
> +According to the ELF specification:
> +
> +When resolving symbolic references, the dynamic linker examines the symbol
> +tables with a breadth-first search.
> +
> +This function was using a depth first search.  By doing so the conflict
> +resolution reported to the prelinker (when LD_TRACE_PRELINKING=1 is set)
> +was incorrect.  This caused problems when their were various circular
> +dependencies between libraries.  The problem usually manifested itself by
> +the wrong IFUNC being executed.
> +
> +[BZ# 20488]
> +
> +Upstream-Status: Pending

Above can be changes to:

Upstream-Status: Submitted [libc-alpha]

> +
> +Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> +---
> + elf/dl-deps.c | 14 ++++++++++----
> + 1 file changed, 10 insertions(+), 4 deletions(-)
> +
> +diff --git a/elf/dl-deps.c b/elf/dl-deps.c
> +index 6a82987..fc37c87 100644
> +--- a/elf/dl-deps.c
> ++++ b/elf/dl-deps.c
> +@@ -73,13 +73,19 @@ _dl_build_local_scope (struct link_map **list, struct link_map *map)
> + {
> +   struct link_map **p = list;
> +   struct link_map **q;
> ++  struct link_map **r;
> + 
> +   *p++ = map;
> +   map->l_reserved = 1;
> +-  if (map->l_initfini)
> +-    for (q = map->l_initfini + 1; *q; ++q)
> +-      if (! (*q)->l_reserved)
> +-	p += _dl_build_local_scope (p, *q);
> ++
> ++  for (r = list; r < p; ++r)
> ++    if ((*r)->l_initfini)
> ++      for (q = (*r)->l_initfini + 1; *q; ++q)
> ++	if (! (*q)->l_reserved)
> ++	  {
> ++	    *p++ = *q;
> ++	    (*q)->l_reserved = 1;
> ++	  }
> +   return p - list;
> + }
> + 
> +-- 
> +2.5.5
> +
> diff --git a/meta/recipes-core/glibc/glibc_2.24.bb b/meta/recipes-core/glibc/glibc_2.24.bb
> index e446e7a..a3ac738 100644
> --- a/meta/recipes-core/glibc/glibc_2.24.bb
> +++ b/meta/recipes-core/glibc/glibc_2.24.bb
> @@ -36,6 +36,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
>             file://0023-eglibc-Install-PIC-archives.patch \
>             file://0024-eglibc-Forward-port-cross-locale-generation-support.patch \
>             file://0025-Define-DUMMY_LOCALE_T-if-not-defined.patch \
> +           file://0026-build_local_scope.patch \
>  "
>  
>  SRC_URI += "\
> 

[PATCH 2/2] Revert "local.conf.sample: Disable ARM and PPC due to prelink test case failures"

[Mark Hatle]

This reverts commit d3e5c1d17f80b3dd7a6d85e07b5c0c516f09ae93.

The problem is not in prelink, but in binutils.  Any programs compiled with
the option will be broken and prelink will not make it worse.

See Yocto Project bug 10168 for more information.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta/conf/local.conf.sample | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
index b35db51..85c5e21 100644
--- a/meta/conf/local.conf.sample
+++ b/meta/conf/local.conf.sample
@@ -131,15 +131,6 @@ EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
 # NOTE: mklibs also needs to be explicitly enabled for a given image, see local.conf.extended
 USER_CLASSES ?= "buildstats image-mklibs image-prelink"
 
-# Prelinker tests are currently failing on ARM and PPC.  Prevent us from
-# running on those architectures until the failure is explained.
-USER_CLASSES_remove_arm = 'image-prelink'
-USER_CLASSES_remove_armeb = 'image-prelink'
-USER_CLASSES_remove_aarch64 = 'image-prelink'
-USER_CLASSES_remove_aarch64eb = 'image-prelink'
-USER_CLASSES_remove_powerpc = 'image-prelink'
-USER_CLASSES_remove_powerpc64 = 'image-prelink'
-
 
 #
 # Runtime testing of images
-- 
2.5.5