prelink support

prelink support

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

It looks like the image-prelink support is still disabled in master because
of an issue with IFUNC symbol relocation.

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf/local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed

Is there still a problem, or is it safe to reenable image-prelink?  I see a
"Fix ARM IFUNC support" in the prelink-cross repo, but that appears to have
been committed several months before image-prelink was disabled.  We'd like
to enable image-prelink on a build off jethro.

Any help or links to a recent discussion thread would be helpful.  Thanks!

[-- Attachment #2: Type: text/html, Size: 894 bytes --]

Re: prelink support

[Richard Purdie]

On Mon, 2016-06-20 at 08:24 -0400, Kyle Russell wrote:
> It looks like the image-prelink support is still disabled in master
> because of an issue with IFUNC symbol relocation.
> 
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf
> /local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed
> 
> Is there still a problem, or is it safe to reenable image-prelink?  I
> see a "Fix ARM IFUNC support" in the prelink-cross repo, but that
> appears to have been committed several months before image-prelink
> was disabled.  We'd like to enable image-prelink on a build off
> jethro.
> 
> Any help or links to a recent discussion thread would be helpful. 
> Thanks!

Basically, its still broken. I actually ran into a bug the other day
caused by the issue where I had prelink enabled locally.

I know Mark is working on some fixes but its an onion problem, you fix
one thing, then find other issues. Mark can give the full story but
right now, its not able to be re-enabled.

Cheers,

Richard

Re: prelink support

[Mark Hatle]

On 6/20/16 7:24 AM, Kyle Russell wrote:
> It looks like the image-prelink support is still disabled in master because of
> an issue with IFUNC symbol relocation.
> 
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf/local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed
> 
> Is there still a problem, or is it safe to reenable image-prelink?  I see a "Fix
> ARM IFUNC support" in the prelink-cross repo, but that appears to have been
> committed several months before image-prelink was disabled.  We'd like to enable
> image-prelink on a build off jethro.
> 
> Any help or links to a recent discussion thread would be helpful.  Thanks!
> 
> 

The ARM fix is for a different IFUNC problem.

Disabling the prelinker was caused by a serious of problems that started with a
fork failure traced back to shared library processing orders.

(For comments below, I'm referring to the repository:
http://git.yoctoproject.org/cgit/cgit.cgi/prelink-cross/)

The initial problem, IFUNC processing, showed that processing order of the
shared libraries could lead to a case where the wrong IFUNC was used.  We
believe this particular issues was fixed in the staging tree.

In the tree you'll see a cross_prelink_staging branch.  In that branch the fix
is commit ID: "8f55afd84b3580b1f1d6af904e8c2a39221055b7"  This fixes the 'fork'
issue by ensuring the proper sort order for shared library processing.  (This
was finally fixed in March.)

However with that we determined there was at least one more issue related to
section ordering.  The prelink test suite was failing due to various integrity
checks showing that once we prelinked something, we could not reverse the
process.  It has taken us months to identify the cause of the problem and the
solution.  (Cause BTW was a change in binutils-2.22 that no longer ensured that
the section order was sorted by offset order... a small amount of the prelink
processing needed to be changed to deal with that behavior.  It's taken far
longer to fix then we had ever expected.)

See commit (33be255d62af533189f1f7bc66c06602b703980a) in the
cross_prelink_staging branch.

With this commit, I think the two major issues have been resolved.  I've got a
small set of additional patches I need to put into place -- and then I have to
finish going through the regression suite to verify everything is working
properly.  (Seems like every time I get to this step, something else comes up
and I'm not getting it done...)

So if you've read this far, the point in all of this is that I THINK that the
cross_prelink_staging branch and current top commit
"33be255d62af533189f1f7bc66c06602b703980a" are working.  However, I've not
completed testing so I'm not sure of that yet.

If you can help with testing, you should modify your prelink recipe to use the
'cross_prelink_staging' branch, and the 33be255 commit.  Verify that this is
working for your use cases.  If you are using glib or graphical environments pay
particular attention to process startup messages that indicate failures.

If this IS working for you, I'd like a reply back with the architecture and
processor you are using so I can add that to my matrix.  (I'm traveling for the
next couple of days, and then should be back in the officer where I hope to
finish my regression testing, apply the last couple of patches from Gentoo
developers, regression test again and prepare a release version.)

Any help is appreciated, thanks for understanding why this is taking so long.

--Mark

Re: prelink support

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 3950 bytes --]

Thanks, Mark and Richard.  Great update!  I'll try out the staging branch
and report back with results.

On Mon, Jun 20, 2016 at 9:15 AM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> On 6/20/16 7:24 AM, Kyle Russell wrote:
> > It looks like the image-prelink support is still disabled in master
> because of
> > an issue with IFUNC symbol relocation.
> >
> >
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf/local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed
> >
> > Is there still a problem, or is it safe to reenable image-prelink?  I
> see a "Fix
> > ARM IFUNC support" in the prelink-cross repo, but that appears to have
> been
> > committed several months before image-prelink was disabled.  We'd like
> to enable
> > image-prelink on a build off jethro.
> >
> > Any help or links to a recent discussion thread would be helpful.
> Thanks!
> >
> >
>
> The ARM fix is for a different IFUNC problem.
>
> Disabling the prelinker was caused by a serious of problems that started
> with a
> fork failure traced back to shared library processing orders.
>
> (For comments below, I'm referring to the repository:
> http://git.yoctoproject.org/cgit/cgit.cgi/prelink-cross/)
>
> The initial problem, IFUNC processing, showed that processing order of the
> shared libraries could lead to a case where the wrong IFUNC was used.  We
> believe this particular issues was fixed in the staging tree.
>
> In the tree you'll see a cross_prelink_staging branch.  In that branch the
> fix
> is commit ID: "8f55afd84b3580b1f1d6af904e8c2a39221055b7"  This fixes the
> 'fork'
> issue by ensuring the proper sort order for shared library processing.
> (This
> was finally fixed in March.)
>
> However with that we determined there was at least one more issue related
> to
> section ordering.  The prelink test suite was failing due to various
> integrity
> checks showing that once we prelinked something, we could not reverse the
> process.  It has taken us months to identify the cause of the problem and
> the
> solution.  (Cause BTW was a change in binutils-2.22 that no longer ensured
> that
> the section order was sorted by offset order... a small amount of the
> prelink
> processing needed to be changed to deal with that behavior.  It's taken far
> longer to fix then we had ever expected.)
>
> See commit (33be255d62af533189f1f7bc66c06602b703980a) in the
> cross_prelink_staging branch.
>
> With this commit, I think the two major issues have been resolved.  I've
> got a
> small set of additional patches I need to put into place -- and then I
> have to
> finish going through the regression suite to verify everything is working
> properly.  (Seems like every time I get to this step, something else comes
> up
> and I'm not getting it done...)
>
> So if you've read this far, the point in all of this is that I THINK that
> the
> cross_prelink_staging branch and current top commit
> "33be255d62af533189f1f7bc66c06602b703980a" are working.  However, I've not
> completed testing so I'm not sure of that yet.
>
> If you can help with testing, you should modify your prelink recipe to use
> the
> 'cross_prelink_staging' branch, and the 33be255 commit.  Verify that this
> is
> working for your use cases.  If you are using glib or graphical
> environments pay
> particular attention to process startup messages that indicate failures.
>
> If this IS working for you, I'd like a reply back with the architecture and
> processor you are using so I can add that to my matrix.  (I'm traveling
> for the
> next couple of days, and then should be back in the officer where I hope to
> finish my regression testing, apply the last couple of patches from Gentoo
> developers, regression test again and prepare a release version.)
>
> Any help is appreciated, thanks for understanding why this is taking so
> long.
>
> --Mark
>

[-- Attachment #2: Type: text/html, Size: 4861 bytes --]

Re: prelink support

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 4132 bytes --]

The cross_prelink_staging branch seems to work for our ARM Cortex-A53
configuration.

The only issue that I ran into was building src/rtld/dl-open.c.  I guess
the GCCVERSION we declared (4.9%) was defaulting to a language standard
that didn't support for loop initial declarations.

On Mon, Jun 20, 2016 at 9:15 AM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> On 6/20/16 7:24 AM, Kyle Russell wrote:
> > It looks like the image-prelink support is still disabled in master
> because of
> > an issue with IFUNC symbol relocation.
> >
> >
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf/local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed
> >
> > Is there still a problem, or is it safe to reenable image-prelink?  I
> see a "Fix
> > ARM IFUNC support" in the prelink-cross repo, but that appears to have
> been
> > committed several months before image-prelink was disabled.  We'd like
> to enable
> > image-prelink on a build off jethro.
> >
> > Any help or links to a recent discussion thread would be helpful.
> Thanks!
> >
> >
>
> The ARM fix is for a different IFUNC problem.
>
> Disabling the prelinker was caused by a serious of problems that started
> with a
> fork failure traced back to shared library processing orders.
>
> (For comments below, I'm referring to the repository:
> http://git.yoctoproject.org/cgit/cgit.cgi/prelink-cross/)
>
> The initial problem, IFUNC processing, showed that processing order of the
> shared libraries could lead to a case where the wrong IFUNC was used.  We
> believe this particular issues was fixed in the staging tree.
>
> In the tree you'll see a cross_prelink_staging branch.  In that branch the
> fix
> is commit ID: "8f55afd84b3580b1f1d6af904e8c2a39221055b7"  This fixes the
> 'fork'
> issue by ensuring the proper sort order for shared library processing.
> (This
> was finally fixed in March.)
>
> However with that we determined there was at least one more issue related
> to
> section ordering.  The prelink test suite was failing due to various
> integrity
> checks showing that once we prelinked something, we could not reverse the
> process.  It has taken us months to identify the cause of the problem and
> the
> solution.  (Cause BTW was a change in binutils-2.22 that no longer ensured
> that
> the section order was sorted by offset order... a small amount of the
> prelink
> processing needed to be changed to deal with that behavior.  It's taken far
> longer to fix then we had ever expected.)
>
> See commit (33be255d62af533189f1f7bc66c06602b703980a) in the
> cross_prelink_staging branch.
>
> With this commit, I think the two major issues have been resolved.  I've
> got a
> small set of additional patches I need to put into place -- and then I
> have to
> finish going through the regression suite to verify everything is working
> properly.  (Seems like every time I get to this step, something else comes
> up
> and I'm not getting it done...)
>
> So if you've read this far, the point in all of this is that I THINK that
> the
> cross_prelink_staging branch and current top commit
> "33be255d62af533189f1f7bc66c06602b703980a" are working.  However, I've not
> completed testing so I'm not sure of that yet.
>
> If you can help with testing, you should modify your prelink recipe to use
> the
> 'cross_prelink_staging' branch, and the 33be255 commit.  Verify that this
> is
> working for your use cases.  If you are using glib or graphical
> environments pay
> particular attention to process startup messages that indicate failures.
>
> If this IS working for you, I'd like a reply back with the architecture and
> processor you are using so I can add that to my matrix.  (I'm traveling
> for the
> next couple of days, and then should be back in the officer where I hope to
> finish my regression testing, apply the last couple of patches from Gentoo
> developers, regression test again and prepare a release version.)
>
> Any help is appreciated, thanks for understanding why this is taking so
> long.
>
> --Mark
>

[-- Attachment #2: Type: text/html, Size: 5062 bytes --]

Re: prelink support

[Mark Hatle]

On 6/22/16 3:06 AM, Kyle Russell wrote:
> The cross_prelink_staging branch seems to work for our ARM Cortex-A53 configuration.
> 
> The only issue that I ran into was building src/rtld/dl-open.c.  I guess the
> GCCVERSION we declared (4.9%) was defaulting to a language standard that didn't
> support for loop initial declarations.

What is the specific error you got?  This is the type of stuff I'd like to resolve.

Are you using a branch and gcc that is part of OpenEmbedded?  If so, I need to
know how to configure the system so I can attempt to reproduce and work around
the issue.

--Mark

> On Mon, Jun 20, 2016 at 9:15 AM, Mark Hatle <mark.hatle@windriver.com
> <mailto:mark.hatle@windriver.com>> wrote:
> 
>     On 6/20/16 7:24 AM, Kyle Russell wrote:
>     > It looks like the image-prelink support is still disabled in master because of
>     > an issue with IFUNC symbol relocation.
>     >
>     >
>     https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/meta-poky/conf/local.conf.sample?id=8debfea81e69d038bd2d56314b272cb74f5582ed
>     >
>     > Is there still a problem, or is it safe to reenable image-prelink?  I see
>     a "Fix
>     > ARM IFUNC support" in the prelink-cross repo, but that appears to have been
>     > committed several months before image-prelink was disabled.  We'd like to
>     enable
>     > image-prelink on a build off jethro.
>     >
>     > Any help or links to a recent discussion thread would be helpful.  Thanks!
>     >
>     >
> 
>     The ARM fix is for a different IFUNC problem.
> 
>     Disabling the prelinker was caused by a serious of problems that started with a
>     fork failure traced back to shared library processing orders.
> 
>     (For comments below, I'm referring to the repository:
>     http://git.yoctoproject.org/cgit/cgit.cgi/prelink-cross/)
> 
>     The initial problem, IFUNC processing, showed that processing order of the
>     shared libraries could lead to a case where the wrong IFUNC was used.  We
>     believe this particular issues was fixed in the staging tree.
> 
>     In the tree you'll see a cross_prelink_staging branch.  In that branch the fix
>     is commit ID: "8f55afd84b3580b1f1d6af904e8c2a39221055b7"  This fixes the 'fork'
>     issue by ensuring the proper sort order for shared library processing.  (This
>     was finally fixed in March.)
> 
>     However with that we determined there was at least one more issue related to
>     section ordering.  The prelink test suite was failing due to various integrity
>     checks showing that once we prelinked something, we could not reverse the
>     process.  It has taken us months to identify the cause of the problem and the
>     solution.  (Cause BTW was a change in binutils-2.22 that no longer ensured that
>     the section order was sorted by offset order... a small amount of the prelink
>     processing needed to be changed to deal with that behavior.  It's taken far
>     longer to fix then we had ever expected.)
> 
>     See commit (33be255d62af533189f1f7bc66c06602b703980a) in the
>     cross_prelink_staging branch.
> 
>     With this commit, I think the two major issues have been resolved.  I've got a
>     small set of additional patches I need to put into place -- and then I have to
>     finish going through the regression suite to verify everything is working
>     properly.  (Seems like every time I get to this step, something else comes up
>     and I'm not getting it done...)
> 
>     So if you've read this far, the point in all of this is that I THINK that the
>     cross_prelink_staging branch and current top commit
>     "33be255d62af533189f1f7bc66c06602b703980a" are working.  However, I've not
>     completed testing so I'm not sure of that yet.
> 
>     If you can help with testing, you should modify your prelink recipe to use the
>     'cross_prelink_staging' branch, and the 33be255 commit.  Verify that this is
>     working for your use cases.  If you are using glib or graphical environments pay
>     particular attention to process startup messages that indicate failures.
> 
>     If this IS working for you, I'd like a reply back with the architecture and
>     processor you are using so I can add that to my matrix.  (I'm traveling for the
>     next couple of days, and then should be back in the officer where I hope to
>     finish my regression testing, apply the last couple of patches from Gentoo
>     developers, regression test again and prepare a release version.)
> 
>     Any help is appreciated, thanks for understanding why this is taking so long.
> 
>     --Mark
> 
> 

Re: prelink support

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On Wed, Jun 22, 2016 at 10:57 AM, Mark Hatle <mark.hatle@windriver.com>
wrote:

> What is the specific error you got?  This is the type of stuff I'd like to
> resolve.
>

I don't have the logs on me at the moment for the specific output, but
there were two errors (one for 'int scope_cnt' and the other for 'unsigned
int cnt') being declared in the for loop initialization, which isn't
supported until -std=c99.  I was able to move the declaration to the
beginning of _dl_show_scope just for testing and prelink finished compiling
successfully.

Are you using a branch and gcc that is part of OpenEmbedded?  If so, I need
> to
> know how to configure the system so I can attempt to reproduce and work
> around
> the issue.
>

I think we're using the gcc compiler that's included in poky.  I was
running poky at the following commit on jethro.

ddbc131 toasterconf.json: exclude releases Toaster can't build

The only configuration change that I'm aware of was setting GCCVERSION back
to "4.9%" as recommended in
http://www.yoctoproject.org/docs/2.1/mega-manual/mega-manual.html#migration-2.0-gcc-5

I had updated the SRCREV and branch in the prelink recipe to point to
cross_prelink_staging as you suggested.  It simply seems that the version
of dl-open.c on cross_prelink_staging just isn't compatible with gcc 4.9,
which seems to default to C90.

I can dig further into our setup and send specific steps tomorrow if you're
not able to get similar results.

[-- Attachment #2: Type: text/html, Size: 2708 bytes --]