* [PATCH][RFC] chrpath.bbclass: strip common parent directories from rpath
@ 2013-03-05 23:55 Andreas Oberritter
2013-03-06 0:32 ` [PATCH v2][RFC] " Andreas Oberritter
0 siblings, 1 reply; 10+ messages in thread
From: Andreas Oberritter @ 2013-03-05 23:55 UTC (permalink / raw)
To: openembedded-core
Allows to use shorter TMPDIRs in corner cases, e.g. with native
perl modules, having a deep directory structure.
The problem is that the original absolute rpath may be shorter
than the newly generated relative rpath.
The failing rpaths were '/ab/cde/tmp/sysroots/x86_64-linux/usr/lib'
(old) and '$ORIGIN/../../../../../../../../../usr/lib' (new) for
'/ab/cde/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.14.3/auto/XML/Parser/Expat/Expat.so'
The new rpath is '$ORIGIN/../../../../../../..'.
The new code doesn't just compare libdir, because I guess it
should also work if only parts of libdir share a parent directory
with the binary.
This patch also adds a check for len(basedir) > 0, because I think
basedir is empty in the cross-compile case, in which case
rpath.find(basedir) would always return 0, but I'm unsure whether
I understood that part of the code correctly or not.
[YOCTO #3989]
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
---
meta/classes/chrpath.bbclass | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 0c7ab77..ed4a0b2 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -55,9 +55,22 @@ def process_dir (directory, d):
rpath = os.path.normpath(rpath)
# If the rpath shares a root with base_prefix determine a new dynamic rpath from the
# base_prefix shared root
- if rpath.find(basedir) != -1:
- depth = fpath.partition(basedir)[2].count('/')
+ if len(basedir) > 0 and rpath.find(basedir) != -1:
+ relfpath = fpath.partition(basedir)[2].strip()
libpath = rpath.partition(basedir)[2].strip()
+ depth = relfpath.count('/')
+ # Compare common parent directories to reduce the length of the new rpath.
+ # Ignore the first (empty) element.
+ libdirs = libpath.split('/')
+ relfdirs = relfpath.split('/')
+ nparents = 0
+ for i in range(1, min(len(libdirs), len(relfdirs))):
+ if libdirs[i] != relfdirs[i]:
+ break
+ nparents += 1
+ if nparents > 0:
+ libpath = '/'.join(libdirs[:-nparents])
+ depth -= nparents
# otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
# NOTE: This will not work reliably for cross packages, particularly in the case
# where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
--
1.7.10.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath
2013-03-05 23:55 [PATCH][RFC] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
@ 2013-03-06 0:32 ` Andreas Oberritter
2013-03-06 7:19 ` [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath# Henning Heinold
2013-07-02 9:50 ` [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
0 siblings, 2 replies; 10+ messages in thread
From: Andreas Oberritter @ 2013-03-06 0:32 UTC (permalink / raw)
To: openembedded-core
Allows to use shorter TMPDIRs in corner cases, e.g. with native
perl modules, having a deep directory structure.
The problem is that the original absolute rpath may be shorter
than the newly generated relative rpath.
The failing rpaths were '/ab/cde/tmp/sysroots/x86_64-linux/usr/lib'
(old) and '$ORIGIN/../../../../../../../../../usr/lib' (new) for
'/ab/cde/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.14.3/auto/XML/Parser/Expat/Expat.so'
The new rpath is '$ORIGIN/../../../../../../..'.
The new code doesn't just compare libdir, because I guess it
should also work if only parts of libdir share a parent directory
with the binary.
This patch also adds a check for len(basedir) > 0, because I think
basedir is empty in the cross-compile case, in which case
rpath.find(basedir) would always return 0, but I'm unsure whether
I understood that part of the code correctly or not.
[YOCTO #3989]
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
---
v2: Fixed reassembly of libpath
meta/classes/chrpath.bbclass | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 0c7ab77..454b587 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -55,9 +55,22 @@ def process_dir (directory, d):
rpath = os.path.normpath(rpath)
# If the rpath shares a root with base_prefix determine a new dynamic rpath from the
# base_prefix shared root
- if rpath.find(basedir) != -1:
- depth = fpath.partition(basedir)[2].count('/')
+ if len(basedir) > 0 and rpath.find(basedir) != -1:
+ relfpath = fpath.partition(basedir)[2].strip()
libpath = rpath.partition(basedir)[2].strip()
+ depth = relfpath.count('/')
+ # Compare common parent directories to reduce the length of the new rpath.
+ # Ignore the first (empty) element.
+ libdirs = libpath.split('/')
+ relfdirs = relfpath.split('/')
+ nparents = 0
+ for i in range(1, min(len(libdirs), len(relfdirs))):
+ if libdirs[i] != relfdirs[i]:
+ break
+ nparents += 1
+ if nparents > 0:
+ libpath = '/' + '/'.join(libdirs[i:])
+ depth -= nparents
# otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
# NOTE: This will not work reliably for cross packages, particularly in the case
# where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
--
1.7.10.4
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath#
2013-03-06 0:32 ` [PATCH v2][RFC] " Andreas Oberritter
@ 2013-03-06 7:19 ` Henning Heinold
2013-07-02 9:50 ` [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
1 sibling, 0 replies; 10+ messages in thread
From: Henning Heinold @ 2013-03-06 7:19 UTC (permalink / raw)
To: openembedded-core
Hi,
I will try your patch on meta-java too, because later openjdk's vm had problem with
the rpath too, until khem made a patch.
Bye Henning
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath
@ 2013-07-01 23:31 Saul Wold
0 siblings, 0 replies; 10+ messages in thread
From: Saul Wold @ 2013-07-01 23:31 UTC (permalink / raw)
To: Andreas Oberritter
On 03/05/2013 04:32 PM, Andreas Oberritter wrote:
>
> Allows to use shorter TMPDIRs in corner cases, e.g. with native
> perl modules, having a deep directory structure.
>
> The problem is that the original absolute rpath may be shorter
> than the newly generated relative rpath.
>
> The failing rpaths were '/ab/cde/tmp/sysroots/x86_64-linux/usr/lib'
> (old) and '$ORIGIN/../../../../../../../../../usr/lib' (new) for
> '/ab/cde/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.14.3/auto/XML/Parser/Expat/Expat.so'
>
> The new rpath is '$ORIGIN/../../../../../../..'.
>
> The new code doesn't just compare libdir, because I guess it
> should also work if only parts of libdir share a parent directory
> with the binary.
>
> This patch also adds a check for len(basedir) > 0, because I think
> basedir is empty in the cross-compile case, in which case
> rpath.find(basedir) would always return 0, but I'm unsure whether
> I understood that part of the code correctly or not.
>
This seems to introduce a problem in slipt_and_strip_files
> ERROR: debugedit failed with exit code 127 (cmd was '/srv/ssd/sgw/builds/sec_flags/build/tmp/sysroots/x86_64-linux/usr/lib/rpm/bin/debugedit' -b '/srv/ssd/sgw/builds/sec_flags/build/tmp/work/i586-poky-linux' -d '/usr/src/debug' -i -l '/srv/ssd/sgw/builds/sec_flags/build/tmp/work/i586-poky-linux/eglibc/2.17-r3/debugsources.list' '/srv/ssd/sgw/builds/sec_flags/build/tmp/work/i586-poky-linux/eglibc/2.17-r3/package/usr/bin/locale')
> ERROR: Function failed: split_and_strip_files
> ERROR: Logfile of failure stored in: /srv/ssd/sgw/builds/sec_flags/build/tmp/work/i586-poky-linux/eglibc/2.17-r3/temp/log.do_package.26019
> ERROR: Task 27 (/srv/ssd/sgw/builds/sec_flags/meta/recipes-core/eglibc/eglibc_2.17.bb, do_package) failed with exit code '1'
That was seen with a clean build, no sstate.
Sau!
> [YOCTO #3989]
>
> Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
> ---
> v2: Fixed reassembly of libpath
>
> meta/classes/chrpath.bbclass | 17 +++++++++++++++--
> 1 file changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
> index 0c7ab77..454b587 100644
> --- a/meta/classes/chrpath.bbclass
> +++ b/meta/classes/chrpath.bbclass
> @@ -55,9 +55,22 @@ def process_dir (directory, d):
> rpath = os.path.normpath(rpath)
> # If the rpath shares a root with base_prefix determine a new dynamic rpath from the
> # base_prefix shared root
> - if rpath.find(basedir) != -1:
> - depth = fpath.partition(basedir)[2].count('/')
> + if len(basedir) > 0 and rpath.find(basedir) != -1:
> + relfpath = fpath.partition(basedir)[2].strip()
> libpath = rpath.partition(basedir)[2].strip()
> + depth = relfpath.count('/')
> + # Compare common parent directories to reduce the length of the new rpath.
> + # Ignore the first (empty) element.
> + libdirs = libpath.split('/')
> + relfdirs = relfpath.split('/')
> + nparents = 0
> + for i in range(1, min(len(libdirs), len(relfdirs))):
> + if libdirs[i] != relfdirs[i]:
> + break
> + nparents += 1
> + if nparents > 0:
> + libpath = '/' + '/'.join(libdirs[i:])
> + depth -= nparents
> # otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
> # NOTE: This will not work reliably for cross packages, particularly in the case
> # where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-03-06 0:32 ` [PATCH v2][RFC] " Andreas Oberritter
2013-03-06 7:19 ` [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath# Henning Heinold
@ 2013-07-02 9:50 ` Andreas Oberritter
2013-07-30 15:24 ` Phil Blundell
1 sibling, 1 reply; 10+ messages in thread
From: Andreas Oberritter @ 2013-07-02 9:50 UTC (permalink / raw)
To: openembedded-core
Allows to use shorter TMPDIRs in corner cases, e.g. with native
perl modules, having a deep directory structure.
The problem is that the original absolute rpath may be shorter
than the newly generated relative rpath.
The failing rpaths were '/ab/cde/tmp/sysroots/x86_64-linux/usr/lib'
(old) and '$ORIGIN/../../../../../../../../../usr/lib' (new) for
'/ab/cde/tmp/sysroots/x86_64-linux/usr/lib/perl-native/perl/5.14.3/auto/XML/Parser/Expat/Expat.so'
The new rpath is '$ORIGIN/../../../../../../..'.
The new code doesn't just compare libdir, because I guess it
should also work if only parts of libdir share a parent directory
with the binary.
This patch also adds a check for len(basedir) > 0, because I think
basedir is empty in the cross-compile case, in which case
rpath.find(basedir) would always return 0, but I'm unsure whether
I understood that part of the code correctly or not.
[YOCTO #3989]
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
---
v2: Fixed reassembly of libpath
v3: Really fixed reassembly of libpath:
libpath = '/' + '/'.join(libdirs[nparents + 1:])
I've been using this version of the patch since about four months ago
and didn't notice any problems.
meta/classes/chrpath.bbclass | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/meta/classes/chrpath.bbclass b/meta/classes/chrpath.bbclass
index 0c7ab77..9255aa7 100644
--- a/meta/classes/chrpath.bbclass
+++ b/meta/classes/chrpath.bbclass
@@ -55,9 +55,22 @@ def process_dir (directory, d):
rpath = os.path.normpath(rpath)
# If the rpath shares a root with base_prefix determine a new dynamic rpath from the
# base_prefix shared root
- if rpath.find(basedir) != -1:
- depth = fpath.partition(basedir)[2].count('/')
+ if len(basedir) > 0 and rpath.find(basedir) != -1:
+ relfpath = fpath.partition(basedir)[2].strip()
libpath = rpath.partition(basedir)[2].strip()
+ depth = relfpath.count('/')
+ # Compare common parent directories to reduce the length of the new rpath.
+ # Ignore the first (empty) element.
+ libdirs = libpath.split('/')
+ relfdirs = relfpath.split('/')
+ nparents = 0
+ for i in range(1, min(len(libdirs), len(relfdirs))):
+ if libdirs[i] != relfdirs[i]:
+ break
+ nparents += 1
+ if nparents > 0:
+ libpath = '/' + '/'.join(libdirs[nparents + 1:])
+ depth -= nparents
# otherwise (i.e. cross packages) determine a shared root based on the TMPDIR
# NOTE: This will not work reliably for cross packages, particularly in the case
# where your TMPDIR is a short path (i.e. /usr/poky) as chrpath cannot insert an
--
1.8.1.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-07-02 9:50 ` [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
@ 2013-07-30 15:24 ` Phil Blundell
2013-07-30 15:27 ` Khem Raj
0 siblings, 1 reply; 10+ messages in thread
From: Phil Blundell @ 2013-07-30 15:24 UTC (permalink / raw)
To: Andreas Oberritter; +Cc: openembedded-core
On Tue, 2013-07-02 at 11:50 +0200, Andreas Oberritter wrote:
> Allows to use shorter TMPDIRs in corner cases, e.g. with native
> perl modules, having a deep directory structure.
>
> The problem is that the original absolute rpath may be shorter
> than the newly generated relative rpath.
Can we not just fix chrpath to cope with the case where the new path is
longer than the old one? It seems a bit sad to require
ever-more-convoluted workarounds for what is essentially a tool problem.
p.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-07-30 15:24 ` Phil Blundell
@ 2013-07-30 15:27 ` Khem Raj
2013-07-30 15:34 ` Phil Blundell
0 siblings, 1 reply; 10+ messages in thread
From: Khem Raj @ 2013-07-30 15:27 UTC (permalink / raw)
To: Phil Blundell; +Cc: openembedded-core
On Jul 30, 2013, at 8:24 AM, Phil Blundell <pb@pbcl.net> wrote:
> On Tue, 2013-07-02 at 11:50 +0200, Andreas Oberritter wrote:
>> Allows to use shorter TMPDIRs in corner cases, e.g. with native
>> perl modules, having a deep directory structure.
>>
>> The problem is that the original absolute rpath may be shorter
>> than the newly generated relative rpath.
>
> Can we not just fix chrpath to cope with the case where the new path is
> longer than the old one? It seems a bit sad to require
> ever-more-convoluted workarounds for what is essentially a tool problem.
>
another option is to dump chrpath in favour of something which does it elegantly - patchelf
I had propsed patch set for that last year here
http://git.openembedded.org/openembedded-core-contrib/log/?h=kraj/patchelf
> p.
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-07-30 15:27 ` Khem Raj
@ 2013-07-30 15:34 ` Phil Blundell
2013-07-30 18:47 ` Andreas Oberritter
0 siblings, 1 reply; 10+ messages in thread
From: Phil Blundell @ 2013-07-30 15:34 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core
On Tue, 2013-07-30 at 08:27 -0700, Khem Raj wrote:
> On Jul 30, 2013, at 8:24 AM, Phil Blundell <pb@pbcl.net> wrote:
>
> > On Tue, 2013-07-02 at 11:50 +0200, Andreas Oberritter wrote:
> >> Allows to use shorter TMPDIRs in corner cases, e.g. with native
> >> perl modules, having a deep directory structure.
> >>
> >> The problem is that the original absolute rpath may be shorter
> >> than the newly generated relative rpath.
> >
> > Can we not just fix chrpath to cope with the case where the new path is
> > longer than the old one? It seems a bit sad to require
> > ever-more-convoluted workarounds for what is essentially a tool problem.
> >
>
> another option is to dump chrpath in favour of something which does it elegantly - patchelf
Right, or that. I've never used patchelf myself but if it works better
than chrpath (which isn't an especially high bar to set) then that
sounds like a decent enough option.
p.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-07-30 15:34 ` Phil Blundell
@ 2013-07-30 18:47 ` Andreas Oberritter
2013-07-31 23:25 ` Trevor Woerner
0 siblings, 1 reply; 10+ messages in thread
From: Andreas Oberritter @ 2013-07-30 18:47 UTC (permalink / raw)
To: Phil Blundell; +Cc: openembedded-core
On 30.07.2013 17:34, Phil Blundell wrote:
> On Tue, 2013-07-30 at 08:27 -0700, Khem Raj wrote:
>> On Jul 30, 2013, at 8:24 AM, Phil Blundell <pb@pbcl.net> wrote:
>>
>>> On Tue, 2013-07-02 at 11:50 +0200, Andreas Oberritter wrote:
>>>> Allows to use shorter TMPDIRs in corner cases, e.g. with native
>>>> perl modules, having a deep directory structure.
>>>>
>>>> The problem is that the original absolute rpath may be shorter
>>>> than the newly generated relative rpath.
>>>
>>> Can we not just fix chrpath to cope with the case where the new path is
>>> longer than the old one? It seems a bit sad to require
>>> ever-more-convoluted workarounds for what is essentially a tool problem.
>>>
I guess this can be done equally well in a later patch. I don't really
like this workaround either, but it improves the current situation
surprisingly well.
>>
>> another option is to dump chrpath in favour of something which does it elegantly - patchelf
>
> Right, or that. I've never used patchelf myself but if it works better
> than chrpath (which isn't an especially high bar to set) then that
> sounds like a decent enough option.
I won't have the time to work on either improving chrpath or integrating
patchelf. If Khem has a working solution, then let's integrate it.
AFAIR, the reasoning against patchelf was the bad availability across
distributions and it being a prerequisite for native builds. Today,
there's still no patchelf package in current releases of Debian or Ubuntu.
Regards,
Andreas
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath
2013-07-30 18:47 ` Andreas Oberritter
@ 2013-07-31 23:25 ` Trevor Woerner
0 siblings, 0 replies; 10+ messages in thread
From: Trevor Woerner @ 2013-07-31 23:25 UTC (permalink / raw)
To: Andreas Oberritter; +Cc: openembedded-core
On 30 July 2013 14:47, Andreas Oberritter <obi@opendreambox.org> wrote:
> AFAIR, the reasoning against patchelf was the bad availability across
> distributions and it being a prerequisite for native builds. Today,
> there's still no patchelf package in current releases of Debian or Ubuntu.
pseudo isn't part of most distributions either; but that doesn't mean
it can't be an important part of OE/Yocto :-) Couldn't patchelf-native
be built as part of the "getting your system ready to build" phase?
Khem, don't forget to add it to buildtools-tarball, the SDKs, and the
build applicance(s).
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-07-31 23:25 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-03-05 23:55 [PATCH][RFC] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
2013-03-06 0:32 ` [PATCH v2][RFC] " Andreas Oberritter
2013-03-06 7:19 ` [PATCH v2][RFC] chrpath.bbclass: strip common parent directories from rpath# Henning Heinold
2013-07-02 9:50 ` [RFC PATCH v3] chrpath.bbclass: strip common parent directories from rpath Andreas Oberritter
2013-07-30 15:24 ` Phil Blundell
2013-07-30 15:27 ` Khem Raj
2013-07-30 15:34 ` Phil Blundell
2013-07-30 18:47 ` Andreas Oberritter
2013-07-31 23:25 ` Trevor Woerner
-- strict thread matches above, loose matches on Subject: below --
2013-07-01 23:31 [PATCH v2][RFC] " Saul Wold
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox