[Buildroot] [PATCH 0/2 v2] Workaround for bug #7172: patchelf (branch yem/patchelf)

[Buildroot] [PATCH 0/2 v2] Workaround for bug #7172: patchelf (branch yem/patchelf)

[Yann E. MORIN]

Hello All!

This series introduces a workaround for the limitations expressed in bug
#7172 [0], which explains that a variable containing a $ sign is incorrectly
expanded.

Since it is not trivial to actually fix #7172, this series implements a
workaround, by:
  - explaining the limitation in the menuconfig and the manual
  - adding patchelf as a host-package, so users can tweak ELF files
    after the build (e.g. in a post-build script)

[0] https://bugs.buildroot.org/show_bug.cgi?id=7172

Changes v1 -> v2:
  - fix typoes  (Thomas DS)


Regards,
Yann E. MORIN.


The following changes since commit 26aefa672c96caefc17adf59a9ab3d327114ad36:

  bind: fix static openssl linking (2014-07-30 23:51:51 +0200)

are available in the git repository at:

  git://gitorious.org/buildroot/buildroot.git yem/patchelf

for you to fetch changes up to fc8f5090621f94da255a6191eec78125047eb318:

  package/patchelf: new host package (2014-07-31 00:44:25 +0200)

----------------------------------------------------------------
Yann E. MORIN (2):
      toolchain: linker options with a $ sign are not supported
      package/patchelf: new host package

 docs/manual/known-issues.txt    |  4 ++++
 package/Config.in.host          |  1 +
 package/patchelf/Config.in.host |  7 +++++++
 package/patchelf/patchelf.mk    | 12 ++++++++++++
 toolchain/toolchain-common.in   |  3 +++
 5 files changed, 27 insertions(+)
 create mode 100644 package/patchelf/Config.in.host
 create mode 100644 package/patchelf/patchelf.mk

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2 v2] toolchain: linker options with a $ sign are not supported

[Yann E. MORIN]

As reported in bug #7172 [0], setting BR2_TARGET_LDFLAGS to a value
containing a $ sign can lead to unexpected results.

This is because it is very hard to know when the $ sign gets evaluated:
  - in the Buildroot-level make
  - in the shell called by the Buildroot-level make
  - in the package's own build-system, either at configure time, in the
    Makefile, in a shell in the Makefile...

So, it is very difficult to know how much escaping that would need.

A proposal is to use a shell variable to pass such values unmolested.
But it is not that simple either, since it still contains a $ sign, and
there is not much certainty as to when it would be evaluated.

Instead, just document this limitation, both in the help text for
BR2_TARGET_LDFLAGS, and in the known-issues section in the manual.

Does not really fix #7172, but at least the limitation is documented.

[0] https://bugs.buildroot.org/show_bug.cgi?id=7172

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Mike Zick <minimod@morethan.org>
---
 docs/manual/known-issues.txt  | 4 ++++
 toolchain/toolchain-common.in | 3 +++
 2 files changed, 7 insertions(+)

diff --git a/docs/manual/known-issues.txt b/docs/manual/known-issues.txt
index 08469e9..5eaded9 100644
--- a/docs/manual/known-issues.txt
+++ b/docs/manual/known-issues.txt
@@ -3,6 +3,10 @@
 
 = Known issues
 
+* It is not possible to pass extra linker options via +BR2_TARGET_LDFLAGS+
+  if such options contain a +$+ sign. For example, the following is known
+  to break: +BR2_TARGET_LDFLAGS="-Wl,-rpath=\'$ORIGIN/../lib'"+
+
 * The +ltp-testsuite+ package does not build with the default uClibc
   configuration used by the Buildroot toolchain backend. The LTP
   testsuite uses several functions that are considered obsolete, such
diff --git a/toolchain/toolchain-common.in b/toolchain/toolchain-common.in
index 13de9e5..2df8ddb 100644
--- a/toolchain/toolchain-common.in
+++ b/toolchain/toolchain-common.in
@@ -132,6 +132,9 @@ config BR2_TARGET_LDFLAGS
 	help
 	  Extra options to pass to the linker when building for the target.
 
+	  Note that options with a '$' sign (eg. -Wl,-rpath='$ORIGIN/../lib')
+	  are not supported.
+
 config BR2_ECLIPSE_REGISTER
 	bool "Register toolchain within Eclipse Buildroot plug-in"
 	help
-- 
1.9.1

[Buildroot] [PATCH 1/2 v2] toolchain: linker options with a $ sign are not supported

[Thomas De Schampheleire]

On Thu, Jul 31, 2014 at 12:57 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> As reported in bug #7172 [0], setting BR2_TARGET_LDFLAGS to a value
> containing a $ sign can lead to unexpected results.
>
> This is because it is very hard to know when the $ sign gets evaluated:
>   - in the Buildroot-level make
>   - in the shell called by the Buildroot-level make
>   - in the package's own build-system, either at configure time, in the
>     Makefile, in a shell in the Makefile...
>
> So, it is very difficult to know how much escaping that would need.
>
> A proposal is to use a shell variable to pass such values unmolested.
> But it is not that simple either, since it still contains a $ sign, and
> there is not much certainty as to when it would be evaluated.
>
> Instead, just document this limitation, both in the help text for
> BR2_TARGET_LDFLAGS, and in the known-issues section in the manual.
>
> Does not really fix #7172, but at least the limitation is documented.
>
> [0] https://bugs.buildroot.org/show_bug.cgi?id=7172
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Mike Zick <minimod@morethan.org>
> ---
>  docs/manual/known-issues.txt  | 4 ++++
>  toolchain/toolchain-common.in | 3 +++
>  2 files changed, 7 insertions(+)
>
> diff --git a/docs/manual/known-issues.txt b/docs/manual/known-issues.txt
> index 08469e9..5eaded9 100644
> --- a/docs/manual/known-issues.txt
> +++ b/docs/manual/known-issues.txt
> @@ -3,6 +3,10 @@
>
>  = Known issues
>
> +* It is not possible to pass extra linker options via +BR2_TARGET_LDFLAGS+
> +  if such options contain a +$+ sign. For example, the following is known
> +  to break: +BR2_TARGET_LDFLAGS="-Wl,-rpath=\'$ORIGIN/../lib'"+
> +
>  * The +ltp-testsuite+ package does not build with the default uClibc
>    configuration used by the Buildroot toolchain backend. The LTP
>    testsuite uses several functions that are considered obsolete, such
> diff --git a/toolchain/toolchain-common.in b/toolchain/toolchain-common.in
> index 13de9e5..2df8ddb 100644
> --- a/toolchain/toolchain-common.in
> +++ b/toolchain/toolchain-common.in
> @@ -132,6 +132,9 @@ config BR2_TARGET_LDFLAGS
>         help
>           Extra options to pass to the linker when building for the target.
>
> +         Note that options with a '$' sign (eg. -Wl,-rpath='$ORIGIN/../lib')
> +         are not supported.
> +
>  config BR2_ECLIPSE_REGISTER
>         bool "Register toolchain within Eclipse Buildroot plug-in"
>         help


Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

[Buildroot] [PATCH 1/2 v2] toolchain: linker options with a $ sign are not supported

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 31 Jul 2014 00:57:15 +0200, Yann E. MORIN wrote:
> As reported in bug #7172 [0], setting BR2_TARGET_LDFLAGS to a value
> containing a $ sign can lead to unexpected results.
> 
> This is because it is very hard to know when the $ sign gets evaluated:
>   - in the Buildroot-level make
>   - in the shell called by the Buildroot-level make
>   - in the package's own build-system, either at configure time, in the
>     Makefile, in a shell in the Makefile...
> 
> So, it is very difficult to know how much escaping that would need.
> 
> A proposal is to use a shell variable to pass such values unmolested.
> But it is not that simple either, since it still contains a $ sign, and
> there is not much certainty as to when it would be evaluated.
> 
> Instead, just document this limitation, both in the help text for
> BR2_TARGET_LDFLAGS, and in the known-issues section in the manual.
> 
> Does not really fix #7172, but at least the limitation is documented.
> 
> [0] https://bugs.buildroot.org/show_bug.cgi?id=7172
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Mike Zick <minimod@morethan.org>
> ---
>  docs/manual/known-issues.txt  | 4 ++++
>  toolchain/toolchain-common.in | 3 +++
>  2 files changed, 7 insertions(+)

Applied, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2 v2] package/patchelf: new host package

[Yann E. MORIN]

In some situations, users may want to tweak the dynamic section of the
binaries (for example to add/set the RPATH to $ORIGIN/../lib).

Because it is not trivial to do it properly from the Buildroot
infrastructure, allow those users to use patchelf (e.g. from a
post-build script) to tweak binaries.

patchelf is able to:
  - modify an existing DT_RUNPATH tags
  - add a DT_RUNPATH tag if not already present
  - do the above to the DT_RPATH tag, too
  - set the path to the interpreter
  - remove DT_NEEDED tags
  - query a binary for the DT_RUNPATH/DT_RPATH tag, or for the
    interpreter path

Does not really fix #7172, but this is an appropriate workaround.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Mike Zick <minimod@morethan.org>
---
 package/Config.in.host          |  1 +
 package/patchelf/Config.in.host |  7 +++++++
 package/patchelf/patchelf.mk    | 12 ++++++++++++
 3 files changed, 20 insertions(+)
 create mode 100644 package/patchelf/Config.in.host
 create mode 100644 package/patchelf/patchelf.mk

diff --git a/package/Config.in.host b/package/Config.in.host
index 062c6c9..e05bbfa 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -13,6 +13,7 @@ menu "Host utilities"
 	source "package/omap-u-boot-utils/Config.in.host"
 	source "package/openocd/Config.in.host"
 	source "package/parted/Config.in.host"
+	source "package/patchelf/Config.in.host"
 	source "package/pwgen/Config.in.host"
 	source "package/sam-ba/Config.in.host"
 	source "package/squashfs/Config.in.host"
diff --git a/package/patchelf/Config.in.host b/package/patchelf/Config.in.host
new file mode 100644
index 0000000..d1c8375
--- /dev/null
+++ b/package/patchelf/Config.in.host
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_HOST_PATCHELF
+	bool "host patchelf"
+	help
+	  PatchELF is a small utility to modify the dynamic linker
+	  and RPATH of ELF executables.
+
+	  http://nixos.org/patchelf.html
diff --git a/package/patchelf/patchelf.mk b/package/patchelf/patchelf.mk
new file mode 100644
index 0000000..1a8e48d
--- /dev/null
+++ b/package/patchelf/patchelf.mk
@@ -0,0 +1,12 @@
+################################################################################
+#
+# patchelf
+#
+################################################################################
+
+PATCHELF_VERSION = 0.8
+PATCHELF_SITE = http://releases.nixos.org/patchelf/patchelf-0.8/
+PATCHELF_LICENSE = GPLv3
+PATCHELF_LICENSE_FILES = COPYING
+
+$(eval $(host-autotools-package))
-- 
1.9.1

[Buildroot] [PATCH 2/2 v2] package/patchelf: new host package

[Baruch Siach]

Hi Yann,

On Thu, Jul 31, 2014 at 12:57:16AM +0200, Yann E. MORIN wrote:
> In some situations, users may want to tweak the dynamic section of the
> binaries (for example to add/set the RPATH to $ORIGIN/../lib).
> 
> Because it is not trivial to do it properly from the Buildroot
> infrastructure, allow those users to use patchelf (e.g. from a
> post-build script) to tweak binaries.
> 
> patchelf is able to:
>   - modify an existing DT_RUNPATH tags
>   - add a DT_RUNPATH tag if not already present
>   - do the above to the DT_RPATH tag, too
>   - set the path to the interpreter
>   - remove DT_NEEDED tags
>   - query a binary for the DT_RUNPATH/DT_RPATH tag, or for the
>     interpreter path
> 
> Does not really fix #7172, but this is an appropriate workaround.

Maybe this is worth a mention as a workaround to the known issue of #7172 in 
the manual?

baruch


-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] [PATCH 2/2 v2] package/patchelf: new host package

[Thomas De Schampheleire]

On Thu, Jul 31, 2014 at 12:57 AM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> In some situations, users may want to tweak the dynamic section of the
> binaries (for example to add/set the RPATH to $ORIGIN/../lib).
>
> Because it is not trivial to do it properly from the Buildroot
> infrastructure, allow those users to use patchelf (e.g. from a
> post-build script) to tweak binaries.
>
> patchelf is able to:
>   - modify an existing DT_RUNPATH tags
>   - add a DT_RUNPATH tag if not already present
>   - do the above to the DT_RPATH tag, too
>   - set the path to the interpreter
>   - remove DT_NEEDED tags
>   - query a binary for the DT_RUNPATH/DT_RPATH tag, or for the
>     interpreter path
>
> Does not really fix #7172, but this is an appropriate workaround.
>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Mike Zick <minimod@morethan.org>
> ---
>  package/Config.in.host          |  1 +
>  package/patchelf/Config.in.host |  7 +++++++
>  package/patchelf/patchelf.mk    | 12 ++++++++++++
>  3 files changed, 20 insertions(+)
>  create mode 100644 package/patchelf/Config.in.host
>  create mode 100644 package/patchelf/patchelf.mk
>
> diff --git a/package/Config.in.host b/package/Config.in.host
> index 062c6c9..e05bbfa 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -13,6 +13,7 @@ menu "Host utilities"
>         source "package/omap-u-boot-utils/Config.in.host"
>         source "package/openocd/Config.in.host"
>         source "package/parted/Config.in.host"
> +       source "package/patchelf/Config.in.host"
>         source "package/pwgen/Config.in.host"
>         source "package/sam-ba/Config.in.host"
>         source "package/squashfs/Config.in.host"
> diff --git a/package/patchelf/Config.in.host b/package/patchelf/Config.in.host
> new file mode 100644
> index 0000000..d1c8375
> --- /dev/null
> +++ b/package/patchelf/Config.in.host
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_HOST_PATCHELF
> +       bool "host patchelf"
> +       help
> +         PatchELF is a small utility to modify the dynamic linker
> +         and RPATH of ELF executables.
> +
> +         http://nixos.org/patchelf.html
> diff --git a/package/patchelf/patchelf.mk b/package/patchelf/patchelf.mk
> new file mode 100644
> index 0000000..1a8e48d
> --- /dev/null
> +++ b/package/patchelf/patchelf.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# patchelf
> +#
> +################################################################################
> +
> +PATCHELF_VERSION = 0.8
> +PATCHELF_SITE = http://releases.nixos.org/patchelf/patchelf-0.8/
> +PATCHELF_LICENSE = GPLv3
> +PATCHELF_LICENSE_FILES = COPYING
> +
> +$(eval $(host-autotools-package))

Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>

I do agree with Baruch's comment that it will be helpful for users
desiring to use $ in the LDFLAGS to know that patchelf exists in
buildroot. This could be added in the first patch of this series, or
as a third patch.

Best regards,
Thomas

[Buildroot] [PATCH 2/2 v2] package/patchelf: new host package

[Thomas Petazzoni]

Dear Yann E. MORIN,

On Thu, 31 Jul 2014 00:57:16 +0200, Yann E. MORIN wrote:
> In some situations, users may want to tweak the dynamic section of the
> binaries (for example to add/set the RPATH to $ORIGIN/../lib).
> 
> Because it is not trivial to do it properly from the Buildroot
> infrastructure, allow those users to use patchelf (e.g. from a
> post-build script) to tweak binaries.
> 
> patchelf is able to:
>   - modify an existing DT_RUNPATH tags
>   - add a DT_RUNPATH tag if not already present
>   - do the above to the DT_RPATH tag, too
>   - set the path to the interpreter
>   - remove DT_NEEDED tags
>   - query a binary for the DT_RUNPATH/DT_RPATH tag, or for the
>     interpreter path
> 
> Does not really fix #7172, but this is an appropriate workaround.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Mike Zick <minimod@morethan.org>
> ---
>  package/Config.in.host          |  1 +
>  package/patchelf/Config.in.host |  7 +++++++
>  package/patchelf/patchelf.mk    | 12 ++++++++++++
>  3 files changed, 20 insertions(+)
>  create mode 100644 package/patchelf/Config.in.host
>  create mode 100644 package/patchelf/patchelf.mk
> 
> diff --git a/package/Config.in.host b/package/Config.in.host
> index 062c6c9..e05bbfa 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -13,6 +13,7 @@ menu "Host utilities"
>  	source "package/omap-u-boot-utils/Config.in.host"
>  	source "package/openocd/Config.in.host"
>  	source "package/parted/Config.in.host"
> +	source "package/patchelf/Config.in.host"
>  	source "package/pwgen/Config.in.host"
>  	source "package/sam-ba/Config.in.host"
>  	source "package/squashfs/Config.in.host"
> diff --git a/package/patchelf/Config.in.host b/package/patchelf/Config.in.host
> new file mode 100644
> index 0000000..d1c8375
> --- /dev/null
> +++ b/package/patchelf/Config.in.host
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_HOST_PATCHELF
> +	bool "host patchelf"
> +	help
> +	  PatchELF is a small utility to modify the dynamic linker
> +	  and RPATH of ELF executables.
> +
> +	  http://nixos.org/patchelf.html
> diff --git a/package/patchelf/patchelf.mk b/package/patchelf/patchelf.mk
> new file mode 100644
> index 0000000..1a8e48d
> --- /dev/null
> +++ b/package/patchelf/patchelf.mk
> @@ -0,0 +1,12 @@
> +################################################################################
> +#
> +# patchelf
> +#
> +################################################################################
> +
> +PATCHELF_VERSION = 0.8
> +PATCHELF_SITE = http://releases.nixos.org/patchelf/patchelf-0.8/
> +PATCHELF_LICENSE = GPLv3

The license is actually GPLv3+, see the README file of the project.

> +PATCHELF_LICENSE_FILES = COPYING
> +
> +$(eval $(host-autotools-package))

Other than that, applied, thanks!

I think it would be good to use that to make the toolchain relocatable,
instead of something like http://patchwork.ozlabs.org/patch/359841/.
What do you think?

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 2/2 v2] package/patchelf: new host package

[Yann E. MORIN]

Thomas, Samuel, All,

On 2014-07-31 23:55 +0200, Thomas Petazzoni spake thusly:
> On Thu, 31 Jul 2014 00:57:16 +0200, Yann E. MORIN wrote:
> > In some situations, users may want to tweak the dynamic section of the
> > binaries (for example to add/set the RPATH to $ORIGIN/../lib).
> > 
> > Because it is not trivial to do it properly from the Buildroot
> > infrastructure, allow those users to use patchelf (e.g. from a
> > post-build script) to tweak binaries.
[--SNIP--]
> > +PATCHELF_LICENSE = GPLv3
> The license is actually GPLv3+, see the README file of the project.
[--SNIP--]
> Other than that, applied, thanks!

Thanks! :-)

> I think it would be good to use that to make the toolchain relocatable,
> instead of something like http://patchwork.ozlabs.org/patch/359841/.
> What do you think?

I think that Samuel already has a WIP branch on making a relocatable
SDK: https://github.com/tSed/buildroot/tree/sma/reloc-sdk/wip

So, Samuel, what do you think of it? ;-)

(Note how fast I was able to elude and reassign the question?)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'