[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Yann E. MORIN]

Hello All!

TL;DR: gcc gets confused by our symlinks trick that we use to fix the
rpath handling. We replace symlinks with script wrappers instead.

See the first commit for the whole story.

Fixes: #11031.


Regards,
Yann E. MORIN.


The following changes since commit 2c57dc0469f7e32182932e5937f61f82b01a743a

  cjson: bump to version 1.7.7 (2018-05-25 16:22:11 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 59b3d35d7c805ebee29dc7185ea63e55aca768ca

  package/elf2flt: replace hard-links with script wrappers to fix rpath (2018-05-25 22:24:32 +0200)


----------------------------------------------------------------
Yann E. MORIN (2):
      package/binutils: switch from symlinks to script wrappers
      package/elf2flt: replace hard-links with script wrappers to fix rpath

 package/binutils/binutils.mk |  9 ++++++---
 package/elf2flt/elf2flt.mk   | 17 +++++++++++++++++
 2 files changed, 23 insertions(+), 3 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2] package/binutils: switch from symlinks to script wrappers

[Yann E. MORIN]

Commit f9cffb6af464 (binutils: replace hard-links with soft-links to
fix rpath) has a side effect that when we build for a noMMU target,
elf2flt will in turn replace some of the programs installed by
binutils, with its own wrappers.

For example, it will rename host/TUPLE/bin/ld to ld.real, and add its
own wrapper in place of the origginal. It does the same for
host/bin/TUPLE-ld and host/bin/TUPLE-ld.real.

However, we had already made ld a symlink to ../../bin/TUPLE-ld, so
host/TUPLE/bin/ld.real will still point to host/bin/TUPLE-ld when we
want it to point to ld.real instead...

This ultimately confuses gcc later on.

Of course, the culprit is also elf2flt, which also installs similar
hardlinks that would ultimately exhibit the same rpath issue as the
one fixed by f9cffb6af464. Note: we haven't had an issue so far with
that, because those tools installed by elf2flt only link with libz,
which is most often present on the host system. So, all seem well,
but is nonetheless broken; this will be fixed in a subsequent commit.

But back on topic. If we were to fix elf2flt with similar symlinks,
gcc still gets confused. The underlying reason for this confusion is
not entirely clear, though... It looks like something is trying to
dereference symlinks and gets confused by the resut somehow...

So, in an attempt to restore some sanity in all this mess, we try to
restore the previous behaviour, we switch from using symlinks to using
shell script wrappers. We ensure that the behaviour really mimicks that
of a hardlink, by telling bash to set arg[0] to the name of the script,
not to that of the wrapped executable:

    #!/usr/bin/env bash
    exec -a "${0}" "${0%/*}/../../bin/TUPLE-${0##*/}" "${@}"

"exec -a FOO" is a bashism that telss bash to use FOO as argv[0]
instead of the name of the executable.

With this trick, we're back to working conditions for building for
noMMU targets.

Fixes: #11031.

Note: the wrappers are effectively all totally identical, indeed. We
can optimise that by using hardlinks in a followup commit, if that
even makes sense...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Christophe Priouzeau <christophe.priouzeau@st.com>
---
 package/binutils/binutils.mk | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/package/binutils/binutils.mk b/package/binutils/binutils.mk
index b24e4334c2..25650b31c7 100644
--- a/package/binutils/binutils.mk
+++ b/package/binutils/binutils.mk
@@ -132,13 +132,16 @@ endif
 
 # Hardlinks between binaries in different directories cause a problem
 # with rpath fixup, so we de-hardlink those binaries, and replace them
-# with symbolic links.
+# with wrapper scripts.
 BINUTILS_TOOLS = ar as ld ld.bfd nm objcopy objdump ranlib readelf strip
 define HOST_BINUTILS_FIXUP_HARDLINKS
 	$(foreach tool,$(BINUTILS_TOOLS),\
-		rm -f $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool) ; \
-		ln -s ../../bin/$(GNU_TARGET_NAME)-$(tool) $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool)
+		rm -f $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool) && \
+		printf '#!/usr/bin/env bash\nexec -a "$${0}" "$${0%%/*}/%s-$${0##*/}" "$${@}"\n' \
+			../../bin/$(GNU_TARGET_NAME) \
+			>$(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool)
 	)
+	chmod +x $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/*
 endef
 HOST_BINUTILS_POST_INSTALL_HOOKS += HOST_BINUTILS_FIXUP_HARDLINKS
 
-- 
2.14.1

[Buildroot] [PATCH 2/2] package/elf2flt: replace hard-links with script wrappers to fix rpath

[Yann E. MORIN]

Do for elf2flt what we did for binutils: replace the hardlinks (which
break rpath handling) with script wrappers.

See previous commit (package/binutils: switch from symlinks to script
wrappers) and commit f9cffb6af4 (binutils: replace hard-links with
soft-links to fix rpath) for the complete story.

Fixes: #11031.

Reported-by: Christophe Priouzeau <christophe.priouzeau@st.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Christophe Priouzeau <christophe.priouzeau@st.com>
---
 package/elf2flt/elf2flt.mk | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/package/elf2flt/elf2flt.mk b/package/elf2flt/elf2flt.mk
index 995dfedebd..6ce5cd34a2 100644
--- a/package/elf2flt/elf2flt.mk
+++ b/package/elf2flt/elf2flt.mk
@@ -29,4 +29,21 @@ endif
 
 HOST_ELF2FLT_CONF_ENV = LIBS="$(HOST_ELF2FLT_LIBS)"
 
+# Hardlinks between binaries in different directories cause a problem
+# # with rpath fixup, so we de-hardlink those binaries, and replace them
+# with wrapper scripts. ld and ld.real are already wrappers, but they
+# point to the binutils, so we have to replace them with wrappers to
+# the elf2flt ones.
+ELF2FLT_TOOLS = elf2flt flthdr ld ld.real
+define HOST_ELF2FLT_FIXUP_HARDLINKS
+	$(foreach tool,$(ELF2FLT_TOOLS),\
+		rm -f $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool) && \
+		printf '#!/usr/bin/env bash\nexec -a "$${0}" "$${0%%/*}/%s-$${0##*/}" "$${@}"\n' \
+			../../bin/$(GNU_TARGET_NAME) \
+			>$(HOST_DIR)/$(GNU_TARGET_NAME)/bin/$(tool)
+	)
+	chmod +x $(HOST_DIR)/$(GNU_TARGET_NAME)/bin/*
+endef
+HOST_ELF2FLT_POST_INSTALL_HOOKS += HOST_ELF2FLT_FIXUP_HARDLINKS
+
 $(eval $(host-autotools-package))
-- 
2.14.1

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Arnout Vandecappelle]

On 25-05-18 22:42, Yann E. MORIN wrote:
> Hello All!
> 
> TL;DR: gcc gets confused by our symlinks trick that we use to fix the
> rpath handling. We replace symlinks with script wrappers instead.

 So maybe, instead of laying hacks upon hacks, we should go back to the drawing
board on the rpath handling? I would feel a lot more comfortable if we could
just fix fix-rpath instead of playing the whack-a-mole game of workarounds in
packages.

 I don't see a full solution yet, but basically I think that we should
- detect when an executable has multiple hardlinks;
- detect if it has already been "fixed" in this fix-rpath run;
- if so, clone the file before fixing it proper.

 The weak point is: if we clone, there may be some hardlinks that actually could
still be reused as hardlinks. So you'd have to detect them first.

 Alternatively we could consider that this is anyway only about the host tree,
which is normally the smallest of the three trees (when you exclude external
toolchain). So maybe we should just always clone for the host tree.


 I realize we're very close to the release, but I still think that always
cloning is probably the safest way to do it.

 Regards,
 Arnout

> 
> See the first commit for the whole story.
> 
> Fixes: #11031.
> 
> 
> Regards,
> Yann E. MORIN.
> 
> 
> The following changes since commit 2c57dc0469f7e32182932e5937f61f82b01a743a
> 
>   cjson: bump to version 1.7.7 (2018-05-25 16:22:11 +0200)
> 
> 
> are available in the git repository at:
> 
>   git://git.buildroot.org/~ymorin/git/buildroot.git
> 
> for you to fetch changes up to 59b3d35d7c805ebee29dc7185ea63e55aca768ca
> 
>   package/elf2flt: replace hard-links with script wrappers to fix rpath (2018-05-25 22:24:32 +0200)
> 
> 
> ----------------------------------------------------------------
> Yann E. MORIN (2):
>       package/binutils: switch from symlinks to script wrappers
>       package/elf2flt: replace hard-links with script wrappers to fix rpath
> 
>  package/binutils/binutils.mk |  9 ++++++---
>  package/elf2flt/elf2flt.mk   | 17 +++++++++++++++++
>  2 files changed, 23 insertions(+), 3 deletions(-)
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Yann E. MORIN]

Arnout, All,

On 2018-05-27 16:34 +0200, Arnout Vandecappelle spake thusly:
> On 25-05-18 22:42, Yann E. MORIN wrote:
> > Hello All!
> > 
> > TL;DR: gcc gets confused by our symlinks trick that we use to fix the
> > rpath handling. We replace symlinks with script wrappers instead.
> 
>  So maybe, instead of laying hacks upon hacks, we should go back to the drawing
> board on the rpath handling? I would feel a lot more comfortable if we could
> just fix fix-rpath instead of playing the whack-a-mole game of workarounds in
> packages.

Well, we're not really playing whack-a-mole, because the isue is only
about packages that install hardlinks in two locations, because of the
bin/TUPLE-foo and TUPLE/bin/foo "duplication".

The only ones doing such a thing AFAIK are binutils and elf2flt.

gcc installs hardlinks, but in the same directory. The tools it installs
in a TUPLE-named directory are not hardlinks.

>  I don't see a full solution yet, but basically I think that we should
> - detect when an executable has multiple hardlinks;
> - detect if it has already been "fixed" in this fix-rpath run;
> - if so, clone the file before fixing it proper.

What did you mean by "clone" in this context? Remove the hardlinks and
replace them by a copy of the file?

>  The weak point is: if we clone, there may be some hardlinks that actually could
> still be reused as hardlinks. So you'd have to detect them first.

Sorry, I don't follow you here. If we replaced hardlinks with actual
copies, how come we'd still have hardlinks?

>  Alternatively we could consider that this is anyway only about the host tree,
> which is normally the smallest of the three trees (when you exclude external
> toolchain). So maybe we should just always clone for the host tree.

I'm afraid that detecting hardlinks will not be cheap, even when we
exclude staging/ (which is a sub-dir of host/) from the search...

>  I realize we're very close to the release, but I still think that always
> cloning is probably the safest way to do it.

Do we envision other impacted packages? I don't think so. Really,
binutils and elf2flt really seem to be the only two packages impacted
by the problem.

That we missed the noMMU and elf2flt cases previously is just an
oversight on our part...

Then. if I am wrong, we're still talking about only two packages for
now, and they only break because of the ARC archtecture, for which we
use bleeding-edge tools that require we have libfl.so. Two. Single.
Packages. We've always refused to add infra when only very few packages
were impacted.

Or were you suggesting that we do not add infra, but we replace the
current two hacks (Thomas' symlinks or my script wrappers) by an actual
copy, still just for binutils and elf2flt? In which case we would not
loose time detecting hard-links in a generic way, obviously.

Regards,
Yann E. MORIN.

>  Regards,
>  Arnout
> 
> > 
> > See the first commit for the whole story.
> > 
> > Fixes: #11031.
> > 
> > 
> > Regards,
> > Yann E. MORIN.
> > 
> > 
> > The following changes since commit 2c57dc0469f7e32182932e5937f61f82b01a743a
> > 
> >   cjson: bump to version 1.7.7 (2018-05-25 16:22:11 +0200)
> > 
> > 
> > are available in the git repository at:
> > 
> >   git://git.buildroot.org/~ymorin/git/buildroot.git
> > 
> > for you to fetch changes up to 59b3d35d7c805ebee29dc7185ea63e55aca768ca
> > 
> >   package/elf2flt: replace hard-links with script wrappers to fix rpath (2018-05-25 22:24:32 +0200)
> > 
> > 
> > ----------------------------------------------------------------
> > Yann E. MORIN (2):
> >       package/binutils: switch from symlinks to script wrappers
> >       package/elf2flt: replace hard-links with script wrappers to fix rpath
> > 
> >  package/binutils/binutils.mk |  9 ++++++---
> >  package/elf2flt/elf2flt.mk   | 17 +++++++++++++++++
> >  2 files changed, 23 insertions(+), 3 deletions(-)
> > 
> 
> -- 
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Yann E. MORIN]

Arnout, all,

On 2018-05-27 17:35 +0200, Yann E. MORIN spake thusly:
> On 2018-05-27 16:34 +0200, Arnout Vandecappelle spake thusly:
> >  So maybe, instead of laying hacks upon hacks, we should go back to the drawing
> > board on the rpath handling? I would feel a lot more comfortable if we could
> > just fix fix-rpath instead of playing the whack-a-mole game of workarounds in
> > packages.
[--SNIP--]
> Or were you suggesting that we do not add infra, but we replace the
> current two hacks (Thomas' symlinks or my script wrappers) by an actual
> copy, still just for binutils and elf2flt? In which case we would not
> loose time detecting hard-links in a generic way, obviously.

Something like the following, maybe?
    https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/binutils-elf2flt-fix-2

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > Arnout, all,
 > On 2018-05-27 17:35 +0200, Yann E. MORIN spake thusly:
 >> On 2018-05-27 16:34 +0200, Arnout Vandecappelle spake thusly:
 >> >  So maybe, instead of laying hacks upon hacks, we should go back to the drawing
 >> > board on the rpath handling? I would feel a lot more comfortable if we could
 >> > just fix fix-rpath instead of playing the whack-a-mole game of workarounds in
 >> > packages.
 > [--SNIP--]
 >> Or were you suggesting that we do not add infra, but we replace the
 >> current two hacks (Thomas' symlinks or my script wrappers) by an actual
 >> copy, still just for binutils and elf2flt? In which case we would not
 >> loose time detecting hard-links in a generic way, obviously.

 > Something like the following, maybe?
 >     https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/binutils-elf2flt-fix-2

Yes, +/- the comments that still refers to wrapper scripts.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH 0/2] package/binutils+elf2flt: fix for noMMU targets

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

Hi,

 >> I don't see a full solution yet, but basically I think that we should
 >> - detect when an executable has multiple hardlinks;
 >> - detect if it has already been "fixed" in this fix-rpath run;
 >> - if so, clone the file before fixing it proper.

 > What did you mean by "clone" in this context? Remove the hardlinks and
 > replace them by a copy of the file?

Yes, I guess that was what Arnout meant. "break" the hard link and
duplicate the file content.

 >> Alternatively we could consider that this is anyway only about the host tree,
 >> which is normally the smallest of the three trees (when you exclude external
 >> toolchain). So maybe we should just always clone for the host tree.

 > I'm afraid that detecting hardlinks will not be cheap, even when we
 > exclude staging/ (which is a sub-dir of host/) from the search...

Just detecting hard linked files should be easy (E.G. st_nlink > 1 when
stat'ing). Figuring out where the other hard links are is indeed hard,
but we afaik don't need to know that.


 >> I realize we're very close to the release, but I still think that always
 >> cloning is probably the safest way to do it.

 > Do we envision other impacted packages? I don't think so. Really,
 > binutils and elf2flt really seem to be the only two packages impacted
 > by the problem.

 > That we missed the noMMU and elf2flt cases previously is just an
 > oversight on our part...

 > Then. if I am wrong, we're still talking about only two packages for
 > now, and they only break because of the ARC archtecture, for which we
 > use bleeding-edge tools that require we have libfl.so. Two. Single.
 > Packages. We've always refused to add infra when only very few packages
 > were impacted.

 > Or were you suggesting that we do not add infra, but we replace the
 > current two hacks (Thomas' symlinks or my script wrappers) by an actual
 > copy, still just for binutils and elf2flt? In which case we would not
 > loose time detecting hard-links in a generic way, obviously.

I can follow Arnout, but I think the script wrappers (or simply copies
of the binaries) is good enough for 2018.05.

-- 
Bye, Peter Korsgaard