export TARGET_LDFLAGS and native sstate

export TARGET_LDFLAGS and native sstate

[Mike Crowe]

We're building for both ARM and MIPS-based MACHINEs in a single source
tree. This seems to result in us compiling (or luckily most of the time
resurrecting from sstate-cache) two different versions of all -native
packages due to different base hashes.

It seems that this difference in base hashes is due to the exported
variable TARGET_LDFLAGS being different between the two CPUs:

< export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
---
> export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"

which in turn is because TARGET_LINK_HASH_STYLE is explicitly set to "sysv"
on MIPS but allowed to be "gnu" on everything else.

Every time I switch back and forth between building ARM-based and
MIPS-based machines the do_populate_sysroot_setscene tasks run for each
required -native recipe.

If I modify native.bbclass to do:

 TARGET_LDFLAGS = "${BUILD_LDFLAGS}"

then the hashes match and this unnecessary sysroot population is avoided.

Is this a sensible thing to be doing? It does mean that TARGET_LDFLAGS
contains a lot of (presumably unnecessary) rpath shenanigans but perhaps
that doesn't matter. :(

Thanks.

Mike.

Re: export TARGET_LDFLAGS and native sstate

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Mon, Apr 7, 2014 at 8:53 AM, Mike Crowe <mac@mcrowe.com> wrote:

> We're building for both ARM and MIPS-based MACHINEs in a single source
> tree. This seems to result in us compiling (or luckily most of the time
> resurrecting from sstate-cache) two different versions of all -native
> packages due to different base hashes.
>
> It seems that this difference in base hashes is due to the exported
> variable TARGET_LDFLAGS being different between the two CPUs:
>
> < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> ---
> > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
>

Heh, this i another case of a likely completely unnecessary export.
Software we build expects LDFLAGS to be used, not TARGET_LDFLAGS, so I
can't imagine that anything is using this export. Of course, it's
non-trivial to confirm that this is the case :)
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 1453 bytes --]

Re: export TARGET_LDFLAGS and native sstate

[Mike Crowe]

On Monday 07 April 2014 at 09:17:38 -0700, Chris Larson wrote:
> On Mon, Apr 7, 2014 at 8:53 AM, Mike Crowe <mac@mcrowe.com> wrote:
> 
> > We're building for both ARM and MIPS-based MACHINEs in a single source
> > tree. This seems to result in us compiling (or luckily most of the time
> > resurrecting from sstate-cache) two different versions of all -native
> > packages due to different base hashes.
> >
> > It seems that this difference in base hashes is due to the exported
> > variable TARGET_LDFLAGS being different between the two CPUs:
> >
> > < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> > ---
> > > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
> >
> 
> Heh, this i another case of a likely completely unnecessary export.
> Software we build expects LDFLAGS to be used, not TARGET_LDFLAGS, so I
> can't imagine that anything is using this export. Of course, it's
> non-trivial to confirm that this is the case :)

It did strike me as an odd thing to be exporting. Given the name I assumed
it had something to do with building the toolchain. I notice though that
the gcc recipes explicitly export LDFLAGS_FOR_TARGET inside tasks based on
TARGET_LDFLAGS anyway so the toolchain "should be fine". :)

I'm happy to try our complete build without exporting TARGET_LDFLAGS as a
first step but I realise that probably wouldn't be enough proof.

Thanks.

Mike.

Re: export TARGET_LDFLAGS and native sstate

[Mike Crowe]

On Monday 07 April 2014 at 17:49:51 +0100, Mike Crowe wrote:
> On Monday 07 April 2014 at 09:17:38 -0700, Chris Larson wrote:
> > On Mon, Apr 7, 2014 at 8:53 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > 
> > > We're building for both ARM and MIPS-based MACHINEs in a single source
> > > tree. This seems to result in us compiling (or luckily most of the time
> > > resurrecting from sstate-cache) two different versions of all -native
> > > packages due to different base hashes.
> > >
> > > It seems that this difference in base hashes is due to the exported
> > > variable TARGET_LDFLAGS being different between the two CPUs:
> > >
> > > < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> > > ---
> > > > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
> > >
> > 
> > Heh, this i another case of a likely completely unnecessary export.
> > Software we build expects LDFLAGS to be used, not TARGET_LDFLAGS, so I
> > can't imagine that anything is using this export. Of course, it's
> > non-trivial to confirm that this is the case :)

My git archaeology shows that this dates from the very first import from
svn back in 2005. Back then it looks like it was necessary for
wpa_supplicant which used it in its defconfig file. This is no longer the
case.

I didn't look at any other layers.

> It did strike me as an odd thing to be exporting. Given the name I assumed
> it had something to do with building the toolchain. I notice though that
> the gcc recipes explicitly export LDFLAGS_FOR_TARGET inside tasks based on
> TARGET_LDFLAGS anyway so the toolchain "should be fine". :)
> 
> I'm happy to try our complete build without exporting TARGET_LDFLAGS as a
> first step but I realise that probably wouldn't be enough proof.

I've tested our build without the "export" in front of TARGET_LDFLAGS in
bitbake.conf and saw no problems at all so I'm in favour of doing that.

Would a patch for this be acceptable? It does cause the world to be
rebuilt. :(

Thanks.

Mike.

Re: export TARGET_LDFLAGS and native sstate

[Chris Larson]

[-- Attachment #1: Type: text/plain, Size: 2825 bytes --]

On Thu, Apr 10, 2014 at 9:15 AM, Mike Crowe <mac@mcrowe.com> wrote:

> On Monday 07 April 2014 at 17:49:51 +0100, Mike Crowe wrote:
> > On Monday 07 April 2014 at 09:17:38 -0700, Chris Larson wrote:
> > > On Mon, Apr 7, 2014 at 8:53 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > >
> > > > We're building for both ARM and MIPS-based MACHINEs in a single
> source
> > > > tree. This seems to result in us compiling (or luckily most of the
> time
> > > > resurrecting from sstate-cache) two different versions of all -native
> > > > packages due to different base hashes.
> > > >
> > > > It seems that this difference in base hashes is due to the exported
> > > > variable TARGET_LDFLAGS being different between the two CPUs:
> > > >
> > > > < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> > > > ---
> > > > > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu
> -Wl,--as-needed"
> > > >
> > >
> > > Heh, this i another case of a likely completely unnecessary export.
> > > Software we build expects LDFLAGS to be used, not TARGET_LDFLAGS, so I
> > > can't imagine that anything is using this export. Of course, it's
> > > non-trivial to confirm that this is the case :)
>
> My git archaeology shows that this dates from the very first import from
> svn back in 2005. Back then it looks like it was necessary for
> wpa_supplicant which used it in its defconfig file. This is no longer the
> case.
>
> I didn't look at any other layers.
>
> > It did strike me as an odd thing to be exporting. Given the name I
> assumed
> > it had something to do with building the toolchain. I notice though that
> > the gcc recipes explicitly export LDFLAGS_FOR_TARGET inside tasks based
> on
> > TARGET_LDFLAGS anyway so the toolchain "should be fine". :)
> >
> > I'm happy to try our complete build without exporting TARGET_LDFLAGS as a
> > first step but I realise that probably wouldn't be enough proof.
>
> I've tested our build without the "export" in front of TARGET_LDFLAGS in
> bitbake.conf and saw no problems at all so I'm in favour of doing that.
>
> Would a patch for this be acceptable? It does cause the world to be
> rebuilt. :(


I'm a fan of this, personally, but you'll likely need to check with folks
like Richard Purdie for a final call, and this particular fix would have to
go in post-1.6, into the 1.7 timeframe (1.6 isn't branched yet) since we're
in the release candidate phase, and this has inherent risk. So, it might be
worth pursuing the merge of the workaround with alters TARGET_LDFLAGS in
native.bbclass to improve sstate reuse for 1.6, then pursue the unexport of
TARGET_LDFLAGS for 1.7.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3543 bytes --]

Re: export TARGET_LDFLAGS and native sstate

[Denys Dmytriyenko]

On Thu, Apr 10, 2014 at 10:36:07AM -0700, Chris Larson wrote:
> On Thu, Apr 10, 2014 at 9:15 AM, Mike Crowe <mac@mcrowe.com> wrote:
> 
> > On Monday 07 April 2014 at 17:49:51 +0100, Mike Crowe wrote:
> > > On Monday 07 April 2014 at 09:17:38 -0700, Chris Larson wrote:
> > > > On Mon, Apr 7, 2014 at 8:53 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > > >
> > > > > We're building for both ARM and MIPS-based MACHINEs in a single
> > source
> > > > > tree. This seems to result in us compiling (or luckily most of the
> > time
> > > > > resurrecting from sstate-cache) two different versions of all -native
> > > > > packages due to different base hashes.
> > > > >
> > > > > It seems that this difference in base hashes is due to the exported
> > > > > variable TARGET_LDFLAGS being different between the two CPUs:
> > > > >
> > > > > < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> > > > > ---
> > > > > > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu
> > -Wl,--as-needed"
> > > > >
> > > >
> > > > Heh, this i another case of a likely completely unnecessary export.
> > > > Software we build expects LDFLAGS to be used, not TARGET_LDFLAGS, so I
> > > > can't imagine that anything is using this export. Of course, it's
> > > > non-trivial to confirm that this is the case :)
> >
> > My git archaeology shows that this dates from the very first import from
> > svn back in 2005. Back then it looks like it was necessary for
> > wpa_supplicant which used it in its defconfig file. This is no longer the
> > case.
> >
> > I didn't look at any other layers.
> >
> > > It did strike me as an odd thing to be exporting. Given the name I
> > assumed
> > > it had something to do with building the toolchain. I notice though that
> > > the gcc recipes explicitly export LDFLAGS_FOR_TARGET inside tasks based
> > on
> > > TARGET_LDFLAGS anyway so the toolchain "should be fine". :)
> > >
> > > I'm happy to try our complete build without exporting TARGET_LDFLAGS as a
> > > first step but I realise that probably wouldn't be enough proof.
> >
> > I've tested our build without the "export" in front of TARGET_LDFLAGS in
> > bitbake.conf and saw no problems at all so I'm in favour of doing that.
> >
> > Would a patch for this be acceptable? It does cause the world to be
> > rebuilt. :(
> 
> 
> I'm a fan of this, personally, but you'll likely need to check with folks
> like Richard Purdie for a final call, and this particular fix would have to
> go in post-1.6, into the 1.7 timeframe (1.6 isn't branched yet) since we're

^^^ looks like it just got branched off less than an hour ago...


> in the release candidate phase, and this has inherent risk. So, it might be
> worth pursuing the merge of the workaround with alters TARGET_LDFLAGS in
> native.bbclass to improve sstate reuse for 1.6, then pursue the unexport of
> TARGET_LDFLAGS for 1.7.
> -- 
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics

> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: export TARGET_LDFLAGS and native sstate

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

-Khem
On Apr 7, 2014 5:53 AM, "Mike Crowe" <mac@mcrowe.com> wrote:
>
> We're building for both ARM and MIPS-based MACHINEs in a single source
> tree. This seems to result in us compiling (or luckily most of the time
> resurrecting from sstate-cache) two different versions of all -native
> packages due to different base hashes.
>
> It seems that this difference in base hashes is due to the exported
> variable TARGET_LDFLAGS being different between the two CPUs:
>
> < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> ---
> > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
>
> which in turn is because TARGET_LINK_HASH_STYLE is explicitly set to
"sysv"
> on MIPS but allowed to be "gnu" on everything else.
>
> Every time I switch back and forth between building ARM-based and
> MIPS-based machines the do_populate_sysroot_setscene tasks run for each
> required -native recipe.
>
> If I modify native.bbclass to do:
>
>  TARGET_LDFLAGS = "${BUILD_LDFLAGS}"

this is reasonable to do
>
> then the hashes match and this unnecessary sysroot population is avoided.
>
> Is this a sensible thing to be doing? It does mean that TARGET_LDFLAGS
> contains a lot of (presumably unnecessary) rpath shenanigans but perhaps
> that doesn't matter. :(
>
> Thanks.
>
> Mike.
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

[-- Attachment #2: Type: text/html, Size: 2111 bytes --]

Re: export TARGET_LDFLAGS and native sstate

[Mike Crowe]

On Monday 07 April 2014 at 12:35:44 -0700, Khem Raj wrote:
> -Khem
> On Apr 7, 2014 5:53 AM, "Mike Crowe" <mac@mcrowe.com> wrote:
> >
> > We're building for both ARM and MIPS-based MACHINEs in a single source
> > tree. This seems to result in us compiling (or luckily most of the time
> > resurrecting from sstate-cache) two different versions of all -native
> > packages due to different base hashes.
> >
> > It seems that this difference in base hashes is due to the exported
> > variable TARGET_LDFLAGS being different between the two CPUs:
> >
> > < export TARGET_LDFLAGS="-Wl,-O1  -Wl,--as-needed"
> > ---
> > > export TARGET_LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
> >
> > which in turn is because TARGET_LINK_HASH_STYLE is explicitly set to
> "sysv"
> > on MIPS but allowed to be "gnu" on everything else.
> >
> > Every time I switch back and forth between building ARM-based and
> > MIPS-based machines the do_populate_sysroot_setscene tasks run for each
> > required -native recipe.
> >
> > If I modify native.bbclass to do:
> >
> >  TARGET_LDFLAGS = "${BUILD_LDFLAGS}"
> 
> this is reasonable to do

I've submitted a patch that does this in
<1397640696-21281-1-git-send-email-mac@mcrowe.com> . The patch also does
the same thing to TARGET_CFLAGS, TARGET_CXXFLAGS and TARGET_CPPFLAGS. I'll
let the experts decided whether this is suitable for Daisy or not.

We're currently running our build with TARGET_LDFLAGS not exported and
haven't noticed any problems yet. I can submit a patch that does this but
I'd feel bound to also stop exporting TARGET_CFLAGS, TARGET_CXXFLAGS and
TARGET_CPPFLAGS for consistency though. Such a patch would definitely not
be suitable for Daisy.

Either (or both) patches solve my problem with task hashes.

Thanks.

Mike.