[PATCH] bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to avoid issues

[PATCH] bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to avoid issues

[Richard Purdie]

This resolves issues related to pigz-native when installing from sstate that people 
have been seeing. It also gives us a way to solve issues like the gzip-native race 
during sstate package creation covered in Yocto #1774.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 0d16d11..1570654 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -153,6 +153,12 @@ def sstate_installpkg(ss, d):
         bb.mkdirhier(dir)
         oe.path.remove(dir)
 
+    # We're adding binaries into the sysroots, we don't want to execute them
+    # whilst they're half installed or being installed so we need to
+    # remove the sysroots from PATH
+    savedpath = d.getVar("PATH")
+    d.setVar("PATH", "${ORIGPATH}")
+
     sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['name'])
     sstatepkg = d.getVar('SSTATE_PKG', True) + '_' + ss['name'] + ".tgz"
 
@@ -190,6 +196,8 @@ def sstate_installpkg(ss, d):
         # conflict with another writer
         os.remove(fixmefn)
 
+    d.setVar("PATH", savedpath)
+
     for state in ss['dirs']:
         prepdir(state[1])
         os.rename(sstateinst + state[0], state[1])
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index d4357d1..145a9a2 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -401,6 +401,7 @@ EXTRA_IMAGEDEPENDS = ""
 # Toolchain info.
 ##################################################################
 
+ORIGPATH := "${PATH}"
 PATH_prepend = "${STAGING_BINDIR_TOOLCHAIN}:${STAGING_BINDIR_CROSS}:${STAGING_DIR_NATIVE}${sbindir_native}:${STAGING_BINDIR_NATIVE}:${STAGING_DIR_NATIVE}${base_sbindir_native}:${STAGING_DIR_NATIVE}/${base_bindir_native}:"
 export PATH
 

Re: [PATCH] bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to avoid issues

[McClintock Matthew-B29882]

On Wed, Mar 21, 2012 at 5:44 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> This resolves issues related to pigz-native when installing from sstate that people
> have been seeing. It also gives us a way to solve issues like the gzip-native race
> during sstate package creation covered in Yocto #1774.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 0d16d11..1570654 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -153,6 +153,12 @@ def sstate_installpkg(ss, d):
>         bb.mkdirhier(dir)
>         oe.path.remove(dir)
>
> +    # We're adding binaries into the sysroots, we don't want to execute them
> +    # whilst they're half installed or being installed so we need to
> +    # remove the sysroots from PATH
> +    savedpath = d.getVar("PATH")
> +    d.setVar("PATH", "${ORIGPATH}")
> +
>     sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['name'])
>     sstatepkg = d.getVar('SSTATE_PKG', True) + '_' + ss['name'] + ".tgz"
>
> @@ -190,6 +196,8 @@ def sstate_installpkg(ss, d):
>         # conflict with another writer
>         os.remove(fixmefn)
>
> +    d.setVar("PATH", savedpath)
> +

So we always use the host tar and gzip here? Isn't there a reason we
build tar-native explicitly? Could be fine if it's not needed for
sstate-cache...

>     for state in ss['dirs']:
>         prepdir(state[1])
>         os.rename(sstateinst + state[0], state[1])
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index d4357d1..145a9a2 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -401,6 +401,7 @@ EXTRA_IMAGEDEPENDS = ""
>  # Toolchain info.
>  ##################################################################
>
> +ORIGPATH := "${PATH}"
>  PATH_prepend = "${STAGING_BINDIR_TOOLCHAIN}:${STAGING_BINDIR_CROSS}:${STAGING_DIR_NATIVE}${sbindir_native}:${STAGING_BINDIR_NATIVE}:${STAGING_DIR_NATIVE}${base_sbindir_native}:${STAGING_DIR_NATIVE}/${base_bindir_native}:"
>  export PATH
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: [PATCH] bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to avoid issues

[Richard Purdie]

On Wed, 2012-03-21 at 17:54 +0000, McClintock Matthew-B29882 wrote:
> On Wed, Mar 21, 2012 at 5:44 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > This resolves issues related to pigz-native when installing from sstate that people
> > have been seeing. It also gives us a way to solve issues like the gzip-native race
> > during sstate package creation covered in Yocto #1774.
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> > index 0d16d11..1570654 100644
> > --- a/meta/classes/sstate.bbclass
> > +++ b/meta/classes/sstate.bbclass
> > @@ -153,6 +153,12 @@ def sstate_installpkg(ss, d):
> >         bb.mkdirhier(dir)
> >         oe.path.remove(dir)
> >
> > +    # We're adding binaries into the sysroots, we don't want to execute them
> > +    # whilst they're half installed or being installed so we need to
> > +    # remove the sysroots from PATH
> > +    savedpath = d.getVar("PATH")
> > +    d.setVar("PATH", "${ORIGPATH}")
> > +
> >     sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['name'])
> >     sstatepkg = d.getVar('SSTATE_PKG', True) + '_' + ss['name'] + ".tgz"
> >
> > @@ -190,6 +196,8 @@ def sstate_installpkg(ss, d):
> >         # conflict with another writer
> >         os.remove(fixmefn)
> >
> > +    d.setVar("PATH", savedpath)
> > +
> 
> So we always use the host tar and gzip here? Isn't there a reason we
> build tar-native explicitly? Could be fine if it's not needed for
> sstate-cache...

I had to hit the archives but:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=577dd4b3e5d4861c31824d920fa170ba3a585f63

so yes, there is a good reason we might want to use our tar and I just
broken our old versions of tar workaround :(. We did check that tar code
for races and its relatively safe. The gzip situation is of course not
so safe and we have the open bugs to prove it :(.

So I should probably revert this patch although I'm reluctant since I
can think of other ways we can break the sysroots which this patch very
neatly solves.

The only other option would be to explicitly allow tar through some
linking/PATH magic :/.

Cheers,

Richard

Re: [PATCH] bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to avoid issues

[McClintock Matthew-B29882]

On Wed, Mar 21, 2012 at 4:12 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2012-03-21 at 17:54 +0000, McClintock Matthew-B29882 wrote:
>> On Wed, Mar 21, 2012 at 5:44 AM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > This resolves issues related to pigz-native when installing from sstate that people
>> > have been seeing. It also gives us a way to solve issues like the gzip-native race
>> > during sstate package creation covered in Yocto #1774.
>> >
>> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> > ---
>> > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
>> > index 0d16d11..1570654 100644
>> > --- a/meta/classes/sstate.bbclass
>> > +++ b/meta/classes/sstate.bbclass
>> > @@ -153,6 +153,12 @@ def sstate_installpkg(ss, d):
>> >         bb.mkdirhier(dir)
>> >         oe.path.remove(dir)
>> >
>> > +    # We're adding binaries into the sysroots, we don't want to execute them
>> > +    # whilst they're half installed or being installed so we need to
>> > +    # remove the sysroots from PATH
>> > +    savedpath = d.getVar("PATH")
>> > +    d.setVar("PATH", "${ORIGPATH}")
>> > +
>> >     sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['name'])
>> >     sstatepkg = d.getVar('SSTATE_PKG', True) + '_' + ss['name'] + ".tgz"
>> >
>> > @@ -190,6 +196,8 @@ def sstate_installpkg(ss, d):
>> >         # conflict with another writer
>> >         os.remove(fixmefn)
>> >
>> > +    d.setVar("PATH", savedpath)
>> > +
>>
>> So we always use the host tar and gzip here? Isn't there a reason we
>> build tar-native explicitly? Could be fine if it's not needed for
>> sstate-cache...
>
> I had to hit the archives but:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=577dd4b3e5d4861c31824d920fa170ba3a585f63
>
> so yes, there is a good reason we might want to use our tar and I just
> broken our old versions of tar workaround :(. We did check that tar code
> for races and its relatively safe. The gzip situation is of course not
> so safe and we have the open bugs to prove it :(.
>
> So I should probably revert this patch although I'm reluctant since I
> can think of other ways we can break the sysroots which this patch very
> neatly solves.
>
> The only other option would be to explicitly allow tar through some
> linking/PATH magic :/.

Not sure what the best course of action is... we do:

ASSUME_PROVIDED += "${SSTATE_PKG_TARZIPPROG}-native"

which fixes the gzip-native issue by not even building it....

-M