Broken parallel build of gcc*

Broken parallel build of gcc*

[Enrico Scholz]

Hi,

some of the recent changes seem to break the parallel build of the gcc*
recipes which are sharing a same source directory:

| $ ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross -c cleansstate
| $ BB_NUMBER_THREADS=3 ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross
| ...
| NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
| NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
| NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Succeeded
| ERROR: Error executing a python function in ...recipes-devtools/gcc/gcc-cross_4.7.bb:
| OSError: [Errno 39] Directory not empty: '...tmp/work-shared/gcc-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7/git/'
| NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Failed

It seems that the work-shared/ directory is not locked and cleaned when
starting the second unpack.



Enrico

Re: Broken parallel build of gcc*

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

On Fri, Jul 20, 2012 at 12:28:29PM +0200, Enrico Scholz wrote:
> Hi,
> 
> some of the recent changes seem to break the parallel build of the gcc*
> recipes which are sharing a same source directory:
> 
> | $ ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross -c cleansstate
> | $ BB_NUMBER_THREADS=3 ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross
> | ...
> | NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
> | NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
> | NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Succeeded
> | ERROR: Error executing a python function in ...recipes-devtools/gcc/gcc-cross_4.7.bb:
> | OSError: [Errno 39] Directory not empty: '...tmp/work-shared/gcc-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7/git/'
> | NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Failed
> 
> It seems that the work-shared/ directory is not locked and cleaned when
> starting the second unpack.

I can confirm the same
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026174.html

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Broken parallel build of gcc*

[Richard Purdie]

On Fri, 2012-07-20 at 12:41 +0200, Martin Jansa wrote:
> On Fri, Jul 20, 2012 at 12:28:29PM +0200, Enrico Scholz wrote:
> > Hi,
> > 
> > some of the recent changes seem to break the parallel build of the gcc*
> > recipes which are sharing a same source directory:
> > 
> > | $ ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross -c cleansstate
> > | $ BB_NUMBER_THREADS=3 ./bitbake gcc-cross-initial gcc-cross-intermediate gcc-cross
> > | ...
> > | NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
> > | NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Started
> > | NOTE: package gcc-cross-initial-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Succeeded
> > | ERROR: Error executing a python function in ...recipes-devtools/gcc/gcc-cross_4.7.bb:
> > | OSError: [Errno 39] Directory not empty: '...tmp/work-shared/gcc-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7/git/'
> > | NOTE: package gcc-cross-4.7.1.0+git1+d07e24f4ab59f264d68d21838795349faab5dede-r7: task do_unpack: Failed
> > 
> > It seems that the work-shared/ directory is not locked and cleaned when
> > starting the second unpack.
> 
> I can confirm the same
> http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026174.html

The autobuilder is not seeing this and I'm not seeing it locally either.

This would imply there is some difference in configuration which is
triggering it. rm_work* is a possibility, as could ASSUME_PROVIDED
settings potentially although unlikely.

Encrico: Which DISTRO are you using and do you use rm_work*.bbclass?

Cheers,

Richard

Re: Broken parallel build of gcc*

[Enrico Scholz]

Richard Purdie
<richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
writes:

> Encrico: Which DISTRO are you using and do you use rm_work*.bbclass?

yes; rm_old_work seems to cause the problem.  Task uses ${PN} and was
declared with

  addtask rm_old_work before do_unpack

which causes different base hashes.


Is there a way to exclude such tasks from the signature handling?



Enrico

Re: Broken parallel build of gcc*

[Richard Purdie]

On Fri, 2012-07-20 at 13:53 +0200, Enrico Scholz wrote:
> Richard Purdie
> <richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
> writes:
> 
> > Encrico: Which DISTRO are you using and do you use rm_work*.bbclass?
> 
> yes; rm_old_work seems to cause the problem.  Task uses ${PN} and was
> declared with
> 
>   addtask rm_old_work before do_unpack
> 
> which causes different base hashes.
> 
> 
> Is there a way to exclude such tasks from the signature handling?

The easiest option is probably something like:

do_rm_old_works[vardepexclude] = "PN"

but this will trigger rebuilds depending on whether rm_work_old is
turned on or off.

The more advanced way to fix this would be in:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/lib/oe/sstatesig.py

probably something like:

if dep.endswith("do_rm_old_works"):
    return False

before the first check.

Cheers,

Richard

Re: Broken parallel build of gcc*

[Enrico Scholz]

Richard Purdie
<richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
writes:

>> Is there a way to exclude such tasks from the signature handling?
>
> The easiest option is probably something like:
>
> do_rm_old_works[vardepexclude] = "PN"

I tried this but it is not really an option because there must be added
a lot of completely unrelated variables:

 $ bitbake-diffsigs gcc-runtime*do_rm_old_work.sigdata* libgcc*do_rm_old_work.sigdata*

 Dependency on variable gcc_cv_collect2_libs was added
 Variable CPPFLAGS value changed from  to ${TARGET_CPPFLAGS}

(none of these is used within do_rm_old_works).


Enrico

Re: Broken parallel build of gcc*

[Richard Purdie]

On Fri, 2012-07-20 at 14:15 +0200, Enrico Scholz wrote:
> Richard Purdie
> <richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
> writes:
> 
> >> Is there a way to exclude such tasks from the signature handling?
> >
> > The easiest option is probably something like:
> >
> > do_rm_old_works[vardepexclude] = "PN"
> 
> I tried this but it is not really an option because there must be added
> a lot of completely unrelated variables:
> 
>  $ bitbake-diffsigs gcc-runtime*do_rm_old_work.sigdata* libgcc*do_rm_old_work.sigdata*
> 
>  Dependency on variable gcc_cv_collect2_libs was added
>  Variable CPPFLAGS value changed from  to ${TARGET_CPPFLAGS}
> 
> (none of these is used within do_rm_old_works).

Can you point me at a copy of the class your using and the full diffsigs
output please?

Thanks,

Richard

Re: Broken parallel build of gcc*

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

On Fri, Jul 20, 2012 at 01:28:03PM +0100, Richard Purdie wrote:
> On Fri, 2012-07-20 at 14:15 +0200, Enrico Scholz wrote:
> > Richard Purdie
> > <richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
> > writes:
> > 
> > >> Is there a way to exclude such tasks from the signature handling?
> > >
> > > The easiest option is probably something like:
> > >
> > > do_rm_old_works[vardepexclude] = "PN"
> > 
> > I tried this but it is not really an option because there must be added
> > a lot of completely unrelated variables:
> > 
> >  $ bitbake-diffsigs gcc-runtime*do_rm_old_work.sigdata* libgcc*do_rm_old_work.sigdata*
> > 
> >  Dependency on variable gcc_cv_collect2_libs was added
> >  Variable CPPFLAGS value changed from  to ${TARGET_CPPFLAGS}
> > 
> > (none of these is used within do_rm_old_works).
> 
> Can you point me at a copy of the class your using and the full diffsigs
> output please?

Mine is here
http://git.shr-project.org/git/?p=meta-smartphone.git;a=blob;f=meta-shr/classes/rm_old_work.bbclass;h=db13aa677958a52cd2c1b9efd6b90e38eadba41d;hb=refs/heads/shr
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Broken parallel build of gcc*

[Enrico Scholz]

Richard Purdie
<richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
writes:

>> >> Is there a way to exclude such tasks from the signature handling?
>> >
>> > The easiest option is probably something like:
>> >
>> > do_rm_old_works[vardepexclude] = "PN"
>> 
>> I tried this but it is not really an option...
> 
> Can you point me at a copy of the class your using and the full diffsigs
> output please?

---
do_rm_old_work() {
  :
}
addtask rm_old_work before do_unpack
---

suffices; the diffsigs difference is caused by different sh environment
between target, -native and -cross recipes.

I solved it for now by modifying sstatesig.py but a more generic way
would be nice.



Enrico

Re: Broken parallel build of gcc*

[Enrico Scholz]

Enrico Scholz
<enrico.scholz-wttK6gPy29v+Hn7q9Vec/7NAH6kLmebB@public.gmane.org>
writes:

>> Encrico: Which DISTRO are you using and do you use rm_work*.bbclass?
>
> yes; rm_old_work seems to cause the problem. 

The signature stuff was one problem; there is another, more trivial (but
probably more difficult) one:

Adding a task like

| addtask do_rm_old_work before do_unpack

causes stampfile dependencies for the shared gcc like

gcc.do_unpack ->  gcc-initial.do_rm_old_work
gcc.do_unpack ->  gcc-intermediate.do_rm_old_work

E.g. the earlier 'do_rm_old_work' stamps are recipe dependent but the
later 'do_unpack' are shared between recipes. Bitbake reexecutes do_unpack
when the corresponding do_rm_old_work is newer which is causing the seen
errors.

For now, I will declare the task as 'before do_compile'.


Enrico

Re: Broken parallel build of gcc*

[Richard Purdie]

On Sat, 2012-07-21 at 16:39 +0200, Enrico Scholz wrote:
> Enrico Scholz
> <enrico.scholz-wttK6gPy29v+Hn7q9Vec/7NAH6kLmebB@public.gmane.org>
> writes:
> 
> >> Encrico: Which DISTRO are you using and do you use rm_work*.bbclass?
> >
> > yes; rm_old_work seems to cause the problem. 
> 
> The signature stuff was one problem; there is another, more trivial (but
> probably more difficult) one:
> 
> Adding a task like
> 
> | addtask do_rm_old_work before do_unpack
> 
> causes stampfile dependencies for the shared gcc like
> 
> gcc.do_unpack ->  gcc-initial.do_rm_old_work
> gcc.do_unpack ->  gcc-intermediate.do_rm_old_work
> 
> E.g. the earlier 'do_rm_old_work' stamps are recipe dependent but the
> later 'do_unpack' are shared between recipes. Bitbake reexecutes do_unpack
> when the corresponding do_rm_old_work is newer which is causing the seen
> errors.

do_rm_old_work[stamp-base] = "${SS}"

in the gcc recipe would probably help that. Your alternative solution
obviously works too.

Cheers,

Richard

Re: Broken parallel build of gcc*

[Enrico Scholz]

Richard Purdie
<richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
writes:

> do_rm_old_work[stamp-base] = "${SS}"

will not work because do_rm_old_work depends on ${PN} (it cleans the
packages + packages-split directories of previously built packages in
${WORKDIR}).  Executing it for only one gcc* recipe will keep the
directories of the other recipes.


Enrico

Re: Broken parallel build of gcc*

[Richard Purdie]

On Sun, 2012-07-22 at 11:43 +0200, Enrico Scholz wrote:
> Richard Purdie
> <richard.purdie-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
> writes:
> 
> > do_rm_old_work[stamp-base] = "${SS}"
> 
> will not work because do_rm_old_work depends on ${PN} (it cleans the
> packages + packages-split directories of previously built packages in
> ${WORKDIR}).  Executing it for only one gcc* recipe will keep the
> directories of the other recipes.

This appears to be a situation I can't win with then...

Cheers,

Richard