* work-shared not being correctly shared
@ 2012-09-24 15:38 Phil Blundell
2012-09-24 15:42 ` Saul Wold
0 siblings, 1 reply; 6+ messages in thread
From: Phil Blundell @ 2012-09-24 15:38 UTC (permalink / raw)
To: openembedded-core
Since updating to the latest versions of bitbake and oe-core I'm
experiencing some sort of conflict to do with the gcc sources in
tmp/work-shared.
What seems to be happening is that all the different recipes which share
that source directory are using different stamp values for do_unpack.
So, for example, I can do:
$ bitbake -f -c unpack gcc-cross
[ source unpacks ]
$ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack*
-rw-rw-r-- 1 pb pb 0 2012-09-24 15:37
tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.ea6dd89ddae3dcc205848b41ce0251cd
$ bitbake -f -c unpack gcc-cross
[ nothing happens ]
$ bitbake -f -c unpack gcc-cross
[ still nothing happens ]
$ bitbake -f -c unpack gcc-runtime
[ source unpacks again ]
$ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack*
-rw-rw-r-- 1 pb pb 0 2012-09-24 15:42
tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.e677db6d46e8c0c6109b02302aa477d0
$ bitbake -f -c unpack gcc-runtime
[ nothing happens ]
Worse, if I set BB_NUMBER_THREADS to some high number, I seem to end up
with multiple competing tasks all trying to unpack in parallel and the
build never makes any real forward progress.
Does anybody have any insight into what is going wrong here and how I
should debug it?
thanks
p.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared
2012-09-24 15:38 work-shared not being correctly shared Phil Blundell
@ 2012-09-24 15:42 ` Saul Wold
2012-09-24 16:22 ` Phil Blundell
0 siblings, 1 reply; 6+ messages in thread
From: Saul Wold @ 2012-09-24 15:42 UTC (permalink / raw)
To: Phil Blundell; +Cc: openembedded-core
On 09/24/2012 08:38 AM, Phil Blundell wrote:
> Since updating to the latest versions of bitbake and oe-core I'm
> experiencing some sort of conflict to do with the gcc sources in
> tmp/work-shared.
>
> What seems to be happening is that all the different recipes which share
> that source directory are using different stamp values for do_unpack.
> So, for example, I can do:
>
> $ bitbake -f -c unpack gcc-cross
> [ source unpacks ]
> $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack*
> -rw-rw-r-- 1 pb pb 0 2012-09-24 15:37
> tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.ea6dd89ddae3dcc205848b41ce0251cd
> $ bitbake -f -c unpack gcc-cross
> [ nothing happens ]
> $ bitbake -f -c unpack gcc-cross
> [ still nothing happens ]
> $ bitbake -f -c unpack gcc-runtime
> [ source unpacks again ]
> $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack*
> -rw-rw-r-- 1 pb pb 0 2012-09-24 15:42
> tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.e677db6d46e8c0c6109b02302aa477d0
> $ bitbake -f -c unpack gcc-runtime
> [ nothing happens ]
>
> Worse, if I set BB_NUMBER_THREADS to some high number, I seem to end up
> with multiple competing tasks all trying to unpack in parallel and the
> build never makes any real forward progress.
>
> Does anybody have any insight into what is going wrong here and how I
> should debug it?
>
Yes, I am seeing something similar with do_patch happening twice on the
work-shared area, I talked with Richard a bit ago about it, and I am
digging into the siginfo files, you can use bitbake-sigdiff to see what
sigs are changed causing it unpack twice (or in my case patch twice).
Sau!
> thanks
>
> p.
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared
2012-09-24 15:42 ` Saul Wold
@ 2012-09-24 16:22 ` Phil Blundell
2012-09-24 19:28 ` Martin Jansa
0 siblings, 1 reply; 6+ messages in thread
From: Phil Blundell @ 2012-09-24 16:22 UTC (permalink / raw)
To: Saul Wold; +Cc: openembedded-core
On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote:
> Yes, I am seeing something similar with do_patch happening twice on the
> work-shared area, I talked with Richard a bit ago about it, and I am
> digging into the siginfo files, you can use bitbake-sigdiff to see what
> sigs are changed causing it unpack twice (or in my case patch twice).
Ah, right, it seems to be rm_old_work that's causing the problem. I'll
disable that for now and figure out how to fix it later.
Thanks
p.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared
2012-09-24 16:22 ` Phil Blundell
@ 2012-09-24 19:28 ` Martin Jansa
2012-10-01 11:50 ` Phil Blundell
0 siblings, 1 reply; 6+ messages in thread
From: Martin Jansa @ 2012-09-24 19:28 UTC (permalink / raw)
To: Phil Blundell; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 726 bytes --]
On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote:
> On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote:
> > Yes, I am seeing something similar with do_patch happening twice on the
> > work-shared area, I talked with Richard a bit ago about it, and I am
> > digging into the siginfo files, you can use bitbake-sigdiff to see what
> > sigs are changed causing it unpack twice (or in my case patch twice).
>
> Ah, right, it seems to be rm_old_work that's causing the problem. I'll
> disable that for now and figure out how to fix it later.
That was already discussed few weeks ago, you should be able to find it
in archive.
Cheers,
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared
2012-09-24 19:28 ` Martin Jansa
@ 2012-10-01 11:50 ` Phil Blundell
2012-10-01 13:38 ` Richard Purdie
0 siblings, 1 reply; 6+ messages in thread
From: Phil Blundell @ 2012-10-01 11:50 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
On Mon, 2012-09-24 at 21:28 +0200, Martin Jansa wrote:
> On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote:
> > On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote:
> > > Yes, I am seeing something similar with do_patch happening twice on the
> > > work-shared area, I talked with Richard a bit ago about it, and I am
> > > digging into the siginfo files, you can use bitbake-sigdiff to see what
> > > sigs are changed causing it unpack twice (or in my case patch twice).
> >
> > Ah, right, it seems to be rm_old_work that's causing the problem. I'll
> > disable that for now and figure out how to fix it later.
>
> That was already discussed few weeks ago, you should be able to find it
> in archive.
I found this thread:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026185.html
but there didn't appear to be any clear conclusion about what the right
fix for the problem was. Does anybody have any further insight into
that?
p.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared
2012-10-01 11:50 ` Phil Blundell
@ 2012-10-01 13:38 ` Richard Purdie
0 siblings, 0 replies; 6+ messages in thread
From: Richard Purdie @ 2012-10-01 13:38 UTC (permalink / raw)
To: Phil Blundell; +Cc: Martin Jansa, openembedded-core
On Mon, 2012-10-01 at 12:50 +0100, Phil Blundell wrote:
> On Mon, 2012-09-24 at 21:28 +0200, Martin Jansa wrote:
> > On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote:
> > > On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote:
> > > > Yes, I am seeing something similar with do_patch happening twice on the
> > > > work-shared area, I talked with Richard a bit ago about it, and I am
> > > > digging into the siginfo files, you can use bitbake-sigdiff to see what
> > > > sigs are changed causing it unpack twice (or in my case patch twice).
> > >
> > > Ah, right, it seems to be rm_old_work that's causing the problem. I'll
> > > disable that for now and figure out how to fix it later.
> >
> > That was already discussed few weeks ago, you should be able to find it
> > in archive.
>
> I found this thread:
>
> http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026185.html
>
> but there didn't appear to be any clear conclusion about what the right
> fix for the problem was. Does anybody have any further insight into
> that?
The issue is that rm_old_work references PN in its task definition.
gcc's shared work directory assumes it can get the same sstate checksums
for fetch/unpack/patch which is broken by:
addtask rm_old_works before do_unpack
The trick would seem to be to change do_unpack for do_configure which is
how others worked around it.
Not a perfect fix but equally not annoying enough to lose lots of sleep
over.
Cheers,
Richard
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-10-01 13:51 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-09-24 15:38 work-shared not being correctly shared Phil Blundell
2012-09-24 15:42 ` Saul Wold
2012-09-24 16:22 ` Phil Blundell
2012-09-24 19:28 ` Martin Jansa
2012-10-01 11:50 ` Phil Blundell
2012-10-01 13:38 ` Richard Purdie
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox