* work-shared not being correctly shared @ 2012-09-24 15:38 Phil Blundell 2012-09-24 15:42 ` Saul Wold 0 siblings, 1 reply; 6+ messages in thread From: Phil Blundell @ 2012-09-24 15:38 UTC (permalink / raw) To: openembedded-core Since updating to the latest versions of bitbake and oe-core I'm experiencing some sort of conflict to do with the gcc sources in tmp/work-shared. What seems to be happening is that all the different recipes which share that source directory are using different stamp values for do_unpack. So, for example, I can do: $ bitbake -f -c unpack gcc-cross [ source unpacks ] $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack* -rw-rw-r-- 1 pb pb 0 2012-09-24 15:37 tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.ea6dd89ddae3dcc205848b41ce0251cd $ bitbake -f -c unpack gcc-cross [ nothing happens ] $ bitbake -f -c unpack gcc-cross [ still nothing happens ] $ bitbake -f -c unpack gcc-runtime [ source unpacks again ] $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack* -rw-rw-r-- 1 pb pb 0 2012-09-24 15:42 tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.e677db6d46e8c0c6109b02302aa477d0 $ bitbake -f -c unpack gcc-runtime [ nothing happens ] Worse, if I set BB_NUMBER_THREADS to some high number, I seem to end up with multiple competing tasks all trying to unpack in parallel and the build never makes any real forward progress. Does anybody have any insight into what is going wrong here and how I should debug it? thanks p. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared 2012-09-24 15:38 work-shared not being correctly shared Phil Blundell @ 2012-09-24 15:42 ` Saul Wold 2012-09-24 16:22 ` Phil Blundell 0 siblings, 1 reply; 6+ messages in thread From: Saul Wold @ 2012-09-24 15:42 UTC (permalink / raw) To: Phil Blundell; +Cc: openembedded-core On 09/24/2012 08:38 AM, Phil Blundell wrote: > Since updating to the latest versions of bitbake and oe-core I'm > experiencing some sort of conflict to do with the gcc sources in > tmp/work-shared. > > What seems to be happening is that all the different recipes which share > that source directory are using different stamp values for do_unpack. > So, for example, I can do: > > $ bitbake -f -c unpack gcc-cross > [ source unpacks ] > $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack* > -rw-rw-r-- 1 pb pb 0 2012-09-24 15:37 > tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.ea6dd89ddae3dcc205848b41ce0251cd > $ bitbake -f -c unpack gcc-cross > [ nothing happens ] > $ bitbake -f -c unpack gcc-cross > [ still nothing happens ] > $ bitbake -f -c unpack gcc-runtime > [ source unpacks again ] > $ ls -l tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack* > -rw-rw-r-- 1 pb pb 0 2012-09-24 15:42 > tmp-eglibc/stamps/work-shared/gcc-4.7.2-r13.do_unpack.e677db6d46e8c0c6109b02302aa477d0 > $ bitbake -f -c unpack gcc-runtime > [ nothing happens ] > > Worse, if I set BB_NUMBER_THREADS to some high number, I seem to end up > with multiple competing tasks all trying to unpack in parallel and the > build never makes any real forward progress. > > Does anybody have any insight into what is going wrong here and how I > should debug it? > Yes, I am seeing something similar with do_patch happening twice on the work-shared area, I talked with Richard a bit ago about it, and I am digging into the siginfo files, you can use bitbake-sigdiff to see what sigs are changed causing it unpack twice (or in my case patch twice). Sau! > thanks > > p. > > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared 2012-09-24 15:42 ` Saul Wold @ 2012-09-24 16:22 ` Phil Blundell 2012-09-24 19:28 ` Martin Jansa 0 siblings, 1 reply; 6+ messages in thread From: Phil Blundell @ 2012-09-24 16:22 UTC (permalink / raw) To: Saul Wold; +Cc: openembedded-core On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote: > Yes, I am seeing something similar with do_patch happening twice on the > work-shared area, I talked with Richard a bit ago about it, and I am > digging into the siginfo files, you can use bitbake-sigdiff to see what > sigs are changed causing it unpack twice (or in my case patch twice). Ah, right, it seems to be rm_old_work that's causing the problem. I'll disable that for now and figure out how to fix it later. Thanks p. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared 2012-09-24 16:22 ` Phil Blundell @ 2012-09-24 19:28 ` Martin Jansa 2012-10-01 11:50 ` Phil Blundell 0 siblings, 1 reply; 6+ messages in thread From: Martin Jansa @ 2012-09-24 19:28 UTC (permalink / raw) To: Phil Blundell; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 726 bytes --] On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote: > On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote: > > Yes, I am seeing something similar with do_patch happening twice on the > > work-shared area, I talked with Richard a bit ago about it, and I am > > digging into the siginfo files, you can use bitbake-sigdiff to see what > > sigs are changed causing it unpack twice (or in my case patch twice). > > Ah, right, it seems to be rm_old_work that's causing the problem. I'll > disable that for now and figure out how to fix it later. That was already discussed few weeks ago, you should be able to find it in archive. Cheers, -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 205 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared 2012-09-24 19:28 ` Martin Jansa @ 2012-10-01 11:50 ` Phil Blundell 2012-10-01 13:38 ` Richard Purdie 0 siblings, 1 reply; 6+ messages in thread From: Phil Blundell @ 2012-10-01 11:50 UTC (permalink / raw) To: Martin Jansa; +Cc: openembedded-core On Mon, 2012-09-24 at 21:28 +0200, Martin Jansa wrote: > On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote: > > On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote: > > > Yes, I am seeing something similar with do_patch happening twice on the > > > work-shared area, I talked with Richard a bit ago about it, and I am > > > digging into the siginfo files, you can use bitbake-sigdiff to see what > > > sigs are changed causing it unpack twice (or in my case patch twice). > > > > Ah, right, it seems to be rm_old_work that's causing the problem. I'll > > disable that for now and figure out how to fix it later. > > That was already discussed few weeks ago, you should be able to find it > in archive. I found this thread: http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026185.html but there didn't appear to be any clear conclusion about what the right fix for the problem was. Does anybody have any further insight into that? p. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: work-shared not being correctly shared 2012-10-01 11:50 ` Phil Blundell @ 2012-10-01 13:38 ` Richard Purdie 0 siblings, 0 replies; 6+ messages in thread From: Richard Purdie @ 2012-10-01 13:38 UTC (permalink / raw) To: Phil Blundell; +Cc: Martin Jansa, openembedded-core On Mon, 2012-10-01 at 12:50 +0100, Phil Blundell wrote: > On Mon, 2012-09-24 at 21:28 +0200, Martin Jansa wrote: > > On Mon, Sep 24, 2012 at 05:22:57PM +0100, Phil Blundell wrote: > > > On Mon, 2012-09-24 at 08:42 -0700, Saul Wold wrote: > > > > Yes, I am seeing something similar with do_patch happening twice on the > > > > work-shared area, I talked with Richard a bit ago about it, and I am > > > > digging into the siginfo files, you can use bitbake-sigdiff to see what > > > > sigs are changed causing it unpack twice (or in my case patch twice). > > > > > > Ah, right, it seems to be rm_old_work that's causing the problem. I'll > > > disable that for now and figure out how to fix it later. > > > > That was already discussed few weeks ago, you should be able to find it > > in archive. > > I found this thread: > > http://lists.linuxtogo.org/pipermail/openembedded-core/2012-July/026185.html > > but there didn't appear to be any clear conclusion about what the right > fix for the problem was. Does anybody have any further insight into > that? The issue is that rm_old_work references PN in its task definition. gcc's shared work directory assumes it can get the same sstate checksums for fetch/unpack/patch which is broken by: addtask rm_old_works before do_unpack The trick would seem to be to change do_unpack for do_configure which is how others worked around it. Not a perfect fix but equally not annoying enough to lose lots of sleep over. Cheers, Richard ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-10-01 13:51 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-09-24 15:38 work-shared not being correctly shared Phil Blundell 2012-09-24 15:42 ` Saul Wold 2012-09-24 16:22 ` Phil Blundell 2012-09-24 19:28 ` Martin Jansa 2012-10-01 11:50 ` Phil Blundell 2012-10-01 13:38 ` Richard Purdie
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox