* Issue with (gcc-cross-|glibc-)initial and packaged staging @ 2010-03-09 11:40 Thomas Horsten 2010-03-09 18:42 ` Khem Raj 2010-03-09 20:07 ` Tom Rini 0 siblings, 2 replies; 5+ messages in thread From: Thomas Horsten @ 2010-03-09 11:40 UTC (permalink / raw) To: openembedded-devel I've spotted a rather annoying interim build failure but I'm not sure how to best tackle the situation. For example, gcc-cross-initial and gcc-cross install some of the same files in staging. Normally this doesn't matter, since gcc-cross overwrites the earlier ones installed by -initial, but when using packaged staging and many threads (BBTHREADS=10 for example), it sometimes happens that the pstaging packages for gcc-cross-initial and gcc-cross get installed into staging in the wrong order (or at the same time in two different threads), resulting in all builds after that failing due to undefined references in the lib files from -initial. Whether this happens or not is a timing dependent issue, so it's sometimes hard to reproduce. Any ideas how to solve this issue? I'm thinking that after gcc-cross (and glibc-cross) have been built there is no need to depend on the -initial packages any more. Another option would be to install them with different paths. Thomas ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Issue with (gcc-cross-|glibc-)initial and packaged staging 2010-03-09 11:40 Issue with (gcc-cross-|glibc-)initial and packaged staging Thomas Horsten @ 2010-03-09 18:42 ` Khem Raj 2010-03-09 20:07 ` Tom Rini 1 sibling, 0 replies; 5+ messages in thread From: Khem Raj @ 2010-03-09 18:42 UTC (permalink / raw) To: openembedded-devel On Tue, Mar 9, 2010 at 3:40 AM, Thomas Horsten <thomas.horsten@citrix.com> wrote: > I've spotted a rather annoying interim build failure but I'm not sure > how to best tackle the situation. > > For example, gcc-cross-initial and gcc-cross install some of the same > files in staging. Normally this doesn't matter, since gcc-cross > overwrites the earlier ones installed by -initial, but when using > packaged staging and many threads (BBTHREADS=10 for example), it > sometimes happens that the pstaging packages for gcc-cross-initial and > gcc-cross get installed into staging in the wrong order (or at the same > time in two different threads), resulting in all builds after that > failing due to undefined references in the lib files from -initial. toolchain build happens in a sequence due to catch 22 it has to get itself bootstrapped. In a better world when staging is always packaged then gcc-initial would create packages which would be obsoleted by gcc-cross. > > Whether this happens or not is a timing dependent issue, so it's > sometimes hard to reproduce. > > Any ideas how to solve this issue? I'm thinking that after gcc-cross > (and glibc-cross) have been built there is no need to depend on the > -initial packages any more. yes but for that you need to have the notion of toolchain is not being build from scratch and in other case when you have already bootstrapped it. This state information needs to be saved somehow obsoleting package feature in dpkg or bitbake would help in this regard when it sees that gcc-cross is already installed it would replace all gcc-cross-initial and gcc-cross-intermediate references with gcc-cross Another option would be to install them with > different paths. I think if there > > Thomas > > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Issue with (gcc-cross-|glibc-)initial and packaged staging 2010-03-09 11:40 Issue with (gcc-cross-|glibc-)initial and packaged staging Thomas Horsten 2010-03-09 18:42 ` Khem Raj @ 2010-03-09 20:07 ` Tom Rini 2010-03-09 21:05 ` Denys Dmytriyenko 1 sibling, 1 reply; 5+ messages in thread From: Tom Rini @ 2010-03-09 20:07 UTC (permalink / raw) To: openembedded-devel On Tue, 2010-03-09 at 11:40 +0000, Thomas Horsten wrote: > I've spotted a rather annoying interim build failure but I'm not sure > how to best tackle the situation. > > For example, gcc-cross-initial and gcc-cross install some of the same > files in staging. Normally this doesn't matter, since gcc-cross > overwrites the earlier ones installed by -initial, but when using > packaged staging and many threads (BBTHREADS=10 for example), it > sometimes happens that the pstaging packages for gcc-cross-initial and > gcc-cross get installed into staging in the wrong order (or at the same > time in two different threads), resulting in all builds after that > failing due to undefined references in the lib files from -initial. So, my guess is that what happens is that when we copy the stamps from the pstage package over to the live stamps directory to check them, another thread comes along and sees they're in and we get out of order. Can you try something like (pastebin'd to avoid patchwork doing something silly): http://pastebin.com/rCijESQ8 This is possibly not ideal, but as I comment there, grab the lock for staging before we copy stamps in (and release it if there's problems). But this should fix the problem, yes? If gcc-cross-initial thread is going and the gcc-cross thread fires, it'll be stuck waiting for gcc-initial to finish up fully. -- Tom Rini <tom_rini@mentor.com> Mentor Graphics Corporation ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Issue with (gcc-cross-|glibc-)initial and packaged staging 2010-03-09 20:07 ` Tom Rini @ 2010-03-09 21:05 ` Denys Dmytriyenko 2010-03-09 22:16 ` Chris Larson 0 siblings, 1 reply; 5+ messages in thread From: Denys Dmytriyenko @ 2010-03-09 21:05 UTC (permalink / raw) To: openembedded-devel On Tue, Mar 09, 2010 at 01:07:29PM -0700, Tom Rini wrote: > On Tue, 2010-03-09 at 11:40 +0000, Thomas Horsten wrote: > > I've spotted a rather annoying interim build failure but I'm not sure > > how to best tackle the situation. > > > > For example, gcc-cross-initial and gcc-cross install some of the same > > files in staging. Normally this doesn't matter, since gcc-cross > > overwrites the earlier ones installed by -initial, but when using > > packaged staging and many threads (BBTHREADS=10 for example), it > > sometimes happens that the pstaging packages for gcc-cross-initial and > > gcc-cross get installed into staging in the wrong order (or at the same > > time in two different threads), resulting in all builds after that > > failing due to undefined references in the lib files from -initial. > > So, my guess is that what happens is that when we copy the stamps from > the pstage package over to the live stamps directory to check them, > another thread comes along and sees they're in and we get out of order. > Can you try something like (pastebin'd to avoid patchwork doing > something silly): > http://pastebin.com/rCijESQ8 > > This is possibly not ideal, but as I comment there, grab the lock for > staging before we copy stamps in (and release it if there's problems). > But this should fix the problem, yes? If gcc-cross-initial thread is > going and the gcc-cross thread fires, it'll be stuck waiting for > gcc-initial to finish up fully. As I've previously seen and complained on IRC about a similar issue with many BB threads failing in packaged staging, but for different packages, I would like to try this patch as well. BTW, looking into the patch it is funny that I also tried to fix it by using SYSROOT_LOCK, but I don't think I was doing it properly. I'll see if Tom's patch fixes it... -- Denys ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Issue with (gcc-cross-|glibc-)initial and packaged staging 2010-03-09 21:05 ` Denys Dmytriyenko @ 2010-03-09 22:16 ` Chris Larson 0 siblings, 0 replies; 5+ messages in thread From: Chris Larson @ 2010-03-09 22:16 UTC (permalink / raw) To: openembedded-devel On Tue, Mar 9, 2010 at 2:05 PM, Denys Dmytriyenko <denis@denix.org> wrote: > On Tue, Mar 09, 2010 at 01:07:29PM -0700, Tom Rini wrote: > > On Tue, 2010-03-09 at 11:40 +0000, Thomas Horsten wrote: > > > I've spotted a rather annoying interim build failure but I'm not sure > > > how to best tackle the situation. > > > > > > For example, gcc-cross-initial and gcc-cross install some of the same > > > files in staging. Normally this doesn't matter, since gcc-cross > > > overwrites the earlier ones installed by -initial, but when using > > > packaged staging and many threads (BBTHREADS=10 for example), it > > > sometimes happens that the pstaging packages for gcc-cross-initial and > > > gcc-cross get installed into staging in the wrong order (or at the same > > > time in two different threads), resulting in all builds after that > > > failing due to undefined references in the lib files from -initial. > > > > So, my guess is that what happens is that when we copy the stamps from > > the pstage package over to the live stamps directory to check them, > > another thread comes along and sees they're in and we get out of order. > > Can you try something like (pastebin'd to avoid patchwork doing > > something silly): > > http://pastebin.com/rCijESQ8 > > > > This is possibly not ideal, but as I comment there, grab the lock for > > staging before we copy stamps in (and release it if there's problems). > > But this should fix the problem, yes? If gcc-cross-initial thread is > > going and the gcc-cross thread fires, it'll be stuck waiting for > > gcc-initial to finish up fully. > > As I've previously seen and complained on IRC about a similar issue with > many > BB threads failing in packaged staging, but for different packages, I would > like to try this patch as well. BTW, looking into the patch it is funny > that I > also tried to fix it by using SYSROOT_LOCK, but I don't think I was doing > it > properly. I'll see if Tom's patch fixes it... Tom's patch looks like a good idea to me, to ensure the area of code where the stamps don't match the staging contents is protected. Note that when the stamp policy is full or whitelist, this issue does not occur, since the setscene's of dependent recipes are run prior to the setscene of the recipe which depends upon them. I could see an argument for that occurring regardless of the stamp policy, but this'll do. -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-03-09 22:19 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-03-09 11:40 Issue with (gcc-cross-|glibc-)initial and packaged staging Thomas Horsten 2010-03-09 18:42 ` Khem Raj 2010-03-09 20:07 ` Tom Rini 2010-03-09 21:05 ` Denys Dmytriyenko 2010-03-09 22:16 ` Chris Larson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.