From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: Mike Crowe <mac@mcrowe.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: populate-sysroot files in sstate cache overwritten by "empty" ones
Date: Thu, 13 Jun 2013 22:21:47 +0100 [thread overview]
Message-ID: <1371158507.20823.64.camel@ted> (raw)
In-Reply-To: <20130613111601.GA27137@mcrowe.com>
On Thu, 2013-06-13 at 12:16 +0100, Mike Crowe wrote:
> I've managed to find a relatively easy way to cause a perfectly valid
> sstate populate-sysroot file to be overwritten with a (virtually) empty
> one. The circumstances that provoke this in our layer are rather more
> complex but end up with the same situation.
>
> The problem seems to be caused by having an ancillary task in recipe1 that
> depends on do_configure and the same ancillary task in recipe2 that depends
> on recipe1:populate-sysroot. The result is that recipe1:do_configure runs
> followed by recipe1:do_populate_sysroot without the intervening important
> tasks such as do_install.
>
> To reproduce the problem, first apply this patch to oe-core to arrange for
> the required dependencies to exist:
>
> diff --git a/meta/recipes-core/mac-test/mac-test.bb b/meta/recipes-core/mac-test/mac-test.bb
> new file mode 100644
> index 0000000..48d656a
> --- /dev/null
> +++ b/meta/recipes-core/mac-test/mac-test.bb
> @@ -0,0 +1,15 @@
> +inherit module
> +LICENSE = "MIT"
> +LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=3f40d7994397109285ec7b81fdeb3b58 \
> + file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
> +DEPENDS = "virtual/kernel"
> +
> +do_extract_config() {
> +}
> +addtask extract_config after do_populate_sysroot
> +
> +do_compile() {
> +}
> +
> +do_install() {
> +}
> diff --git a/meta/recipes-kernel/linux/linux-yocto_3.8.bb b/meta/recipes-kernel/linux/linux-yocto_3.8.bb
> index 1517f40..3f14603 100644
> --- a/meta/recipes-kernel/linux/linux-yocto_3.8.bb
> +++ b/meta/recipes-kernel/linux/linux-yocto_3.8.bb
> @@ -28,3 +28,9 @@ KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
> KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
> KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
> KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"
> +
> +do_extract_config() {
> + cp ${B}/.config /tmp/somewhere-safe
> +}
> +
> +addtask extract_config after do_configure
>
>
> Now do:
>
> export BB_NUMBER_THREADS=8
> bitbake kernel-yocto mac-test
>
> # At this point kernel-yocto's populate-sysroot.tgz file in the sstate
> # cache is correct.
>
> bitbake -c clean kernel-yocto mac-test
> bitbake -c extract_config kernel-yocto mac-test
>
> # Now kernel-yocto's populate-sysroot.tgz file is virtually empty
>
>
> The log from the failing build looks like:
>
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: Running setscene task 35 of 36 (/home/mac/nobackup/git/oe-core/meta/recipes-core/mac-test/mac-test.bb, do_populate_sysroot_setscene)
> NOTE: recipe mac-test-1.0-r0: task do_populate_sysroot_setscene: Started
> NOTE: recipe mac-test-1.0-r0: task do_populate_sysroot_setscene: Succeeded
> NOTE: Running setscene task 36 of 36 (/home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_populate_sysroot_setscene)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_populate_sysroot_setscene: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_populate_sysroot_setscene: Succeeded
> NOTE: Executing RunQueue Tasks
> NOTE: Running task 175 of 269 (ID: 6, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_fetch)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_fetch: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_fetch: Succeeded
> NOTE: Running task 248 of 269 (ID: 2, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_unpack)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_unpack: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_unpack: Succeeded
> NOTE: Running task 249 of 269 (ID: 1, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_kernel_checkout)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_kernel_checkout: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_kernel_checkout: Succeeded
> NOTE: Running task 250 of 269 (ID: 0, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_validate_branches)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_validate_branches: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_validate_branches: Succeeded
> NOTE: Running task 251 of 269 (ID: 3, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_patch)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_patch: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_patch: Succeeded
> NOTE: Running task 252 of 269 (ID: 7, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_kernel_configme)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_kernel_configme: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_kernel_configme: Succeeded
> NOTE: Running task 253 of 269 (ID: 8, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_configure)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_configure: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_configure: Succeeded
> NOTE: Running task 262 of 269 (ID: 5, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_populate_sysroot)
> NOTE: Running task 263 of 269 (ID: 16, /home/mac/nobackup/git/oe-core/meta/recipes-kernel/linux/linux-yocto_3.8.bb, do_extract_config)
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_populate_sysroot: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_extract_config: Started
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_extract_config: Succeeded
> NOTE: recipe linux-yocto-3.8.13+gitedd6461602_AUTOINC+1f973c0fc8-r4.1: task do_populate_sysroot: Succeeded
> NOTE: Running task 269 of 269 (ID: 25, /home/mac/nobackup/git/oe-core/meta/recipes-core/mac-test/mac-test.bb, do_extract_config)
> NOTE: recipe mac-test-1.0-r0: task do_extract_config: Started
> NOTE: recipe mac-test-1.0-r0: task do_extract_config: Succeeded
> NOTE: Tasks Summary: Attempted 269 tasks of which 259 didn't need to be rerun and all succeeded.
Hmm, interesting.
There is some code in staging.bbclass:
BB_SETSCENE_VERIFY_FUNCTION = "sysroot_checkhashes"
where sysroot_checkhashes() is meant to notice that do_configure reruns
and hence forces do_populate_sysroot to rerun. It appears to do this
successfully, however its missing out the dependencies of
do_populate_sysroot (compile, install) which is why the package ends up
broken.
I think this is a collision of two sets of data, there are some tasks
being skipped which shouldn't be. So I'd guess that the values from
BB_SETSCENE_VERIFY_FUNCTION aren't being processed with respect to
removing things from the setscene covered list.
Cheers,
Richard
next prev parent reply other threads:[~2013-06-13 21:21 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-13 11:16 populate-sysroot files in sstate cache overwritten by "empty" ones Mike Crowe
2013-06-13 21:21 ` Richard Purdie [this message]
2013-06-14 11:19 ` Mike Crowe
2013-06-14 11:32 ` Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1371158507.20823.64.camel@ted \
--to=richard.purdie@linuxfoundation.org \
--cc=mac@mcrowe.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.