sstate.bbclass: Ensure machine specific stamps are only wiped for the current task

Openembedded Core Discussions
 help / color / mirror / Atom feed

* sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
@ 2011-10-06 14:19 Richard Purdie
  2011-10-06 20:57 ` Koen Kooi
  2011-10-09 11:25 ` Koen Kooi
  0 siblings, 2 replies; 14+ messages in thread
From: Richard Purdie @ 2011-10-06 14:19 UTC (permalink / raw)
  To: openembedded-core

sstate was being a little too ethusiastic about removing stamp files and
was removing stamp files for other machines when it shouldn't have been.

This patch teaches sstate about machine specific stamp extensions and
allows it to only remove the current task's stampfiles.

Based on a patch from Phil Blundell <philb@gnu.org> with some tweaks
from me.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index eee04ab..6abf55b 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -259,10 +259,15 @@ def sstate_clean(ss, d):
             bb.utils.unlockfile(lock)
 
     stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
+    extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info')
     oe.path.remove(stfile)
     oe.path.remove(stfile + "_setscene")
-    oe.path.remove(stfile + ".*")
-    oe.path.remove(stfile + "_setscene" + ".*")
+    if extrainf:
+        oe.path.remove(stfile + ".*" + extrainf)
+        oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
+    else:
+        oe.path.remove(stfile + ".*")
+        oe.path.remove(stfile + "_setscene" + ".*")
 
 CLEANFUNCS += "sstate_cleanall"
 





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-06 14:19 sstate.bbclass: Ensure machine specific stamps are only wiped for the current task Richard Purdie
@ 2011-10-06 20:57 ` Koen Kooi
  2011-10-09 11:25 ` Koen Kooi
  1 sibling, 0 replies; 14+ messages in thread
From: Koen Kooi @ 2011-10-06 20:57 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

let's get this in before my buildserver melts a hole in the floor :)

Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:

> sstate was being a little too ethusiastic about removing stamp files and
> was removing stamp files for other machines when it shouldn't have been.
> 
> This patch teaches sstate about machine specific stamp extensions and
> allows it to only remove the current task's stampfiles.
> 
> Based on a patch from Phil Blundell <philb@gnu.org> with some tweaks
> from me.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index eee04ab..6abf55b 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -259,10 +259,15 @@ def sstate_clean(ss, d):
>             bb.utils.unlockfile(lock)
> 
>     stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
> +    extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info')
>     oe.path.remove(stfile)
>     oe.path.remove(stfile + "_setscene")
> -    oe.path.remove(stfile + ".*")
> -    oe.path.remove(stfile + "_setscene" + ".*")
> +    if extrainf:
> +        oe.path.remove(stfile + ".*" + extrainf)
> +        oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
> +    else:
> +        oe.path.remove(stfile + ".*")
> +        oe.path.remove(stfile + "_setscene" + ".*")
> 
> CLEANFUNCS += "sstate_cleanall"
> 
> 
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core




^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-06 14:19 sstate.bbclass: Ensure machine specific stamps are only wiped for the current task Richard Purdie
  2011-10-06 20:57 ` Koen Kooi
@ 2011-10-09 11:25 ` Koen Kooi
  2011-10-09 18:10   ` Martin Jansa
  1 sibling, 1 reply; 14+ messages in thread
From: Koen Kooi @ 2011-10-09 11:25 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer


Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:

> sstate was being a little too ethusiastic about removing stamp files and
> was removing stamp files for other machines when it shouldn't have been.
> 
> This patch teaches sstate about machine specific stamp extensions and
> allows it to only remove the current task's stampfiles.

Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:

1) wipe out tmp, sstate-cache, pseudodone
2) MACHINE=beagleboard bitbake u-boot
3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
4) MACHINE=brokensstate bitbake u-boot

And behold:

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started
NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started
NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started
NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded
NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started
NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started
NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded
NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded
NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded
NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started
NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started
NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started
NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started
NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded
NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started
NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started
NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded
NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded
NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started
NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started
NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started
NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded
NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded
NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started
NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started
NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded
NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded
NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded
NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started
NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded
NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded
NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
NOTE: Executing RunQueue Tasks
NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic)
NOTE: Running task 549 of 920 (ID: 452, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/binutils/binutils-cross_2.20.1.bb, do_fetch)
NOTE: Running task 553 of 920 (ID: 504, /OE/tentacle/sources/openembedded-core/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.39.bb, do_fetch)
NOTE: Running task 558 of 920 (ID: 538, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-intermediate_4.5.bb, do_populate_lic)
NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_lic: Started
NOTE: package binutils-cross-2.20.1-r7: task do_fetch: Started
[..]
NOTE: package eglibc-2.12-r27: task do_compile: Started


What I suspect is happing is that angstrom is somehow putting $MACHINE in a var that upsets the sstate logic and/or rm_work upsets sstate and/or something else is broken. The problem is that I have no idea how to debug this except to ping Richard and get the magic diffsigs incantation.

regards,

Koen




^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-09 11:25 ` Koen Kooi
@ 2011-10-09 18:10   ` Martin Jansa
  2011-10-09 18:11     ` Koen Kooi
  0 siblings, 1 reply; 14+ messages in thread
From: Martin Jansa @ 2011-10-09 18:10 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 6216 bytes --]

On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> 
> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> 
> > sstate was being a little too ethusiastic about removing stamp files and
> > was removing stamp files for other machines when it shouldn't have been.
> > 
> > This patch teaches sstate about machine specific stamp extensions and
> > allows it to only remove the current task's stampfiles.
> 
> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> 
> 1) wipe out tmp, sstate-cache, pseudodone
> 2) MACHINE=beagleboard bitbake u-boot
> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> 4) MACHINE=brokensstate bitbake u-boot

Same here.. :/ (building 3 armv7 machines.. always "from scratch").

> And behold:
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
> NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started
> NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
> NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started
> NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded
> NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started
> NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
> NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started
> NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started
> NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded
> NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded
> NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded
> NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
> NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
> NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started
> NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
> NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started
> NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded
> NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started
> NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started
> NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded
> NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded
> NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started
> NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started
> NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded
> NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
> NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started
> NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started
> NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started
> NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded
> NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded
> NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started
> NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started
> NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded
> NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started
> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
> NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded
> NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded
> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
> NOTE: Executing RunQueue Tasks
> NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic)
> NOTE: Running task 549 of 920 (ID: 452, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/binutils/binutils-cross_2.20.1.bb, do_fetch)
> NOTE: Running task 553 of 920 (ID: 504, /OE/tentacle/sources/openembedded-core/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.39.bb, do_fetch)
> NOTE: Running task 558 of 920 (ID: 538, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-intermediate_4.5.bb, do_populate_lic)
> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_lic: Started
> NOTE: package binutils-cross-2.20.1-r7: task do_fetch: Started
> [..]
> NOTE: package eglibc-2.12-r27: task do_compile: Started
> 
> 
> What I suspect is happing is that angstrom is somehow putting $MACHINE in a var that upsets the sstate logic and/or rm_work upsets sstate and/or something else is broken. The problem is that I have no idea how to debug this except to ping Richard and get the magic diffsigs incantation.
> 
> regards,
> 
> Koen
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-09 18:10   ` Martin Jansa
@ 2011-10-09 18:11     ` Koen Kooi
  2011-10-10 11:45       ` Martin Jansa
  0 siblings, 1 reply; 14+ messages in thread
From: Koen Kooi @ 2011-10-09 18:11 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer


Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:

> On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
>> 
>> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
>> 
>>> sstate was being a little too ethusiastic about removing stamp files and
>>> was removing stamp files for other machines when it shouldn't have been.
>>> 
>>> This patch teaches sstate about machine specific stamp extensions and
>>> allows it to only remove the current task's stampfiles.
>> 
>> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
>> 
>> 1) wipe out tmp, sstate-cache, pseudodone
>> 2) MACHINE=beagleboard bitbake u-boot
>> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
>> 4) MACHINE=brokensstate bitbake u-boot
> 
> Same here.. :/ (building 3 armv7 machines.. always "from scratch").

To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?

> 
>> And behold:
>> 
>> NOTE: Resolving any missing task queue dependencies
>> NOTE: Preparing runqueue
>> NOTE: Executing SetScene Tasks
>> NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
>> NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Started
>> NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
>> NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Started
>> NOTE: package mtd-utils-1.4.6-r0: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Started
>> NOTE: package bash-4.1-r2: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gcc-cross-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Started
>> NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Started
>> NOTE: package eglibc-2.12-r27: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package binutils-cross-2.20.1-r7: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package linux-libc-headers-2.6.39-r2: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
>> NOTE: package gcc-runtime-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
>> NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Started
>> NOTE: package libgcc-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Started
>> NOTE: package zlib-1.2.5-r0: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Started
>> NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Started
>> NOTE: package lzo-2.05-r0: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package e2fsprogs-1.41.14-r2: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Started
>> NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Started
>> NOTE: package libtool-cross-2.4-r4: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package util-linux-2.19.1-r7: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gcc-cross-intermediate-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Started
>> NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Started
>> NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Started
>> NOTE: package eglibc-initial-2.12-r27: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package ncurses-5.9-r1.1: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Started
>> NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Started
>> NOTE: package expat-2.0.1-r0: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package perl-5.12.3-r5: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gettext-0.18.1.1-r4: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Started
>> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Started
>> NOTE: package gdbm-1.8.3-r4: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package db-5.1.19-r3: task do_populate_sysroot_setscene: Succeeded
>> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_sysroot_setscene: Succeeded
>> NOTE: Executing RunQueue Tasks
>> NOTE: Running task 536 of 920 (ID: 918, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-initial_4.5.bb, do_populate_lic)
>> NOTE: Running task 549 of 920 (ID: 452, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/binutils/binutils-cross_2.20.1.bb, do_fetch)
>> NOTE: Running task 553 of 920 (ID: 504, /OE/tentacle/sources/openembedded-core/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_2.6.39.bb, do_fetch)
>> NOTE: Running task 558 of 920 (ID: 538, /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-devtools/gcc/gcc-cross-intermediate_4.5.bb, do_populate_lic)
>> NOTE: package gcc-cross-initial-4.5-r43+svnr178923: task do_populate_lic: Started
>> NOTE: package binutils-cross-2.20.1-r7: task do_fetch: Started
>> [..]
>> NOTE: package eglibc-2.12-r27: task do_compile: Started
>> 
>> 
>> What I suspect is happing is that angstrom is somehow putting $MACHINE in a var that upsets the sstate logic and/or rm_work upsets sstate and/or something else is broken. The problem is that I have no idea how to debug this except to ping Richard and get the magic diffsigs incantation.
>> 
>> regards,
>> 
>> Koen
>> 
>> 
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
> -- 
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core




^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-09 18:11     ` Koen Kooi
@ 2011-10-10 11:45       ` Martin Jansa
  2011-10-10 14:16         ` Richard Purdie
  2011-10-10 14:17         ` Richard Purdie
  0 siblings, 2 replies; 14+ messages in thread
From: Martin Jansa @ 2011-10-10 11:45 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 2465 bytes --]

On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> 
> Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> 
> > On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> >> 
> >> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> >> 
> >>> sstate was being a little too ethusiastic about removing stamp files and
> >>> was removing stamp files for other machines when it shouldn't have been.
> >>> 
> >>> This patch teaches sstate about machine specific stamp extensions and
> >>> allows it to only remove the current task's stampfiles.
> >> 
> >> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> >> 
> >> 1) wipe out tmp, sstate-cache, pseudodone
> >> 2) MACHINE=beagleboard bitbake u-boot
> >> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> >> 4) MACHINE=brokensstate bitbake u-boot
> > 
> > Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> 
> To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?

DISTRO=shr

MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok

without wiping anything (same tmp, same sstate-cache, same pseudodone)

MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok

palmpre and palmpre2 are "the same"
$ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
require conf/machine/include/palmpre.inc

$ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
require conf/machine/include/palmpre.inc

nokia900 is also armv7a-vfp-neon (that's why all packages except
MACHINE_ARCH were always reused without rebuilding in OE-classic).

I have tried to debug sigdata files to see why and also tried to add
MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
as RP suggested, but sofar haven't found solution at least to reuse
already built sstate packages for armv7a-vfp-neon.

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 11:45       ` Martin Jansa
@ 2011-10-10 14:16         ` Richard Purdie
  2011-10-10 14:37           ` Koen Kooi
  2011-10-10 14:17         ` Richard Purdie
  1 sibling, 1 reply; 14+ messages in thread
From: Richard Purdie @ 2011-10-10 14:16 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
> On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> > 
> > Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> > 
> > > On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> > >> 
> > >> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> > >> 
> > >>> sstate was being a little too ethusiastic about removing stamp files and
> > >>> was removing stamp files for other machines when it shouldn't have been.
> > >>> 
> > >>> This patch teaches sstate about machine specific stamp extensions and
> > >>> allows it to only remove the current task's stampfiles.
> > >> 
> > >> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> > >> 
> > >> 1) wipe out tmp, sstate-cache, pseudodone
> > >> 2) MACHINE=beagleboard bitbake u-boot
> > >> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> > >> 4) MACHINE=brokensstate bitbake u-boot
> > > 
> > > Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> > 
> > To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
> 
> DISTRO=shr
> 
> MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
> MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
> 
> without wiping anything (same tmp, same sstate-cache, same pseudodone)
> 
> MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
> MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
> 
> palmpre and palmpre2 are "the same"
> $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
> require conf/machine/include/palmpre.inc
> 
> $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
> require conf/machine/include/palmpre.inc
> 
> nokia900 is also armv7a-vfp-neon (that's why all packages except
> MACHINE_ARCH were always reused without rebuilding in OE-classic).
> 
> I have tried to debug sigdata files to see why and also tried to add
> MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
> as RP suggested, but sofar haven't found solution at least to reuse
> already built sstate packages for armv7a-vfp-neon.

Also as a tip for debugging this, you can do what is in the following
commit (patch also inline below):

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9&id=1261cc69b152f33b934c2448020dfa3c95f312c6

which will mean bitbake -n generates stamp files for a given bitbake
run. You can then quickly do:

MACHINE=A bitbake -n foo
MACHINE=B bitbake -n foo

for example and then look for cases where two stamp sigdata files were
generated. If you see them you can "bitbake-diffsigs A B" and I'll be
very interested in the results of that and "bitbake-diffsigs A"...

Cheers,

Richard

bitbake: Allow easier dry run stamp debugging

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
index 8937f08..efa9ffe 100644
--- a/bitbake/lib/bb/build.py
+++ b/bitbake/lib/bb/build.py
@@ -255,7 +255,7 @@ def _task_data(fn, task, d):
     data.expandKeys(localdata)
     return localdata
 
-def _exec_task(fn, task, d, quieterr):
+def _exec_task(fn, task, d, quieterr, dryrun=False):
     """Execute a BB 'task'
 
     Execution of a task involves a bit more setup than executing a function,
@@ -317,10 +317,13 @@ def _exec_task(fn, task, d, quieterr):
     event.fire(TaskStarted(task, localdata), localdata)
     try:
         for func in (prefuncs or '').split():
-            exec_func(func, localdata)
-        exec_func(task, localdata)
+            if not dryrun:
+                exec_func(func, localdata)
+        if not dryrun:
+            exec_func(task, localdata)
         for func in (postfuncs or '').split():
-            exec_func(func, localdata)
+            if not dryrun:
+                exec_func(func, localdata)
     except FuncFailed as exc:
         if not quieterr:
             logger.error(str(exc))
@@ -355,13 +358,13 @@ def _exec_task(fn, task, d, quieterr):
 
     return 0
 
-def exec_task(fn, task, d):
+def exec_task(fn, task, d, dryrun=False):
     try: 
         quieterr = False
         if d.getVarFlag(task, "quieterrors") is not None:
             quieterr = True
 
-        return _exec_task(fn, task, d, quieterr)
+        return _exec_task(fn, task, d, quieterr, dryrun)
     except Exception:
         from traceback import format_exc
         if not quieterr:
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index aca06b5..e53da56 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1156,8 +1156,7 @@ class RunQueueExecute:
                     logger.critical(str(exc))
                 os._exit(1)
             try:
-                if not self.cooker.configuration.dry_run:
-                    ret = bb.build.exec_task(fn, taskname, the_data)
+                ret = bb.build.exec_task(fn, taskname, the_data, self.cooker.configuration.dry_run)
                 os._exit(ret)
             except:
                 os._exit(1)







^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 14:16         ` Richard Purdie
@ 2011-10-10 14:37           ` Koen Kooi
  2011-10-10 14:41             ` Martin Jansa
  0 siblings, 1 reply; 14+ messages in thread
From: Koen Kooi @ 2011-10-10 14:37 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer


Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven:

> On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
>> On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
>>> 
>>> Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
>>> 
>>>> On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
>>>>> 
>>>>> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
>>>>> 
>>>>>> sstate was being a little too ethusiastic about removing stamp files and
>>>>>> was removing stamp files for other machines when it shouldn't have been.
>>>>>> 
>>>>>> This patch teaches sstate about machine specific stamp extensions and
>>>>>> allows it to only remove the current task's stampfiles.
>>>>> 
>>>>> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
>>>>> 
>>>>> 1) wipe out tmp, sstate-cache, pseudodone
>>>>> 2) MACHINE=beagleboard bitbake u-boot
>>>>> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
>>>>> 4) MACHINE=brokensstate bitbake u-boot
>>>> 
>>>> Same here.. :/ (building 3 armv7 machines.. always "from scratch").
>>> 
>>> To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
>> 
>> DISTRO=shr
>> 
>> MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
>> MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
>> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
>> 
>> without wiping anything (same tmp, same sstate-cache, same pseudodone)
>> 
>> MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
>> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
>> MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
>> 
>> palmpre and palmpre2 are "the same"
>> $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
>> require conf/machine/include/palmpre.inc
>> 
>> $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
>> require conf/machine/include/palmpre.inc
>> 
>> nokia900 is also armv7a-vfp-neon (that's why all packages except
>> MACHINE_ARCH were always reused without rebuilding in OE-classic).
>> 
>> I have tried to debug sigdata files to see why and also tried to add
>> MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
>> as RP suggested, but sofar haven't found solution at least to reuse
>> already built sstate packages for armv7a-vfp-neon.
> 
> Also as a tip for debugging this, you can do what is in the following
> commit (patch also inline below):
> 
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9&id=1261cc69b152f33b934c2448020dfa3c95f312c6
> 
> which will mean bitbake -n generates stamp files for a given bitbake
> run. You can then quickly do:
> 
> MACHINE=A bitbake -n foo
> MACHINE=B bitbake -n foo
> 
> for example and then look for cases where two stamp sigdata files were
> generated. If you see them you can "bitbake-diffsigs A B" and I'll be
> very interested in the results of that and "bitbake-diffsigs A"...
> 
> Cheers,
> 
> Richard
> 
> bitbake: Allow easier dry run stamp debugging
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py

This seems to be against poky instead of bitbake and doesn't apply against bitbake master:

koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 < dry 
patching file lib/bb/build.py
Hunk #1 succeeded at 257 (offset 2 lines).
Hunk #2 succeeded at 319 (offset 2 lines).
Hunk #3 succeeded at 360 (offset 2 lines).
patching file lib/bb/runqueue.py
Hunk #1 FAILED at 1156.
1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej




^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 14:37           ` Koen Kooi
@ 2011-10-10 14:41             ` Martin Jansa
  2011-10-10 15:17               ` Richard Purdie
  0 siblings, 1 reply; 14+ messages in thread
From: Martin Jansa @ 2011-10-10 14:41 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 5153 bytes --]

On Mon, Oct 10, 2011 at 04:37:51PM +0200, Koen Kooi wrote:
> 
> Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven:
> 
> > On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
> >> On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> >>> 
> >>> Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> >>> 
> >>>> On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> >>>>> 
> >>>>> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> >>>>> 
> >>>>>> sstate was being a little too ethusiastic about removing stamp files and
> >>>>>> was removing stamp files for other machines when it shouldn't have been.
> >>>>>> 
> >>>>>> This patch teaches sstate about machine specific stamp extensions and
> >>>>>> allows it to only remove the current task's stampfiles.
> >>>>> 
> >>>>> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> >>>>> 
> >>>>> 1) wipe out tmp, sstate-cache, pseudodone
> >>>>> 2) MACHINE=beagleboard bitbake u-boot
> >>>>> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> >>>>> 4) MACHINE=brokensstate bitbake u-boot
> >>>> 
> >>>> Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> >>> 
> >>> To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
> >> 
> >> DISTRO=shr
> >> 
> >> MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
> >> MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
> >> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
> >> 
> >> without wiping anything (same tmp, same sstate-cache, same pseudodone)
> >> 
> >> MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
> >> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
> >> MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
> >> 
> >> palmpre and palmpre2 are "the same"
> >> $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
> >> require conf/machine/include/palmpre.inc
> >> 
> >> $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
> >> require conf/machine/include/palmpre.inc
> >> 
> >> nokia900 is also armv7a-vfp-neon (that's why all packages except
> >> MACHINE_ARCH were always reused without rebuilding in OE-classic).
> >> 
> >> I have tried to debug sigdata files to see why and also tried to add
> >> MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
> >> as RP suggested, but sofar haven't found solution at least to reuse
> >> already built sstate packages for armv7a-vfp-neon.
> > 
> > Also as a tip for debugging this, you can do what is in the following
> > commit (patch also inline below):
> > 
> > http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9&id=1261cc69b152f33b934c2448020dfa3c95f312c6
> > 
> > which will mean bitbake -n generates stamp files for a given bitbake
> > run. You can then quickly do:
> > 
> > MACHINE=A bitbake -n foo
> > MACHINE=B bitbake -n foo
> > 
> > for example and then look for cases where two stamp sigdata files were
> > generated. If you see them you can "bitbake-diffsigs A B" and I'll be
> > very interested in the results of that and "bitbake-diffsigs A"...
> > 
> > Cheers,
> > 
> > Richard
> > 
> > bitbake: Allow easier dry run stamp debugging
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
> 
> This seems to be against poky instead of bitbake and doesn't apply against bitbake master:
> 
> koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 < dry 
> patching file lib/bb/build.py
> Hunk #1 succeeded at 257 (offset 2 lines).
> Hunk #2 succeeded at 319 (offset 2 lines).
> Hunk #3 succeeded at 360 (offset 2 lines).
> patching file lib/bb/runqueue.py
> Hunk #1 FAILED at 1156.
> 1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej

Same problem here.. seems like diff is bigger then it was before:
both current master

OE @ ~/projects $ diff -rq poky/bitbake/ bitbake/ | grep -v pyc | grep -v pyo | grep -v .git
Only in bitbake/: MANIFEST.in
Only in bitbake/: TODO
Only in poky/bitbake/bin: bitbake-runtask
Only in bitbake/: classes
Only in bitbake/: conf
Files poky/bitbake/lib/bb/__init__.py and bitbake/lib/bb/__init__.py differ
Files poky/bitbake/lib/bb/build.py and bitbake/lib/bb/build.py differ
Files poky/bitbake/lib/bb/cooker.py and bitbake/lib/bb/cooker.py differ
Files poky/bitbake/lib/bb/parse/parse_py/BBHandler.py and bitbake/lib/bb/parse/parse_py/BBHandler.py differ
Files poky/bitbake/lib/bb/pysh/pyshtables.py and bitbake/lib/bb/pysh/pyshtables.py differ
Files poky/bitbake/lib/bb/runqueue.py and bitbake/lib/bb/runqueue.py differ
Only in poky/bitbake/lib/bb: shell.py
Only in bitbake/: setup.py

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 14:41             ` Martin Jansa
@ 2011-10-10 15:17               ` Richard Purdie
  0 siblings, 0 replies; 14+ messages in thread
From: Richard Purdie @ 2011-10-10 15:17 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

On Mon, 2011-10-10 at 16:41 +0200, Martin Jansa wrote:
> On Mon, Oct 10, 2011 at 04:37:51PM +0200, Koen Kooi wrote:
> > 
> > Op 10 okt. 2011, om 16:16 heeft Richard Purdie het volgende geschreven:
> > 
> > > On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
> > >> On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> > >>> 
> > >>> Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> > >>> 
> > >>>> On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> > >>>>> 
> > >>>>> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> > >>>>> 
> > >>>>>> sstate was being a little too ethusiastic about removing stamp files and
> > >>>>>> was removing stamp files for other machines when it shouldn't have been.
> > >>>>>> 
> > >>>>>> This patch teaches sstate about machine specific stamp extensions and
> > >>>>>> allows it to only remove the current task's stampfiles.
> > >>>>> 
> > >>>>> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> > >>>>> 
> > >>>>> 1) wipe out tmp, sstate-cache, pseudodone
> > >>>>> 2) MACHINE=beagleboard bitbake u-boot
> > >>>>> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> > >>>>> 4) MACHINE=brokensstate bitbake u-boot
> > >>>> 
> > >>>> Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> > >>> 
> > >>> To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
> > >> 
> > >> DISTRO=shr
> > >> 
> > >> MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
> > >> MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
> > >> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
> > >> 
> > >> without wiping anything (same tmp, same sstate-cache, same pseudodone)
> > >> 
> > >> MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
> > >> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
> > >> MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
> > >> 
> > >> palmpre and palmpre2 are "the same"
> > >> $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
> > >> require conf/machine/include/palmpre.inc
> > >> 
> > >> $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
> > >> require conf/machine/include/palmpre.inc
> > >> 
> > >> nokia900 is also armv7a-vfp-neon (that's why all packages except
> > >> MACHINE_ARCH were always reused without rebuilding in OE-classic).
> > >> 
> > >> I have tried to debug sigdata files to see why and also tried to add
> > >> MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
> > >> as RP suggested, but sofar haven't found solution at least to reuse
> > >> already built sstate packages for armv7a-vfp-neon.
> > > 
> > > Also as a tip for debugging this, you can do what is in the following
> > > commit (patch also inline below):
> > > 
> > > http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/wip9&id=1261cc69b152f33b934c2448020dfa3c95f312c6
> > > 
> > > which will mean bitbake -n generates stamp files for a given bitbake
> > > run. You can then quickly do:
> > > 
> > > MACHINE=A bitbake -n foo
> > > MACHINE=B bitbake -n foo
> > > 
> > > for example and then look for cases where two stamp sigdata files were
> > > generated. If you see them you can "bitbake-diffsigs A B" and I'll be
> > > very interested in the results of that and "bitbake-diffsigs A"...
> > > 
> > > Cheers,
> > > 
> > > Richard
> > > 
> > > bitbake: Allow easier dry run stamp debugging
> > > 
> > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > > ---
> > > diff --git a/bitbake/lib/bb/build.py b/bitbake/lib/bb/build.py
> > 
> > This seems to be against poky instead of bitbake and doesn't apply against bitbake master:
> > 
> > koen@dominion:/OE/tentacle/sources/bitbake$ patch -p2 < dry 
> > patching file lib/bb/build.py
> > Hunk #1 succeeded at 257 (offset 2 lines).
> > Hunk #2 succeeded at 319 (offset 2 lines).
> > Hunk #3 succeeded at 360 (offset 2 lines).
> > patching file lib/bb/runqueue.py
> > Hunk #1 FAILED at 1156.
> > 1 out of 1 hunk FAILED -- saving rejects to file lib/bb/runqueue.py.rej
> 
> Same problem here.. seems like diff is bigger then it was before:
> both current master
> 
> OE @ ~/projects $ diff -rq poky/bitbake/ bitbake/ | grep -v pyc | grep -v pyo | grep -v .git
> Only in bitbake/: MANIFEST.in
> Only in bitbake/: TODO
> Only in bitbake/: classes
> Only in bitbake/: conf
>Only in bitbake/: setup.py

These are deliberate and don't make sense in poky.

>Only in poky/bitbake/bin: bitbake-runtask
>Files poky/bitbake/lib/bb/cooker.py and bitbake/lib/bb/cooker.py differ

These are related and I need to get this fixed, there is an open bug
about this.

> Files poky/bitbake/lib/bb/__init__.py and bitbake/lib/bb/__init__.py differ

Not showing up here for me...

> Files poky/bitbake/lib/bb/parse/parse_py/BBHandler.py and bitbake/lib/bb/parse/parse_py/BBHandler.py differ

This is whitespace only.

> Files poky/bitbake/lib/bb/pysh/pyshtables.py and bitbake/lib/bb/pysh/pyshtables.py differ

This is a generated file and in neither repository.

> Only in poky/bitbake/lib/bb: shell.py

This is a reminder that we'd like a bitbake shell to come back.

>Files poky/bitbake/lib/bb/runqueue.py and bitbake/lib/bb/runqueue.py differ
>Files poky/bitbake/lib/bb/build.py and bitbake/lib/bb/build.py differ

and these are the main "problem". Some is related to the bitbake-runtask
piece above, there were also a couple of patches I didn't like to
bitbake and didn't take into poky.

There isn't anything particularly major here, just reminders of areas
that could do with some dedicated cleanup or rethinking. Bitbake master
takes the short cut of skipping tasks during dry run without forking. My
patch assumes that the fork is made to generate the stamps. An updated
patch is below.

Cheers,

Richard


diff --git a/lib/bb/build.py b/lib/bb/build.py
index 70cc809..5ae8ba4 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -257,7 +257,7 @@ def _task_data(fn, task, d):
     data.expandKeys(localdata)
     return localdata
 
-def _exec_task(fn, task, d, quieterr):
+def _exec_task(fn, task, d, quieterr, dryrun=False):
     """Execute a BB 'task'
 
     Execution of a task involves a bit more setup than executing a function,
@@ -319,10 +319,13 @@ def _exec_task(fn, task, d, quieterr):
     event.fire(TaskStarted(task, localdata), localdata)
     try:
         for func in (prefuncs or '').split():
-            exec_func(func, localdata)
-        exec_func(task, localdata)
+            if not dryrun:
+                exec_func(func, localdata)
+        if not dryrun:
+            exec_func(task, localdata)
         for func in (postfuncs or '').split():
-            exec_func(func, localdata)
+            if not dryrun:
+                exec_func(func, localdata)
     except FuncFailed as exc:
         if not quieterr:
             logger.error(str(exc))
@@ -357,13 +360,13 @@ def _exec_task(fn, task, d, quieterr):
 
     return 0
 
-def exec_task(fn, task, d):
+def exec_task(fn, task, d, dryrun=False):
     try: 
         quieterr = False
         if d.getVarFlag(task, "quieterrors") is not None:
             quieterr = True
 
-        return _exec_task(fn, task, d, quieterr)
+        return _exec_task(fn, task, d, quieterr, dryrun)
     except Exception:
         from traceback import format_exc
         if not quieterr:
diff --git a/lib/bb/runqueue.py b/lib/bb/runqueue.py
index 9a1bad7..61aba1b 100644
--- a/lib/bb/runqueue.py
+++ b/lib/bb/runqueue.py
@@ -1156,7 +1156,7 @@ class RunQueueExecute:
                     logger.critical(str(exc))
                 os._exit(1)
             try:
-                ret = bb.build.exec_task(fn, taskname, the_data)
+                ret = bb.build.exec_task(fn, taskname, the_data, self.cooker.configuration.dry_run)
                 os._exit(ret)
             except:
                 os._exit(1)
@@ -1329,12 +1329,6 @@ class RunQueueExecuteTasks(RunQueueExecute):
                                 self.rqdata.get_user_idstring(task))
                 self.task_skip(task)
                 return True
-            elif self.cooker.configuration.dry_run:
-                self.runq_running[task] = 1
-                self.runq_buildable[task] = 1
-                self.stats.taskActive()
-                self.task_complete(task)
-                return True
 
             taskdep = self.rqdata.dataCache.task_deps[fn]
             if 'noexec' in taskdep and taskname in taskdep['noexec']:





^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 11:45       ` Martin Jansa
  2011-10-10 14:16         ` Richard Purdie
@ 2011-10-10 14:17         ` Richard Purdie
  2011-11-03 16:33           ` Martin Jansa
  1 sibling, 1 reply; 14+ messages in thread
From: Richard Purdie @ 2011-10-10 14:17 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
> On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> > 
> > Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> > 
> > > On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> > >> 
> > >> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> > >> 
> > >>> sstate was being a little too ethusiastic about removing stamp files and
> > >>> was removing stamp files for other machines when it shouldn't have been.
> > >>> 
> > >>> This patch teaches sstate about machine specific stamp extensions and
> > >>> allows it to only remove the current task's stampfiles.
> > >> 
> > >> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> > >> 
> > >> 1) wipe out tmp, sstate-cache, pseudodone
> > >> 2) MACHINE=beagleboard bitbake u-boot
> > >> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> > >> 4) MACHINE=brokensstate bitbake u-boot
> > > 
> > > Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> > 
> > To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
> 
> DISTRO=shr
> 
> MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
> MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
> 
> without wiping anything (same tmp, same sstate-cache, same pseudodone)
> 
> MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
> MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
> MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
> 
> palmpre and palmpre2 are "the same"
> $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
> require conf/machine/include/palmpre.inc
> 
> $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
> require conf/machine/include/palmpre.inc
> 
> nokia900 is also armv7a-vfp-neon (that's why all packages except
> MACHINE_ARCH were always reused without rebuilding in OE-classic).
> 
> I have tried to debug sigdata files to see why and also tried to add
> MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
> as RP suggested, but sofar haven't found solution at least to reuse
> already built sstate packages for armv7a-vfp-neon.

I've posted a patch which at least locally solves the issue with the "cp
MACHINEA MACHINEB". If there are more issues I'd be interested, see my
separate email with some debugging tips.

Cheers,

Richard





^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-10-10 14:17         ` Richard Purdie
@ 2011-11-03 16:33           ` Martin Jansa
  2011-11-03 22:08             ` Martin Jansa
  0 siblings, 1 reply; 14+ messages in thread
From: Martin Jansa @ 2011-11-03 16:33 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 4450 bytes --]

On Mon, Oct 10, 2011 at 03:17:51PM +0100, Richard Purdie wrote:
> On Mon, 2011-10-10 at 13:45 +0200, Martin Jansa wrote:
> > On Sun, Oct 09, 2011 at 08:11:45PM +0200, Koen Kooi wrote:
> > > 
> > > Op 9 okt. 2011, om 20:10 heeft Martin Jansa het volgende geschreven:
> > > 
> > > > On Sun, Oct 09, 2011 at 01:25:09PM +0200, Koen Kooi wrote:
> > > >> 
> > > >> Op 6 okt. 2011, om 16:19 heeft Richard Purdie het volgende geschreven:
> > > >> 
> > > >>> sstate was being a little too ethusiastic about removing stamp files and
> > > >>> was removing stamp files for other machines when it shouldn't have been.
> > > >>> 
> > > >>> This patch teaches sstate about machine specific stamp extensions and
> > > >>> allows it to only remove the current task's stampfiles.
> > > >> 
> > > >> Not sure if it's related to this problem or not but sstate is still not working as intented for me when switching machines, it keeps rebuilding the toolchain when switching machine. What I did this morning:
> > > >> 
> > > >> 1) wipe out tmp, sstate-cache, pseudodone
> > > >> 2) MACHINE=beagleboard bitbake u-boot
> > > >> 3) cp conf/machine/beagleboard.conf conf/machine/brokensstate.conf
> > > >> 4) MACHINE=brokensstate bitbake u-boot
> > > > 
> > > > Same here.. :/ (building 3 armv7 machines.. always "from scratch").
> > > 
> > > To eliminate varables: Angstrom/shr/aurora/micro/other/all of the above?
> > 
> > DISTRO=shr
> > 
> > MACHINE=nokia900 bitbake shr-image -> builds from scratch, finished ok
> > MACHINE=palmpre bitbake shr-image -> builds from scratch, finished ok
> > MACHINE=palmpre2 bitbake shr-image -> builds from scratch, finished ok
> > 
> > without wiping anything (same tmp, same sstate-cache, same pseudodone)
> > 
> > MACHINE=palmpre bitbake shr-image -> builds from scratch, again!, finished ok
> > MACHINE=palmpre2 bitbake shr-image -> builds from scratch, again!, finished ok
> > MACHINE=nokia900 bitbake shr-image -> builds from scratch, again!, finished ok
> > 
> > palmpre and palmpre2 are "the same"
> > $ cat meta-smartphone/meta-palm/conf/machine/palmpre.conf | grep -v ^#
> > require conf/machine/include/palmpre.inc
> > 
> > $ cat meta-smartphone/meta-palm/conf/machine/palmpre2.conf | grep -v ^#
> > require conf/machine/include/palmpre.inc
> > 
> > nokia900 is also armv7a-vfp-neon (that's why all packages except
> > MACHINE_ARCH were always reused without rebuilding in OE-classic).
> > 
> > I have tried to debug sigdata files to see why and also tried to add
> > MACHINEOVERRIDES[vardepsexclude] = "MACHINE"
> > as RP suggested, but sofar haven't found solution at least to reuse
> > already built sstate packages for armv7a-vfp-neon.
> 
> I've posted a patch which at least locally solves the issue with the "cp
> MACHINEA MACHINEB". If there are more issues I'd be interested, see my
> separate email with some debugging tips.

Hi,

to recap what we have talked about on ELCE;

Currently it doesn't seem to change sstate checksums after switching
MACHINE, but still it rebuilds everything at least once after switch.

Today I've tried to wipe tmp and rebuild from sstate-cache and I've
noticed interesting thing:

All recipes are starting with 
do_populate_lic_setscene 
not 
do_populate_sysroot_setscene
like ie koen's build does

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started
NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started
NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded
NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded

Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it 
causes the rebuild instead of population of sysroot from sstate?

IIRC I had similar issue when I was starting to use oe-core and it was 
in the end caused by inheriting package_ipk (once directly from distro config 
and once through sane-feed-ipk.inc) and it was fixed 6 months ago by:
http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a
And IIRC we've talked about some ways to make sstate less error-prone
for such things.

Now I have similar symptoms but only after switching MACHINE.

Any hint?

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-11-03 16:33           ` Martin Jansa
@ 2011-11-03 22:08             ` Martin Jansa
  2011-11-04  9:32               ` Martin Jansa
  0 siblings, 1 reply; 14+ messages in thread
From: Martin Jansa @ 2011-11-03 22:08 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 26435 bytes --]

On Thu, Nov 03, 2011 at 05:33:28PM +0100, Martin Jansa wrote:
> Hi,
> 
> to recap what we have talked about on ELCE;
> 
> Currently it doesn't seem to change sstate checksums after switching
> MACHINE, but still it rebuilds everything at least once after switch.
> 
> Today I've tried to wipe tmp and rebuild from sstate-cache and I've
> noticed interesting thing:
> 
> All recipes are starting with 
> do_populate_lic_setscene 
> not 
> do_populate_sysroot_setscene
> like ie koen's build does
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started
> NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started
> NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded
> NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded
> 
> Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it 
> causes the rebuild instead of population of sysroot from sstate?
> 
> IIRC I had similar issue when I was starting to use oe-core and it was 
> in the end caused by inheriting package_ipk (once directly from distro config 
> and once through sane-feed-ipk.inc) and it was fixed 6 months ago by:
> http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a
> And IIRC we've talked about some ways to make sstate less error-prone
> for such things.
> 
> Now I have similar symptoms but only after switching MACHINE.
> 
> Any hint?

With few more hours of debuging this.. I can say that sometimes it works, 
but sometimes it's too strict to be usefull anyway.

Whole logs are here:
http://build.shr-project.org/tests/jama/sstate.test
here I'll try to shorten as much as possible.

OE @shr ~/shr-core $ export MACHINE=qemuarm
OE qemuarm@shr ~/shr-core $ bitbake -k core-image-core
.....
NOTE: package core-image-core-1.0-r0: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 5306 tasks of which 5250 didn't need to be rerun and 0 failed.
OE qemuarm@shr ~/shr-core $ bitbake -k expat
Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all)
NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started
NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed.

All done

OE qemuarmB@shr ~/shr-core $ cp openembedded-core/meta/conf/machine/qemuarm.conf openembedded-core/meta/conf/machine/qemuarmcopy.conf
OE qemuarmB@shr ~/shr-core $ export MACHINE=qemuarmcopy
OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
..
libproxy fail (around task 2232)
fix

OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
libproxy built
NOTE: Tasks Summary: Attempted 2428 tasks of which 2232 didn't need to be rerun and 0 failed.

So after 3 hours we have qemuarm and qemuarmcopy (identical machine) with expat built! hurray
But seems like nothing was reused from qemuarm build.

once again for check

OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm
OE qemuarm@shr ~/shr-core $ bitbake -k expat
Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
...
NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.

OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy
OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
...
NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.

great

OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* | sed 's/.*bitbake//g'
  460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk
 1.2K Oct 30 00:54 manifest-qemuarm-expat.package
  138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic
  732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot
  460 Nov  3 19:50 manifest-qemuarmcopy-expat.deploy-ipk
 1.2K Nov  3 19:50 manifest-qemuarmcopy-expat.package
  138 Nov  3 19:17 manifest-qemuarmcopy-expat.populate-lic
  788 Nov  3 19:50 manifest-qemuarmcopy-expat.populate-sysroot
  460 Oct 25 15:35 manifest-spitz-expat.deploy-ipk
 1.2K Oct 25 15:35 manifest-spitz-expat.package
  138 Oct 25 15:35 manifest-spitz-expat.populate-lic
  704 Oct 25 15:35 manifest-spitz-expat.populate-sysroot

OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
    926 Nov  3 19:17 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
   8444 Nov  3 19:17 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
  12409 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
  12419 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-757ebe15c6afc27935bb3f3b823f1a12_deploy-ipk.tgz.siginfo
  12419 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz.siginfo
  21852 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
  21920 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-d813b67ed935225757b90d014f631652_populate-sysroot.tgz.siginfo
  21927 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e90bb07db6e100f14ee8f8eed99a7d7e_populate-sysroot.tgz.siginfo
  91310 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-f22b72a744d826703a553960f3402bc9_package.tgz.siginfo
  91317 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-634056338ea09623cd84437f710e04d5_package.tgz.siginfo
  91826 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
 339323 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-d813b67ed935225757b90d014f631652_populate-sysroot.tgz
 339468 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e90bb07db6e100f14ee8f8eed99a7d7e_populate-sysroot.tgz
 339470 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
 437319 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-757ebe15c6afc27935bb3f3b823f1a12_deploy-ipk.tgz
 437805 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
 437864 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz
 882262 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-f22b72a744d826703a553960f3402bc9_package.tgz
 882411 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz
 882415 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-634056338ea09623cd84437f710e04d5_package.tgz

So it seems we have the same populate-lic.tgz.siginfo for all 3 machines but remaining tasks are 1 for each machine.

so lets check deploy-ipk for qemuarm and qemuarmcopy..

OE qemuarmcopy@shr ~/shr-core/sstate-cache $ bitbake-diffsigs sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz.siginfo sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
basehash changed from 1d549bac0c3a8e400f1b56b7df9cf79e to 7558b2191b73c3ef5d99e608c2701215
Variable OPKGBUILDCMD value changed from opkg-build -o 0 -g 0 to opkg-build
Hash for dependent task virtual:native:/OE/shr-core/openembedded-core/meta/recipes-devtools/opkg-utils/opkg-utils_svn.bb.do_populate_sysroot changed from ed94400e8710f5a838f362f1ce67fb78 to 23c7d629ee4bf98f81cb85af901e7396
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb.do_package changed from 634056338ea09623cd84437f710e04d5 to 0bd093f2a681edc159eff0dc784022f9
Hash for dependent task virtual:native:/OE/shr-core/openembedded-core/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb.do_populate_sysroot changed from dde1a4ad1e7479005c272dda013d87ef to fca554e63de71499466d62d6c6d0f38c

ahh lots of changes.. but wait!
So MACHINE=qemuarm first said
NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed.
but sstate checksums were invalid already for qemuarm (ie due to OPKGBUILDCMD change)

So let's retry with clean sstate:

OE qemuarmcopy@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb
..
OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
ls: cannot access sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-*: No such file or directory

OK no sstate tgz for all 3 armv5t machines

OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* |  sed 's/.*bitbake//g'
  460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk
 1.2K Oct 30 00:54 manifest-qemuarm-expat.package
  138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic
  732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot
  460 Oct 25 15:35 manifest-spitz-expat.deploy-ipk
 1.2K Oct 25 15:35 manifest-spitz-expat.package
  138 Oct 25 15:35 manifest-spitz-expat.populate-lic
  704 Oct 25 15:35 manifest-spitz-expat.populate-sysroot

Remaining 2 armv5t machines still have their manifests, lets clean them too.

OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm    
OE qemuarm@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb 
....

OE qemuarm@shr ~/shr-core $ export MACHINE=spitz
OE spitz@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb 
...

OE qemuarm@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
ls: cannot access manifest-qemuarm*expat*: No such file or directory

OK good, clean start

OE qemuarm@shr ~/shr-core $ bitbake expat
Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
...
NOTE: Tasks Summary: Attempted 2428 tasks of which 2414 didn't need to be rerun and 0 failed.

OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy                                                                                                      
OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
    926 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
   8444 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
  12409 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
  21852 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
  91826 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
 339474 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
 437687 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
 882413 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz

OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
  460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
 1.2K Nov  3 21:51 manifest-qemuarm-expat.package
  138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
  732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot

sofar good

OE qemuarmcopy@shr ~/shr-core $ bitbake expat
Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: package expat-2.0.1-r1: task do_package_write_ipk_setscene: Started
NOTE: package expat-2.0.1-r1: task do_populate_sysroot_setscene: Started
NOTE: package expat-2.0.1-r1: task do_package_write_ipk_setscene: Succeeded
NOTE: package expat-2.0.1-r1: task do_populate_sysroot_setscene: Succeeded
NOTE: package expat-2.0.1-r1: task do_package_setscene: Started
NOTE: package expat-2.0.1-r1: task do_package_setscene: Succeeded
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.

OE spitz@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
  460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
 1.2K Nov  3 21:51 manifest-qemuarm-expat.package
  138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
  732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot
  460 Nov  3 21:54 manifest-qemuarmcopy-expat.deploy-ipk
 1.2K Nov  3 21:54 manifest-qemuarmcopy-expat.package
  788 Nov  3 21:54 manifest-qemuarmcopy-expat.populate-sysroot

nice! everything reused! lets try spitz..

OE spitz@shr ~/shr-core $ bitbake expat
NOTE: Out of date cache found, rebuilding...
NOTE: shr DOES NOT support sysvinit because use systemd now                                                                                    | ETA:  0:01:26
NOTE: shr DOES NOT support external-poky-toolchain because use internal toolchain                                                              | ETA:  0:01:29
NOTE: shr DOES NOT support external-csl-toolchain because use internal toolchain
NOTE: shr DOES NOT support console-tools because use kbd now
NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC                                                                       | ETA:  0:01:30
NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC
NOTE: shr DOES NOT support gconf-dbus because gconf-dbus is provided by gconf                                                                  | ETA:  0:01:32
NOTE: shr DOES NOT support xserver-common because use x11-common now###########################################                                | ETA:  0:00:35
Parsing recipes: 100% |########################################################################################################################| Time: 0:04:23
Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 52 skipped, 0 masked, 0 errors.

OE Build Configuration:
BB_VERSION        = "1.13.3"
TARGET_ARCH       = "arm"
TARGET_OS         = "linux-gnueabi"
MACHINE           = "spitz"
DISTRO            = "shr"
DISTRO_VERSION    = "20111103"
TUNE_FEATURES     = "armv5 dsp thumb xscale"
TARGET_FPU        = "soft"
meta-jama         = "master:f699df1c7aa69f669cf0e6f0e1fe5cc1e7f3e29e"
meta-shr          
meta-aurora       
meta-fso          = "jama:b159c41f87b6349a1b3963b77a696e3414012fc9"
meta-efl          
meta-oe           
meta-gnome        
meta-gpe          = "shr:b4ef7ac3f70ef15c45ef01af87a665d2a4135890"
meta-nokia        
meta-htc          
meta-palm         
meta-openmoko     
meta-samsung      = "jama:b159c41f87b6349a1b3963b77a696e3414012fc9"
meta-handheld     = "jansa/spitz:b228c7de95d49afc31866523c1c542a53f6229a7"
meta              = "jansa/test:9f46daa3b656ccd83ac3322ab8c8a027eb09b260"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 1961 of 2428 (ID: 4, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_fetch)
NOTE: Running task 2142 of 2428 (ID: 1248, virtual:native:/OE/shr-core/meta-openembedded/meta-gnome/recipes-gnome/gobject-introspection/gobject-introspection_
git.bb, do_fetch)
NOTE: package expat-2.0.1-r1: task do_fetch: Started
NOTE: package expat-2.0.1-r1: task do_fetch: Succeeded
NOTE: Running task 2143 of 2428 (ID: 0, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_unpack)
NOTE: package expat-2.0.1-r1: task do_unpack: Started
NOTE: package gobject-introspection-native-1.29.0+gitr3+8d64bc23d2b837421ecf9c7b0e4b8d5d95ca0d21-r0: task do_fetch: Started
...
NOTE: package gobject-introspection-native-1.29.0+gitr3+8d64bc23d2b837421ecf9c7b0e4b8d5d95ca0d21-r0: task do_rm_work: Succeeded
NOTE: package libproxy-0.4.7-r1: task do_fetch: Started
...
NOTE: package expat-2.0.1-r1: task do_rm_work: Succeeded
NOTE: package libproxy-0.4.7-r1: task do_rm_work: Started
NOTE: package libproxy-0.4.7-r1: task do_rm_work: Succeeded
NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all)
NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started
NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 2428 tasks of which 2388 didn't need to be rerun and 0 failed.

libproxy was built because I had to cleansstate it in first qemuarmcopy build - remember?
gobject-introspection-native was rebuilt for some reason, but probably not important

but why did it build expat again?

OE spitz@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
    924 Nov  3 22:02 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
   8444 Nov  3 22:02 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
  12409 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
  12409 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-8ddf3a5ca6b77e09cb0c64fcc7ff5279_deploy-ipk.tgz.siginfo
  21845 Nov  3 22:03 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-20b5b1202729ee864303af147d64958d_populate-sysroot.tgz.siginfo
  21852 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
  91819 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz.siginfo
  91826 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
 339330 Nov  3 22:03 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-20b5b1202729ee864303af147d64958d_populate-sysroot.tgz
 339474 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
 437571 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-8ddf3a5ca6b77e09cb0c64fcc7ff5279_deploy-ipk.tgz
 437687 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
 882277 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz
 882413 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz

OE spitz@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* |  sed 's/.*bitbake//g'
  460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
 1.2K Nov  3 21:51 manifest-qemuarm-expat.package
  138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
  732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot
  460 Nov  3 21:54 manifest-qemuarmcopy-expat.deploy-ipk
 1.2K Nov  3 21:54 manifest-qemuarmcopy-expat.package
  788 Nov  3 21:54 manifest-qemuarmcopy-expat.populate-sysroot
  460 Nov  3 22:04 manifest-spitz-expat.deploy-ipk
 1.2K Nov  3 22:04 manifest-spitz-expat.package
  138 Nov  3 22:02 manifest-spitz-expat.populate-lic
  704 Nov  3 22:03 manifest-spitz-expat.populate-sysroot

OE qemuarm@shr ~/shr-core/sstate-cache $ bitbake-diffsigs sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz.siginfo sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
basehash changed from 46af1626aaa837363f75be1feb68a887 to 538e0e2d40c961a31a4347625af0b0f2
Variable TUNE_CCARGS value changed from  ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} ${@bb.utils.contains("TUNE_FEATURES", "xscale", "-mtune=xscale", "", d)} to  ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} ${@bb.utils.contains("TUNE_FEATURES", "arm926ejs", "-mtune=arm926ej-s", "", d)}
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb.do_install changed from caca792bd48071654a7061388de25373 to 18e1dbc2128e23f65b5a8d8fc418a38d
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_4.6.bb.do_package changed from dd50ecd95cc6795a9dd0cd2bf46f402d to b781ef14e88af52bab8e86f18fd44e8a
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/eglibc/eglibc_2.14.bb.do_package changed from 585a01290d2062f02ec6618a9b4b661d to 27699293b72b7b12aa2662b9a1a66a53
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/desktop-file-utils/desktop-file-utils-native_0.15.bb.do_populate_sysroot changed from 67753340d7173dcf7d0a50f750b322b1 to 9d6d5166a23472faa5b66c8ab86a9448
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/gcc/gcc-runtime_4.6.bb.do_package changed from 66c434a5afcb5895d45e5f78a5375240 to de96ebfb46addf8e7a945a56da436af5
Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.bb.do_package changed from c2e9e854f6c13ae537f6d6edc6b6ed54 to 34ccdc741d85abba60afe3ce93676e77

aaah so they're compatible only as far as they have completely the same TUNE_CCARGS?

few more white-spaces to see the diff:
Variable TUNE_CCARGS value changed from  
${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} 
${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} 
${@bb.utils.contains("TUNE_FEATURES", "xscale", "-mtune=xscale", "", d)} 
to  
${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} 
${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} 
${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)}
${@bb.utils.contains("TUNE_FEATURES", "arm926ejs", "-mtune=arm926ej-s", "", d)}

That's pity as they are overwritting the same files in feeds:

OE qemuarm@shr ~/shr-core $ ls -lah tmp/deploy/ipk/armv5te/libexpat*
-rw-r--r-- 1 bitbake bitbake 8.3K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-bin_2.0.1-r1_armv5te.ipk
-rw-r--r-- 1 bitbake bitbake 191K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-dbg_2.0.1-r1_armv5te.ipk
-rw-r--r-- 1 bitbake bitbake  13K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-dev_2.0.1-r1_armv5te.ipk
-rw-r--r-- 1 bitbake bitbake 4.4K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-doc_2.0.1-r1_armv5te.ipk
-rw-r--r-- 1 bitbake bitbake 171K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-staticdev_2.0.1-r1_armv5te.ipk
-rw-r--r-- 1 bitbake bitbake  48K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat1_2.0.1-r1_armv5te.ipk

So user won't know if the package he downloads was built as qemuarm or qemuarmcopy or spitz..
all he will notice is that md5 sums for ipk files are changing alot.

Cheers,
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: sstate.bbclass: Ensure machine specific stamps are only wiped for the current task
  2011-11-03 22:08             ` Martin Jansa
@ 2011-11-04  9:32               ` Martin Jansa
  0 siblings, 0 replies; 14+ messages in thread
From: Martin Jansa @ 2011-11-04  9:32 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 27566 bytes --]

On Thu, Nov 03, 2011 at 11:08:37PM +0100, Martin Jansa wrote:
> On Thu, Nov 03, 2011 at 05:33:28PM +0100, Martin Jansa wrote:
> > Hi,
> > 
> > to recap what we have talked about on ELCE;
> > 
> > Currently it doesn't seem to change sstate checksums after switching
> > MACHINE, but still it rebuilds everything at least once after switch.
> > 
> > Today I've tried to wipe tmp and rebuild from sstate-cache and I've
> > noticed interesting thing:
> > 
> > All recipes are starting with 
> > do_populate_lic_setscene 
> > not 
> > do_populate_sysroot_setscene
> > like ie koen's build does
> > 
> > NOTE: Resolving any missing task queue dependencies
> > NOTE: Preparing runqueue
> > NOTE: Executing SetScene Tasks
> > NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Started
> > NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Started
> > NOTE: package quilt-native-0.48-r0: task do_populate_lic_setscene: Succeeded
> > NOTE: package shr-image-2.0-r18: task do_populate_lic_setscene: Succeeded
> > 
> > Does do_populate_lic_setscene wipe all stamps after do_populate_lic so it 
> > causes the rebuild instead of population of sysroot from sstate?
> > 
> > IIRC I had similar issue when I was starting to use oe-core and it was 
> > in the end caused by inheriting package_ipk (once directly from distro config 
> > and once through sane-feed-ipk.inc) and it was fixed 6 months ago by:
> > http://git.shr-project.org/git/?p=meta-shr.git;a=commit;h=00623b3c6dd01c4a227008f7cfdb0801cb06f49a
> > And IIRC we've talked about some ways to make sstate less error-prone
> > for such things.
> > 
> > Now I have similar symptoms but only after switching MACHINE.
> > 
> > Any hint?
> 
> With few more hours of debuging this.. I can say that sometimes it works, 
> but sometimes it's too strict to be usefull anyway.

This is also related to older thread about sstate
http://lists.linuxtogo.org/pipermail/openembedded-core/2011-May/003308.html

I'm going to try
BB_SIGNATURE_HANDLER = "basichash"
as it's supposed to become default anyway IIRC.

Regards,

> 
> Whole logs are here:
> http://build.shr-project.org/tests/jama/sstate.test
> here I'll try to shorten as much as possible.
> 
> OE @shr ~/shr-core $ export MACHINE=qemuarm
> OE qemuarm@shr ~/shr-core $ bitbake -k core-image-core
> .....
> NOTE: package core-image-core-1.0-r0: task do_rm_work_all: Succeeded
> NOTE: Tasks Summary: Attempted 5306 tasks of which 5250 didn't need to be rerun and 0 failed.
> OE qemuarm@shr ~/shr-core $ bitbake -k expat
> Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: Executing RunQueue Tasks
> NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all)
> NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started
> NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed.
> 
> All done
> 
> OE qemuarmB@shr ~/shr-core $ cp openembedded-core/meta/conf/machine/qemuarm.conf openembedded-core/meta/conf/machine/qemuarmcopy.conf
> OE qemuarmB@shr ~/shr-core $ export MACHINE=qemuarmcopy
> OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
> Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> ..
> libproxy fail (around task 2232)
> fix
> 
> OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
> libproxy built
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2232 didn't need to be rerun and 0 failed.
> 
> So after 3 hours we have qemuarm and qemuarmcopy (identical machine) with expat built! hurray
> But seems like nothing was reused from qemuarm build.
> 
> once again for check
> 
> OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm
> OE qemuarm@shr ~/shr-core $ bitbake -k expat
> Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> ...
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.
> 
> OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy
> OE qemuarmcopy@shr ~/shr-core $ bitbake -k expat
> Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> ...
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.
> 
> great
> 
> OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* | sed 's/.*bitbake//g'
>   460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk
>  1.2K Oct 30 00:54 manifest-qemuarm-expat.package
>   138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic
>   732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot
>   460 Nov  3 19:50 manifest-qemuarmcopy-expat.deploy-ipk
>  1.2K Nov  3 19:50 manifest-qemuarmcopy-expat.package
>   138 Nov  3 19:17 manifest-qemuarmcopy-expat.populate-lic
>   788 Nov  3 19:50 manifest-qemuarmcopy-expat.populate-sysroot
>   460 Oct 25 15:35 manifest-spitz-expat.deploy-ipk
>  1.2K Oct 25 15:35 manifest-spitz-expat.package
>   138 Oct 25 15:35 manifest-spitz-expat.populate-lic
>   704 Oct 25 15:35 manifest-spitz-expat.populate-sysroot
> 
> OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
>     926 Nov  3 19:17 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
>    8444 Nov  3 19:17 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
>   12409 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
>   12419 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-757ebe15c6afc27935bb3f3b823f1a12_deploy-ipk.tgz.siginfo
>   12419 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz.siginfo
>   21852 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
>   21920 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-d813b67ed935225757b90d014f631652_populate-sysroot.tgz.siginfo
>   21927 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e90bb07db6e100f14ee8f8eed99a7d7e_populate-sysroot.tgz.siginfo
>   91310 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-f22b72a744d826703a553960f3402bc9_package.tgz.siginfo
>   91317 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-634056338ea09623cd84437f710e04d5_package.tgz.siginfo
>   91826 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
>  339323 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-d813b67ed935225757b90d014f631652_populate-sysroot.tgz
>  339468 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e90bb07db6e100f14ee8f8eed99a7d7e_populate-sysroot.tgz
>  339470 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
>  437319 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-757ebe15c6afc27935bb3f3b823f1a12_deploy-ipk.tgz
>  437805 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
>  437864 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz
>  882262 Oct 25 15:35 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-f22b72a744d826703a553960f3402bc9_package.tgz
>  882411 Nov  3 19:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz
>  882415 Oct 30 00:54 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-634056338ea09623cd84437f710e04d5_package.tgz
> 
> So it seems we have the same populate-lic.tgz.siginfo for all 3 machines but remaining tasks are 1 for each machine.
> 
> so lets check deploy-ipk for qemuarm and qemuarmcopy..
> 
> OE qemuarmcopy@shr ~/shr-core/sstate-cache $ bitbake-diffsigs sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-e2d316a9a054855d4433a3734e343db8_deploy-ipk.tgz.siginfo sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
> basehash changed from 1d549bac0c3a8e400f1b56b7df9cf79e to 7558b2191b73c3ef5d99e608c2701215
> Variable OPKGBUILDCMD value changed from opkg-build -o 0 -g 0 to opkg-build
> Hash for dependent task virtual:native:/OE/shr-core/openembedded-core/meta/recipes-devtools/opkg-utils/opkg-utils_svn.bb.do_populate_sysroot changed from ed94400e8710f5a838f362f1ce67fb78 to 23c7d629ee4bf98f81cb85af901e7396
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb.do_package changed from 634056338ea09623cd84437f710e04d5 to 0bd093f2a681edc159eff0dc784022f9
> Hash for dependent task virtual:native:/OE/shr-core/openembedded-core/meta/recipes-devtools/pseudo/pseudo_1.1.1.bb.do_populate_sysroot changed from dde1a4ad1e7479005c272dda013d87ef to fca554e63de71499466d62d6c6d0f38c
> 
> ahh lots of changes.. but wait!
> So MACHINE=qemuarm first said
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2427 didn't need to be rerun and 0 failed.
> but sstate checksums were invalid already for qemuarm (ie due to OPKGBUILDCMD change)
> 
> So let's retry with clean sstate:
> 
> OE qemuarmcopy@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb
> ..
> OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
> ls: cannot access sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-*: No such file or directory
> 
> OK no sstate tgz for all 3 armv5t machines
> 
> OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* |  sed 's/.*bitbake//g'
>   460 Oct 30 00:54 manifest-qemuarm-expat.deploy-ipk
>  1.2K Oct 30 00:54 manifest-qemuarm-expat.package
>   138 Oct 30 00:34 manifest-qemuarm-expat.populate-lic
>   732 Oct 30 00:54 manifest-qemuarm-expat.populate-sysroot
>   460 Oct 25 15:35 manifest-spitz-expat.deploy-ipk
>  1.2K Oct 25 15:35 manifest-spitz-expat.package
>   138 Oct 25 15:35 manifest-spitz-expat.populate-lic
>   704 Oct 25 15:35 manifest-spitz-expat.populate-sysroot
> 
> Remaining 2 armv5t machines still have their manifests, lets clean them too.
> 
> OE qemuarmcopy@shr ~/shr-core $ export MACHINE=qemuarm    
> OE qemuarm@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb 
> ....
> 
> OE qemuarm@shr ~/shr-core $ export MACHINE=spitz
> OE spitz@shr ~/shr-core $ bitbake -c cleansstate -b openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb 
> ...
> 
> OE qemuarm@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
> ls: cannot access manifest-qemuarm*expat*: No such file or directory
> 
> OK good, clean start
> 
> OE qemuarm@shr ~/shr-core $ bitbake expat
> Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> ...
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2414 didn't need to be rerun and 0 failed.
> 
> OE qemuarm@shr ~/shr-core $ export MACHINE=qemuarmcopy                                                                                                      
> OE qemuarmcopy@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
>     926 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
>    8444 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
>   12409 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
>   21852 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
>   91826 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
>  339474 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
>  437687 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
>  882413 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz
> 
> OE qemuarmcopy@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
>   460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
>  1.2K Nov  3 21:51 manifest-qemuarm-expat.package
>   138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
>   732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot
> 
> sofar good
> 
> OE qemuarmcopy@shr ~/shr-core $ bitbake expat
> Parsing of 1436 .bb files complete (1423 cached, 13 parsed). 1837 targets, 56 skipped, 0 masked, 0 errors.
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: package expat-2.0.1-r1: task do_package_write_ipk_setscene: Started
> NOTE: package expat-2.0.1-r1: task do_populate_sysroot_setscene: Started
> NOTE: package expat-2.0.1-r1: task do_package_write_ipk_setscene: Succeeded
> NOTE: package expat-2.0.1-r1: task do_populate_sysroot_setscene: Succeeded
> NOTE: package expat-2.0.1-r1: task do_package_setscene: Started
> NOTE: package expat-2.0.1-r1: task do_package_setscene: Succeeded
> NOTE: Executing RunQueue Tasks
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2428 didn't need to be rerun and 0 failed.
> 
> OE spitz@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* |  sed 's/.*bitbake//g'
>   460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
>  1.2K Nov  3 21:51 manifest-qemuarm-expat.package
>   138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
>   732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot
>   460 Nov  3 21:54 manifest-qemuarmcopy-expat.deploy-ipk
>  1.2K Nov  3 21:54 manifest-qemuarmcopy-expat.package
>   788 Nov  3 21:54 manifest-qemuarmcopy-expat.populate-sysroot
> 
> nice! everything reused! lets try spitz..
> 
> OE spitz@shr ~/shr-core $ bitbake expat
> NOTE: Out of date cache found, rebuilding...
> NOTE: shr DOES NOT support sysvinit because use systemd now                                                                                    | ETA:  0:01:26
> NOTE: shr DOES NOT support external-poky-toolchain because use internal toolchain                                                              | ETA:  0:01:29
> NOTE: shr DOES NOT support external-csl-toolchain because use internal toolchain
> NOTE: shr DOES NOT support console-tools because use kbd now
> NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC                                                                       | ETA:  0:01:30
> NOTE: shr DOES NOT support libiconv because libiconv is provided by LIBC
> NOTE: shr DOES NOT support gconf-dbus because gconf-dbus is provided by gconf                                                                  | ETA:  0:01:32
> NOTE: shr DOES NOT support xserver-common because use x11-common now###########################################                                | ETA:  0:00:35
> Parsing recipes: 100% |########################################################################################################################| Time: 0:04:23
> Parsing of 1436 .bb files complete (0 cached, 1436 parsed). 1837 targets, 52 skipped, 0 masked, 0 errors.
> 
> OE Build Configuration:
> BB_VERSION        = "1.13.3"
> TARGET_ARCH       = "arm"
> TARGET_OS         = "linux-gnueabi"
> MACHINE           = "spitz"
> DISTRO            = "shr"
> DISTRO_VERSION    = "20111103"
> TUNE_FEATURES     = "armv5 dsp thumb xscale"
> TARGET_FPU        = "soft"
> meta-jama         = "master:f699df1c7aa69f669cf0e6f0e1fe5cc1e7f3e29e"
> meta-shr          
> meta-aurora       
> meta-fso          = "jama:b159c41f87b6349a1b3963b77a696e3414012fc9"
> meta-efl          
> meta-oe           
> meta-gnome        
> meta-gpe          = "shr:b4ef7ac3f70ef15c45ef01af87a665d2a4135890"
> meta-nokia        
> meta-htc          
> meta-palm         
> meta-openmoko     
> meta-samsung      = "jama:b159c41f87b6349a1b3963b77a696e3414012fc9"
> meta-handheld     = "jansa/spitz:b228c7de95d49afc31866523c1c542a53f6229a7"
> meta              = "jansa/test:9f46daa3b656ccd83ac3322ab8c8a027eb09b260"
> 
> NOTE: Resolving any missing task queue dependencies
> NOTE: Preparing runqueue
> NOTE: Executing SetScene Tasks
> NOTE: Executing RunQueue Tasks
> NOTE: Running task 1961 of 2428 (ID: 4, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_fetch)
> NOTE: Running task 2142 of 2428 (ID: 1248, virtual:native:/OE/shr-core/meta-openembedded/meta-gnome/recipes-gnome/gobject-introspection/gobject-introspection_
> git.bb, do_fetch)
> NOTE: package expat-2.0.1-r1: task do_fetch: Started
> NOTE: package expat-2.0.1-r1: task do_fetch: Succeeded
> NOTE: Running task 2143 of 2428 (ID: 0, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_unpack)
> NOTE: package expat-2.0.1-r1: task do_unpack: Started
> NOTE: package gobject-introspection-native-1.29.0+gitr3+8d64bc23d2b837421ecf9c7b0e4b8d5d95ca0d21-r0: task do_fetch: Started
> ...
> NOTE: package gobject-introspection-native-1.29.0+gitr3+8d64bc23d2b837421ecf9c7b0e4b8d5d95ca0d21-r0: task do_rm_work: Succeeded
> NOTE: package libproxy-0.4.7-r1: task do_fetch: Started
> ...
> NOTE: package expat-2.0.1-r1: task do_rm_work: Succeeded
> NOTE: package libproxy-0.4.7-r1: task do_rm_work: Started
> NOTE: package libproxy-0.4.7-r1: task do_rm_work: Succeeded
> NOTE: Running task 2428 of 2428 (ID: 11, /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb, do_rm_work_all)
> NOTE: package expat-2.0.1-r1: task do_rm_work_all: Started
> NOTE: package expat-2.0.1-r1: task do_rm_work_all: Succeeded
> NOTE: Tasks Summary: Attempted 2428 tasks of which 2388 didn't need to be rerun and 0 failed.
> 
> libproxy was built because I had to cleansstate it in first qemuarmcopy build - remember?
> gobject-introspection-native was rebuilt for some reason, but probably not important
> 
> but why did it build expat again?
> 
> OE spitz@shr ~/shr-core/sstate-cache $ ls -la1 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-* | sort | sed 's/.*bitbake//g'
>     924 Nov  3 22:02 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz
>    8444 Nov  3 22:02 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-b198883dcffdb3af1a2c669f97d379ca_populate-lic.tgz.siginfo
>   12409 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz.siginfo
>   12409 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-8ddf3a5ca6b77e09cb0c64fcc7ff5279_deploy-ipk.tgz.siginfo
>   21845 Nov  3 22:03 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-20b5b1202729ee864303af147d64958d_populate-sysroot.tgz.siginfo
>   21852 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz.siginfo
>   91819 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz.siginfo
>   91826 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
>  339330 Nov  3 22:03 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-20b5b1202729ee864303af147d64958d_populate-sysroot.tgz
>  339474 Nov  3 21:50 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-1a807cb22931fac0fbd848d258cafecd_populate-sysroot.tgz
>  437571 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-8ddf3a5ca6b77e09cb0c64fcc7ff5279_deploy-ipk.tgz
>  437687 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-62f6df13c2e883d59f8e73e5eb24fdab_deploy-ipk.tgz
>  882277 Nov  3 22:04 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz
>  882413 Nov  3 21:51 sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz
> 
> OE spitz@shr ~/shr-core/tmp/sstate-control $ ls -lah manifest-qemuarm*expat* manifest-spitz-*expat* |  sed 's/.*bitbake//g'
>   460 Nov  3 21:51 manifest-qemuarm-expat.deploy-ipk
>  1.2K Nov  3 21:51 manifest-qemuarm-expat.package
>   138 Nov  3 21:50 manifest-qemuarm-expat.populate-lic
>   732 Nov  3 21:50 manifest-qemuarm-expat.populate-sysroot
>   460 Nov  3 21:54 manifest-qemuarmcopy-expat.deploy-ipk
>  1.2K Nov  3 21:54 manifest-qemuarmcopy-expat.package
>   788 Nov  3 21:54 manifest-qemuarmcopy-expat.populate-sysroot
>   460 Nov  3 22:04 manifest-spitz-expat.deploy-ipk
>  1.2K Nov  3 22:04 manifest-spitz-expat.package
>   138 Nov  3 22:02 manifest-spitz-expat.populate-lic
>   704 Nov  3 22:03 manifest-spitz-expat.populate-sysroot
> 
> OE qemuarm@shr ~/shr-core/sstate-cache $ bitbake-diffsigs sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-cab79e114cc993bf250b13dfa8a93624_package.tgz.siginfo sstate-expat-armv5te-oe-linux-gnueabi-2.0.1-r1-armv5te-2-0bd093f2a681edc159eff0dc784022f9_package.tgz.siginfo
> basehash changed from 46af1626aaa837363f75be1feb68a887 to 538e0e2d40c961a31a4347625af0b0f2
> Variable TUNE_CCARGS value changed from  ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} ${@bb.utils.contains("TUNE_FEATURES", "xscale", "-mtune=xscale", "", d)} to  ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} ${@bb.utils.contains("TUNE_FEATURES", "arm926ejs", "-mtune=arm926ej-s", "", d)}
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/expat/expat_2.0.1.bb.do_install changed from caca792bd48071654a7061388de25373 to 18e1dbc2128e23f65b5a8d8fc418a38d
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_4.6.bb.do_package changed from dd50ecd95cc6795a9dd0cd2bf46f402d to b781ef14e88af52bab8e86f18fd44e8a
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-core/eglibc/eglibc_2.14.bb.do_package changed from 585a01290d2062f02ec6618a9b4b661d to 27699293b72b7b12aa2662b9a1a66a53
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/desktop-file-utils/desktop-file-utils-native_0.15.bb.do_populate_sysroot changed from 67753340d7173dcf7d0a50f750b322b1 to 9d6d5166a23472faa5b66c8ab86a9448
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/gcc/gcc-runtime_4.6.bb.do_package changed from 66c434a5afcb5895d45e5f78a5375240 to de96ebfb46addf8e7a945a56da436af5
> Hash for dependent task /OE/shr-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.bb.do_package changed from c2e9e854f6c13ae537f6d6edc6b6ed54 to 34ccdc741d85abba60afe3ce93676e77
> 
> aaah so they're compatible only as far as they have completely the same TUNE_CCARGS?
> 
> few more white-spaces to see the diff:
> Variable TUNE_CCARGS value changed from  
> ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "xscale", "-mtune=xscale", "", d)} 
> to  
> ${@bb.utils.contains("TUNE_FEATURES", "armv5", "-march=armv5${ARMPKGSFX_THUMB}${ARMPKGSFX_DSP}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "armv4", "-march=armv4${ARMPKGSFX_THUMB}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "thumb", "${ARM_THUMB_M_OPT}", "", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", "-mno-thumb-interwork", "-mthumb-interwork", d)} 
> ${@bb.utils.contains("TUNE_FEATURES", "vfp", bb.utils.contains("TUNE_FEATURES", "callconvention-hard", "-mfloat-abi=hard", "-mfloat-abi=softfp", d), "" ,d)}
> ${@bb.utils.contains("TUNE_FEATURES", "arm926ejs", "-mtune=arm926ej-s", "", d)}
> 
> That's pity as they are overwritting the same files in feeds:
> 
> OE qemuarm@shr ~/shr-core $ ls -lah tmp/deploy/ipk/armv5te/libexpat*
> -rw-r--r-- 1 bitbake bitbake 8.3K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-bin_2.0.1-r1_armv5te.ipk
> -rw-r--r-- 1 bitbake bitbake 191K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-dbg_2.0.1-r1_armv5te.ipk
> -rw-r--r-- 1 bitbake bitbake  13K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-dev_2.0.1-r1_armv5te.ipk
> -rw-r--r-- 1 bitbake bitbake 4.4K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-doc_2.0.1-r1_armv5te.ipk
> -rw-r--r-- 1 bitbake bitbake 171K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat-staticdev_2.0.1-r1_armv5te.ipk
> -rw-r--r-- 1 bitbake bitbake  48K Nov  3 22:04 tmp/deploy/ipk/armv5te/libexpat1_2.0.1-r1_armv5te.ipk
> 
> So user won't know if the package he downloads was built as qemuarm or qemuarmcopy or spitz..
> all he will notice is that md5 sums for ipk files are changing alot.
> 
> Cheers,
> -- 
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com



-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2011-11-04  9:38 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-10-06 14:19 sstate.bbclass: Ensure machine specific stamps are only wiped for the current task Richard Purdie
2011-10-06 20:57 ` Koen Kooi
2011-10-09 11:25 ` Koen Kooi
2011-10-09 18:10   ` Martin Jansa
2011-10-09 18:11     ` Koen Kooi
2011-10-10 11:45       ` Martin Jansa
2011-10-10 14:16         ` Richard Purdie
2011-10-10 14:37           ` Koen Kooi
2011-10-10 14:41             ` Martin Jansa
2011-10-10 15:17               ` Richard Purdie
2011-10-10 14:17         ` Richard Purdie
2011-11-03 16:33           ` Martin Jansa
2011-11-03 22:08             ` Martin Jansa
2011-11-04  9:32               ` Martin Jansa

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox