Machine dependencies validation with sstate

Machine dependencies validation with sstate

[Richard Purdie]

In the process of diving into the issues in bug 1916
(http://bugzilla.yoctoproject.org/show_bug.cgi?id=1916), I've come up
with a good way of detecting machine specific changes to generic
PACKAGE_ARCH packages.

The idea is to have two identical machines, qemux86 and qemux86copy.
These can be setup by:

$ cp meta/conf/machine/qemux86.conf meta/conf/machine/qemux86copy.conf 
[Add SRCREV entry for qemux86copy to meta/recipes-kernel/linux/linux-yocto_3.0.bb]
$ cp -r meta/recipes-core/netbase/netbase-4.47/qemux86 meta/recipes-core/netbase/netbase-4.47/qemux86copy
[Add PACKAGE_ARCH_qemux86copy to meta/recipes-core/netbase/netbase_4.47.bb]
[Add RRECOMMENDS_${PN}_qemux86copy to meta/recipes-multimedia/gstreamer/gstreamer_0.10.35.bb]

You can then generate all of the sigdata files for two machine builds by
running the following commands. The nice thing about these is that you
don't have to run a complete build, it just writes out the data the
build would have generated:

rm tmp/stamps/*/*sigdata*
MACHINE=qemux86 bitbake core-image-sato -S
find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
MACHINE=qemux86copy bitbake core-image-sato -S
find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2

and then comparing the files l1 and l2, you can see which sigdata files
differ. Running bitbake-diffsigs on the files, e.g.:

bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*

can show what changed and you can then consider whether that is a valid
change and how it might be avoided if necessary. This is how some of the
patches I've just posted were developed.

Comparing qemux86 and qemux86copy as above with core-image-sato now
yields only two differences, in the package_write* tasks of iptables and
gstreamer. This is due to the dependencies these two recipes have on
kernel-module-* packages which in turn depend on linux-yocto which is
machine specific. As yet I can't find a good way to avoid the kernel
dependencies. If we could break debian.bbclass's hold in the
kernel-module case (which never get renamed), that would be one way to
avoid this problem.

I thought people might find this intersting and it was worth documenting
in the list archives if nothing else.

Cheers,

Richard

Re: Machine dependencies validation with sstate

[McClintock Matthew-B29882]

On Tue, Jan 24, 2012 at 11:22 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> rm tmp/stamps/*/*sigdata*
> MACHINE=qemux86 bitbake core-image-sato -S
> find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
> find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
> MACHINE=qemux86copy bitbake core-image-sato -S
> find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
> find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2

I never knew about the '-S' option... that's helpful and saves a lot of time.

> and then comparing the files l1 and l2, you can see which sigdata files
> differ. Running bitbake-diffsigs on the files, e.g.:
>
> bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*

Have you found an easy way to traverse up a dependency chain to the
initial signature? It can take a while running the above command until
you get to the *real* signature change on not just a signature change
because one of the dependencies signature changed...

Some sort of tool at this point seems to be the solution. It could do
a lot more as well. Tracking signature changes between builds would be
nice as well. The list could go on and on...

-M

Re: Machine dependencies validation with sstate

[Richard Purdie]

On Mon, 2012-01-30 at 16:48 +0000, McClintock Matthew-B29882 wrote:
> Have you found an easy way to traverse up a dependency chain to the
> initial signature? It can take a while running the above command until
> you get to the *real* signature change on not just a signature change
> because one of the dependencies signature changed...
> 
> Some sort of tool at this point seems to be the solution. It could do
> a lot more as well. Tracking signature changes between builds would be
> nice as well. The list could go on and on...

There isn't an easy way at the moment, I've just been relying on an
educated guess from looking at two sets of files where the differences
appear to start from.

I agree we really need a tool in this kind of area. You could file an
enhancement request but it could really do with someone working on it to
help drive it forward. The bitbake core code at least already has many
useful functions to help perform the analysis.

Cheers,

Richard

Re: Machine dependencies validation with sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5352 bytes --]

On Tue, Jan 24, 2012 at 05:22:09PM +0000, Richard Purdie wrote:
> In the process of diving into the issues in bug 1916
> (http://bugzilla.yoctoproject.org/show_bug.cgi?id=1916), I've come up
> with a good way of detecting machine specific changes to generic
> PACKAGE_ARCH packages.
> 
> The idea is to have two identical machines, qemux86 and qemux86copy.
> These can be setup by:
> 
> $ cp meta/conf/machine/qemux86.conf meta/conf/machine/qemux86copy.conf 
> [Add SRCREV entry for qemux86copy to meta/recipes-kernel/linux/linux-yocto_3.0.bb]
> $ cp -r meta/recipes-core/netbase/netbase-4.47/qemux86 meta/recipes-core/netbase/netbase-4.47/qemux86copy
> [Add PACKAGE_ARCH_qemux86copy to meta/recipes-core/netbase/netbase_4.47.bb]
> [Add RRECOMMENDS_${PN}_qemux86copy to meta/recipes-multimedia/gstreamer/gstreamer_0.10.35.bb]
> 
> You can then generate all of the sigdata files for two machine builds by
> running the following commands. The nice thing about these is that you
> don't have to run a complete build, it just writes out the data the
> build would have generated:
> 
> rm tmp/stamps/*/*sigdata*
> MACHINE=qemux86 bitbake core-image-sato -S
> find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
> find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
> MACHINE=qemux86copy bitbake core-image-sato -S
> find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
> find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2
> 
> and then comparing the files l1 and l2, you can see which sigdata files
> differ. Running bitbake-diffsigs on the files, e.g.:
> 
> bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*
> 
> can show what changed and you can then consider whether that is a valid
> change and how it might be avoided if necessary. This is how some of the
> patches I've just posted were developed.
> 
> Comparing qemux86 and qemux86copy as above with core-image-sato now
> yields only two differences, in the package_write* tasks of iptables and
> gstreamer. This is due to the dependencies these two recipes have on
> kernel-module-* packages which in turn depend on linux-yocto which is
> machine specific. As yet I can't find a good way to avoid the kernel
> dependencies. If we could break debian.bbclass's hold in the
> kernel-module case (which never get renamed), that would be one way to
> avoid this problem.
> 
> I thought people might find this intersting and it was worth documenting
> in the list archives if nothing else.

Thanks for this how-to!

Helped me to find e.g. this:
http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=c6be068de2f716c9fdfc54970fd9908b4f2cd0e6

Now I'm looking into why busybox depends on MACHINE variable.

And interesting case is when something depends on linux-libc-headers even task-* recipe 
which includes ie iptables (which rdepends or rrecommends some kernel modules)

Any idea why identical hash of linux-libc-headers_3.1.bb.do_package_write is 
once compared with linux-gta04_git.bb.do_deploy and once linux-nokia900-meego_git.bb.do_package_write?

Is it bug in bitbake-diffsigs output or fault of signature handler?

om-gta02@shr ~/shr-core $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.e772dc750d69dd217f6c7dfe7de98246 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.ec74b075811b5316554475fb6360313a
Task dependency hash changed from c38789fba09937fbb2c2ba96b13fb2e8 to 4068dabb3a553492e678b90434f6c893 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy)
Task dependency hash changed from 9470058e15b666452c6209230c2d445b to 742657c20542706b17592336b1a0e09a (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write)
Task dependency hash changed from e547a64f5b883431b63dcbcaa0ad1f30 to c38789fba09937fbb2c2ba96b13fb2e8 (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write)
Task dependency hash changed from 7f72940519de5e7f16fe1ec5f7108df5 to bd3580a29e1922eacec0c798b009f8e9 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write)
Task dependency hash changed from bcf02bbe9d89f4dff8d3c7b289202fbb to 644a5cc5cd8e2a05716bd84f694d2511 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write)

or more simple case
$ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.6cc951872c73234753749ba739be9796 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.7eb3867f1291db17de32fb142e8c9af2
Task dependency hash changed from 4d510f798d684afd0edaf8c15720e595 to ef18cfc890c39697919a7a6f2ae3e2cb (for linux-nokia900-meego_git.bb.do_package and linux-gta04_git.bb.do_package)

So everything what somehow rdepends/reccomends on machine specific kernel should be also machine specific?
Or can we remove kernel dependency somehow to way that we want package manager to fill use RRECOMMENDS_${PN}
but we don't care (from sstate pov) how kernel was built. We already don't need such module to be built at all.

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Machine dependencies validation with sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 6253 bytes --]

On Thu, Feb 16, 2012 at 02:44:47PM +0100, Martin Jansa wrote:
> On Tue, Jan 24, 2012 at 05:22:09PM +0000, Richard Purdie wrote:
> > In the process of diving into the issues in bug 1916
> > (http://bugzilla.yoctoproject.org/show_bug.cgi?id=1916), I've come up
> > with a good way of detecting machine specific changes to generic
> > PACKAGE_ARCH packages.
> > 
> > The idea is to have two identical machines, qemux86 and qemux86copy.
> > These can be setup by:
> > 
> > $ cp meta/conf/machine/qemux86.conf meta/conf/machine/qemux86copy.conf 
> > [Add SRCREV entry for qemux86copy to meta/recipes-kernel/linux/linux-yocto_3.0.bb]
> > $ cp -r meta/recipes-core/netbase/netbase-4.47/qemux86 meta/recipes-core/netbase/netbase-4.47/qemux86copy
> > [Add PACKAGE_ARCH_qemux86copy to meta/recipes-core/netbase/netbase_4.47.bb]
> > [Add RRECOMMENDS_${PN}_qemux86copy to meta/recipes-multimedia/gstreamer/gstreamer_0.10.35.bb]
> > 
> > You can then generate all of the sigdata files for two machine builds by
> > running the following commands. The nice thing about these is that you
> > don't have to run a complete build, it just writes out the data the
> > build would have generated:
> > 
> > rm tmp/stamps/*/*sigdata*
> > MACHINE=qemux86 bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
> > MACHINE=qemux86copy bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2
> > 
> > and then comparing the files l1 and l2, you can see which sigdata files
> > differ. Running bitbake-diffsigs on the files, e.g.:
> > 
> > bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*
> > 
> > can show what changed and you can then consider whether that is a valid
> > change and how it might be avoided if necessary. This is how some of the
> > patches I've just posted were developed.
> > 
> > Comparing qemux86 and qemux86copy as above with core-image-sato now
> > yields only two differences, in the package_write* tasks of iptables and
> > gstreamer. This is due to the dependencies these two recipes have on
> > kernel-module-* packages which in turn depend on linux-yocto which is
> > machine specific. As yet I can't find a good way to avoid the kernel
> > dependencies. If we could break debian.bbclass's hold in the
> > kernel-module case (which never get renamed), that would be one way to
> > avoid this problem.
> > 
> > I thought people might find this intersting and it was worth documenting
> > in the list archives if nothing else.
> 
> Thanks for this how-to!
> 
> Helped me to find e.g. this:
> http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=c6be068de2f716c9fdfc54970fd9908b4f2cd0e6
> 
> Now I'm looking into why busybox depends on MACHINE variable.
> 
> And interesting case is when something depends on linux-libc-headers even task-* recipe 
> which includes ie iptables (which rdepends or rrecommends some kernel modules)
> 
> Any idea why identical hash of linux-libc-headers_3.1.bb.do_package_write is 
> once compared with linux-gta04_git.bb.do_deploy and once linux-nokia900-meego_git.bb.do_package_write?
> 
> Is it bug in bitbake-diffsigs output or fault of signature handler?
> 
> om-gta02@shr ~/shr-core $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.e772dc750d69dd217f6c7dfe7de98246 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.ec74b075811b5316554475fb6360313a
> Task dependency hash changed from c38789fba09937fbb2c2ba96b13fb2e8 to 4068dabb3a553492e678b90434f6c893 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy)
> Task dependency hash changed from 9470058e15b666452c6209230c2d445b to 742657c20542706b17592336b1a0e09a (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write)
> Task dependency hash changed from e547a64f5b883431b63dcbcaa0ad1f30 to c38789fba09937fbb2c2ba96b13fb2e8 (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write)
> Task dependency hash changed from 7f72940519de5e7f16fe1ec5f7108df5 to bd3580a29e1922eacec0c798b009f8e9 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write)
> Task dependency hash changed from bcf02bbe9d89f4dff8d3c7b289202fbb to 644a5cc5cd8e2a05716bd84f694d2511 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write)
> 
> or more simple case
> $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.6cc951872c73234753749ba739be9796 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.7eb3867f1291db17de32fb142e8c9af2
> Task dependency hash changed from 4d510f798d684afd0edaf8c15720e595 to ef18cfc890c39697919a7a6f2ae3e2cb (for linux-nokia900-meego_git.bb.do_package and linux-gta04_git.bb.do_package)
> 
> So everything what somehow rdepends/reccomends on machine specific kernel should be also machine specific?
> Or can we remove kernel dependency somehow to way that we want package manager to fill use RRECOMMENDS_${PN}
> but we don't care (from sstate pov) how kernel was built. We already don't need such module to be built at all.

and one more weird issue, why are some -native packages depending on
kernel.do_deploy?

bitbake-diffsigs
tmp-eglibc/stamps/x86_64-linux/evas-native-2_1.1.0+svnr67991-r3.1.do_build.sigdata.958e626749c6668426b053c235679492
tmp-eglibc/stamps/x86_64-linux/evas-native-2_1.1.0+svnr67991-r3.1.do_build.sigdata.653789da75b68fd24290dc8364950ccc
Task dependency hash changed from 9470058e15b666452c6209230c2d445b to
4068dabb3a553492e678b90434f6c893 (for
linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_deploy)

If I grep for linux/kernel/deploy in whole meta-efl I don't see how it
could became dependant task of it (the same for edje-native and
ecore-native).

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Machine dependencies validation with sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5765 bytes --]

On Thu, Feb 16, 2012 at 02:44:47PM +0100, Martin Jansa wrote:
> On Tue, Jan 24, 2012 at 05:22:09PM +0000, Richard Purdie wrote:
> > In the process of diving into the issues in bug 1916
> > (http://bugzilla.yoctoproject.org/show_bug.cgi?id=1916), I've come up
> > with a good way of detecting machine specific changes to generic
> > PACKAGE_ARCH packages.
> > 
> > The idea is to have two identical machines, qemux86 and qemux86copy.
> > These can be setup by:
> > 
> > $ cp meta/conf/machine/qemux86.conf meta/conf/machine/qemux86copy.conf 
> > [Add SRCREV entry for qemux86copy to meta/recipes-kernel/linux/linux-yocto_3.0.bb]
> > $ cp -r meta/recipes-core/netbase/netbase-4.47/qemux86 meta/recipes-core/netbase/netbase-4.47/qemux86copy
> > [Add PACKAGE_ARCH_qemux86copy to meta/recipes-core/netbase/netbase_4.47.bb]
> > [Add RRECOMMENDS_${PN}_qemux86copy to meta/recipes-multimedia/gstreamer/gstreamer_0.10.35.bb]
> > 
> > You can then generate all of the sigdata files for two machine builds by
> > running the following commands. The nice thing about these is that you
> > don't have to run a complete build, it just writes out the data the
> > build would have generated:
> > 
> > rm tmp/stamps/*/*sigdata*
> > MACHINE=qemux86 bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
> > MACHINE=qemux86copy bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2
> > 
> > and then comparing the files l1 and l2, you can see which sigdata files
> > differ. Running bitbake-diffsigs on the files, e.g.:
> > 
> > bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*
> > 
> > can show what changed and you can then consider whether that is a valid
> > change and how it might be avoided if necessary. This is how some of the
> > patches I've just posted were developed.
> > 
> > Comparing qemux86 and qemux86copy as above with core-image-sato now
> > yields only two differences, in the package_write* tasks of iptables and
> > gstreamer. This is due to the dependencies these two recipes have on
> > kernel-module-* packages which in turn depend on linux-yocto which is
> > machine specific. As yet I can't find a good way to avoid the kernel
> > dependencies. If we could break debian.bbclass's hold in the
> > kernel-module case (which never get renamed), that would be one way to
> > avoid this problem.
> > 
> > I thought people might find this intersting and it was worth documenting
> > in the list archives if nothing else.
> 
> Thanks for this how-to!
> 
> Helped me to find e.g. this:
> http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=c6be068de2f716c9fdfc54970fd9908b4f2cd0e6
> 
> Now I'm looking into why busybox depends on MACHINE variable.

Ah this is because meta-oe adds systemd support which rrecommends some
kernel-modules.. so same issue as below :/.

> And interesting case is when something depends on linux-libc-headers even task-* recipe 
> which includes ie iptables (which rdepends or rrecommends some kernel modules)
> 
> Any idea why identical hash of linux-libc-headers_3.1.bb.do_package_write is 
> once compared with linux-gta04_git.bb.do_deploy and once linux-nokia900-meego_git.bb.do_package_write?
> 
> Is it bug in bitbake-diffsigs output or fault of signature handler?
> 
> om-gta02@shr ~/shr-core $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.e772dc750d69dd217f6c7dfe7de98246 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.ec74b075811b5316554475fb6360313a
> Task dependency hash changed from c38789fba09937fbb2c2ba96b13fb2e8 to 4068dabb3a553492e678b90434f6c893 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy)
> Task dependency hash changed from 9470058e15b666452c6209230c2d445b to 742657c20542706b17592336b1a0e09a (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write)
> Task dependency hash changed from e547a64f5b883431b63dcbcaa0ad1f30 to c38789fba09937fbb2c2ba96b13fb2e8 (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write)
> Task dependency hash changed from 7f72940519de5e7f16fe1ec5f7108df5 to bd3580a29e1922eacec0c798b009f8e9 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write)
> Task dependency hash changed from bcf02bbe9d89f4dff8d3c7b289202fbb to 644a5cc5cd8e2a05716bd84f694d2511 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write)
> 
> or more simple case
> $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.6cc951872c73234753749ba739be9796 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.7eb3867f1291db17de32fb142e8c9af2
> Task dependency hash changed from 4d510f798d684afd0edaf8c15720e595 to ef18cfc890c39697919a7a6f2ae3e2cb (for linux-nokia900-meego_git.bb.do_package and linux-gta04_git.bb.do_package)
> 
> So everything what somehow rdepends/reccomends on machine specific kernel should be also machine specific?
> Or can we remove kernel dependency somehow to way that we want package manager to fill use RRECOMMENDS_${PN}
> but we don't care (from sstate pov) how kernel was built. We already don't need such module to be built at all.
> 
> Regards,
> 
> -- 
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com



-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: Machine dependencies validation with sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5810 bytes --]

On Thu, Feb 16, 2012 at 02:44:47PM +0100, Martin Jansa wrote:
> On Tue, Jan 24, 2012 at 05:22:09PM +0000, Richard Purdie wrote:
> > In the process of diving into the issues in bug 1916
> > (http://bugzilla.yoctoproject.org/show_bug.cgi?id=1916), I've come up
> > with a good way of detecting machine specific changes to generic
> > PACKAGE_ARCH packages.
> > 
> > The idea is to have two identical machines, qemux86 and qemux86copy.
> > These can be setup by:
> > 
> > $ cp meta/conf/machine/qemux86.conf meta/conf/machine/qemux86copy.conf 
> > [Add SRCREV entry for qemux86copy to meta/recipes-kernel/linux/linux-yocto_3.0.bb]
> > $ cp -r meta/recipes-core/netbase/netbase-4.47/qemux86 meta/recipes-core/netbase/netbase-4.47/qemux86copy
> > [Add PACKAGE_ARCH_qemux86copy to meta/recipes-core/netbase/netbase_4.47.bb]
> > [Add RRECOMMENDS_${PN}_qemux86copy to meta/recipes-multimedia/gstreamer/gstreamer_0.10.35.bb]
> > 
> > You can then generate all of the sigdata files for two machine builds by
> > running the following commands. The nice thing about these is that you
> > don't have to run a complete build, it just writes out the data the
> > build would have generated:
> > 
> > rm tmp/stamps/*/*sigdata*
> > MACHINE=qemux86 bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l1
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l1
> > MACHINE=qemux86copy bitbake core-image-sato -S
> > find tmp/stamps/i586-poky-linux/ -name \*sigdata* | sort > l2
> > find tmp/stamps/all-poky-linux/ -name \*sigdata* | sort >> l2
> > 
> > and then comparing the files l1 and l2, you can see which sigdata files
> > differ. Running bitbake-diffsigs on the files, e.g.:
> > 
> > bitbake-diffsigs tmp/stamps/all-poky-linux/x11-common-0.1-r44.do_package_write_ipk.sigdata.*
> > 
> > can show what changed and you can then consider whether that is a valid
> > change and how it might be avoided if necessary. This is how some of the
> > patches I've just posted were developed.
> > 
> > Comparing qemux86 and qemux86copy as above with core-image-sato now
> > yields only two differences, in the package_write* tasks of iptables and
> > gstreamer. This is due to the dependencies these two recipes have on
> > kernel-module-* packages which in turn depend on linux-yocto which is
> > machine specific. As yet I can't find a good way to avoid the kernel
> > dependencies. If we could break debian.bbclass's hold in the
> > kernel-module case (which never get renamed), that would be one way to
> > avoid this problem.
> > 
> > I thought people might find this intersting and it was worth documenting
> > in the list archives if nothing else.
> 
> Thanks for this how-to!
> 
> Helped me to find e.g. this:
> http://git.shr-project.org/git/?p=meta-smartphone.git;a=commit;h=c6be068de2f716c9fdfc54970fd9908b4f2cd0e6
> 
> Now I'm looking into why busybox depends on MACHINE variable.
> 
> And interesting case is when something depends on linux-libc-headers even task-* recipe 
> which includes ie iptables (which rdepends or rrecommends some kernel modules)
> 
> Any idea why identical hash of linux-libc-headers_3.1.bb.do_package_write is 
> once compared with linux-gta04_git.bb.do_deploy and once linux-nokia900-meego_git.bb.do_package_write?
> 
> Is it bug in bitbake-diffsigs output or fault of signature handler?

patch to improve bitbake-diffsigs output for this case sent to
bitbake-devel:
http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-February/001878.html

> 
> om-gta02@shr ~/shr-core $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.e772dc750d69dd217f6c7dfe7de98246 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/ecalc-git-r1.do_build.sigdata.ec74b075811b5316554475fb6360313a
> Task dependency hash changed from c38789fba09937fbb2c2ba96b13fb2e8 to 4068dabb3a553492e678b90434f6c893 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy)
> Task dependency hash changed from 9470058e15b666452c6209230c2d445b to 742657c20542706b17592336b1a0e09a (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write)
> Task dependency hash changed from e547a64f5b883431b63dcbcaa0ad1f30 to c38789fba09937fbb2c2ba96b13fb2e8 (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write)
> Task dependency hash changed from 7f72940519de5e7f16fe1ec5f7108df5 to bd3580a29e1922eacec0c798b009f8e9 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write)
> Task dependency hash changed from bcf02bbe9d89f4dff8d3c7b289202fbb to 644a5cc5cd8e2a05716bd84f694d2511 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write)
> 
> or more simple case
> $ bitbake-diffsigs tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.6cc951872c73234753749ba739be9796 tmp-eglibc/stamps/armv7a-vfp-neon-oe-linux-gnueabi/iptables-1.4.12.2-r0.do_package_write_ipk.sigdata.7eb3867f1291db17de32fb142e8c9af2
> Task dependency hash changed from 4d510f798d684afd0edaf8c15720e595 to ef18cfc890c39697919a7a6f2ae3e2cb (for linux-nokia900-meego_git.bb.do_package and linux-gta04_git.bb.do_package)
> 
> So everything what somehow rdepends/reccomends on machine specific kernel should be also machine specific?
> Or can we remove kernel dependency somehow to way that we want package manager to fill use RRECOMMENDS_${PN}
> but we don't care (from sstate pov) how kernel was built. We already don't need such module to be built at all.
> 
> Regards,
> 
> -- 
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com



-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]