Re: [oe-commits] Richard Purdie : bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP

Re: [oe-commits] Richard Purdie : bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 5708 bytes --]

On Tue, Nov 20, 2012 at 03:29:56PM +0000, git@git.openembedded.org wrote:
> Module: openembedded-core.git
> Branch: master
> Commit: 05075cf3138d1c61f5cf4fe0e1a4587acc00c692
> URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=05075cf3138d1c61f5cf4fe0e1a4587acc00c692
> 
> Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> Date:   Fri Nov 16 15:35:53 2012 +0000
> 
> bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP
> 
> This means some of the hacks we have to tell where the package name ends and
> the version starts in the directory layout becomes obsolete, simplifying the
> work of some of the cleanup scripts. It also makes the layout slightly more
> intuitive to the user.
> 
> It does force a rebuild onto the user but it will reuse sstate successfully.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> 
> ---
> 
>  meta/classes/sanity.bbclass |    2 ++
>  meta/conf/abi_version.conf  |    2 +-
>  meta/conf/bitbake.conf      |    6 +++---
>  3 files changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 3806bc5..5b4f070 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -605,6 +605,8 @@ def check_sanity(sanity_data):
>              f.write(current_abi)
>          elif abi == "7" and current_abi == "8":
>              messages = messages + "Your configuration is using stamp files including the sstate hash but your build directory was built with stamp files that do not include this.\nTo continue, either rebuild or switch back to the OEBasic signature handler with BB_SIGNATURE_HANDLER = 'OEBasic'.\n"
> +        elif (abi != current_abi and current_abi == "9"):
> +            messages = messages + "The layout of the TMPDIR STAMPS directory has changed. Please clean out TMPDIR and rebuild (sstate will be still be valid and reused)\n"

The part about valid and reused sstate doesn't seem to be true, if I compare 
sstate checksums with and without this patch (current master and master with 
this patch reverted), they are different:

$ bitbake-diffsigs stamps.1353438701/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers/3.4.3-r0.do_package.sigdata.9bf64da2bd20a51a9f34d0e909ea94de stamps.1353439128/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers-3.4.3-r0.do_package.sigdata.e2d8b3b59ff70b134b71f02b92f9d952
basehash changed from fd8635729e529ea26a963c47c1af647d to 0fd2c572da433536e317e1c7c12c4f60
List of dependencies for variable WORKDIR changed from set(['PR', 'PV', 'MULTIMACH_TARGET_SYS', 'PN', 'EXTENDPE', 'TMPDIR']) to set(['PF', 'TMPDIR', 'MULTIMACH_TARGET_SYS'])
changed items: set(['PR', 'PV', 'PF', 'PN', 'EXTENDPE'])
Variable WORKDIR value changed from ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} to ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}
Hash for dependent task linux-libc-headers_3.4.3.bb.do_install changed from ac25fb45e330ee0f9848210a1ad71924 to fe785052d57bae397d0ab38fa69af65d

$ bitbake-diffsigs stamps.1353438701/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers/3.4.3-r0.do_patch.sigdata.ab675d07d2d05b3621d3d68752deb1d5 stamps.1353439128/om-gta02/armv4t-oe-linux-gnueabi/linux-libc-headers-3.4.3-r0.do_patch.sigdata.1da915f84f81b15ca5f4f76176add79e
Task dependencies changed from:
['ABIEXTENSION', 'ARMPKGARCH', 'ARMPKGSFX_DSP', 'ARMPKGSFX_EABI', 'ARMPKGSFX_ENDIAN', 'ARMPKGSFX_FPU', 'ARMPKGSFX_THUMB', 'EXTENDPE', 'LIBCEXTENSION', 'MULTIMACH_TARGET_SYS', 'PACKAGE_ARCH', 'PATCHTOOL', 'PE', 'PN', 'PR', 'PV', 'S', 'SRCREV', 'TARGET_OS', 'TARGET_VENDOR', 'TUNE_PKGARCH', 'WORKDIR', 'base_do_patch', 'patch_do_patch', 'patch_path', 'should_apply', 'src_patches']
to:
['ABIEXTENSION', 'ARMPKGARCH', 'ARMPKGSFX_DSP', 'ARMPKGSFX_EABI', 'ARMPKGSFX_ENDIAN', 'ARMPKGSFX_FPU', 'ARMPKGSFX_THUMB', 'EXTENDPE', 'LIBCEXTENSION', 'MULTIMACH_TARGET_SYS', 'PACKAGE_ARCH', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'S', 'SRCREV', 'TARGET_OS', 'TARGET_VENDOR', 'TUNE_PKGARCH', 'WORKDIR', 'base_do_patch', 'patch_do_patch', 'patch_path', 'should_apply', 'src_patches']
basehash changed from 944e176cca5d006ba0306f57e85084bc to 5fb73074cd1a6bf4b5ba41f893232325
List of dependencies for variable WORKDIR changed from set(['PR', 'PV', 'MULTIMACH_TARGET_SYS', 'PN', 'EXTENDPE', 'TMPDIR']) to set(['PF', 'TMPDIR', 'MULTIMACH_TARGET_SYS'])
changed items: set(['PR', 'PV', 'PF', 'PN', 'EXTENDPE'])
Dependency on Variable PF was removed
Variable WORKDIR value changed from ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} to ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}

> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index cf779a4..450a0ee 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -337,9 +337,9 @@ GITDIR = "${CO_DIR}/git2"
>  BZRDIR = "${CO_DIR}/bzr"
>  HGDIR = "${CO_DIR}/hg"
>  
> -STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-${EXTENDPE}${PV}-${PR}"
> -STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-[0-9]*-*"
> -WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}"
> +STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
> +STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/*-*"
> +WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"

This breaks paths in all PN-dbg, as reported on previous patch doing this with
WORKDIR:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-September/030031.html

Cheers,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [oe-commits] Richard Purdie : bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP

[Richard Purdie]

On Tue, 2012-11-20 at 20:26 +0100, Martin Jansa wrote:
> On Tue, Nov 20, 2012 at 03:29:56PM +0000, git@git.openembedded.org wrote:
> > Module: openembedded-core.git
> > Branch: master
> > Commit: 05075cf3138d1c61f5cf4fe0e1a4587acc00c692
> > URL:    http://git.openembedded.org/?p=openembedded-core.git&a=commit;h=05075cf3138d1c61f5cf4fe0e1a4587acc00c692
> > 
> > Author: Richard Purdie <richard.purdie@linuxfoundation.org>
> > Date:   Fri Nov 16 15:35:53 2012 +0000
> > 
> > bitbake.conf/sanity: Separate versions and PN stamp components into separate directories for WORKDIR and STAMP
> > 
> > This means some of the hacks we have to tell where the package name ends and
> > the version starts in the directory layout becomes obsolete, simplifying the
> > work of some of the cleanup scripts. It also makes the layout slightly more
> > intuitive to the user.
> > 
> > It does force a rebuild onto the user but it will reuse sstate successfully.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > 
> > ---
> > 
> >  meta/classes/sanity.bbclass |    2 ++
> >  meta/conf/abi_version.conf  |    2 +-
> >  meta/conf/bitbake.conf      |    6 +++---
> >  3 files changed, 6 insertions(+), 4 deletions(-)
> > 
> > diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> > index 3806bc5..5b4f070 100644
> > --- a/meta/classes/sanity.bbclass
> > +++ b/meta/classes/sanity.bbclass
> > @@ -605,6 +605,8 @@ def check_sanity(sanity_data):
> >              f.write(current_abi)
> >          elif abi == "7" and current_abi == "8":
> >              messages = messages + "Your configuration is using stamp files including the sstate hash but your build directory was built with stamp files that do not include this.\nTo continue, either rebuild or switch back to the OEBasic signature handler with BB_SIGNATURE_HANDLER = 'OEBasic'.\n"
> > +        elif (abi != current_abi and current_abi == "9"):
> > +            messages = messages + "The layout of the TMPDIR STAMPS directory has changed. Please clean out TMPDIR and rebuild (sstate will be still be valid and reused)\n"
> 
> The part about valid and reused sstate doesn't seem to be true, if I compare 
> sstate checksums with and without this patch (current master and master with 
> this patch reverted), they are different:

They are indeed, sorry about that :/.

> > -STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-${EXTENDPE}${PV}-${PR}"
> > -STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}-[0-9]*-*"
> > -WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PF}"
> > +STAMP = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
> > +STAMPCLEAN = "${TMPDIR}/stamps/${MULTIMACH_TARGET_SYS}/${PN}/*-*"
> > +WORKDIR = "${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}"
> 
> This breaks paths in all PN-dbg, as reported on previous patch doing this with
> WORKDIR:
> http://lists.linuxtogo.org/pipermail/openembedded-core/2012-September/030031.html

I pushed a fix for this earlier in the week, thanks for reminding me
about that problem, I had forgotten about it :(.

Cheers,

Richard