[PATCH] migrate_localcount.bbclass: use PRAUTOINX instead of PN & PV

[PATCH] migrate_localcount.bbclass: use PRAUTOINX instead of PN & PV

[Constantin Musca]

- we need to use PRAUTOINX (it covers all cases)
- it addresses bumped PE issues

[YOCTO #3071]

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
---
 meta/classes/migrate_localcount.bbclass | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/meta/classes/migrate_localcount.bbclass b/meta/classes/migrate_localcount.bbclass
index 59f14e8..3916ad4 100644
--- a/meta/classes/migrate_localcount.bbclass
+++ b/meta/classes/migrate_localcount.bbclass
@@ -18,14 +18,13 @@ python migrate_localcount_handler () {
         if not revs or not counts:
             return
 
-        srcrev = bb.fetch2.get_srcrev(e.data)
-        bpv = pv[:pv.find(srcrev)]
-
         if len(revs) != len(counts):
             bb.warn("The number of revs and localcounts don't match in %s" % pn)
             return
 
-        version = 'AUTOINC-%s-%s' % (pn, bpv)
+        version = e.data.getVar('PRAUTOINX', True)
+        srcrev = bb.fetch2.get_srcrev(e.data)
+        base_ver = 'AUTOINC-%s' % version[:version.find(srcrev)]
         pkgarch = e.data.getVar('PACKAGE_ARCH', True)
         value = max(int(count) for count in counts)
 
@@ -40,7 +39,7 @@ python migrate_localcount_handler () {
         flock = bb.utils.lockfile("%s.lock" % df)
         with open(df, 'a') as fd:
             fd.write('PRAUTO$%s$%s$%s = "%s"\n' %
-                    (version, pkgarch, srcrev, str(value)))
+                    (base_ver, pkgarch, srcrev, str(value)))
         bb.utils.unlockfile(flock)
 }
 
-- 
1.7.11.7

Re: [PATCH] migrate_localcount.bbclass: use PRAUTOINX instead of PN & PV

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]

On Mon, Jan 28, 2013 at 11:01:20AM +0200, Constantin Musca wrote:
> - we need to use PRAUTOINX (it covers all cases)
> - it addresses bumped PE issues

Build is still runing, but entries in prserv db look correct now,
thanks!

Cheers,

> 
> [YOCTO #3071]
> 
> Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
> ---
>  meta/classes/migrate_localcount.bbclass | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/classes/migrate_localcount.bbclass b/meta/classes/migrate_localcount.bbclass
> index 59f14e8..3916ad4 100644
> --- a/meta/classes/migrate_localcount.bbclass
> +++ b/meta/classes/migrate_localcount.bbclass
> @@ -18,14 +18,13 @@ python migrate_localcount_handler () {
>          if not revs or not counts:
>              return
>  
> -        srcrev = bb.fetch2.get_srcrev(e.data)
> -        bpv = pv[:pv.find(srcrev)]
> -
>          if len(revs) != len(counts):
>              bb.warn("The number of revs and localcounts don't match in %s" % pn)
>              return
>  
> -        version = 'AUTOINC-%s-%s' % (pn, bpv)
> +        version = e.data.getVar('PRAUTOINX', True)
> +        srcrev = bb.fetch2.get_srcrev(e.data)
> +        base_ver = 'AUTOINC-%s' % version[:version.find(srcrev)]
>          pkgarch = e.data.getVar('PACKAGE_ARCH', True)
>          value = max(int(count) for count in counts)
>  
> @@ -40,7 +39,7 @@ python migrate_localcount_handler () {
>          flock = bb.utils.lockfile("%s.lock" % df)
>          with open(df, 'a') as fd:
>              fd.write('PRAUTO$%s$%s$%s = "%s"\n' %
> -                    (version, pkgarch, srcrev, str(value)))
> +                    (base_ver, pkgarch, srcrev, str(value)))
>          bb.utils.unlockfile(flock)
>  }
>  
> -- 
> 1.7.11.7
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH] migrate_localcount.bbclass: use PRAUTOINX instead of PN & PV

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3088 bytes --]

On Mon, Jan 28, 2013 at 01:56:37PM +0100, Martin Jansa wrote:
> On Mon, Jan 28, 2013 at 11:01:20AM +0200, Constantin Musca wrote:
> > - we need to use PRAUTOINX (it covers all cases)
> > - it addresses bumped PE issues
> 
> Build is still runing, but entries in prserv db look correct now,
> thanks!

It's already too late for my local build, but maybe it would be
interesting for people still using OEBasic (instead of default
OEBasicHash) to use PRSERV _only_ as LOCALCOUNT replacement as that was
removed completely.

Now I see all packages are upgraded on target because of extra .0 
in PR.

And it will stay constant .0 with OEBasic if I understood RP's answer
correctly.

"The service works with both OEBasic and OEBasicHash
generators, with the understanding that PR bumps happen when the
checksum changes and the different generator mechanisms change
signatures under different circumstances."

Only manual PE/PV/PR bump will change OEBasic signature, but also use 
different key in PRMAIN_nohist, so again with '.0'.

Cheers,

> 
> Cheers,
> 
> > 
> > [YOCTO #3071]
> > 
> > Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
> > ---
> >  meta/classes/migrate_localcount.bbclass | 9 ++++-----
> >  1 file changed, 4 insertions(+), 5 deletions(-)
> > 
> > diff --git a/meta/classes/migrate_localcount.bbclass b/meta/classes/migrate_localcount.bbclass
> > index 59f14e8..3916ad4 100644
> > --- a/meta/classes/migrate_localcount.bbclass
> > +++ b/meta/classes/migrate_localcount.bbclass
> > @@ -18,14 +18,13 @@ python migrate_localcount_handler () {
> >          if not revs or not counts:
> >              return
> >  
> > -        srcrev = bb.fetch2.get_srcrev(e.data)
> > -        bpv = pv[:pv.find(srcrev)]
> > -
> >          if len(revs) != len(counts):
> >              bb.warn("The number of revs and localcounts don't match in %s" % pn)
> >              return
> >  
> > -        version = 'AUTOINC-%s-%s' % (pn, bpv)
> > +        version = e.data.getVar('PRAUTOINX', True)
> > +        srcrev = bb.fetch2.get_srcrev(e.data)
> > +        base_ver = 'AUTOINC-%s' % version[:version.find(srcrev)]
> >          pkgarch = e.data.getVar('PACKAGE_ARCH', True)
> >          value = max(int(count) for count in counts)
> >  
> > @@ -40,7 +39,7 @@ python migrate_localcount_handler () {
> >          flock = bb.utils.lockfile("%s.lock" % df)
> >          with open(df, 'a') as fd:
> >              fd.write('PRAUTO$%s$%s$%s = "%s"\n' %
> > -                    (version, pkgarch, srcrev, str(value)))
> > +                    (base_ver, pkgarch, srcrev, str(value)))
> >          bb.utils.unlockfile(flock)
> >  }
> >  
> > -- 
> > 1.7.11.7
> > 
> > 
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
> -- 
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com



-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH] migrate_localcount.bbclass: use PRAUTOINX instead of PN & PV

[Richard Purdie]

On Mon, 2013-01-28 at 15:35 +0100, Martin Jansa wrote:
> On Mon, Jan 28, 2013 at 01:56:37PM +0100, Martin Jansa wrote:
> > On Mon, Jan 28, 2013 at 11:01:20AM +0200, Constantin Musca wrote:
> > > - we need to use PRAUTOINX (it covers all cases)
> > > - it addresses bumped PE issues
> > 
> > Build is still runing, but entries in prserv db look correct now,
> > thanks!
> 
> It's already too late for my local build, but maybe it would be
> interesting for people still using OEBasic (instead of default
> OEBasicHash) to use PRSERV _only_ as LOCALCOUNT replacement as that was
> removed completely.
> 
> Now I see all packages are upgraded on target because of extra .0 
> in PR.
> 
> And it will stay constant .0 with OEBasic if I understood RP's answer
> correctly.
> 
> "The service works with both OEBasic and OEBasicHash
> generators, with the understanding that PR bumps happen when the
> checksum changes and the different generator mechanisms change
> signatures under different circumstances."

It won't just stick as .0. The underlying hashes change a fair bit with
both variants. The OEBasic version just doesn't inject them into stamps.
I therefore don't think this is going to work well for your use case.

> Only manual PE/PV/PR bump will change OEBasic signature, but also use 
> different key in PRMAIN_nohist, so again with '.0'.

I've been giving the OEBasic situation a bit of thought and the split
between OEBasic and OEBasicHash probably doesn't make sense. As things
stand, OEBasic is probably going to become less and less useful. How
about we redesign OEBasic so that:

a) It uses hashes as part of the stamp filenames (same as OEBasicHash)
b) It doesn't include other tasks outside of the current recipe in the 
   hash construction.

This means it will behave like classic OE with things not rebuilding
that often. However certain classes of changes will get accounted for
and those are probably useful and make sense. For example:

a) If you edit patches of a given recipe it will rebuild
b) If you change a task like do_configure, things after it will rerun in
a given recipe.
c) If a dependent task in another recipe changes, it won't rerun.

I *think* the changes to do this are simple:

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 79a410e..11c4208 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -1,5 +1,11 @@
 import bb.siggen
 
+def sstate_rundepfilter_basic(siggen, fn, recipename, task, dep, depname, dataCache):
+    # Always include our own inter-task dependencies
+    if recipename == depname:
+        return True
+    return False
+
 def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     # Return True if we should keep the dependency, False to drop it
     def isNative(x):
@@ -53,14 +59,14 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     # Default to keep dependencies
     return True
 
-class SignatureGeneratorOEBasic(bb.siggen.SignatureGeneratorBasic):
+class SignatureGeneratorOEBasic(bb.siggen.SignatureGeneratorBasicHash):
     name = "OEBasic"
     def init_rundepcheck(self, data):
         self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE", True) or "").split()
         self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS", True) or "").split()
         pass
     def rundep_check(self, fn, recipename, task, dep, depname, dataCache = None):
-        return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCache)
+        return sstate_rundepfilter_basic(self, fn, recipename, task, dep, depname, dataCache)
 
 class SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
     name = "OEBasicHash"

I'd be interested to see if this works better for people's use cases. It
will need a clean tmpdir as the stamps directory layout changes with
this patch.

Cheers,

Richard