* [PATCH 1/2] lib/oe/sstatesig: fix finding native siginfo files in sstate-cache
2017-04-06 22:01 [PATCH 0/2] OE-side fixes for bitbake-diffsigs Paul Eggleton
@ 2017-04-06 22:01 ` Paul Eggleton
2017-04-06 22:01 ` [PATCH 2/2] lib/oe/sstatesig: avoid reporting duplicate siginfo files from sstate Paul Eggleton
1 sibling, 0 replies; 3+ messages in thread
From: Paul Eggleton @ 2017-04-06 22:01 UTC (permalink / raw)
To: openembedded-core
When comparing signatures with bitbake-diffsigs -t or bitbake -S
printdiff, we use this find_siginfo() function implemented in this
module to find the siginfo/sigdata files corresponding to the tasks
we're looking for. However, native sstate files go into a
NATIVELSBSTRING subdirectory and there was no handling for this when
asking about native recipes.
I'm not even sure why we were walking SSTATE_DIR in order to find
this - we don't need to, we just need to run glob.glob() on the filespec
we calculate, which should be a little bit more efficient.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/sstatesig.py | 25 +++++++++----------------
1 file changed, 9 insertions(+), 16 deletions(-)
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 13fd3bd..a76a031 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -318,22 +318,15 @@ def find_siginfo(pn, taskname, taskhashlist, d):
sstatename = taskname[3:]
filespec = '%s_%s.*.siginfo' % (localdata.getVar('SSTATE_PKG'), sstatename)
- if hashval != '*':
- sstatedir = "%s/%s" % (d.getVar('SSTATE_DIR'), hashval[:2])
- else:
- sstatedir = d.getVar('SSTATE_DIR')
-
- for root, dirs, files in os.walk(sstatedir):
- for fn in files:
- fullpath = os.path.join(root, fn)
- if fnmatch.fnmatch(fullpath, filespec):
- if taskhashlist:
- hashfiles[hashval] = fullpath
- else:
- try:
- filedates[fullpath] = os.stat(fullpath).st_mtime
- except:
- continue
+ matchedfiles = glob.glob(filespec)
+ for fullpath in matchedfiles:
+ if taskhashlist:
+ hashfiles[hashval] = fullpath
+ else:
+ try:
+ filedates[fullpath] = os.stat(fullpath).st_mtime
+ except:
+ continue
if taskhashlist:
return hashfiles
--
2.9.3
^ permalink raw reply related [flat|nested] 3+ messages in thread* [PATCH 2/2] lib/oe/sstatesig: avoid reporting duplicate siginfo files from sstate
2017-04-06 22:01 [PATCH 0/2] OE-side fixes for bitbake-diffsigs Paul Eggleton
2017-04-06 22:01 ` [PATCH 1/2] lib/oe/sstatesig: fix finding native siginfo files in sstate-cache Paul Eggleton
@ 2017-04-06 22:01 ` Paul Eggleton
1 sibling, 0 replies; 3+ messages in thread
From: Paul Eggleton @ 2017-04-06 22:01 UTC (permalink / raw)
To: openembedded-core
In find_siginfo(), which is used by bitbake-diffsigs among other things,
avoid adding a siginfo file from the sstate-cache where we've already
collected a sigdata file from the stamps directory with the same hash.
This avoids the possibility that the top two files (as picked by default
using the bitbake-diffsigs -t option) are for the same signature and
thus the tool would report no differences. In order to do that, just use
the hashfiles dict that we already have - we just need to change the
code to populate that even if we're collecting matching files without
looking for a fixed set of hashes (i.e. taskhashlist isn't set).
This replaces previous code in bitbake-diffsigs that attempted to filter
these out with limited success.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/sstatesig.py | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index a76a031..8d93013 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -254,9 +254,6 @@ def find_siginfo(pn, taskname, taskhashlist, d):
import fnmatch
import glob
- if taskhashlist:
- hashfiles = {}
-
if not taskname:
# We have to derive pn and taskname
key = pn
@@ -266,8 +263,15 @@ def find_siginfo(pn, taskname, taskhashlist, d):
if key.startswith('virtual:native:'):
pn = pn + '-native'
+ hashfiles = {}
filedates = {}
+ def get_hashval(siginfo):
+ if siginfo.endswith('.siginfo'):
+ return siginfo.rpartition(':')[2].partition('_')[0]
+ else:
+ return siginfo.rpartition('.')[2]
+
# First search in stamps dir
localdata = d.createCopy()
localdata.setVar('MULTIMACH_TARGET_SYS', '*')
@@ -297,6 +301,8 @@ def find_siginfo(pn, taskname, taskhashlist, d):
filedates[fullpath] = os.stat(fullpath).st_mtime
except OSError:
continue
+ hashval = get_hashval(fullpath)
+ hashfiles[hashval] = fullpath
if not taskhashlist or (len(filedates) < 2 and not foundall):
# That didn't work, look in sstate-cache
@@ -320,9 +326,11 @@ def find_siginfo(pn, taskname, taskhashlist, d):
matchedfiles = glob.glob(filespec)
for fullpath in matchedfiles:
- if taskhashlist:
- hashfiles[hashval] = fullpath
- else:
+ actual_hashval = get_hashval(fullpath)
+ if actual_hashval in hashfiles:
+ continue
+ hashfiles[hashval] = fullpath
+ if not taskhashlist:
try:
filedates[fullpath] = os.stat(fullpath).st_mtime
except:
--
2.9.3
^ permalink raw reply related [flat|nested] 3+ messages in thread