[PATCH 0/3 V2] bb: siggen: Make dump_sigfile and compare_sigfiles print uuid4

[PATCH 0/3 V2] bb: siggen: Make dump_sigfile and compare_sigfiles print uuid4

[Robert Yang]

* V2
  - Tweak the output of diffsigs/dumpsigs to print uuid4 which makes it easier
    for debugging.
  - Add another commit "bitbake-diffsigs: Use 4 spaces as indent for recursecb"

* V1
  - Initial version

The following changes since commit c23c8ebc7f66a92110bfc9e3c4d633a432d1353b:

  core-image-sato-sdk-ptest: Try and keep image below 4GB limit (2019-04-16 11:10:02 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib rbt/sigs
  http://git.pokylinux.org/cgit.cgi//log/?h=rbt/sigs

Robert Yang (3):
  bitbake-diffsigs: Use 4 spaces as indent for recursecb
  bb: siggen: Make dump_sigfile and compare_sigfiles print uuid4
  bb: siggen: Print more info when basehash are mis-matched

 bitbake/bin/bitbake-diffsigs |  2 +-
 bitbake/lib/bb/runqueue.py   |  2 +-
 bitbake/lib/bb/siggen.py     | 16 +++++++++++++++-
 3 files changed, 17 insertions(+), 3 deletions(-)

-- 
2.7.4

[PATCH 1/3 V2] bitbake-diffsigs: Use 4 spaces as indent for recursecb

[Robert Yang]

It used 2 spaces as indent which wasn't clear enough, and might cause
confusions, people might think it was in wrong format.

Fixed:
$ bitbake bc-native -ccleansstate -Snone
$ bitbake bc-native -ccleansstate -Snone
$ bitbake-diffsigs tmp/stamps/x86_64-linux/bc-native/1.07.1-r0.do_cleansstate.sigdata.*

* Before:
Hash for dependent task bc/bc_1.07.1.bb.do_clean:virtual:native changed from [foo]
  Taint (by forced/invalidated task) changed from [foo]
Taint (by forced/invalidated task) changed from [foo]

* Now
Hash for dependent task bc/bc_1.07.1.bb.do_clean:virtual:native changed from [foo]
    Taint (by forced/invalidated task) changed from [foo]
Taint (by forced/invalidated task) changed from [foo]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/bin/bitbake-diffsigs | 2 +-
 bitbake/lib/bb/runqueue.py   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/bitbake/bin/bitbake-diffsigs b/bitbake/bin/bitbake-diffsigs
index fa430bb..73229b7 100755
--- a/bitbake/bin/bitbake-diffsigs
+++ b/bitbake/bin/bitbake-diffsigs
@@ -105,7 +105,7 @@ def recursecb(key, hash1, hash2):
         out2 = bb.siggen.compare_sigfiles(hashfiles[hash1], hashfiles[hash2], recursecb, color=color)
         for change in out2:
             for line in change.splitlines():
-                recout.append('  ' + line)
+                recout.append('    ' + line)
 
     return recout
 
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 329cda3..524c85d 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -1667,7 +1667,7 @@ class RunQueue:
             recout = []
             if len(hashfiles) == 2:
                 out2 = bb.siggen.compare_sigfiles(hashfiles[hash1], hashfiles[hash2], recursecb)
-                recout.extend(list('  ' + l for l in out2))
+                recout.extend(list('    ' + l for l in out2))
             else:
                 recout.append("Unable to find matching sigdata for %s with hashes %s or %s" % (key, hash1, hash2))
 
-- 
2.7.4

[PATCH 2/3 V2] bb: siggen: Make dump_sigfile and compare_sigfiles print uuid4

[Robert Yang]

This can make people easier to understand bitbake-diffsigs/dumpsig's output,
otherwise, it's hard to know it is a random uuid unless look into the code.

E.g.:
$ bitbake bc-native -ccleansstate -Snone
$ bitbake bc-native -ccleansstate -Snone
$ bitbake-diffsigs tmp/stamps/x86_64-linux/bc-native/1.07.1-r0.do_cleansstate.sigdata.*

* Before:
  Taint (by forced/invalidated task) changed from nostamp:fe79d162-c4a8-4174-8007-f6d4aa09abdc to nostamp:28192187-5021-40c1-9e21-45483b62c910

* Now:
  Taint (by forced/invalidated task) changed from nostamp(uuid4):fe79d162-c4a8-4174-8007-f6d4aa09abdc to nostamp(uuid4):28192187-5021-40c1-9e21-45483b62c910

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/lib/bb/siggen.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
index 09c9c8a..89bf533 100644
--- a/bitbake/lib/bb/siggen.py
+++ b/bitbake/lib/bb/siggen.py
@@ -633,6 +633,10 @@ def compare_sigfiles(a, b, recursecb=None, color=False, collapsed=False):
     a_taint = a_data.get('taint', None)
     b_taint = b_data.get('taint', None)
     if a_taint != b_taint:
+        if a_taint.startswith('nostamp:'):
+            a_taint = a_taint.replace('nostamp:', 'nostamp(uuid4):')
+        if b_taint.startswith('nostamp:'):
+            b_taint = b_taint.replace('nostamp:', 'nostamp(uuid4):')
         output.append(color_format("{color_title}Taint (by forced/invalidated task) changed{color_default} from %s to %s") % (a_taint, b_taint))
 
     return output
@@ -705,7 +709,11 @@ def dump_sigfile(a):
             output.append("Hash for dependent task %s is %s" % (dep, a_data['runtaskhashes'][dep]))
 
     if 'taint' in a_data:
-        output.append("Tainted (by forced/invalidated task): %s" % a_data['taint'])
+        if a_data['taint'].startswith('nostamp:'):
+            msg = a_data['taint'].replace('nostamp:', 'nostamp(uuid4):')
+        else:
+            msg = a_data['taint']
+        output.append("Tainted (by forced/invalidated task): %s" % msg)
 
     if 'task' in a_data:
         computed_basehash = calc_basehash(a_data)
-- 
2.7.4

[PATCH 3/3 V2] bb: siggen: Print more info when basehash are mis-matched

[Robert Yang]

This is useful for debugging.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/lib/bb/siggen.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/bitbake/lib/bb/siggen.py b/bitbake/lib/bb/siggen.py
index 89bf533..7b9f3de 100644
--- a/bitbake/lib/bb/siggen.py
+++ b/bitbake/lib/bb/siggen.py
@@ -119,6 +119,12 @@ class SignatureGeneratorBasic(SignatureGenerator):
             k = fn + "." + task
             if not ignore_mismatch and k in self.basehash and self.basehash[k] != basehash[k]:
                 bb.error("When reparsing %s, the basehash value changed from %s to %s. The metadata is not deterministic and this needs to be fixed." % (k, self.basehash[k], basehash[k]))
+                bb.error("The following commands may help:")
+                cmd = "$ bitbake %s -c%s" % (d.getVar('PN'), task)
+                # Make sure sigdata is dumped before run printdiff
+                bb.error("%s -Snone" % cmd)
+                bb.error("Then:")
+                bb.error("%s -Sprintdiff\n" % cmd)
             self.basehash[k] = basehash[k]
 
         self.taskdeps[fn] = taskdeps
-- 
2.7.4