[PATCH 0/4] BitBake signature tool improvements

[PATCH 0/4] BitBake signature tool improvements

[Paul Eggleton]

Make bitbake-diffsigs and bitbake-dumpsig a little easier to use.

The following changes (against poky, but apply cleanly against bitbake
master with -p2) are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib paule/bb-sig-tools
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/bb-sig-tools

Paul Eggleton (4):
  bitbake-diffsigs: handle if task name is specified without do_ prefix
  bitbake-diffsigs: refactor argument parsing slightly
  bitbake-diffsigs: improve error handling
  bitbake-dumpsig: introduce command line and error handling

 bitbake/bin/bitbake-diffsigs | 57 ++++++++++++++++++++++++++++-------------
 bitbake/bin/bitbake-dumpsig  | 60 +++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 96 insertions(+), 21 deletions(-)

-- 
1.8.1.2

[PATCH 1/4] bitbake-diffsigs: handle if task name is specified without do_ prefix

[Paul Eggleton]

Warn the user and adjust the task name automatically if the -t option
is specified with a task name that doesn't start with do_ (e.g.
"configure" instead of "do_configure").

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/bin/bitbake-diffsigs | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/bitbake/bin/bitbake-diffsigs b/bitbake/bin/bitbake-diffsigs
index 3ce70da..6e4c44a 100755
--- a/bitbake/bin/bitbake-diffsigs
+++ b/bitbake/bin/bitbake-diffsigs
@@ -39,6 +39,10 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data files')
         sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        logger.warn('Invalid task name "%s" - assuming you meant "do_%s"' % (taskname, taskname))
+        taskname = 'do_%s' % taskname
+
     filedates = bb.siggen.find_siginfo(pn, taskname, None, bbhandler.config_data)
     latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
     if not latestfiles:
-- 
1.8.1.2

[PATCH 2/4] bitbake-diffsigs: refactor argument parsing slightly

[Paul Eggleton]

* Use OptionParser to parse the two options to -t rather than trying to
  pick them out ourselves.
* Add a description shown with --help output

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/bin/bitbake-diffsigs | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/bitbake/bin/bitbake-diffsigs b/bitbake/bin/bitbake-diffsigs
index 6e4c44a..6f24981 100755
--- a/bitbake/bin/bitbake-diffsigs
+++ b/bitbake/bin/bitbake-diffsigs
@@ -75,6 +75,7 @@ def find_compare_task(bbhandler, pn, taskname):
 
 
 parser = optparse.OptionParser(
+    description = "Compares siginfo/sigdata files written out by BitBake",
     usage = """
   %prog -t recipename taskname
   %prog sigdatafile1 sigdatafile2
@@ -82,25 +83,21 @@ parser = optparse.OptionParser(
 
 parser.add_option("-t", "--task",
         help = "find the signature data files for last two runs of the specified task and compare them",
-        action="store_true", dest="taskmode")
+        action="store", dest="taskargs", nargs=2, metavar='recipename taskname')
 
 options, args = parser.parse_args(sys.argv)
 
-if len(args) == 1:
-    parser.print_help()
+if options.taskargs:
+    tinfoil = bb.tinfoil.Tinfoil()
+    tinfoil.prepare(config_only = True)
+    find_compare_task(tinfoil, options.taskargs[0], options.taskargs[1])
 else:
-    if options.taskmode:
-        tinfoil = bb.tinfoil.Tinfoil()
-        if len(args) < 3:
-            logger.error("Please specify a recipe and task name")
-            sys.exit(1)
-        tinfoil.prepare(config_only = True)
-        find_compare_task(tinfoil, args[1], args[2])
+    if len(args) == 1:
+        parser.print_help()
+    elif len(args) == 2:
+        output = bb.siggen.dump_sigfile(sys.argv[1])
     else:
-        if len(args) == 2:
-            output = bb.siggen.dump_sigfile(sys.argv[1])
-        else:
-            output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
 
-        if output:
-            print '\n'.join(output)
+    if output:
+        print '\n'.join(output)
-- 
1.8.1.2

[PATCH 3/4] bitbake-diffsigs: improve error handling

[Paul Eggleton]

* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/bin/bitbake-diffsigs | 30 +++++++++++++++++++++++++-----
 1 file changed, 25 insertions(+), 5 deletions(-)

diff --git a/bitbake/bin/bitbake-diffsigs b/bitbake/bin/bitbake-diffsigs
index 6f24981..53b47e0 100755
--- a/bitbake/bin/bitbake-diffsigs
+++ b/bitbake/bin/bitbake-diffsigs
@@ -3,7 +3,7 @@
 # bitbake-diffsigs
 # BitBake task signature data comparison utility
 #
-# Copyright (C) 2012 Intel Corporation
+# Copyright (C) 2013 Intel Corporation
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 2 as
@@ -30,7 +30,18 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
 import bb.tinfoil
 import bb.siggen
 
-logger = logging.getLogger('BitBake')
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-diffsigs')
 
 def find_compare_task(bbhandler, pn, taskname):
     """ Find the most recent signature files for the specified PN/task and compare them """
@@ -94,10 +105,19 @@ if options.taskargs:
 else:
     if len(args) == 1:
         parser.print_help()
-    elif len(args) == 2:
-        output = bb.siggen.dump_sigfile(sys.argv[1])
     else:
-        output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        import cPickle
+        try:
+            if len(args) == 2:
+                output = bb.siggen.dump_sigfile(sys.argv[1])
+            else:
+                output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        except IOError as e:
+            logger.error(str(e))
+            sys.exit(1)
+        except cPickle.UnpicklingError, EOFError:
+            logger.error('Invalid signature data - ensure you are specifying sigdata/siginfo files')
+            sys.exit(1)
 
     if output:
         print '\n'.join(output)
-- 
1.8.1.2

[PATCH 4/4] bitbake-dumpsig: introduce command line and error handling

[Paul Eggleton]

This utility doesn't take any special arguments, but it's nice if it at
least knows how to deal with no arguments, --help and errors properly.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/bin/bitbake-dumpsig | 60 ++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 57 insertions(+), 3 deletions(-)

diff --git a/bitbake/bin/bitbake-dumpsig b/bitbake/bin/bitbake-dumpsig
index ccbc412..656d93a 100755
--- a/bitbake/bin/bitbake-dumpsig
+++ b/bitbake/bin/bitbake-dumpsig
@@ -1,11 +1,65 @@
 #!/usr/bin/env python
+
+# bitbake-dumpsig
+# BitBake task signature dump utility
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
 import os
 import sys
 import warnings
+import optparse
+import logging
+
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), 'lib'))
 
 import bb.siggen
 
-output = bb.siggen.dump_sigfile(sys.argv[1])
-if output:
-    print '\n'.join(output)
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-dumpsig')
+
+parser = optparse.OptionParser(
+    description = "Dumps siginfo/sigdata files written out by BitBake",
+    usage = """
+  %prog sigdatafile""")
+
+options, args = parser.parse_args(sys.argv)
+
+if len(args) == 1:
+    parser.print_help()
+else:
+    import cPickle
+    try:
+        output = bb.siggen.dump_sigfile(args[1])
+    except IOError as e:
+        logger.error(str(e))
+        sys.exit(1)
+    except cPickle.UnpicklingError, EOFError:
+        logger.error('Invalid signature data - ensure you are specifying a sigdata/siginfo file')
+        sys.exit(1)
+
+    if output:
+        print '\n'.join(output)
-- 
1.8.1.2

Re: [PATCH 1/4] bitbake-diffsigs: handle if task name is specified without do_ prefix

[Paul Eggleton]

On Friday 04 October 2013 17:44:25 Paul Eggleton wrote:
> Warn the user and adjust the task name automatically if the -t option
> is specified with a task name that doesn't start with do_ (e.g.
> "configure" instead of "do_configure").
> 
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> ---
>  bitbake/bin/bitbake-diffsigs | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/bitbake/bin/bitbake-diffsigs b/bitbake/bin/bitbake-diffsigs
> index 3ce70da..6e4c44a 100755
> --- a/bitbake/bin/bitbake-diffsigs
> +++ b/bitbake/bin/bitbake-diffsigs
> @@ -39,6 +39,10 @@ def find_compare_task(bbhandler, pn, taskname):
>          logger.error('Metadata does not support finding signature data
> files') sys.exit(1)
> 
> +    if not taskname.startswith('do_'):
> +        logger.warn('Invalid task name "%s" - assuming you meant "do_%s"' %
> (taskname, taskname)) +        taskname = 'do_%s' % taskname
> +
>      filedates = bb.siggen.find_siginfo(pn, taskname, None,
> bbhandler.config_data) latestfiles = sorted(filedates.keys(), key=lambda f:
> filedates[f])[-2:] if not latestfiles:

I've dropped the warning message on the branch after discussion with Richard - 
the do_ prefix is really an internal detail and users are used to specifying 
these without the prefix when using bitbake -c.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre