[PATCH 00/10] BitBake GUI fixes

[PATCH 00/10] BitBake GUI fixes

[Joshua Lock]

This series of patches fixes the BitBake GUI's and some of the code they
interact with such that I can now successfully use the goggle UI with the
XMLRPC server.

The series is currently pretty useless due to bug#561
(http://bugzilla.pokylinux.org/show_bug.cgi?id=561) preventing use of the
goggle UI with the none server however I would appreciate review on the patch
set before it becomes to large.

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: josh/master
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=josh/master

Thanks,
    Joshua Lock <josh@linux.intel.com>
---


Joshua Lock (10):
  bitbake/crumbs: Fix crumbs UI for bitbake event class name changes
  bitbake/crumbs: do the test for ignored messages sooner
  bitbake/crumbs: fix the event name determination
  bitbake/goggle: interaction tweaks
  bitbake/depexp: Factor ProgressBar into a separate class in crumbs/
  bitbake/crumbs: add optional pbar parameter to
    RunningBuild.handle_event()
  bitbake/goggle: add a ProgressBar for parse progress
  bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7
  bitbake/cooker: fix idle command processing in servers
  bitbake/goggle: automatically show most recently added  message

 bitbake/lib/bb/cooker.py                 |    2 +-
 bitbake/lib/bb/server/xmlrpc.py          |   68 +++++++++++++++++++++++++++++-
 bitbake/lib/bb/ui/crumbs/progress.py     |   17 +++++++
 bitbake/lib/bb/ui/crumbs/runningbuild.py |   26 +++++++----
 bitbake/lib/bb/ui/depexp.py              |   19 ++-------
 bitbake/lib/bb/ui/goggle.py              |   15 +++++-
 6 files changed, 117 insertions(+), 30 deletions(-)
 create mode 100644 bitbake/lib/bb/ui/crumbs/progress.py

-- 
1.7.3.2

[PATCH 05/10] bitbake/depexp: Factor ProgressBar into a separate class in crumbs/

[Joshua Lock]

ProgressBar will be useful in other UI elements so make it it's own class.
Make ProgressBar a subclass of gtk.Dialog, rather than gtk.Window, so that we
can suggest the window manager parent the ProgressBar to the widget passed at
as parent.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/crumbs/progress.py |   17 +++++++++++++++++
 bitbake/lib/bb/ui/depexp.py          |   19 ++++---------------
 2 files changed, 21 insertions(+), 15 deletions(-)
 create mode 100644 bitbake/lib/bb/ui/crumbs/progress.py

diff --git a/bitbake/lib/bb/ui/crumbs/progress.py b/bitbake/lib/bb/ui/crumbs/progress.py
new file mode 100644
index 0000000..8bd8710
--- /dev/null
+++ b/bitbake/lib/bb/ui/crumbs/progress.py
@@ -0,0 +1,17 @@
+import gtk
+
+class ProgressBar(gtk.Dialog):
+    def __init__(self, parent):
+
+        gtk.Dialog.__init__(self)
+        self.set_title("Parsing metadata, please wait...")
+        self.set_default_size(500, 0)
+        self.set_transient_for(parent)
+        self.set_destroy_with_parent(True)
+        self.progress = gtk.ProgressBar()
+        self.vbox.pack_start(self.progress)
+        self.show_all()
+
+    def update(self, x, y):
+        self.progress.set_fraction(float(x)/float(y))
+        self.progress.set_text("%d/%d (%2d %%)" % (x, y, x*100/y))
diff --git a/bitbake/lib/bb/ui/depexp.py b/bitbake/lib/bb/ui/depexp.py
index 1cd58ca..6fd18d1 100644
--- a/bitbake/lib/bb/ui/depexp.py
+++ b/bitbake/lib/bb/ui/depexp.py
@@ -21,6 +21,7 @@ import gobject
 import gtk
 import threading
 import xmlrpclib
+from bb.ui.crumbs.progress import ProgressBar
 
 # Package Model
 (COL_PKG_NAME) = (0)
@@ -171,18 +172,6 @@ def parse(depgraph, pkg_model, depends_model):
                               COL_DEP_PARENT, package,
                               COL_DEP_PACKAGE, rdepend)
 
-class ProgressBar(gtk.Window):
-    def __init__(self):
-
-        gtk.Window.__init__(self)
-        self.set_title("Parsing .bb files, please wait...")
-        self.set_default_size(500, 0)
-        self.connect("delete-event", gtk.main_quit)
-
-        self.progress = gtk.ProgressBar()
-        self.add(self.progress)
-        self.show_all()
-
 class gtkthread(threading.Thread):
     quit = threading.Event()
     def __init__(self, shutdown):
@@ -217,8 +206,8 @@ def init(server, eventHandler):
     gtkgui.start()
 
     gtk.gdk.threads_enter()
-    pbar = ProgressBar()
     dep = DepExplorer()
+    pbar = ProgressBar(dep)
     gtk.gdk.threads_leave()
 
     while True:
@@ -236,9 +225,9 @@ def init(server, eventHandler):
                     print(("\nParsing finished. %d cached, %d parsed, %d skipped, %d masked, %d errors."
                         % ( event.cached, event.parsed, event.skipped, event.masked, event.errors)))
                     pbar.hide()
+                    return
                 gtk.gdk.threads_enter()
-                pbar.progress.set_fraction(float(x)/float(y))
-                pbar.progress.set_text("%d/%d (%2d %%)" % (x, y, x*100/y))
+                pbar.update(x, y)
                 gtk.gdk.threads_leave()
                 continue
 
-- 
1.7.3.2

[PATCH 06/10] bitbake/crumbs: add optional pbar parameter to RunningBuild.handle_event()

[Joshua Lock]

Defaults to None, but if set will pass the ParseProgress sofar and total to
pbar's update() method.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/crumbs/runningbuild.py |   10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diff --git a/bitbake/lib/bb/ui/crumbs/runningbuild.py b/bitbake/lib/bb/ui/crumbs/runningbuild.py
index d4ba4b7..9730bfd 100644
--- a/bitbake/lib/bb/ui/crumbs/runningbuild.py
+++ b/bitbake/lib/bb/ui/crumbs/runningbuild.py
@@ -48,7 +48,7 @@ class RunningBuild (gobject.GObject):
         gobject.GObject.__init__ (self)
         self.model = RunningBuildModel()
 
-    def handle_event (self, event):
+    def handle_event (self, event, pbar=None):
         # Handle an event from the event queue, this may result in updating
         # the model and thus the UI. Or it may be to tell us that the build
         # has finished successfully (or not, as the case may be.)
@@ -162,6 +162,14 @@ class RunningBuild (gobject.GObject):
             else:
                 self.emit ("build-succeeded")
 
+        elif isinstance(event, bb.event.ParseProgress) and pbar:
+            x = event.sofar
+            y = event.total
+            if x == y:
+                pbar.hide()
+                return
+            pbar.update(x, y)
+
 class RunningBuildTreeView (gtk.TreeView):
     def __init__ (self):
         gtk.TreeView.__init__ (self)
-- 
1.7.3.2

[PATCH 07/10] bitbake/goggle: add a ProgressBar for parse progress

[Joshua Lock]

Construct a ProgressBar and pass it to the RunningBuild.handle_event() so
that goggle users are notified of metadata parsing progress.

UI's with status make users less nervous

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/goggle.py |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/ui/goggle.py b/bitbake/lib/bb/ui/goggle.py
index d29d576..57322ee 100644
--- a/bitbake/lib/bb/ui/goggle.py
+++ b/bitbake/lib/bb/ui/goggle.py
@@ -22,13 +22,14 @@ import gobject
 import gtk
 import xmlrpclib
 from bb.ui.crumbs.runningbuild import RunningBuildTreeView, RunningBuild
+from bb.ui.crumbs.progress import ProgressBar
 
-def event_handle_idle_func (eventHandler, build):
+def event_handle_idle_func (eventHandler, build, pbar):
 
     # Consume as many messages as we can in the time available to us
     event = eventHandler.getEvent()
     while event:
-        build.handle_event (event)
+        build.handle_event (event, pbar)
         event = eventHandler.getEvent()
 
     return True
@@ -51,6 +52,7 @@ def init (server, eventHandler):
 
     window = MainWindow ()
     window.show_all ()
+    pbar = ProgressBar(window)
 
     # Create the object for the current build
     running_build = RunningBuild ()
@@ -73,6 +75,7 @@ def init (server, eventHandler):
     gobject.timeout_add (200,
                          event_handle_idle_func,
                          eventHandler,
-                         running_build)
+                         running_build,
+                         pbar)
 
     gtk.main()
-- 
1.7.3.2

[PATCH 08/10] bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7

[Joshua Lock]

Python 2.7's library changes some of xmlrpclib's internal implementation such
that interacting with a proxy to BitBakes SimpleXMLRPCServer would cause
BitBake to crash.

The issue was traced to changes in the xmlrpclib.Transport implementation and
Python bug #8194 (http://bugs.python.org/issue8194).

This patch introduces a workaround by create a subclass of
xmlrpclib.Transport, which overrides the offending methods with the Python
2.6.6 implementation copy and pasted from the Python 2.6.6 xmlrpclib, and
using this BBTransport implementation for both xmlrpclib.Server objects we
create.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/server/xmlrpc.py |   68 +++++++++++++++++++++++++++++++++++++-
 1 files changed, 66 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/server/xmlrpc.py b/bitbake/lib/bb/server/xmlrpc.py
index 5d3cc3e..c2bfe12 100644
--- a/bitbake/lib/bb/server/xmlrpc.py
+++ b/bitbake/lib/bb/server/xmlrpc.py
@@ -45,6 +45,68 @@ if sys.hexversion < 0x020600F0:
     print("Sorry, python 2.6 or later is required for bitbake's XMLRPC mode")
     sys.exit(1)
 
+##
+# The xmlrpclib.Transport class has undergone various changes in Python 2.7
+# which break BitBake's XMLRPC implementation.
+# To work around this we subclass Transport and have a copy/paste of method
+# implementations from Python 2.6.6's xmlrpclib.
+#
+# Upstream Python bug is #8194 (http://bugs.python.org/issue8194)
+##
+
+class BBTransport(xmlrpclib.Transport):
+    def request(self, host, handler, request_body, verbose=0):
+        h = self.make_connection(host)
+        if verbose:
+            h.set_debuglevel(1)
+
+        self.send_request(h, handler, request_body)
+        self.send_host(h, host)
+        self.send_user_agent(h)
+        self.send_content(h, request_body)
+
+        errcode, errmsg, headers = h.getreply()
+
+        if errcode != 200:
+            raise ProtocolError(
+                host + handler,
+                errcode, errmsg,
+                headers
+                )
+
+        self.verbose = verbose
+
+        try:
+            sock = h._conn.sock
+        except AttributeError:
+            sock = None
+
+        return self._parse_response(h.getfile(), sock)
+
+    def make_connection(self, host):
+        import httplib
+        host, extra_headers, x509 = self.get_host_info(host)
+        return httplib.HTTP(host)
+
+    def _parse_response(self, file, sock):
+        p, u = self.getparser()
+
+        while 1:
+            if sock:
+                response = sock.recv(1024)
+            else:
+                response = file.read(1024)
+            if not response:
+                break
+            if self.verbose:
+                print "body:", repr(response)
+            p.feed(response)
+
+        file.close()
+        p.close()
+
+        return u.close()
+
 class BitBakeServerCommands():
     def __init__(self, server, cooker):
         self.cooker = cooker
@@ -54,7 +116,8 @@ class BitBakeServerCommands():
         """
         Register a remote UI Event Handler
         """
-        s = xmlrpclib.Server("http://%s:%d" % (host, port), allow_none=True)
+        t = BBTransport()
+        s = xmlrpclib.Server("http://%s:%d/" % (host, port), transport=t, allow_none=True)
         return bb.event.register_UIHhandler(s)
 
     def unregisterEventHandler(self, handlerNum):
@@ -177,7 +240,8 @@ class BitbakeUILauch():
 
 class BitBakeServerConnection():
     def __init__(self, serverinfo):
-        self.connection = xmlrpclib.Server("http://%s:%s" % (serverinfo.host, serverinfo.port),  allow_none=True)
+        t = BBTransport()
+        self.connection = xmlrpclib.Server("http://%s:%s" % (serverinfo.host, serverinfo.port), transport=t, allow_none=True)
         self.events = uievent.BBUIEventQueue(self.connection)
 
     def terminate(self):
-- 
1.7.3.2

[PATCH 09/10] bitbake/cooker: fix idle command processing in servers

[Joshua Lock]

idle command processing in each of the servers does not handle an explicit
None return value, which means the goggle UI ends up repeatedly adding
"Tasks Summary:" rows to the list.

This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False
when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle.
It may be that the correct way to fix this is to change the idle command
processing in the servers.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/cooker.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index 054dac8..8c55e3e 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -734,7 +734,7 @@ class BBCooker:
             if not retval:
                 bb.event.fire(bb.event.BuildCompleted(buildname, targets, failures), self.configuration.event_data)
                 self.command.finishAsyncCommand()
-                return None
+                return False
             if retval is True:
                 return True
             return 0.5
-- 
1.7.3.2

[PATCH 10/10] bitbake/goggle: automatically show most recently added message

[Joshua Lock]

It seems likely that the user would want to view the most recently emitted
messages so this patch sets the message dislay treeview to scroll to any
newly added rows.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/goggle.py |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/bitbake/lib/bb/ui/goggle.py b/bitbake/lib/bb/ui/goggle.py
index 57322ee..858ca14 100644
--- a/bitbake/lib/bb/ui/goggle.py
+++ b/bitbake/lib/bb/ui/goggle.py
@@ -34,6 +34,9 @@ def event_handle_idle_func (eventHandler, build, pbar):
 
     return True
 
+def scroll_tv_cb (model, path, iter, view):
+    view.scroll_to_cell (path)
+
 class MainWindow (gtk.Window):
     def __init__ (self):
         gtk.Window.__init__ (self, gtk.WINDOW_TOPLEVEL)
@@ -57,6 +60,7 @@ def init (server, eventHandler):
     # Create the object for the current build
     running_build = RunningBuild ()
     window.cur_build_tv.set_model (running_build.model)
+    running_build.model.connect("row-inserted", scroll_tv_cb, window.cur_build_tv)
     try:
         cmdline = server.runCommand(["getCmdLineAction"])
         print(cmdline)
-- 
1.7.3.2

[PATCH 01/10] bitbake/crumbs: Fix crumbs UI for bitbake event class name changes

[Joshua Lock]

Some of the events we where trying to look for have had their class names
changed, fix these references.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/crumbs/runningbuild.py |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/bitbake/lib/bb/ui/crumbs/runningbuild.py b/bitbake/lib/bb/ui/crumbs/runningbuild.py
index b4416ec..6f77b83 100644
--- a/bitbake/lib/bb/ui/crumbs/runningbuild.py
+++ b/bitbake/lib/bb/ui/crumbs/runningbuild.py
@@ -67,11 +67,12 @@ class RunningBuild (gobject.GObject):
                 (package, task) = self.pids_to_task[pid]
                 parent = self.tasks_to_iter[(package, task)]
 
-        if isinstance(event, bb.msg.Msg):
+        if isinstance(event, bb.msg.MsgBase):
+
             # Set a pretty icon for the message based on it's type.
             if isinstance(event, bb.msg.MsgWarn):
                 icon = "dialog-warning"
-            elif isinstance(event, bb.msg.MsgErr):
+            elif isinstance(event, bb.msg.MsgError):
                 icon = "dialog-error"
             else:
                 icon = None
@@ -128,7 +129,7 @@ class RunningBuild (gobject.GObject):
             # Mark this task as active.
             self.model.set(i, self.model.COL_ICON, "gtk-execute")
 
-        elif isinstance(event, bb.build.Task):
+        elif isinstance(event, bb.build.TaskBase):
 
             if isinstance(event, bb.build.TaskFailed):
                 # Mark the task as failed
-- 
1.7.3.2

[PATCH 02/10] bitbake/crumbs: do the test for ignored messages sooner

[Joshua Lock]

Move the test for ignored messages to the start of the message handling loop to
avoid doing work for messages which are only going to be ignored.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/crumbs/runningbuild.py |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/bitbake/lib/bb/ui/crumbs/runningbuild.py b/bitbake/lib/bb/ui/crumbs/runningbuild.py
index 6f77b83..711697c 100644
--- a/bitbake/lib/bb/ui/crumbs/runningbuild.py
+++ b/bitbake/lib/bb/ui/crumbs/runningbuild.py
@@ -68,6 +68,9 @@ class RunningBuild (gobject.GObject):
                 parent = self.tasks_to_iter[(package, task)]
 
         if isinstance(event, bb.msg.MsgBase):
+            # Ignore the "Running task i of n .."
+            if (event._message.startswith ("Running task")):
+                return # don't add these to the list
 
             # Set a pretty icon for the message based on it's type.
             if isinstance(event, bb.msg.MsgWarn):
@@ -77,10 +80,6 @@ class RunningBuild (gobject.GObject):
             else:
                 icon = None
 
-            # Ignore the "Running task i of n .." messages
-            if (event._message.startswith ("Running task")):
-                return
-
             # Add the message to the tree either at the top level if parent is
             # None otherwise as a descendent of a task.
             self.model.append (parent,
-- 
1.7.3.2

[PATCH 03/10] bitbake/crumbs: fix the event name determination

[Joshua Lock]

Due to some recent change *somewhere* we need to explicitly look at the
name attribute on the instances class, rather than the name attribute of
the instance.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/crumbs/runningbuild.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/bitbake/lib/bb/ui/crumbs/runningbuild.py b/bitbake/lib/bb/ui/crumbs/runningbuild.py
index 711697c..d4ba4b7 100644
--- a/bitbake/lib/bb/ui/crumbs/runningbuild.py
+++ b/bitbake/lib/bb/ui/crumbs/runningbuild.py
@@ -83,7 +83,7 @@ class RunningBuild (gobject.GObject):
             # Add the message to the tree either at the top level if parent is
             # None otherwise as a descendent of a task.
             self.model.append (parent,
-                               (event.__name__.split()[-1], # e.g. MsgWarn, MsgError
+                               (event.__class__.__name__.split()[-1], # e.g. MsgWarn, MsgError
                                 package,
                                 task,
                                 event._message,
-- 
1.7.3.2

[PATCH 04/10] bitbake/goggle: interaction tweaks

[Joshua Lock]

Set the goggle window to a more sane default size (640x480) and hook up the
close button.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 bitbake/lib/bb/ui/goggle.py |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/bitbake/lib/bb/ui/goggle.py b/bitbake/lib/bb/ui/goggle.py
index 2cfa002..d29d576 100644
--- a/bitbake/lib/bb/ui/goggle.py
+++ b/bitbake/lib/bb/ui/goggle.py
@@ -41,6 +41,8 @@ class MainWindow (gtk.Window):
         scrolled_window = gtk.ScrolledWindow ()
         self.add (scrolled_window)
         self.cur_build_tv = RunningBuildTreeView()
+        self.connect("delete-event", gtk.main_quit)
+        self.set_default_size(640, 480)
         scrolled_window.add (self.cur_build_tv)
 
 def init (server, eventHandler):
-- 
1.7.3.2

Re: [PATCH 00/10] BitBake GUI fixes

[Richard Purdie]

On Mon, 2010-12-06 at 15:59 +0000, Joshua Lock wrote:
> This series of patches fixes the BitBake GUI's and some of the code they
> interact with such that I can now successfully use the goggle UI with the
> XMLRPC server.
> 
> The series is currently pretty useless due to bug#561
> (http://bugzilla.pokylinux.org/show_bug.cgi?id=561) preventing use of the
> goggle UI with the none server however I would appreciate review on the patch
> set before it becomes to large.
> 
> Pull URL: git://git.pokylinux.org/poky-contrib.git
>   Branch: josh/master
>   Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=josh/master
> 
> Thanks,
>     Joshua Lock <josh@linux.intel.com>

I merged these apart from:

http://git.pokylinux.org/cgit.cgi/poky-contrib/commit/?h=josh/master&id=69396fc01d7401d6a110ba30e7a06d2abd45e007

which as you say in the commit, I'm not sure if this is a cooker or a
server problem. I think we need to dig into and clearly define that.

Cheers,

Richard