[PATCH 0/8] Various hob related patches

[PATCH 0/8] Various hob related patches

[Joshua Lock]

This pull request includes the series I submitted yesterday as well as some
additional bug fixes.

Regards,
Joshua

The following changes since commit 705d14d1e1108e0544c7eab827f1242f0839add9:

  bb/cooker: only emit ConfigFilePathFound for files which were parsed (2011-08-10 13:38:11 +0100)

are available in the git repository at:
  git://github.com/incandescant/bitbake hob
  https://github.com/incandescant/bitbake/tree/hob

Joshua Lock (8):
  bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark()
  bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list
  bb/ui/crumbs/tasklistmodel: don't include an item in its own depends
  bb/cache: rename confusing variable
  bb/command|cooker: refactor the reparseFiles logic
  bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change
  bb/ui/crumbs/configurator: introduce writeConfFile method for all
    writes
  lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

 lib/bb/cache.py                     |    6 ++--
 lib/bb/command.py                   |    7 +++++
 lib/bb/cooker.py                    |    9 ++++--
 lib/bb/ui/crumbs/configurator.py    |   46 ++++++++++++++--------------------
 lib/bb/ui/crumbs/hobeventhandler.py |    1 +
 lib/bb/ui/crumbs/hobprefs.py        |    1 +
 lib/bb/ui/crumbs/tasklistmodel.py   |   28 +++++++++++++++------
 7 files changed, 57 insertions(+), 41 deletions(-)

-- 
1.7.6

[PATCH 1/8] bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark()

[Joshua Lock]

Two similarly named variables in the mark() method resulted in the wrong
variable being used in a couple of places. This patch adresses this in
several ways:
1) Renames the variables to be less similar
2) Uses the correct variables
3) Adds some coments to document the methods intent

Partially addresses [YOCTO #1355]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/tasklistmodel.py |   22 +++++++++++++++-------
 1 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/lib/bb/ui/crumbs/tasklistmodel.py b/lib/bb/ui/crumbs/tasklistmodel.py
index 3e09757..96814c2 100644
--- a/lib/bb/ui/crumbs/tasklistmodel.py
+++ b/lib/bb/ui/crumbs/tasklistmodel.py
@@ -316,9 +316,13 @@ class TaskListModel(gtk.ListStore):
     def mark(self, opath):
         usersel = {}
         removed = []
+
         it = self.get_iter_first()
-        name = self[opath][self.COL_NAME]
+        # The name of the item we're removing, so that we can use it to find
+        # other items which either depend on it, or were brought in by it
+        marked_name = self[opath][self.COL_NAME]
 
+        # Remove the passed item
         self.remove_item_path(opath)
 
         # Remove all dependent packages, update binb
@@ -330,7 +334,7 @@ class TaskListModel(gtk.ListStore):
             deps = self[path][self.COL_DEPS]
             binb = self[path][self.COL_BINB]
             itype = self[path][self.COL_TYPE]
-            iname = self[path][self.COL_NAME]
+            itname = self[path][self.COL_NAME]
 
             # We ignore anything that isn't a package
             if not itype == "package":
@@ -341,16 +345,20 @@ class TaskListModel(gtk.ListStore):
             # is to save its name and re-mark it for inclusion once dependency
             # processing is complete
             if binb == "User Selected":
-                usersel[iname] = self[path][self.COL_IMG]
+                usersel[itname] = self[path][self.COL_IMG]
 
+            # If the iterated item is included and depends on the removed
+            # item it should also be removed.
             # FIXME: need to ensure partial name matching doesn't happen
-            if inc and deps.count(name) and name not in removed:
+            if inc and deps.count(marked_name) and itname not in removed:
                 # found a dependency, remove it
-                removed.append(name)
+                removed.append(itname)
                 self.mark(path)
 
-            if inc and binb.count(name):
-                bib = self.find_alt_dependency(name)
+            # If the iterated item was brought in by the removed (passed) item
+            # try and find an alternative dependee and update the binb column
+            if inc and binb.count(marked_name):
+                bib = self.find_alt_dependency(itname)
                 self[path][self.COL_BINB] = bib
 
         # Re-add any removed user selected items
-- 
1.7.6

[PATCH 2/8] bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list

[Joshua Lock]

Fix thinko - the squish method returns a uniquified list, it doesn't modify
the list in place. Therefore the call to squish() was useless as its return
value was never assigned.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/tasklistmodel.py |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/lib/bb/ui/crumbs/tasklistmodel.py b/lib/bb/ui/crumbs/tasklistmodel.py
index 96814c2..aec80e2 100644
--- a/lib/bb/ui/crumbs/tasklistmodel.py
+++ b/lib/bb/ui/crumbs/tasklistmodel.py
@@ -235,7 +235,8 @@ class TaskListModel(gtk.ListStore):
                         if pn:
                             depends.append(pn)
 
-            self.squish(depends)
+            # uniquify the list of depends
+            depends = self.squish(depends)
             deps = " ".join(depends)
 
             if name.count('task-') > 0:
-- 
1.7.6

[PATCH 3/8] bb/ui/crumbs/tasklistmodel: don't include an item in its own depends

[Joshua Lock]

This causes the simple removal algorithm to perform needless circular logic

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/tasklistmodel.py |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/lib/bb/ui/crumbs/tasklistmodel.py b/lib/bb/ui/crumbs/tasklistmodel.py
index aec80e2..7a463a6 100644
--- a/lib/bb/ui/crumbs/tasklistmodel.py
+++ b/lib/bb/ui/crumbs/tasklistmodel.py
@@ -237,6 +237,9 @@ class TaskListModel(gtk.ListStore):
 
             # uniquify the list of depends
             depends = self.squish(depends)
+            # remove circular dependencies
+            if name in depends:
+                depends.remove(name)
             deps = " ".join(depends)
 
             if name.count('task-') > 0:
-- 
1.7.6

[PATCH 4/8] bb/cache: rename confusing variable

[Joshua Lock]

The bNeedUpdate variable doesn't reflect its purpose, and doesn't match
coding style (type encoded in variable name, camel case) - rename to
cache_ok.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/cache.py |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/bb/cache.py b/lib/bb/cache.py
index d805d46..d495f9e 100644
--- a/lib/bb/cache.py
+++ b/lib/bb/cache.py
@@ -286,14 +286,14 @@ class Cache(object):
         old_mtimes.append(newest_mtime)
         newest_mtime = max(old_mtimes)
 
-        bNeedUpdate = True
+        cache_ok = True
         if self.caches_array:
             for cache_class in self.caches_array:
                 if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
                     cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
-                    bNeedUpdate = bNeedUpdate and (bb.parse.cached_mtime_noerror(cachefile) >= newest_mtime)
+                    cache_ok = cache_ok and (bb.parse.cached_mtime_noerror(cachefile) >= newest_mtime)
                     cache_class.init_cacheData(self)
-        if bNeedUpdate:
+        if cache_ok:
             self.load_cachefile()
         elif os.path.isfile(self.cachefile):
             logger.info("Out of date cache found, rebuilding...")
-- 
1.7.6

[PATCH 5/8] bb/command|cooker: refactor the reparseFiles logic

[Joshua Lock]

Turn the reparseFiles logic into a command to reset the cooker's state
machine and a noop which triggers a cache rebuild.

The resetCooker command resets the cookers state machine such that a cache
update will be triggered by any async command which requires the cache.
The reparseFiles command remains as a noop async command that has the
needcache property set to True so that when called it ensures the cache is
built.

Patch from Richard with the addition of removing the force parameter from
the updateCache method.

CC: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/command.py |    7 +++++++
 lib/bb/cooker.py  |    9 ++++++---
 2 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/lib/bb/command.py b/lib/bb/command.py
index 893a6d9..41796f6 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -172,6 +172,13 @@ class CommandsSync:
         value = params[1]
         bb.data.setVar(varname, value, command.cooker.configuration.data)
 
+    def resetCooker(self, command, params):
+        """
+        Reset the cooker to its initial state, thus forcing a reparse for
+        any async command that has the needcache property set to True
+        """
+        command.cooker.reset()
+
 
 class CommandsAsync:
     """
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index a8edfe7..7641d60 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1091,8 +1091,8 @@ class BBCooker:
 
         self.server_registration_cb(buildTargetsIdle, rq)
 
-    def updateCache(self, force=False):
-        if self.state == state.running and not force:
+    def updateCache(self):
+        if self.state == state.running:
             return
 
         if self.state in (state.shutdown, state.stop):
@@ -1277,8 +1277,11 @@ class BBCooker:
         self.state = state.stop
 
     def reparseFiles(self):
+        return
+
+    def reset(self):
+        self.state = state.initial
         self.loadConfigurationData()
-        self.updateCache(force=True)
 
 def server_main(cooker, func, *args):
     cooker.pre_serve()
-- 
1.7.6

[PATCH 6/8] bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change

[Joshua Lock]

The API has changed from just a reparse call to a reset call followed by
a reparse call.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/hobeventhandler.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/lib/bb/ui/crumbs/hobeventhandler.py b/lib/bb/ui/crumbs/hobeventhandler.py
index 2e0a999..1a4c7b5 100644
--- a/lib/bb/ui/crumbs/hobeventhandler.py
+++ b/lib/bb/ui/crumbs/hobeventhandler.py
@@ -120,6 +120,7 @@ class HobHandler(gobject.GObject):
                 self.current_command = self.BUILD_IMAGE
             else:
                 self.current_command = self.CFG_PATH_LAYERS
+            self.server.runCommand(["resetCooker"])
             self.server.runCommand(["reparseFiles"])
         elif self.current_command == self.BUILD_IMAGE:
             self.building = "image"
-- 
1.7.6

[PATCH 7/8] bb/ui/crumbs/configurator: introduce writeConfFile method for all writes

[Joshua Lock]

Configuration files are written in several places, this refactors the code
to use a common method.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/configurator.py |   46 +++++++++++++++----------------------
 1 files changed, 19 insertions(+), 27 deletions(-)

diff --git a/lib/bb/ui/crumbs/configurator.py b/lib/bb/ui/crumbs/configurator.py
index e17d555..c37e917 100644
--- a/lib/bb/ui/crumbs/configurator.py
+++ b/lib/bb/ui/crumbs/configurator.py
@@ -207,6 +207,19 @@ class Configurator(gobject.GObject):
 
         return "".join(layer_entry)
 
+    def writeConfFile(self, conffile, contents):
+        """
+        Make a backup copy of conffile and write a new file in its stead with
+        the lines in the contents list.
+        """
+        # Create a backup of the conf file
+        bkup = "%s~" % conffile
+        os.rename(conffile, bkup)
+
+        # Write the contents list object to the conf file
+        with open(conffile, "w") as new:
+            new.write("".join(contents))
+
     def writeLocalConf(self):
         # Dictionary containing only new or modified variables
         changed_values = {}
@@ -218,12 +231,8 @@ class Configurator(gobject.GObject):
         if not len(changed_values):
             return
 
-        # Create a backup of the local.conf
-        bkup = "%s~" % self.local
-        os.rename(self.local, bkup)
-
         # read the original conf into a list
-        with open(bkup, 'r') as config:
+        with open(self.local, 'r') as config:
             config_lines = config.readlines()
 
         new_config_lines = ["\n"]
@@ -259,20 +268,14 @@ class Configurator(gobject.GObject):
         # Add the modified variables
         config_lines.extend(new_config_lines)
 
-        # Write the updated lines list object to the local.conf
-        with open(self.local, "w") as n:
-            n.write("".join(config_lines))
+        self.writeConfFile(self.local, config_lines)
 
         del self.orig_config
         self.orig_config = copy.deepcopy(self.config)
 
     def insertTempBBPath(self, bbpath, bbfiles):
-        # Create a backup of the local.conf
-        bkup = "%s~" % self.local
-        os.rename(self.local, bkup)
-
         # read the original conf into a list
-        with open(bkup, 'r') as config:
+        with open(self.local, 'r') as config:
             config_lines = config.readlines()
 
         if bbpath:
@@ -280,9 +283,7 @@ class Configurator(gobject.GObject):
         if bbfiles:
             config_lines.append("BBFILES := \"${BBFILES} %s\"\n" % bbfiles)
 
-        # Write the updated lines list object to the local.conf
-        with open(self.local, "w") as n:
-            n.write("".join(config_lines))
+        self.writeConfFile(self.local, config_lines)
 
     def writeLayerConf(self):
         # If we've not added/removed new layers don't write
@@ -292,23 +293,14 @@ class Configurator(gobject.GObject):
         # This pattern should find the existing BBLAYERS
         pattern = 'BBLAYERS\s=\s\".*\"'
 
-        # Backup the users bblayers.conf
-        bkup = "%s~" % self.bblayers
-        os.rename(self.bblayers, bkup)
-
         replacement = self._constructLayerEntry()
 
-        with open(bkup, "r") as f:
+        with open(self.bblayers, "r") as f:
             contents = f.read()
             p = re.compile(pattern, re.DOTALL)
             new = p.sub(replacement, contents)
 
-        with open(self.bblayers, "w") as n:
-            n.write(new)
-
-        # At some stage we should remove the backup we've created
-        # though we should probably verify it first
-        #os.remove(bkup)
+        self.writeConfFile(self.bblayers, new)
 
         # set loaded_layers for dirtiness tracking
         self.loaded_layers = copy.deepcopy(self.enabled_layers)
-- 
1.7.6

[PATCH 8/8] lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

[Joshua Lock]

We need to unset the reload_required variable once we've triggered a reload
so that we don't cause a reload each time the preferences dialog is shown and
dismissed, regardless of whether anything has changed.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/hobprefs.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/lib/bb/ui/crumbs/hobprefs.py b/lib/bb/ui/crumbs/hobprefs.py
index 8ebfba2..014087b 100644
--- a/lib/bb/ui/crumbs/hobprefs.py
+++ b/lib/bb/ui/crumbs/hobprefs.py
@@ -163,6 +163,7 @@ class HobPrefs(gtk.Dialog):
     def prefs_response_cb(self, dialog, response):
         if self.reload_required:
             glib.idle_add(self.handler.reload_data)
+            self.reload_required = False
 
     def __init__(self, configurator, handler, curr_sdk_mach, curr_distro, pclass,
                  cpu_cnt, pmake, bbthread, selected_image_types, all_image_types,
-- 
1.7.6

Re: [PATCH 0/8] Various hob related patches

[Richard Purdie]

On Thu, 2011-08-11 at 15:57 -0700, Joshua Lock wrote:
> This pull request includes the series I submitted yesterday as well as some
> additional bug fixes.
> 
> Regards,
> Joshua
> 
> The following changes since commit 705d14d1e1108e0544c7eab827f1242f0839add9:
> 
>   bb/cooker: only emit ConfigFilePathFound for files which were parsed (2011-08-10 13:38:11 +0100)
> 
> are available in the git repository at:
>   git://github.com/incandescant/bitbake hob
>   https://github.com/incandescant/bitbake/tree/hob
> 
> Joshua Lock (8):
>   bb/ui/crumbs/tasklistmodel: fix some typos and add comments to mark()
>   bb/ui/crumbs/tasklistmodel: correctly uniquify dependency list
>   bb/ui/crumbs/tasklistmodel: don't include an item in its own depends
>   bb/cache: rename confusing variable
>   bb/command|cooker: refactor the reparseFiles logic
>   bb/ui/crumbs/hobeventhandler: adapt to reset -> reparse change
>   bb/ui/crumbs/configurator: introduce writeConfFile method for all
>     writes
>   lib/bb/ui/crumbs/hobprefs: fix erroneous save/reparse

Merged to master thanks.

Richard