[RFC PATCH 0/3] Tracking parsed configuration files

[RFC PATCH 0/3] Tracking parsed configuration files

[Joshua Lock]

All,

This series attempts to address Yocto bug 1246: "Configuration changes could
be saved to a file the cooker isn't using".

It's an ugly series and the issue will only present itself if users run the
UI trough alternative ways. My main issue is that we can't easily track all
loaded conf files at parseConfigurationFiles time so it feels a little
brittle.

Thoughts? Comments?

Regards,
Joshua

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 2650be190afc05f9472aca8b11af99205a342838:

  ui/crumbs/tasklistmodel: fix loading a saved recipe (2011-07-26 09:42:28 -0700)

are available in the git repository at:
  git://github.com/incandescant/bitbake configfiles
  https://github.com/incandescant/bitbake/tree/configfiles

Joshua Lock (3):
  cooker: track conf files parsed outside of the usual bitbake.conf
    inclusion
  cooker|event: notify if pre/post file in ConfigFilePathFound event
  ui/crumbs/[hobeventhandler|configurator]: don't use unparsed files

 lib/bb/cooker.py                    |   10 +++++++++-
 lib/bb/event.py                     |    3 ++-
 lib/bb/ui/crumbs/configurator.py    |    8 ++++----
 lib/bb/ui/crumbs/hobeventhandler.py |    6 ++++--
 4 files changed, 19 insertions(+), 8 deletions(-)

-- 
1.7.6

[RFC PATCH 1/3] cooker: track conf files parsed outside of the usual bitbake.conf inclusion

[Joshua Lock]

Save a list of configuration files loaded pre and post parse of bitbake.conf

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/cooker.py |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 1f305d9..765c58d 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -82,6 +82,7 @@ class BBCooker:
         self.status = None
         self.appendlist = {}
         self.skiplist = {}
+        self.extraconf = []
 
         self.server_registration_cb = server_registration_cb
 
@@ -741,6 +742,7 @@ class BBCooker:
         # Parse files for loading *before* bitbake.conf and any includes
         for f in prefiles:
             data = _parse(f, data)
+            self.extraconf.append(f)
 
         layerconf = self._findLayerConf()
         if layerconf:
@@ -766,6 +768,7 @@ class BBCooker:
         # Parse files for loading *after* bitbake.conf and any includes
         for p in postfiles:
             data = _parse(p, data)
+            self.extraconf.append(p)
 
         # Handle any INHERITs and inherit the base class
         bbclasses  = ["base"] + (data.getVar('INHERIT', True) or "").split()
-- 
1.7.6

[RFC PATCH 2/3] cooker|event: notify if pre/post file in ConfigFilePathFound event

[Joshua Lock]

When findConfigFilePath called the emitted ConfigFilePathFound event will
also include a boolean value indiciating whether the specified config file
was loaded using the prefile or postfile options to BitBake.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/cooker.py |    7 ++++++-
 lib/bb/event.py  |    3 ++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 765c58d..f0c07b5 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -605,9 +605,14 @@ class BBCooker:
                 collectlog.warn("No bb files matched BBFILE_PATTERN_%s '%s'" % (collection, pattern))
 
     def findConfigFilePath(self, configfile):
+        additional = False
         path = self._findConfigFile(configfile)
         if path:
-            bb.event.fire(bb.event.ConfigFilePathFound(path), self.configuration.data)
+            for path in self.extraconf:
+                _, cf, _ = path.rpartition(configfile)
+                if cf:
+                    additional = True
+            bb.event.fire(bb.event.ConfigFilePathFound(path, additional), self.configuration.data)
 
     def findFilesMatchingInDir(self, filepattern, directory):
         """
diff --git a/lib/bb/event.py b/lib/bb/event.py
index 93c04ba..01700dc 100644
--- a/lib/bb/event.py
+++ b/lib/bb/event.py
@@ -412,9 +412,10 @@ class ConfigFilePathFound(Event):
     """
     Event when a path for a config file has been found
     """
-    def __init__(self, path):
+    def __init__(self, path, extraconf):
         Event.__init__(self)
         self._path = path
+        self._extraconf = extraconf
 
 class MsgBase(Event):
     """Base class for messages"""
-- 
1.7.6

[RFC PATCH 3/3] ui/crumbs/[hobeventhandler|configurator]: don't use unparsed files

[Joshua Lock]

Use the new field in the ConfigFilePathFound event to track whether the
hob.local.conf file was parsed when BitBake was launched.

Fixes [YOCTO #1246]

Signed-off-by: Joshua Lock <josh@linux.intel.com>
---
 lib/bb/ui/crumbs/configurator.py    |    8 ++++----
 lib/bb/ui/crumbs/hobeventhandler.py |    6 ++++--
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/bb/ui/crumbs/configurator.py b/lib/bb/ui/crumbs/configurator.py
index ec48a4f..f5eae28 100644
--- a/lib/bb/ui/crumbs/configurator.py
+++ b/lib/bb/ui/crumbs/configurator.py
@@ -115,9 +115,9 @@ class Configurator(gobject.GObject):
         self.enabled_layers = copy.deepcopy(self.loaded_layers)
         self.emit("layers-loaded")
 
-    def _addConfigFile(self, path):
+    def _addConfigFile(self, path, additional):
         pref, sep, filename = path.rpartition("/")
-        if filename == "local.conf" or filename == "hob.local.conf":
+        if filename == "local.conf" or (filename == "hob.local.conf" and additional):
             self._loadLocalConf(path)
         elif filename == "bblayers.conf":
             self._loadLayerConf(path)
@@ -293,8 +293,8 @@ class Configurator(gobject.GObject):
 
         self.emit("layers-changed")
 
-    def configFound(self, handler, path):
-        self._addConfigFile(path)
+    def configFound(self, handler, path, additional):
+        self._addConfigFile(path, additional)
 
     def loadConfig(self, path):
         self._addConfigFile(path)
diff --git a/lib/bb/ui/crumbs/hobeventhandler.py b/lib/bb/ui/crumbs/hobeventhandler.py
index 4897bcc..9cfa7c2 100644
--- a/lib/bb/ui/crumbs/hobeventhandler.py
+++ b/lib/bb/ui/crumbs/hobeventhandler.py
@@ -42,7 +42,8 @@ class HobHandler(gobject.GObject):
                                   (gobject.TYPE_PYOBJECT,)),
          "config-found"        : (gobject.SIGNAL_RUN_LAST,
                                   gobject.TYPE_NONE,
-                                  (gobject.TYPE_STRING,)),
+                                  (gobject.TYPE_STRING,
+                                   gobject.TYPE_BOOLEAN)),
          "generating-data"     : (gobject.SIGNAL_RUN_LAST,
                                   gobject.TYPE_NONE,
                                   ()),
@@ -150,7 +151,8 @@ class HobHandler(gobject.GObject):
                 self.emit("sdk-machines-updated", sdk_machines)
         elif isinstance(event, bb.event.ConfigFilePathFound):
             path = event._path
-            self.emit("config-found", path)
+            extraconf = event._extraconf
+            self.emit("config-found", path, extraconf)
         elif isinstance(event, bb.event.FilesMatchingFound):
             # FIXME: hard coding, should at least be a variable shared between
             # here and the caller
-- 
1.7.6

Re: [RFC PATCH 0/3] Tracking parsed configuration files

[Richard Purdie]

On Tue, 2011-07-26 at 21:46 -0700, Joshua Lock wrote:
> All,
> 
> This series attempts to address Yocto bug 1246: "Configuration changes could
> be saved to a file the cooker isn't using".
> 
> It's an ugly series and the issue will only present itself if users run the
> UI trough alternative ways. My main issue is that we can't easily track all
> loaded conf files at parseConfigurationFiles time so it feels a little
> brittle.
> 
> Thoughts? Comments?

Likely bitbake should be maintaining a list of files the given parsing
process has included up to a given point. I believe we already have
something along those lines and might just want to expose it...

Cheers,

Richard