Poky Project Discussions
 help / color / mirror / Atom feed
* [PATCH 1/3] bitbake: allow excluding dependencies of variables
  2010-12-14 18:30 [PATCH 0/3] reworked environment variable whitelisting fixes Paul Eggleton
@ 2010-12-10 14:29 ` Paul Eggleton
  2010-12-10 15:30 ` [PATCH 2/3] bitbake: add optional expansion to getVarFlag() Paul Eggleton
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2010-12-10 14:29 UTC (permalink / raw)
  To: poky

Adds a vardepsexclude flag that can be used to exclude a dependency of
a variable (the opposite of vardeps). This will allow the exclusion of
variables from the hash generation much more selectively than blanket
whitelisting using BB_HASHBASE_WHITELIST.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/lib/bb/data.py |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/bitbake/lib/bb/data.py b/bitbake/lib/bb/data.py
index d4d43fd..0c95ebb 100644
--- a/bitbake/lib/bb/data.py
+++ b/bitbake/lib/bb/data.py
@@ -297,6 +297,7 @@ def build_dependencies(key, keys, shelldeps, d):
             deps |= parser.references
             deps = deps | (keys & parser.execs)
         deps |= set((d.getVarFlag(key, "vardeps") or "").split())
+        deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split())
     except:
         bb.note("Error expanding variable %s" % key) 
         raise
-- 
1.7.1



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/3] bitbake: add optional expansion to getVarFlag()
  2010-12-14 18:30 [PATCH 0/3] reworked environment variable whitelisting fixes Paul Eggleton
  2010-12-10 14:29 ` [PATCH 1/3] bitbake: allow excluding dependencies of variables Paul Eggleton
@ 2010-12-10 15:30 ` Paul Eggleton
  2010-12-14 18:09 ` [PATCH 3/3] poky.conf: adjust variable whitelisting to fix sstate checksums Paul Eggleton
  2010-12-15  9:08 ` [PATCH 0/3] reworked environment variable whitelisting fixes Richard Purdie
  3 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2010-12-10 15:30 UTC (permalink / raw)
  To: poky

Add a parameter to getVarFlag() to auto-expand the value of the flag. This
makes getVarFlag() more consistent with getVar(), and allows expansion of
vardeps and vardepsexclude (which has been done in this commit).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 bitbake/lib/bb/data.py       |    4 ++--
 bitbake/lib/bb/data_smart.py |    9 ++++++---
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/bitbake/lib/bb/data.py b/bitbake/lib/bb/data.py
index 0c95ebb..0aa8b40 100644
--- a/bitbake/lib/bb/data.py
+++ b/bitbake/lib/bb/data.py
@@ -296,8 +296,8 @@ def build_dependencies(key, keys, shelldeps, d):
             parser = d.expandWithRefs(d.getVar(key, False), key)
             deps |= parser.references
             deps = deps | (keys & parser.execs)
-        deps |= set((d.getVarFlag(key, "vardeps") or "").split())
-        deps -= set((d.getVarFlag(key, "vardepsexclude") or "").split())
+        deps |= set((d.getVarFlag(key, "vardeps", True) or "").split())
+        deps -= set((d.getVarFlag(key, "vardepsexclude", True) or "").split())
     except:
         bb.note("Error expanding variable %s" % key) 
         raise
diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index c8cd8f8..30f9cbc 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -277,12 +277,15 @@ class DataSmart:
             self._makeShadowCopy(var)
         self.dict[var][flag] = flagvalue
 
-    def getVarFlag(self, var, flag):
+    def getVarFlag(self, var, flag, exp = False):
         local_var = self._findVar(var)
+        value = None
         if local_var:
             if flag in local_var:
-                return copy.copy(local_var[flag])
-        return None
+                value = copy.copy(local_var[flag])
+        if exp and value:
+            value = self.expand(value, None)
+        return value
 
     def delVarFlag(self, var, flag):
         local_var = self._findVar(var)
-- 
1.7.1



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 3/3] poky.conf: adjust variable whitelisting to fix sstate checksums
  2010-12-14 18:30 [PATCH 0/3] reworked environment variable whitelisting fixes Paul Eggleton
  2010-12-10 14:29 ` [PATCH 1/3] bitbake: allow excluding dependencies of variables Paul Eggleton
  2010-12-10 15:30 ` [PATCH 2/3] bitbake: add optional expansion to getVarFlag() Paul Eggleton
@ 2010-12-14 18:09 ` Paul Eggleton
  2010-12-15  9:08 ` [PATCH 0/3] reworked environment variable whitelisting fixes Richard Purdie
  3 siblings, 0 replies; 5+ messages in thread
From: Paul Eggleton @ 2010-12-14 18:09 UTC (permalink / raw)
  To: poky

After some analysis of task dependencies I have adjusted the global whitelist
used to filter out variables that would otherwise cause unwanted changes in
task checksums for sstate packages.

The following changes have been made to BB_HASHBASE_WHITELIST:

 * Fix typo: FILESEXTRAPATHS not FILESEXTRPATHS
 * Add variables FILE_DIRNAME HOME LOGNAME SHELL TERM USER
 * Remove variables DATE and TIME (these are referred to by a number of tasks
   where they should affect the checksum, e.g. because they influence PV)
 * Remove variable _ (not found in dependencies)

Additionally DATE and SRCDATE are excluded but only explicitly for
patch_do_patch, which uses these only to provide a warning if patches are
out of date.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/conf/distro/poky.conf |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/meta/conf/distro/poky.conf b/meta/conf/distro/poky.conf
index 10b3165..95ca097 100644
--- a/meta/conf/distro/poky.conf
+++ b/meta/conf/distro/poky.conf
@@ -143,4 +143,5 @@ require conf/distro/include/world-broken.inc
 # Setup our hash policy
 BB_SIGNATURE_HANDLER = "basic"
 BB_HASHTASK_WHITELIST = "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
-BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH TIME DATE BBPATH DL_DIR SSTATE_DIR THISDIR _ FILESEXTRPATHS"
+BB_HASHBASE_WHITELIST = "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
+patch_do_patch[vardepsexclude] = "DATE SRCDATE"
-- 
1.7.1



^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 0/3] reworked environment variable whitelisting fixes
@ 2010-12-14 18:30 Paul Eggleton
  2010-12-10 14:29 ` [PATCH 1/3] bitbake: allow excluding dependencies of variables Paul Eggleton
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Paul Eggleton @ 2010-12-14 18:30 UTC (permalink / raw)
  To: poky

Adjustments to variable whitelisting to fix sstate package hashing. This supersedes
the paule/sstate contrib branch.

The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.

Pull URL: git://git.pokylinux.org/poky-contrib.git
  Branch: paule/sstate2
  Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2

Thanks,
    Paul Eggleton <paul.eggleton@linux.intel.com>
---


Paul Eggleton (3):
  bitbake: allow excluding dependencies of variables
  bitbake: add optional expansion to getVarFlag()
  poky.conf: adjust variable whitelisting to fix sstate checksums

 bitbake/lib/bb/data.py       |    3 ++-
 bitbake/lib/bb/data_smart.py |    9 ++++++---
 meta/conf/distro/poky.conf   |    3 ++-
 3 files changed, 10 insertions(+), 5 deletions(-)



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 0/3] reworked environment variable whitelisting fixes
  2010-12-14 18:30 [PATCH 0/3] reworked environment variable whitelisting fixes Paul Eggleton
                   ` (2 preceding siblings ...)
  2010-12-14 18:09 ` [PATCH 3/3] poky.conf: adjust variable whitelisting to fix sstate checksums Paul Eggleton
@ 2010-12-15  9:08 ` Richard Purdie
  3 siblings, 0 replies; 5+ messages in thread
From: Richard Purdie @ 2010-12-15  9:08 UTC (permalink / raw)
  To: Paul Eggleton; +Cc: poky

On Tue, 2010-12-14 at 18:30 +0000, Paul Eggleton wrote:
> Adjustments to variable whitelisting to fix sstate package hashing. This supersedes
> the paule/sstate contrib branch.
> 
> The bitbake changes have been discussed briefly with Richard Purdie & Chris Larson.
> 
> Pull URL: git://git.pokylinux.org/poky-contrib.git
>   Branch: paule/sstate2
>   Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=paule/sstate2
> 
> Thanks,
>     Paul Eggleton <paul.eggleton@linux.intel.com>

Merged into master, thanks!

Cheers,

Richard




^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2010-12-15  9:09 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-12-14 18:30 [PATCH 0/3] reworked environment variable whitelisting fixes Paul Eggleton
2010-12-10 14:29 ` [PATCH 1/3] bitbake: allow excluding dependencies of variables Paul Eggleton
2010-12-10 15:30 ` [PATCH 2/3] bitbake: add optional expansion to getVarFlag() Paul Eggleton
2010-12-14 18:09 ` [PATCH 3/3] poky.conf: adjust variable whitelisting to fix sstate checksums Paul Eggleton
2010-12-15  9:08 ` [PATCH 0/3] reworked environment variable whitelisting fixes Richard Purdie

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox