[PATCH] data_smart: Expand overrides cache recursively

[PATCH] data_smart: Expand overrides cache recursively

[Richard Purdie]

If the values that make up OVERRIDES are themselves overridden, 
we end up into some horrible circular logic. Unfortunately some
metadata does depend on this functionality.

e.g:
DEFAULTTUNE_virtclass-multilib-xxx = Y
which changes TUNE_ARCH 
which changes TARGET_ARCH
which changes OVERRIDES

As a solution, we iterate override expansion until the values don't
change. If we iterate more than 5 times we abort and tell the user to
report the issue.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index f245d99..7651a5e 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -420,9 +420,13 @@ class DataSmart(MutableMapping):
         self.overrides = None
 
     def need_overrides(self):
-        if self.overrides is None:
-            if self.inoverride:
-                return
+        if self.overrides is not None:
+            return
+        changed = True
+        count = 0
+        if self.inoverride:
+            return
+        while changed:
             self.inoverride = True
             # Can end up here recursively so setup dummy values
             self.overrides = []
@@ -431,6 +435,14 @@ class DataSmart(MutableMapping):
             self.overridesset = set(self.overrides)
             self.inoverride = False
             self.expand_cache = {}
+            newoverrides = (self.getVar("OVERRIDES", True) or "").split(":") or []
+            if newoverrides == self.overrides:
+                changed = False
+            self.overrides = newoverrides
+            self.overridesset = set(self.overrides)
+            count += 1
+            if count > 5:
+                bb.fatal("Overrides could not be expanded into a stable state after 5 iterations, overrides must be being referenced by other overridden variables in some recursive fashion. Please provide your configuration to bitbake-devel so we can laugh, er, I mean try and understand how to make it work.")
 
     def initVar(self, var):
         self.expand_cache = {}

Re: [PATCH] data_smart: Expand overrides cache recursively

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

On Wed, Sep 16, 2015 at 1:57 PM, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> +        while changed:
>              self.inoverride = True
>              # Can end up here recursively so setup dummy values
>              self.overrides = []
> @@ -431,6 +435,14 @@ class DataSmart(MutableMapping):
>              self.overridesset = set(self.overrides)
>              self.inoverride = False
>              self.expand_cache = {}
> +            newoverrides = (self.getVar("OVERRIDES", True) or
> "").split(":") or []
> +            if newoverrides == self.overrides:
> +                changed = False
> +            self.overrides = newoverrides
> +            self.overridesset = set(self.overrides)
> +            count += 1
> +            if count > 5:
> +                bb.fatal("Overrides could not be expanded into a stable
> state after 5 iterations, overrides must be being referenced by other
> overridden variables in some recursive fashion. Please provide your
> configuration to bitbake-devel so we can laugh, er, I mean try and
> understand how to make it work.")
>

We know the maximum number of iterations here, so it might be worth doing a
for count in range(5): .. if newoverrides == self.overrides: break, and the
else block on the loop would indicate it never broke and therefore we hit
the count > 5 case. That'd eliminate the manual count bits.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 2212 bytes --]