[PATCH] eSDK: fix locked signature handling for multiconfig builds

[PATCH] eSDK: fix locked signature handling for multiconfig builds

[Mehmet Fide]

Building an extensible SDK with populate_sdk_ext fails for machines
that use BBMULTICONFIG.  Three separate issues contribute to the
failure:

1. generate_locked_sigs() in copy_buildsystem.py builds the task
   filter from BB_TASKDEPDATA which does not include multiconfig
   tasks.  The resulting locked-sigs.inc lacks entries for mc builds,
   so the eSDK internal build cannot find sstate for those tasks and
   fails with setscene enforcement errors.

   Fix by also collecting mc task IDs from siggen.runtaskdeps.

2. sstate_lockedsigs() in sstatesig.py stores a single hash per
   pn:task pair.  When the same PN appears in multiple tune-specific
   type sections with different hashes (as happens with multiconfig),
   the last type overwrites earlier entries.  get_taskhash() then
   returns the wrong locked hash for one of the multiconfigs.

   Fix by storing a list of entries per pn:task and selecting the
   entry whose hash matches the computed hash in get_taskhash().

3. translate_virtualfns() in oe-check-sstate crashes with a KeyError
   when multiconfig tasks reference recipe files not present in the
   recipe cache for that mc context.

   Fix by skipping unresolvable tasks gracefully.

Signed-off-by: Mehmet Fide <mehmet.fide@gmail.com>
---
 meta/lib/oe/copy_buildsystem.py | 10 +++++++++-
 meta/lib/oe/sstatesig.py        | 17 ++++++++++++++---
 scripts/oe-check-sstate         |  4 ++++
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
index 81abfbf9e2..c62fbd3fbb 100644
--- a/meta/lib/oe/copy_buildsystem.py
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -172,7 +172,15 @@ class BuildSystem(object):
 def generate_locked_sigs(sigfile, d):
     bb.utils.mkdirhier(os.path.dirname(sigfile))
     depd = d.getVar('BB_TASKDEPDATA', False)
-    tasks = ['%s:%s' % (v[2], v[1]) for v in depd.values()]
+    tasks = set(['%s:%s' % (v[2], v[1]) for v in depd.values()])
+    # BB_TASKDEPDATA does not include multiconfig tasks, so the eSDK
+    # locked-sigs.inc would lack entries for mc builds causing setscene
+    # enforcement failures.  Add any mc tasks tracked by siggen.
+    for tid in bb.parse.siggen.runtaskdeps:
+        if isinstance(tid, tuple):
+            tid = tid[1]
+        if tid.startswith('mc:'):
+            tasks.add(tid)
     bb.parse.siggen.dump_lockedsigs(sigfile, tasks)
 
 def prune_lockedsigs(excluded_tasks, excluded_targets, lockedsigs, onlynative, pruned_output):
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index d818fce8f1..9736e0b4e6 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -90,7 +90,9 @@ def sstate_lockedsigs(d):
             pn, task, h = ls.split(":", 2)
             if pn not in sigs:
                 sigs[pn] = {}
-            sigs[pn][task] = [h, siggen_lockedsigs_var]
+            if task not in sigs[pn]:
+                sigs[pn][task] = []
+            sigs[pn][task].append([h, siggen_lockedsigs_var])
     return sigs
 
 class SignatureGeneratorOEBasicHashMixIn(object):
@@ -182,8 +184,17 @@ class SignatureGeneratorOEBasicHashMixIn(object):
 
         if not unlocked and recipename in self.lockedsigs:
             if task in self.lockedsigs[recipename]:
-                h_locked = self.lockedsigs[recipename][task][0]
-                var = self.lockedsigs[recipename][task][1]
+                entries = self.lockedsigs[recipename][task]
+                # When multiple locked hashes exist for the same pn:task
+                # (e.g. from multiconfig builds), pick the one matching
+                # the computed hash.  Fall back to the first entry.
+                h_locked = entries[0][0]
+                var = entries[0][1]
+                for entry in entries:
+                    if entry[0] == h:
+                        h_locked = entry[0]
+                        var = entry[1]
+                        break
                 self.lockedhashes[tid] = h_locked
                 self._internal = True
                 unihash = self.get_unihash(tid)
diff --git a/scripts/oe-check-sstate b/scripts/oe-check-sstate
index 0d171c4463..239aff0a97 100755
--- a/scripts/oe-check-sstate
+++ b/scripts/oe-check-sstate
@@ -35,6 +35,10 @@ def translate_virtualfns(tasks):
             (mc, fn, taskname) = bb.runqueue.split_tid(task)
             if taskname.endswith('_setscene'):
                 taskname = taskname[:-9]
+            if mc not in recipecaches or fn not in recipecaches[mc].pkg_fn:
+                # Multiconfig tasks may reference fns not in the recipe
+                # cache for this mc context.  Skip them quietly.
+                continue
             outtasks.append('%s:%s' % (recipecaches[mc].pkg_fn[fn], taskname))
     finally:
         tinfoil.shutdown()
-- 
2.43.0

Re: [PATCH] eSDK: fix locked signature handling for multiconfig builds

[Richard Purdie]

On Sat, 2026-03-14 at 23:46 +0100, Mehmet Fide wrote:
> Building an extensible SDK with populate_sdk_ext fails for machines
> that use BBMULTICONFIG.  Three separate issues contribute to the
> failure:
> 
> 1. generate_locked_sigs() in copy_buildsystem.py builds the task
>    filter from BB_TASKDEPDATA which does not include multiconfig
>    tasks.  The resulting locked-sigs.inc lacks entries for mc builds,
>    so the eSDK internal build cannot find sstate for those tasks and
>    fails with setscene enforcement errors.
> 
>    Fix by also collecting mc task IDs from siggen.runtaskdeps.
> 
> 2. sstate_lockedsigs() in sstatesig.py stores a single hash per
>    pn:task pair.  When the same PN appears in multiple tune-specific
>    type sections with different hashes (as happens with multiconfig),
>    the last type overwrites earlier entries.  get_taskhash() then
>    returns the wrong locked hash for one of the multiconfigs.
> 
>    Fix by storing a list of entries per pn:task and selecting the
>    entry whose hash matches the computed hash in get_taskhash().
> 
> 3. translate_virtualfns() in oe-check-sstate crashes with a KeyError
>    when multiconfig tasks reference recipe files not present in the
>    recipe cache for that mc context.
> 
>    Fix by skipping unresolvable tasks gracefully.
> 
> Signed-off-by: Mehmet Fide <mehmet.fide@gmail.com>
> ---
>  meta/lib/oe/copy_buildsystem.py | 10 +++++++++-
>  meta/lib/oe/sstatesig.py        | 17 ++++++++++++++---
>  scripts/oe-check-sstate         |  4 ++++
>  3 files changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
> index 81abfbf9e2..c62fbd3fbb 100644
> --- a/meta/lib/oe/copy_buildsystem.py
> +++ b/meta/lib/oe/copy_buildsystem.py
> @@ -172,7 +172,15 @@ class BuildSystem(object):
>  def generate_locked_sigs(sigfile, d):
>      bb.utils.mkdirhier(os.path.dirname(sigfile))
>      depd = d.getVar('BB_TASKDEPDATA', False)
> -    tasks = ['%s:%s' % (v[2], v[1]) for v in depd.values()]
> +    tasks = set(['%s:%s' % (v[2], v[1]) for v in depd.values()])
> +    # BB_TASKDEPDATA does not include multiconfig tasks, so the eSDK
> +    # locked-sigs.inc would lack entries for mc builds causing setscene
> +    # enforcement failures.  Add any mc tasks tracked by siggen.
> +    for tid in bb.parse.siggen.runtaskdeps:
> +        if isinstance(tid, tuple):
> +            tid = tid[1]
> +        if tid.startswith('mc:'):
> +            tasks.add(tid)
>      bb.parse.siggen.dump_lockedsigs(sigfile, tasks)

BB_TASKDEPDATA only contains the tasks that the current task depends
upon. This is by design so that you can't "see" outside your own
current scope, which would be a dependency issue.

The way you've changed the code, it will add any task with a
multiconfig set, regardless of whether you depend on it or not.

It is also possible you're running this code within a multiconfig, in
which case you'd want to non-mc tasks, so it isn't correct from that
angle either.

The complexities of this are why we've so far not chosen to try and
handle it.

>  def prune_lockedsigs(excluded_tasks, excluded_targets, lockedsigs, onlynative, pruned_output):
> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index d818fce8f1..9736e0b4e6 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -90,7 +90,9 @@ def sstate_lockedsigs(d):
>              pn, task, h = ls.split(":", 2)
>              if pn not in sigs:
>                  sigs[pn] = {}
> -            sigs[pn][task] = [h, siggen_lockedsigs_var]
> +            if task not in sigs[pn]:
> +                sigs[pn][task] = []
> +            sigs[pn][task].append([h, siggen_lockedsigs_var])
>      return sigs
>  
>  class SignatureGeneratorOEBasicHashMixIn(object):
> @@ -182,8 +184,17 @@ class SignatureGeneratorOEBasicHashMixIn(object):
>  
>          if not unlocked and recipename in self.lockedsigs:
>              if task in self.lockedsigs[recipename]:
> -                h_locked = self.lockedsigs[recipename][task][0]
> -                var = self.lockedsigs[recipename][task][1]
> +                entries = self.lockedsigs[recipename][task]
> +                # When multiple locked hashes exist for the same pn:task
> +                # (e.g. from multiconfig builds), pick the one matching
> +                # the computed hash.  Fall back to the first entry.
> +                h_locked = entries[0][0]
> +                var = entries[0][1]
> +                for entry in entries:
> +                    if entry[0] == h:
> +                        h_locked = entry[0]
> +                        var = entry[1]
> +                        break
>                  self.lockedhashes[tid] = h_locked
>                  self._internal = True
>                  unihash = self.get_unihash(tid)

Multiconfigs are deliberately meant to behave like separate builds,
which don't need to be multiconfig aware. The way you're changing this,
changes that. Instead for example, the locked sigs inc file could
include the MC in the filename, then you'd just need each config to
write it out.

Cheers,

Richard