[PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[Paul Barker]

Where a task (such as do_package) runs under fakeroot, the corresponding
setscene task (do_package_setscene) will also run under fakeroot when
restoring from sstate. Assuming pseudo is used as the fakeroot
implementation, we need pseudo-native and all its runtime dependencies
to be available in the sysroot before running any setscene tasks under
fakeroot.

We already add a hard dependency from all do_package_setscene tasks to
virtual/fakeroot-native:do_populate_sysroot in base.bbclass, but this
does not cover transitive dependencies. So, extend the dependencies of
pseudo-native:do_populate_sysroot_setscene to ensure that the sqlite3
library and attr binaries are also available in the sysroot before
running fakeroot setscene tasks.

[YOCTO #15963]

Signed-off-by: Paul Barker <paul@pbarker.dev>
---
 meta/recipes-devtools/pseudo/pseudo.inc | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/recipes-devtools/pseudo/pseudo.inc b/meta/recipes-devtools/pseudo/pseudo.inc
index 22c934977d9b..82499cdd74da 100644
--- a/meta/recipes-devtools/pseudo/pseudo.inc
+++ b/meta/recipes-devtools/pseudo/pseudo.inc
@@ -155,3 +155,10 @@ do_install:append:class-nativesdk () {
 }
 
 BBCLASSEXTEND = "native nativesdk"
+
+# Setscene tasks which run under fakeroot must not be executed before
+# pseudo-native and *all* its runtime dependencies are available in the
+# sysroot.
+PSEUDO_SETSCENE_DEPS = ""
+PSEUDO_SETSCENE_DEPS:class-native = "sqlite3-native:do_populate_sysroot attr-native:do_populate_sysroot"
+do_populate_sysroot_setscene[depends] += "${PSEUDO_SETSCENE_DEPS}"

---
base-commit: 58558b97c157469f060bb2ad59a40254fb6181e4
change-id: 20251016-fix-pseudo-native-221fe3fceea4

Best regards,
-- 
Paul Barker

Re: [OE-core] [PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[Gyorgy Sarvari]

Tested-by: Gyorgy Sarvari <skandigraun@gmail.com>

On 10/16/25 21:11, Paul Barker via lists.openembedded.org wrote:
> Where a task (such as do_package) runs under fakeroot, the corresponding
> setscene task (do_package_setscene) will also run under fakeroot when
> restoring from sstate. Assuming pseudo is used as the fakeroot
> implementation, we need pseudo-native and all its runtime dependencies
> to be available in the sysroot before running any setscene tasks under
> fakeroot.
>
> We already add a hard dependency from all do_package_setscene tasks to
> virtual/fakeroot-native:do_populate_sysroot in base.bbclass, but this
> does not cover transitive dependencies. So, extend the dependencies of
> pseudo-native:do_populate_sysroot_setscene to ensure that the sqlite3
> library and attr binaries are also available in the sysroot before
> running fakeroot setscene tasks.
>
> [YOCTO #15963]
>
> Signed-off-by: Paul Barker <paul@pbarker.dev>
> ---
>  meta/recipes-devtools/pseudo/pseudo.inc | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/meta/recipes-devtools/pseudo/pseudo.inc b/meta/recipes-devtools/pseudo/pseudo.inc
> index 22c934977d9b..82499cdd74da 100644
> --- a/meta/recipes-devtools/pseudo/pseudo.inc
> +++ b/meta/recipes-devtools/pseudo/pseudo.inc
> @@ -155,3 +155,10 @@ do_install:append:class-nativesdk () {
>  }
>  
>  BBCLASSEXTEND = "native nativesdk"
> +
> +# Setscene tasks which run under fakeroot must not be executed before
> +# pseudo-native and *all* its runtime dependencies are available in the
> +# sysroot.
> +PSEUDO_SETSCENE_DEPS = ""
> +PSEUDO_SETSCENE_DEPS:class-native = "sqlite3-native:do_populate_sysroot attr-native:do_populate_sysroot"
> +do_populate_sysroot_setscene[depends] += "${PSEUDO_SETSCENE_DEPS}"
>
> ---
> base-commit: 58558b97c157469f060bb2ad59a40254fb6181e4
> change-id: 20251016-fix-pseudo-native-221fe3fceea4
>
> Best regards,
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#224980): https://lists.openembedded.org/g/openembedded-core/message/224980
> Mute This Topic: https://lists.openembedded.org/mt/115795308/6084445
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [skandigraun@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

RE: [OE-core] [PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[Marko, Peter]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Paul Barker
> Sent: Thursday, October 16, 2025 21:11
> To: openembedded-core@lists.openembedded.org
> Cc: Alexander Kanavin <alex.kanavin@gmail.com>; Paul Barker
> <paul@pbarker.dev>
> Subject: [OE-core] [PATCH] pseudo: Add hard sstate dependencies for pseudo-
> native
> 
> Where a task (such as do_package) runs under fakeroot, the corresponding
> setscene task (do_package_setscene) will also run under fakeroot when
> restoring from sstate. Assuming pseudo is used as the fakeroot
> implementation, we need pseudo-native and all its runtime dependencies
> to be available in the sysroot before running any setscene tasks under
> fakeroot.
> 
> We already add a hard dependency from all do_package_setscene tasks to
> virtual/fakeroot-native:do_populate_sysroot in base.bbclass, but this
> does not cover transitive dependencies. So, extend the dependencies of
> pseudo-native:do_populate_sysroot_setscene to ensure that the sqlite3
> library and attr binaries are also available in the sysroot before
> running fakeroot setscene tasks.
> 
> [YOCTO #15963]
> 
> Signed-off-by: Paul Barker <paul@pbarker.dev>
> ---
>  meta/recipes-devtools/pseudo/pseudo.inc | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/meta/recipes-devtools/pseudo/pseudo.inc b/meta/recipes-
> devtools/pseudo/pseudo.inc
> index 22c934977d9b..82499cdd74da 100644
> --- a/meta/recipes-devtools/pseudo/pseudo.inc
> +++ b/meta/recipes-devtools/pseudo/pseudo.inc
> @@ -155,3 +155,10 @@ do_install:append:class-nativesdk () {
>  }
> 
>  BBCLASSEXTEND = "native nativesdk"
> +
> +# Setscene tasks which run under fakeroot must not be executed before
> +# pseudo-native and *all* its runtime dependencies are available in the
> +# sysroot.
> +PSEUDO_SETSCENE_DEPS = ""
> +PSEUDO_SETSCENE_DEPS:class-native = "sqlite3-native:do_populate_sysroot
> attr-native:do_populate_sysroot"

I have a doubt here.

$ grep -rI 'setscene[[]depends[]]'
bitbake/lib/bb/tests/runqueue-tests/recipes/d1.bb:do_package_setscene[depends] = "a1:do_populate_sysroot_setscene"
bitbake/lib/bb/runqueue.py:    # e.g. do_sometask_setscene[depends] = "targetname:do_someothertask_setscene"
meta/classes/useradd.bbclass:do_package_setscene[depends] += "${USERADDSETSCENEDEPS}"
meta/classes/useradd.bbclass:do_populate_sysroot_setscene[depends] += "${USERADDSETSCENEDEPS}"
meta/recipes-core/sysvinit/sysvinit_3.14.bb:do_package_setscene[depends] = "${MLPREFIX}base-passwd:do_populate_sysroot"

useradd.bbclass with USERADDSETSCENEDEPS also resolves into depending on setscene tasks.

So tests and useradd depend on setscene which is also documented and handled in requeue.
To me it looks like sysvinit (where this fix for pseudo was presumable coped from) uses wrong task dependency which works only by chance.
But this is deep bitbake stuff which may have special nuances.
What is the difference between using these two variants?

Peter


> +do_populate_sysroot_setscene[depends] += "${PSEUDO_SETSCENE_DEPS}"
> 
> ---
> base-commit: 58558b97c157469f060bb2ad59a40254fb6181e4
> change-id: 20251016-fix-pseudo-native-221fe3fceea4
> 
> Best regards,
> --
> Paul Barker

Re: [OE-core] [PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 3931 bytes --]

On Tue, 2025-10-28 at 14:02 +0000, Marko, Peter wrote:
> 
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-
> > core@lists.openembedded.org> On Behalf Of Paul Barker
> > Sent: Thursday, October 16, 2025 21:11
> > To: openembedded-core@lists.openembedded.org
> > Cc: Alexander Kanavin <alex.kanavin@gmail.com>; Paul Barker
> > <paul@pbarker.dev>
> > Subject: [OE-core] [PATCH] pseudo: Add hard sstate dependencies for pseudo-
> > native
> > 
> > Where a task (such as do_package) runs under fakeroot, the corresponding
> > setscene task (do_package_setscene) will also run under fakeroot when
> > restoring from sstate. Assuming pseudo is used as the fakeroot
> > implementation, we need pseudo-native and all its runtime dependencies
> > to be available in the sysroot before running any setscene tasks under
> > fakeroot.
> > 
> > We already add a hard dependency from all do_package_setscene tasks to
> > virtual/fakeroot-native:do_populate_sysroot in base.bbclass, but this
> > does not cover transitive dependencies. So, extend the dependencies of
> > pseudo-native:do_populate_sysroot_setscene to ensure that the sqlite3
> > library and attr binaries are also available in the sysroot before
> > running fakeroot setscene tasks.
> > 
> > [YOCTO #15963]
> > 
> > Signed-off-by: Paul Barker <paul@pbarker.dev>
> > ---
> >  meta/recipes-devtools/pseudo/pseudo.inc | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/pseudo/pseudo.inc b/meta/recipes-
> > devtools/pseudo/pseudo.inc
> > index 22c934977d9b..82499cdd74da 100644
> > --- a/meta/recipes-devtools/pseudo/pseudo.inc
> > +++ b/meta/recipes-devtools/pseudo/pseudo.inc
> > @@ -155,3 +155,10 @@ do_install:append:class-nativesdk () {
> >  }
> > 
> >  BBCLASSEXTEND = "native nativesdk"
> > +
> > +# Setscene tasks which run under fakeroot must not be executed before
> > +# pseudo-native and *all* its runtime dependencies are available in the
> > +# sysroot.
> > +PSEUDO_SETSCENE_DEPS = ""
> > +PSEUDO_SETSCENE_DEPS:class-native = "sqlite3-native:do_populate_sysroot
> > attr-native:do_populate_sysroot"
> 
> I have a doubt here.
> 
> $ grep -rI 'setscene[[]depends[]]'
> bitbake/lib/bb/tests/runqueue-tests/recipes/d1.bb:do_package_setscene[depends] = "a1:do_populate_sysroot_setscene"
> bitbake/lib/bb/runqueue.py:    # e.g. do_sometask_setscene[depends] = "targetname:do_someothertask_setscene"
> meta/classes/useradd.bbclass:do_package_setscene[depends] += "${USERADDSETSCENEDEPS}"
> meta/classes/useradd.bbclass:do_populate_sysroot_setscene[depends] += "${USERADDSETSCENEDEPS}"
> meta/recipes-core/sysvinit/sysvinit_3.14.bb:do_package_setscene[depends] = "${MLPREFIX}base-passwd:do_populate_sysroot"
> 
> useradd.bbclass with USERADDSETSCENEDEPS also resolves into depending on setscene tasks.
> 
> So tests and useradd depend on setscene which is also documented and handled in requeue.
> To me it looks like sysvinit (where this fix for pseudo was presumable coped from) uses wrong task dependency which works only by chance.
> But this is deep bitbake stuff which may have special nuances.
> What is the difference between using these two variants?

The hard setscene dependency on do_populate_sysroot definitely works, it's used
in base.bbclass for do_package_setscene:

    d.appendVarFlag('do_package_setscene', 'depends', ' virtual/fakeroot-native:do_populate_sysroot')

And looking in runqueue.py in bitbake, there is a line to trim '_setscene' off
the end of the task we're depending on if it is present:

    deptid = depfn + ":" + idependtask.replace("_setscene", "")

So both variants work and result in the same hard dependencies within bitbake.
As "_setscene" is removed if present, I'd say it's slightly more correct to
specify the dependency without it.

Thanks,

-- 
Paul Barker

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [PATCH] pseudo: Add hard sstate dependencies for pseudo-native

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 3125 bytes --]

On Thu, 2025-10-16 at 20:11 +0100, Paul Barker wrote:
> Where a task (such as do_package) runs under fakeroot, the corresponding
> setscene task (do_package_setscene) will also run under fakeroot when
> restoring from sstate. Assuming pseudo is used as the fakeroot
> implementation, we need pseudo-native and all its runtime dependencies
> to be available in the sysroot before running any setscene tasks under
> fakeroot.
> 
> We already add a hard dependency from all do_package_setscene tasks to
> virtual/fakeroot-native:do_populate_sysroot in base.bbclass, but this
> does not cover transitive dependencies. So, extend the dependencies of
> pseudo-native:do_populate_sysroot_setscene to ensure that the sqlite3
> library and attr binaries are also available in the sysroot before
> running fakeroot setscene tasks.
> 
> [YOCTO #15963]
> 
> Signed-off-by: Paul Barker <paul@pbarker.dev>
> ---
>  meta/recipes-devtools/pseudo/pseudo.inc | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/meta/recipes-devtools/pseudo/pseudo.inc b/meta/recipes-devtools/pseudo/pseudo.inc
> index 22c934977d9b..82499cdd74da 100644
> --- a/meta/recipes-devtools/pseudo/pseudo.inc
> +++ b/meta/recipes-devtools/pseudo/pseudo.inc
> @@ -155,3 +155,10 @@ do_install:append:class-nativesdk () {
>  }
>  
>  BBCLASSEXTEND = "native nativesdk"
> +
> +# Setscene tasks which run under fakeroot must not be executed before
> +# pseudo-native and *all* its runtime dependencies are available in the
> +# sysroot.
> +PSEUDO_SETSCENE_DEPS = ""
> +PSEUDO_SETSCENE_DEPS:class-native = "sqlite3-native:do_populate_sysroot attr-native:do_populate_sysroot"
> +do_populate_sysroot_setscene[depends] += "${PSEUDO_SETSCENE_DEPS}"

Richard,

I've spent some time looking at this. I can't find a good reason for the
hard setscene dependency on attr-native, this appears to be a build time
dependency only. I thought I saw invocation of the setfattr command, but
looking again that's just in the configure script.

I think setting the hard setscene dependency here in the pseudo recipe
is good. Placing it here as a dependency of pseudo-native gives us the
following dependency graph for the test case of gcc-runtime:do_package:

  gcc-runtime:do_package_setscene -> pseudo-native:do_populate_sysroot_setscene -> sqlite3-native:do_populate_sysroot_setscene

The extra hop there doesn't really cause us any build performance
issues. It prevents us from running do_populate_sysroot_setscene in
parallel for pseudo-native and sqlite3-native, but neither of those are
large sstate archives and they're very quick to unpack.

I'd be concerned if the graph of hard dependencies got complicated, but
one level of indirection for a single dependency is ok.

It's good to keep the dependency in the pseudo recipe where it can be
easily understood, rather than moving it to a new variable named
something like FAKEROOT_SETSCENE_DEPENDS in base.bbclass.

So, I plan to send a v2 which drops the attr-native:do_populate_sysroot
dependency but is otherwise the same.

Thanks,

-- 
Paul Barker

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]