[bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing

[bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing

[Joshua Watt]

Parse out PN, PV, and PR from the "originating" recipe (the .bb file
being parsed) and provide them at parse time. This allows .bbappends,
.inc, etc. files to know what recipe is currently being parsed.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 lib/bb/cookerdata.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 65c153a5b..8abf5e56e 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -504,6 +504,12 @@ class CookerDataBuilder(object):
         bb_data.setVar("__BBMULTICONFIG", mc)
         bb_data.setVar("FILE_LAYERNAME", layername)
 
+        pn, pv, pr = bb.parse.vars_from_file(bbfile, bb_data)
+
+        bb_data.setVar("ORIG_PN", pn or "defaultpkgname")
+        bb_data.setVar("ORIG_PV", pv or "1.0")
+        bb_data.setVar("ORIG_PR", pr or "r0")
+
         bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
         bb.parse.cached_mtime_noerror(bbfile_loc)
 
-- 
2.51.1

RE: [bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing

[Peter Kjellerstedt]

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Joshua Watt via lists.openembedded.org
> Sent: den 30 oktober 2025 15:19
> To: bitbake-devel@lists.openembedded.org
> Cc: Joshua Watt <JPEWhacker@gmail.com>
> Subject: [bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing
> 
> Parse out PN, PV, and PR from the "originating" recipe (the .bb file
> being parsed) and provide them at parse time. This allows .bbappends,
> .inc, etc. files to know what recipe is currently being parsed.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  lib/bb/cookerdata.py | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> index 65c153a5b..8abf5e56e 100644
> --- a/lib/bb/cookerdata.py
> +++ b/lib/bb/cookerdata.py
> @@ -504,6 +504,12 @@ class CookerDataBuilder(object):
>          bb_data.setVar("__BBMULTICONFIG", mc)
>          bb_data.setVar("FILE_LAYERNAME", layername)
> 
> +        pn, pv, pr = bb.parse.vars_from_file(bbfile, bb_data)
> +
> +        bb_data.setVar("ORIG_PN", pn or "defaultpkgname")
> +        bb_data.setVar("ORIG_PV", pv or "1.0")
> +        bb_data.setVar("ORIG_PR", pr or "r0")
> +
>          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
>          bb.parse.cached_mtime_noerror(bbfile_loc)
> 
> --
> 2.51.1

You want to include PE as well.

//Peter

Re: [bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing

[Joshua Watt]

On Thu, Oct 30, 2025 at 11:09 AM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> > -----Original Message-----
> > From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Joshua Watt via lists.openembedded.org
> > Sent: den 30 oktober 2025 15:19
> > To: bitbake-devel@lists.openembedded.org
> > Cc: Joshua Watt <JPEWhacker@gmail.com>
> > Subject: [bitbake-devel][RFC] cookerdata: Include "originating" recipe data when parsing
> >
> > Parse out PN, PV, and PR from the "originating" recipe (the .bb file
> > being parsed) and provide them at parse time. This allows .bbappends,
> > .inc, etc. files to know what recipe is currently being parsed.
> >
> > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > ---
> >  lib/bb/cookerdata.py | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> > index 65c153a5b..8abf5e56e 100644
> > --- a/lib/bb/cookerdata.py
> > +++ b/lib/bb/cookerdata.py
> > @@ -504,6 +504,12 @@ class CookerDataBuilder(object):
> >          bb_data.setVar("__BBMULTICONFIG", mc)
> >          bb_data.setVar("FILE_LAYERNAME", layername)
> >
> > +        pn, pv, pr = bb.parse.vars_from_file(bbfile, bb_data)
> > +
> > +        bb_data.setVar("ORIG_PN", pn or "defaultpkgname")
> > +        bb_data.setVar("ORIG_PV", pv or "1.0")
> > +        bb_data.setVar("ORIG_PR", pr or "r0")
> > +
> >          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
> >          bb.parse.cached_mtime_noerror(bbfile_loc)
> >
> > --
> > 2.51.1
>
> You want to include PE as well.

Ya. The other option would just be to do:

  bb_data.setVar("ORIG_BBFILE", bbfile)

And let users parse it out for themselves if necessary.

>
> //Peter
>

[bitbake-devel][PATCH] toaster: support bitbake-setup

[Reyna, David]

From c063178ab0971fe297a9e9bca6e6b04eafc5149c Mon Sep 17 00:00:00 2001
From: David Reyna <David.Reyna@windriver.com>
Date: Thu, 30 Oct 2025 14:03:35 -0700
Subject: [PATCH] toaster: support bitbake-setup

    This adds support for the new bitbake-setup:
      * Maintain backwards compatibility for poky-classic releases
      * Assume bitbake-setup for all releases 'master' and forward
      * When no ".templateconf" assume Poky and not OE
      * Generate and use "init-build-env" script instead of 'oe-init-build-env'
        directly, but "cd" to the clone and not the local installation.
      * Include "bblayers.conf" because "cookerdata.py" now explicitly
        required it, and let it be over-ridden by the usual "toaster-bblayers.conf".
      * Force the value "TOPDIR" in "toaster.conf" to the local project's directory.

    Normally "cookerdata.py" sets TOPDIR from getcwd(), but somehow in bitbake-setup
    mode the CWD at this point has become ".../layers/openembedded-core", so
    "bitbake.conf and all other files cannot be found. This non-obvious problem was
    hard to discover. Until this is understood, the forced value will make things work.

    [YOCTO #16012]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 bin/toaster                                   | 21 ++--
 .../bldcontrol/localhostbecontroller.py       | 99 ++++++++++++++-----
 2 files changed, 89 insertions(+), 31 deletions(-)

diff --git a/bin/toaster b/bin/toaster
index f002c8c1590..d20cbe90ba8 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -164,14 +164,20 @@ fi
 # in the local layers that currently make using an arbitrary
 # toasterconf.json difficult.
 
-. $OE_ROOT/.templateconf
-if [ -n "$TEMPLATECONF" ]; then
-    if [ ! -d "$TEMPLATECONF" ]; then
-        # Allow TEMPLATECONF=meta-xyz/conf as a shortcut
-        if [ -d "$OE_ROOT/$TEMPLATECONF" ]; then
-            TEMPLATECONF="$OE_ROOT/$TEMPLATECONF"
+# BBSETUP no longer supports .templateconf
+if [ -f $OE_ROOT/.templateconf ] ; then 
+    . $OE_ROOT/.templateconf
+    if [ -n "$TEMPLATECONF" ]; then
+        if [ ! -d "$TEMPLATECONF" ]; then
+            # Allow TEMPLATECONF=meta-xyz/conf as a shortcut
+            if [ -d "$OE_ROOT/$TEMPLATECONF" ]; then
+                TEMPLATECONF="$OE_ROOT/$TEMPLATECONF"
+            fi
         fi
     fi
+else
+    # Force poky
+    TEMPLATECONF="layers/meta-yocto/meta-poky/conf/templates/default"
 fi
 
 unset OE_ROOT
@@ -255,7 +261,8 @@ fi
 # 2) the build dir (in build)
 # 3) the sqlite db if that is being used.
 # 4) pid's we need to clean up on exit/shutdown
-export TOASTER_DIR=$TOASTERDIR
+# By default we move this all into the Toaster's parent build directory
+export TOASTER_DIR=$BUILDDIR
 export BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS TOASTER_DIR"
 
 # Determine the action. If specified by arguments, fine, if not, toggle it
diff --git a/lib/toaster/bldcontrol/localhostbecontroller.py b/lib/toaster/bldcontrol/localhostbecontroller.py
index 577e765f112..a7bc5722fad 100644
--- a/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -6,6 +6,11 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 
+# DEBUGGING NOTE: if you need to add debug statements into the files within a
+# Toaster-imported layer, you must disable the "git fetch ..." command after
+# the initial clone else any changes will be automatically replaced with the
+# original code
+
 import os
 import re
 import shutil
@@ -183,6 +188,8 @@ class LocalhostBEController(BuildEnvironmentController):
             if commit != "HEAD":
                 logger.debug("localhostbecontroller: checking out commit %s to %s " % (commit, localdirname))
                 ref = commit if re.match('^[a-fA-F0-9]+$', commit) else 'origin/%s' % commit
+                # DEBUGGING NOTE: this is the 'git fetch" to disable after the initial clone to
+                # prevent inserted debugging commands from being lost
                 self._shellcmd('git fetch && git reset --hard "%s"' % ref, localdirname,env=git_env)
 
             # take the localdirname as poky dir if we can find the oe-init-build-env
@@ -319,7 +326,28 @@ class LocalhostBEController(BuildEnvironmentController):
         layers = self.setLayers(bitbake, layers, targets)
         is_merged_attr = bitbake.req.project.merged_attr
 
+        # Is this build for a Bitbake Setup installation?
+        isBBSetup = False
+        release_description = bitbake.req.project.release.description
+        if 'master' in release_description:
+            # This release going forward is Bitbake Setup
+            isBBSetup = True
+        else:
+            try:
+                # Yocto Project 5.3 "Whinlatter"
+                params = release_description.split(' ')
+                # Work past YP "10.1"
+                release_params = params[2].split('.')
+                if int(release_params[0]) > 5:
+                    isBBSetup = True
+                if (release_params[0] == '5') and (int(release_params[1]) >= 3):
+                    isBBSetup = True
+            except:
+                # If release schema not recognized, then this release is in the future
+                isBBSetup = True
+
         git_env = os.environ.copy()
+        toaster_run_dir = git_env['BUILDDIR']
         # (note: add custom environment settings here)
         try:
             # insure that the project init/build uses the selected bitbake, and not Toaster's
@@ -335,28 +363,44 @@ class LocalhostBEController(BuildEnvironmentController):
         else:
             builddir = '%s-toaster-%d' % (self.be.builddir, bitbake.req.project.id)
         oe_init = os.path.join(self.pokydirname, 'oe-init-build-env')
-        # init build environment
+        setup_init = os.path.join(builddir, 'init-build-env')
+        # init build environment and build directory structure
         try:
             custom_script = ToasterSetting.objects.get(name="CUSTOM_BUILD_INIT_SCRIPT").value
             custom_script = custom_script.replace("%BUILDDIR%" ,builddir)
             self._shellcmd("bash -c 'source %s'" % (custom_script),env=git_env)
         except ToasterSetting.DoesNotExist:
-            self._shellcmd("bash -c 'source %s %s'" % (oe_init, builddir),
-                       self.be.sourcedir,env=git_env)
+            # Normal setup
+            if isBBSetup:
+                # Create an 'init-build-env' for the build directory that points to respective cloned bitbake
+                os.makedirs(builddir, exist_ok = True)
+                build_init_script = os.path.join(builddir,'init-build-env')
+                os.system(f"cp {os.path.join(toaster_run_dir,'init-build-env')} {build_init_script}")
+                os.system(f"sed -i -e 's|^cd .*$|cd {self.pokydirname}|g' {build_init_script}")
+                os.system(f"sed -i -e 's|^set .*$|set {builddir}|g' {build_init_script}")
+                # Execute the init
+                self._shellcmd(f"bash -c 'source {build_init_script}'", env=git_env)
+            else:
+                self._shellcmd("bash -c 'source %s %s'" % (oe_init, builddir),
+                           self.be.sourcedir,env=git_env)
 
         # update bblayers.conf
         if not is_merged_attr:
             bblconfpath = os.path.join(builddir, "conf/toaster-bblayers.conf")
             with open(bblconfpath, 'w') as bblayers:
                 bblayers.write('# line added by toaster build control\n'
-                               'BBLAYERS = "%s"' % ' '.join(layers))
+                               'BBLAYERS = "%s"\n' % ' '.join(layers))
 
             # write configuration file
             confpath = os.path.join(builddir, 'conf/toaster.conf')
             with open(confpath, 'w') as conf:
+                if isBBSetup:
+                    # Force TOPDIR
+                    conf.write(f'# Force TOPDIR\n')
+                    conf.write(f'TOPDIR = "{builddir}"\n\n')
                 for var in variables:
-                    conf.write('%s="%s"\n' % (var.name, var.value))
-                conf.write('INHERIT+="toaster buildhistory"')
+                    conf.write('%s = "%s"\n' % (var.name, var.value))
+                conf.write('INHERIT += "toaster buildhistory"\n')
         else:
             # Append the Toaster-specific values directly to the bblayers.conf
             bblconfpath = os.path.join(builddir, "conf/bblayers.conf")
@@ -443,15 +487,25 @@ class LocalhostBEController(BuildEnvironmentController):
             else:
                 logger.error("Looks like Bitbake is not available, please fix your environment")
 
+        # NOTE: BB SETUP requires a 'bblayers.conf' even though Toaster immediately overrides it with 'toaster-bblayers.conf'
+        bblayers = os.path.join(builddir,"conf/bblayers.conf")
         toasterlayers = os.path.join(builddir,"conf/toaster-bblayers.conf")
-        if not is_merged_attr:
-            self._shellcmd('%s bash -c \"source %s %s; BITBAKE_UI="knotty" %s --read %s --read %s '
-                           '--server-only -B 0.0.0.0:0\"' % (env_clean, oe_init,
-                           builddir, bitbake, confpath, toasterlayers), self.be.sourcedir)
+        if isBBSetup:
+            # Use 'init-build-env' model
+            if not is_merged_attr:
+                self._shellcmd(f'{env_clean} bash -c \"source {setup_init}; BITBAKE_UI="knotty" {bitbake} --read {confpath} --read {bblayers} --read {toasterlayers} '
+                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+            else:
+                self._shellcmd(f'{env_clean} bash -c \"source {setup_init}; BITBAKE_UI="knotty" {bitbake} '
+                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
         else:
-            self._shellcmd('%s bash -c \"source %s %s; BITBAKE_UI="knotty" %s '
-                           '--server-only -B 0.0.0.0:0\"' % (env_clean, oe_init,
-                           builddir, bitbake), self.be.sourcedir)
+            # Use 'oe-init-build-env' model
+            if not is_merged_attr:
+                self._shellcmd(f'{env_clean} bash -c \"source {oe_init} {builddir}; BITBAKE_UI="knotty" {bitbake} --read {confpath} --read {bblayers} --read {toasterlayers} '
+                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+            else:
+                self._shellcmd(f'{env_clean} bash -c \"source {oe_init} {builddir}; BITBAKE_UI="knotty" {bitbake} '
+                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
 
         # read port number from bitbake.lock
         self.be.bbport = -1
@@ -497,20 +551,17 @@ class LocalhostBEController(BuildEnvironmentController):
         log = os.path.join(builddir, 'toaster_ui.log')
         local_bitbake = os.path.join(os.path.dirname(os.getenv('BBBASEDIR')),
                                      'bitbake')
+
         if not is_merged_attr:
-            self._shellcmd(['%s bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
-                        '%s %s -u toasterui  --read %s --read %s --token="" >>%s 2>&1;'
-                        'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:%s %s -m)&\"' \
-                        % (env_clean, brbe, self.be.bbport, local_bitbake, bbtargets, confpath, toasterlayers, log,
-                        self.be.bbport, bitbake,)],
+            self._shellcmd([f'{env_clean} bash -c \"(TOASTER_BRBE="{brbe}" BBSERVER="0.0.0.0:{self.be.bbport}" '
+                        f'{bitbake} {bbtargets} -u toasterui  --read {confpath} --read {bblayers} --read {toasterlayers} --token="" >>{log} 2>&1;'
+                        f'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:{self.be.bbport} {bitbake} -m)&\"'],
                         builddir, nowait=True)
         else:
-            self._shellcmd(['%s bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
-                        '%s %s -u toasterui  --token="" >>%s 2>&1;'
-                        'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:%s %s -m)&\"' \
-                        % (env_clean, brbe, self.be.bbport, local_bitbake, bbtargets, log,
-                        self.be.bbport, bitbake,)],
+            self._shellcmd([f'{env_clean} bash -c \"(TOASTER_BRBE="{brbe}" BBSERVER="0.0.0.0:{self.be.bbport}" '
+                        f'{local_bitbake} {bbtargets} -u toasterui  --token="" >>{log} 2>&1;'
+                        f'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:{self.be.bbport} {bitbake} -m)&\"'],
                         builddir, nowait=True)
 
         logger.debug('localhostbecontroller: Build launched, exiting. '
-                     'Follow build logs at %s' % log)
+                     f'Follow build logs at {log}')
-- 
2.43.0

[bitbake-devel][PATCH] toaster: remove dependency on poky distro

[Reyna, David]

From 12b96eba959f298bb8edcbcbd9b35a53d104ecea Mon Sep 17 00:00:00 2001
From: David Reyna <David.Reyna@windriver.com>
Date: Wed, 12 Nov 2025 00:04:24 -0800
Subject: [PATCH] toaster: remove dependency on poky distro

The poky distro is now stripped of its repositories. Toaster had used that
repo as both a quick way to get the three core layers (openembedded-code,
meta-poky, and meta-yocto-bsp) but also a way to get the related bitbake,
which was important because bitbake is not in the Layer Index. Now, Toaster
must explicitly clone each of the core layers plus the proper bitbake layer.

Notes on "orm/fixtures":
  * The "orm/fixtures" files used to hard code the three core layers to
    point to the poky distro, ignoring the Layer Index values
  * The script "lsupdates.py" is now allowed to use the Layer Index to
    properly initialize these layers.
  * The fixture files are still used because (a) they define the supported
    releases for Toaster, plus (b) they now define the proper bitbake repo
    path and branches to associate with the YP releases.
Notes on "localhostbecontroller.py":
  * Always insert the proper bitbake into the layer clone list
  * Insure that both 'BBBASEDIR' and 'BITBAKEDIR' are defined in the
    sub-shells so that oe-init finds the right bitbake (e.g.
    'oe-buildenv-internal')

[YOCTO #16012]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 .../bldcontrol/localhostbecontroller.py       | 67 ++++++++-----------
 lib/toaster/orm/fixtures/gen_fixtures.py      | 10 +--
 lib/toaster/orm/fixtures/poky.xml             | 38 +++++------
 .../orm/management/commands/lsupdates.py      |  6 +-
 4 files changed, 55 insertions(+), 66 deletions(-)

diff --git a/lib/toaster/bldcontrol/localhostbecontroller.py b/lib/toaster/bldcontrol/localhostbecontroller.py
index 37b0a7519..dafda7e16 100644
--- a/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -40,7 +40,8 @@ class LocalhostBEController(BuildEnvironmentController):
 
     def __init__(self, be):
         super(LocalhostBEController, self).__init__(be)
-        self.pokydirname = None
+        self.bitbake_dirname = None
+        self.oecore_dirname = None
         self.islayerset = False
 
     def _shellcmd(self, command, cwd=None, nowait=False,env=None):
@@ -100,7 +101,7 @@ class LocalhostBEController(BuildEnvironmentController):
         # 1. get a list of repos with branches, and map dirpaths for each layer
         gitrepos = {}
 
-        # if we're using a remotely fetched version of bitbake add its git
+        # We are using a remotely fetched version of bitbake - add its git
         # details to the list of repos to clone
         if bitbake.giturl and bitbake.commit:
             gitrepos[(bitbake.giturl, bitbake.commit)] = []
@@ -154,12 +155,20 @@ class LocalhostBEController(BuildEnvironmentController):
         clone_total=len(gitrepos.keys())
         self.setCloneStatus(bitbake,'Started',clone_total,clone_count,'')
         for giturl, commit in gitrepos.keys():
-            self.setCloneStatus(bitbake,'progress',clone_total,clone_count,gitrepos[(giturl, commit)][0][0])
+            layer_name = gitrepos[(giturl, commit)][0][0]
+            self.setCloneStatus(bitbake,'progress',clone_total,clone_count,layer_name)
             clone_count += 1
 
             localdirname = os.path.join(self.be.sourcedir, self.getGitCloneDirectory(giturl, commit))
             logger.debug("localhostbecontroller: giturl %s:%s checking out in current directory %s" % (giturl, commit, localdirname))
 
+            # Capture local paths to release's bitbake and openembedded-core layers
+            if 'bitbake' == layer_name:
+                self.bitbake_dirname = localdirname
+            # Capture local path to release's openembedded-core layer (for 'oe-init-build-env')
+            if 'openembedded-core' == layer_name:
+                self.oecore_dirname = localdirname
+
             # see if our directory is a git repository
             if os.path.exists(localdirname):
                 try:
@@ -192,16 +201,6 @@ class LocalhostBEController(BuildEnvironmentController):
                 # prevent inserted debugging commands from being lost
                 self._shellcmd('git fetch && git reset --hard "%s"' % ref, localdirname,env=git_env)
 
-            # take the localdirname as poky dir if we can find the oe-init-build-env
-            if self.pokydirname is None and os.path.exists(os.path.join(localdirname, "oe-init-build-env")):
-                logger.debug("localhostbecontroller: selected poky dir name %s" % localdirname)
-                self.pokydirname = localdirname
-
-                # make sure we have a working bitbake
-                if not os.path.exists(os.path.join(self.pokydirname, 'bitbake')):
-                    logger.debug("localhostbecontroller: checking bitbake into the poky dirname %s " % self.pokydirname)
-                    self._shellcmd("git clone -b \"%s\" \"%s\" \"%s\" " % (bitbake.commit, bitbake.giturl, os.path.join(self.pokydirname, 'bitbake')),env=git_env)
-
             # verify our repositories
             for name, dirpath, index in gitrepos[(giturl, commit)]:
                 localdirpath = os.path.join(localdirname, dirpath)
@@ -215,21 +214,11 @@ class LocalhostBEController(BuildEnvironmentController):
         self.setCloneStatus(bitbake,'complete',clone_total,clone_count,'')
         logger.debug("localhostbecontroller: current layer list %s " % pformat(layerlist))
 
-        # Resolve self.pokydirname if not resolved yet, consider the scenario
-        # where all layers are local, that's the else clause
-        if self.pokydirname is None:
-            if os.path.exists(os.path.join(self.be.sourcedir, "oe-init-build-env")):
-                logger.debug("localhostbecontroller: selected poky dir name %s" % self.be.sourcedir)
-                self.pokydirname = self.be.sourcedir
-            else:
-                # Alternatively, scan local layers for relative "oe-init-build-env" location
-                for layer in layers:
-                    if os.path.exists(os.path.join(layer.layer_version.layer.local_source_dir,"..","oe-init-build-env")):
-                        logger.debug("localhostbecontroller, setting pokydirname to %s" % (layer.layer_version.layer.local_source_dir))
-                        self.pokydirname = os.path.join(layer.layer_version.layer.local_source_dir,"..")
-                        break
-                else:
-                    logger.error("pokydirname is not set, you will run into trouble!")
+        # If self.bitbake_dirname still missing (e.g. 'local' build), use local bitbake
+        if not self.bitbake_dirname:
+            self.bitbake_dirname = os.environ.get('BBBASEDIR','')
+        if not self.bitbake_dirname:
+            logger.error("bitbake_dirname is not set, you will run into trouble!")
 
         # 5. create custom layer and add custom recipes to it
         for target in targets:
@@ -352,8 +341,11 @@ class LocalhostBEController(BuildEnvironmentController):
         try:
             # insure that the project init/build uses the selected bitbake, and not Toaster's
             del git_env['TEMPLATECONF']
-            del git_env['BBBASEDIR']
             del git_env['BUILDDIR']
+            # Insure that oe-init finds the right bitbake (e.g. 'oe-buildenv-internal')
+            git_env['BBBASEDIR'] = self.bitbake_dirname
+            git_env['BITBAKEDIR'] = self.bitbake_dirname
+
         except KeyError:
             pass
 
@@ -362,7 +354,7 @@ class LocalhostBEController(BuildEnvironmentController):
             builddir = bitbake.req.project.builddir
         else:
             builddir = '%s-toaster-%d' % (self.be.builddir, bitbake.req.project.id)
-        oe_init = os.path.join(self.pokydirname, 'oe-init-build-env')
+        oe_init = os.path.join(self.oecore_dirname, 'oe-init-build-env')
         setup_init = os.path.join(builddir, 'init-build-env')
         # init build environment and build directory structure
         try:
@@ -376,7 +368,7 @@ class LocalhostBEController(BuildEnvironmentController):
                 os.makedirs(builddir, exist_ok = True)
                 build_init_script = os.path.join(builddir,'init-build-env')
                 os.system(f"cp {os.path.join(toaster_run_dir,'init-build-env')} {build_init_script}")
-                os.system(f"sed -i -e 's|^cd .*$|cd {self.pokydirname}|g' {build_init_script}")
+                os.system(f"sed -i -e 's|^cd .*$|cd {self.oecore_dirname}|g' {build_init_script}")
                 os.system(f"sed -i -e 's|^set .*$|set {builddir}|g' {build_init_script}")
                 # Execute the init
                 self._shellcmd(f"bash -c 'source {build_init_script}'", env=git_env)
@@ -476,10 +468,10 @@ class LocalhostBEController(BuildEnvironmentController):
 
         # run bitbake server from the clone if available
         # otherwise pick it from the PATH
-        bitbake = os.path.join(self.pokydirname, 'bitbake', 'bin', 'bitbake')
+        bitbake = os.path.join(self.bitbake_dirname, 'bin', 'bitbake')
         if not os.path.exists(bitbake):
             logger.info("Bitbake not available under %s, will try to use it from PATH" %
-                        self.pokydirname)
+                        self.bitbake_dirname)
             for path in os.environ["PATH"].split(os.pathsep):
                 if os.path.exists(os.path.join(path, 'bitbake')):
                     bitbake = os.path.join(path, 'bitbake')
@@ -494,18 +486,18 @@ class LocalhostBEController(BuildEnvironmentController):
             # Use 'init-build-env' model
             if not is_merged_attr:
                 self._shellcmd(f'{env_clean} bash -c \"source {setup_init}; BITBAKE_UI="knotty" {bitbake} --read {confpath} --read {bblayers} --read {toasterlayers} '
-                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir,env=git_env)
             else:
                 self._shellcmd(f'{env_clean} bash -c \"source {setup_init}; BITBAKE_UI="knotty" {bitbake} '
-                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir,env=git_env)
         else:
             # Use 'oe-init-build-env' model
             if not is_merged_attr:
                 self._shellcmd(f'{env_clean} bash -c \"source {oe_init} {builddir}; BITBAKE_UI="knotty" {bitbake} --read {confpath} --read {bblayers} --read {toasterlayers} '
-                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+                               '--server-only -B 0.0.0.0:0\"', self.be.sourcedir,env=git_env)
             else:
                 self._shellcmd(f'{env_clean} bash -c \"source {oe_init} {builddir}; BITBAKE_UI="knotty" {bitbake} '
-                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir)
+                           '--server-only -B 0.0.0.0:0\"', self.be.sourcedir,env=git_env)
 
         # read port number from bitbake.lock
         self.be.bbport = -1
@@ -551,7 +543,6 @@ class LocalhostBEController(BuildEnvironmentController):
         log = os.path.join(builddir, 'toaster_ui.log')
         local_bitbake = os.path.join(os.path.dirname(os.getenv('BBBASEDIR')),
                                      'bitbake')
-
         if not is_merged_attr:
             self._shellcmd([f'{env_clean} bash -c \"(TOASTER_BRBE="{brbe}" BBSERVER="0.0.0.0:{self.be.bbport}" '
                         f'{bitbake} {bbtargets} -u toasterui  --read {confpath} --read {bblayers} --read {toasterlayers} --token="" >>{log} 2>&1;'
diff --git a/lib/toaster/orm/fixtures/gen_fixtures.py b/lib/toaster/orm/fixtures/gen_fixtures.py
index 6201f679b..c5b0ab7a7 100755
--- a/lib/toaster/orm/fixtures/gen_fixtures.py
+++ b/lib/toaster/orm/fixtures/gen_fixtures.py
@@ -32,7 +32,7 @@ verbose = False
 # It is also policy to include all active LTS releases.
 #
 
-# [Codename, Yocto Project Version, Release Date, Current Version, Support Level, Poky Version, BitBake branch]
+# [0=Codename, 1=Yocto Project Version, 2=Release Date, 3=Current Version, 4=Support Level, 5=Poky Version, 6=BitBake branch]
 current_releases = [
     # Release slot #1
     ['Scarthgap','5.0','April 2024','5.0.0 (April 2024)','Long Term Support (until April 2028)','','2.8'],
@@ -43,7 +43,7 @@ current_releases = [
     # Release slot #4
     ['Whinlatter','5.3','October 2025','5.3.0 (October 2024)','Support for 7 months (until May 2026)','','2.14'],
     ['Walnascar','5.2','April 2025','5.2.0 (April 2025)','Support for 7 months (until October 2025)','','2.12'],
-    #['Styhead','5.1','November 2024','5.1.0 (November 2024)','Support for 7 months (until May 2025)','','2.10'],
+    ['Styhead','5.1','November 2024','5.1.0 (November 2024)','Support for 7 months (until May 2025)','','2.10'],
     #['Nanbield','4.3','November 2023','4.3.0 (November 2023)','Support for 7 months (until May 2024)','','2.6'],
     #['Mickledore','4.2','April 2023','4.2.0 (April 2023)','Support for 7 months (until October 2023)','','2.4'],
     #['Langdale','4.1','October 2022','4.1.2 (January 2023)','Support for 7 months (until May 2023)','','2.2'],
@@ -81,9 +81,9 @@ prolog_template = '''\
 bitbakeversion_poky_template = '''\
   <object model="orm.bitbakeversion" pk="{{bitbake_id}}">
     <field type="CharField" name="name">{{name}}</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
     <field type="CharField" name="branch">{{branch}}</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
 '''
 bitbakeversion_oecore_template = '''\
@@ -224,7 +224,7 @@ def generate_poky():
             params['release'] = params['release'][0].lower() + params['release'][1:]
         params['name'] = params['release']
         params['bitbake_id'] = str(i+1)
-        params['branch'] = params['release']
+        params['branch'] = release[6]   # Bitbake branch
         print_template(bitbakeversion_poky_template,params,fd)
     print_str('',fd)
 
diff --git a/lib/toaster/orm/fixtures/poky.xml b/lib/toaster/orm/fixtures/poky.xml
index 6cf4f0687..34b83e971 100644
--- a/lib/toaster/orm/fixtures/poky.xml
+++ b/lib/toaster/orm/fixtures/poky.xml
@@ -9,45 +9,45 @@
   <!-- Bitbake versions which correspond to the metadata release -->
   <object model="orm.bitbakeversion" pk="1">
     <field type="CharField" name="name">scarthgap</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">scarthgap</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
+    <field type="CharField" name="branch">2.8</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="2">
     <field type="CharField" name="name">HEAD</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
     <field type="CharField" name="branch">HEAD</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="3">
     <field type="CharField" name="name">master</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
     <field type="CharField" name="branch">master</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="4">
     <field type="CharField" name="name">whinlatter</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">whinlatter</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
+    <field type="CharField" name="branch">2.14</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="5">
     <field type="CharField" name="name">walnascar</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">walnascar</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
+    <field type="CharField" name="branch">2.12</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="6">
     <field type="CharField" name="name">styhead</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">styhead</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
+    <field type="CharField" name="branch">2.10</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
   <object model="orm.bitbakeversion" pk="7">
     <field type="CharField" name="name">kirkstone</field>
-    <field type="CharField" name="giturl">git://git.yoctoproject.org/poky</field>
-    <field type="CharField" name="branch">kirkstone</field>
-    <field type="CharField" name="dirpath">bitbake</field>
+    <field type="CharField" name="giturl">git://git.openembedded.org/bitbake</field>
+    <field type="CharField" name="branch">2.0</field>
+    <field type="CharField" name="dirpath"></field>
   </object>
 
 
diff --git a/lib/toaster/orm/management/commands/lsupdates.py b/lib/toaster/orm/management/commands/lsupdates.py
index 6d64830eb..0ee00aa15 100644
--- a/lib/toaster/orm/management/commands/lsupdates.py
+++ b/lib/toaster/orm/management/commands/lsupdates.py
@@ -138,9 +138,7 @@ class Command(BaseCommand):
                 l.summary = index.layerItems[id].summary
                 l.description = index.layerItems[id].description
 
-                if created:
-                    # predefined layers in the fixtures (for example poky.xml)
-                    # always preempt the Layer Index for these values
+                if True: # (if created:) always override fixture defaults now that poky distro is stripped
                     l.vcs_url = index.layerItems[id].vcs_url
                     l.vcs_web_url = index.layerItems[id].vcs_web_url
                     l.vcs_web_tree_base_url = index.layerItems[id].vcs_web_tree_base_url
@@ -179,7 +177,7 @@ class Command(BaseCommand):
                     index.layerBranches[id].layer_id)
                 continue
 
-            if created:
+            if True: # (if created:) always override fixture defaults now that poky distro is stripped
                 lv.release = li_branch_id_to_toaster_release[index.layerBranches[id].branch_id]
                 lv.up_date = index.layerBranches[id].updated
                 lv.commit = index.layerBranches[id].actual_branch
-- 
2.43.0

[bitbake-devel][PATCH v2] cookerdata: Include "originating" recipe name when parsing

[Joshua Watt]

When a bbappend file is parsed, the FILE variable is set to the name of
the actual file being parsed (e.g. the name of the bbappend). Since
PN/PV etc. are derived from FILE, this means that they can be
misleading in the event that bbappend is using wildcards (e.g. PV might
have the value of "%" instead of the actual version number).

In order to allow bbappends to derived the actual information of the
recipe, capture the name of the original recipe being parsed as
ORIG_FILE when parsing a new recipe.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 lib/bb/cookerdata.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 22ac95eac..fe41bb3bf 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -512,6 +512,8 @@ class CookerDataBuilder(object):
         bb_data.setVar("__BBMULTICONFIG", mc)
         bb_data.setVar("FILE_LAYERNAME", layername)
 
+        bb_data.setVar("ORIG_FILE", bbfile)
+
         bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
         bb.parse.cached_mtime_noerror(bbfile_loc)
 
-- 
2.52.0

Re: [bitbake-devel][PATCH v2] cookerdata: Include "originating" recipe name when parsing

[Richard Purdie]

On Fri, 2025-12-19 at 15:36 -0700, Joshua Watt via lists.openembedded.org wrote:
> When a bbappend file is parsed, the FILE variable is set to the name of
> the actual file being parsed (e.g. the name of the bbappend). Since
> PN/PV etc. are derived from FILE, this means that they can be
> misleading in the event that bbappend is using wildcards (e.g. PV might
> have the value of "%" instead of the actual version number).
> 
> In order to allow bbappends to derived the actual information of the
> recipe, capture the name of the original recipe being parsed as
> ORIG_FILE when parsing a new recipe.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  lib/bb/cookerdata.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> index 22ac95eac..fe41bb3bf 100644
> --- a/lib/bb/cookerdata.py
> +++ b/lib/bb/cookerdata.py
> @@ -512,6 +512,8 @@ class CookerDataBuilder(object):
>          bb_data.setVar("__BBMULTICONFIG", mc)
>          bb_data.setVar("FILE_LAYERNAME", layername)
>  
> +        bb_data.setVar("ORIG_FILE", bbfile)
> +
>          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
>          bb.parse.cached_mtime_noerror(bbfile_loc)

We discussed this a bit on the review call today, I appreciate Joshua
wasn't there but it made sense to try and unblock this.

In brief, the concerns mentioned were:

* no tests/docs
* not wanting to encourage wide use of this
  (conflicting with the above)
* whether the name is right

The idea of using a __ prefix for the variable was mentioned. That
would allow Joshua to address the issues he has but not make it
'official' recommended API. That might be the compromise we need.

For the name itself, I'm really torn. For variables which bitbake
itself responds to, I've tried to standardise on a BB_ prefix. This is
more a variable bitbake exposes for use elsewhere and we have the FILE_
prefix already being used but I'm torn on whether we should continue
that theme or use something with BB_...

Cheers,

Richard

Re: [bitbake-devel][PATCH v2] cookerdata: Include "originating" recipe name when parsing

[Joshua Watt]

On Thu, Feb 19, 2026 at 9:32 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2025-12-19 at 15:36 -0700, Joshua Watt via lists.openembedded.org wrote:
> > When a bbappend file is parsed, the FILE variable is set to the name of
> > the actual file being parsed (e.g. the name of the bbappend). Since
> > PN/PV etc. are derived from FILE, this means that they can be
> > misleading in the event that bbappend is using wildcards (e.g. PV might
> > have the value of "%" instead of the actual version number).
> >
> > In order to allow bbappends to derived the actual information of the
> > recipe, capture the name of the original recipe being parsed as
> > ORIG_FILE when parsing a new recipe.
> >
> > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > ---
> >  lib/bb/cookerdata.py | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> > index 22ac95eac..fe41bb3bf 100644
> > --- a/lib/bb/cookerdata.py
> > +++ b/lib/bb/cookerdata.py
> > @@ -512,6 +512,8 @@ class CookerDataBuilder(object):
> >          bb_data.setVar("__BBMULTICONFIG", mc)
> >          bb_data.setVar("FILE_LAYERNAME", layername)
> >
> > +        bb_data.setVar("ORIG_FILE", bbfile)
> > +
> >          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
> >          bb.parse.cached_mtime_noerror(bbfile_loc)
>
> We discussed this a bit on the review call today, I appreciate Joshua
> wasn't there but it made sense to try and unblock this.
>
> In brief, the concerns mentioned were:
>
> * no tests/docs

A test should be pretty easy to do, I'll see if I can write one up; I
can try to write some docs also.

> * not wanting to encourage wide use of this
>   (conflicting with the above)
> * whether the name is right
>
> The idea of using a __ prefix for the variable was mentioned. That
> would allow Joshua to address the issues he has but not make it
> 'official' recommended API. That might be the compromise we need.
>
> For the name itself, I'm really torn. For variables which bitbake
> itself responds to, I've tried to standardise on a BB_ prefix. This is
> more a variable bitbake exposes for use elsewhere and we have the FILE_
> prefix already being used but I'm torn on whether we should continue
> that theme or use something with BB_...

BB_ORIG_FILE then? I'd probably avoid the __BB prefix since that is
reserved for variables which bitbake used internally (only).

Alternatively, if we want to set a new convention of "hidden"
variables, we could perhaps to do something like "bb_ORIG_FILE", but
that is kinda hard on the eyes :)

>
> Cheers,
>
> Richard
>
>
>
>
>
>
>
>
>

Re: [bitbake-devel][PATCH v2] cookerdata: Include "originating" recipe name when parsing

[Richard Purdie]

On Thu, 2026-02-19 at 09:39 -0700, Joshua Watt wrote:
> On Thu, Feb 19, 2026 at 9:32 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > On Fri, 2025-12-19 at 15:36 -0700, Joshua Watt via lists.openembedded.org wrote:
> > > When a bbappend file is parsed, the FILE variable is set to the name of
> > > the actual file being parsed (e.g. the name of the bbappend). Since
> > > PN/PV etc. are derived from FILE, this means that they can be
> > > misleading in the event that bbappend is using wildcards (e.g. PV might
> > > have the value of "%" instead of the actual version number).
> > > 
> > > In order to allow bbappends to derived the actual information of the
> > > recipe, capture the name of the original recipe being parsed as
> > > ORIG_FILE when parsing a new recipe.
> > > 
> > > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > > ---
> > >  lib/bb/cookerdata.py | 2 ++
> > >  1 file changed, 2 insertions(+)
> > > 
> > > diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> > > index 22ac95eac..fe41bb3bf 100644
> > > --- a/lib/bb/cookerdata.py
> > > +++ b/lib/bb/cookerdata.py
> > > @@ -512,6 +512,8 @@ class CookerDataBuilder(object):
> > >          bb_data.setVar("__BBMULTICONFIG", mc)
> > >          bb_data.setVar("FILE_LAYERNAME", layername)
> > > 
> > > +        bb_data.setVar("ORIG_FILE", bbfile)
> > > +
> > >          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
> > >          bb.parse.cached_mtime_noerror(bbfile_loc)
> > 
> > We discussed this a bit on the review call today, I appreciate Joshua
> > wasn't there but it made sense to try and unblock this.
> > 
> > In brief, the concerns mentioned were:
> > 
> > * no tests/docs
> 
> A test should be pretty easy to do, I'll see if I can write one up; I
> can try to write some docs also.

Right, those are straightforward in many ways but it also establishes
official API which is a concern. That isn't what is holding this up.

> 
> > * not wanting to encourage wide use of this
> >   (conflicting with the above)
> > * whether the name is right
> > 
> > The idea of using a __ prefix for the variable was mentioned. That
> > would allow Joshua to address the issues he has but not make it
> > 'official' recommended API. That might be the compromise we need.
> > 
> > For the name itself, I'm really torn. For variables which bitbake
> > itself responds to, I've tried to standardise on a BB_ prefix. This is
> > more a variable bitbake exposes for use elsewhere and we have the FILE_
> > prefix already being used but I'm torn on whether we should continue
> > that theme or use something with BB_...
> 
> BB_ORIG_FILE then? I'd probably avoid the __BB prefix since that is
> reserved for variables which bitbake used internally (only).

Someone did mention "recipe" instead of "file" might be an idea so more
like BB_RECIPE_FILE maybe?

> Alternatively, if we want to set a new convention of "hidden"
> variables, we could perhaps to do something like "bb_ORIG_FILE", but
> that is kinda hard on the eyes :)

I want to try and unblock you, yet at the same time discourage people
from using this. '__' is the best chance of that we have. I really
don't want to do bb_ as that is horrible on the eyes. Part of me
realises it is completely futile to even try and do that and thinks I
may as well not try.

I think the horrors people could implement with this are problematic
but I really am not sure we have many other good choices. Layer
specific dangling bbappend controls aren't particularly nice either.

Cheers,

Richard

Re: [bitbake-devel][PATCH v2] cookerdata: Include "originating" recipe name when parsing

[Joshua Watt]

On Thu, Feb 19, 2026 at 9:55 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2026-02-19 at 09:39 -0700, Joshua Watt wrote:
> > On Thu, Feb 19, 2026 at 9:32 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > >
> > > On Fri, 2025-12-19 at 15:36 -0700, Joshua Watt via lists.openembedded.org wrote:
> > > > When a bbappend file is parsed, the FILE variable is set to the name of
> > > > the actual file being parsed (e.g. the name of the bbappend). Since
> > > > PN/PV etc. are derived from FILE, this means that they can be
> > > > misleading in the event that bbappend is using wildcards (e.g. PV might
> > > > have the value of "%" instead of the actual version number).
> > > >
> > > > In order to allow bbappends to derived the actual information of the
> > > > recipe, capture the name of the original recipe being parsed as
> > > > ORIG_FILE when parsing a new recipe.
> > > >
> > > > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > > > ---
> > > >  lib/bb/cookerdata.py | 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
> > > > index 22ac95eac..fe41bb3bf 100644
> > > > --- a/lib/bb/cookerdata.py
> > > > +++ b/lib/bb/cookerdata.py
> > > > @@ -512,6 +512,8 @@ class CookerDataBuilder(object):
> > > >          bb_data.setVar("__BBMULTICONFIG", mc)
> > > >          bb_data.setVar("FILE_LAYERNAME", layername)
> > > >
> > > > +        bb_data.setVar("ORIG_FILE", bbfile)
> > > > +
> > > >          bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
> > > >          bb.parse.cached_mtime_noerror(bbfile_loc)
> > >
> > > We discussed this a bit on the review call today, I appreciate Joshua
> > > wasn't there but it made sense to try and unblock this.
> > >
> > > In brief, the concerns mentioned were:
> > >
> > > * no tests/docs
> >
> > A test should be pretty easy to do, I'll see if I can write one up; I
> > can try to write some docs also.
>
> Right, those are straightforward in many ways but it also establishes
> official API which is a concern. That isn't what is holding this up.
>
> >
> > > * not wanting to encourage wide use of this
> > >   (conflicting with the above)
> > > * whether the name is right
> > >
> > > The idea of using a __ prefix for the variable was mentioned. That
> > > would allow Joshua to address the issues he has but not make it
> > > 'official' recommended API. That might be the compromise we need.
> > >
> > > For the name itself, I'm really torn. For variables which bitbake
> > > itself responds to, I've tried to standardise on a BB_ prefix. This is
> > > more a variable bitbake exposes for use elsewhere and we have the FILE_
> > > prefix already being used but I'm torn on whether we should continue
> > > that theme or use something with BB_...
> >
> > BB_ORIG_FILE then? I'd probably avoid the __BB prefix since that is
> > reserved for variables which bitbake used internally (only).
>
> Someone did mention "recipe" instead of "file" might be an idea so more
> like BB_RECIPE_FILE maybe?
>
> > Alternatively, if we want to set a new convention of "hidden"
> > variables, we could perhaps to do something like "bb_ORIG_FILE", but
> > that is kinda hard on the eyes :)
>
> I want to try and unblock you, yet at the same time discourage people
> from using this. '__' is the best chance of that we have. I really
> don't want to do bb_ as that is horrible on the eyes. Part of me
> realises it is completely futile to even try and do that and thinks I
> may as well not try.

Fair enough. __BB_RECIPE_FILE works for me

>
> I think the horrors people could implement with this are problematic
> but I really am not sure we have many other good choices. Layer
> specific dangling bbappend controls aren't particularly nice either.
>
> Cheers,
>
> Richard
>
>
>

[bitbake-devel][PATCH v3] cookerdata: Include "originating" recipe name when parsing

[Joshua Watt]

From: Joshua Watt <jpewhacker@gmail.com>

When a bbappend file is parsed, the FILE variable is set to the name of
the actual file being parsed (e.g. the name of the bbappend). Since
PN/PV etc. are derived from FILE, this means that they can be
misleading in the event that bbappend is using wildcards (e.g. PV might
have the value of "%" instead of the actual version number).

In order to allow bbappends to derived the actual information of the
recipe, capture the name of the original recipe being parsed as
__BB_RECIPE_FILE when parsing a new recipe. The value of this variable
doesn't change when parsing .bbappend or .inc file associated with the
recipe

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
V3: Rename to __BB_RECIPE_FILE, add documentation and tests


 .../bitbake-user-manual-ref-variables.rst     |   4 +
 lib/bb/cookerdata.py                          |   2 +
 lib/bb/tests/parse-tests/classes/base.bbclass |   5 +
 .../classes/recipe-file-class.bbclass         |   2 +
 lib/bb/tests/parse-tests/conf/bitbake.conf    |  15 ++
 lib/bb/tests/parse.py                         | 132 +++++++++++++++++-
 6 files changed, 159 insertions(+), 1 deletion(-)
 create mode 100644 lib/bb/tests/parse-tests/classes/base.bbclass
 create mode 100644 lib/bb/tests/parse-tests/classes/recipe-file-class.bbclass
 create mode 100644 lib/bb/tests/parse-tests/conf/bitbake.conf

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
index 3a01144ff..5940e6670 100644
--- a/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
+++ b/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -720,6 +720,10 @@ overview of their function and contents.
          pressure on the system. Monitor the varying value after ``Mem:`` above
          to set a sensible value.
 
+   :term:`__BB_RECIPE_FILE`
+      Specifies the name of the recipe file that is being parsed. This is the
+      same even if when evalated in a bbappend, include or bbclass file.
+
    :term:`BB_RUNFMT`
       Specifies the name of the executable script files (i.e. run files)
       saved into ``${``\ :term:`T`\ ``}``. By default, the
diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py
index 22ac95eac..59f808c96 100644
--- a/lib/bb/cookerdata.py
+++ b/lib/bb/cookerdata.py
@@ -512,6 +512,8 @@ class CookerDataBuilder(object):
         bb_data.setVar("__BBMULTICONFIG", mc)
         bb_data.setVar("FILE_LAYERNAME", layername)
 
+        bb_data.setVar("__BB_RECIPE_FILE", bbfile)
+
         bbfile_loc = os.path.abspath(os.path.dirname(bbfile))
         bb.parse.cached_mtime_noerror(bbfile_loc)
 
diff --git a/lib/bb/tests/parse-tests/classes/base.bbclass b/lib/bb/tests/parse-tests/classes/base.bbclass
new file mode 100644
index 000000000..db8898e12
--- /dev/null
+++ b/lib/bb/tests/parse-tests/classes/base.bbclass
@@ -0,0 +1,5 @@
+# At least one task is required for bitbake to parse
+do_fetch() {
+    :
+}
+addtask do_fetch
diff --git a/lib/bb/tests/parse-tests/classes/recipe-file-class.bbclass b/lib/bb/tests/parse-tests/classes/recipe-file-class.bbclass
new file mode 100644
index 000000000..4682dc6c3
--- /dev/null
+++ b/lib/bb/tests/parse-tests/classes/recipe-file-class.bbclass
@@ -0,0 +1,2 @@
+BBCLASS_RECIPE_FILE := "${@os.path.basename(d.getVar('__BB_RECIPE_FILE'))}"
+BBCLASS_FILE := "${@os.path.basename(d.getVar('FILE'))}"
diff --git a/lib/bb/tests/parse-tests/conf/bitbake.conf b/lib/bb/tests/parse-tests/conf/bitbake.conf
new file mode 100644
index 000000000..e03625061
--- /dev/null
+++ b/lib/bb/tests/parse-tests/conf/bitbake.conf
@@ -0,0 +1,15 @@
+CACHE = "${TOPDIR}/cache"
+THISDIR = "${@os.path.dirname(d.getVar('FILE'))}"
+COREBASE := "${@os.path.normpath(os.path.dirname(d.getVar('FILE')+'/../../'))}"
+EXTRA_BBFILES ?= ""
+BBFILES = "${COREBASE}/recipes/*.bb ${COREBASE}/recipes/*.bbappend ${EXTRA_BBFILES}"
+PROVIDES = "${PN}"
+PN = "${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0]}"
+PF = "${BB_CURRENT_MC}:${PN}"
+export PATH
+TMPDIR ??= "${TOPDIR}"
+STAMP = "${TMPDIR}/stamps/${PN}"
+T = "${TMPDIR}/workdir/${PN}/temp"
+BB_NUMBER_THREADS = "4"
+
+BB_BASEHASH_IGNORE_VARS = "BB_CURRENT_MC BB_HASHSERVE TMPDIR TOPDIR SLOWTASKS SSTATEVALID FILE BB_CURRENTTASK"
diff --git a/lib/bb/tests/parse.py b/lib/bb/tests/parse.py
index d3867ece9..6ac2137e0 100644
--- a/lib/bb/tests/parse.py
+++ b/lib/bb/tests/parse.py
@@ -11,6 +11,8 @@ import tempfile
 import logging
 import bb
 import os
+import subprocess
+import textwrap
 
 logger = logging.getLogger('BitBake.TestParse')
 
@@ -210,7 +212,7 @@ python () {
     #
     # Test based upon a real world data corruption issue. One
     # data store changing a variable poked through into a different data
-    # store. This test case replicates that issue where the value 'B' would 
+    # store. This test case replicates that issue where the value 'B' would
     # become unset/disappear.
     #
     def test_parse_classextend_contamination(self):
@@ -508,3 +510,131 @@ EXTRA_OECONF:append = " foobar"
             test_helper("require some3.conf", " foobar")
             test_helper("include_all some.conf", " bar foo")
             test_helper("include_all some3.conf", " foobar")
+
+    def test_file_variables(self):
+        # Tests the values of FILE and __BB_RECIPE_FILE in different
+        # combinations of bbappends, includes, and inherits
+
+        def write_file(path, data):
+            with open(path, "w") as f:
+                f.write(textwrap.dedent(data))
+
+        def run_bitbake(cmd, builddir, extraenv={}):
+            env = os.environ.copy()
+            env["BBPATH"] = os.path.realpath(os.path.join(os.path.dirname(__file__), "parse-tests"))
+            env["BB_ENV_PASSTHROUGH_ADDITIONS"] = "TOPDIR"
+            env["TOPDIR"] = builddir
+            for k, v in extraenv.items():
+                env[k] = v
+                env["BB_ENV_PASSTHROUGH_ADDITIONS"] = env["BB_ENV_PASSTHROUGH_ADDITIONS"] + " " + k
+            try:
+                return subprocess.check_output(cmd, env=env, stderr=subprocess.STDOUT, universal_newlines=True, cwd=builddir)
+            except subprocess.CalledProcessError as e:
+                self.fail("Command %s failed with %s" % (cmd, e.output))
+
+        with tempfile.TemporaryDirectory(prefix="parserecipes") as recipes, tempfile.TemporaryDirectory(prefix="parsetest") as builddir:
+            extraenv = {
+                "EXTRA_BBFILES": f"{recipes}/*.bb {recipes}/*.bbappend",
+            }
+
+            inc_path = f"{recipes}/recipe-file.inc"
+            bbappend_path = f"{recipes}/recipe-%.bbappend"
+            recipe_path = f"{recipes}/recipe-file1.bb"
+
+            # __BB_RECIPE_FILE should always be the name of .bb file, even
+            # when set in a bbappend. FILE is the name of the bbappend
+            write_file(recipe_path, "")
+            write_file(bbappend_path,
+                """\
+                BBAPPEND_RECIPE_FILE := "${@os.path.basename(d.getVar('__BB_RECIPE_FILE'))}"
+                BBAPPEND_FILE := "${@os.path.basename(d.getVar('FILE'))}"
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('BBAPPEND_FILE="recipe-%.bbappend"', output)
+            self.assertIn(f'BBAPPEND_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # __BB_RECIPE_FILE should always be the name of .bb file, even when
+            # set in an include file. FILE is the name of the include
+            write_file(recipe_path,
+                """\
+                require recipe-file.inc
+                """
+            )
+            write_file(inc_path,
+                """\
+                INC_RECIPE_FILE := "${@os.path.basename(d.getVar('__BB_RECIPE_FILE'))}"
+                INC_FILE := "${@os.path.basename(d.getVar('FILE'))}"
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('INC_FILE="recipe-file.inc"', output)
+            self.assertIn(f'INC_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # Test when the include file is included from a bbappend
+            write_file(recipe_path, "")
+            write_file(bbappend_path,
+                """\
+                require recipe-file.inc
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('INC_FILE="recipe-file.inc"', output)
+            self.assertIn(f'INC_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # Test the variables in a bbclass when inherited directly in the
+            # recipe. Note that FILE still refers to the recipe in a bbclass
+            write_file(recipe_path,
+                """\
+                inherit recipe-file-class
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('BBCLASS_FILE="recipe-file1.bb"', output)
+            self.assertIn(f'BBCLASS_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # Test the variables when the inherit is in a bbappend. In this
+            # case, FILE is the bbappend
+            write_file(recipe_path, "")
+            write_file(bbappend_path,
+                """\
+                inherit recipe-file-class
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('BBCLASS_FILE="recipe-%.bbappend"', output)
+            self.assertIn(f'BBCLASS_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # Test the variables when the inherit is in a include. In this
+            # case, FILE is the include file
+            write_file(recipe_path,
+                """\
+                require recipe-file.inc
+                """
+            )
+            write_file(bbappend_path, "")
+            write_file(inc_path,
+                """\
+                inherit recipe-file-class
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('BBCLASS_FILE="recipe-file.inc"', output)
+            self.assertIn(f'BBCLASS_RECIPE_FILE="recipe-file1.bb"', output)
+
+            # Test the variables when the inherit is in a include included from
+            # a bbappend. In this case, FILE is the include file
+            write_file(recipe_path, "")
+            write_file(bbappend_path,
+                """\
+                require recipe-file.inc
+                """
+            )
+            write_file(inc_path,
+                """\
+                inherit recipe-file-class
+                """
+            )
+            output = run_bitbake(["bitbake", "-e", "recipe-file1"], builddir, extraenv).splitlines()
+            self.assertIn('BBCLASS_FILE="recipe-file.inc"', output)
+            self.assertIn(f'BBCLASS_RECIPE_FILE="recipe-file1.bb"', output)
-- 
2.53.0