[PATCH] bitbake-layers: layerindex-fetch: respect --branch for already-configured layers

[PATCH] bitbake-layers: layerindex-fetch: respect --branch for already-configured layers

[Osama Abdelkader]

[YOCTO #7852]

The layerindex-fetch command has two issues when --branch is specified:

1. If a layer is already in bblayers.conf, the command exits early with
   "You already have the requested layer(s)" without checking if the
   layer is on the requested branch.

2. For layers in bblayers.conf (cooker layers), the branch checking code
   added in commit 138dd7883ee (2022) never executes because cooker layers
   are skipped entirely in the fetch loop.

This patch fixes both issues by:

- Preventing the early exit (fast path) when --branch is specified, forcing
  branch verification even for already-configured layers.

- Adding branch check/switch logic for cooker layers when --branch is
  specified. The code finds each layer's git repository and calls
  get_fetch_layer() with the requested branch to perform the validation
  and checkout if needed.

- Fixing the branch detection in get_fetch_layer() to use
  'git rev-parse --abbrev-ref HEAD' instead of parsing 'git branch' output.

- Fixing undefined 'stderr' variable (should be 'completed_proc.stderr').

This completes the fix started in commit 138dd7883ee, allowing users to
use --branch to switch already-configured layers to different branches.

Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
---
 bitbake/lib/bblayers/layerindex.py | 51 ++++++++++++++++++++++++++----
 1 file changed, 45 insertions(+), 6 deletions(-)

diff --git a/bitbake/lib/bblayers/layerindex.py b/bitbake/lib/bblayers/layerindex.py
index ba91fac669..72d2eb8fb7 100644
--- a/bitbake/lib/bblayers/layerindex.py
+++ b/bitbake/lib/bblayers/layerindex.py
@@ -55,13 +55,15 @@ class LayerIndexPlugin(ActionPlugin):
             switching branches if necessary and possible.
             """
             base_cmd = ['git', '--git-dir=%s/.git' % repodir, '--work-tree=%s' % repodir]
-            cmd = base_cmd + ['branch']
+            # Get current branch name
+            cmd = base_cmd + ['rev-parse', '--abbrev-ref', 'HEAD']
             completed_proc = subprocess.run(cmd, text=True, capture_output=True)
             if completed_proc.returncode:
-                logger.error("Unable to validate repo %s (%s)" % (repodir, stderr))
+                logger.error("Unable to validate repo %s (%s)" % (repodir, completed_proc.stderr))
                 return None, None, None
             else:
-                if branch != completed_proc.stdout[2:-1]:
+                current_branch = completed_proc.stdout.strip()
+                if branch != current_branch:
                     cmd = base_cmd + ['status', '--short']
                     completed_proc = subprocess.run(cmd, text=True, capture_output=True)
                     if completed_proc.stdout.count('\n') != 0:
@@ -118,7 +120,8 @@ class LayerIndexPlugin(ActionPlugin):
 
         # Fast path, check if we already have what has been requested!
         (dependencies, invalidnames) = cookerIndex.find_dependencies(names=args.layername, ignores=ignore_layers)
-        if not args.show_only and not invalidnames:
+        if not args.show_only and not invalidnames and not args.branch:
+            # Only skip if no specific branch was requested
             logger.plain("You already have the requested layer(s): %s" % args.layername)
             return 0
 
@@ -202,8 +205,44 @@ class LayerIndexPlugin(ActionPlugin):
             for deplayerbranch in dependencies:
                 layerBranch = dependencies[deplayerbranch][0]
 
-                if layerBranch.index.config['TYPE'] == 'cooker':
-                    # Anything loaded via cooker is already local, skip it
+                is_cooker_layer = layerBranch.index.config['TYPE'] == 'cooker'
+
+                if is_cooker_layer:
+                    # Anything loaded via cooker is already local
+                    # If a branch was specified, we still need to check/switch branches
+                    if args.branch:
+                        # Map collection names to layer paths (same as cooker plugin does)
+                        layerconfs = self.tinfoil.config_data.varhistory.get_variable_items_files('BBFILE_COLLECTIONS')
+                        bbfile_collections = {layer: os.path.dirname(os.path.dirname(path)) for layer, path in layerconfs.items()}
+
+                        # Get the layer path for this cooker layer's collection
+                        collection_name = layerBranch.collection
+                        if collection_name in bbfile_collections:
+                            bblayer = bbfile_collections[collection_name]
+                            if os.path.exists(bblayer):
+                                try:
+                                    # Find the git repository root
+                                    gitdir_cmd = ['git', '-C', bblayer, 'rev-parse', '--show-toplevel']
+                                    result = subprocess.run(gitdir_cmd, capture_output=True, text=True)
+                                    if result.returncode == 0:
+                                        repo_root = result.stdout.strip()
+                                        # Get the subdir relative to repo root
+                                        rel_subdir = os.path.relpath(bblayer, repo_root) if bblayer != repo_root else ''
+                                        # Get parent dir of repo as fetchdir
+                                        parent_fetchdir = os.path.dirname(repo_root)
+                                        # Now call get_fetch_layer to check/switch branch
+                                        # Use args.branch (the requested branch) not layerBranch.actual_branch (current branch)
+                                        _, name, _ = self.get_fetch_layer(parent_fetchdir,
+                                                                          layerBranch.layer.vcs_url,
+                                                                          rel_subdir,
+                                                                          not args.show_only,
+                                                                          args.branch,
+                                                                          args.shallow)
+                                        if not name:
+                                            # Error during branch check/switch
+                                            return 1
+                                except subprocess.SubprocessError:
+                                    pass  # Not a git repo, skip
                     continue
 
                 subdir, name, layerdir = self.get_fetch_layer(fetchdir,
-- 
2.43.0

Re: [PATCH] bitbake-layers: layerindex-fetch: respect --branch for already-configured layers

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 2283 bytes --]

On Sat, 2025-11-01 at 15:56 +0200, Osama Abdelkader wrote:
> [YOCTO #7852]
> 
> The layerindex-fetch command has two issues when --branch is specified:
> 
> 1. If a layer is already in bblayers.conf, the command exits early with
>    "You already have the requested layer(s)" without checking if the
>    layer is on the requested branch.
> 
> 2. For layers in bblayers.conf (cooker layers), the branch checking code
>    added in commit 138dd7883ee (2022) never executes because cooker layers
>    are skipped entirely in the fetch loop.
> 
> This patch fixes both issues by:
> 
> - Preventing the early exit (fast path) when --branch is specified, forcing
>   branch verification even for already-configured layers.
> 
> - Adding branch check/switch logic for cooker layers when --branch is
>   specified. The code finds each layer's git repository and calls
>   get_fetch_layer() with the requested branch to perform the validation
>   and checkout if needed.
> 
> - Fixing the branch detection in get_fetch_layer() to use
>   'git rev-parse --abbrev-ref HEAD' instead of parsing 'git branch' output.
> 
> - Fixing undefined 'stderr' variable (should be 'completed_proc.stderr').
> 
> This completes the fix started in commit 138dd7883ee, allowing users to
> use --branch to switch already-configured layers to different branches.
> 
> Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>

Hi,

Thanks for the contribution! Apologies for the delay in reviewing it, we've
been focused on finishing the 5.3 release.

This looks like a useful change but it's difficult to review in its current
form. Could you split this into multiple commits? Based on the above detail
there are 4 different logical changes here so we would prefer each of those as
a separate patch.

I'd especially like the completed_proc.stderr fix to be isolated as it
may need to be backported to a stable release. So this fix should come
first in the series - best practice is to put patches that fix bugs
first, then patches that add features.

I think you'll also need to apply these changes to the bitbake
repository rather than the poky repository to remove the unnecessary
'bitbake/' prefix on the file paths.

Best regards,

-- 
Paul Barker

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [PATCH] bitbake-layers: layerindex-fetch: respect --branch for already-configured layers

[Osama Abdelkader]

On Mon, Dec 08, 2025 at 09:40:05PM +0000, Paul Barker wrote:
> On Sat, 2025-11-01 at 15:56 +0200, Osama Abdelkader wrote:
> > [YOCTO #7852]
> > 
> > The layerindex-fetch command has two issues when --branch is specified:
> > 
> > 1. If a layer is already in bblayers.conf, the command exits early with
> >    "You already have the requested layer(s)" without checking if the
> >    layer is on the requested branch.
> > 
> > 2. For layers in bblayers.conf (cooker layers), the branch checking code
> >    added in commit 138dd7883ee (2022) never executes because cooker layers
> >    are skipped entirely in the fetch loop.
> > 
> > This patch fixes both issues by:
> > 
> > - Preventing the early exit (fast path) when --branch is specified, forcing
> >   branch verification even for already-configured layers.
> > 
> > - Adding branch check/switch logic for cooker layers when --branch is
> >   specified. The code finds each layer's git repository and calls
> >   get_fetch_layer() with the requested branch to perform the validation
> >   and checkout if needed.
> > 
> > - Fixing the branch detection in get_fetch_layer() to use
> >   'git rev-parse --abbrev-ref HEAD' instead of parsing 'git branch' output.
> > 
> > - Fixing undefined 'stderr' variable (should be 'completed_proc.stderr').
> > 
> > This completes the fix started in commit 138dd7883ee, allowing users to
> > use --branch to switch already-configured layers to different branches.
> > 
> > Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
> 
> Hi,
> 
> Thanks for the contribution! Apologies for the delay in reviewing it, we've
> been focused on finishing the 5.3 release.
> 
> This looks like a useful change but it's difficult to review in its current
> form. Could you split this into multiple commits? Based on the above detail
> there are 4 different logical changes here so we would prefer each of those as
> a separate patch.
> 
> I'd especially like the completed_proc.stderr fix to be isolated as it
> may need to be backported to a stable release. So this fix should come
> first in the series - best practice is to put patches that fix bugs
> first, then patches that add features.
> 
> I think you'll also need to apply these changes to the bitbake
> repository rather than the poky repository to remove the unnecessary
> 'bitbake/' prefix on the file paths.
> 
> Best regards,
> 
> -- 
> Paul Barker

Thanks for the feedback, I'm going to that.
Best regards,
Osama