[PATCH v4 0/5] devtool: fix standalone clone conversion for nested git repos

[PATCH v4 0/5] devtool: fix standalone clone conversion for nested git repos

[Jamin Lin]

When a recipe has multiple git SRC_URI entries with destsuffix values
nested inside S, devtool modify left all but the top-level repo with a
.git/objects/info/alternates file pointing into the downloads cache.

Running 'bitbake -c cleanall' then removed those shared objects, breaking
all subsequent git operations in the workspace.
Fix git_convert_standalone_clone() to walk all git repos nested inside S
and repack each one to a fully standalone clone.

v1:
  - Fix git_convert_standalone_clone() to walk all git repos nested
    inside S and repack each one to a standalone clone
v2:
  - Add selftest recipe devtool-test-multi-destsuffix with six nested
    git SRC_URI entries to reproduce the scenario
  - Add test_devtool_modify_multi_git_destsuffix_standalone to verify
    all nested repos have their alternates removed after devtool modify
    and remain usable after 'bitbake -c cleanall'
v3:
  - Read SRC_URI directly instead of walking the tree for '.git' dirs -
    also fixes repos nested inside another repo's own working tree
  - Fix setup_git_repo() to register nested repos as submodules before
    the initial commit (was dead code), processing bottom-up to avoid
    stale submodule references
  - Fix commitIgnored() to skip committing when nothing is staged, so
    do_patch no longer fails on a submodule that's dirty only because
    of its own nested content
  - Simplify test recipe to three genuinely-nested destsuffix entries
    (level1/level1/level2/level1/level2/level3), update selftest to match
v4:
   - Fix regression from v3:
    setup_git_repo() now detects nested repos both for fresh workspace
    repos and for recipes whose source is already a git repo (fixes
    test_devtool_modify_nested_gitsm).

Jamin Lin (5):
  oe/patch: Skip commitIgnored when nothing is actually staged
  devtool: Register nested git repos before the initial commit
  devtool-source: Make nested destsuffix git repos standalone
  meta-selftest: Add devtool-test-multi-destsuffix recipe
  oeqa/selftest/devtool: Add test for multiple nested git destsuffix
    repos

 .../devtool-test-multi-destsuffix_git.bb      | 27 +++++++
 meta/classes/devtool-source.bbclass           | 16 ++++
 meta/lib/oe/patch.py                          | 12 +++
 meta/lib/oeqa/selftest/cases/devtool.py       | 64 +++++++++++++++
 scripts/lib/devtool/__init__.py               | 79 +++++++++++++------
 5 files changed, 175 insertions(+), 23 deletions(-)
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb

-- 
2.43.0

[PATCH v4 1/5] oe/patch: Skip commitIgnored when nothing is actually staged

[Jamin Lin]

patch.bbclass's patch_task_postfunc calls GitApplyTree.commitIgnored()
whenever 'git status --porcelain .' reports the source tree as dirty
after do_patch, to snapshot those changes into the devtool tracking
repo with 'git add' + 'git commit'.

A path can be reported as dirty by git status purely because it is a
submodule (or an unregistered embedded git repo) that itself has
modified or untracked content - for example a recipe with multiple git
SRC_URI entries where one destsuffix places a repo inside another
repo's own working tree. The outer repo's tracked commit hash for that
submodule hasn't changed, so 'git add' has nothing new to stage for it,
and git refuses to fold the submodule's own dirty state into a plain
commit without it being resolved first:

  $ git commit -m ... --no-verify --no-gpg-sign
  Changes not staged for commit:
    (commit or discard the untracked or modified content in submodules)
          modified:   level1 (modified content)
  no changes added to commit (use "git add" and/or "git commit -a")

'git commit' then exits non-zero with nothing to commit, and
commitIgnored() propagates that failure straight up, taking the whole
do_patch task down with it.

Fix by checking 'git diff --cached --name-only' after 'git add': if
nothing was actually staged, there is nothing meaningful to snapshot,
so skip the commit (and the note it would otherwise add) instead of
failing.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
---
 meta/lib/oe/patch.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 1d50e83ab7..944dde1e03 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -516,6 +516,18 @@ class GitApplyTree(PatchTree):
     def commitIgnored(subject, dir=None, files=None, d=None):
         if files:
             runcmd(['git', 'add'] + files, dir)
+
+        # 'git add' can leave nothing actually staged even though the caller
+        # saw a dirty status: a path can show as modified purely because it
+        # is a submodule/embedded git repository with modified or untracked
+        # content of its own (e.g. a further nested git repo from another
+        # destsuffix SRC_URI entry) - git refuses to record that via a plain
+        # 'git add'/'git commit' without resolving the submodule's own state,
+        # so the commit below would fail with "no changes added to commit".
+        # Skip the commit if there is nothing actually staged.
+        if not runcmd(['git', 'diff', '--cached', '--name-only'], dir).strip():
+            return
+
         cmd = ["git"]
         GitApplyTree.gitCommandUserOptions(cmd, d=d)
         cmd += ["commit", "-m", subject, "--no-verify", "--no-gpg-sign"]
-- 
2.43.0

Re: [PATCH v4 1/5] oe/patch: Skip commitIgnored when nothing is actually staged

[Paul Barker]

On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> patch.bbclass's patch_task_postfunc calls GitApplyTree.commitIgnored()
> whenever 'git status --porcelain .' reports the source tree as dirty
> after do_patch, to snapshot those changes into the devtool tracking
> repo with 'git add' + 'git commit'.
> 
> A path can be reported as dirty by git status purely because it is a
> submodule (or an unregistered embedded git repo) that itself has
> modified or untracked content - for example a recipe with multiple git
> SRC_URI entries where one destsuffix places a repo inside another
> repo's own working tree. The outer repo's tracked commit hash for that
> submodule hasn't changed, so 'git add' has nothing new to stage for it,
> and git refuses to fold the submodule's own dirty state into a plain
> commit without it being resolved first:
> 
>   $ git commit -m ... --no-verify --no-gpg-sign
>   Changes not staged for commit:
>     (commit or discard the untracked or modified content in submodules)
>           modified:   level1 (modified content)
>   no changes added to commit (use "git add" and/or "git commit -a")
> 
> 'git commit' then exits non-zero with nothing to commit, and
> commitIgnored() propagates that failure straight up, taking the whole
> do_patch task down with it.
> 
> Fix by checking 'git diff --cached --name-only' after 'git add': if
> nothing was actually staged, there is nothing meaningful to snapshot,
> so skip the commit (and the note it would otherwise add) instead of
> failing.
> 
> Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> ---
>  meta/lib/oe/patch.py | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
> index 1d50e83ab7..944dde1e03 100644
> --- a/meta/lib/oe/patch.py
> +++ b/meta/lib/oe/patch.py
> @@ -516,6 +516,18 @@ class GitApplyTree(PatchTree):
>      def commitIgnored(subject, dir=None, files=None, d=None):
>          if files:
>              runcmd(['git', 'add'] + files, dir)
> +
> +        # 'git add' can leave nothing actually staged even though the caller
> +        # saw a dirty status: a path can show as modified purely because it
> +        # is a submodule/embedded git repository with modified or untracked
> +        # content of its own (e.g. a further nested git repo from another
> +        # destsuffix SRC_URI entry) - git refuses to record that via a plain
> +        # 'git add'/'git commit' without resolving the submodule's own state,
> +        # so the commit below would fail with "no changes added to commit".
> +        # Skip the commit if there is nothing actually staged.
> +        if not runcmd(['git', 'diff', '--cached', '--name-only'], dir).strip():
> +            return

This is a lot of text for a simple message, was this AI generated?

We can just say "Skip the commit if there is nothing actually staged"
here.

Best regards,

-- 
Paul Barker

RE: [PATCH v4 1/5] oe/patch: Skip commitIgnored when nothing is actually staged

[Jamin Lin]

> -----Original Message-----
> From: Paul Barker <paul@pbarker.dev>
> Sent: Sunday, August 16, 2026 6:50 PM
> To: Jamin Lin <jamin_lin@aspeedtech.com>;
> openembedded-core@lists.openembedded.org; alex.kanavin@gmail.com;
> mathieu.dubois-briand@bootlin.com
> Cc: Troy Lee <troy_lee@aspeedtech.com>
> Subject: Re: [PATCH v4 1/5] oe/patch: Skip commitIgnored when nothing is
> actually staged
> 
> On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> > patch.bbclass's patch_task_postfunc calls GitApplyTree.commitIgnored()
> > whenever 'git status --porcelain .' reports the source tree as dirty
> > after do_patch, to snapshot those changes into the devtool tracking
> > repo with 'git add' + 'git commit'.
> >
> > A path can be reported as dirty by git status purely because it is a
> > submodule (or an unregistered embedded git repo) that itself has
> > modified or untracked content - for example a recipe with multiple git
> > SRC_URI entries where one destsuffix places a repo inside another
> > repo's own working tree. The outer repo's tracked commit hash for that
> > submodule hasn't changed, so 'git add' has nothing new to stage for
> > it, and git refuses to fold the submodule's own dirty state into a
> > plain commit without it being resolved first:
> >
> >   $ git commit -m ... --no-verify --no-gpg-sign
> >   Changes not staged for commit:
> >     (commit or discard the untracked or modified content in submodules)
> >           modified:   level1 (modified content)
> >   no changes added to commit (use "git add" and/or "git commit -a")
> >
> > 'git commit' then exits non-zero with nothing to commit, and
> > commitIgnored() propagates that failure straight up, taking the whole
> > do_patch task down with it.
> >
> > Fix by checking 'git diff --cached --name-only' after 'git add': if
> > nothing was actually staged, there is nothing meaningful to snapshot,
> > so skip the commit (and the note it would otherwise add) instead of
> > failing.
> >
> > Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> > ---
> >  meta/lib/oe/patch.py | 12 ++++++++++++
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py index
> > 1d50e83ab7..944dde1e03 100644
> > --- a/meta/lib/oe/patch.py
> > +++ b/meta/lib/oe/patch.py
> > @@ -516,6 +516,18 @@ class GitApplyTree(PatchTree):
> >      def commitIgnored(subject, dir=None, files=None, d=None):
> >          if files:
> >              runcmd(['git', 'add'] + files, dir)
> > +
> > +        # 'git add' can leave nothing actually staged even though the
> caller
> > +        # saw a dirty status: a path can show as modified purely because
> it
> > +        # is a submodule/embedded git repository with modified or
> untracked
> > +        # content of its own (e.g. a further nested git repo from another
> > +        # destsuffix SRC_URI entry) - git refuses to record that via a plain
> > +        # 'git add'/'git commit' without resolving the submodule's own
> state,
> > +        # so the commit below would fail with "no changes added to
> commit".
> > +        # Skip the commit if there is nothing actually staged.
> > +        if not runcmd(['git', 'diff', '--cached', '--name-only'], dir).strip():
> > +            return
> 
> This is a lot of text for a simple message, was this AI generated?
> 
Yes, will add AI-Generated: Uses Claud

> We can just say "Skip the commit if there is nothing actually staged"
> here.
> 
Will do.

> Best regards,
> 
> --
> Paul Barker

[PATCH v4 2/5] devtool: Register nested git repos before the initial commit

[Jamin Lin]

setup_git_repo() is meant to convert a git repo that a recipe unpacks
inside S (e.g. via multiple git SRC_URI entries with different
destsuffix values) into a regular git submodule, so devtool can later
tag branches on it and extract patches from it via finish/update.

That detection never actually triggered, because of the order the
function ran things in when it had to create the workspace repo itself:

  1. 'git init'
  2. 'git add -A .' + initial commit  <- commits the nested repo as a
                                         bare, unregistered gitlink
  3. checkout devbranch, tag basetag
  4. scan 'git status --porcelain' for still-untracked directories
     ("?? <dir>/") and convert any that are git repos into submodules

By the time step 4 ran, the nested repo had already been swept up by
step 2's 'git add -A .': git treats a directory containing its own .git
as an embedded repo and stages it as a gitlink pointing at its current
HEAD, without registering it as a submodule. Once that gitlink is
committed, 'git status --porcelain' reports it as e.g. " M <dir>"
(already tracked) rather than "?? <dir>/" (untracked), so step 4's
"line.endswith('/')" check could never match it and the conversion to a
real submodule silently never happened.

There is also a second entry path with the same root cause: when the
recipe's top-level source is itself fetched via git://, repodir is
already a git repo, so the 'if not .git' block above (init + initial
commit) is skipped entirely - and so was the detection that lived inside
it. In that case the nested repo instead gets committed as a bare
gitlink later, by patch_task_postfunc's 'git add' after do_patch.

Fix this by extracting the detection into a helper and calling it before
anything can commit the nested repo as a bare gitlink, in both cases:
  - freshly-created workspace repo: right after 'git init', before
    'git add -A .' and the initial commit;
  - repodir already a git repo: at function entry, before the later
    'git add' in patch_task_postfunc.
At those points the nested repo is still untracked and reported with a
trailing "/", so it is correctly picked up and registered via
'git submodule add'.

Nested repos are discovered top-down (so a repo that manages its own
submodules via .gitmodules can be skipped rather than descended into),
but registered bottom-up (deepest first): a parent's commit recording
its child's HEAD must happen after that child is fully finalized,
otherwise registering a still-deeper repo afterwards moves the child's
HEAD forward again and leaves the parent pointing at a stale revision.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
---
 scripts/lib/devtool/__init__.py | 79 +++++++++++++++++++++++----------
 1 file changed, 56 insertions(+), 23 deletions(-)

diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 58b02eb460..030ba7edd9 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -197,9 +197,59 @@ def setup_git_repo(repodir, version, devbranch, basetag='devtool-base', d=None):
     """
     import bb.process
     import oe.patch
+
+    def register_nested_git_submodules():
+        # If the recipe unpacks another git repo inside S (e.g. multiple git
+        # SRC_URI entries with destsuffix), declare it as a regular git
+        # submodule now, so we will be able to tag branches on it and extract
+        # patches when doing finish/update on the recipe. This must happen
+        # before anything else commits that nested repo as a bare,
+        # unregistered gitlink: once that happens 'git status' no longer
+        # reports it as untracked ("?? <dir>/"), so this detection can never
+        # find it. That can happen either from 'git add -A .' below (for a
+        # freshly-initialized repo) or, when repodir is already its own git
+        # repo (e.g. a recipe fetched via plain git://), from a later 'git
+        # add' done elsewhere (patch_task_postfunc, after do_patch) - so this
+        # is called both from the fresh-repo branch below and from the
+        # already-a-repo branch, before either has a chance to do that.
+        #
+        # Discover nested repos top-down (so we can still skip descending into
+        # a repo that manages its own submodules via .gitmodules), but do the
+        # actual 'git submodule add' + commit bottom-up (deepest repo first):
+        # a parent's commit recording its child's current HEAD must happen
+        # after that child is fully finalized, otherwise a deeper repo added
+        # later on gets its own registration commit, moving the child's HEAD
+        # forward again and leaving the parent's already-made commit pointing
+        # at a stale, superseded revision of it.
+        stdout, _ = bb.process.run("git status --porcelain", cwd=repodir)
+        nested_repos = []
+        for line in stdout.splitlines():
+            if line.endswith("/"):
+                new_dir = line.split()[1]
+                for root, dirs, files in os.walk(os.path.join(repodir, new_dir)):
+                    if ".git" in dirs + files:
+                        nested_repos.append(root)
+                        # Do not descend into nested git repos that have submodules themselves.
+                        if ".gitmodules" in files:
+                            logger.warning('Nested git repository with submodules %s; devtool will not recurse into it', root)
+                            dirs[:] = []
+
+        for root in reversed(nested_repos):
+            parentdir = os.path.join(root, "..")
+            (stdout, _) = bb.process.run('git remote', cwd=root)
+            remote = stdout.splitlines()[0]
+            (stdout, _) = bb.process.run('git remote get-url %s' % remote, cwd=root)
+            remote_url = stdout.splitlines()[0]
+            logger.error(os.path.relpath(parentdir, root))
+            bb.process.run('git submodule add %s %s' % (remote_url, os.path.relpath(root, parentdir)), cwd=parentdir)
+            oe.patch.GitApplyTree.commitIgnored("Add additional submodule from SRC_URI", dir=parentdir, d=d)
+
     if not os.path.exists(os.path.join(repodir, '.git')):
         bb.process.run('git init', cwd=repodir)
         bb.process.run('git config --local gc.autodetach 0', cwd=repodir)
+
+        register_nested_git_submodules()
+
         bb.process.run('git add -f -A .', cwd=repodir)
         commit_cmd = ['git']
         oe.patch.GitApplyTree.gitCommandUserOptions(commit_cmd, d=d)
@@ -214,6 +264,12 @@ def setup_git_repo(repodir, version, devbranch, basetag='devtool-base', d=None):
             commitmsg = "Initial commit from upstream"
         commit_cmd += ['-m', commitmsg]
         bb.process.run(commit_cmd, cwd=repodir)
+    else:
+        # repodir is already a git repo in its own right (e.g. a recipe whose
+        # top-level source is fetched via plain git://), so there was no
+        # fresh init/initial commit above to interfere with detecting nested
+        # repos - do it here instead, at the earliest point available.
+        register_nested_git_submodules()
 
     # Ensure singletask.lock (as used by externalsrc.bbclass) is ignored by git
     gitinfodir = os.path.join(repodir, '.git', 'info')
@@ -237,29 +293,6 @@ def setup_git_repo(repodir, version, devbranch, basetag='devtool-base', d=None):
     bb.process.run('git checkout -b %s' % devbranch, cwd=repodir)
     bb.process.run('git tag -f --no-sign %s' % basetag, cwd=repodir)
 
-    # if recipe unpacks another git repo inside S, we need to declare it as a regular git submodule now,
-    # so we will be able to tag branches on it and extract patches when doing finish/update on the recipe
-    stdout, _ = bb.process.run("git status --porcelain", cwd=repodir)
-    found = False
-    for line in stdout.splitlines():
-        if line.endswith("/"):
-            new_dir = line.split()[1]
-            for root, dirs, files in os.walk(os.path.join(repodir, new_dir)):
-                if ".git" in dirs + files:
-                    (stdout, _) = bb.process.run('git remote', cwd=root)
-                    remote = stdout.splitlines()[0]
-                    (stdout, _) = bb.process.run('git remote get-url %s' % remote, cwd=root)
-                    remote_url = stdout.splitlines()[0]
-                    logger.error(os.path.relpath(os.path.join(root, ".."), root))
-                    bb.process.run('git submodule add %s %s' % (remote_url, os.path.relpath(root, os.path.join(root, ".."))), cwd=os.path.join(root, ".."))
-                    # Do not descend into nested git repos that have submodules themselves.
-                    if ".gitmodules" in files:
-                        logger.warning('Nested git repository with submodules %s; devtool will not recurse into it', root)
-                        dirs[:] = []
-                    found = True
-                if found:
-                    oe.patch.GitApplyTree.commitIgnored("Add additional submodule from SRC_URI", dir=os.path.join(root, ".."), d=d)
-                    found = False
     if os.path.exists(os.path.join(repodir, '.gitmodules')):
         bb.process.run('git submodule foreach --recursive  "git tag -f --no-sign %s"' % basetag, cwd=repodir)
 
-- 
2.43.0

Re: [PATCH v4 2/5] devtool: Register nested git repos before the initial commit

[Paul Barker]

On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> setup_git_repo() is meant to convert a git repo that a recipe unpacks
> inside S (e.g. via multiple git SRC_URI entries with different
> destsuffix values) into a regular git submodule, so devtool can later
> tag branches on it and extract patches from it via finish/update.
> 
> That detection never actually triggered, because of the order the
> function ran things in when it had to create the workspace repo itself:
> 
>   1. 'git init'
>   2. 'git add -A .' + initial commit  <- commits the nested repo as a
>                                          bare, unregistered gitlink
>   3. checkout devbranch, tag basetag
>   4. scan 'git status --porcelain' for still-untracked directories
>      ("?? <dir>/") and convert any that are git repos into submodules
> 
> By the time step 4 ran, the nested repo had already been swept up by
> step 2's 'git add -A .': git treats a directory containing its own .git
> as an embedded repo and stages it as a gitlink pointing at its current
> HEAD, without registering it as a submodule. Once that gitlink is
> committed, 'git status --porcelain' reports it as e.g. " M <dir>"
> (already tracked) rather than "?? <dir>/" (untracked), so step 4's
> "line.endswith('/')" check could never match it and the conversion to a
> real submodule silently never happened.
> 
> There is also a second entry path with the same root cause: when the
> recipe's top-level source is itself fetched via git://, repodir is
> already a git repo, so the 'if not .git' block above (init + initial
> commit) is skipped entirely - and so was the detection that lived inside
> it. In that case the nested repo instead gets committed as a bare
> gitlink later, by patch_task_postfunc's 'git add' after do_patch.
> 
> Fix this by extracting the detection into a helper and calling it before
> anything can commit the nested repo as a bare gitlink, in both cases:
>   - freshly-created workspace repo: right after 'git init', before
>     'git add -A .' and the initial commit;
>   - repodir already a git repo: at function entry, before the later
>     'git add' in patch_task_postfunc.
> At those points the nested repo is still untracked and reported with a
> trailing "/", so it is correctly picked up and registered via
> 'git submodule add'.
> 
> Nested repos are discovered top-down (so a repo that manages its own
> submodules via .gitmodules can be skipped rather than descended into),
> but registered bottom-up (deepest first): a parent's commit recording
> its child's HEAD must happen after that child is fully finalized,
> otherwise registering a still-deeper repo afterwards moves the child's
> HEAD forward again and leaves the parent pointing at a stale revision.
> 
> Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>

This commit is doing three things:
- Refactoring existing code into register_nested_git_submodules()
- Modifying the refactored code
- Moving the call site earlier

It's very hard to review this commit and be confident that it is
correct. This should be split into a two or three logical steps with
simpler commit messages.

The wall-of-text commit message and comment suggests to me this may be
generated with AI. Have you read the contributor guide [1]?

[1]: https://docs.yoctoproject.org/contributor-guide/submit-changes.html#acceptance-of-ai-generated-code

Best regards,

-- 
Paul Barker

RE: [PATCH v4 2/5] devtool: Register nested git repos before the initial commit

[Jamin Lin]

> Subject: Re: [PATCH v4 2/5] devtool: Register nested git repos before the initial
> commit
> 
> On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> > setup_git_repo() is meant to convert a git repo that a recipe unpacks
> > inside S (e.g. via multiple git SRC_URI entries with different
> > destsuffix values) into a regular git submodule, so devtool can later
> > tag branches on it and extract patches from it via finish/update.
> >
> > That detection never actually triggered, because of the order the
> > function ran things in when it had to create the workspace repo itself:
> >
> >   1. 'git init'
> >   2. 'git add -A .' + initial commit  <- commits the nested repo as a
> >                                          bare, unregistered gitlink
> >   3. checkout devbranch, tag basetag
> >   4. scan 'git status --porcelain' for still-untracked directories
> >      ("?? <dir>/") and convert any that are git repos into submodules
> >
> > By the time step 4 ran, the nested repo had already been swept up by
> > step 2's 'git add -A .': git treats a directory containing its own
> > .git as an embedded repo and stages it as a gitlink pointing at its
> > current HEAD, without registering it as a submodule. Once that gitlink
> > is committed, 'git status --porcelain' reports it as e.g. " M <dir>"
> > (already tracked) rather than "?? <dir>/" (untracked), so step 4's
> > "line.endswith('/')" check could never match it and the conversion to
> > a real submodule silently never happened.
> >
> > There is also a second entry path with the same root cause: when the
> > recipe's top-level source is itself fetched via git://, repodir is
> > already a git repo, so the 'if not .git' block above (init + initial
> > commit) is skipped entirely - and so was the detection that lived
> > inside it. In that case the nested repo instead gets committed as a
> > bare gitlink later, by patch_task_postfunc's 'git add' after do_patch.
> >
> > Fix this by extracting the detection into a helper and calling it
> > before anything can commit the nested repo as a bare gitlink, in both cases:
> >   - freshly-created workspace repo: right after 'git init', before
> >     'git add -A .' and the initial commit;
> >   - repodir already a git repo: at function entry, before the later
> >     'git add' in patch_task_postfunc.
> > At those points the nested repo is still untracked and reported with a
> > trailing "/", so it is correctly picked up and registered via 'git
> > submodule add'.
> >
> > Nested repos are discovered top-down (so a repo that manages its own
> > submodules via .gitmodules can be skipped rather than descended into),
> > but registered bottom-up (deepest first): a parent's commit recording
> > its child's HEAD must happen after that child is fully finalized,
> > otherwise registering a still-deeper repo afterwards moves the child's
> > HEAD forward again and leaves the parent pointing at a stale revision.
> >
> > Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> 
> This commit is doing three things:
> - Refactoring existing code into register_nested_git_submodules()
> - Modifying the refactored code
> - Moving the call site earlier
> 
> It's very hard to review this commit and be confident that it is correct. This
> should be split into a two or three logical steps with simpler commit messages.
> 
> The wall-of-text commit message and comment suggests to me this may be
> generated with AI. Have you read the contributor guide [1]?
> 
After I read this contribution guide, will add 
AI-Generated: Uses Claud in commit message.

> [1]:
> https://docs.yoctoproject.org/contributor-guide/submit-changes.html#acceptan
> ce-of-ai-generated-code
> 
> Best regards,
> 
> --
> Paul Barker

[PATCH v4 3/5] devtool-source: Make nested destsuffix git repos standalone

[Jamin Lin]

When a recipe uses multiple git SRC_URI entries with different destsuffix
values (e.g. recipes with separate repositories for the kernel, modules
and application), do_unpack clones each source tree with
'git clone -n -s'.

The -s flag uses git's shared-object mechanism:
instead of copying objects locally it writes a .git/objects/info/alternates file
pointing back to the bare repository under the downloads directory (DL_DIR/git2/).

scriptutils.git_convert_standalone_clone() is called by devtool_post_unpack to
make the top-level source directory standalone: it runs 'git repack -a' to copy
all objects into the local object store and then removes the alternates file.

However it only processes the top-level source directory. Each nested git repo
created by a separate SRC_URI entry retains its own alternates file still
pointing into downloads/.

Steps to reproduce:
  1. devtool modify <recipe-with-multiple-git-SRC_URI>
  2. bitbake -c cleanall <recipe>
  3. bitbake <recipe>

At step 2, 'bitbake -c cleanall' calls fetcher.clean() which deletes
the bare repositories from downloads/git2/. The top-level workspace
repo is standalone (alternates already removed by the original code),
but the nested repos still hold alternates pointing to the now-deleted
paths.

At step 3, srctree_hash_files() runs 'git add -A .' with a custom
GIT_INDEX_FILE. Git internally calls 'git status --porcelain=2' on
each nested repo to check for changes; this fails with exit 128 because
the nested alternates are broken:
  error: unable to normalize alternate object path:
         .../downloads/git2/github.com.example.module//objects
  fatal: bad object HEAD
  fatal: 'git status --porcelain=2' failed in submodule modules/lib/module

This halts the BitBake parse phase with a CalledProcessError and leaves
the workspace in an unrecoverable state without manual intervention.

Fix by having devtool_post_unpack() look at the recipe's SRC_URI directly:
any git entry with an explicit destsuffix param names an additional
checkout nested under the source tree, so convert each of those to a
standalone clone the same way as the top-level tree.

This is deliberately metadata-driven rather than walking the unpacked
source tree looking for '.git' directories: a directory walk has no way
to tell a nested checkout from an ordinary subdirectory of one, so it
either has to stop at the first git repo it finds - which then misses a
destsuffix repo nested inside another repo's own working tree - or keep
walking into every repo's contents, which is wasted work for large trees.
Reading SRC_URI instead gives the exact, authoritative set of paths that
need converting, regardless of how they happen to be nested on disk.

Only entries with an explicit destsuffix are handled, since that is the
only way a recipe ends up with more than one git checkout under S; this
also avoids having to duplicate the git fetcher's internal logic for
computing an implicit default destsuffix (which depends on the subdir/
subpath params and BB_GIT_DEFAULT_DESTSUFFIX).

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
---
 meta/classes/devtool-source.bbclass | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index f29f40588f..940cdedbac 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -97,6 +97,22 @@ python devtool_post_unpack() {
 
     scriptutils.git_convert_standalone_clone(srcsubdir)
 
+    # Recipes can use multiple git SRC_URI entries with an explicit destsuffix to
+    # unpack several repositories as nested subdirectories of the source tree
+    # (e.g. recipes with separate repos for the kernel, modules and
+    # application). Each such entry is unpacked as its own 'git clone -s' and
+    # needs the same standalone conversion as srcsubdir above, otherwise it keeps
+    # referencing objects in the downloads dir that 'bitbake -c cleanall' removes.
+    # We only look at entries with an explicit destsuffix param, since that's the
+    # only way a recipe ends up with more than one git checkout under S - this
+    # avoids having to duplicate the fetcher's internal default-destsuffix logic.
+    import bb.fetch2
+    fetch = bb.fetch2.Fetch(d.getVar('SRC_URI').split(), d)
+    for url in fetch.urls:
+        ud = fetch.ud[url]
+        if ud.type == 'git' and ud.parm.get('destsuffix'):
+            scriptutils.git_convert_standalone_clone(os.path.join(unpackdir, ud.parm['destsuffix']))
+
     # Make sure that srcsubdir exists
     bb.utils.mkdirhier(srcsubdir)
     if not os.listdir(srcsubdir):
-- 
2.43.0

Re: [PATCH v4 3/5] devtool-source: Make nested destsuffix git repos standalone

[Paul Barker]

On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> When a recipe uses multiple git SRC_URI entries with different destsuffix
> values (e.g. recipes with separate repositories for the kernel, modules
> and application), do_unpack clones each source tree with
> 'git clone -n -s'.
> 
> The -s flag uses git's shared-object mechanism:
> instead of copying objects locally it writes a .git/objects/info/alternates file
> pointing back to the bare repository under the downloads directory (DL_DIR/git2/).
> 
> scriptutils.git_convert_standalone_clone() is called by devtool_post_unpack to
> make the top-level source directory standalone: it runs 'git repack -a' to copy
> all objects into the local object store and then removes the alternates file.
> 
> However it only processes the top-level source directory. Each nested git repo
> created by a separate SRC_URI entry retains its own alternates file still
> pointing into downloads/.
> 
> Steps to reproduce:
>   1. devtool modify <recipe-with-multiple-git-SRC_URI>
>   2. bitbake -c cleanall <recipe>
>   3. bitbake <recipe>
> 
> At step 2, 'bitbake -c cleanall' calls fetcher.clean() which deletes
> the bare repositories from downloads/git2/. The top-level workspace
> repo is standalone (alternates already removed by the original code),
> but the nested repos still hold alternates pointing to the now-deleted
> paths.
> 
> At step 3, srctree_hash_files() runs 'git add -A .' with a custom
> GIT_INDEX_FILE. Git internally calls 'git status --porcelain=2' on
> each nested repo to check for changes; this fails with exit 128 because
> the nested alternates are broken:
>   error: unable to normalize alternate object path:
>          .../downloads/git2/github.com.example.module//objects
>   fatal: bad object HEAD
>   fatal: 'git status --porcelain=2' failed in submodule modules/lib/module
> 
> This halts the BitBake parse phase with a CalledProcessError and leaves
> the workspace in an unrecoverable state without manual intervention.
> 
> Fix by having devtool_post_unpack() look at the recipe's SRC_URI directly:
> any git entry with an explicit destsuffix param names an additional
> checkout nested under the source tree, so convert each of those to a
> standalone clone the same way as the top-level tree.
> 
> This is deliberately metadata-driven rather than walking the unpacked
> source tree looking for '.git' directories: a directory walk has no way
> to tell a nested checkout from an ordinary subdirectory of one, so it
> either has to stop at the first git repo it finds - which then misses a
> destsuffix repo nested inside another repo's own working tree - or keep
> walking into every repo's contents, which is wasted work for large trees.
> Reading SRC_URI instead gives the exact, authoritative set of paths that
> need converting, regardless of how they happen to be nested on disk.
> 
> Only entries with an explicit destsuffix are handled, since that is the
> only way a recipe ends up with more than one git checkout under S; this
> also avoids having to duplicate the git fetcher's internal logic for
> computing an implicit default destsuffix (which depends on the subdir/
> subpath params and BB_GIT_DEFAULT_DESTSUFFIX).
> 
> Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> ---
>  meta/classes/devtool-source.bbclass | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
> index f29f40588f..940cdedbac 100644
> --- a/meta/classes/devtool-source.bbclass
> +++ b/meta/classes/devtool-source.bbclass
> @@ -97,6 +97,22 @@ python devtool_post_unpack() {
>  
>      scriptutils.git_convert_standalone_clone(srcsubdir)
>  
> +    # Recipes can use multiple git SRC_URI entries with an explicit destsuffix to
> +    # unpack several repositories as nested subdirectories of the source tree
> +    # (e.g. recipes with separate repos for the kernel, modules and
> +    # application). Each such entry is unpacked as its own 'git clone -s' and
> +    # needs the same standalone conversion as srcsubdir above, otherwise it keeps
> +    # referencing objects in the downloads dir that 'bitbake -c cleanall' removes.
> +    # We only look at entries with an explicit destsuffix param, since that's the
> +    # only way a recipe ends up with more than one git checkout under S - this
> +    # avoids having to duplicate the fetcher's internal default-destsuffix logic.
> +    import bb.fetch2
> +    fetch = bb.fetch2.Fetch(d.getVar('SRC_URI').split(), d)
> +    for url in fetch.urls:
> +        ud = fetch.ud[url]
> +        if ud.type == 'git' and ud.parm.get('destsuffix'):
> +            scriptutils.git_convert_standalone_clone(os.path.join(unpackdir, ud.parm['destsuffix']))
> +
>      # Make sure that srcsubdir exists
>      bb.utils.mkdirhier(srcsubdir)
>      if not os.listdir(srcsubdir):

Why still handle srcsubdir separately above this loop?

What about SRC_URI containing multiple git repositories that unpack
side-by-side instead of nested?

It may be better to just iterate through all git repositories in SRC_URI
rather than just the nested ones.

Best regards,

-- 
Paul Barker

[PATCH v4 4/5] meta-selftest: Add devtool-test-multi-destsuffix recipe

[Jamin Lin]

Add a test recipe with three git SRC_URI entries using nested destsuffix
values, where each repo's checkout lives inside the working tree of the
previous one (level1, then level1/level2, then level1/level2/level3).
This recipe is used by the devtool selftest to verify that devtool modify
correctly converts all nested git repos to standalone clones, including
the case where a repo is nested inside another repo's own working tree
rather than merely under a shared plain directory.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
---
 .../devtool-test-multi-destsuffix_git.bb      | 27 +++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb

diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb b/meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb
new file mode 100644
index 0000000000..880f0f0bb4
--- /dev/null
+++ b/meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb
@@ -0,0 +1,27 @@
+SUMMARY = "Test recipe for multiple git SRC_URI entries with nested destsuffix values"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
+
+# Three git entries genuinely nested inside each other's working tree (not just
+# nested by path under a shared plain directory): level2's destsuffix places it
+# inside level1's checkout, and level3's destsuffix places it inside level2's.
+# This exercises the devtool code path that must convert every nested git repo
+# to a standalone clone even when one repo's checkout lives inside another
+# repo's working tree: the initial fetch uses a shared clone whose alternates
+# point into downloads/git2/; git repack copies those objects locally so the
+# workspace survives 'bitbake -c cleanall'.
+SRC_URI = "git://git.yoctoproject.org/dbus-wait;nobranch=1;protocol=https;name=level1;destsuffix=level1 \
+    git://git.yoctoproject.org/dbus-wait;nobranch=1;protocol=https;name=level2;destsuffix=level1/level2 \
+    git://git.yoctoproject.org/dbus-wait;nobranch=1;protocol=https;name=level3;destsuffix=level1/level2/level3 \
+"
+
+SRCREV_level1 = "64bc7c8fae61ded0c4e555aa775911f84c56e438"
+SRCREV_level2 = "64bc7c8fae61ded0c4e555aa775911f84c56e438"
+SRCREV_level3 = "64bc7c8fae61ded0c4e555aa775911f84c56e438"
+SRCREV_FORMAT = "level1_level2_level3"
+
+S = "${UNPACKDIR}"
+
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+do_install[noexec] = "1"
-- 
2.43.0

[PATCH v4 5/5] oeqa/selftest/devtool: Add test for multiple nested git destsuffix repos

[Jamin Lin]

Add test_devtool_modify_multi_git_destsuffix_standalone to verify that
devtool modify converts all nested git repos (from multiple SRC_URI git
entries with different destsuffix values, including a repo nested inside
another repo's own working tree) to standalone clones so the workspace
survives 'bitbake -c cleanall'.

Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
---
 meta/lib/oeqa/selftest/cases/devtool.py | 64 +++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index a10eb0c784..de73a2e620 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -1265,6 +1265,70 @@ class DevtoolModifyTests(DevtoolBase):
         self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake'), 'Submodule not found')
         self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake-gitsm-test1'), 'Submodule not found')
 
+    def test_devtool_modify_multi_git_destsuffix_standalone(self):
+        """
+        Verify that devtool modify converts all nested git repos (from multiple
+        SRC_URI git entries with different destsuffix values) to standalone clones
+        so that 'bitbake -c cleanall' does not break the devtool workspace.
+
+        The recipe (devtool-test-multi-destsuffix) has three git SRC_URI entries
+        with S = ${UNPACKDIR}, each nested inside the previous repo's own
+        working tree:
+            destsuffix=level1               -> srcdir/level1/
+            destsuffix=level1/level2        -> srcdir/level1/level2/
+            destsuffix=level1/level2/level3 -> srcdir/level1/level2/level3/
+
+        This mirrors real-world recipes that embed multiple module repos
+        as nested subdirectories of the primary source tree, including the
+        case where one repo's checkout lives inside another repo's working
+        tree rather than merely under a shared plain directory.
+        """
+        testrecipe = 'devtool-test-multi-destsuffix'
+        src_uri = get_bb_var('SRC_URI', testrecipe)
+        self.assertIn('git://', src_uri,
+                      'This test expects %s to have git SRC_URI entries' % testrecipe)
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('devtool reset %s' % testrecipe)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        result = runCmd('devtool modify %s' % testrecipe)
+        self.assertEqual(result.status, 0,
+                         'devtool modify failed: %s' % result.output)
+        srcdir = os.path.join(self.workspacedir, 'sources', testrecipe)
+        nested_paths = [
+            ('level1', 'level1'),
+            ('level2', 'level1/level2'),
+            ('level3', 'level1/level2/level3'),
+        ]
+
+        for name, subpath in nested_paths:
+            repo_path = os.path.join(srcdir, subpath)
+            self.assertExists(os.path.join(repo_path, '.git'),
+                              'Repo %s (.git) not found in devtool workspace' % name)
+
+        # Key assertion: no nested repo should retain a git alternates file.
+        # devtool modify must repack objects locally so the workspace does not
+        # depend on the downloads cache, which 'bitbake -c cleanall' will delete.
+        for name, subpath in nested_paths:
+            repo_path = os.path.join(srcdir, subpath)
+            alternates_file = os.path.join(repo_path, '.git', 'objects',
+                                           'info', 'alternates')
+            self.assertNotExists(alternates_file,
+                                 'Repo %s still has a git alternates file after '
+                                 'devtool modify' % name)
+
+        # Verify the workspace survives cleanall, which removes the shared
+        # objects in the downloads cache that alternates would reference.
+        bitbake('%s -c cleanall' % testrecipe)
+
+        # After cleanall all repos must still be usable.
+        # A broken alternates file would cause git operations to fail.
+        for name, subpath in nested_paths:
+            repo_path = os.path.join(srcdir, subpath)
+            result = runCmd('git status', cwd=repo_path)
+            self.assertEqual(result.status, 0,
+                             'git status failed in repo %s after cleanall: %s'
+                             % (name, result.output))
+
 class DevtoolUpdateTests(DevtoolBase):
 
     def test_devtool_update_recipe(self):
-- 
2.43.0

Re: [PATCH v4 5/5] oeqa/selftest/devtool: Add test for multiple nested git destsuffix repos

[Paul Barker]

On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> Add test_devtool_modify_multi_git_destsuffix_standalone to verify that
> devtool modify converts all nested git repos (from multiple SRC_URI git
> entries with different destsuffix values, including a repo nested inside
> another repo's own working tree) to standalone clones so the workspace
> survives 'bitbake -c cleanall'.
> 
> Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> ---
>  meta/lib/oeqa/selftest/cases/devtool.py | 64 +++++++++++++++++++++++++
>  1 file changed, 64 insertions(+)
> 
> diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
> index a10eb0c784..de73a2e620 100644
> --- a/meta/lib/oeqa/selftest/cases/devtool.py
> +++ b/meta/lib/oeqa/selftest/cases/devtool.py
> @@ -1265,6 +1265,70 @@ class DevtoolModifyTests(DevtoolBase):
>          self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake'), 'Submodule not found')
>          self.assertExists(os.path.join(source_repo_gitsm_gitmodules, 'bitbake-gitsm-test1'), 'Submodule not found')
>  
> +    def test_devtool_modify_multi_git_destsuffix_standalone(self):
> +        """
> +        Verify that devtool modify converts all nested git repos (from multiple
> +        SRC_URI git entries with different destsuffix values) to standalone clones
> +        so that 'bitbake -c cleanall' does not break the devtool workspace.
> +
> +        The recipe (devtool-test-multi-destsuffix) has three git SRC_URI entries
> +        with S = ${UNPACKDIR}, each nested inside the previous repo's own
> +        working tree:
> +            destsuffix=level1               -> srcdir/level1/
> +            destsuffix=level1/level2        -> srcdir/level1/level2/
> +            destsuffix=level1/level2/level3 -> srcdir/level1/level2/level3/
> +
> +        This mirrors real-world recipes that embed multiple module repos
> +        as nested subdirectories of the primary source tree, including the
> +        case where one repo's checkout lives inside another repo's working
> +        tree rather than merely under a shared plain directory.
> +        """
> +        testrecipe = 'devtool-test-multi-destsuffix'
> +        src_uri = get_bb_var('SRC_URI', testrecipe)
> +        self.assertIn('git://', src_uri,
> +                      'This test expects %s to have git SRC_URI entries' % testrecipe)
> +        self.track_for_cleanup(self.workspacedir)
> +        self.add_command_to_tearDown('devtool reset %s' % testrecipe)
> +        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
> +        result = runCmd('devtool modify %s' % testrecipe)
> +        self.assertEqual(result.status, 0,
> +                         'devtool modify failed: %s' % result.output)
> +        srcdir = os.path.join(self.workspacedir, 'sources', testrecipe)
> +        nested_paths = [
> +            ('level1', 'level1'),
> +            ('level2', 'level1/level2'),
> +            ('level3', 'level1/level2/level3'),
> +        ]
> +
> +        for name, subpath in nested_paths:
> +            repo_path = os.path.join(srcdir, subpath)
> +            self.assertExists(os.path.join(repo_path, '.git'),
> +                              'Repo %s (.git) not found in devtool workspace' % name)
> +
> +        # Key assertion: no nested repo should retain a git alternates file.
> +        # devtool modify must repack objects locally so the workspace does not
> +        # depend on the downloads cache, which 'bitbake -c cleanall' will delete.
> +        for name, subpath in nested_paths:
> +            repo_path = os.path.join(srcdir, subpath)
> +            alternates_file = os.path.join(repo_path, '.git', 'objects',
> +                                           'info', 'alternates')
> +            self.assertNotExists(alternates_file,
> +                                 'Repo %s still has a git alternates file after '
> +                                 'devtool modify' % name)
> +
> +        # Verify the workspace survives cleanall, which removes the shared
> +        # objects in the downloads cache that alternates would reference.
> +        bitbake('%s -c cleanall' % testrecipe)

We can't use cleanall here as it deletes data from the main downloads
directory. Is the above check for an alternates file not sufficient? If
not then this test needs to use an isolated downloads directory.

Best regards,

-- 
Paul Barker

RE: [PATCH v4 5/5] oeqa/selftest/devtool: Add test for multiple nested git destsuffix repos

[Jamin Lin]

> On Fri, 2026-07-31 at 09:26 +0000, Jamin Lin wrote:
> > Add test_devtool_modify_multi_git_destsuffix_standalone to verify that
> > devtool modify converts all nested git repos (from multiple SRC_URI
> > git entries with different destsuffix values, including a repo nested
> > inside another repo's own working tree) to standalone clones so the
> > workspace survives 'bitbake -c cleanall'.
> >
> > Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> > ---
> >  meta/lib/oeqa/selftest/cases/devtool.py | 64
> > +++++++++++++++++++++++++
> >  1 file changed, 64 insertions(+)
> >
> > diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
> > b/meta/lib/oeqa/selftest/cases/devtool.py
> > index a10eb0c784..de73a2e620 100644
> > --- a/meta/lib/oeqa/selftest/cases/devtool.py
> > +++ b/meta/lib/oeqa/selftest/cases/devtool.py
> > @@ -1265,6 +1265,70 @@ class DevtoolModifyTests(DevtoolBase):
> >          self.assertExists(os.path.join(source_repo_gitsm_gitmodules,
> 'bitbake'), 'Submodule not found')
> >          self.assertExists(os.path.join(source_repo_gitsm_gitmodules,
> > 'bitbake-gitsm-test1'), 'Submodule not found')
> >
> > +    def test_devtool_modify_multi_git_destsuffix_standalone(self):
> > +        """
> > +        Verify that devtool modify converts all nested git repos (from
> multiple
> > +        SRC_URI git entries with different destsuffix values) to standalone
> clones
> > +        so that 'bitbake -c cleanall' does not break the devtool workspace.
> > +
> > +        The recipe (devtool-test-multi-destsuffix) has three git SRC_URI
> entries
> > +        with S = ${UNPACKDIR}, each nested inside the previous repo's
> own
> > +        working tree:
> > +            destsuffix=level1               -> srcdir/level1/
> > +            destsuffix=level1/level2        -> srcdir/level1/level2/
> > +            destsuffix=level1/level2/level3 ->
> > + srcdir/level1/level2/level3/
> > +
> > +        This mirrors real-world recipes that embed multiple module repos
> > +        as nested subdirectories of the primary source tree, including the
> > +        case where one repo's checkout lives inside another repo's
> working
> > +        tree rather than merely under a shared plain directory.
> > +        """
> > +        testrecipe = 'devtool-test-multi-destsuffix'
> > +        src_uri = get_bb_var('SRC_URI', testrecipe)
> > +        self.assertIn('git://', src_uri,
> > +                      'This test expects %s to have git SRC_URI entries'
> % testrecipe)
> > +        self.track_for_cleanup(self.workspacedir)
> > +        self.add_command_to_tearDown('devtool reset %s' % testrecipe)
> > +        self.add_command_to_tearDown('bitbake-layers remove-layer
> */workspace')
> > +        result = runCmd('devtool modify %s' % testrecipe)
> > +        self.assertEqual(result.status, 0,
> > +                         'devtool modify failed: %s' % result.output)
> > +        srcdir = os.path.join(self.workspacedir, 'sources', testrecipe)
> > +        nested_paths = [
> > +            ('level1', 'level1'),
> > +            ('level2', 'level1/level2'),
> > +            ('level3', 'level1/level2/level3'),
> > +        ]
> > +
> > +        for name, subpath in nested_paths:
> > +            repo_path = os.path.join(srcdir, subpath)
> > +            self.assertExists(os.path.join(repo_path, '.git'),
> > +                              'Repo %s (.git) not found in devtool
> > + workspace' % name)
> > +
> > +        # Key assertion: no nested repo should retain a git alternates file.
> > +        # devtool modify must repack objects locally so the workspace
> does not
> > +        # depend on the downloads cache, which 'bitbake -c cleanall' will
> delete.
> > +        for name, subpath in nested_paths:
> > +            repo_path = os.path.join(srcdir, subpath)
> > +            alternates_file = os.path.join(repo_path, '.git', 'objects',
> > +                                           'info', 'alternates')
> > +            self.assertNotExists(alternates_file,
> > +                                 'Repo %s still has a git alternates
> file after '
> > +                                 'devtool modify' % name)
> > +
> > +        # Verify the workspace survives cleanall, which removes the
> shared
> > +        # objects in the downloads cache that alternates would reference.
> > +        bitbake('%s -c cleanall' % testrecipe)
> 
Will remove cleanall test.

> We can't use cleanall here as it deletes data from the main downloads
> directory. Is the above check for an alternates file not sufficient? If not then
> this test needs to use an isolated downloads directory.
> 
> Best regards,
> 
> --
> Paul Barker

RE: [PATCH v4 0/5] devtool: fix standalone clone conversion for nested git repos

[Jamin Lin]

All,

Thanks all for the review and the time you spent on this.

I originally ran into this problem myself and used AI to help put
together the fix. However, the design questions raised here have gone
beyond my own understanding of this code and its original intent, and I
don't feel I can properly stand behind the patches or address the review
comments with confidence. 

So I'm going to drop this series rather than submit something I can't fully explain.

For anyone who hits the same issue: as a workaround, use
'bitbake -c cleansstate <recipe>' instead of 'bitbake -c cleanall'.
cleansstate does not delete the bare repositories under DL_DIR/git2/,
so the nested repos' alternates stay valid and the devtool workspace
keeps working.

Thanks again for the review.

Jamin

> -----Original Message-----
> From: Jamin Lin
> Sent: Friday, July 31, 2026 5:27 PM
> To: openembedded-core@lists.openembedded.org; alex.kanavin@gmail.com;
> paul@pbarker.dev; mathieu.dubois-briand@bootlin.com
> Cc: Troy Lee <troy_lee@aspeedtech.com>; Jamin Lin
> <jamin_lin@aspeedtech.com>
> Subject: [PATCH v4 0/5] devtool: fix standalone clone conversion for nested git
> repos
> 
> When a recipe has multiple git SRC_URI entries with destsuffix values nested
> inside S, devtool modify left all but the top-level repo with
> a .git/objects/info/alternates file pointing into the downloads cache.
> 
> Running 'bitbake -c cleanall' then removed those shared objects, breaking all
> subsequent git operations in the workspace.
> Fix git_convert_standalone_clone() to walk all git repos nested inside S and
> repack each one to a fully standalone clone.
> 
> v1:
>   - Fix git_convert_standalone_clone() to walk all git repos nested
>     inside S and repack each one to a standalone clone
> v2:
>   - Add selftest recipe devtool-test-multi-destsuffix with six nested
>     git SRC_URI entries to reproduce the scenario
>   - Add test_devtool_modify_multi_git_destsuffix_standalone to verify
>     all nested repos have their alternates removed after devtool modify
>     and remain usable after 'bitbake -c cleanall'
> v3:
>   - Read SRC_URI directly instead of walking the tree for '.git' dirs -
>     also fixes repos nested inside another repo's own working tree
>   - Fix setup_git_repo() to register nested repos as submodules before
>     the initial commit (was dead code), processing bottom-up to avoid
>     stale submodule references
>   - Fix commitIgnored() to skip committing when nothing is staged, so
>     do_patch no longer fails on a submodule that's dirty only because
>     of its own nested content
>   - Simplify test recipe to three genuinely-nested destsuffix entries
>     (level1/level1/level2/level1/level2/level3), update selftest to match
> v4:
>    - Fix regression from v3:
>     setup_git_repo() now detects nested repos both for fresh workspace
>     repos and for recipes whose source is already a git repo (fixes
>     test_devtool_modify_nested_gitsm).
> 
> Jamin Lin (5):
>   oe/patch: Skip commitIgnored when nothing is actually staged
>   devtool: Register nested git repos before the initial commit
>   devtool-source: Make nested destsuffix git repos standalone
>   meta-selftest: Add devtool-test-multi-destsuffix recipe
>   oeqa/selftest/devtool: Add test for multiple nested git destsuffix
>     repos
> 
>  .../devtool-test-multi-destsuffix_git.bb      | 27 +++++++
>  meta/classes/devtool-source.bbclass           | 16 ++++
>  meta/lib/oe/patch.py                          | 12 +++
>  meta/lib/oeqa/selftest/cases/devtool.py       | 64 +++++++++++++++
>  scripts/lib/devtool/__init__.py               | 79 +++++++++++++------
>  5 files changed, 175 insertions(+), 23 deletions(-)  create mode 100644
> meta-selftest/recipes-test/devtool/devtool-test-multi-destsuffix_git.bb
> 
> --
> 2.43.0