Re: [OE-core] [PATCH v3 1/5] devtool: Detect nested git repos before the initial workspace commit

["Mathieu Dubois-Briand" <mathieu.dubois-briand@bootlin.com>]

On Thu Jul 23, 2026 at 10:11 AM CEST, Jamin Lin via lists.openembedded.org wrote:
> setup_git_repo() is meant to convert a git repo that a recipe unpacks
> inside S (e.g. via multiple git SRC_URI entries with different
> destsuffix values) into a regular git submodule, so devtool can later
> tag branches on it and extract patches from it via finish/update.
>
> That detection never actually triggers, because of the order the
> function runs things in:
>
>   1. 'git init'
>   2. 'git add -A .' + initial commit  <- commits the nested repo as a
>                                          bare, unregistered gitlink
>   3. checkout devbranch, tag basetag
>   4. scan 'git status --porcelain' for still-untracked directories
>      ("?? <dir>/") and convert any that are git repos into submodules
>
> By the time step 4 runs, the nested repo was already swept up by step
> 2's 'git add -A .': git treats a directory containing its own .git as
> an embedded repo and stages it as a gitlink pointing at its current
> HEAD, without registering it as a submodule. Once that gitlink is
> committed, 'git status --porcelain' reports it as e.g. " M leveldir"
> (already tracked) rather than "?? leveldir/" (untracked), so step 4's
> "line.endswith('/')" check can never match it, and the conversion to a
> real submodule silently never happens.
>
> This isn't just a missed feature: an unregistered gitlink that has its
> own untracked content (e.g. a further nested git repo underneath it)
> shows up as "dirty" to git status even though the tracked commit hash
> hasn't changed. patch.bbclass's patch_task_postfunc sees that dirtiness
> after do_patch and tries to commit it, but 'git add' has nothing new to
> stage for a gitlink whose hash is unchanged, so the follow-up 'git
> commit' fails with "nothing to commit" and do_patch fails outright.
>
> Fix this by moving the nested-repo detection and submodule conversion
> to run right after 'git init', before 'git add -A .' and the initial
> commit. At that point the nested repo is still untracked and reported
> with a trailing "/", so it's correctly picked up and registered via
> 'git submodule add' before anything commits it as a bare gitlink.
>
> Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
> ---

Hi Jamin,

Thanks for your patch.

It looks like this is breaking two selftests:

2026-07-23 17:18:01,965 - oe-selftest - INFO - devtool.DevtoolUpgradeTests.test_devtool_finish_update_patch (subunit.RemotedTestCase)
2026-07-23 17:18:01,966 - oe-selftest - INFO -  ... FAIL
...
2026-07-23 17:18:01,967 - oe-selftest - INFO - 1: 20/50 268/758 (81.53s) (0 failed) (devtool.DevtoolUpgradeTests.test_devtool_finish_update_patch)
2026-07-23 17:18:01,967 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/devtool.py", line 2629, in test_devtool_finish_update_patch
    self._check_repo_status(recipedir, expected_status)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/devtool.py", line 138, in _check_repo_status
    self.fail('Unexpected modified file in line: %s' % line)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
    raise self.failureException(msg)
AssertionError: Unexpected modified file in line:  D recipes-extended/sysdig/sysdig-selftest/0055-Add-cstdint-for-uintXX_t-types.patch
...
2026-07-23 18:03:48,429 - oe-selftest - INFO - devtool.DevtoolModifyTests.test_devtool_modify_nested_gitsm (subunit.RemotedTestCase)
2026-07-23 18:03:48,429 - oe-selftest - INFO -  ... FAIL
...
2026-07-23 18:03:48,430 - oe-selftest - INFO - 12: 15/38 645/758 (50.47s) (0 failed) (devtool.DevtoolModifyTests.test_devtool_modify_nested_gitsm)
2026-07-23 18:03:48,430 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/devtool.py", line 1256, in test_devtool_modify_nested_gitsm
    result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/layers/openembedded-core/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
    raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'devtool modify devtool-test-git-gitsm -x /tmp/devtoolqac4wc88gw' returned non-zero exit status 1:

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/4411
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4358
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4177

Can you have a look at the issue?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com