[PATCH 0/4] devtool: standard: Fix file copy on finish --force

[PATCH 0/4] devtool: standard: Fix file copy on finish --force

[Mathieu Dubois-Briand]

Fix "devtool finish --force" command, by allowing to copy folders that
were not previously existing but also by preventing it from trying to
remove non-existent files. Devtool was particularly confused when using
the finish subcommand on a recipe that was just added.

Reproducer:

devtool add --version 2.10 https://ftp.gnu.org/gnu/hello/hello-2.10.tar.gz
...
devtool build hello
...
devtool finish hello -f ../openembedded-core/meta/
...
Traceback (most recent call last):
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/devtool", line 352, in <module>
    ret = main()
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/devtool", line 338, in main
    ret = args.func(args, config, basepath, workspace)
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/lib/devtool/standard.py", line 2214, in finish
    updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
                                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/lib/devtool/standard.py", line 1903, in _update_recipe
    updated, appendf, removed = _update_recipe_patch(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, initial_rev, dry_run_outdir, force_patch_refresh)
                                ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/lib/devtool/standard.py", line 1675, in _update_recipe_patch
    upd_f, new_f, del_f = _export_local_files(srctree, rd, local_files_dir, srctreebase)
                          ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/gits/openembedded-core/scripts/lib/devtool/standard.py", line 1469, in _export_local_files
    shutil.copy2(fullfile, os.path.join(destdir, f))
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/shutil.py", line 468, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/shutil.py", line 262, in copyfile
    with open(dst, 'wb') as fdst:
         ~~~~^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/devtool77jejua2/tmpyuo3c_ay/build-aux/compile'

Using --force is arguably a bad idea here, but as we do provide this
possibility, we should make sure the code does not crash.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
Mathieu Dubois-Briand (4):
      scripts: scriptutils: Use LicenseRef-Proprietary LICENSE while fetching
      devtool: standard: Fix file copy on finish --force
      devtool: standard: Remove unused variable
      oe-selftest: devtool: Add test for add/finish workflow

 meta/lib/oeqa/selftest/cases/devtool.py | 58 +++++++++++++++++++++++++++++++++
 scripts/lib/devtool/standard.py         |  9 +++--
 scripts/lib/scriptutils.py              |  2 +-
 3 files changed, 65 insertions(+), 4 deletions(-)
---
base-commit: dd003e147db548d9c86821a08450d6461d061cea
change-id: 20260813-mathieu-devtool-9b0b2768098f

Best regards,
-- 
Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>

[PATCH 1/4] scripts: scriptutils: Use LicenseRef-Proprietary LICENSE while fetching

[Mathieu Dubois-Briand]

Switching away of the "CLOSED" license, as devtool users would encounter
deprecation warnings on "devtool add" command.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
 scripts/lib/scriptutils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/lib/scriptutils.py b/scripts/lib/scriptutils.py
index 32e749dbb1a2..b4fb6cc1845f 100644
--- a/scripts/lib/scriptutils.py
+++ b/scripts/lib/scriptutils.py
@@ -170,7 +170,7 @@ def fetch_url(tinfoil, srcuri, srcrev, destdir, logger, preserve_tmp=False, mirr
             logger.debug('Generating initial recipe %s for fetching' % fetchrecipe)
             with open(fetchrecipe, 'w') as f:
                 # We don't want to have to specify LIC_FILES_CHKSUM
-                f.write('LICENSE = "CLOSED"\n')
+                f.write('LICENSE = "LicenseRef-Proprietary"\n')
                 # We don't need the cross-compiler
                 f.write('INHIBIT_DEFAULT_DEPS = "1"\n')
                 # We don't have the checksums yet so we can't require them

-- 
2.47.3

[PATCH 2/4] devtool: standard: Fix file copy on finish --force

[Mathieu Dubois-Briand]

Fix "devtool finish --force" command, by allowing to copy folders that
were not previously existing but also by preventing it from trying to
remove non-existent files. Devtool was particularly confused when using
the finish subcommand on a recipe that was just added.

Fixes: ce8190c51905: devtool: Drop oe-local-files and simplify
---
 scripts/lib/devtool/standard.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 9b2643604d1d..536411efef6a 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1461,13 +1461,17 @@ def _export_local_files(srctree, rd, destdir, srctreebase):
         if os.path.exists(os.path.join(fullfile, ".git")):
             # submodules handled elsewhere
             continue
-        if f not in existing_files:
+        if f not in existing_files and os.path.exists(fullfile):
             added[f] = {}
+            parentdir = os.path.normpath(os.path.join(destdir, f, os.pardir))
+            if not os.path.isdir(parentdir):
+                os.makedirs(parentdir)
+
             if os.path.isdir(os.path.join(srctree, f)):
                 shutil.copytree(fullfile, os.path.join(destdir, f))
             else:
                 shutil.copy2(fullfile, os.path.join(destdir, f))
-        elif not os.path.exists(fullfile):
+        elif f in existing_files and not os.path.exists(fullfile):
             removed[f] = existing_files[f]
         elif f in existing_files:
             updated[f] = {'path' : existing_files[f]}

-- 
2.47.3

Patchtest results for [PATCH 2/4] devtool: standard: Fix file copy on finish --force

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 3218 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/2-4-devtool-standard-Fix-file-copy-on-finish---force.patch

FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping test (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

Re: [OE-core] Patchtest results for [PATCH 2/4] devtool: standard: Fix file copy on finish --force

[Mathieu Dubois-Briand]

On Fri Aug 14, 2026 at 6:32 PM CEST, Patchtest via lists.openembedded.org wrote:
> Thank you for your submission. Patchtest identified one
> or more issues with the patch. Please see the log below for
> more information:
>
> ---
> Testing patch /home/patchtest/share/mboxes/2-4-devtool-standard-Fix-file-copy-on-finish---force.patch
>
> FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence)
>

Not sure how I managed to skip the Signed-off-by. I will fix it in my
branch...

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH 3/4] devtool: standard: Remove unused variable

[Mathieu Dubois-Briand]

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
 scripts/lib/devtool/standard.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 536411efef6a..ac9e7cd61a58 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1437,7 +1437,6 @@ def _export_local_files(srctree, rd, destdir, srctreebase):
     # recipe space).
     existing_files = oe.recipeutils.get_recipe_local_files(rd)
 
-    new_set = None
     updated = OrderedDict()
     added = OrderedDict()
     removed = OrderedDict()

-- 
2.47.3

Patchtest results for [PATCH 3/4] devtool: standard: Remove unused variable

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 3198 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/3-4-devtool-standard-Remove-unused-variable.patch

FAIL: test commit message presence: Please include a commit message on your patch explaining the change (test_mbox.TestMbox.test_commit_message_presence)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping test (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

[PATCH 4/4] oe-selftest: devtool: Add test for add/finish workflow

[Mathieu Dubois-Briand]

Add test for devtool add/build/finish workflow, inclusing a test for
finish --force.
---
 meta/lib/oeqa/selftest/cases/devtool.py | 58 +++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index d84a18e8b69b..d549be9f7965 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -713,6 +713,64 @@ class DevtoolAddTests(DevtoolBase):
         self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
         result = runCmd('devtool add %s %s -f %s' % (testrecipe, srcdir, url))
 
+    def test_devtool_add_build_finish(self):
+        url = 'https://ftp.gnu.org/gnu/hello/hello-2.12.3.tar.gz'
+        pn = 'hello'
+        recipe = 'hello_2.12.3.bb'
+        # Test devtool add
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake -c cleansstate %s' % pn)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        result = runCmd('devtool add %s' % (url, ))
+        self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'),
+                          'Workspace directory not created')
+        # Test devtool build
+        result = runCmd('devtool build %s' % pn)
+        bb_vars = get_bb_vars(['D', 'bindir'], pn)
+        installdir = bb_vars['D']
+        self.assertTrue(installdir, 'Could not query installdir variable')
+        # devtool finish
+        result = runCmd('devtool finish %s meta-selftest' % pn,
+                        ignore_status=True)
+        self.assertNotEqual(result.status, 0,
+                            'devtool finish should have failed on a dirty directory. devtool output: %s' %
+                            (result.output))
+        # clean directory
+        sourcedir = os.path.join(self.workspacedir, 'sources', 'hello')
+        runCmd('git reset --hard; git clean -dxf', cwd=sourcedir)
+        # devtool finish (again)
+        self.assertExists(os.path.join(self.workspacedir, 'recipes', pn),
+                          'Recipe directory should exist before finish')
+        result = runCmd('devtool finish %s meta-selftest' % pn)
+        self.assertNotExists(os.path.join(self.workspacedir, 'recipes', pn),
+                             'Recipe directory should not exist after finish')
+        # Check recipe got created as expected
+        self.assertExists(os.path.join(get_test_layer(), f'recipes-{pn}', pn, recipe),
+                          'Recipe should exist in the layer after finish')
+
+    def test_devtool_add_build_finish_force(self):
+        url = 'https://ftp.gnu.org/gnu/hello/hello-2.12.3.tar.gz'
+        pn = 'hello'
+        recipe = 'hello_2.12.3.bb'
+        # Test devtool add
+        self.track_for_cleanup(self.workspacedir)
+        self.add_command_to_tearDown('bitbake -c cleansstate %s' % pn)
+        self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+        result = runCmd('devtool add %s' % (url, ))
+        self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'),
+                          'Workspace directory not created')
+        # Test devtool build
+        result = runCmd('devtool build %s' % pn)
+        bb_vars = get_bb_vars(['D', 'bindir'], pn)
+        installdir = bb_vars['D']
+        self.assertTrue(installdir, 'Could not query installdir variable')
+        # devtool finish
+        result = runCmd('devtool finish %s meta-selftest --force' % pn)
+        # Check recipe got created as expected
+        self.assertExists(os.path.join(get_test_layer(), f'recipes-{pn}', pn, recipe),
+                          'Recipe should exist in the layer after finish')
+
+
 class DevtoolModifyTests(DevtoolBase):
 
     def test_devtool_modify(self):

-- 
2.47.3

Patchtest results for [PATCH 4/4] oe-selftest: devtool: Add test for add/finish workflow

[patchtest]

[-- Attachment #1: Type: text/plain, Size: 3222 bytes --]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/4-4-oe-selftest-devtool-Add-test-for-add-finish-workflow.patch

FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping test (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!