* [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes
@ 2026-08-30 21:48 AdrianF
2026-08-30 21:48 ` [PATCH v2 01/25] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction AdrianF
` (24 more replies)
0 siblings, 25 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
changes v1 --> v2:
- Complaints from Patchwork
- One more fix commit
This series improves `devtool ide-sdk` and its underlying deploy-target
infrastructure:
- Adds `--nfs` rootfs debugging support (`rootfs`/`rootfs-dbg`), allowing the
target to boot from a combined companion `rootfs-dbg` — opening up
possibilities such as code coverage and profiling (e.g. `perf`) and other
tools that need debug symbols present on the target.
- Runs `do_install` through BitBake instead of shelling out, using a new
tinfoil prepared-task runner — much better functionally and also much
quicker than before.
Requires the latest bitbake master (adds `Tinfoil.run_prepared_task`).
- Adds `runqemu` slirp networking support for ide-sdk (with auto
port-forwarding and automatic SSH host-key check disabling for loopback
targets).
- Adds `--package`/`--file-glob` deploy filters to `devtool deploy-target`,
plus support for deploying directly into a local rootfs path (e.g. an NFS
export), refactoring the ssh deployment path into its own function.
- Refactors `runqemu-extract-sdk` and `runqemu-export-rootfs` into Python.
- Various ide-sdk fixes: auto-write image debug settings to the bbappend,
dedupe `solib_search_path` entries, VSCode IntelliSense for rootfs-dbg
sources, and cpp-example breakpoint/attach fixes.
Each feature/fix is covered by a corresponding oe-selftest addition.
A documentation update covering these changes will be sent as a follow-up series shortly.
Adrian Freihofer (25):
runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction
devtool: ide-sdk: dedupe solib_search_path entries
devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources
cpp-example: fix stuck breakpoints when attaching and daemonize
properly
devtool: ide-sdk: auto-write image debug settings to bbappend
oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug
settings
oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams
devtool: ide-sdk: support runqemu slirp
devtool: ide-sdk: auto-disable ssh host key checking for loopback
targets
oe-selftest: devtool ide-sdk: add slirp networking test
devtool: deploy-target: add --package/--file-glob filters
oe-selftest: devtool deploy-target: test --package/--file-glob filters
devtool: ide-sdk: forward --package/--file-glob deploy filters
oe-selftest: devtool ide-sdk: cover --package filters
runqemu-extract-sdk: refactor in Python
runqemu-export-rootfs: refactor in Python
devtool: deploy: split ssh deployment into a separate function
devtool: deploy-target: allow deploying directly into a local rootfs
oe-selftest: devtool deploy-target: test deploying into a local rootfs
path
oe-selftest: tinfoil: test prepared task runner
devtool: ide-sdk: run do_install through BitBake
oe-selftest: devtool ide-sdk: test install task
devtool: ide-sdk: support NFS rootfs
oe-selftest: devtool ide-sdk: test NFS debug rootfs
devtool: ide-sdk: fix GDB loading stale libs instead of recipe's own
build
.../recipes-test/cpp/cmake-example.inc | 1 +
.../recipes-test/cpp/cpp-example.inc | 4 +
.../recipes-test/cpp/files/CMakeLists.txt | 6 +
.../recipes-test/cpp/files/cpp-example.cpp | 91 ++-
.../recipes-test/cpp/files/cpp-example.init | 10 +-
.../recipes-test/cpp/files/daemonize.cpp | 57 ++
.../recipes-test/cpp/files/daemonize.hpp | 27 +
.../recipes-test/cpp/files/meson.build | 9 +-
.../recipes-test/cpp/files/meson.options | 2 +
.../recipes-test/cpp/meson-example.inc | 1 +
.../pseudo-pyc-test/pseudo-pyc-test.bb | 45 ++
meta/lib/oeqa/selftest/cases/devtool.py | 632 +++++++++++++----
meta/lib/oeqa/selftest/cases/tinfoil.py | 60 ++
meta/lib/oeqa/targetcontrol.py | 23 +-
.../qemu/nativesdk-qemu-helper_1.0.bb | 3 +
scripts/lib/devtool/deploy.py | 392 +++++++++--
scripts/lib/devtool/ide_plugins/__init__.py | 41 +-
scripts/lib/devtool/ide_plugins/ide_code.py | 93 ++-
scripts/lib/devtool/ide_plugins/ide_none.py | 8 +-
scripts/lib/devtool/ide_sdk.py | 649 +++++++++++++++---
scripts/lib/runqemu_utils.py | 246 +++++++
scripts/runqemu-export-rootfs | 126 +---
scripts/runqemu-extract-sdk | 101 +--
23 files changed, 2090 insertions(+), 537 deletions(-)
create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.cpp
create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.hpp
create mode 100644 scripts/lib/runqemu_utils.py
--
2.55.0
^ permalink raw reply [flat|nested] 31+ messages in thread
* [PATCH v2 01/25] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries AdrianF
` (23 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
pseudo inverted its path-matching logic some time ago: when set,
PSEUDO_INCLUDE_PATHS acts as an ownership-tracking allowlist, and
anything outside it silently falls through to real chown()/chmod()
instead of being recorded by pseudo. This script never set it, so an
inherited restrictive value could leave the extracted rootfs with
wrong ownership. Since this script only ever touches SDK_ROOTFS_DIR,
setting PSEUDO_INCLUDE_PATHS to it is enough.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
| 2 ++
1 file changed, 2 insertions(+)
--git a/scripts/runqemu-extract-sdk b/scripts/runqemu-extract-sdk
index 528ef76430..db9813e1dc 100755
--- a/scripts/runqemu-extract-sdk
+++ b/scripts/runqemu-extract-sdk
@@ -85,6 +85,8 @@ mkdir -p "$pseudo_state_dir"
touch "$pseudo_state_dir/pseudo.pid"
PSEUDO_LOCALSTATEDIR="$pseudo_state_dir"
export PSEUDO_LOCALSTATEDIR
+PSEUDO_INCLUDE_PATHS="$SDK_ROOTFS_DIR"
+export PSEUDO_INCLUDE_PATHS
echo "Extracting rootfs tarball using pseudo..."
echo "$PSEUDO $PSEUDO_OPTS tar -C \"$SDK_ROOTFS_DIR\" $TAR_OPTS \"$ROOTFS_TARBALL\""
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
2026-08-30 21:48 ` [PATCH v2 01/25] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-09-03 11:18 ` [OE-core] " Richard Purdie
2026-08-30 21:48 ` [PATCH v2 03/25] devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources AdrianF
` (22 subsequent siblings)
24 siblings, 1 reply; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
With usrmerge in DISTRO_FEATURES (the default), root_prefix becomes
exec_prefix, so base_libdir and libdir resolve to the same path. This
made most entries of solib_search_path() literal duplicates. Dedupe
while preserving order.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_sdk.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 4fdeeec24d..6f08940c9c 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -643,7 +643,8 @@ class RecipeModified:
os.path.join(image.rootfs, base_libdir),
os.path.join(image.rootfs, libdir)
]
- return so_paths
+ # base_libdir == libdir when usrmerge is in DISTRO_FEATURES (the default), dedupe
+ return list(dict.fromkeys(so_paths))
def solib_search_path_str(self, image):
"""Return a : separated list of paths usable by GDB's set solib-search-path"""
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 03/25] devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
2026-08-30 21:48 ` [PATCH v2 01/25] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction AdrianF
2026-08-30 21:48 ` [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 04/25] cpp-example: fix stuck breakpoints when attaching and daemonize properly AdrianF
` (21 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Add includePath fallback entries to c_cpp_properties.json pointing at
recipe-sysroot/usr/include and rootfs-dbg/usr/src/debug, so the C/C++
extension can resolve symbols in other recipes' sources whenever they
are opened from rootfs-dbg, e.g. while stepping into them with the
debugger.
Also widen files.readonlyInclude from just the recipe's sysroots to
the whole TMPDIR: more robust, since it also covers oe-workdir, rootfs,
and other generated content reachable from the workspace. oe-logs and
oe-workdir are kept visible in the VSCode explorer instead of being
hidden via files.exclude, so build logs and temp scripts can still be
browsed, while staying out of the file watcher/indexer and read-only.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 8 ++--
scripts/lib/devtool/ide_plugins/ide_code.py | 44 +++++++++++++++------
scripts/lib/devtool/ide_sdk.py | 2 +
3 files changed, 38 insertions(+), 16 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 7c989f9dc2..c0df13b718 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3945,11 +3945,11 @@ class DevtoolIdeSdkKernelTests(DevtoolIdeSdkTests):
os.path.join(tempdir, 'Makefile'),
'makefilePath should point to the Makefile in the source tree')
- # Verify kernel sources are set read-only
+ # Verify kernel sources (under TMPDIR) are set read-only
+ tmpdir = get_bb_var('TMPDIR', recipe_name)
readonly_includes = settings_d.get('files.readonlyInclude', {})
- self.assertTrue(
- any(k for k in readonly_includes if 'staging_kernel' in k.lower() or 'linux' in k.lower()),
- 'Kernel staging dir should be set read-only in files.readonlyInclude: %s' % readonly_includes)
+ self.assertIn(os.path.realpath(tmpdir) + '/**', readonly_includes,
+ 'TMPDIR should be set read-only in files.readonlyInclude: %s' % readonly_includes)
# Verify the cross-build environment is exported for the terminal
self.assertIn('terminal.integrated.env.linux', settings_d,
diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index 1a8a79f623..5abf0492e8 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -205,9 +205,6 @@ class IdeVSCode(IdeBase):
settings_dict["files.watcherExclude"].update(files_excludes_kernel)
settings_dict["python.analysis.exclude"] += kernel_exclude_patterns
- # protect the kernel sources
- settings_dict["files.readonlyInclude"][modified_recipe.staging_kernel_dir + '/**'] = True
-
# Export the complete cross-build environment
settings_dict["terminal.integrated.env.linux"] = modified_recipe.exported_vars
@@ -232,12 +229,15 @@ class IdeVSCode(IdeBase):
]
def vscode_settings(self, modified_recipe, image_recipe):
- files_excludes = {
+ files_hide = {
"**/.git/**": True,
- "**/oe-logs/**": True,
- "**/oe-workdir/**": True,
"**/source-date-epoch/**": True
}
+ files_watcher_exclude = dict(files_hide)
+ files_watcher_exclude.update({
+ "**/oe-logs/**": True,
+ "**/oe-workdir/**": True,
+ })
python_exclude = [
"**/.git/**",
"**/oe-logs/**",
@@ -245,14 +245,16 @@ class IdeVSCode(IdeBase):
"**/source-date-epoch/**"
]
files_readonly = {
- modified_recipe.recipe_sysroot + '/**': True,
- modified_recipe.recipe_sysroot_native + '/**': True,
+ modified_recipe.tmpdir + '/**': True,
+ "**/oe-logs/**": True,
+ "**/oe-workdir/**": True,
}
if image_recipe.rootfs_dbg is not None:
files_readonly[image_recipe.rootfs_dbg + '/**'] = True
settings_dict = {
- "files.watcherExclude": files_excludes,
- "files.exclude": files_excludes,
+ "files.watcherExclude": files_watcher_exclude,
+ "files.exclude": files_hide,
+ "search.exclude": dict(files_watcher_exclude),
"files.readonlyInclude": files_readonly,
"python.analysis.exclude": python_exclude
}
@@ -287,7 +289,7 @@ class IdeVSCode(IdeBase):
IdeBase.update_json_file(
self.dot_code_dir(modified_recipe), extensions_file, {"recommendations": recommendations})
- def vscode_c_cpp_properties(self, modified_recipe):
+ def vscode_c_cpp_properties(self, modified_recipe, image_recipe):
properties_dict = {
"name": modified_recipe.recipe_id_pretty,
}
@@ -322,6 +324,24 @@ class IdeVSCode(IdeBase):
else: # no C/C++ build
return
+ # configurationProvider/compileCommands only cover the recipe under
+ # development. Add includePath as a fallback so the C/C++ extension
+ # also resolves symbols in other recipes sources found in rootfs-dbg.
+ if image_recipe.rootfs_dbg is not None:
+ recipe_sysroot_include = os.path.join(modified_recipe.recipe_sysroot, "usr", "include")
+ # rootfs_dbg/usr/include is empty, target headers come from recipe-sysroot,
+ # consistent with the GDB sourceFileMap for "/usr/include".
+ rootfs_dbg_src_debug = os.path.join(image_recipe.rootfs_dbg, "usr", "src", "debug")
+ include_path = properties_dict.get("includePath", ["${workspaceFolder}/**"])
+ for path in (recipe_sysroot_include, rootfs_dbg_src_debug + "/**"):
+ if path not in include_path:
+ include_path.append(path)
+ properties_dict["includePath"] = include_path
+ # That's the default, but make it easy to change if a big index is preferred.
+ properties_dict["browse"] = {
+ "limitSymbolsToIncludedHeaders": True
+ }
+
properties_dicts = {
"configurations": [
properties_dict
@@ -873,7 +893,7 @@ class IdeVSCode(IdeBase):
def setup_modified_recipe(self, args, image_recipe, modified_recipe):
self.vscode_settings(modified_recipe, image_recipe)
self.vscode_extensions(modified_recipe)
- self.vscode_c_cpp_properties(modified_recipe)
+ self.vscode_c_cpp_properties(modified_recipe, image_recipe)
if args.target:
if modified_recipe.toolchain == 'clang':
self.initialize_cross_debug_configs(
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 6f08940c9c..d7575ec523 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -457,6 +457,7 @@ class RecipeModified:
self.staging_incdir = None
self.strip_cmd = None
self.target_arch = None
+ self.tmpdir = None
self.toolchain = None
self.topdir = None
self.workdir = None
@@ -545,6 +546,7 @@ class RecipeModified:
recipe_d.getVar('STAGING_INCDIR'))
self.strip_cmd = recipe_d.getVar('STRIP')
self.target_arch = recipe_d.getVar('TARGET_ARCH')
+ self.tmpdir = os.path.realpath(recipe_d.getVar('TMPDIR'))
self.toolchain = recipe_d.getVar('TOOLCHAIN')
self.topdir = recipe_d.getVar('TOPDIR')
self.workdir = os.path.realpath(recipe_d.getVar('WORKDIR'))
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 04/25] cpp-example: fix stuck breakpoints when attaching and daemonize properly
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (2 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 03/25] devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 05/25] devtool: ide-sdk: auto-write image debug settings to bbappend AdrianF
` (20 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Attaching gdbserver to the already-running SysV-init-started instance
and then adding a breakpoint left it unbound (grey/hollow) forever,
even though setting the same breakpoint before starting the debugger
worked fine.
Root cause: cpp-example.init started the daemon via
`start-stop-daemon --background`, which double-forks to detach from
the terminal. That leaves the final daemon process in a process group
whose leader has already exited, i.e. its pgid no longer equals its
own pid. gdbserver's remote-interrupt mechanism pauses a running
inferior with `kill(-pid, SIGINT)` (signal the whole process group);
with a mismatched pgid this fails silently (ESRCH), GDB's
`-exec-interrupt` never actually stops the target, and MIEngine's
pending-breakpoint bind times out and reports it as unbound.
Fix by dropping the double-fork daemonization we inherited from
`start-stop-daemon`/SysV-init cargo culting in favor of the classic,
correct Unix daemon recipe: a single fork() + setsid(), which makes
the process its own session AND process-group leader (pgid == pid)
and keeps gdbserver's SIGINT-to-process-group interrupt working. As
part of doing this properly:
- cpp-example itself now daemonizes (--daemonize), writes its own
pidfile (--pidfile), and drops root privileges (--uid/--gid)
instead of depending on start-stop-daemon/su for that.
- stdio is redirected to /dev/null after setsid() so the service
actually releases the console instead of holding it open forever.
- Since stdio is gone once daemonized, log through syslog instead so
the service's messages aren't silently discarded.
- All of this legacy SysV-init-only behavior lives in a new
daemonize.cpp/.hpp, compiled in only when WITH_SYSTEMD is off;
under systemd (Type=simple) none of it is needed since systemd
already daemonizes, drops privileges, and tracks the pid itself.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
.../recipes-test/cpp/cmake-example.inc | 1 +
.../recipes-test/cpp/cpp-example.inc | 4 +
.../recipes-test/cpp/files/CMakeLists.txt | 6 ++
.../recipes-test/cpp/files/cpp-example.cpp | 91 +++++++++++++++++--
.../recipes-test/cpp/files/cpp-example.init | 10 +-
.../recipes-test/cpp/files/daemonize.cpp | 57 ++++++++++++
.../recipes-test/cpp/files/daemonize.hpp | 27 ++++++
.../recipes-test/cpp/files/meson.build | 9 +-
.../recipes-test/cpp/files/meson.options | 2 +
.../recipes-test/cpp/meson-example.inc | 1 +
meta/lib/oeqa/selftest/cases/devtool.py | 8 +-
11 files changed, 198 insertions(+), 18 deletions(-)
create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.cpp
create mode 100644 meta-selftest/recipes-test/cpp/files/daemonize.hpp
diff --git a/meta-selftest/recipes-test/cpp/cmake-example.inc b/meta-selftest/recipes-test/cpp/cmake-example.inc
index eb023d389a..38d53af7b8 100644
--- a/meta-selftest/recipes-test/cpp/cmake-example.inc
+++ b/meta-selftest/recipes-test/cpp/cmake-example.inc
@@ -11,6 +11,7 @@ SRC_URI += "file://CMakeLists.txt"
inherit cmake-qemu
PACKAGECONFIG[failing_test] = "-DFAILING_TEST=ON"
+PACKAGECONFIG[systemd] = "-DWITH_SYSTEMD=ON,-DWITH_SYSTEMD=OFF"
# Support installing all recipe variants in parallel
EXTRA_OECMAKE += "\
diff --git a/meta-selftest/recipes-test/cpp/cpp-example.inc b/meta-selftest/recipes-test/cpp/cpp-example.inc
index 0070d17201..3934345f17 100644
--- a/meta-selftest/recipes-test/cpp/cpp-example.inc
+++ b/meta-selftest/recipes-test/cpp/cpp-example.inc
@@ -15,6 +15,8 @@ SRC_URI = "\
file://cpp-example.cpp \
file://cpp-example-lib.hpp \
file://cpp-example-lib.cpp \
+ file://daemonize.cpp \
+ file://daemonize.hpp \
file://test-cpp-example.cpp \
file://cpp-example.conf \
file://config.h.in \
@@ -33,6 +35,8 @@ SYSTEMD_SERVICE:${PN} = "${BPN}.service"
INITSCRIPT_NAME = "${BPN}"
INITSCRIPT_PARAMS = "defaults 99"
+PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', '', d)}"
+
# Create cpp-example user and group
USERADD_PACKAGES = "${PN}"
GROUPADD_PARAM:${PN} = "--system ${EX_SERVICE_USER}"
diff --git a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt
index 8802839702..e063bdafb3 100644
--- a/meta-selftest/recipes-test/cpp/files/CMakeLists.txt
+++ b/meta-selftest/recipes-test/cpp/files/CMakeLists.txt
@@ -13,6 +13,7 @@ project(cmake-example
option(BUILD_SHARED_LIBS "Build using shared libraries" ON)
option(FAILING_TEST "Compile a failing unit test to test the test infrastructure" OFF)
+option(WITH_SYSTEMD "Target is managed by systemd: skip the legacy SysV daemonize/pidfile/privilege-drop code" OFF)
set(BINARY_NAME "cmake-example" CACHE STRING "Name of the installed executable and library prefix")
set(TEST_BINARY_NAME "test-cmake-example" CACHE STRING "Name of the installed test executable")
@@ -54,6 +55,11 @@ install(TARGETS ${BINARY_NAME}-lib
add_executable(${BINARY_NAME} cpp-example.cpp)
target_include_directories(${BINARY_NAME} PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
target_link_libraries(${BINARY_NAME} PRIVATE ${BINARY_NAME}-lib)
+if (WITH_SYSTEMD)
+ target_compile_definitions(${BINARY_NAME} PRIVATE WITH_SYSTEMD)
+else()
+ target_sources(${BINARY_NAME} PRIVATE daemonize.cpp daemonize.hpp)
+endif(WITH_SYSTEMD)
install(TARGETS ${BINARY_NAME}
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp
index a376419c13..f18911554c 100644
--- a/meta-selftest/recipes-test/cpp/files/cpp-example.cpp
+++ b/meta-selftest/recipes-test/cpp/files/cpp-example.cpp
@@ -6,44 +6,117 @@
#include "cpp-example-lib.hpp"
+#include <cstdlib>
#include <iostream>
#include <unistd.h>
#include <string>
#include <vector>
+#ifndef WITH_SYSTEMD
+#include <sys/types.h>
+#include <syslog.h>
+
+#include "daemonize.hpp"
+#endif
+
+namespace {
+
+#ifndef WITH_SYSTEMD
+bool g_use_syslog = false;
+#endif
+
+// Prints an informational message. Under systemd, stdout is already captured
+// by the journal. Otherwise, use stdout until daemonized (stdio is then
+// redirected to /dev/null), and syslog afterwards so messages aren't lost.
+void log_info(const std::string& msg)
+{
+#ifndef WITH_SYSTEMD
+ if (g_use_syslog) {
+ syslog(LOG_INFO, "%s", msg.c_str());
+ return;
+ }
+#endif
+ std::cout << msg << std::endl;
+}
+
+} // namespace
+
int main(int argc, char* argv[])
{
bool endless_mode = false;
+#ifndef WITH_SYSTEMD
+ bool daemonize_mode = false;
+ std::string pidfile_path;
+ bool have_uid = false;
+ bool have_gid = false;
+ uid_t target_uid = 0;
+ gid_t target_gid = 0;
+#endif
// Parse command line arguments
for (int i = 1; i < argc; i++) {
- if (std::string(argv[i]) == "--endless") {
+ std::string arg = argv[i];
+ if (arg == "--endless") {
endless_mode = true;
- } else if (std::string(argv[i]) == "--help" || std::string(argv[i]) == "-h") {
+#ifndef WITH_SYSTEMD
+ } else if (arg == "--daemonize") {
+ daemonize_mode = true;
+ } else if (arg == "--pidfile" && i + 1 < argc) {
+ pidfile_path = argv[++i];
+ } else if (arg == "--uid" && i + 1 < argc) {
+ target_uid = static_cast<uid_t>(std::strtoul(argv[++i], nullptr, 10));
+ have_uid = true;
+ } else if (arg == "--gid" && i + 1 < argc) {
+ target_gid = static_cast<gid_t>(std::strtoul(argv[++i], nullptr, 10));
+ have_gid = true;
+#endif
+ } else if (arg == "--help" || arg == "-h") {
std::cout << "Usage: " << argv[0] << " [OPTIONS]" << std::endl;
std::cout << "Options:" << std::endl;
- std::cout << " --endless Run in endless loop mode (for service)" << std::endl;
- std::cout << " --help, -h Show this help message" << std::endl;
+ std::cout << " --endless Run in endless loop mode (for service)" << std::endl;
+#ifndef WITH_SYSTEMD
+ std::cout << " --daemonize Detach from the controlling terminal" << std::endl;
+ std::cout << " --pidfile <path> Write the daemon's PID to <path>" << std::endl;
+ std::cout << " --uid <uid> Drop root privileges to this user ID" << std::endl;
+ std::cout << " --gid <gid> Drop root privileges to this group ID" << std::endl;
+#endif
+ std::cout << " --help, -h Show this help message" << std::endl;
return 0;
}
}
+#ifndef WITH_SYSTEMD
+ if (daemonize_mode) {
+ daemonize();
+ openlog(argv[0], LOG_PID, LOG_DAEMON);
+ g_use_syslog = true;
+ }
+
+ if (!pidfile_path.empty()) {
+ write_pidfile(pidfile_path);
+ }
+
+ // Drop privileges after daemonizing/writing the pidfile (both may need
+ // root, e.g. to create files under /var/run), but before doing any work.
+ drop_privileges(have_gid, target_gid, have_uid, target_uid);
+#endif
+
auto cpp_example = CppExample();
if (endless_mode) {
- std::cout << "Starting cpp-example service in endless mode..." << std::endl;
+ log_info("Starting cpp-example service in endless mode...");
} else {
- std::cout << "Running cpp-example once..." << std::endl;
+ log_info("Running cpp-example once...");
}
- std::cout << "C++ example linking " << cpp_example.get_string() << std::endl;
- std::cout << "Linking json-c version " << cpp_example.get_json_c_version() << std::endl;
+ log_info("C++ example linking " + cpp_example.get_string());
+ log_info(std::string("Linking json-c version ") + cpp_example.get_json_c_version());
cpp_example.print_json();
do {
// Read and print message from config file
std::string config_message = cpp_example.read_config_message();
- std::cout << "Config file message: " << config_message << std::endl;
+ log_info("Config file message: " + config_message);
if (endless_mode) {
// Sleep for 1 second
diff --git a/meta-selftest/recipes-test/cpp/files/cpp-example.init b/meta-selftest/recipes-test/cpp/files/cpp-example.init
index 30b8486eeb..6ef9aaac38 100644
--- a/meta-selftest/recipes-test/cpp/files/cpp-example.init
+++ b/meta-selftest/recipes-test/cpp/files/cpp-example.init
@@ -25,8 +25,7 @@ start() {
fi
echo -n "Starting $DAEMON: "
- start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile \
- --background --chuid $USER --exec $DAEMON_PATH -- $DAEMON_ARGS
+ $DAEMON_PATH --daemonize --pidfile "$PIDFILE" --uid "$(id -u "$USER")" --gid "$(id -g "$USER")" $DAEMON_ARGS
RETVAL=$?
if [ $RETVAL -eq 0 ]; then
echo "OK"
@@ -39,8 +38,11 @@ start() {
stop() {
echo -n "Stopping $DAEMON: "
- start-stop-daemon --stop --quiet --pidfile $PIDFILE
- RETVAL=$?
+ if [ -f "$PIDFILE" ] && kill "$(cat "$PIDFILE")" 2>/dev/null; then
+ RETVAL=0
+ else
+ RETVAL=1
+ fi
if [ $RETVAL -eq 0 ]; then
echo "OK"
rm -f $PIDFILE $LOCK_FILE
diff --git a/meta-selftest/recipes-test/cpp/files/daemonize.cpp b/meta-selftest/recipes-test/cpp/files/daemonize.cpp
new file mode 100644
index 0000000000..7d91259bfa
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/files/daemonize.cpp
@@ -0,0 +1,57 @@
+/*
+ * Copyright OpenEmbedded Contributors
+ *
+ * SPDX-License-Identifier: MIT
+ */
+
+#include "daemonize.hpp"
+
+#include <cstdlib>
+#include <fcntl.h>
+#include <fstream>
+#include <grp.h>
+#include <unistd.h>
+
+void daemonize()
+{
+ pid_t pid = fork();
+ if (pid < 0) {
+ std::perror("fork");
+ std::exit(1);
+ }
+ if (pid > 0) {
+ _exit(0);
+ }
+ setsid();
+
+ // setsid() only drops the controlling-terminal association; stdio still
+ // points at the console, so redirect it or an --endless service keeps
+ // the console open/busy with its output forever.
+ int null_fd = open("/dev/null", O_RDWR);
+ if (null_fd >= 0) {
+ dup2(null_fd, STDIN_FILENO);
+ dup2(null_fd, STDOUT_FILENO);
+ dup2(null_fd, STDERR_FILENO);
+ if (null_fd > STDERR_FILENO) {
+ close(null_fd);
+ }
+ }
+}
+
+void write_pidfile(const std::string& path)
+{
+ std::ofstream pidfile(path, std::ios::trunc);
+ pidfile << getpid() << std::endl;
+}
+
+void drop_privileges(bool have_gid, gid_t gid, bool have_uid, uid_t uid)
+{
+ if (have_gid && (setgroups(0, nullptr) != 0 || setgid(gid) != 0)) {
+ std::perror("setgid");
+ std::exit(1);
+ }
+ if (have_uid && setuid(uid) != 0) {
+ std::perror("setuid");
+ std::exit(1);
+ }
+}
diff --git a/meta-selftest/recipes-test/cpp/files/daemonize.hpp b/meta-selftest/recipes-test/cpp/files/daemonize.hpp
new file mode 100644
index 0000000000..17d85d690a
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/files/daemonize.hpp
@@ -0,0 +1,27 @@
+/*
+ * Copyright OpenEmbedded Contributors
+ *
+ * SPDX-License-Identifier: MIT
+ */
+
+#pragma once
+
+// Legacy SysV-init daemonizing helpers. Not needed (and not built) when
+// WITH_SYSTEMD is set, since systemd already daemonizes, drops privileges,
+// and tracks the pid itself.
+
+#include <string>
+#include <sys/types.h>
+
+// Detaches from the controlling terminal. Uses a single fork (not the classic
+// double-fork) so this process becomes its own session AND process-group
+// leader (pgid == pid); a second fork would hand it off to a child with a
+// stale pgid, which breaks gdbserver's SIGINT-to-process-group interrupt.
+void daemonize();
+
+// Writes the current process's pid to the given path.
+void write_pidfile(const std::string& path);
+
+// Permanently drops from root to the given group/user. Must run gid before
+// uid: once uid is dropped, the process no longer has permission to setgid.
+void drop_privileges(bool have_gid, gid_t gid, bool have_uid, uid_t uid);
diff --git a/meta-selftest/recipes-test/cpp/files/meson.build b/meta-selftest/recipes-test/cpp/files/meson.build
index 3cb4669dfa..620c220dbd 100644
--- a/meta-selftest/recipes-test/cpp/files/meson.build
+++ b/meta-selftest/recipes-test/cpp/files/meson.build
@@ -30,6 +30,13 @@ configure_file(input : 'config.h.in',
# Include the build directory for config.h
inc_dir = include_directories('.')
+example_sources = ['cpp-example.cpp']
+if get_option('WITH_SYSTEMD').enabled()
+ add_project_arguments('-DWITH_SYSTEMD', language: 'cpp')
+else
+ example_sources += ['daemonize.cpp', 'daemonize.hpp']
+endif
+
exlib = shared_library(binary_name + 'lib',
'cpp-example-lib.cpp', 'cpp-example-lib.hpp',
version: meson.project_version(),
@@ -40,7 +47,7 @@ exlib = shared_library(binary_name + 'lib',
)
executable(binary_name,
- 'cpp-example.cpp',
+ example_sources,
link_with : exlib,
include_directories : inc_dir,
install : true
diff --git a/meta-selftest/recipes-test/cpp/files/meson.options b/meta-selftest/recipes-test/cpp/files/meson.options
index 374e346197..a507518990 100644
--- a/meta-selftest/recipes-test/cpp/files/meson.options
+++ b/meta-selftest/recipes-test/cpp/files/meson.options
@@ -1,6 +1,8 @@
option('FAILING_TEST', type : 'feature', value : 'disabled',
description : 'Compile a failing unit test to test the test infrastructure')
+option('WITH_SYSTEMD', type : 'feature', value : 'disabled',
+ description : 'Target is managed by systemd: skip the legacy SysV daemonize/pidfile/privilege-drop code')
option('BINARY_NAME', type : 'string', value : 'mesonex',
description : 'Name of the installed executable')
option('TEST_BINARY_NAME', type : 'string', value : 'test-mesonex',
diff --git a/meta-selftest/recipes-test/cpp/meson-example.inc b/meta-selftest/recipes-test/cpp/meson-example.inc
index 2937be27f8..eb60649636 100644
--- a/meta-selftest/recipes-test/cpp/meson-example.inc
+++ b/meta-selftest/recipes-test/cpp/meson-example.inc
@@ -16,6 +16,7 @@ SRC_URI += "\
inherit pkgconfig meson
PACKAGECONFIG[failing_test] = "-DFAILING_TEST=enabled"
+PACKAGECONFIG[systemd] = "-DWITH_SYSTEMD=enabled,-DWITH_SYSTEMD=disabled"
# Support installing all recipes variants in parallel
EXTRA_OEMESON += "\
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index c0df13b718..87efa1ee80 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3123,7 +3123,7 @@ class DevtoolIdeSdkTests(DevtoolBase):
# the first _gdb_cross_debugging_multi call above.
self._gdb_cross_debugging_multi(
qemu, recipe_name, example_exe, MAGIC_STRING_NEW,
- exe_break_line=63 + LINE_SHIFT, exe_list_line=55 + LINE_SHIFT,
+ exe_break_line=136 + LINE_SHIFT, exe_list_line=128 + LINE_SHIFT,
hpp_break_line=21 + LINE_SHIFT, lib_break_line=31 + LINE_SHIFT)
def _verify_cmake_preset(self, tempdir):
@@ -3239,7 +3239,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
self.assertIn("GNU gdb", r.output)
def _gdb_debug_cpp_example(self, magic_string, gdb_start_cmd="run",
- exe_break_line=63, exe_list_line=55, hpp_break_line=21,
+ exe_break_line=136, exe_list_line=128, hpp_break_line=21,
lib_break_line=31):
"""Get a series of gdb commands to debug the cpp-example-lib example"""
gdb_batch_cmd = " -ex 'break main' -ex '%s'" % gdb_start_cmd
@@ -3285,7 +3285,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
gdb_batch_cmd += " -ex 'continue'"
return gdb_batch_cmd
- def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=55, lib_break_line=31):
+ def _gdb_debug_cpp_example_check(self, gdb_output, magic_string, exe_list_line=128, lib_break_line=31):
self.assertIn("Breakpoint 1, main", gdb_output)
self.assertIn("$1 = 0", gdb_output) # test.string.compare equal
self.assertIn("$2 = -3", gdb_output) # test.string.compare longer
@@ -3313,7 +3313,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
self.assertIn("exited normally", gdb_output)
def _gdb_cross_debugging_multi(self, qemu, recipe_name, example_exe, magic_string,
- exe_break_line=63, exe_list_line=55, hpp_break_line=21,
+ exe_break_line=136, exe_list_line=128, hpp_break_line=21,
lib_break_line=31):
"""Verify gdb-cross is working
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 05/25] devtool: ide-sdk: auto-write image debug settings to bbappend
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (3 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 04/25] cpp-example: fix stuck breakpoints when attaching and daemonize properly AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 06/25] oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug settings AdrianF
` (19 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Simplify devtool ide-sdk usage by automatically generating the image
modifications required for debugging, instead of requiring manual
local.conf edits.
Generate the needed IMAGE_* and debug-server settings in the workspace
image bbappend based on the selected recipes and debugger. Add only
settings that are still missing, and remove the generated section
again when it becomes unnecessary, so configuration stays minimal and
up to date.
This was not done before because rewriting image metadata after parse
caused basehash inconsistency warnings. That is now handled by
controlling the parse flow and allowing the required additional parse
round(s) after bbappend updates.
Build in two phases so debugger ports are known before the image build,
and handle bbappend updates in a way that avoids basehash
non-determinism warnings.
This allows to simplify the documentation. We can remove the instructions
for manually configuring IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS,
IMAGE_CLASSES as well as appends to IMAGE_INSTALL in local.conf. That
just happens automatically now when running devtool ide-sdk.
Support also bbappend files with version in the file name like for
example: foo_1.2.bbappend or foo_git.bbappend.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_sdk.py | 251 +++++++++++++++++++++++++++------
1 file changed, 206 insertions(+), 45 deletions(-)
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index d7575ec523..b62a0f612e 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -185,32 +185,79 @@ class RecipeImage:
rootfs must be created as part of the SDK.
"""
- def __init__(self, name):
- self.combine_dbg_image = False
- self.gdbserver_missing = False
+ MARKER = '# devtool ide-sdk: image debug settings'
+
+ def __init__(self, name, orig_bbappend_content=None):
self.name = name
self.rootfs = None
self.__rootfs_dbg = None
self.bootstrap_tasks = [self.name + ':do_build']
+ # Debug settings already provided by the base configuration (e.g.
+ # local.conf, MACHINE, DISTRO, the recipe itself) plus any bbappend
+ # content other than devtool ide-sdk's own section (see
+ # strip_bbappend_sections()). Populated by initialize().
+ self.base_image_gen_debugfs = False
+ self.base_image_fstypes_debugfs = ''
+ self.base_has_combined_dbg = False
+ self.base_image_install = set()
+ self._bbappend = None
+ # Content of the bbappend before strip_bbappend_sections() ran.
+ self._orig_bbappend_content = orig_bbappend_content
+
+ @classmethod
+ def _strip_marker_section(cls, content):
+ """Remove devtool ide-sdk's own image debug settings section, if any"""
+ return re.sub(
+ r'^' + re.escape(cls.MARKER) + r'\n(?:[^\n]+\n)*',
+ '', content, flags=re.MULTILINE)
+
+ @classmethod
+ def strip_bbappend_sections(cls, config, recipe_names):
+ """Remove devtool ide-sdk's own bbappend section from earlier runs
+
+ Returns a {recipe name: content before stripping} dict for the recipes
+ that have a bbappend, to be passed on to the RecipeImage constructor.
+ """
+ originals = {}
+ appends_dir = os.path.join(config.workspace_path, 'appends')
+ for name in recipe_names:
+ bbappend = os.path.join(appends_dir, name + '.bbappend')
+ if not os.path.exists(bbappend):
+ continue
+ with open(bbappend, 'r') as f:
+ content = f.read()
+ originals[name] = content
+ stripped = cls._strip_marker_section(content)
+ if stripped != content:
+ with open(bbappend, 'w') as f:
+ f.write(stripped)
+ return originals
def initialize(self, config, tinfoil):
+ appends_dir = os.path.join(config.workspace_path, 'appends')
+ self._bbappend = os.path.join(appends_dir, self.name + '.bbappend')
+
+ # strip_bbappend_sections() ran before the tinfoil session started, so
+ # this parse sees the same bbappend content as any other parse of this
+ # recipe until the tinfoil session ends.
image_d = parse_recipe(
config, tinfoil, self.name, appends=True, filter_workspace=False)
if not image_d:
raise DevtoolError(
"Parsing image recipe %s failed" % self.name)
- self.combine_dbg_image = bb.data.inherits_class(
+ self.base_image_gen_debugfs = image_d.getVar(
+ 'IMAGE_GEN_DEBUGFS') == '1'
+ self.base_image_fstypes_debugfs = image_d.getVar(
+ 'IMAGE_FSTYPES_DEBUGFS') or ''
+ self.base_has_combined_dbg = bb.data.inherits_class(
'image-combined-dbg', image_d)
+ self.base_image_install = set(
+ (image_d.getVar('IMAGE_INSTALL') or '').split())
workdir = image_d.getVar('WORKDIR')
self.rootfs = os.path.join(workdir, 'rootfs')
- if image_d.getVar('IMAGE_GEN_DEBUGFS') == "1":
- self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg')
-
- package_install = image_d.getVar('PACKAGE_INSTALL').split()
- self.gdbserver_missing = 'gdbserver' not in package_install
- self.lldb_server_missing = 'lldb-server' not in package_install
+ self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg')
@property
def debug_support(self):
@@ -222,6 +269,76 @@ class RecipeImage:
return self.__rootfs_dbg
return None
+ def update_image_bbappend(self, recipes_modified):
+ """Write debug settings for modified-mode recipes into the image bbappend.
+
+ Writes IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS, IMAGE_CLASSES for
+ image-combined-dbg, the appropriate debug server (gdbserver or
+ lldb-server), and IMAGE_INSTALL entries for each modified recipe
+ (including the ptest package when the recipe inherits ptest).
+
+ initialize() already stripped this section from the bbappend on
+ disk before parsing, so it only needs to be added back here, if
+ still needed. Returns True if the resulting bbappend content
+ actually differs from what was on disk when initialize() ran, False
+ if it is left exactly as it was.
+ """
+ wants_gdbserver = any(
+ r.wants_gdbserver and r.toolchain != 'clang'
+ for r in recipes_modified)
+ wants_lldb_server = any(
+ r.wants_gdbserver and r.toolchain == 'clang'
+ for r in recipes_modified)
+
+ # Only add what the base configuration (e.g. local.conf) does not
+ # already provide, to avoid duplicate/conflicting settings.
+ lines = []
+ if not self.base_image_gen_debugfs:
+ lines.append('IMAGE_GEN_DEBUGFS = "1"')
+ if self.base_image_fstypes_debugfs != '':
+ lines.append('IMAGE_FSTYPES_DEBUGFS = ""')
+ if not self.base_has_combined_dbg:
+ lines.append('IMAGE_CLASSES += "image-combined-dbg"')
+ if wants_gdbserver and 'gdbserver' not in self.base_image_install:
+ lines.append('IMAGE_INSTALL:append = " gdbserver"')
+ if wants_lldb_server and 'lldb-server' not in self.base_image_install:
+ lines.append('IMAGE_INSTALL:append = " lldb-server"')
+ for r in recipes_modified:
+ if r.name not in self.base_image_install:
+ lines.append('IMAGE_INSTALL:append = " %s"' % r.name)
+ if r.has_ptest and (r.name + '-ptest') not in self.base_image_install:
+ lines.append('IMAGE_INSTALL:append = " %s-ptest"' % r.name)
+
+ original_content = self._orig_bbappend_content or ''
+ # strip_bbappend_sections() left this on disk, and it is what bitbake
+ # parsed. Any difference from it invalidates the parsed basehashes.
+ if os.path.exists(self._bbappend):
+ with open(self._bbappend, 'r') as f:
+ parsed_content = f.read()
+ else:
+ parsed_content = ''
+
+ if not lines:
+ # The base configuration already provides everything needed.
+ if parsed_content != original_content:
+ logger.info(
+ "Removed image debug settings from %s: already provided by the base configuration", self._bbappend)
+ return False
+
+ new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n'
+ new_content = parsed_content
+ if new_content and not new_content.endswith('\n'):
+ new_content += '\n'
+ new_content += new_section
+
+ appends_dir = os.path.dirname(self._bbappend)
+ os.makedirs(appends_dir, exist_ok=True)
+ with open(self._bbappend, 'w') as f:
+ f.write(new_content)
+
+ logger.info("Updated image bbappend %s", self._bbappend)
+ return True
+
class RecipeMetaIdeSupport:
"""For the shared sysroots mode meta-ide-support is needed
@@ -475,6 +592,8 @@ class RecipeModified:
# Whether to warn when DEBUG_BUILD is not set. Kernel modules are built
# by the kernel's build system and DEBUG_BUILD does not influence them.
self.wants_debug_build = True
+ # Whether this recipe provides a ptest package
+ self.has_ptest = False
# build_tool = cmake
self.oecmake_generator = None
self.cmake_cache_vars = None
@@ -592,6 +711,8 @@ class RecipeModified:
self.reverse_debug_prefix_map = self._init_reverse_debug_prefix_map(
recipe_d.getVar('DEBUG_PREFIX_MAP'))
+ self.has_ptest = bb.data.inherits_class('ptest', recipe_d)
+
# Recipe ID is the identifier for IDE config sections
self.recipe_id = self.bpn + "-" + self.package_arch
self.recipe_id_pretty = self.bpn + ": " + self.package_arch
@@ -1235,9 +1356,21 @@ def ide_setup(args, config, basepath, workspace):
if recipe in ['meta-ide-support', 'build-sysroots']:
raise DevtoolError("Invalid recipe: %s." % recipe)
- # Collect information about tasks which need to be bitbaked
+ # Collect information about tasks which need to be bitbaked.
+ # In modified mode the image build is held back until after
+ # setup_modified_recipe() has assigned the debugger port numbers and
+ # update_image_bbappend() has written the complete bbappend. That way
+ # the image is built with a single, stable recipe hash so that no
+ # basehash-changed warnings are emitted.
bootstrap_tasks = []
bootstrap_tasks_late = []
+ image_bootstrap_tasks = []
+ # Must happen before setup_tinfoil() so that every parse in this session
+ # sees the same bbappend content. Which of the recipes is the image is only
+ # known after parsing, so this covers all of them; for a recipe without an
+ # ide-sdk section it is a no-op.
+ orig_bbappend_contents = RecipeImage.strip_bbappend_sections(
+ config, args.recipenames)
tinfoil = setup_tinfoil(config_only=False, basepath=basepath)
try:
# define mode depending on recipes which need to be processed
@@ -1269,8 +1402,9 @@ def ide_setup(args, config, basepath, workspace):
if not recipes_modified_names:
appends_dir = os.path.join(config.workspace_path, 'appends')
recipes_modified_names = sorted(
- os.path.splitext(os.path.basename(path))[0]
- for path in glob.glob(os.path.join(appends_dir, '*.bbappend')))
+ bb.parse.vars_from_file(path, None)[0]
+ for path in glob.glob(os.path.join(appends_dir, '*.bbappend'))
+ if bb.parse.vars_from_file(path, None)[0] not in recipes_image_names)
if recipes_modified_names:
logger.info(
"No modified recipes specified, using workspace bbappends from %s: %s",
@@ -1302,9 +1436,17 @@ def ide_setup(args, config, basepath, workspace):
recipes_images = []
for recipes_image_name in recipes_image_names:
logger.info("Using image: %s" % recipes_image_name)
- recipe_image = RecipeImage(recipes_image_name)
+ recipe_image = RecipeImage(
+ recipes_image_name,
+ orig_bbappend_contents.get(recipes_image_name))
recipe_image.initialize(config, tinfoil)
- bootstrap_tasks += recipe_image.bootstrap_tasks
+ if args.mode == DevtoolIdeMode.modified:
+ # Keep the image build separate so that the complete bbappend
+ # can be written in one step before the image is built,
+ # avoiding sstate hash mismatches.
+ image_bootstrap_tasks += recipe_image.bootstrap_tasks
+ else:
+ bootstrap_tasks += recipe_image.bootstrap_tasks
recipes_images.append(recipe_image)
# Provide a Direct SDK with shared sysroots
@@ -1357,37 +1499,22 @@ def ide_setup(args, config, basepath, workspace):
finally:
tinfoil.shutdown()
+ bb_cmd = 'bitbake '
+ if args.bitbake_k:
+ bb_cmd += "-k "
+
if not args.skip_bitbake:
- bb_cmd = 'bitbake '
- if args.bitbake_k:
- bb_cmd += "-k "
- bb_cmd_early = bb_cmd + ' '.join(bootstrap_tasks)
- exec_build_env_command(
- config.init_path, basepath, bb_cmd_early, watch=True)
- if bootstrap_tasks_late:
- bb_cmd_late = bb_cmd + ' '.join(bootstrap_tasks_late)
+ # Phase 1: build modified recipes and debug tools so that
+ # installed_binaries is populated and port numbers can be assigned.
+ # The image is built in phase 2, after the complete bbappend is written.
+ if bootstrap_tasks:
exec_build_env_command(
- config.init_path, basepath, bb_cmd_late, watch=True)
-
- wants_gdbserver = any(
- r.wants_gdbserver and r.toolchain == 'gcc'
- for r in recipes_modified)
- wants_lldb_server = any(
- r.wants_gdbserver and r.toolchain == 'clang'
- for r in recipes_modified)
- for recipe_image in recipes_images:
- if wants_gdbserver and recipe_image.gdbserver_missing:
- logger.warning(
- "gdbserver not installed in image %s. Remote debugging will not be available" % recipe_image)
- if wants_lldb_server and recipe_image.lldb_server_missing:
- logger.warning(
- "lldb-server not installed in image %s. "
- "Remote debugging with LLDB (CodeLLDB) will not be available. "
- "Add 'lldb-server' to IMAGE_INSTALL." % recipe_image)
-
- if (wants_gdbserver or wants_lldb_server) and recipe_image.combine_dbg_image is False:
- logger.warning(
- 'IMAGE_CLASSES += "image-combined-dbg" is missing for image %s. Remote debugging will not find debug symbols from rootfs-dbg.' % recipe_image)
+ config.init_path, basepath,
+ bb_cmd + ' '.join(bootstrap_tasks), watch=True)
+ if bootstrap_tasks_late:
+ exec_build_env_command(
+ config.init_path, basepath,
+ bb_cmd + ' '.join(bootstrap_tasks_late), watch=True)
# Instantiate the active IDE plugin
ide = ide_plugins[args.ide]()
@@ -1408,6 +1535,39 @@ def ide_setup(args, config, basepath, workspace):
'You might want to add DEBUG_BUILD = "1" to %s. '
'Note that devtool modify --debug-build can do this automatically.',
recipe_modified.name, recipe_modified.bbappend)
+
+ # Ports are now assigned. Write the complete image bbappend in a
+ # single step so that the image is built with exactly one recipe
+ # hash. This avoids the sstate basehash-changed warnings that
+ # arise when the bbappend is modified after the image has
+ # already been built. This also runs with --skip-bitbake, otherwise
+ # the section removed by strip_bbappend_sections() would be lost.
+ bbappend_changed = False
+ for ri in recipes_images:
+ if ri.update_image_bbappend(recipes_modified):
+ bbappend_changed = True
+
+ if not args.skip_bitbake:
+ if image_bootstrap_tasks:
+ if bbappend_changed:
+ # The bbappend content just written differs from the one
+ # bitbake parsed during the tinfoil session above. With a
+ # memory resident server that session's basehashes are
+ # still cached, so reparsing would report "basehash value
+ # changed ... not deterministic" for every task. Tell
+ # bitbake the recipe intentionally changed by resetting the
+ # cooker: this clears its basehash history.
+ reparse_tinfoil = setup_tinfoil(config_only=True, basepath=basepath)
+ try:
+ reparse_tinfoil.run_command('resetCooker')
+ reparse_tinfoil.parse_recipes()
+ finally:
+ reparse_tinfoil.shutdown()
+
+ # Phase 2: build the image
+ exec_build_env_command(
+ config.init_path, basepath,
+ bb_cmd + ' '.join(image_bootstrap_tasks), watch=True)
else:
raise DevtoolError("Must not end up here.")
@@ -1483,7 +1643,8 @@ def register_commands(subparsers, context):
parser_ide_sdk.add_argument(
'-I', '--key', help='Specify ssh private key for connection to the target')
parser_ide_sdk.add_argument(
- '--skip-bitbake', help='Generate IDE configuration but skip calling bitbake to update the SDK', action='store_true')
+ '--skip-bitbake', help='Skip the bitbake builds which update the SDK. The recipes are still parsed, '
+ 'the IDE configuration is generated from their metadata', action='store_true')
parser_ide_sdk.add_argument(
'-k', '--bitbake-k', help='Pass -k parameter to bitbake', action='store_true')
parser_ide_sdk.add_argument(
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 06/25] oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug settings
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (4 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 05/25] devtool: ide-sdk: auto-write image debug settings to bbappend AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 07/25] oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams AdrianF
` (18 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
devtool ide-sdk now writes IMAGE_GEN_DEBUGFS, IMAGE_CLASSES +=
"image-combined-dbg" and the required IMAGE_INSTALL entries
(gdbserver/lldb-server, per-recipe ptest packages) to the image
workspace bbappend automatically. _write_bb_config() therefore no
longer sets these in local.conf, so tests also exercise that ide-sdk
does not need help from local.conf to enable image debug support.
Tests that use runqemu now call devtool ide-sdk once before starting
QEMU to build the image with the debug settings already applied, and
again with --skip-bitbake -t root@<ip> once the real target IP is
known, to update the deploy scripts without triggering a redundant
image rebuild.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 111 +++++++++++++-----------
1 file changed, 61 insertions(+), 50 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 87efa1ee80..1ecf4c24f5 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -2889,15 +2889,15 @@ class DevtoolIdeSdkTests(DevtoolBase):
if self.logger.isEnabledFor(logging.DEBUG):
self._cmd_logger = self.logger
- def _write_bb_config(self, recipe_names, extra_packages=None):
- """Helper to write the bitbake local.conf file"""
- image_install = 'gdbserver ' + ' '.join([r + '-ptest' for r in recipe_names])
- if extra_packages:
- image_install += ' ' + ' '.join(extra_packages)
+ def _write_bb_config(self):
+ """Helper to write the bitbake local.conf file.
+
+ Image debug settings (IMAGE_GEN_DEBUGFS, IMAGE_CLASSES += image-combined-dbg,
+ IMAGE_INSTALL for gdbserver/lldb-server and the recipe under test) are written
+ automatically by devtool ide-sdk to the image workspace bbappend.
+ Only settings not managed by ide_sdk are configured here.
+ """
conf_lines = [
- 'IMAGE_CLASSES += "image-combined-dbg"',
- 'IMAGE_GEN_DEBUGFS = "1"',
- 'IMAGE_INSTALL:append = " %s"' % image_install,
'DISTRO_FEATURES:append = " ptest"',
# Static UIDs/GIDs are required so that files installed via
# "install -o ${BPN}" in do_install embed the same UID that gets
@@ -3374,70 +3374,67 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
testimage = "oe-selftest-image"
self._check_workspace()
- self._write_bb_config(recipe_names)
+ self._write_bb_config()
self._check_runqemu_prerequisites()
# Verify deployment to Qemu (system mode) works
- bitbake(testimage)
+ tempdir_cmake = self._devtool_ide_sdk_recipe("cmake-example", "CMakeLists.txt", None)
+ tempdir_meson = self._devtool_ide_sdk_recipe("meson-example", "meson.build", testimage)
+ runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none' % testimage,
+ output_log=self._cmd_logger)
+
with runqemu(testimage, runqemuparams="nographic") as qemu:
# cmake-example recipe
recipe_name = "cmake-example"
example_exe = "cmake-example"
example_user_group = "cmake-example"
conf_file = "/etc/cmake-example.conf"
- build_file = "CMakeLists.txt"
# Verify the cmake-example service is running on the target
self._verify_service_running(qemu, example_exe)
# Verify /etc/cmake-example.conf is owned by the cmake-example user
self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group)
- # Setup the recipe with devtool ide-sdk cmake-example ...
- tempdir = self._devtool_ide_sdk_recipe(
- recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % (
+ # Re-run ide-sdk with the actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._gdb_cross()
- compile_cmd = self._verify_cmake_preset(tempdir)
- self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name, example_exe, compile_cmd)
+ compile_cmd = self._verify_cmake_preset(tempdir_cmake)
+ self._devtool_ide_sdk_qemu(tempdir_cmake, qemu, recipe_name, example_exe, compile_cmd)
# Verify the oe-scripts sym-link is valid
self.assertEqual(self._workspace_scripts_dir(
- recipe_name), self._sources_scripts_dir(tempdir))
+ recipe_name), self._sources_scripts_dir(tempdir_cmake))
# Verify /etc/cmake-example.conf is still owned by the cmake-example user
# after the install and deploy scripts updated the file
self._verify_conf_file(qemu, conf_file, example_exe, example_exe)
-
# meson-example recipe
recipe_name = "meson-example"
example_exe = "mesonex"
example_user_group = "meson-example"
conf_file = "/etc/meson-example.conf"
- build_file = "meson.build"
# Verify the meson-example service is running on the target
self._verify_service_running(qemu, example_exe)
# Verify /etc/meson-example.conf is owned by the meson-example user
self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group)
- # Setup the recipe with devtool ide-sdk meson-example ...
- tempdir = self._devtool_ide_sdk_recipe(
- recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % (
+ # Re-run ide-sdk with the actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._gdb_cross()
- compile_cmd = self._verify_meson_build(tempdir, recipe_name)
- self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name, example_exe, compile_cmd)
+ compile_cmd = self._verify_meson_build(tempdir_meson, recipe_name)
+ self._devtool_ide_sdk_qemu(tempdir_meson, qemu, recipe_name, example_exe, compile_cmd)
# Verify the oe-scripts sym-link is valid
self.assertEqual(self._workspace_scripts_dir(
- recipe_name), self._sources_scripts_dir(tempdir))
+ recipe_name), self._sources_scripts_dir(tempdir_meson))
# Verify /etc/meson-example.conf is still owned by the meson-example user
# after the install and deploy scripts updated the file
@@ -3784,15 +3781,17 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
build_file = "CMakeLists.txt"
self._check_workspace()
- self._write_bb_config([recipe_name])
+ self._write_bb_config()
- # Verify deployment to Qemu (system mode) works
+ # Build image with debug settings before starting QEMU
self._check_runqemu_prerequisites()
- bitbake(testimage)
+ tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
with runqemu(testimage, runqemuparams="nographic") as qemu:
- tempdir = self._devtool_ide_sdk_recipe(
- recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % (
+ # Re-run with actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._verify_cmake_preset(tempdir)
@@ -3814,15 +3813,18 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
testimage = "oe-selftest-image"
self._check_workspace()
- self._write_bb_config([recipe_name])
+ self._write_bb_config()
- # Verify deployment to Qemu (system mode) works
+ # Build image with debug settings before starting QEMU
self._check_runqemu_prerequisites()
- bitbake(testimage)
+ tempdir = self._devtool_ide_sdk_recipe(
+ recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
with runqemu(testimage, runqemuparams="nographic") as qemu:
- tempdir = self._devtool_ide_sdk_recipe(
- recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % (
+ # Re-run with actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
@@ -4507,13 +4509,17 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests):
testimage = "oe-selftest-image"
self._check_workspace()
- self._write_bb_config([recipe_name], extra_packages=['lldb-server'])
+ self._write_bb_config()
+ # Build image with debug settings (lldb-server for clang) before starting QEMU
self._check_runqemu_prerequisites()
- bitbake(testimage)
+ tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
with runqemu(testimage, runqemuparams="nographic") as qemu:
- tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=code' % (
+ # Re-run with actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
@@ -4566,7 +4572,7 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests):
testimage = "oe-selftest-image"
self._check_workspace()
- self._write_bb_config([recipe_name], extra_packages=['lldb-server'])
+ self._write_bb_config()
# Build image with debug settings (lldb-server for clang) before starting QEMU
self._check_runqemu_prerequisites()
@@ -4678,12 +4684,17 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests):
testimage = 'oe-selftest-image'
self._check_workspace()
- self._write_bb_config([recipe_name], extra_packages=['lldb-server'])
+ self._write_bb_config()
self._check_runqemu_prerequisites()
- bitbake(testimage)
+
+ # Build image with debug settings (lldb-server for clang) before starting QEMU
+ tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=none' % (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
with runqemu(testimage, runqemuparams='nographic') as qemu:
- tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --ide=none' % (
+ # Re-run with actual QEMU IP; image is already built
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % (
recipe_name, testimage, qemu.ip)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
@@ -4742,7 +4753,7 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests):
testimage = 'oe-selftest-image'
self._check_workspace()
- self._write_bb_config([recipe_name], extra_packages=['lldb-server'])
+ self._write_bb_config()
self._check_runqemu_prerequisites()
# Build image with debug settings (lldb-server for clang) before starting QEMU
@@ -4803,7 +4814,7 @@ class DevtoolIdeSdkMiscTests(DevtoolIdeSdkTests):
shared_recipe_name = "cmake-example"
self._check_workspace()
- self._write_bb_config([modified_recipe_name])
+ self._write_bb_config()
tempdir = self._devtool_ide_sdk_recipe(
modified_recipe_name, modified_build_file, None)
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 07/25] oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (5 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 06/25] oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug settings AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 08/25] devtool: ide-sdk: support runqemu slirp AdrianF
` (17 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
QemuRunner needs use_slirp=True to parse the guest IP address from the
hostfwd=tcp: entry in the QEMU command line rather than from the TAP-style
ip= kernel argument. QemuRunner is constructed before the runqemuparams
are known, so use_slirp defaults to False.
Set self.runner.use_slirp = True in QemuTarget.start() when 'slirp'
appears in runqemuparams, before calling the runner. This is the
earliest point at which runqemuparams is available and is the natural
place to propagate mode flags to the already-constructed runner.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/targetcontrol.py | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index fa6b43bb35..1deea2e5ce 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -24,6 +24,7 @@ class BaseTarget(object, metaclass=ABCMeta):
def __init__(self, d, logger):
self.connection = None
self.ip = None
+ self.port = None
self.server_ip = None
self.datetime = d.getVar('DATETIME')
self.testdir = d.getVar("TEST_LOG_DIR")
@@ -163,6 +164,8 @@ class QemuTarget(BaseTarget):
super(QemuTarget, self).deploy()
def start(self, params=None, ssh=True, extra_bootparams='', runqemuparams='', launch_cmd='', discard_writes=True):
+ if 'slirp' in runqemuparams.split():
+ self.runner.use_slirp = True
if launch_cmd:
start = self.runner.launch(get_ip=ssh, launch_cmd=launch_cmd, qemuparams=params)
else:
@@ -171,8 +174,16 @@ class QemuTarget(BaseTarget):
if start:
if ssh:
self.ip = self.runner.ip
+ self.port = None
+ if self.runner.use_slirp:
+ # In slirp mode the runner's ip is "host:port"
+ ip_port = self.runner.ip.split(':')
+ if len(ip_port) == 2:
+ self.ip, self.port = ip_port
+ else:
+ self.logger.error("Could not get host machine port to connect qemu with slirp, ssh will not be able to connect to qemu with slirp")
self.server_ip = self.runner.server_ip
- self.connection = SSHControl(ip=self.ip, logfile=self.sshlog)
+ self.connection = SSHControl(ip=self.ip, logfile=self.sshlog, port=self.port)
else:
self.stop()
if os.path.exists(self.qemulog):
@@ -192,13 +203,21 @@ class QemuTarget(BaseTarget):
self.loggerhandler.close()
self.connection = None
self.ip = None
+ self.port = None
self.server_ip = None
def restart(self, params=None):
if self.runner.restart(params):
self.ip = self.runner.ip
+ self.port = None
+ if self.runner.use_slirp:
+ ip_port = self.runner.ip.split(':')
+ if len(ip_port) == 2:
+ self.ip, self.port = ip_port
+ else:
+ self.logger.error("Could not get host machine port to connect qemu with slirp, ssh will not be able to connect to qemu with slirp")
self.server_ip = self.runner.server_ip
- self.connection = SSHControl(ip=self.ip, logfile=self.sshlog)
+ self.connection = SSHControl(ip=self.ip, logfile=self.sshlog, port=self.port)
else:
raise RuntimeError("%s - FAILED to re-start qemu - check the task log and the boot log" % self.pn)
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 08/25] devtool: ide-sdk: support runqemu slirp
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (6 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 07/25] oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 09/25] devtool: ide-sdk: auto-disable ssh host key checking for loopback targets AdrianF
` (16 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
ide-sdk needs to make sure the debugger server ports are reachable, so
when runqemu uses slirp networking, add the required port forwards to
QB_SLIRP_OPT in the image bbappend.
Example:
devtool ide-sdk oe-selftest-image -t root@localhost -P 2222 -c
runqemu slirp serialstdio snapshot
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_plugins/__init__.py | 15 ++-
scripts/lib/devtool/ide_plugins/ide_none.py | 6 +-
scripts/lib/devtool/ide_sdk.py | 133 +++++++++++++++-----
3 files changed, 121 insertions(+), 33 deletions(-)
diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py
index 136ddd8914..a5069cae85 100644
--- a/scripts/lib/devtool/ide_plugins/__init__.py
+++ b/scripts/lib/devtool/ide_plugins/__init__.py
@@ -65,6 +65,10 @@ class DebuggerCrossConfig:
DebuggerCrossConfig._port_next += 1
self.debug_server_port = self.debug_server_ports[self.default_mode]
self.id_pretty = "%d_%s" % (self.debug_server_port, self.binary_pretty)
+ # Hook for subclasses needing additional fixed ports forwarded through
+ # slirp beyond the one-per-mode debug_server_ports (e.g. lldb-server's
+ # spawned gdbserver instances).
+ self.extra_ports = []
if self.id_pretty in DebuggerCrossConfig._configs:
raise DevtoolError(
@@ -269,6 +273,13 @@ class LldbServerConfig(DebuggerCrossConfig):
default_mode=DebuggerServerModes.MULTI):
super().__init__(image_recipe, modified_recipe, binary,
default_mode)
+ # lldb-server platform spawns a separate gdb-remote-protocol
+ # "gdbserver" instance per debug session; without --gdbserver-port it
+ # picks a random port, which cannot be forwarded through slirp NAT.
+ # Pin it to a fixed, dedicated port that gets slirp-forwarded too.
+ self.gdbserver_port = DebuggerCrossConfig._port_next
+ DebuggerCrossConfig._port_next += 1
+ self.extra_ports.append(self.gdbserver_port)
def _lldb_server_tmp_dir(self, mode):
return os.path.join('/tmp', 'lldb_server_%s' % self.id_pretty_mode(mode))
@@ -294,8 +305,8 @@ class LldbServerConfig(DebuggerCrossConfig):
cmd = self._target_tcp_port_check_cmd() + " && exit 0; "
cmd += "mkdir -p %s; " % tmp_dir
cmd += "cd %s; " % tmp_dir
- cmd += "%s platform --server --listen *:%s > %s 2>&1 & _lldb_server_pid=\\$!; " % (
- lldb_server, self.debug_server_port, log_file)
+ cmd += "%s platform --server --listen *:%s --gdbserver-port %s > %s 2>&1 & _lldb_server_pid=\\$!; " % (
+ lldb_server, self.debug_server_port, self.gdbserver_port, log_file)
cmd += "echo \\$_lldb_server_pid > %s; " % pid_file
cmd += self._target_wait_for_tcp_port_cmd(
"lldb_server_pid", log_file)
diff --git a/scripts/lib/devtool/ide_plugins/ide_none.py b/scripts/lib/devtool/ide_plugins/ide_none.py
index 959140cedb..e4b255f2fa 100644
--- a/scripts/lib/devtool/ide_plugins/ide_none.py
+++ b/scripts/lib/devtool/ide_plugins/ide_none.py
@@ -207,10 +207,12 @@ class LldbServerConfigNone(LldbServerConfig):
lines = ['# This file is generated by devtool ide-sdk']
if server_mode == DebuggerServerModes.MULTI:
lines.append('# On the remote target:')
- lines.append('# lldb-server platform --server --listen *:%d' % self.debug_server_port)
+ lines.append('# lldb-server platform --server --listen *:%d --gdbserver-port %d' % (
+ self.debug_server_port, self.gdbserver_port))
else:
lines.append('# On the remote target:')
- lines.append('# lldb-server platform --one-shot --server --listen *:%d' % self.debug_server_port)
+ lines.append('# lldb-server platform --one-shot --server --listen *:%d --gdbserver-port %d' % (
+ self.debug_server_port, self.gdbserver_port))
lines.append('# On the build machine:')
lines.append('# cd ' + self.modified_recipe.real_srctree)
lines.append('# ' + self.debugger_cross.lldb + ' -s ' + self.lldbinit)
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index b62a0f612e..719648a3eb 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -23,7 +23,7 @@ import scriptutils
import bb
from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe
from devtool.standard import get_real_srctree
-from devtool.ide_plugins import BuildTool
+from devtool.ide_plugins import BuildTool, DebuggerCrossConfig
from oe.kernel_module import kernel_module_os_env
@@ -186,15 +186,17 @@ class RecipeImage:
"""
MARKER = '# devtool ide-sdk: image debug settings'
+ QB_SLIRP_MARKER = '# devtool ide-sdk: QB_SLIRP_OPT'
def __init__(self, name, orig_bbappend_content=None):
self.name = name
self.rootfs = None
self.__rootfs_dbg = None
+ self.qb_slirp_opt = ''
self.bootstrap_tasks = [self.name + ':do_build']
# Debug settings already provided by the base configuration (e.g.
# local.conf, MACHINE, DISTRO, the recipe itself) plus any bbappend
- # content other than devtool ide-sdk's own section (see
+ # content other than devtool ide-sdk's own sections (see
# strip_bbappend_sections()). Populated by initialize().
self.base_image_gen_debugfs = False
self.base_image_fstypes_debugfs = ''
@@ -204,11 +206,11 @@ class RecipeImage:
# Content of the bbappend before strip_bbappend_sections() ran.
self._orig_bbappend_content = orig_bbappend_content
- @classmethod
- def _strip_marker_section(cls, content):
- """Remove devtool ide-sdk's own image debug settings section, if any"""
+ @staticmethod
+ def _strip_marker_section(content, marker):
+ """Remove one devtool ide-sdk marker section, if present"""
return re.sub(
- r'^' + re.escape(cls.MARKER) + r'\n(?:[^\n]+\n)*',
+ r'^' + re.escape(marker) + r'\n(?:[^\n]+\n)*',
'', content, flags=re.MULTILINE)
@classmethod
@@ -227,7 +229,8 @@ class RecipeImage:
with open(bbappend, 'r') as f:
content = f.read()
originals[name] = content
- stripped = cls._strip_marker_section(content)
+ stripped = cls._strip_marker_section(content, cls.MARKER)
+ stripped = cls._strip_marker_section(stripped, cls.QB_SLIRP_MARKER)
if stripped != content:
with open(bbappend, 'w') as f:
f.write(stripped)
@@ -259,6 +262,8 @@ class RecipeImage:
self.rootfs = os.path.join(workdir, 'rootfs')
self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg')
+ self.qb_slirp_opt = image_d.getVar('QB_SLIRP_OPT') or ''
+
@property
def debug_support(self):
return bool(self.rootfs_dbg)
@@ -279,9 +284,10 @@ class RecipeImage:
initialize() already stripped this section from the bbappend on
disk before parsing, so it only needs to be added back here, if
- still needed. Returns True if the resulting bbappend content
- actually differs from what was on disk when initialize() ran, False
- if it is left exactly as it was.
+ still needed. Also updates QB_SLIRP_OPT with the debugger server
+ port forwards (see update_qb_slirp_opt()). Returns True if the
+ resulting bbappend content actually differs from what was on disk
+ when initialize() ran, False if it is left exactly as it was.
"""
wants_gdbserver = any(
r.wants_gdbserver and r.toolchain != 'clang'
@@ -319,24 +325,90 @@ class RecipeImage:
parsed_content = ''
if not lines:
- # The base configuration already provides everything needed.
- if parsed_content != original_content:
+ if self.MARKER in original_content:
logger.info(
"Removed image debug settings from %s: already provided by the base configuration", self._bbappend)
+ image_changed = False
+ else:
+ new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n'
+ new_content = parsed_content
+ if new_content and not new_content.endswith('\n'):
+ new_content += '\n'
+ new_content += new_section
+
+ appends_dir = os.path.dirname(self._bbappend)
+ os.makedirs(appends_dir, exist_ok=True)
+ with open(self._bbappend, 'w') as f:
+ f.write(new_content)
+ logger.info("Updated image bbappend %s", self._bbappend)
+ image_changed = True
+
+ slirp_changed = self.update_qb_slirp_opt()
+ return image_changed or slirp_changed
+
+ def update_qb_slirp_opt(self):
+ """Update QB_SLIRP_OPT in the image bbappend
+
+ Support connecting to a debugger server running on the target device via
+ runqemu's slirp network:
+ - If the base value is non-empty (recipe/machine sets QB_SLIRP_OPT):
+ only missing port forwards are appended via QB_SLIRP_OPT:append.
+ - If the base value is empty (runqemu would use its own built-in default
+ of SSH 2222, telnet 2323, tftp): a full QB_SLIRP_OPT assignment is
+ written that mirrors that default plus the debugger ports, so that
+ runqemu reads the complete set from the .qemuboot.conf.
+
+ Returns True if the bbappend content actually changed, False otherwise.
+ """
+ ports = sorted({port for cfg in DebuggerCrossConfig._configs.values()
+ for port in list(cfg.debug_server_ports.values()) + cfg.extra_ports})
+ if not ports:
return False
- new_section = self.MARKER + '\n' + '\n'.join(lines) + '\n'
- new_content = parsed_content
+ # Determine which ports are already in the base value
+ already = {int(m.group(1))
+ for m in re.finditer(r':(\d+)-:\d+', self.qb_slirp_opt)}
+ missing_ports = [p for p in ports if p not in already]
+ if not missing_ports:
+ logger.info("QB_SLIRP_OPT already contains all needed port forwards")
+ return False
+
+ if self.qb_slirp_opt:
+ # Base value exists: :append only the missing port forwards
+ extra = ''.join(
+ ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports)
+ new_line = 'QB_SLIRP_OPT:append = "%s"' % extra
+ else:
+ # No base value: mirror runqemu's built-in default (SSH 2222, telnet
+ # 2323, tftp) and add the debugger ports.
+ all_hostfwds = (
+ 'hostfwd=tcp:127.0.0.1:2222-:22,'
+ 'hostfwd=tcp:127.0.0.1:2323-:23'
+ )
+ all_hostfwds += ''.join(
+ ',hostfwd=tcp:127.0.0.1:%d-:%d' % (p, p) for p in missing_ports)
+ new_line = 'QB_SLIRP_OPT = "-netdev user,id=net0,%s,tftp=${DEPLOY_DIR_IMAGE}"' % all_hostfwds
+
+ if os.path.exists(self._bbappend):
+ with open(self._bbappend, 'r') as f:
+ content = f.read()
+ else:
+ content = ''
+ stripped_content = self._strip_marker_section(content, self.QB_SLIRP_MARKER)
+ new_content = stripped_content
if new_content and not new_content.endswith('\n'):
new_content += '\n'
- new_content += new_section
+ new_content += self.QB_SLIRP_MARKER + '\n' + new_line + '\n'
+
+ if new_content == content:
+ logger.debug("QB_SLIRP_OPT in %s is already up to date", self._bbappend)
+ return False
appends_dir = os.path.dirname(self._bbappend)
os.makedirs(appends_dir, exist_ok=True)
with open(self._bbappend, 'w') as f:
f.write(new_content)
-
- logger.info("Updated image bbappend %s", self._bbappend)
+ logger.info("Updated QB_SLIRP_OPT in %s: %s", self._bbappend, new_line)
return True
@@ -1359,9 +1431,9 @@ def ide_setup(args, config, basepath, workspace):
# Collect information about tasks which need to be bitbaked.
# In modified mode the image build is held back until after
# setup_modified_recipe() has assigned the debugger port numbers and
- # update_image_bbappend() has written the complete bbappend. That way
- # the image is built with a single, stable recipe hash so that no
- # basehash-changed warnings are emitted.
+ # update_image_bbappend() has written the complete bbappend (including
+ # QB_SLIRP_OPT). That way the image is built with a single, stable
+ # recipe hash so that no basehash-changed warnings are emitted.
bootstrap_tasks = []
bootstrap_tasks_late = []
image_bootstrap_tasks = []
@@ -1442,8 +1514,8 @@ def ide_setup(args, config, basepath, workspace):
recipe_image.initialize(config, tinfoil)
if args.mode == DevtoolIdeMode.modified:
# Keep the image build separate so that the complete bbappend
- # can be written in one step before the image is built,
- # avoiding sstate hash mismatches.
+ # (IMAGE_ vars + QB_SLIRP_OPT) can be written in one step
+ # before the image is built, avoiding sstate hash mismatches.
image_bootstrap_tasks += recipe_image.bootstrap_tasks
else:
bootstrap_tasks += recipe_image.bootstrap_tasks
@@ -1536,12 +1608,13 @@ def ide_setup(args, config, basepath, workspace):
'Note that devtool modify --debug-build can do this automatically.',
recipe_modified.name, recipe_modified.bbappend)
- # Ports are now assigned. Write the complete image bbappend in a
- # single step so that the image is built with exactly one recipe
- # hash. This avoids the sstate basehash-changed warnings that
- # arise when the bbappend is modified after the image has
- # already been built. This also runs with --skip-bitbake, otherwise
- # the section removed by strip_bbappend_sections() would be lost.
+ # Ports are now assigned. Write the complete image bbappend --
+ # IMAGE_ debug settings and QB_SLIRP_OPT -- in a single step so
+ # that the image is built with exactly one recipe hash. This
+ # avoids the sstate basehash-changed warnings that arise when
+ # the bbappend is modified after the image has already been
+ # built. This also runs with --skip-bitbake, otherwise the sections
+ # removed by strip_bbappend_sections() would be lost.
bbappend_changed = False
for ri in recipes_images:
if ri.update_image_bbappend(recipes_modified):
@@ -1564,7 +1637,9 @@ def ide_setup(args, config, basepath, workspace):
finally:
reparse_tinfoil.shutdown()
- # Phase 2: build the image
+ # Phase 2: build the image. do_image -> do_write_qemuboot_conf
+ # picks up QB_SLIRP_OPT from the bbappend written above, so no
+ # separate write_qemuboot_conf step is needed.
exec_build_env_command(
config.init_path, basepath,
bb_cmd + ' '.join(image_bootstrap_tasks), watch=True)
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 09/25] devtool: ide-sdk: auto-disable ssh host key checking for loopback targets
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (7 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 08/25] devtool: ide-sdk: support runqemu slirp AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 10/25] oe-selftest: devtool ide-sdk: add slirp networking test AdrianF
` (15 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
QEMU instances reached via slirp/hostfwd (e.g. root@localhost) get a new
ephemeral ssh host key on every boot, so StrictHostKeyChecking would
fail on the second and later runs unless --no-host-check is passed
explicitly. Detect loopback targets (localhost, 127.0.0.1, ::1) and
disable host key checking automatically, both in TargetDevice and in
the generated do_install-through-bitbake script that re-parses the
target args on the build host.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_sdk.py | 50 ++++++++++++++++++++++++++--------
1 file changed, 38 insertions(+), 12 deletions(-)
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 719648a3eb..e45752aff1 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -45,22 +45,24 @@ class DevtoolIdeMode(Enum):
shared = 'shared'
+# Hosts a ssh target is considered to loop back to the local machine, e.g. a
+# QEMU instance reached through slirp/hostfwd port forwarding (root@localhost)
+# which has an ephemeral ssh host key that changes on every boot.
+LOOPBACK_HOSTS = ('localhost', '127.0.0.1', '::1')
+
+
+def target_host(target):
+ return target.split('@')[-1]
+
+
+def is_loopback_target(target):
+ return target_host(target) in LOOPBACK_HOSTS
+
+
class TargetDevice:
"""SSH remote login parameters"""
def __init__(self, args):
- self.extraoptions = []
- if args.no_host_check:
- self.extraoptions += ['-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no']
- self.ssh_sshexec = 'ssh'
- if args.ssh_exec:
- self.ssh_sshexec = args.ssh_exec
- self.ssh_port = []
- if args.port:
- self.ssh_port = ['-p', args.port]
- if args.key:
- self.extraoptions += ['-i', args.key]
-
self.target = args.target
target_sp = args.target.split('@')
if len(target_sp) == 1:
@@ -72,6 +74,25 @@ class TargetDevice:
else:
logger.error("Invalid target argument: %s" % args.target)
+ no_host_check = args.no_host_check
+ if not no_host_check and is_loopback_target(args.target):
+ logger.debug(
+ "Target %s is a loopback address, disabling ssh host key checking "
+ "(assuming a QEMU instance with an ephemeral host key)." % args.target)
+ no_host_check = True
+
+ self.extraoptions = []
+ if no_host_check:
+ self.extraoptions += ['-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no']
+ self.ssh_sshexec = 'ssh'
+ if args.ssh_exec:
+ self.ssh_sshexec = args.ssh_exec
+ self.ssh_port = []
+ if args.port:
+ self.ssh_port = ['-p', args.port]
+ if args.key:
+ self.extraoptions += ['-i', args.key]
+
class RecipeNative:
"""Base class for calling bitbake to provide a -native recipe"""
@@ -1324,6 +1345,8 @@ class RecipeModified:
'no_preserve', 'port', 'show_status', 'ssh_exec', 'strip', 'target']
filtered_args_dict = {key: value for key, value in vars(
args).items() if key in args_filter}
+ if is_loopback_target(filtered_args_dict['target']):
+ filtered_args_dict['no_host_check'] = True
cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict))
cmd_lines.append('class Dict2Class(object):')
cmd_lines.append(' def __init__(self, my_dict):')
@@ -1340,6 +1363,9 @@ class RecipeModified:
cmd_lines.append(' i += 2')
cmd_lines.append(' else:')
cmd_lines.append(' i += 1')
+ cmd_lines.append(
+ "if filtered_args.target.split('@')[-1] in %s:" % str(LOOPBACK_HOSTS))
+ cmd_lines.append(' filtered_args.no_host_check = True')
cmd_lines.append(
'setattr(filtered_args, "recipename", "%s")' % self.bpn)
cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' %
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 10/25] oe-selftest: devtool ide-sdk: add slirp networking test
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (8 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 09/25] devtool: ide-sdk: auto-disable ssh host key checking for loopback targets AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters AdrianF
` (14 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Add test_devtool_ide_sdk_none_qemu_slirp which verifies the complete
devtool ide-sdk workflow using runqemu slirp networking instead of TAP
devices.
The test checks that:
- update_qb_slirp_opt() writes QB_SLIRP_OPT (including the SSH
hostfwd entry) to the image workspace bbappend before the build.
- The image boots in runqemu slirp mode (no TAP device setup needed).
- devtool deploy-target successfully reaches the target via the slirp
SSH port forward (localhost:2222 by default).
- The full edit/compile/deploy/verify cycle works over the slirp
connection, just as it does in the TAP-based tests.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 50 +++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 1ecf4c24f5..c18b94ab5a 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3864,6 +3864,56 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
# Verify deployment and remote debugging works
self._verify_launch_json_debugging(tempdir, qemu, example_exe)
+ @OETestTag("runqemu")
+ def test_devtool_ide_sdk_none_qemu_slirp(self):
+ """Verify devtool ide-sdk works with runqemu slirp networking.
+
+ Slirp mode uses SSH port forwarding (default: localhost:2222 -> guest:22).
+ This test checks that:
+ - update_qb_slirp_opt() writes the QB_SLIRP_OPT host-forwarding
+ entries to the image workspace bbappend before the build.
+ - runqemu boots the image in slirp mode.
+ - devtool deploy-target reaches the target via localhost:2222.
+ """
+ recipe_name = "cmake-example"
+ build_file = "CMakeLists.txt"
+ testimage = "oe-selftest-image"
+
+ self._check_workspace()
+ self._write_bb_config()
+
+ # devtool modify and build image; QB_SLIRP_OPT is written to the bbappend.
+ tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, None)
+ runCmd('devtool ide-sdk %s %s -c --ide=none' % (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
+ # Verify QB_SLIRP_OPT was written to the workspace bbappend.
+ appends_dir = os.path.join(self.workspacedir, 'appends')
+ bbappend = os.path.join(appends_dir, testimage + '.bbappend')
+ self.assertExists(bbappend, 'Image bbappend not created at %s' % bbappend)
+ with open(bbappend) as f:
+ bbappend_content = f.read()
+ self.assertIn('QB_SLIRP_OPT', bbappend_content,
+ 'QB_SLIRP_OPT not written to image bbappend')
+ self.assertIn('hostfwd=tcp:127.0.0.1:2222-:22', bbappend_content,
+ 'SSH slirp port forward missing from QB_SLIRP_OPT')
+
+ with runqemu(testimage, runqemuparams="nographic slirp") as qemu:
+ slirp_host = qemu.ip
+ slirp_port = qemu.port or '2222'
+
+ # Re-run ide-sdk with the actual slirp address; image is already built.
+ bitbake_sdk_cmd = (
+ 'devtool ide-sdk %s %s -t root@%s -P %s --skip-bitbake --ide=none'
+ % (recipe_name, testimage, slirp_host, slirp_port))
+ runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
+
+ self._gdb_cross()
+ compile_cmd = self._verify_cmake_preset(tempdir)
+ self._devtool_ide_sdk_qemu(tempdir, qemu, recipe_name,
+ recipe_name, compile_cmd)
+
+
class DevtoolIdeSdkKernelTests(DevtoolIdeSdkTests):
@OETestTag("runqemu")
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (9 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 10/25] oe-selftest: devtool ide-sdk: add slirp networking test AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 22:11 ` Patchtest results for " patchtest
2026-08-30 21:48 ` [PATCH v2 12/25] oe-selftest: devtool deploy-target: test " AdrianF
` (13 subsequent siblings)
24 siblings, 1 reply; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Allow deploying only a subset of a recipe's installed files instead of
always sending the whole do_install output. This can be crucial when a
recipe's PACKAGES include additional content (e.g. -doc, -staticdev, or
optional extras) that does not even fit into the target device's
storage/memory; it also speeds up deployment significantly by avoiding
the transfer of files that are not actually needed for testing on the
target.
- --package PACKAGE filters the deployed files down to those listed in
that package's FILES variable.
- --file-glob GLOB filters by an arbitrary glob pattern on the installed
path. --package and --file-glob combine as a union when both are given.
Examples:
devtool deploy-target mdadm root@target
deploy everything (minus -dbg/-src/-staticdev, see above)
devtool deploy-target mdadm root@target --package mdadm-doc
deploy only the mdadm-doc package's files
devtool deploy-target mdadm root@target --package mdadm,mdadm-doc
deploy the main and -doc packages together
devtool deploy-target mdadm root@target --file-glob '/usr/bin/*'
deploy only files under /usr/bin
devtool ide-sdk mdadm cmake-example core-image-minimal \
--package "mdadm:,-doc" --package "cmake-example:,-doc"
scope each --package entry to its own recipe when several
modified recipes are targeted at once
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/deploy.py | 161 ++++++++++++++++++++++++++++++++--
1 file changed, 155 insertions(+), 6 deletions(-)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index 35ec0b1b4d..267d124415 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -17,6 +17,7 @@ import tempfile
import bb.utils
import argparse_oe
import oe.types
+import oe.package
from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError
@@ -137,6 +138,92 @@ def _prepare_remote_script(deploy, destdir='/', verbose=False, dryrun=False, und
return '\n'.join(lines)
+def parse_file_globs_arg(entries, recipename=None):
+ """Flatten --file-glob entries ("[RECIPE:]GLOB") into glob patterns for recipename.
+
+ E.g. ["other:/usr/bin/*", "/etc/*"] with recipename="myrecipe" -> ["/etc/*"].
+ """
+ globs = []
+ for entry in entries or []:
+ recipe, spec = entry.split(':', 1) if ':' in entry else (None, entry)
+ if recipe is not None and recipename is not None and recipe != recipename:
+ continue
+ if spec:
+ globs.append(spec)
+ return globs
+
+def _match_deploy_files(recipe_outdir, file_globs, recipename=None):
+ """Match file_globs (FILES-variable-style patterns) against recipe_outdir.
+
+ Returns the matched files (relative to recipe_outdir), or None if no glob
+ applies to recipename.
+ """
+ file_globs = parse_file_globs_arg(file_globs, recipename)
+ if not file_globs:
+ return None
+ cwd = os.getcwd()
+ os.chdir(recipe_outdir)
+ try:
+ matched, _ = oe.package.files_from_filevars(file_globs)
+ finally:
+ os.chdir(cwd)
+ return {os.path.normpath(f) for f in matched}
+
+def parse_packages_arg(entries, recipename=None):
+ """Flatten --package entries ("[RECIPE:]PKG[,PKG...]") into package names.
+
+ E.g. with recipename="myrecipe": "myrecipe:,-doc,-ptest" ->
+ ["myrecipe", "myrecipe-doc", "myrecipe-ptest"]; "other:foo" -> [].
+ """
+ packages = []
+ for entry in entries or []:
+ recipe, spec = entry.split(':', 1) if ':' in entry else (None, entry)
+ if recipe is not None and recipename is not None and recipe != recipename:
+ continue
+ for item in spec.split(','):
+ if not item:
+ if recipe is not None:
+ packages.append(recipe)
+ elif item.startswith('-'):
+ if recipe is None:
+ raise DevtoolError('Package suffix "%s" requires a "RECIPE:" prefix, '
+ 'e.g. "RECIPE:%s"' % (item, item))
+ packages.append(recipe + item)
+ else:
+ packages.append(item)
+ return packages
+
+def is_default_excluded_package(pkg):
+ """True for packages left out of a deploy"""
+ return pkg.endswith(('-dbg', '-src', '-staticdev'))
+
+def _match_package_files(recipe_outdir, packages_files, packages, recipename=None):
+ """Resolve packages (see parse_packages_arg) into their files under recipe_outdir."""
+ packages = parse_packages_arg(packages, recipename)
+ all_package_names = [pkg for pkg, _ in packages_files]
+ for pkg in packages:
+ if pkg not in all_package_names:
+ raise DevtoolError('Package "%s" is not one of the packages produced '
+ 'by this recipe (PACKAGES: %s)' % (pkg, ' '.join(all_package_names)))
+ cwd = os.getcwd()
+ os.chdir(recipe_outdir)
+ try:
+ seen = set()
+ result = set() if packages else None
+ default_excluded = set()
+ for pkg, files_var in packages_files:
+ matched, _ = oe.package.files_from_filevars((files_var or '').split())
+ matched = {os.path.normpath(f) for f in matched} - seen
+ seen |= matched
+ if pkg in packages:
+ if result is not None:
+ result |= matched
+ elif is_default_excluded_package(pkg):
+ default_excluded |= matched
+ finally:
+ os.chdir(cwd)
+ return result, default_excluded
+
def deploy(args, config, basepath, workspace):
"""Entry point for the devtool 'deploy' subcommand"""
import oe.utils
@@ -160,14 +247,15 @@ def deploy(args, config, basepath, workspace):
max_process = oe.utils.get_bb_number_threads(rd)
fakerootcmd = rd.getVar('FAKEROOTCMD')
fakerootenv = rd.getVar('FAKEROOTENV')
+ packages_files = [(pkg, rd.getVar('FILES:' + pkg) or '')
+ for pkg in (rd.getVar('PACKAGES') or '').split()]
finally:
tinfoil.shutdown()
- return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args)
+ return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files)
-def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args):
+def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None):
import math
- import oe.package
try:
host, destdir = args.target.split(':')
@@ -208,11 +296,35 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
if ret != 0:
raise DevtoolError('Failed to strip files for deployment')
+ allowed_files = None
+ default_excluded_files = set()
+ file_sets = []
+ if file_globs:
+ deploy_files = _match_deploy_files(recipe_outdir, file_globs, getattr(args, 'recipename', None))
+ if deploy_files is not None:
+ file_sets.append(deploy_files)
+ if packages_files:
+ package_files, default_excluded_files = _match_package_files(recipe_outdir, packages_files, getattr(args, 'package', None),
+ getattr(args, 'recipename', None))
+ if package_files is not None:
+ file_sets.append(package_files)
+ if file_sets:
+ allowed_files = set().union(*file_sets)
+
filelist = []
+ tar_relpaths = []
inodes = set({})
ftotalsize = 0
for root, _, files in os.walk(recipe_outdir):
for fn in files:
+ relpath = os.path.normpath(os.path.join(os.path.relpath(root, recipe_outdir), fn))
+ if allowed_files is not None:
+ if relpath not in allowed_files:
+ continue
+ elif relpath in default_excluded_files:
+ # No explicit --package/--file-glob filter was given: still leave
+ # out packages like -staticdev that aren't needed on a live target.
+ continue
fstat = os.lstat(os.path.join(root, fn))
# Get the size in kiB (since we'll be comparing it to the output of du -k)
# MUST use lstat() here not stat() or getfilesize() since we don't want to
@@ -226,6 +338,12 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
# The path as it would appear on the target
fpath = os.path.join(destdir, os.path.relpath(root, recipe_outdir), fn)
filelist.append((fpath, fsize))
+ tar_relpaths.append(relpath)
+
+ if allowed_files is not None and not filelist:
+ raise DevtoolError('No files to deploy for %s - the --package/--file-glob '
+ 'filter(s) did not match any of the files installed by this '
+ 'recipe.' % args.recipename)
if args.dry_run:
print('Files to be deployed for %s on target %s:' % (args.recipename, args.target))
@@ -282,8 +400,28 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
finally:
shutil.rmtree(tmpdir)
- # Now run the script
- ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, 'tar cf - . | %s %s %s %s \'sh %s %s %s %s\'' % (ssh_sshexec, ssh_port, extraoptions, args.target, tmpscript, args.recipename, destdir, tmpfilelist), cwd=recipe_outdir, shell=True)
+ # Now run the script. When a package/glob filter narrowed down filelist,
+ # tar is given an explicit list of relative paths (-T) instead of packing
+ # the whole recipe_outdir tree.
+ tar_filelist_path = None
+ try:
+ if allowed_files is not None:
+ tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
+ with os.fdopen(tar_fd, 'w') as f:
+ for relpath in tar_relpaths:
+ # './' prefix matches what 'tar cf - .' itself would produce, which
+ # the remote script's manifest handling (sed "s!^./!$2!") relies on.
+ f.write('./' + relpath + '\n')
+ tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
+ else:
+ tar_cmd = 'tar cf - .'
+ remote_cmd = '%s | %s %s %s %s \'sh %s %s %s %s\'' % (
+ tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target,
+ tmpscript, args.recipename, destdir, tmpfilelist)
+ ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, remote_cmd, cwd=recipe_outdir, shell=True)
+ finally:
+ if tar_filelist_path:
+ os.remove(tar_filelist_path)
if ret != 0:
raise DevtoolError('Deploy failed - rerun with -s to get a complete '
'error message')
@@ -362,7 +500,7 @@ def register_commands(subparsers, context):
parser_deploy = subparsers.add_parser('deploy-target',
help='Deploy recipe output files to live target machine',
- description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand.',
+ description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand. Use --package/--file-glob to deploy only a subset of the recipe\'s installed files.',
group='testbuild')
parser_deploy.add_argument('recipename', help='Recipe to deploy')
parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]')
@@ -375,6 +513,17 @@ def register_commands(subparsers, context):
parser_deploy.add_argument('-P', '--port', help='Specify port to use for connection to the target')
parser_deploy.add_argument('-I', '--key',
help='Specify ssh private key for connection to the target')
+ parser_deploy.add_argument('--package', action='append', metavar='PACKAGE',
+ help='Only deploy files belonging to PACKAGE, as defined by that '
+ 'package\'s FILES variable in the recipe metadata. May be a '
+ 'comma-separated list and/or specified multiple times to '
+ 'include several packages. May be prefixed with "RECIPE:" (must '
+ 'match RECIPENAME), e.g. "RECIPE:,-doc,-ptest" is short for '
+ '"RECIPE,RECIPE-doc,RECIPE-ptest".')
+ parser_deploy.add_argument('--file-glob', action='append', dest='file_globs', metavar='GLOB',
+ help='Only deploy files whose installed path matches this glob '
+ 'pattern (e.g. "/usr/bin/*" or "${bindir}/myprog"). May be '
+ 'specified multiple times. Combined with --package if both are given.')
strip_opts = parser_deploy.add_mutually_exclusive_group(required=False)
strip_opts.add_argument('-S', '--strip',
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 12/25] oe-selftest: devtool deploy-target: test --package/--file-glob filters
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (10 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 13/25] devtool: ide-sdk: forward --package/--file-glob deploy filters AdrianF
` (12 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Extend test_devtool_deploy_target to validate --package/--file-glob
filter behavior for both default deploy and --strip deploy.
The test now covers package-only, glob-only, and combined filtering,
plus recipe-scoped "RECIPE:" forms and shorthand expansion.
Examples covered by the test:
- devtool deploy-target -c mdadm root@<ip> --package mdadm
- devtool deploy-target -c mdadm root@<ip> --package mdadm-doc
- devtool deploy-target -c mdadm root@<ip> --package mdadm,mdadm-doc
- devtool deploy-target -c mdadm root@<ip> --file-glob /usr/sbin/mdadm
- devtool deploy-target -c mdadm root@<ip> --package mdadm --file-glob /usr/sbin/mdadm
- devtool deploy-target -c mdadm root@<ip> --package mdadm:-doc
- devtool deploy-target -c mdadm root@<ip> --package mdadm:,-doc
- devtool deploy-target -c mdadm root@<ip> --file-glob mdadm:/usr/sbin/mdadm
This also verifies that non-matching "other-recipe:" prefixes are
ignored for the current recipe.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 135 +++++++++++++++++-------
scripts/lib/devtool/deploy.py | 10 +-
2 files changed, 108 insertions(+), 37 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index c18b94ab5a..b3eb3c28fb 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -2038,8 +2038,14 @@ class DevtoolDeployTargetTests(DevtoolBase):
# Definitions
testrecipe = 'mdadm'
testfile = '/sbin/mdadm'
+ # mdmon is installed by the same do_install, used to check --package/--file-glob filtering
+ otherfile = '/sbin/mdmon'
if "usrmerge" in get_bb_var('DISTRO_FEATURES'):
testfile = '/usr/sbin/mdadm'
+ otherfile = '/usr/sbin/mdmon'
+ # Use the mdadm-doc package for testing --package and --file-glob filtering
+ mandir = get_bb_var('mandir', testrecipe)
+ docfile = os.path.join(mandir, 'man8', '%s.8' % testrecipe)
testimage = 'oe-selftest-image'
testcommand = '/sbin/mdadm --help'
# Build an image to run
@@ -2066,48 +2072,105 @@ class DevtoolDeployTargetTests(DevtoolBase):
self.assertIn(' %s' % testfile, result.output)
# Boot the image
with runqemu(testimage) as qemu:
- # Now really test deploy-target
- for extra_opt in ['', '--strip']:
- deploy_cmd= 'devtool deploy-target -c %s root@%s %s' % (testrecipe, qemu.ip, extra_opt)
- self.logger.debug(deploy_cmd)
+ def _deploy_and_check(extra_args, check_full_filelist, expected_files):
+ """Deploy with extra_args, verify the result, then undeploy and verify removal.
+
+ check_full_filelist is only meaningful for an unfiltered deploy: it
+ compares every file installed by do_install against what actually
+ landed on the target. For a --package/--file-glob filtered deploy,
+ expected_files instead maps a handful of representative paths
+ (testfile, otherfile, docfile) to whether they should have been
+ deployed by this particular filter.
+ """
+ deploy_cmd = 'devtool deploy-target -c %s root@%s %s' % (testrecipe, qemu.ip, extra_args)
+ if self.logger.isEnabledFor(logging.DEBUG):
+ deploy_cmd += ' -s'
result = runCmd(deploy_cmd)
- # Run a test command to see if it was installed properly
- status, _ = qemu.run(testcommand)
- self.assertEqual(status, 0)
- # Check if it deployed all of the files with the right ownership/perms
- # First look on the host - need to do this under pseudo to get the correct ownership/perms
- bb_vars = get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD', 'PATH'], testrecipe)
- installdir = bb_vars['D']
- fakerootenv = bb_vars['FAKEROOTENV']
- fakerootcmd = bb_vars['FAKEROOTCMD']
- path = bb_vars['PATH']
- result = runCmd('PATH="%s" %s %s find . -type f -exec ls -l {} \\;' % (path, fakerootenv, fakerootcmd), cwd=installdir)
- filelist1 = self._process_ls_output(result.output)
+ self.logger.debug('%s\n%s' % (deploy_cmd, result.output))
+
+ if check_full_filelist:
+ # Run a test command to see if it was installed properly
+ status, _ = qemu.run(testcommand)
+ self.assertEqual(status, 0, '%s: %s was not deployed' % (extra_args, testfile))
+
+ # Check if it deployed all of the files with the right ownership/perms
+ # First look on the host - need to do this under pseudo to get the correct ownership/perms
+ bb_vars = get_bb_vars(['D', 'FAKEROOTENV', 'FAKEROOTCMD', 'PATH'], testrecipe)
+ installdir = bb_vars['D']
+ fakerootenv = bb_vars['FAKEROOTENV']
+ fakerootcmd = bb_vars['FAKEROOTCMD']
+ path = bb_vars['PATH']
+ result = runCmd('PATH="%s" %s %s find . -type f -exec ls -l {} \\;' % (path, fakerootenv, fakerootcmd), cwd=installdir)
+ filelist1 = self._process_ls_output(result.output)
+
+ # Now look on the target
+ tempdir2 = tempfile.mkdtemp(prefix='devtoolqa')
+ self.track_for_cleanup(tempdir2)
+ tmpfilelist = os.path.join(tempdir2, 'files.txt')
+ with open(tmpfilelist, 'w') as f:
+ for line in filelist1:
+ splitline = line.split()
+ f.write(splitline[-1] + '\n')
+ remotefilelist = '/tmp/%s' % os.path.basename(tmpfilelist)
+ status, _ = qemu.copy_to(tmpfilelist, remotefilelist)
+ self.assertEqual(status, 0)
+ status, output = qemu.run(
+ 'xargs ls -l < %s; status=$?; rm -f %s; exit $status' % (
+ remotefilelist, remotefilelist))
+ self.assertEqual(status, 0)
+ filelist2 = self._process_ls_output(output)
+ filelist1.sort(key=lambda item: item.split()[-1])
+ filelist2.sort(key=lambda item: item.split()[-1])
+ self.assertEqual(filelist1, filelist2)
+ else:
+ for path, expected in expected_files.items():
+ status, _ = qemu.run('test -e %s' % path)
+ if expected:
+ self.assertEqual(status, 0, '%s: %s should have been deployed' % (extra_args, path))
+ else:
+ self.assertNotEqual(status, 0, '%s: %s should not have been deployed' % (extra_args, path))
- # Now look on the target
- tempdir2 = tempfile.mkdtemp(prefix='devtoolqa')
- self.track_for_cleanup(tempdir2)
- tmpfilelist = os.path.join(tempdir2, 'files.txt')
- with open(tmpfilelist, 'w') as f:
- for line in filelist1:
- splitline = line.split()
- f.write(splitline[-1] + '\n')
- remotefilelist = '/tmp/%s' % os.path.basename(tmpfilelist)
- status, _ = qemu.copy_to(tmpfilelist, remotefilelist)
- self.assertEqual(status, 0)
- status, output = qemu.run(
- 'xargs ls -l < %s; status=$?; rm -f %s; exit $status' % (
- remotefilelist, remotefilelist))
- self.assertEqual(status, 0)
- filelist2 = self._process_ls_output(output)
- filelist1.sort(key=lambda item: item.split()[-1])
- filelist2.sort(key=lambda item: item.split()[-1])
- self.assertEqual(filelist1, filelist2)
# Test undeploy-target
- result = runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
+ runCmd('devtool undeploy-target -c %s root@%s' % (testrecipe, qemu.ip))
status, _ = qemu.run(testcommand)
self.assertNotEqual(status, 0, 'undeploy-target did not remove command as it should have')
+ filter_cases = [
+ ('', {}),
+ ('--package %s' % testrecipe,
+ {testfile: True, otherfile: True, docfile: False}),
+ ('--package %s-doc' % testrecipe,
+ {testfile: False, otherfile: False, docfile: True}),
+ ('--package %s,%s-doc' % (testrecipe, testrecipe),
+ {testfile: True, otherfile: True, docfile: True}),
+ ('--file-glob %s' % testfile,
+ {testfile: True, otherfile: False, docfile: False}),
+ ('--package %s --file-glob %s' % (testrecipe, testfile),
+ {testfile: True, otherfile: True, docfile: False}),
+ # "RECIPE:" prefix (needed so a single 'devtool ide-sdk' invocation
+ # can scope --package/--file-glob entries to one of several recipes)
+ ('--package %s:%s' % (testrecipe, testrecipe),
+ {testfile: True, otherfile: True, docfile: False}),
+ # shorthand: "RECIPE:-suffix" expands to "RECIPE-suffix"
+ ('--package %s:-doc' % testrecipe,
+ {testfile: False, otherfile: False, docfile: True}),
+ # shorthand: empty item + "-suffix" combined in one entry
+ ('--package %s:,-doc' % testrecipe,
+ {testfile: True, otherfile: True, docfile: True}),
+ # a "RECIPE:" prefix that doesn't match this recipe is skipped,
+ # leaving no package filter applied (same as passing none at all)
+ ('--package other-recipe:%s' % testrecipe,
+ {testfile: True, otherfile: True, docfile: True}),
+ ('--file-glob %s:%s' % (testrecipe, testfile),
+ {testfile: True, otherfile: False, docfile: False}),
+ ('--file-glob other-recipe:%s' % testfile,
+ {testfile: True, otherfile: True, docfile: True}),
+ ]
+ for strip_opt in ['', '--strip']:
+ for filter_args, expected_files in filter_cases:
+ extra_args = ' '.join(a for a in (strip_opt, filter_args) if a)
+ _deploy_and_check(extra_args, check_full_filelist=not filter_args, expected_files=expected_files)
+
class DevtoolBuildImageTests(DevtoolBase):
def test_devtool_build_image(self):
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index 267d124415..d11b04aea2 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -500,7 +500,15 @@ def register_commands(subparsers, context):
parser_deploy = subparsers.add_parser('deploy-target',
help='Deploy recipe output files to live target machine',
- description='Deploys a recipe\'s build output (i.e. the output of the do_install task) to a live target machine over ssh. By default, any existing files will be preserved instead of being overwritten and will be restored if you run devtool undeploy-target. Note: this only deploys the recipe itself and not any runtime dependencies, so it is assumed that those have been installed on the target beforehand. Use --package/--file-glob to deploy only a subset of the recipe\'s installed files.',
+ description='Deploys a recipe\'s build output (i.e. the output of '
+ 'the do_install task) to a live target machine over ssh. '
+ 'By default, any existing files will be preserved instead '
+ 'of being overwritten and will be restored if you run '
+ 'devtool undeploy-target. Note: this only deploys the '
+ 'recipe itself and not any runtime dependencies, so it is '
+ 'assumed that those have been installed on the target '
+ 'beforehand. Use --package/--file-glob to deploy only a '
+ 'subset of the recipe\'s installed files.',
group='testbuild')
parser_deploy.add_argument('recipename', help='Recipe to deploy')
parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]')
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 13/25] devtool: ide-sdk: forward --package/--file-glob deploy filters
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (11 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 12/25] oe-selftest: devtool deploy-target: test " AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 14/25] oe-selftest: devtool ide-sdk: cover --package filters AdrianF
` (11 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Wire deploy-target's --package and --file-glob filtering into
devtool ide-sdk so IDE-driven deploys can limit what is copied to the
target. This keeps ide-sdk's generated fast deploy script behavior
aligned with devtool deploy-target.
RecipeModified now snapshots PACKAGES and FILES:* during initialize(),
and the generated deploy script passes those mappings to deploy_no_d().
Requested packages are validated early so unknown package names fail
with a clear error. The generated script also accepts runtime
-g/--file-glob overrides, so users can refine filters from tasks.json
without regenerating the SDK.
Examples:
- devtool ide-sdk my-recipe --package my-recipe-doc
- devtool ide-sdk my-recipe --file-glob '/usr/bin/*'
- devtool ide-sdk my-recipe --package my-recipe:,-doc,-ptest \
--file-glob 'my-recipe:/usr/lib/*.so*'
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_plugins/__init__.py | 4 ++
scripts/lib/devtool/ide_plugins/ide_code.py | 14 ++++++-
scripts/lib/devtool/ide_sdk.py | 41 ++++++++++++++++++++-
3 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py
index a5069cae85..0d0117344d 100644
--- a/scripts/lib/devtool/ide_plugins/__init__.py
+++ b/scripts/lib/devtool/ide_plugins/__init__.py
@@ -427,4 +427,8 @@ def get_devtool_deploy_opts(args):
devtool_deploy_opts += ["-I", args.key]
if args.strip is False:
devtool_deploy_opts += ["--no-strip"]
+ for package in (args.package or []):
+ devtool_deploy_opts += ["--package", package]
+ for file_glob in (args.file_globs or []):
+ devtool_deploy_opts += ["--file-glob", file_glob]
return devtool_deploy_opts
diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index 5abf0492e8..937c63a4a5 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -596,6 +596,11 @@ class IdeVSCode(IdeBase):
def vscode_tasks_cpp(self, args, modified_recipe):
run_install_deploy = modified_recipe.gen_install_deploy_script(args)
install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty
+ deploy_args = ["--target", args.target]
+ if args.port:
+ deploy_args += ["--port", args.port]
+ for package in args.package or []:
+ deploy_args += ["--package", package]
tasks_dict = {
"version": "2.0.0",
"tasks": [
@@ -603,7 +608,7 @@ class IdeVSCode(IdeBase):
"label": install_task_name,
"type": "shell",
"command": run_install_deploy,
- "args": ["--target", args.target] + (["--port", args.port] if args.port else []),
+ "args": deploy_args,
"problemMatcher": []
}
]
@@ -712,6 +717,11 @@ class IdeVSCode(IdeBase):
reload_task_name = "reload module %s" % modified_recipe.recipe_id_pretty
verify_task_name = "verify module %s" % modified_recipe.recipe_id_pretty
run_install_deploy = modified_recipe.gen_install_deploy_script(args)
+ deploy_args = ["--target", args.target]
+ if args.port:
+ deploy_args += ["--port", args.port]
+ for package in args.package or []:
+ deploy_args += ["--package", package]
tasks_dict = {
"version": "2.0.0",
"tasks": [
@@ -719,7 +729,7 @@ class IdeVSCode(IdeBase):
"label": install_task_name,
"type": "shell",
"command": run_install_deploy,
- "args": ["--target", args.target] + (["--port", args.port] if args.port else []),
+ "args": deploy_args,
"problemMatcher": []
},
{
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index e45752aff1..ac4930af50 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -23,6 +23,7 @@ import scriptutils
import bb
from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, DevtoolError, parse_recipe
from devtool.standard import get_real_srctree
+from devtool.deploy import parse_packages_arg
from devtool.ide_plugins import BuildTool, DebuggerCrossConfig
from oe.kernel_module import kernel_module_os_env
@@ -671,6 +672,8 @@ class RecipeModified:
self.toolchain = None
self.topdir = None
self.workdir = None
+ # Maps each package name (from PACKAGES) to the glob patterns from its FILES variable
+ self.packages_files = {}
# Service management
self.systemd_services = {}
self.init_scripts = {}
@@ -745,6 +748,8 @@ class RecipeModified:
self.package_arch = recipe_d.getVar('PACKAGE_ARCH')
self.package_debug_split_style = recipe_d.getVar(
'PACKAGE_DEBUG_SPLIT_STYLE')
+ for package in (recipe_d.getVar('PACKAGES') or '').split():
+ self.packages_files[package] = recipe_d.getVar('FILES:' + package) or ''
self.path = recipe_d.getVar('PATH')
self.pn = recipe_d.getVar('PN')
self.recipe_sysroot = os.path.realpath(
@@ -1326,6 +1331,17 @@ class RecipeModified:
self._installed_binaries = dict(sorted(binaries.items()))
return self._installed_binaries
+ def _validate_requested_packages(self, args):
+ """Raise if --package (once scoped to this recipe and expanded via
+ parse_packages_arg) references a package this recipe doesn't produce.
+ """
+ packages = parse_packages_arg(getattr(args, 'package', None), self.bpn)
+ for package in packages:
+ if package not in self.packages_files:
+ raise DevtoolError('Package "%s" is not one of the packages produced '
+ 'by the %s recipe (PACKAGES: %s)' %
+ (package, self.pn, ' '.join(self.packages_files.keys())))
+
def gen_deploy_target_script(self, args):
"""Generate a script which does what devtool deploy-target does
@@ -1333,6 +1349,7 @@ class RecipeModified:
does not need to start a bitbake server. All information from tinfoil
is hard-coded in the generated script.
"""
+ self._validate_requested_packages(args)
cmd_lines = ['#!%s' % str(sys.executable)]
cmd_lines.append('import sys')
cmd_lines.append('devtool_sys_path = %s' % str(sys.path))
@@ -1353,6 +1370,8 @@ class RecipeModified:
cmd_lines.append(' for key in my_dict:')
cmd_lines.append(' setattr(self, key, my_dict[key])')
cmd_lines.append('filtered_args = Dict2Class(filtered_args_dict)')
+ cmd_lines.append('packages_files = %s' % repr(list(self.packages_files.items())))
+ cmd_lines.append('file_globs = %s' % repr(list(getattr(args, 'file_globs', None) or []) or None))
cmd_lines.append('i = 1')
cmd_lines.append('while i < len(sys.argv) - 1:')
cmd_lines.append(' if sys.argv[i] in ("-t", "--target"):')
@@ -1361,6 +1380,13 @@ class RecipeModified:
cmd_lines.append(' elif sys.argv[i] in ("-P", "--port"):')
cmd_lines.append(' setattr(filtered_args, "port", sys.argv[i + 1])')
cmd_lines.append(' i += 2')
+ cmd_lines.append(' elif sys.argv[i] in ("-g", "--file-glob"):')
+ cmd_lines.append(' file_globs = (file_globs or []) + [sys.argv[i + 1]]')
+ cmd_lines.append(' i += 2')
+ cmd_lines.append(' elif sys.argv[i] in ("-p", "--package"):')
+ cmd_lines.append(' packages = getattr(filtered_args, "package", None) or []')
+ cmd_lines.append(' setattr(filtered_args, "package", packages + [sys.argv[i + 1]])')
+ cmd_lines.append(' i += 2')
cmd_lines.append(' else:')
cmd_lines.append(' i += 1')
cmd_lines.append(
@@ -1368,7 +1394,7 @@ class RecipeModified:
cmd_lines.append(' filtered_args.no_host_check = True')
cmd_lines.append(
'setattr(filtered_args, "recipename", "%s")' % self.bpn)
- cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args)' %
+ cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files)' %
(self.d, self.workdir, self.path, self.strip_cmd,
self.libdir, self.base_libdir, self.max_process,
self.fakerootcmd, self.fakerootenv))
@@ -1758,4 +1784,17 @@ def register_commands(subparsers, context):
'-p', '--no-preserve', help='Do not preserve existing files', action='store_true')
parser_ide_sdk.add_argument(
'--no-check-space', help='Do not check for available space before deploying', action='store_true')
+ parser_ide_sdk.add_argument(
+ '--package', action='append', metavar='PACKAGE',
+ help='Only deploy files belonging to PACKAGE, as defined by that package\'s '
+ 'FILES variable in the recipe metadata. May be a comma-separated list '
+ 'and/or specified multiple times. May be prefixed with "RECIPE:" to target '
+ 'one of several recipes at once, e.g. "RECIPE:,-doc,-ptest" is short for '
+ '"RECIPE,RECIPE-doc,RECIPE-ptest".')
+ parser_ide_sdk.add_argument(
+ '--file-glob', action='append', dest='file_globs', metavar='GLOB',
+ help='Only deploy files whose installed path matches this glob pattern '
+ '(e.g. "/usr/bin/*"). May be specified multiple times. Combined with '
+ '--package if both are given. May be prefixed with "RECIPE:" to scope '
+ 'the entry to one of the recipes being processed e.g. "RECIPE:/usr/bin/*".')
parser_ide_sdk.set_defaults(func=ide_setup)
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 14/25] oe-selftest: devtool ide-sdk: cover --package filters
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (12 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 13/25] devtool: ide-sdk: forward --package/--file-glob deploy filters AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python AdrianF
` (10 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Extend ide-sdk selftests to pass recipe-scoped --package filters through
both ide=none and ide=code flows.
The ide=none multi-recipe test now exercises repeated filter forwarding:
- --package cmake-example:,-ptest
- --package meson-example:,-ptest
The ide=code cmake and meson tests now also pass:
- --package <recipe>:,-ptest
Example command covered by the test:
- devtool ide-sdk cmake-example meson-example oe-selftest-image \
-c --ide=none --package cmake-example:,-ptest \
--package meson-example:,-ptest
Example ide=code command covered by the test:
- devtool ide-sdk cmake-example oe-selftest-image \
-c --ide=code --package cmake-example:,-ptest
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 55 ++++++++++++++++++-------
1 file changed, 39 insertions(+), 16 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index b3eb3c28fb..3e0eb7948f 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3039,7 +3039,7 @@ class DevtoolIdeSdkTests(DevtoolBase):
recipe_id_pretty = recipe_name + ": " + package_arch
return (recipe_id, recipe_id_pretty)
- def _verify_install_script_code(self, tempdir, recipe_name):
+ def _verify_install_script_code(self, tempdir, recipe_name, package_args=None):
"""Verify the scripts referred by the tasks.json file are fine.
This function does not depend on Qemu. Therefore it verifies the scripts
@@ -3053,12 +3053,22 @@ class DevtoolIdeSdkTests(DevtoolBase):
task_install = next(
(task for task in tasks if task["label"] == "install && deploy-target %s" % recipe_id_pretty), None)
self.assertIsNot(task_install, None)
+ for package_arg in package_args or []:
+ self.assertIn("--package", task_install["args"])
+ self.assertIn(package_arg, task_install["args"])
# execute only the bb_run_do_install script since the deploy would require e.g. Qemu running.
i_and_d_script = "install_and_deploy_" + recipe_id
i_and_d_script_path = os.path.join(
self._workspace_scripts_dir(recipe_name), i_and_d_script)
self.assertExists(i_and_d_script_path)
+ deploy_script_path = os.path.join(
+ self._workspace_scripts_dir(recipe_name), 'deploy_target_' + recipe_id)
+ with open(deploy_script_path) as deploy_script:
+ deploy_script_content = deploy_script.read()
+ self.assertIn('packages_files = ', deploy_script_content)
+ self.assertNotIn("'package':", deploy_script_content)
+
def _devtool_ide_sdk_qemu(self, tempdir, qemu, recipe_name, example_exe, compile_cmd):
"""Verify deployment, execution and remote debugging in Qemu system work for one recipe.
@@ -3434,6 +3444,12 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
def test_devtool_ide_sdk_none_qemu(self):
"""Start qemu-system and run tests for multiple recipes. ide=none is used."""
recipe_names = ["cmake-example", "meson-example"]
+ package_filters = [
+ # Exercise multi-recipe package scoping via repeated --package:
+ # RECIPE:,-ptest expands to RECIPE and RECIPE-ptest.
+ "cmake-example:,-ptest",
+ "meson-example:,-ptest",
+ ]
testimage = "oe-selftest-image"
self._check_workspace()
@@ -3443,7 +3459,8 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
# Verify deployment to Qemu (system mode) works
tempdir_cmake = self._devtool_ide_sdk_recipe("cmake-example", "CMakeLists.txt", None)
tempdir_meson = self._devtool_ide_sdk_recipe("meson-example", "meson.build", testimage)
- runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none' % testimage,
+ package_opts = ' '.join('--package %s' % p for p in package_filters)
+ runCmd('devtool ide-sdk cmake-example meson-example %s -c --ide=none %s' % (testimage, package_opts),
output_log=self._cmd_logger)
with runqemu(testimage, runqemuparams="nographic") as qemu:
@@ -3458,9 +3475,11 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
# Verify /etc/cmake-example.conf is owned by the cmake-example user
self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group)
- # Re-run ide-sdk with the actual QEMU IP; image is already built
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % (
- recipe_name, testimage, qemu.ip)
+ # Re-run ide-sdk with only the image name. Modified recipes must
+ # be discovered from workspace appends; the image bbappend itself
+ # must not be mistaken for a modified recipe.
+ bitbake_sdk_cmd = 'devtool ide-sdk %s -t root@%s -c --skip-bitbake --ide=none %s' % (
+ testimage, qemu.ip, package_opts)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._gdb_cross()
@@ -3486,9 +3505,9 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
# Verify /etc/meson-example.conf is owned by the meson-example user
self._verify_conf_file(qemu, conf_file, example_user_group, example_user_group)
- # Re-run ide-sdk with the actual QEMU IP; image is already built
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none' % (
- recipe_name, testimage, qemu.ip)
+ # Re-run ide-sdk with the actual QEMU IP for this recipe
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=none %s' % (
+ recipe_name, testimage, qemu.ip, package_opts)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._gdb_cross()
@@ -3839,6 +3858,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
"""Verify a cmake recipe works with ide=code mode"""
recipe_name = "cmake-example"
example_exe = "cmake-example"
+ package_opt = '--package %s:,-ptest' % recipe_name
build_file = "CMakeLists.txt"
testimage = "oe-selftest-image"
build_file = "CMakeLists.txt"
@@ -3849,16 +3869,17 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
# Build image with debug settings before starting QEMU
self._check_runqemu_prerequisites()
tempdir = self._devtool_ide_sdk_recipe(recipe_name, build_file, testimage)
- runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage),
+ runCmd('devtool ide-sdk %s %s -c --ide=code %s' % (recipe_name, testimage, package_opt),
output_log=self._cmd_logger)
with runqemu(testimage, runqemuparams="nographic") as qemu:
# Re-run with actual QEMU IP; image is already built
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % (
- recipe_name, testimage, qemu.ip)
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code %s' % (
+ recipe_name, testimage, qemu.ip, package_opt)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
self._verify_cmake_preset(tempdir)
- self._verify_install_script_code(tempdir, recipe_name)
+ self._verify_install_script_code(
+ tempdir, recipe_name, ['%s:,-ptest' % recipe_name])
self._gdb_cross()
# Verify the launch.json file created is valid
@@ -3872,6 +3893,7 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
"""Verify a meson recipe works with ide=code mode"""
recipe_name = "meson-example"
example_exe = "mesonex"
+ package_opt = '--package %s:,-ptest' % recipe_name
build_file = "meson.build"
testimage = "oe-selftest-image"
@@ -3882,13 +3904,13 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
self._check_runqemu_prerequisites()
tempdir = self._devtool_ide_sdk_recipe(
recipe_name, build_file, testimage)
- runCmd('devtool ide-sdk %s %s -c --ide=code' % (recipe_name, testimage),
+ runCmd('devtool ide-sdk %s %s -c --ide=code %s' % (recipe_name, testimage, package_opt),
output_log=self._cmd_logger)
with runqemu(testimage, runqemuparams="nographic") as qemu:
# Re-run with actual QEMU IP; image is already built
- bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code' % (
- recipe_name, testimage, qemu.ip)
+ bitbake_sdk_cmd = 'devtool ide-sdk %s %s -t root@%s -c --skip-bitbake --ide=code %s' % (
+ recipe_name, testimage, qemu.ip, package_opt)
runCmd(bitbake_sdk_cmd, output_log=self._cmd_logger)
with open(os.path.join(tempdir, '.vscode', 'settings.json')) as settings_j:
@@ -3918,7 +3940,8 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
runCmd('%s test -C %s' % (meson_exe, meson_build_folder), cwd=tempdir,
output_log=self._cmd_logger)
- self._verify_install_script_code(tempdir, recipe_name)
+ self._verify_install_script_code(
+ tempdir, recipe_name, ['%s:,-ptest' % recipe_name])
self._gdb_cross()
# Verify the launch.json file created is valid
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (13 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 14/25] oe-selftest: devtool ide-sdk: cover --package filters AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-09-03 11:36 ` [OE-core] " Richard Purdie
2026-08-30 21:48 ` [PATCH v2 16/25] runqemu-export-rootfs: " AdrianF
` (9 subsequent siblings)
24 siblings, 1 reply; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Move the pseudo-backed rootfs extraction logic into a reusable Python
module while preserving the existing command interface.
Install the module with qemu-helper so the wrapper also works from an SDK.
AI-Generated: Uses GitHub Copilot
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
.../qemu/nativesdk-qemu-helper_1.0.bb | 3 +
scripts/lib/runqemu_utils.py | 124 ++++++++++++++++++
| 103 +--------------
3 files changed, 133 insertions(+), 97 deletions(-)
create mode 100644 scripts/lib/runqemu_utils.py
diff --git a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
index e1f343989a..16fc07fadf 100644
--- a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
+++ b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
@@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \
LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f"
SRC_URI = "file://${COREBASE}/scripts/runqemu \
+ file://${COREBASE}/scripts/lib/runqemu_utils.py \
file://${COREBASE}/scripts/runqemu-addptable2image \
file://${COREBASE}/scripts/runqemu-gen-tapdevs \
file://${COREBASE}/scripts/runqemu-ifup \
@@ -30,4 +31,6 @@ do_install() {
install -d ${D}${bindir}
install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/
install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/
+ # The runqemu-* wrappers import this module from their own directory.
+ install -m 0644 ${S}${COREBASE}/scripts/lib/runqemu_utils.py ${D}${bindir}/
}
diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py
new file mode 100644
index 0000000000..dff0ca8d3a
--- /dev/null
+++ b/scripts/lib/runqemu_utils.py
@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+#
+# Helpers shared by runqemu and NFS-rootfs preparation tools.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+
+"""Extract and export rootfs tarballs for NFS booting."""
+
+import os
+import subprocess
+import sys
+
+
+class RunQemuRootfsError(Exception):
+ """Raised when an NFS rootfs cannot be prepared or exported."""
+
+
+def native_environment():
+ """Return the qemu-helper-native environment set by oe-find-native-sysroot."""
+ command = '''
+ helper=$(command -v oe-find-native-sysroot) || exit 127
+ . "$helper" qemu-helper-native >/dev/null
+ env -0
+ printf 'PSEUDO=%s\\0OECORE_NATIVE_SYSROOT=%s\\0' "$PSEUDO" "$OECORE_NATIVE_SYSROOT"
+ '''
+ result = subprocess.run(['bash', '-c', command], capture_output=True)
+ if result.returncode:
+ if result.returncode == 127:
+ raise RunQemuRootfsError(
+ 'Unable to find the oe-find-native-sysroot script.\n'
+ 'Did you forget to source your build system environment setup script?')
+ raise RunQemuRootfsError(
+ result.stderr.decode(errors='replace').strip() or
+ 'Unable to set up the qemu-helper-native sysroot')
+
+ environment = {}
+ for entry in result.stdout.split(b'\0'):
+ if b'=' in entry:
+ key, value = entry.split(b'=', 1)
+ environment[key.decode()] = value.decode()
+ return environment
+
+
+def _tar_options(rootfs_tarball):
+ if rootfs_tarball.endswith('.tar.xz'):
+ return ['--numeric-owner', '-xJf']
+ if rootfs_tarball.endswith('.tar.bz2'):
+ return ['--numeric-owner', '-xjf']
+ if rootfs_tarball.endswith('.tar.gz'):
+ return ['--numeric-owner', '-xzf']
+ if rootfs_tarball.endswith('.tar.zst'):
+ return ['--numeric-owner', '--zstd', '-xf']
+ if rootfs_tarball.endswith('.tar'):
+ return ['--numeric-owner', '-xf']
+ raise RunQemuRootfsError(
+ 'Unable to determine sdk tarball format\n'
+ 'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst')
+
+
+def pseudo_state_dir(rootfs_dir):
+ """Return the pseudo database location associated with an extracted rootfs."""
+ rootfs_dir = os.path.realpath(rootfs_dir)
+ return os.path.join(os.path.dirname(rootfs_dir),
+ os.path.basename(rootfs_dir) + '.pseudo_state')
+
+
+def extract_sdk_rootfs(rootfs_tarball, rootfs_dir):
+ """Extract a rootfs tarball under pseudo and return its absolute directory."""
+ if not os.path.exists(rootfs_tarball):
+ raise RunQemuRootfsError("sdk tarball '%s' does not exist" % rootfs_tarball)
+
+ rootfs_tarball = os.path.realpath(rootfs_tarball)
+ rootfs_dir = os.path.realpath(rootfs_dir)
+ tar_options = _tar_options(rootfs_tarball)
+ state_dir = pseudo_state_dir(rootfs_dir)
+ debug_image = '-dbg' in os.path.basename(rootfs_tarball)
+
+ if os.path.exists(state_dir) and not debug_image:
+ raise RunQemuRootfsError(
+ '%s already exists!\n'
+ 'Please delete the rootfs tree and pseudo directory manually\n'
+ 'if this is really what you want.' % state_dir)
+
+ os.makedirs(rootfs_dir, exist_ok=True)
+ os.makedirs(state_dir, exist_ok=True)
+ open(os.path.join(state_dir, 'pseudo.pid'), 'a').close()
+
+ environment = native_environment()
+ environment['PSEUDO_LOCALSTATEDIR'] = state_dir
+ environment['PSEUDO_INCLUDE_PATHS'] = rootfs_dir
+ pseudo = environment.get('PSEUDO')
+ native_sysroot = environment.get('OECORE_NATIVE_SYSROOT')
+ if not pseudo or not native_sysroot:
+ raise RunQemuRootfsError('qemu-helper-native did not provide pseudo')
+
+ command = [pseudo, '-P', os.path.join(native_sysroot, 'usr'), 'tar', '-C', rootfs_dir]
+ command.extend(tar_options)
+ command.append(rootfs_tarball)
+ print('Extracting rootfs tarball using pseudo...')
+ print(' '.join(command))
+ try:
+ subprocess.run(command, env=environment, check=True)
+ except subprocess.CalledProcessError as exc:
+ raise RunQemuRootfsError('Failed to extract rootfs tarball') from exc
+
+ if len(os.listdir(rootfs_dir)) < 4:
+ print("Warning: I don't see many files in %s" % rootfs_dir)
+ print('Please double-check the extraction worked as intended')
+ else:
+ print('SDK image successfully extracted to %s' % rootfs_dir)
+ return rootfs_dir
+
+
+def extract_sdk_main(argv=None):
+ argv = sys.argv[1:] if argv is None else argv
+ if len(argv) != 2:
+ print('Usage: %s <image-tarball> <extract-dir>' % sys.argv[0])
+ return 1
+ try:
+ extract_sdk_rootfs(*argv)
+ except RunQemuRootfsError as exc:
+ print('Error: %s' % exc)
+ return 1
+ return 0
--git a/scripts/runqemu-extract-sdk b/scripts/runqemu-extract-sdk
index db9813e1dc..6cad2fdf90 100755
--- a/scripts/runqemu-extract-sdk
+++ b/scripts/runqemu-extract-sdk
@@ -1,104 +1,13 @@
-#!/bin/bash
-#
-# This utility extracts an SDK image tarball using pseudo, and stores
-# the pseudo database in var/pseudo within the rootfs. If you want to
-# boot QEMU using an nfsroot, you *must* use this script to create the
-# rootfs to ensure it is done correctly with pseudo.
-#
-# Copyright (c) 2010 Intel Corp.
+#!/usr/bin/env python3
#
# SPDX-License-Identifier: GPL-2.0-only
-#
-function usage() {
- echo "Usage: $0 <image-tarball> <extract-dir>"
-}
+import os
+import sys
-if [ $# -ne 2 ]; then
- usage
- exit 1
-fi
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
-SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot 2> /dev/null`
-if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then
- echo "Error: Unable to find the oe-find-native-sysroot script"
- echo "Did you forget to source your build system environment setup script?"
- exit 1
-fi
-. $SYSROOT_SETUP_SCRIPT qemu-helper-native
-PSEUDO_OPTS="-P $OECORE_NATIVE_SYSROOT/usr"
+from runqemu_utils import extract_sdk_main
-ROOTFS_TARBALL=$1
-SDK_ROOTFS_DIR=$2
-if [ ! -e "$ROOTFS_TARBALL" ]; then
- echo "Error: sdk tarball '$ROOTFS_TARBALL' does not exist"
- usage
- exit 1
-fi
-
-# Convert SDK_ROOTFS_DIR to a full pathname
-if [[ ${SDK_ROOTFS_DIR:0:1} != "/" ]]; then
- SDK_ROOTFS_DIR=$(readlink -f $(pwd)/$SDK_ROOTFS_DIR)
-fi
-
-TAR_OPTS=""
-if [[ "$ROOTFS_TARBALL" =~ tar\.xz$ ]]; then
- TAR_OPTS="--numeric-owner -xJf"
-fi
-if [[ "$ROOTFS_TARBALL" =~ tar\.bz2$ ]]; then
- TAR_OPTS="--numeric-owner -xjf"
-fi
-if [[ "$ROOTFS_TARBALL" =~ tar\.gz$ ]]; then
- TAR_OPTS="--numeric-owner -xzf"
-fi
-if [[ "$ROOTFS_TARBALL" =~ tar\.zst$ ]]; then
- TAR_OPTS="--numeric-owner --zstd -xf"
-fi
-if [[ "$ROOTFS_TARBALL" =~ \.tar$ ]]; then
- TAR_OPTS="--numeric-owner -xf"
-fi
-if [ -z "$TAR_OPTS" ]; then
- echo "Error: Unable to determine sdk tarball format"
- echo "Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / tar.zst"
- exit 1
-fi
-
-if [ ! -d "$SDK_ROOTFS_DIR" ]; then
- echo "Creating directory $SDK_ROOTFS_DIR"
- mkdir -p "$SDK_ROOTFS_DIR"
-fi
-
-pseudo_state_dir="$SDK_ROOTFS_DIR/../$(basename "$SDK_ROOTFS_DIR").pseudo_state"
-pseudo_state_dir="$(readlink -f $pseudo_state_dir)"
-
-debug_image="`echo $ROOTFS_TARBALL | grep '\-dbg\.rootfs\.tar'`"
-
-if [ -e "$pseudo_state_dir" -a -z "$debug_image" ]; then
- echo "Error: $pseudo_state_dir already exists!"
- echo "Please delete the rootfs tree and pseudo directory manually"
- echo "if this is really what you want."
- exit 1
-fi
-
-mkdir -p "$pseudo_state_dir"
-touch "$pseudo_state_dir/pseudo.pid"
-PSEUDO_LOCALSTATEDIR="$pseudo_state_dir"
-export PSEUDO_LOCALSTATEDIR
-PSEUDO_INCLUDE_PATHS="$SDK_ROOTFS_DIR"
-export PSEUDO_INCLUDE_PATHS
-
-echo "Extracting rootfs tarball using pseudo..."
-echo "$PSEUDO $PSEUDO_OPTS tar -C \"$SDK_ROOTFS_DIR\" $TAR_OPTS \"$ROOTFS_TARBALL\""
-$PSEUDO $PSEUDO_OPTS tar -C "$SDK_ROOTFS_DIR" $TAR_OPTS "$ROOTFS_TARBALL"
-
-DIRCHECK=`ls -l "$SDK_ROOTFS_DIR" | wc -l`
-if [ "$DIRCHECK" -lt 5 ]; then
- echo "Warning: I don't see many files in $SDK_ROOTFS_DIR"
- echo "Please double-check the extraction worked as intended"
- exit 0
-fi
-
-echo "SDK image successfully extracted to $SDK_ROOTFS_DIR"
-
-exit 0
+sys.exit(extract_sdk_main())
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 16/25] runqemu-export-rootfs: refactor in Python
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (14 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 17/25] devtool: deploy: split ssh deployment into a separate function AdrianF
` (8 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Move the userspace NFS export lifecycle into the reusable runqemu helper
module while preserving the existing command interface.
AI-Generated: Uses GitHub Copilot
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/runqemu_utils.py | 122 ++++++++++++++++++++++++++++++++
scripts/runqemu-export-rootfs | 126 ++--------------------------------
2 files changed, 128 insertions(+), 120 deletions(-)
diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py
index dff0ca8d3a..c36e30d76b 100644
--- a/scripts/lib/runqemu_utils.py
+++ b/scripts/lib/runqemu_utils.py
@@ -7,6 +7,7 @@
"""Extract and export rootfs tarballs for NFS booting."""
import os
+import signal
import subprocess
import sys
@@ -122,3 +123,124 @@ def extract_sdk_main(argv=None):
print('Error: %s' % exc)
return 1
return 0
+
+
+_NFS_ACTIONS = ('start', 'stop', 'restart')
+
+
+def _nfs_paths(instance):
+ state_dir = os.path.join(os.path.expanduser('~'), '.runqemu-sdk')
+ os.makedirs(state_dir, exist_ok=True)
+ return {
+ 'exports': os.path.join(state_dir, 'exports%s' % instance),
+ 'nfs_pid': os.path.join(state_dir, 'nfs%s.pid' % instance),
+ }
+
+
+def _nfs_ports(instance):
+ nfs_port = int(os.environ.get('NFSD_PORT', 3049 + 2 * instance))
+ mount_port = int(os.environ.get('MOUNTD_PORT', 3048 + 2 * instance))
+ return nfs_port, mount_port
+
+
+def _export_usage():
+ return 'Usage: %s {%s} <nfs-export-dir>' % (
+ sys.argv[0], '|'.join(_NFS_ACTIONS))
+
+
+def export_rootfs(action, rootfs_dir):
+ """Start, stop, or restart the userspace NFS server for *rootfs_dir*."""
+ if action not in _NFS_ACTIONS:
+ raise RunQemuRootfsError("Unknown command '%s'" % action)
+ if not os.path.isdir(rootfs_dir):
+ raise RunQemuRootfsError("'%s' does not exist" % rootfs_dir)
+
+ rootfs_dir = os.path.realpath(rootfs_dir)
+ state_dir = pseudo_state_dir(rootfs_dir)
+ if not os.path.isdir(state_dir):
+ raise RunQemuRootfsError(
+ '%s does not exist.\n'
+ 'Did you create the export directory using runqemu-extract-sdk?' % state_dir)
+
+ if action == 'restart':
+ export_rootfs('stop', rootfs_dir)
+ return export_rootfs('start', rootfs_dir)
+
+ instance = int(os.environ.get('NFS_INSTANCE', '0'))
+ paths = _nfs_paths(instance)
+ if action == 'stop':
+ if os.path.exists(paths['nfs_pid']):
+ print('Stopping rpc.nfsd')
+ with open(paths['nfs_pid']) as pid_file:
+ pid = pid_file.read().strip()
+ try:
+ os.kill(int(pid), signal.SIGTERM)
+ except (ValueError, ProcessLookupError):
+ # A stale PID file must not stop the cleanup below.
+ print('rpc.nfsd is not running')
+ os.unlink(paths['nfs_pid'])
+ else:
+ print('No PID file, not stopping rpc.nfsd')
+ if os.path.exists(paths['exports']):
+ print('Removing exports file')
+ os.unlink(paths['exports'])
+ return
+
+ environment = native_environment()
+ native_sysroot = environment.get('OECORE_NATIVE_SYSROOT')
+ pseudo = environment.get('PSEUDO')
+ if not native_sysroot or not pseudo:
+ raise RunQemuRootfsError('qemu-helper-native did not provide pseudo')
+
+ unfsd = os.path.join(native_sysroot, 'usr', 'bin', 'unfsd')
+ if not os.path.exists(unfsd):
+ raise RunQemuRootfsError(
+ 'Unable to find unfsd binary in %s/usr/bin/\n'
+ "This shouldn't happen - something is missing from your toolchain installation"
+ % native_sysroot)
+
+ nfs_port, mount_port = _nfs_ports(instance)
+ environment['PSEUDO_LOCALSTATEDIR'] = state_dir
+ with open(paths['exports'], 'w') as exports_file:
+ exports_file.write('%s (rw,no_root_squash,no_all_squash,insecure)\n' % rootfs_dir)
+
+ command = [pseudo, '-P', os.path.join(native_sysroot, 'usr'), unfsd,
+ '-p', '-i', paths['nfs_pid'], '-e', paths['exports'],
+ '-n', str(nfs_port), '-m', str(mount_port)]
+ print('Creating exports file...')
+ print('Starting User Mode nfsd')
+ print(' %s' % ' '.join(command))
+ try:
+ subprocess.run(command, env=environment, check=True)
+ except subprocess.CalledProcessError as exc:
+ raise RunQemuRootfsError('Error starting nfsd') from exc
+
+ if not os.path.exists(paths['nfs_pid']):
+ raise RunQemuRootfsError('rpc.nfsd did not start correctly')
+ with open(paths['nfs_pid']) as pid_file:
+ try:
+ os.kill(int(pid_file.read()), 0)
+ except OSError as exc:
+ raise RunQemuRootfsError('rpc.nfsd did not start correctly') from exc
+
+ print('')
+ print('On your target please remember to add the following options for NFS')
+ print('nfsroot=IP_ADDRESS:%s,nfsvers=3,port=%s,udp,mountport=%s' %
+ (rootfs_dir, nfs_port, mount_port))
+
+
+def export_rootfs_main(argv=None):
+ argv = sys.argv[1:] if argv is None else argv
+ if len(argv) != 2:
+ print(_export_usage())
+ return 1
+ if argv[0] not in _NFS_ACTIONS:
+ print("Unknown command '%s'" % argv[0])
+ print(_export_usage())
+ return 1
+ try:
+ export_rootfs(*argv)
+ except RunQemuRootfsError as exc:
+ print('Error: %s' % exc)
+ return 1
+ return 0
diff --git a/scripts/runqemu-export-rootfs b/scripts/runqemu-export-rootfs
index 6a8acd0d5a..dde364d37e 100755
--- a/scripts/runqemu-export-rootfs
+++ b/scripts/runqemu-export-rootfs
@@ -1,127 +1,13 @@
-#!/bin/bash
-#
-# Copyright (c) 2005-2009 Wind River Systems, Inc.
+#!/usr/bin/env python3
#
# SPDX-License-Identifier: GPL-2.0-only
-#
-usage() {
- echo "Usage: $0 {start|stop|restart} <nfs-export-dir>"
-}
+import os
+import sys
-if [ $# != 2 ]; then
- usage
- exit 1
-fi
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
-if [[ "$1" != "start" && "$1" != "stop" && "$1" != "restart" ]]; then
- echo "Unknown command '$1'"
- usage
- exit 1
-fi
+from runqemu_utils import export_rootfs_main
-if [ ! -d "$2" ]; then
- echo "Error: '$2' does not exist"
- usage
- exit 1
-fi
-# Ensure the nfs-export-dir is an absolute path
-NFS_EXPORT_DIR=$(cd "$2" && pwd)
-SYSROOT_SETUP_SCRIPT=`which oe-find-native-sysroot 2> /dev/null`
-if [ -z "$SYSROOT_SETUP_SCRIPT" ]; then
- echo "Error: Unable to find the oe-find-native-sysroot script"
- echo "Did you forget to source your build environment setup script?"
- exit 1
-fi
-. $SYSROOT_SETUP_SCRIPT qemu-helper-native
-
-if [ ! -e "$OECORE_NATIVE_SYSROOT/usr/bin/unfsd" ]; then
- echo "Error: Unable to find unfsd binary in $OECORE_NATIVE_SYSROOT/usr/bin/"
-
- echo "This shouldn't happen - something is missing from your toolchain installation"
- exit 1
-fi
-
-if [ ! -d ~/.runqemu-sdk ]; then
- mkdir -p ~/.runqemu-sdk
-fi
-
-NFS_INSTANCE=${NFS_INSTANCE:=0}
-EXPORTS=~/.runqemu-sdk/exports$NFS_INSTANCE
-RMTAB=~/.runqemu-sdk/rmtab$NFS_INSTANCE
-NFSPID=~/.runqemu-sdk/nfs$NFS_INSTANCE.pid
-MOUNTPID=~/.runqemu-sdk/mount$NFS_INSTANCE.pid
-
-PSEUDO_OPTS="-P $OECORE_NATIVE_SYSROOT/usr"
-PSEUDO_LOCALSTATEDIR="$NFS_EXPORT_DIR/../$(basename $NFS_EXPORT_DIR).pseudo_state"
-export PSEUDO_LOCALSTATEDIR
-
-if [ ! -d "$PSEUDO_LOCALSTATEDIR" ]; then
- echo "Error: $PSEUDO_LOCALSTATEDIR does not exist."
- echo "Did you create the export directory using runqemu-extract-sdk?"
- exit 1
-fi
-
-# NFS server port number
-NFSD_PORT=${NFSD_PORT:=$[ 3049 + 2 * $NFS_INSTANCE ]}
-# mountd port number
-MOUNTD_PORT=${MOUNTD_PORT:=$[ 3048 + 2 * $NFS_INSTANCE ]}
-
-## For debugging you would additionally add
-## --debug all
-UNFSD_OPTS="-p -i $NFSPID -e $EXPORTS -n $NFSD_PORT -m $MOUNTD_PORT"
-
-# See how we were called.
-case "$1" in
- start)
- echo "Creating exports file..."
- echo "$NFS_EXPORT_DIR (rw,no_root_squash,no_all_squash,insecure)" > $EXPORTS
-
- echo "Starting User Mode nfsd"
- echo " $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/bin/unfsd $UNFSD_OPTS"
- $PSEUDO $PSEUDO_OPTS $OECORE_NATIVE_SYSROOT/usr/bin/unfsd $UNFSD_OPTS
- if [ ! $? = 0 ]; then
- echo "Error starting nfsd"
- exit 1
- fi
- # Check to make sure everything started ok.
- if [ ! -f $NFSPID ]; then
- echo "rpc.nfsd did not start correctly"
- exit 1
- fi
- ps -fp `cat $NFSPID` > /dev/null 2> /dev/null
- if [ ! $? = 0 ]; then
- echo "rpc.nfsd did not start correctly"
- exit 1
- fi
- echo " "
- echo "On your target please remember to add the following options for NFS"
- echo "nfsroot=IP_ADDRESS:$NFS_EXPORT_DIR,nfsvers=3,port=$NFSD_PORT,udp,mountport=$MOUNTD_PORT"
- ;;
- stop)
- if [ -f "$NFSPID" ]; then
- echo "Stopping rpc.nfsd"
- kill `cat $NFSPID`
- rm -f $NFSPID
- else
- echo "No PID file, not stopping rpc.nfsd"
- fi
- if [ -f "$EXPORTS" ]; then
- echo "Removing exports file"
- rm -f $EXPORTS
- fi
- ;;
- restart)
- $0 stop $NFS_EXPORT_DIR
- $0 start $NFS_EXPORT_DIR
- if [ ! $? = 0 ]; then
- exit 1
- fi
- ;;
- *)
- echo "$0 {start|stop|restart} <nfs-export-dir>"
- ;;
-esac
-
-exit 0
+sys.exit(export_rootfs_main())
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 17/25] devtool: deploy: split ssh deployment into a separate function
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (15 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 16/25] runqemu-export-rootfs: " AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 18/25] devtool: deploy-target: allow deploying directly into a local rootfs AdrianF
` (7 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
This is a refactoring preparation for adding a new local deployment
method that will be used for NFS rootfs deployments.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/deploy.py | 174 ++++++++++++++++++----------------
1 file changed, 90 insertions(+), 84 deletions(-)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index d11b04aea2..d413fa7b91 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -254,6 +254,94 @@ def deploy(args, config, basepath, workspace):
return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files)
+def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files,
+ fakerootcmd, fakerootenv, path, recipe_outdir):
+ """Copy files to target_dir over ssh/scp (user@hostname[:destdir])."""
+ extraoptions = ''
+ if args.no_host_check:
+ extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
+ if not args.show_status:
+ extraoptions += ' -q'
+
+ scp_sshexec = ''
+ ssh_sshexec = 'ssh'
+ if args.ssh_exec:
+ scp_sshexec = "-S %s" % args.ssh_exec
+ ssh_sshexec = args.ssh_exec
+ scp_port = ''
+ ssh_port = ''
+ if args.port:
+ scp_port = "-P %s" % args.port
+ ssh_port = "-p %s" % args.port
+
+ if args.key:
+ extraoptions += ' -i %s' % args.key
+
+ # In order to delete previously deployed files and have the manifest file on
+ # the target, we write out a shell script and then copy it to the target
+ # so we can then run it (piping tar output to it).
+ # (We cannot use scp here, because it doesn't preserve symlinks.)
+ tmpdir = tempfile.mkdtemp(prefix='devtool')
+ try:
+ tmpscript = '/tmp/devtool_deploy.sh'
+ tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list')
+ shellscript = _prepare_remote_script(deploy=True,
+ destdir=destdir,
+ verbose=args.show_status,
+ nopreserve=args.no_preserve,
+ nocheckspace=args.no_check_space)
+ # Write out the script to a file
+ with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f:
+ f.write(shellscript)
+ # Write out the file list
+ with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f:
+ f.write('%d\n' % ftotalsize)
+ for fpath, fsize in filelist:
+ f.write('%s %d\n' % (fpath, fsize))
+ # Copy them to the target
+ ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True)
+ if ret != 0:
+ raise DevtoolError('Failed to copy script to %s - rerun with -s to '
+ 'get a complete error message' % args.target)
+ finally:
+ shutil.rmtree(tmpdir)
+
+ # Now run the script. When a package/glob filter narrowed down filelist,
+ # tar is given an explicit list of relative paths (-T) instead of packing
+ # the whole recipe_outdir tree.
+ tar_filelist_path = None
+ try:
+ if allowed_files is not None:
+ tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
+ with os.fdopen(tar_fd, 'w') as f:
+ for relpath in tar_relpaths:
+ # './' prefix matches what 'tar cf - .' itself would produce, which
+ # the remote script's manifest handling (sed "s!^./!$2!") relies on.
+ f.write('./' + relpath + '\n')
+ tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
+ else:
+ tar_cmd = 'tar cf - .'
+ remote_cmd = '%s | %s %s %s %s \'sh %s %s %s %s\'' % (
+ tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target,
+ tmpscript, args.recipename, destdir, tmpfilelist)
+ ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, remote_cmd, cwd=recipe_outdir, shell=True)
+ finally:
+ if tar_filelist_path:
+ os.remove(tar_filelist_path)
+ if ret != 0:
+ raise DevtoolError('Deploy failed - rerun with -s to get a complete '
+ 'error message')
+
+ logger.info('Successfully deployed %s' % recipe_outdir)
+
+ files_list = []
+ for root, _, files in os.walk(recipe_outdir):
+ for filename in files:
+ filename = os.path.relpath(os.path.join(root, filename), recipe_outdir)
+ files_list.append(os.path.join(destdir, filename))
+
+ return 0
+
def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None):
import math
@@ -351,90 +439,8 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
print(' %s' % item)
return 0
- extraoptions = ''
- if args.no_host_check:
- extraoptions += '-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
- if not args.show_status:
- extraoptions += ' -q'
-
- scp_sshexec = ''
- ssh_sshexec = 'ssh'
- if args.ssh_exec:
- scp_sshexec = "-S %s" % args.ssh_exec
- ssh_sshexec = args.ssh_exec
- scp_port = ''
- ssh_port = ''
- if args.port:
- scp_port = "-P %s" % args.port
- ssh_port = "-p %s" % args.port
-
- if args.key:
- extraoptions += ' -i %s' % args.key
-
- # In order to delete previously deployed files and have the manifest file on
- # the target, we write out a shell script and then copy it to the target
- # so we can then run it (piping tar output to it).
- # (We cannot use scp here, because it doesn't preserve symlinks.)
- tmpdir = tempfile.mkdtemp(prefix='devtool')
- try:
- tmpscript = '/tmp/devtool_deploy.sh'
- tmpfilelist = os.path.join(os.path.dirname(tmpscript), 'devtool_deploy.list')
- shellscript = _prepare_remote_script(deploy=True,
- destdir=destdir,
- verbose=args.show_status,
- nopreserve=args.no_preserve,
- nocheckspace=args.no_check_space)
- # Write out the script to a file
- with open(os.path.join(tmpdir, os.path.basename(tmpscript)), 'w') as f:
- f.write(shellscript)
- # Write out the file list
- with open(os.path.join(tmpdir, os.path.basename(tmpfilelist)), 'w') as f:
- f.write('%d\n' % ftotalsize)
- for fpath, fsize in filelist:
- f.write('%s %d\n' % (fpath, fsize))
- # Copy them to the target
- ret = subprocess.call("scp %s %s %s %s/* %s:%s" % (scp_sshexec, scp_port, extraoptions, tmpdir, args.target, os.path.dirname(tmpscript)), shell=True)
- if ret != 0:
- raise DevtoolError('Failed to copy script to %s - rerun with -s to '
- 'get a complete error message' % args.target)
- finally:
- shutil.rmtree(tmpdir)
-
- # Now run the script. When a package/glob filter narrowed down filelist,
- # tar is given an explicit list of relative paths (-T) instead of packing
- # the whole recipe_outdir tree.
- tar_filelist_path = None
- try:
- if allowed_files is not None:
- tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
- with os.fdopen(tar_fd, 'w') as f:
- for relpath in tar_relpaths:
- # './' prefix matches what 'tar cf - .' itself would produce, which
- # the remote script's manifest handling (sed "s!^./!$2!") relies on.
- f.write('./' + relpath + '\n')
- tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
- else:
- tar_cmd = 'tar cf - .'
- remote_cmd = '%s | %s %s %s %s \'sh %s %s %s %s\'' % (
- tar_cmd, ssh_sshexec, ssh_port, extraoptions, args.target,
- tmpscript, args.recipename, destdir, tmpfilelist)
- ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path, remote_cmd, cwd=recipe_outdir, shell=True)
- finally:
- if tar_filelist_path:
- os.remove(tar_filelist_path)
- if ret != 0:
- raise DevtoolError('Deploy failed - rerun with -s to get a complete '
- 'error message')
-
- logger.info('Successfully deployed %s' % recipe_outdir)
-
- files_list = []
- for root, _, files in os.walk(recipe_outdir):
- for filename in files:
- filename = os.path.relpath(os.path.join(root, filename), recipe_outdir)
- files_list.append(os.path.join(destdir, filename))
-
- return 0
+ return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths,
+ allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
def undeploy(args, config, basepath, workspace):
"""Entry point for the devtool 'undeploy' subcommand"""
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 18/25] devtool: deploy-target: allow deploying directly into a local rootfs
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (16 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 17/25] devtool: deploy: split ssh deployment into a separate function AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 19/25] oe-selftest: devtool deploy-target: test deploying into a local rootfs path AdrianF
` (6 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Accept an absolute path as the deploy-target "target" argument to copy
files directly into a local pseudo-managed rootfs directory (e.g. one
extracted by 'devtool ide-sdk --nfs' for NFS booting), instead of
always requiring an ssh connection.
Deploying over ssh to the target would also work for an NFS-mounted
rootfs, but that means dealing with the target's IP address and sshd
port, which is way more cumbersome than just passing a local path.
Copying the files in directly is also faster, since it avoids the
network/ssh round trip entirely.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/deploy.py | 119 ++++++++++++++++++++++++++++++----
1 file changed, 105 insertions(+), 14 deletions(-)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index d413fa7b91..4974a2fbae 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -20,6 +20,7 @@ import oe.types
import oe.package
from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, DevtoolError
+from runqemu_utils import native_environment, pseudo_state_dir
logger = logging.getLogger('devtool')
@@ -254,6 +255,85 @@ def deploy(args, config, basepath, workspace):
return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files)
+def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_files,
+ fakerootcmd, fakerootenv, path, recipe_outdir):
+ """Copy files directly into target_dir instead of over ssh/scp.
+
+ target_dir is expected to be a pseudo-managed rootfs previously extracted
+ on the host, e.g. by runqemu-extract-sdk or 'devtool ide-sdk --nfs' for a
+ target device or QEMU instance that NFS-mounts it.
+ """
+ if not os.path.isdir(target_dir):
+ raise DevtoolError('Target directory %s does not exist' % target_dir)
+ state_dir = pseudo_state_dir(target_dir)
+ if not os.path.isdir(state_dir):
+ raise DevtoolError(
+ '%s does not exist - %s does not look like a pseudo-managed rootfs '
+ '(e.g. one extracted by devtool ide-sdk --nfs).' % (state_dir, target_dir))
+
+ environment = native_environment()
+ pseudo = environment.get('PSEUDO')
+ native_sysroot = environment.get('OECORE_NATIVE_SYSROOT')
+ if not pseudo or not native_sysroot:
+ raise DevtoolError('qemu-helper-native did not provide pseudo')
+
+ if not args.no_check_space:
+ freespace = shutil.disk_usage(target_dir).free // 1024
+ if ftotalsize > freespace:
+ raise DevtoolError('Deploy failed - insufficient space on target '
+ '(available %d, needed %d)' % (freespace, ftotalsize))
+
+ # destdir is target_dir itself: the script writes directly into the
+ # local pseudo-managed rootfs directory, not into the real filesystem root.
+ destdir = target_dir
+ shellscript = _prepare_remote_script(deploy=True,
+ destdir=destdir,
+ verbose=args.show_status,
+ nopreserve=args.no_preserve,
+ nocheckspace=True)
+
+ tmpdir = tempfile.mkdtemp(prefix='devtool')
+ tar_filelist_path = None
+ try:
+ script_path = os.path.join(tmpdir, 'devtool_deploy.sh')
+ with open(script_path, 'w') as f:
+ f.write(shellscript)
+ filelist_path = os.path.join(tmpdir, 'devtool_deploy.list')
+ with open(filelist_path, 'w') as f:
+ f.write('%d\n' % ftotalsize)
+ for fpath, fsize in filelist:
+ f.write('%s %d\n' % (fpath, fsize))
+
+ if allowed_files is not None:
+ tar_fd, tar_filelist_path = tempfile.mkstemp(prefix='devtool-deploy-filelist-')
+ with os.fdopen(tar_fd, 'w') as f:
+ for relpath in tar_relpaths:
+ f.write('./' + relpath + '\n')
+ tar_cmd = 'tar cf - -T %s' % shlex.quote(tar_filelist_path)
+ else:
+ tar_cmd = 'tar cf - .'
+
+ # The extraction side needs its own pseudo session (with different database) than the recipe's own.
+ extract_cmd = 'PSEUDO_LOCALSTATEDIR=%s PSEUDO_INCLUDE_PATHS=%s %s -P %s sh %s %s %s %s' % (
+ shlex.quote(state_dir), shlex.quote(destdir), shlex.quote(pseudo),
+ shlex.quote(os.path.join(native_sysroot, 'usr')),
+ shlex.quote(script_path), shlex.quote(args.recipename),
+ shlex.quote(destdir), shlex.quote(filelist_path))
+ ret = exec_fakeroot_no_d(fakerootcmd, fakerootenv, path,
+ '%s | %s' % (tar_cmd, extract_cmd),
+ cwd=recipe_outdir, shell=True)
+ finally:
+ if tar_filelist_path:
+ os.remove(tar_filelist_path)
+ shutil.rmtree(tmpdir)
+
+ if ret != 0:
+ raise DevtoolError('Deploy failed - rerun with -s to get a complete '
+ 'error message')
+
+ logger.info('Successfully deployed %s to %s' % (recipe_outdir, target_dir))
+ return 0
+
def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files,
fakerootcmd, fakerootenv, path, recipe_outdir):
"""Copy files to target_dir over ssh/scp (user@hostname[:destdir])."""
@@ -345,12 +425,16 @@ def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files
def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None):
import math
- try:
- host, destdir = args.target.split(':')
- except ValueError:
- destdir = '/'
+ if os.path.isabs(args.target):
+ # A local pseudo-managed rootfs directory (e.g. NFS-exported)
+ destdir = os.path.realpath(args.target)
else:
- args.target = host
+ try:
+ host, destdir = args.target.split(':')
+ except ValueError:
+ destdir = '/'
+ else:
+ args.target = host
if not destdir.endswith('/'):
destdir += '/'
@@ -439,6 +523,12 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
print(' %s' % item)
return 0
+ if os.path.isabs(args.target):
+ # A local directory (e.g. an NFS-exported rootfs) rather than a
+ # user@host ssh target: copy the files in directly, no network needed.
+ return _deploy_local(args, os.path.realpath(args.target), filelist, ftotalsize, tar_relpaths,
+ allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
+
return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths,
allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
@@ -507,17 +597,18 @@ def register_commands(subparsers, context):
parser_deploy = subparsers.add_parser('deploy-target',
help='Deploy recipe output files to live target machine',
description='Deploys a recipe\'s build output (i.e. the output of '
- 'the do_install task) to a live target machine over ssh. '
- 'By default, any existing files will be preserved instead '
- 'of being overwritten and will be restored if you run '
- 'devtool undeploy-target. Note: this only deploys the '
- 'recipe itself and not any runtime dependencies, so it is '
- 'assumed that those have been installed on the target '
- 'beforehand. Use --package/--file-glob to deploy only a '
- 'subset of the recipe\'s installed files.',
+ 'the do_install task) to a live target machine over ssh, '
+ 'or directly into a local pseudo-managed rootfs directory '
+ '(e.g. one extracted for NFS booting). Existing files are '
+ 'preserved by default and restored by devtool '
+ 'undeploy-target. Only the recipe itself is deployed, not '
+ 'its runtime dependencies. Use --package/--file-glob to '
+ 'deploy only a subset of the recipe\'s installed files.',
group='testbuild')
parser_deploy.add_argument('recipename', help='Recipe to deploy')
- parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]')
+ parser_deploy.add_argument('target', help='Live target machine running an ssh server: user@hostname[:destdir]. '
+ 'Alternatively, an absolute path to a local pseudo-managed rootfs directory '
+ '(e.g. one extracted by devtool ide-sdk --nfs) to copy the files into directly, without ssh.')
parser_deploy.add_argument('-c', '--no-host-check', help='Disable ssh host key checking', action='store_true')
parser_deploy.add_argument('-s', '--show-status', help='Show progress/status output', action='store_true')
parser_deploy.add_argument('-n', '--dry-run', help='List files to be deployed only', action='store_true')
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 19/25] oe-selftest: devtool deploy-target: test deploying into a local rootfs path
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (17 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 18/25] devtool: deploy-target: allow deploying directly into a local rootfs AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 20/25] oe-selftest: tinfoil: test prepared task runner AdrianF
` (5 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Extract an image's rootfs tarball with runqemu-extract-sdk, boot it via
NFS with runqemu, then deploy a devtool-modified recipe directly into
that directory with plain 'devtool deploy-target <recipe> <path>' (no
ssh) while the target has it mounted live, and confirm the target
immediately sees the change.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 73 +++++++++++++++++++++++++
1 file changed, 73 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 3e0eb7948f..35f2aa2e6c 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -2171,6 +2171,79 @@ class DevtoolDeployTargetTests(DevtoolBase):
extra_args = ' '.join(a for a in (strip_opt, filter_args) if a)
_deploy_and_check(extra_args, check_full_filelist=not filter_args, expected_files=expected_files)
+ @OETestTag("runqemu")
+ def test_devtool_deploy_target_path(self):
+ """Verify 'devtool deploy-target <recipe> <path>' deploys straight into
+ a local pseudo-managed rootfs directory (no ssh), and that a target
+ booting that same directory via NFS immediately sees the change.
+ """
+ self._check_runqemu_prerequisites()
+ self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
+ testrecipe = 'mdadm'
+ testfile = '/sbin/mdadm'
+ if "usrmerge" in get_bb_var('DISTRO_FEATURES'):
+ testfile = '/usr/sbin/mdadm'
+ testcommand = '/sbin/mdadm --help'
+ testimage = 'oe-selftest-image'
+
+ # A tar rootfs is needed both to extract a local copy of it (below)
+ # and for runqemu to NFS-boot straight from that extracted directory.
+ self.append_config('IMAGE_FSTYPES:append = " tar"\n')
+ bitbake("%s qemu-native qemu-helper-native" % testimage)
+ bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'IMAGE_LINK_NAME'], testimage)
+ deploy_dir_image = bb_vars['DEPLOY_DIR_IMAGE']
+ image_link_name = bb_vars['IMAGE_LINK_NAME']
+ self.add_command_to_tearDown('bitbake -c clean %s' % testimage)
+ self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, testimage))
+
+ tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+ self.track_for_cleanup(tempdir)
+ self.track_for_cleanup(self.workspacedir)
+ self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
+ self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+ runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+ runCmd('devtool build %s' % testrecipe)
+
+ # Extract a local pseudo-managed rootfs the same way
+ rootfs_tarball = os.path.join(deploy_dir_image, image_link_name + '.tar')
+ self.assertExists(rootfs_tarball)
+ extractdir = tempfile.mkdtemp(prefix='devtoolqa')
+ self.track_for_cleanup(extractdir)
+ nfs_rootfs = os.path.join(extractdir, 'rootfs')
+ runCmd('runqemu-extract-sdk %s %s' % (rootfs_tarball, nfs_rootfs))
+ self.assertExists(nfs_rootfs)
+ self.assertExists(nfs_rootfs + '.pseudo_state')
+
+ # oe-selftest-image does not install mdadm by default, so the target must not see it yet.
+ self.assertNotExists(os.path.join(nfs_rootfs, testfile.lstrip('/')))
+
+ qemuboot = os.path.join(deploy_dir_image, image_link_name + '.qemuboot.conf')
+ self.assertExists(qemuboot)
+ launch_cmd = 'runqemu %s %s nographic' % (shlex.quote(qemuboot), shlex.quote(nfs_rootfs))
+ with runqemu(testimage, launch_cmd=launch_cmd) as qemu:
+ status, output = qemu.run("awk '$2 == \"/\" {print $3}' /proc/mounts")
+ self.assertEqual(status, 0)
+ self.assertEqual(output.strip(), 'nfs')
+
+ status, _ = qemu.run(testcommand)
+ self.assertNotEqual(status, 0, '%s should not be deployed yet' % testfile)
+
+ # Deploy directly into the local rootfs path (no ssh) while the target has it NFS-mounted live
+ deploy_cmd = 'devtool deploy-target %s %s' % (testrecipe, nfs_rootfs)
+ if self.logger.isEnabledFor(logging.DEBUG):
+ deploy_cmd += ' -s'
+ result = runCmd(deploy_cmd)
+ self.assertEqual(result.status, 0)
+ self.assertExists(os.path.join(nfs_rootfs, testfile.lstrip('/')))
+
+ status, _ = qemu.run(testcommand)
+ self.assertEqual(status, 0, '%s was not deployed' % testfile)
+
+ # Deploying again while the target still has this directory NFS-mounted live must still succeed.
+ result = runCmd(deploy_cmd)
+ self.assertEqual(result.status, 0)
+
+
class DevtoolBuildImageTests(DevtoolBase):
def test_devtool_build_image(self):
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 20/25] oe-selftest: tinfoil: test prepared task runner
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (18 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 19/25] oe-selftest: devtool deploy-target: test deploying into a local rootfs path AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 21/25] devtool: ide-sdk: run do_install through BitBake AdrianF
` (4 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Add shell and Python marker tasks to pseudo-pyc-test, with and without
fakeroot. Build the tasks normally to prepare their dependencies, then run
them again through Tinfoil's run_prepared_task() API.
This verifies that a prepared task can be rerun through BitBake's worker
path without resolving its dependencies, including shell and Python task
execution within and outside a pseudo session as needed by devtool ide-sdk.
Exercise Tinfoil.run_prepared_task() with shell and Python tasks, both
with and without fakeroot. Build the exact task targets normally to
prepare their prerequisites, remove their markers, then rerun them
through Tinfoil.
This verifies the no-dependency rerun uses BitBake's worker path for
each task type and pseudo context.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
.../pseudo-pyc-test/pseudo-pyc-test.bb | 45 ++++++++++++++
meta/lib/oeqa/selftest/cases/tinfoil.py | 60 +++++++++++++++++++
2 files changed, 105 insertions(+)
diff --git a/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
index 12dc91a8f3..f6b31b63cd 100644
--- a/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
+++ b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
@@ -13,3 +13,48 @@ python do_install() {
import pseudo_pyc_test2
print(pseudo_pyc_test2.STRING)
}
+
+TINFOIL_TEST_MARKER_DIR = "${TMPDIR}/tinfoil-prepared-task"
+TINFOIL_TEST_MARKER_VALUE ?= ""
+
+write_tinfoil_shell_marker() {
+ mkdir -p ${TINFOIL_TEST_MARKER_DIR}
+ printf '%s%s' "$1" "${TINFOIL_TEST_MARKER_VALUE}" > ${TINFOIL_TEST_MARKER_DIR}/"$1"
+}
+
+do_tinfoil_dep() {
+ write_tinfoil_shell_marker dep
+}
+addtask tinfoil_dep
+
+do_tinfoil_shell() {
+ write_tinfoil_shell_marker shell
+}
+addtask tinfoil_shell after do_tinfoil_dep
+
+do_tinfoil_shell_fakeroot() {
+ write_tinfoil_shell_marker shell-fakeroot
+}
+do_tinfoil_shell_fakeroot[fakeroot] = "1"
+do_tinfoil_shell_fakeroot[depends] += "virtual/fakeroot-native:do_populate_sysroot"
+addtask tinfoil_shell_fakeroot after do_tinfoil_dep
+
+def write_tinfoil_marker(d, name):
+ import os
+
+ marker_dir = d.getVar('TINFOIL_TEST_MARKER_DIR')
+ os.makedirs(marker_dir, exist_ok=True)
+ with open(os.path.join(marker_dir, name), 'w') as marker:
+ marker.write(name + d.getVar('TINFOIL_TEST_MARKER_VALUE'))
+
+python do_tinfoil_python() {
+ write_tinfoil_marker(d, 'python')
+}
+addtask tinfoil_python after do_tinfoil_dep
+
+python do_tinfoil_python_fakeroot() {
+ write_tinfoil_marker(d, 'python-fakeroot')
+}
+do_tinfoil_python_fakeroot[fakeroot] = "1"
+do_tinfoil_python_fakeroot[depends] += "virtual/fakeroot-native:do_populate_sysroot"
+addtask tinfoil_python_fakeroot after do_tinfoil_dep
diff --git a/meta/lib/oeqa/selftest/cases/tinfoil.py b/meta/lib/oeqa/selftest/cases/tinfoil.py
index 21c8686b2a..d8009eddfe 100644
--- a/meta/lib/oeqa/selftest/cases/tinfoil.py
+++ b/meta/lib/oeqa/selftest/cases/tinfoil.py
@@ -6,11 +6,14 @@
import os
import re
+import shutil
import time
import logging
+import uuid
import bb.tinfoil
from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake
class TinfoilTests(OESelftestTestCase):
""" Basic tests for the tinfoil API """
@@ -77,6 +80,63 @@ class TinfoilTests(OESelftestTestCase):
localdata.setVar('PN', 'hello')
self.assertEqual('hello', localdata.getVar('BPN'))
+ def test_run_prepared_task(self):
+ """Verify run_prepared_task() runs only the requested task, not its
+ dependency.
+
+ pseudo-pyc-test's do_tinfoil_* tasks depend on do_tinfoil_dep; both
+ write a marker file containing the task name plus the current value
+ of TINFOIL_TEST_MARKER_VALUE. A normal bitbake build is expected to
+ update both the dependency's and the requested tasks' markers to a
+ new value. Calling run_prepared_task() with another new value must
+ update only the requested tasks' markers, leaving do_tinfoil_dep's
+ marker at the value written by the earlier bitbake build.
+ """
+ marker_dir = os.path.join(
+ self.builddir, 'tmp', 'tinfoil-prepared-task')
+ shutil.rmtree(marker_dir, ignore_errors=True)
+ self.track_for_cleanup(marker_dir)
+
+ tasks = ('shell', 'shell_fakeroot', 'python', 'python_fakeroot')
+
+ # Use a fresh, random value each run so the tasks' signatures change
+ # and bitbake can't skip them as "up to date" from a previous run.
+ value1 = uuid.uuid4().hex
+ bitbake(' '.join('pseudo-pyc-test:do_tinfoil_%s' % task
+ for task in tasks),
+ postconfig='TINFOIL_TEST_MARKER_VALUE = "%s"\n' % value1)
+
+ # do_tinfoil_dep is a dependency of all the tasks above, so a normal
+ # bitbake build is expected to have run it too, with the same value.
+ with open(os.path.join(marker_dir, 'dep')) as marker:
+ self.assertEqual(marker.read(), 'dep' + value1)
+ for task in tasks:
+ marker_name = task.replace('_', '-')
+ with open(os.path.join(marker_dir, marker_name)) as marker:
+ self.assertEqual(marker.read(), marker_name + value1)
+
+ with bb.tinfoil.Tinfoil() as tinfoil:
+ tinfoil.prepare(config_only=False, quiet=2)
+ # A distinct value per call proves each run_prepared_task()
+ # invocation re-reads the variable rather than reusing a value
+ # cached from an earlier call in this loop.
+ values2 = {task: uuid.uuid4().hex for task in tasks}
+ for task in tasks:
+ tinfoil.run_command('setVariable', 'TINFOIL_TEST_MARKER_VALUE', values2[task])
+ tinfoil.run_prepared_task(
+ 'pseudo-pyc-test', 'do_tinfoil_%s' % task)
+
+ # The requested tasks must have re-run and picked up their own value
+ for task in tasks:
+ marker_name = task.replace('_', '-')
+ with open(os.path.join(marker_dir, marker_name)) as marker:
+ self.assertEqual(marker.read(), marker_name + values2[task])
+
+ # ...but run_prepared_task() must not have run do_tinfoil_dep, so its
+ # marker should still hold the value written by the earlier bitbake build
+ with open(os.path.join(marker_dir, 'dep')) as marker:
+ self.assertEqual(marker.read(), 'dep' + value1)
+
# The config_data API to parse_recipe_file is used by:
# layerindex-web layerindex/update_layer.py
def test_parse_recipe_custom_data(self):
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 21/25] devtool: ide-sdk: run do_install through BitBake
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (19 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 20/25] oe-selftest: tinfoil: test prepared task runner AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 22/25] oe-selftest: devtool ide-sdk: test install task AdrianF
` (3 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Generate a Python install-task runner for the IDE SDK environment. It
prepares Tinfoil normally, then invokes run_prepared_task() to rerun
do_install through BitBake's worker path before deploying the result.
The IDE owns configure, compile, and unit-test steps in the modified source
tree. At deployment time, resolving task dependencies could rerun those
steps, discard the IDE's build results, and add unnecessary delay. The
prepared-task runner therefore executes only do_install after BitBake has
prepared its prerequisites.
This supports shell and Python task bodies and lets BitBake establish the
fakeroot environment when the task requires pseudo.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/ide_sdk.py | 42 +++++++++++++++++++---------------
1 file changed, 24 insertions(+), 18 deletions(-)
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index ac4930af50..ee3c50460a 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -1400,26 +1400,32 @@ class RecipeModified:
self.fakerootcmd, self.fakerootenv))
return self.write_script(cmd_lines, 'deploy_target')
+ def gen_install_task_script(self):
+ """Generate a script which runs do_install through BitBake."""
+ cmd_lines = ['#!%s' % sys.executable]
+ if self.cmd_compile:
+ cmd_lines += ['import subprocess',
+ 'subprocess.run(%s, cwd=%r, shell=True, check=True)' %
+ (repr(self.cmd_compile), self.real_srctree)]
+ cmd_lines += ['import os',
+ 'import sys',
+ 'sys.path.insert(0, %r)' % os.path.realpath(
+ os.path.join(self.bitbakepath, '..', 'lib')),
+ 'import bb.tinfoil',
+ 'os.chdir(%r)' % self.topdir,
+ 'tinfoil = bb.tinfoil.Tinfoil()',
+ 'try:',
+ ' tinfoil.prepare(config_only=False, quiet=2)',
+ ' tinfoil.run_prepared_task(%r, "do_install")' % self.pn,
+ 'finally:',
+ ' tinfoil.shutdown()']
+ return self.write_script(cmd_lines, 'bb_run_do_install')
+
def gen_install_deploy_script(self, args):
"""Generate a script which does install and deploy"""
- cmd_lines = ['#!/bin/sh']
-
- # Save the original command-line args before 'set' overwrites $@
- cmd_lines.append('_args="$@"')
- # . oe-init-build-env $BUILDDIR $BITBAKEDIR
- # Using 'set' to pass the build directory to oe-init-build-env in sh syntax
- cmd_lines.append('cd "%s" || { echo "cd %s failed"; exit 1; }' % (
- self.oe_init_dir, self.oe_init_dir))
- cmd_lines.append('set %s %s' % (self.topdir, self.bitbakepath.rstrip('/bin')))
- cmd_lines.append('. "%s" || { echo ". %s %s failed"; exit 1; }' % (
- self.oe_init_build_env, self.oe_init_build_env, self.topdir))
-
- # bitbake -c install
- cmd_lines.append(
- 'bitbake %s -c install --force || { echo "bitbake %s -c install --force failed"; exit 1; }' % (self.bpn, self.bpn))
-
- # Self contained devtool deploy-target - use saved args, not $@ (overwritten by 'set')
- cmd_lines.append(self.gen_deploy_target_script(args) + ' $_args')
+ cmd_lines = ['#!/bin/sh -e']
+ cmd_lines.append(self.gen_install_task_script())
+ cmd_lines.append(self.gen_deploy_target_script(args) + ' "$@"')
return self.write_script(cmd_lines, 'install_and_deploy')
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 22/25] oe-selftest: devtool ide-sdk: test install task
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (20 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 21/25] devtool: ide-sdk: run do_install through BitBake AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 23/25] devtool: ide-sdk: support NFS rootfs AdrianF
` (2 subsequent siblings)
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Verify that the ide-sdk setup generates the do_install task script and that
the script runs successfully. This covers the prepared-task runner used by
the IDE deployment workflow without requiring Qemu.
Replace the ad-hoc dependsOn handling in _verify_launch_config() with a
new helper that mirrors VS Code's task dependency resolution and
execution order. This ensures that the selftest runs the same sequence
of tasks as VS Code would, including compile and install/deploy
prerequisites, before starting the debug server. This also aligns the
gdb and lldb selftest paths to avoid behavior drift and improve test
reliability.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 86 ++++++++++++++++++++-----
1 file changed, 70 insertions(+), 16 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 35f2aa2e6c..c05cf7bc82 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3116,7 +3116,7 @@ class DevtoolIdeSdkTests(DevtoolBase):
"""Verify the scripts referred by the tasks.json file are fine.
This function does not depend on Qemu. Therefore it verifies the scripts
- exists and the delete step works as expected. But it does not try to
+ exists and the install step works as expected. But it does not try to
deploy to Qemu.
"""
recipe_id, recipe_id_pretty = self._get_recipe_ids(recipe_name)
@@ -3134,6 +3134,11 @@ class DevtoolIdeSdkTests(DevtoolBase):
i_and_d_script_path = os.path.join(
self._workspace_scripts_dir(recipe_name), i_and_d_script)
self.assertExists(i_and_d_script_path)
+ install_script = 'bb_run_do_install_' + recipe_id
+ install_script_path = os.path.join(
+ self._workspace_scripts_dir(recipe_name), install_script)
+ self.assertExists(install_script_path)
+ runCmd(install_script_path, cwd=tempdir, output_log=self._cmd_logger)
deploy_script_path = os.path.join(
self._workspace_scripts_dir(recipe_name), 'deploy_target_' + recipe_id)
@@ -3368,6 +3373,60 @@ class DevtoolIdeSdkTests(DevtoolBase):
self.assertEqual(actual_group, group,
msg="%s not owned by group %s: got %s" % (conf_file, group, actual_group))
+ def _run_vscode_task_with_dependencies(self, tempdir, task_label, run_main_task=True):
+ """Run a VS Code shell task and its dependsOn chain.
+
+ The helper mirrors VS Code task dependency ordering for selftests, so
+ preLaunchTask prerequisites (e.g. compile and do_install/deploy tasks)
+ run before the debug server task.
+ """
+ tasks_path = os.path.join(tempdir, '.vscode', 'tasks.json')
+ self.assertExists(tasks_path, 'tasks.json not found at %s' % tasks_path)
+
+ with open(tasks_path) as tasks_j:
+ tasks_d = json.load(tasks_j)
+
+ task_by_label = {
+ task.get('label'): task
+ for task in tasks_d.get('tasks', [])
+ if task.get('label')
+ }
+ self.assertIn(task_label, task_by_label,
+ "Task '%s' not found in tasks.json" % task_label)
+
+ ran = set()
+
+ def run_task(label, execute_task):
+ if label in ran:
+ return
+
+ task = task_by_label[label]
+ deps = task.get('dependsOn', [])
+ if isinstance(deps, str):
+ deps = [deps]
+
+ for dep in deps:
+ self.assertIn(dep, task_by_label,
+ "DependsOn task '%s' not found in tasks.json" % dep)
+ run_task(dep, True)
+
+ if execute_task:
+ command = task.get('command')
+ if command:
+ args = task.get('args', [])
+ if not isinstance(args, list):
+ args = [args]
+ task_cmd = [command] + [str(arg) for arg in args]
+
+ task_cwd = task.get('options', {}).get('cwd', tempdir)
+ if isinstance(task_cwd, str):
+ task_cwd = task_cwd.replace('${workspaceFolder}', tempdir)
+ runCmd(task_cmd, cwd=task_cwd, output_log=self._cmd_logger)
+
+ ran.add(label)
+
+ run_task(task_label, run_main_task)
+
class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
@@ -3853,21 +3912,10 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
(task for task in tasks if task["label"] == prelaunch_task_name), None)
self.assertIsNotNone(prelaunch_task, f"PreLaunchTask '{prelaunch_task_name}' not found in tasks.json")
- # Find the dependsOn task if exists (install and deploy-target)
- if "dependsOn" in prelaunch_task:
- depends_task_names = prelaunch_task["dependsOn"]
- for depends_task_name in depends_task_names:
- depends_task = next(
- (task for task in tasks if task["label"] == depends_task_name), None)
- self.assertIsNotNone(depends_task, f"DependsOn task '{depends_task_name}' not found in tasks.json")
- # For simplicity, we assume the dependsOn task is a prerequisite and does not affect the main command
- self.logger.debug(f"PreLaunchTask '{prelaunch_task_name}' depends on '{depends_task_name}'")
-
- # Extract command details from dependsOn task
- depends_task_command = depends_task["command"]
- depends_task_args = depends_task.get("args", [])
- self.logger.debug(f"Would execute dependsOn task: {depends_task_command} {' '.join(depends_task_args)}")
- runCmd(f"{depends_task_command} {' '.join(depends_task_args)}", output_log=self._cmd_logger)
+ # Execute preLaunchTask prerequisites exactly like VS Code dependsOn
+ # handling, but keep starting the debug server in this method.
+ self._run_vscode_task_with_dependencies(
+ tempdir, prelaunch_task_name, run_main_task=False)
# Verify task structure and extract command details
self.assertEqual(prelaunch_task["type"], "shell", f"Task '{prelaunch_task_name}' should be shell type")
@@ -4635,6 +4683,12 @@ class DevtoolIdeSdkClangTests(DevtoolIdeSdkTests):
self.assertIsNotNone(prelaunch_task,
"preLaunchTask '%s' not found in tasks.json" % prelaunch_task_name)
+ # Run compile/install/deploy prerequisites exactly as VS Code would,
+ # but keep starting the debug server itself in this method so we can
+ # retain explicit readiness handling and diagnostics.
+ self._run_vscode_task_with_dependencies(
+ tempdir, prelaunch_task_name, run_main_task=False)
+
# Extract the SSH command and start lldb-server on the target
task_command = prelaunch_task["command"]
task_args = prelaunch_task["args"]
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 23/25] devtool: ide-sdk: support NFS rootfs
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (21 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 22/25] oe-selftest: devtool ide-sdk: test install task AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 24/25] oe-selftest: devtool ide-sdk: test NFS debug rootfs AdrianF
2026-08-30 21:48 ` [PATCH v2 25/25] devtool: ide-sdk: fix GDB loading stale libs instead of recipe's own build AdrianF
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Add --nfs=rootfs and --nfs=rootfs-dbg to build the matching tar image,
extract it below the workspace image directory, and use it as the
deploy-target/debug destination instead of ssh.
Configure remote debuggers to search the extracted workspace rootfs-dbg
when --nfs=rootfs-dbg is selected, matching the filesystem exported to
the target.
devtool deploy-target and the generated deploy_target/install_and_deploy
scripts learn to copy files directly into a local pseudo-managed rootfs
directory (deploy_no_d()/_deploy_local()) when the target is an absolute
path, instead of requiring ssh. The pseudo binary needed for this is
resolved from qemu-helper-native's STAGING_DIR_NATIVE at devtool ide-sdk
generation time, when the build environment is known to be sourced, and
baked into the generated script so it keeps working when later re-run
standalone, e.g. as a VS Code task, without oe-find-native-sysroot on
PATH.
tasks.json generation (vscode_tasks_cpp/vscode_tasks_kernel_module) now
resolves the effective deploy target via resolve_deploy_target(), so the
--nfs local directory is used consistently instead of the ssh target
when both apply.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
scripts/lib/devtool/deploy.py | 22 ++-
scripts/lib/devtool/ide_plugins/__init__.py | 22 ++-
scripts/lib/devtool/ide_plugins/ide_code.py | 39 +++--
scripts/lib/devtool/ide_plugins/ide_none.py | 2 +-
scripts/lib/devtool/ide_sdk.py | 153 ++++++++++++++++++--
5 files changed, 203 insertions(+), 35 deletions(-)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index 4974a2fbae..243bc9a18b 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -256,7 +256,7 @@ def deploy(args, config, basepath, workspace):
return deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=args.file_globs, packages_files=packages_files)
def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_files,
- fakerootcmd, fakerootenv, path, recipe_outdir):
+ fakerootcmd, fakerootenv, path, recipe_outdir, qemu_helper_native_sysroot=None):
"""Copy files directly into target_dir instead of over ssh/scp.
target_dir is expected to be a pseudo-managed rootfs previously extracted
@@ -271,10 +271,17 @@ def _deploy_local(args, target_dir, filelist, ftotalsize, tar_relpaths, allowed_
'%s does not exist - %s does not look like a pseudo-managed rootfs '
'(e.g. one extracted by devtool ide-sdk --nfs).' % (state_dir, target_dir))
- environment = native_environment()
- pseudo = environment.get('PSEUDO')
- native_sysroot = environment.get('OECORE_NATIVE_SYSROOT')
- if not pseudo or not native_sysroot:
+ if qemu_helper_native_sysroot:
+ # Already resolved at script-generation time (when the build
+ # environment was sourced), so this can run standalone, e.g. from a
+ # devtool ide-sdk generated script/VS Code task.
+ native_sysroot = qemu_helper_native_sysroot
+ pseudo = os.path.join(native_sysroot, 'usr', 'bin', 'pseudo')
+ else:
+ environment = native_environment()
+ pseudo = environment.get('PSEUDO')
+ native_sysroot = environment.get('OECORE_NATIVE_SYSROOT')
+ if not pseudo or not native_sysroot or not os.path.exists(pseudo):
raise DevtoolError('qemu-helper-native did not provide pseudo')
if not args.no_check_space:
@@ -422,7 +429,7 @@ def _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths, allowed_files
return 0
-def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None):
+def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_process, fakerootcmd, fakerootenv, args, file_globs=None, packages_files=None, qemu_helper_native_sysroot=None):
import math
if os.path.isabs(args.target):
@@ -527,7 +534,8 @@ def deploy_no_d(srcdir, workdir, path, strip_cmd, libdir, base_libdir, max_proce
# A local directory (e.g. an NFS-exported rootfs) rather than a
# user@host ssh target: copy the files in directly, no network needed.
return _deploy_local(args, os.path.realpath(args.target), filelist, ftotalsize, tar_relpaths,
- allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
+ allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir,
+ qemu_helper_native_sysroot=qemu_helper_native_sysroot)
return _deploy_ssh(args, destdir, filelist, ftotalsize, tar_relpaths,
allowed_files, fakerootcmd, fakerootenv, path, recipe_outdir)
diff --git a/scripts/lib/devtool/ide_plugins/__init__.py b/scripts/lib/devtool/ide_plugins/__init__.py
index 0d0117344d..fd66060682 100644
--- a/scripts/lib/devtool/ide_plugins/__init__.py
+++ b/scripts/lib/devtool/ide_plugins/__init__.py
@@ -406,11 +406,27 @@ class IdeBase:
raise err
-def get_devtool_deploy_opts(args):
+def resolve_deploy_target(args, target_override=None):
+ """Resolve the effective devtool deploy-target destination.
+
+ target_override, e.g. the local NFS rootfs directory when --nfs was used
+ (see devtool.deploy for how a directory target is handled without ssh),
+ always takes precedence over the ssh target from -t/--target. Returns
+ None if neither is set.
+ """
+ if target_override:
+ return target_override
+ if args.target:
+ return args.target
+ return None
+
+
+def get_devtool_deploy_opts(args, target_override=None):
"""Filter args for devtool deploy-target args"""
- if not args.target:
+ target = resolve_deploy_target(args, target_override)
+ if not target:
return None
- devtool_deploy_opts = [args.target]
+ devtool_deploy_opts = [target]
if args.no_host_check:
devtool_deploy_opts += ["-c"]
if args.show_status:
diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py b/scripts/lib/devtool/ide_plugins/ide_code.py
index 937c63a4a5..88aa3ff805 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -9,7 +9,8 @@ import json
import logging
import os
import shutil
-from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts
+from devtool import DevtoolError
+from devtool.ide_plugins import BuildTool, IdeBase, GdbCrossConfig, DebuggerServerModes, LldbServerConfig, get_devtool_deploy_opts, resolve_deploy_target
logger = logging.getLogger('devtool')
@@ -593,10 +594,15 @@ class IdeVSCode(IdeBase):
IdeBase.update_json_file(
self.dot_code_dir(modified_recipe), launch_file, launch_dict)
- def vscode_tasks_cpp(self, args, modified_recipe):
- run_install_deploy = modified_recipe.gen_install_deploy_script(args)
+ def vscode_tasks_cpp(self, args, image_recipe, modified_recipe):
+ run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir)
install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty
- deploy_args = ["--target", args.target]
+ deploy_target = resolve_deploy_target(args, image_recipe.nfs_deploy_dir)
+ if not deploy_target:
+ raise DevtoolError('No deploy target available for %s' % modified_recipe.recipe_id_pretty)
+ # Same target the script already has baked in as its default, so this
+ # is a harmless no-op override, but keeps the task self-contained.
+ deploy_args = ["--target", deploy_target]
if args.port:
deploy_args += ["--port", args.port]
for package in args.package or []:
@@ -693,7 +699,7 @@ class IdeVSCode(IdeBase):
args += [target_device.target, remote_cmd]
return args
- def vscode_tasks_kernel_module(self, args, modified_recipe):
+ def vscode_tasks_kernel_module(self, args, image_recipe, modified_recipe):
"""Generate tasks.json for kernel module recipes.
Three tasks are generated and chained in sequence:
@@ -716,8 +722,13 @@ class IdeVSCode(IdeBase):
install_task_name = "install && deploy-target %s" % modified_recipe.recipe_id_pretty
reload_task_name = "reload module %s" % modified_recipe.recipe_id_pretty
verify_task_name = "verify module %s" % modified_recipe.recipe_id_pretty
- run_install_deploy = modified_recipe.gen_install_deploy_script(args)
- deploy_args = ["--target", args.target]
+ run_install_deploy = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir)
+ deploy_target = resolve_deploy_target(args, image_recipe.nfs_deploy_dir)
+ if not deploy_target:
+ raise DevtoolError('No deploy target available for %s' % modified_recipe.recipe_id_pretty)
+ # Same target the script already has baked in as its default, so this
+ # is a harmless no-op override, but keeps the task self-contained.
+ deploy_args = ["--target", deploy_target]
if args.port:
deploy_args += ["--port", args.port]
for package in args.package or []:
@@ -759,7 +770,7 @@ class IdeVSCode(IdeBase):
IdeBase.update_json_file(
self.dot_code_dir(modified_recipe), tasks_file, tasks_dict)
- def vscode_tasks_fallback(self, args, modified_recipe):
+ def vscode_tasks_fallback(self, args, image_recipe, modified_recipe):
oe_init_dir = modified_recipe.oe_init_dir
oe_init = ". %s %s > /dev/null && " % (modified_recipe.oe_init_build_env, modified_recipe.topdir)
dt_build = "devtool build "
@@ -772,7 +783,7 @@ class IdeVSCode(IdeBase):
dt_deploy_label = dt_deploy + modified_recipe.recipe_id_pretty
dt_deploy_cmd = dt_deploy + modified_recipe.bpn
dt_build_deploy_label = "devtool build & deploy-target %s" % modified_recipe.recipe_id_pretty
- deploy_opts = ' '.join(get_devtool_deploy_opts(args))
+ deploy_opts = ' '.join(get_devtool_deploy_opts(args, image_recipe.nfs_deploy_dir))
tasks_dict = {
"version": "2.0.0",
"tasks": [
@@ -892,13 +903,13 @@ class IdeVSCode(IdeBase):
IdeBase.update_json_file(
self.dot_code_dir(modified_recipe), tasks_file, tasks_dict)
- def vscode_tasks(self, args, modified_recipe):
+ def vscode_tasks(self, args, image_recipe, modified_recipe):
if modified_recipe.build_tool.is_c_cpp:
- self.vscode_tasks_cpp(args, modified_recipe)
+ self.vscode_tasks_cpp(args, image_recipe, modified_recipe)
elif modified_recipe.build_tool == BuildTool.KERNEL_MODULE:
- self.vscode_tasks_kernel_module(args, modified_recipe)
+ self.vscode_tasks_kernel_module(args, image_recipe, modified_recipe)
else:
- self.vscode_tasks_fallback(args, modified_recipe)
+ self.vscode_tasks_fallback(args, image_recipe, modified_recipe)
def setup_modified_recipe(self, args, image_recipe, modified_recipe):
self.vscode_settings(modified_recipe, image_recipe)
@@ -912,7 +923,7 @@ class IdeVSCode(IdeBase):
self.initialize_cross_debug_configs(
image_recipe, modified_recipe, GdbCrossConfigVSCode)
self.vscode_launch(args, modified_recipe)
- self.vscode_tasks(args, modified_recipe)
+ self.vscode_tasks(args, image_recipe, modified_recipe)
def register_ide_plugin(ide_plugins):
diff --git a/scripts/lib/devtool/ide_plugins/ide_none.py b/scripts/lib/devtool/ide_plugins/ide_none.py
index e4b255f2fa..559940fda2 100644
--- a/scripts/lib/devtool/ide_plugins/ide_none.py
+++ b/scripts/lib/devtool/ide_plugins/ide_none.py
@@ -312,7 +312,7 @@ class IdeNone(IdeBase):
- Generate a gdbinit file per executable
- Generate the oe-scripts sym-link
"""
- script_path = modified_recipe.gen_install_deploy_script(args)
+ script_path = modified_recipe.gen_install_deploy_script(args, image_recipe.nfs_deploy_dir)
logger.info("Created: %s" % script_path)
if modified_recipe.toolchain == 'clang':
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index ee3c50460a..8090117959 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -26,6 +26,7 @@ from devtool.standard import get_real_srctree
from devtool.deploy import parse_packages_arg
from devtool.ide_plugins import BuildTool, DebuggerCrossConfig
from oe.kernel_module import kernel_module_os_env
+from runqemu_utils import RunQemuRootfsError, extract_sdk_rootfs, pseudo_state_dir
logger = logging.getLogger('devtool')
@@ -212,8 +213,14 @@ class RecipeImage:
def __init__(self, name, orig_bbappend_content=None):
self.name = name
- self.rootfs = None
+ self.pn = None
+ self.__rootfs = None
self.__rootfs_dbg = None
+ self.__nfs_rootfs = None
+ self.__nfs_rootfs_dbg = None
+ self.__nfs_deploy_dir = None
+ self.deploy_dir_image = None
+ self.image_link_name = None
self.qb_slirp_opt = ''
self.bootstrap_tasks = [self.name + ':do_build']
# Debug settings already provided by the base configuration (e.g.
@@ -221,6 +228,7 @@ class RecipeImage:
# content other than devtool ide-sdk's own sections (see
# strip_bbappend_sections()). Populated by initialize().
self.base_image_gen_debugfs = False
+ self.base_image_fstypes = set()
self.base_image_fstypes_debugfs = ''
self.base_has_combined_dbg = False
self.base_image_install = set()
@@ -271,8 +279,11 @@ class RecipeImage:
raise DevtoolError(
"Parsing image recipe %s failed" % self.name)
+ self.pn = image_d.getVar('PN')
self.base_image_gen_debugfs = image_d.getVar(
'IMAGE_GEN_DEBUGFS') == '1'
+ self.base_image_fstypes = set(
+ (image_d.getVar('IMAGE_FSTYPES') or '').split())
self.base_image_fstypes_debugfs = image_d.getVar(
'IMAGE_FSTYPES_DEBUGFS') or ''
self.base_has_combined_dbg = bb.data.inherits_class(
@@ -281,22 +292,75 @@ class RecipeImage:
(image_d.getVar('IMAGE_INSTALL') or '').split())
workdir = image_d.getVar('WORKDIR')
- self.rootfs = os.path.join(workdir, 'rootfs')
+ self.__rootfs = os.path.join(workdir, 'rootfs')
self.__rootfs_dbg = os.path.join(workdir, 'rootfs-dbg')
+ self.deploy_dir_image = image_d.getVar('DEPLOY_DIR_IMAGE')
+ self.image_link_name = image_d.getVar('IMAGE_LINK_NAME')
self.qb_slirp_opt = image_d.getVar('QB_SLIRP_OPT') or ''
@property
def debug_support(self):
return bool(self.rootfs_dbg)
+ @property
+ def rootfs(self):
+ """Prefer the live NFS-exported rootfs (if --nfs=rootfs is used) over the
+ static WORKDIR/rootfs left over from the image build, so solib_search_path()
+ finds files as devtool deploy-target actually updates them."""
+ if self.__nfs_rootfs:
+ return self.__nfs_rootfs
+ return self.__rootfs
+
@property
def rootfs_dbg(self):
+ if self.__nfs_rootfs_dbg:
+ return self.__nfs_rootfs_dbg
if self.__rootfs_dbg and os.path.isdir(self.__rootfs_dbg):
return self.__rootfs_dbg
return None
- def update_image_bbappend(self, recipes_modified):
+ def set_nfs_rootfs(self, workspace_path, nfs):
+ """Select the workspace NFS rootfs for generated debugger paths and deploys."""
+ if not nfs:
+ return
+ self.__nfs_deploy_dir = self.nfs_rootfs_dir(workspace_path, nfs)
+ if nfs == 'rootfs-dbg':
+ self.__nfs_rootfs_dbg = self.__nfs_deploy_dir
+ elif nfs == 'rootfs':
+ self.__nfs_rootfs = self.__nfs_deploy_dir
+
+ @property
+ def nfs_deploy_dir(self):
+ """Directory to deploy modified-recipe files into instead of over ssh, if --nfs was used."""
+ return self.__nfs_deploy_dir
+
+ def nfs_rootfs_dir(self, workspace_path, nfs):
+ """Return the workspace directory for the selected NFS rootfs."""
+ return os.path.join(workspace_path, 'nfs-exports', self.pn, nfs)
+
+ def nfs_runqemu_helper(self, workspace_path, nfs):
+ """Create a helper that boots the selected rootfs through runqemu."""
+ export_dir = os.path.join(workspace_path, 'nfs-exports', self.pn)
+ rootfs_dir = self.nfs_rootfs_dir(workspace_path, nfs)
+ qemuboot = os.path.join(
+ self.deploy_dir_image, self.image_link_name + '.qemuboot.conf')
+ if not os.path.exists(qemuboot):
+ logger.info(
+ 'No qemuboot configuration was generated for %s; '
+ 'not creating a runqemu helper.', self.name)
+ return None
+
+ helper = os.path.join(export_dir, 'runqemu-' + nfs)
+ with open(helper, 'w') as helper_file:
+ helper_file.write('#!/bin/sh\n')
+ helper_file.write(
+ 'exec runqemu %s %s "$@"\n' % (
+ shlex.quote(qemuboot), shlex.quote(rootfs_dir)))
+ os.chmod(helper, os.stat(helper).st_mode | stat.S_IEXEC)
+ return helper
+
+ def update_image_bbappend(self, recipes_modified, nfs=None):
"""Write debug settings for modified-mode recipes into the image bbappend.
Writes IMAGE_GEN_DEBUGFS, IMAGE_FSTYPES_DEBUGFS, IMAGE_CLASSES for
@@ -323,7 +387,17 @@ class RecipeImage:
lines = []
if not self.base_image_gen_debugfs:
lines.append('IMAGE_GEN_DEBUGFS = "1"')
- if self.base_image_fstypes_debugfs != '':
+ if nfs == 'rootfs':
+ if 'tar' not in self.base_image_fstypes:
+ lines.append('IMAGE_FSTYPES:append = " tar"')
+ elif nfs == 'rootfs-dbg':
+ if self.base_image_fstypes_debugfs:
+ if 'tar' not in self.base_image_fstypes_debugfs.split():
+ lines.append('IMAGE_FSTYPES_DEBUGFS:append = " tar"')
+ else:
+ lines.append('IMAGE_FSTYPES_DEBUGFS = "tar"')
+ elif self.base_image_fstypes_debugfs != '':
+ # Without --nfs no debug filesystem image is needed at all.
lines.append('IMAGE_FSTYPES_DEBUGFS = ""')
if not self.base_has_combined_dbg:
lines.append('IMAGE_CLASSES += "image-combined-dbg"')
@@ -368,6 +442,33 @@ class RecipeImage:
slirp_changed = self.update_qb_slirp_opt()
return image_changed or slirp_changed
+ def extract_nfs_rootfs(self, workspace_path, nfs, target):
+ """Refresh the selected rootfs under the devtool workspace."""
+ suffix = '-dbg' if nfs == 'rootfs-dbg' else ''
+ rootfs_tarball = os.path.join(
+ self.deploy_dir_image, self.image_link_name + suffix + '.tar')
+ rootfs_dir = self.nfs_rootfs_dir(workspace_path, nfs)
+ state_dir = pseudo_state_dir(rootfs_dir)
+
+ for stale_dir in (rootfs_dir, state_dir):
+ if os.path.exists(stale_dir):
+ shutil.rmtree(stale_dir)
+
+ try:
+ extract_sdk_rootfs(rootfs_tarball, rootfs_dir)
+ except RunQemuRootfsError as exc:
+ raise DevtoolError('Unable to prepare NFS rootfs: %s' % exc) from exc
+
+ logger.info('NFS rootfs extracted to %s', rootfs_dir)
+ helper = self.nfs_runqemu_helper(workspace_path, nfs)
+ if helper:
+ opts = 'slirp' if is_loopback_target(target) else ''
+ logger.info(
+ 'With the build environment sourced, start QEMU with NFS rootfs:\n'
+ ' %s %s\n'
+ 'Pass any additional runqemu options to this helper.',
+ helper, opts)
+
def update_qb_slirp_opt(self):
"""Update QB_SLIRP_OPT in the image bbappend
@@ -655,6 +756,11 @@ class RecipeModified:
self.reverse_debug_prefix_map = {}
self.fakerootcmd = None
self.fakerootenv = None
+ # STAGING_DIR_NATIVE of qemu-helper-native, needed by deploy_no_d() to
+ # locate the pseudo binary for a local (e.g. NFS-exported) deploy
+ # target without depending on oe-find-native-sysroot / a sourced
+ # build environment at script run time.
+ self.qemu_helper_native_sysroot = None
self.libdir = None
self.max_process = None
self.package_arch = None
@@ -742,6 +848,12 @@ class RecipeModified:
self.debug_build = recipe_d.getVar('DEBUG_BUILD')
self.fakerootcmd = recipe_d.getVar('FAKEROOTCMD')
self.fakerootenv = recipe_d.getVar('FAKEROOTENV')
+ qemu_helper_native_d = parse_recipe(
+ config, tinfoil, 'qemu-helper-native', appends=True, filter_workspace=False)
+ if not qemu_helper_native_d:
+ raise DevtoolError("Parsing qemu-helper-native recipe failed")
+ self.qemu_helper_native_sysroot = os.path.realpath(
+ qemu_helper_native_d.getVar('STAGING_DIR_NATIVE'))
self.libdir = recipe_d.getVar('libdir')
self.max_process = int(recipe_d.getVar(
"BB_NUMBER_THREADS") or os.cpu_count() or 1)
@@ -1342,12 +1454,17 @@ class RecipeModified:
'by the %s recipe (PACKAGES: %s)' %
(package, self.pn, ' '.join(self.packages_files.keys())))
- def gen_deploy_target_script(self, args):
+ def gen_deploy_target_script(self, args, deploy_target=None):
"""Generate a script which does what devtool deploy-target does
This script is much quicker than devtool target-deploy. Because it
does not need to start a bitbake server. All information from tinfoil
is hard-coded in the generated script.
+
+ deploy_target overrides args.target as the baked-in default, e.g. with
+ the local NFS rootfs directory when --nfs was used (see devtool.deploy
+ for how a directory target is handled without ssh). A runtime -t/--target
+ can still override this default, same as without --nfs.
"""
self._validate_requested_packages(args)
cmd_lines = ['#!%s' % str(sys.executable)]
@@ -1362,6 +1479,8 @@ class RecipeModified:
'no_preserve', 'port', 'show_status', 'ssh_exec', 'strip', 'target']
filtered_args_dict = {key: value for key, value in vars(
args).items() if key in args_filter}
+ if deploy_target:
+ filtered_args_dict['target'] = deploy_target
if is_loopback_target(filtered_args_dict['target']):
filtered_args_dict['no_host_check'] = True
cmd_lines.append('filtered_args_dict = %s' % str(filtered_args_dict))
@@ -1394,10 +1513,11 @@ class RecipeModified:
cmd_lines.append(' filtered_args.no_host_check = True')
cmd_lines.append(
'setattr(filtered_args, "recipename", "%s")' % self.bpn)
- cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files)' %
+ cmd_lines.append('deploy_no_d("%s", "%s", "%s", "%s", "%s", "%s", %d, "%s", "%s", filtered_args, file_globs=file_globs, packages_files=packages_files, qemu_helper_native_sysroot="%s")' %
(self.d, self.workdir, self.path, self.strip_cmd,
self.libdir, self.base_libdir, self.max_process,
- self.fakerootcmd, self.fakerootenv))
+ self.fakerootcmd, self.fakerootenv,
+ self.qemu_helper_native_sysroot))
return self.write_script(cmd_lines, 'deploy_target')
def gen_install_task_script(self):
@@ -1421,11 +1541,11 @@ class RecipeModified:
' tinfoil.shutdown()']
return self.write_script(cmd_lines, 'bb_run_do_install')
- def gen_install_deploy_script(self, args):
+ def gen_install_deploy_script(self, args, deploy_target=None):
"""Generate a script which does install and deploy"""
cmd_lines = ['#!/bin/sh -e']
cmd_lines.append(self.gen_install_task_script())
- cmd_lines.append(self.gen_deploy_target_script(args) + ' "$@"')
+ cmd_lines.append(self.gen_deploy_target_script(args, deploy_target) + ' "$@"')
return self.write_script(cmd_lines, 'install_and_deploy')
@@ -1528,6 +1648,9 @@ def ide_setup(args, config, basepath, workspace):
logger.error("In shared sysroots mode modified recipes %s cannot be handled." % str(
recipes_modified_names))
invalid_params = True
+ if args.nfs:
+ logger.error("--nfs is only supported in modified mode.")
+ invalid_params = True
if args.mode == DevtoolIdeMode.modified:
if not recipes_modified_names:
appends_dir = os.path.join(config.workspace_path, 'appends')
@@ -1570,6 +1693,7 @@ def ide_setup(args, config, basepath, workspace):
recipes_image_name,
orig_bbappend_contents.get(recipes_image_name))
recipe_image.initialize(config, tinfoil)
+ recipe_image.set_nfs_rootfs(config.workspace_path, args.nfs)
if args.mode == DevtoolIdeMode.modified:
# Keep the image build separate so that the complete bbappend
# (IMAGE_ vars + QB_SLIRP_OPT) can be written in one step
@@ -1675,7 +1799,7 @@ def ide_setup(args, config, basepath, workspace):
# removed by strip_bbappend_sections() would be lost.
bbappend_changed = False
for ri in recipes_images:
- if ri.update_image_bbappend(recipes_modified):
+ if ri.update_image_bbappend(recipes_modified, args.nfs):
bbappend_changed = True
if not args.skip_bitbake:
@@ -1701,6 +1825,10 @@ def ide_setup(args, config, basepath, workspace):
exec_build_env_command(
config.init_path, basepath,
bb_cmd + ' '.join(image_bootstrap_tasks), watch=True)
+
+ if args.nfs and not args.skip_bitbake:
+ for ri in recipes_images:
+ ri.extract_nfs_rootfs(config.workspace_path, args.nfs, args.target)
else:
raise DevtoolError("Must not end up here.")
@@ -1775,6 +1903,11 @@ def register_commands(subparsers, context):
'-P', '--port', help='Specify ssh port to use for connection to the target')
parser_ide_sdk.add_argument(
'-I', '--key', help='Specify ssh private key for connection to the target')
+ parser_ide_sdk.add_argument(
+ '--nfs', choices=('rootfs', 'rootfs-dbg'),
+ help='Build and extract the selected image rootfs below '
+ '<workspace>/nfs-exports/<image-PN>/ '
+ 'for NFS booting.')
parser_ide_sdk.add_argument(
'--skip-bitbake', help='Skip the bitbake builds which update the SDK. The recipes are still parsed, '
'the IDE configuration is generated from their metadata', action='store_true')
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 24/25] oe-selftest: devtool ide-sdk: test NFS debug rootfs
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (22 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 23/25] devtool: ide-sdk: support NFS rootfs AdrianF
@ 2026-08-30 21:48 ` AdrianF
2026-08-30 21:48 ` [PATCH v2 25/25] devtool: ide-sdk: fix GDB loading stale libs instead of recipe's own build AdrianF
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Exercise --nfs=rootfs-dbg with a real image build and verify the
extracted rootfs, pseudo state, and generated VS Code debug paths.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 118 ++++++++++++++++++++++++
1 file changed, 118 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index c05cf7bc82..8d68809552 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3539,8 +3539,18 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
r = runCmd(gdbserver_script, output_log=self._cmd_logger)
self.assertEqual(r.status, 0)
+ # The start script already waited for gdbserver's port to come up
+ # before returning, so the pid file it wrote must exist by now. Still
+ # retry the read a few times: on an NFS-root target (--nfs=rootfs-dbg)
+ # this check runs over a fresh SSH connection right after the one that
+ # wrote the file, and a brief visibility lag has been observed there.
pid_file = '/tmp/gdbserver_1234_usr-bin-%s_multi/gdbserver.pid' % example_exe
status, output = qemu.run('cat %s' % pid_file)
+ for _ in range(10):
+ if status == 0:
+ break
+ time.sleep(1)
+ status, output = qemu.run('cat %s' % pid_file)
self.assertEqual(status, 0)
gdbserver_pid = output.strip()
self.assertRegex(gdbserver_pid, r'^\d+$')
@@ -3974,6 +3984,114 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
self.assertEqual(r.status, 0)
debug_check_func(r.output, DevtoolIdeSdkTests.MAGIC_STRING_ORIG)
+ def test_devtool_ide_sdk_code_nfs_debug_rootfs(self):
+ """Verify ide-sdk extracts an NFS debug rootfs for VS Code debugging."""
+ recipe_name = "cmake-example"
+ build_file = "CMakeLists.txt"
+ testimage = "oe-selftest-image"
+
+ self._check_workspace()
+ self._write_bb_config()
+ tempdir = self._devtool_ide_sdk_recipe(
+ recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=code --nfs=rootfs-dbg' %
+ (recipe_name, testimage), output_log=self._cmd_logger)
+
+ nfs_rootfs = os.path.join(
+ self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg')
+ self.assertExists(nfs_rootfs)
+ self.assertExists(nfs_rootfs + '.pseudo_state')
+ self.assertExists(os.path.join(nfs_rootfs, 'usr', 'bin', recipe_name))
+ runqemu_helper = os.path.join(
+ self.workspacedir, 'nfs-exports', testimage,
+ 'runqemu-rootfs-dbg')
+ self.assertExists(runqemu_helper)
+ self.assertTrue(os.access(runqemu_helper, os.X_OK))
+ with open(runqemu_helper) as helper_file:
+ helper = helper_file.read()
+ self.assertIn('exec runqemu ', helper)
+ self.assertIn(nfs_rootfs, helper)
+
+ bbappend = os.path.join(
+ self.workspacedir, 'appends', testimage + '.bbappend')
+ with open(bbappend) as append_file:
+ self.assertRegex(
+ append_file.read(),
+ r'IMAGE_FSTYPES_DEBUGFS(?::append)? = " ?tar"')
+
+ with open(os.path.join(tempdir, '.vscode', 'launch.json')) as launch_file:
+ launch_configurations = json.load(launch_file)['configurations']
+ for configuration in launch_configurations:
+ self.assertIn(
+ nfs_rootfs, configuration['additionalSOLibSearchPath'])
+ self.assertEqual(
+ os.path.join(nfs_rootfs, 'usr', 'src', 'debug'),
+ configuration['sourceFileMap']['/usr/src/debug'])
+
+ def _test_devtool_ide_sdk_nfs_debug_rootfs_qemu(self, slirp=False):
+ """Boot the extracted NFS debug rootfs and exercise remote GDB."""
+ recipe_name = "cmake-example"
+ build_file = "CMakeLists.txt"
+ testimage = "oe-selftest-image"
+
+ self._check_workspace()
+ self._write_bb_config()
+ if not slirp:
+ self._check_runqemu_prerequisites()
+
+ tempdir = self._devtool_ide_sdk_recipe(
+ recipe_name, build_file, testimage)
+ runCmd('devtool ide-sdk %s %s -c --ide=none --nfs=rootfs-dbg' %
+ (recipe_name, testimage),
+ output_log=self._cmd_logger)
+
+ nfs_rootfs = os.path.join(
+ self.workspacedir, 'nfs-exports', testimage, 'rootfs-dbg')
+ runqemu_helper = os.path.join(
+ self.workspacedir, 'nfs-exports', testimage,
+ 'runqemu-rootfs-dbg')
+ self.assertExists(nfs_rootfs)
+ self.assertExists(runqemu_helper)
+
+ launch_cmd = '%s nographic' % shlex.quote(runqemu_helper)
+ runqemuparams = ''
+ if slirp:
+ launch_cmd += ' slirp'
+ # QemuTarget uses this only to select its localhost SSH endpoint;
+ # launch_cmd supplies the actual runqemu option.
+ runqemuparams = 'slirp'
+
+ with runqemu(testimage, runqemuparams=runqemuparams,
+ launch_cmd=launch_cmd) as qemu:
+ status, output = qemu.run(
+ "awk '$2 == \"/\" {print $3}' /proc/mounts")
+ self.assertEqual(status, 0)
+ self.assertEqual(output.strip(), 'nfs')
+
+ # Re-run with the real QEMU target address, instead of 192.168.7.2 IP
+ if slirp:
+ target_options = '-t root@%s -P %s ' % (qemu.ip, qemu.port or '2222')
+ else:
+ target_options = '-t root@%s -c ' % qemu.ip
+ runCmd('devtool ide-sdk %s %s %s--skip-bitbake --ide=none --nfs=rootfs-dbg' %
+ (recipe_name, testimage, target_options),
+ output_log=self._cmd_logger)
+
+ self._gdb_cross()
+ compile_cmd = self._verify_cmake_preset(tempdir)
+ self._devtool_ide_sdk_qemu(
+ tempdir, qemu, recipe_name, recipe_name, compile_cmd)
+
+ @OETestTag("runqemu")
+ def test_devtool_ide_sdk_none_nfs_qemu(self):
+ """Verify remote GDB debugging through an NFS-root QEMU target."""
+ self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu()
+
+ @OETestTag("runqemu")
+ def test_devtool_ide_sdk_none_nfs_qemu_slirp(self):
+ """Verify remote GDB debugging through an NFS-root slirp target."""
+ self._test_devtool_ide_sdk_nfs_debug_rootfs_qemu(slirp=True)
+
@OETestTag("runqemu")
def test_devtool_ide_sdk_code_cmake(self):
"""Verify a cmake recipe works with ide=code mode"""
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* [PATCH v2 25/25] devtool: ide-sdk: fix GDB loading stale libs instead of recipe's own build
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
` (23 preceding siblings ...)
2026-08-30 21:48 ` [PATCH v2 24/25] oe-selftest: devtool ide-sdk: test NFS debug rootfs AdrianF
@ 2026-08-30 21:48 ` AdrianF
24 siblings, 0 replies; 31+ messages in thread
From: AdrianF @ 2026-08-30 21:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Adrian Freihofer
From: Adrian Freihofer <adrian.freihofer@siemens.com>
Setting GDB's sysroot to the recipe's D (image/) directory is often not
enough: D only contains "usr/lib", but the target reports merged-usr
paths like "/lib/libfoo.so.0", so the sysroot lookup fails and GDB
falls back to solib-search-path, resolving the library from the full
image's rootfs-dbg/rootfs instead. Recipes that install libraries
outside the standard libdir hit this on every rebuild, silently
debugging with stale symbols from the last full image build. Confirmed
against gdb-17.2 gdb/solib.c (solib_find_1()) by capturing the GDB MI
trace of an attach session and cross-checking against that code.
Fix: add RecipeModified._find_elf_dirs(), which walks D and returns
every directory containing an ELF file, and prepend those directories
to additionalSOLibSearchPath, ahead of the rootfs-dbg/rootfs entries,
in every generated launch config.
Also extend the launch.json oe-selftest assertions to check that the
first additionalSOLibSearchPath entry is the recipe's own image (D)
directory, not rootfs-dbg.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 6 ++++
scripts/lib/devtool/ide_sdk.py | 42 ++++++++++++++++++++-----
2 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 8d68809552..7d7d228b22 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -3733,6 +3733,12 @@ class DevtoolIdeSdkGccTests(DevtoolIdeSdkTests):
so_paths = config.get("additionalSOLibSearchPath", [])
self.assertIn("/.debug", so_paths, f"Configuration '{config['name']}' should include debug symbol paths")
self.assertIn("/rootfs-dbg/", so_paths, f"Configuration '{config['name']}' should include rootfs-dbg paths")
+ # The recipe's own D/usr/lib must come first, so GDB's solib-search-path
+ # basename fallback prefers freshly rebuilt libraries over a stale
+ # rootfs-dbg/rootfs copy from the last full image build.
+ first_so_path = so_paths.split(":", 1)[0]
+ self.assertIn("/image/", first_so_path, f"Configuration '{config['name']}' first SO lib search path should be the recipe's own image dir: {first_so_path}")
+ self.assertNotIn("rootfs-dbg", first_so_path, f"Configuration '{config['name']}' first SO lib search path should not be rootfs-dbg: {first_so_path}")
# Verify source file mappings
source_map = config.get("sourceFileMap", {})
diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
index 8090117959..c2e7a34b7a 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -939,27 +939,53 @@ class RecipeModified:
return True
return False
+ @staticmethod
+ def _find_elf_dirs(root):
+ """Return every directory under root that contains an ELF file"""
+ elf_dirs = []
+ for dirpath, _dirnames, filenames in os.walk(root):
+ for filename in filenames:
+ file_path = os.path.join(dirpath, filename)
+ if os.path.islink(file_path):
+ continue
+ try:
+ with open(file_path, 'rb') as elf_file:
+ is_elf = elf_file.read(4) == b'\x7fELF'
+ except OSError:
+ continue
+ if is_elf:
+ elf_dirs.append(dirpath)
+ break
+ return elf_dirs
+
def solib_search_path(self, image):
- """Search for debug symbols in the rootfs and rootfs-dbg
+ """Search for debug symbols
The debug symbols of shared libraries which are provided by other packages
are grabbed from the -dbg packages in the rootfs-dbg.
-
But most cross debugging tools like gdb, perf, and systemtap need to find
executable/library first and through it debuglink note find corresponding
symbols file. Therefore the library paths from the rootfs are added as well.
Note: For the devtool modified recipe compiled from the IDE, the debug
- symbols are taken from the unstripped binaries in the image folder.
- Also, devtool deploy-target takes the files from the image folder.
- debug symbols in the image folder refer to the corresponding source files
- with absolute paths of the build machine. Debug symbols found in the
- rootfs-dbg are relocated and contain paths which refer to the source files
- installed on the target device e.g. /usr/src/...
+ symbols are taken from the unstripped binaries in the image=${D} directory.
+ Also, devtool deploy-target takes the files from the image directory.
+ This recipe's own D is searched (for ELF files, wherever they are
+ installed) and listed first, ahead of rootfs-dbg/rootfs, so that this
+ recipe's freshly rebuilt libraries take precedence over any stale copy
+ from the last full image build. "set sysroot D" alone cannot achieve
+ this: the target reports merged-usr paths (e.g. "/lib/libfoo.so.0"), but
+ D only contains "usr/lib" (raw do_install output, no top-level "lib"
+ symlink), so GDB's sysroot-prefixed lookup always fails open() and it
+ falls back to solib-search-path's basename matching instead. This also
+ does not work for libraries which are not installed into the standard
+ libdirs.
"""
base_libdir = self.base_libdir.lstrip('/')
libdir = self.libdir.lstrip('/')
so_paths = [
+ # This recipe's own rebuilt libraries/plugins, ahead of rootfs-dbg/rootfs.
+ *self._find_elf_dirs(self.d),
# debug symbols for package_debug_split_style: debug-with-srcpkg or .debug
os.path.join(image.rootfs_dbg, base_libdir, ".debug"),
os.path.join(image.rootfs_dbg, libdir, ".debug"),
--
2.55.0
^ permalink raw reply related [flat|nested] 31+ messages in thread
* Patchtest results for [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters
2026-08-30 21:48 ` [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters AdrianF
@ 2026-08-30 22:11 ` patchtest
0 siblings, 0 replies; 31+ messages in thread
From: patchtest @ 2026-08-30 22:11 UTC (permalink / raw)
To: AdrianF; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 3208 bytes --]
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:
---
Testing patch /home/patchtest/share/mboxes/v2-11-25-devtool-deploy-target-add---package---file-glob-filters.patch
FAIL: test max line length: Patch line too long (current length 534, maximum is 200) (test_metadata.TestMetadata.test_max_line_length)
PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping test (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [OE-core] [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries
2026-08-30 21:48 ` [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries AdrianF
@ 2026-09-03 11:18 ` Richard Purdie
2026-09-03 16:06 ` Freihofer, Adrian
0 siblings, 1 reply; 31+ messages in thread
From: Richard Purdie @ 2026-09-03 11:18 UTC (permalink / raw)
To: adrian.freihofer, openembedded-core
On Sun, 2026-08-30 at 23:48 +0200, Adrian Freihofer via lists.openembedded.org wrote:
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
>
> With usrmerge in DISTRO_FEATURES (the default), root_prefix becomes
> exec_prefix, so base_libdir and libdir resolve to the same path. This
> made most entries of solib_search_path() literal duplicates. Dedupe
> while preserving order.
>
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
> scripts/lib/devtool/ide_sdk.py | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py
> index 4fdeeec24d..6f08940c9c 100755
> --- a/scripts/lib/devtool/ide_sdk.py
> +++ b/scripts/lib/devtool/ide_sdk.py
> @@ -643,7 +643,8 @@ class RecipeModified:
> os.path.join(image.rootfs, base_libdir),
> os.path.join(image.rootfs, libdir)
> ]
> - return so_paths
> + # base_libdir == libdir when usrmerge is in DISTRO_FEATURES (the default), dedupe
> + return list(dict.fromkeys(so_paths))
This is a little hard to read, would set(so_paths) work here?
Cheers,
Richard
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [OE-core] [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python
2026-08-30 21:48 ` [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python AdrianF
@ 2026-09-03 11:36 ` Richard Purdie
0 siblings, 0 replies; 31+ messages in thread
From: Richard Purdie @ 2026-09-03 11:36 UTC (permalink / raw)
To: adrian.freihofer, openembedded-core
On Sun, 2026-08-30 at 23:48 +0200, Adrian Freihofer via lists.openembedded.org wrote:
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
>
> Move the pseudo-backed rootfs extraction logic into a reusable Python
> module while preserving the existing command interface.
> Install the module with qemu-helper so the wrapper also works from an SDK.
>
> AI-Generated: Uses GitHub Copilot
>
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
> .../qemu/nativesdk-qemu-helper_1.0.bb | 3 +
> scripts/lib/runqemu_utils.py | 124 ++++++++++++++++++
> scripts/runqemu-extract-sdk | 103 +--------------
> 3 files changed, 133 insertions(+), 97 deletions(-)
> create mode 100644 scripts/lib/runqemu_utils.py
>
> diff --git a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> index e1f343989a..16fc07fadf 100644
> --- a/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> +++ b/meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb
> @@ -9,6 +9,7 @@ RDEPENDS:${PN} = "nativesdk-qemu nativesdk-unfs3 nativesdk-pseudo \
> LIC_FILES_CHKSUM = "file://${COREBASE}/scripts/runqemu;beginline=5;endline=10;md5=ac2b489a58739c7628a2604698db5e7f"
>
> SRC_URI = "file://${COREBASE}/scripts/runqemu \
> + file://${COREBASE}/scripts/lib/runqemu_utils.py \
> file://${COREBASE}/scripts/runqemu-addptable2image \
> file://${COREBASE}/scripts/runqemu-gen-tapdevs \
> file://${COREBASE}/scripts/runqemu-ifup \
> @@ -30,4 +31,6 @@ do_install() {
> install -d ${D}${bindir}
> install -m 0755 ${S}${COREBASE}/scripts/oe-* ${D}${bindir}/
> install -m 0755 ${S}${COREBASE}/scripts/runqemu* ${D}${bindir}/
> + # The runqemu-* wrappers import this module from their own directory.
> + install -m 0644 ${S}${COREBASE}/scripts/lib/runqemu_utils.py ${D}${bindir}/
> }
> diff --git a/scripts/lib/runqemu_utils.py b/scripts/lib/runqemu_utils.py
> new file mode 100644
> index 0000000000..dff0ca8d3a
> --- /dev/null
> +++ b/scripts/lib/runqemu_utils.py
> @@ -0,0 +1,124 @@
> +#!/usr/bin/env python3
> +#
> +# Helpers shared by runqemu and NFS-rootfs preparation tools.
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +"""Extract and export rootfs tarballs for NFS booting."""
> +
> +import os
> +import subprocess
> +import sys
> +
> +
> +class RunQemuRootfsError(Exception):
> + """Raised when an NFS rootfs cannot be prepared or exported."""
> +
> +
> +def native_environment():
> + """Return the qemu-helper-native environment set by oe-find-native-sysroot."""
> + command = '''
> + helper=$(command -v oe-find-native-sysroot) || exit 127
> + . "$helper" qemu-helper-native >/dev/null
> + env -0
> + printf 'PSEUDO=%s\\0OECORE_NATIVE_SYSROOT=%s\\0' "$PSEUDO" "$OECORE_NATIVE_SYSROOT"
> + '''
> + result = subprocess.run(['bash', '-c', command], capture_output=True)
> + if result.returncode:
> + if result.returncode == 127:
> + raise RunQemuRootfsError(
> + 'Unable to find the oe-find-native-sysroot script.\n'
> + 'Did you forget to source your build system environment setup script?')
> + raise RunQemuRootfsError(
> + result.stderr.decode(errors='replace').strip() or
> + 'Unable to set up the qemu-helper-native sysroot')
> +
> + environment = {}
> + for entry in result.stdout.split(b'\0'):
> + if b'=' in entry:
> + key, value = entry.split(b'=', 1)
> + environment[key.decode()] = value.decode()
> + return environment
This takes a relatively simple shell script and turns it into python
but the python is very machine generated and probably not great for
readability or maintainability in the future.
For example, in the above we're using the shell to find oe-find-native-
sysroot. Instead we could use shutil.which(). That would make the code
a lot more pythonic and readable. It also special cases some variables
specifically so it isn't quite a generic function but it is named
generically.
> +def _tar_options(rootfs_tarball):
> + if rootfs_tarball.endswith('.tar.xz'):
> + return ['--numeric-owner', '-xJf']
> + if rootfs_tarball.endswith('.tar.bz2'):
> + return ['--numeric-owner', '-xjf']
> + if rootfs_tarball.endswith('.tar.gz'):
> + return ['--numeric-owner', '-xzf']
> + if rootfs_tarball.endswith('.tar.zst'):
> + return ['--numeric-owner', '--zstd', '-xf']
> + if rootfs_tarball.endswith('.tar'):
> + return ['--numeric-owner', '-xf']
> + raise RunQemuRootfsError(
> + 'Unable to determine sdk tarball format\n'
> + 'Accepted types: .tar / .tar.gz / .tar.bz2 / .tar.xz / .tar.zst')
A human doing this would create a table and not duplicate the data. Its
a pain to maintain in future. I appreciate it matches the shell code
but if we're going to do it, we may as well do it well.
> +def pseudo_state_dir(rootfs_dir):
> + """Return the pseudo database location associated with an extracted rootfs."""
> + rootfs_dir = os.path.realpath(rootfs_dir)
> + return os.path.join(os.path.dirname(rootfs_dir),
> + os.path.basename(rootfs_dir) + '.pseudo_state')
os.path.join(os.path.dirname(rootfs_dir), os.path.basename(rootfs_dir)) ?
really? :)
I think this needs a bit more human TLC before it could be merged...
Cheers,
Richard
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [OE-core] [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries
2026-09-03 11:18 ` [OE-core] " Richard Purdie
@ 2026-09-03 16:06 ` Freihofer, Adrian
2026-09-03 20:21 ` Richard Purdie
0 siblings, 1 reply; 31+ messages in thread
From: Freihofer, Adrian @ 2026-09-03 16:06 UTC (permalink / raw)
To: Richard Purdie, openembedded-core@lists.openembedded.org
On Thu, 2026-09-03 at 12:18 +0100, Richard Purdie wrote:
> On Sun, 2026-08-30 at 23:48 +0200, Adrian Freihofer via
> lists.openembedded.org wrote:
> > From: Adrian Freihofer <adrian.freihofer@siemens.com>
> >
> > With usrmerge in DISTRO_FEATURES (the default), root_prefix becomes
> > exec_prefix, so base_libdir and libdir resolve to the same path.
> > This
> > made most entries of solib_search_path() literal duplicates. Dedupe
> > while preserving order.
> >
> > Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> > ---
> > scripts/lib/devtool/ide_sdk.py | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/lib/devtool/ide_sdk.py
> > b/scripts/lib/devtool/ide_sdk.py
> > index 4fdeeec24d..6f08940c9c 100755
> > --- a/scripts/lib/devtool/ide_sdk.py
> > +++ b/scripts/lib/devtool/ide_sdk.py
> > @@ -643,7 +643,8 @@ class RecipeModified:
> > os.path.join(image.rootfs, base_libdir),
> > os.path.join(image.rootfs, libdir)
> > ]
> > - return so_paths
> > + # base_libdir == libdir when usrmerge is in
> > DISTRO_FEATURES (the default), dedupe
> > + return list(dict.fromkeys(so_paths))
>
> This is a little hard to read, would set(so_paths) work here?
The order of these paths is essential. Therefore set would not work.
I'm working on a v3 which will improve the comments for this
complicated function as much as possible.
Thanks you
Regards,
Adrian
>
> Cheers,
>
> Richard
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [OE-core] [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries
2026-09-03 16:06 ` Freihofer, Adrian
@ 2026-09-03 20:21 ` Richard Purdie
0 siblings, 0 replies; 31+ messages in thread
From: Richard Purdie @ 2026-09-03 20:21 UTC (permalink / raw)
To: Freihofer, Adrian, openembedded-core@lists.openembedded.org
On Thu, 2026-09-03 at 16:06 +0000, Freihofer, Adrian wrote:
> On Thu, 2026-09-03 at 12:18 +0100, Richard Purdie wrote:
> > On Sun, 2026-08-30 at 23:48 +0200, Adrian Freihofer via
> > lists.openembedded.org wrote:
> > > From: Adrian Freihofer <adrian.freihofer@siemens.com>
> > >
> > > With usrmerge in DISTRO_FEATURES (the default), root_prefix becomes
> > > exec_prefix, so base_libdir and libdir resolve to the same path.
> > > This
> > > made most entries of solib_search_path() literal duplicates. Dedupe
> > > while preserving order.
> > >
> > > Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> > > ---
> > > scripts/lib/devtool/ide_sdk.py | 3 ++-
> > > 1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/scripts/lib/devtool/ide_sdk.py
> > > b/scripts/lib/devtool/ide_sdk.py
> > > index 4fdeeec24d..6f08940c9c 100755
> > > --- a/scripts/lib/devtool/ide_sdk.py
> > > +++ b/scripts/lib/devtool/ide_sdk.py
> > > @@ -643,7 +643,8 @@ class RecipeModified:
> > > os.path.join(image.rootfs, base_libdir),
> > > os.path.join(image.rootfs, libdir)
> > > ]
> > > - return so_paths
> > > + # base_libdir == libdir when usrmerge is in
> > > DISTRO_FEATURES (the default), dedupe
> > > + return list(dict.fromkeys(so_paths))
> >
> > This is a little hard to read, would set(so_paths) work here?
>
> The order of these paths is essential. Therefore set would not work.
>
> I'm working on a v3 which will improve the comments for this
> complicated function as much as possible.
I wondered if order was important. I think in that case I'd change the
way you're building it in the code above to use a dict, assume dict
ordering and then just make a list from that...
Cheers,
Richard
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2026-09-03 20:21 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-30 21:48 [PATCH v2 00/25] devtool: ide-sdk: NFS/slirp support, deploy filtering, and robustness fixes AdrianF
2026-08-30 21:48 ` [PATCH v2 01/25] runqemu-extract-sdk: set PSEUDO_INCLUDE_PATHS for the extraction AdrianF
2026-08-30 21:48 ` [PATCH v2 02/25] devtool: ide-sdk: dedupe solib_search_path entries AdrianF
2026-09-03 11:18 ` [OE-core] " Richard Purdie
2026-09-03 16:06 ` Freihofer, Adrian
2026-09-03 20:21 ` Richard Purdie
2026-08-30 21:48 ` [PATCH v2 03/25] devtool: ide-sdk: VSCode IntelliSense for rootfs-dbg sources AdrianF
2026-08-30 21:48 ` [PATCH v2 04/25] cpp-example: fix stuck breakpoints when attaching and daemonize properly AdrianF
2026-08-30 21:48 ` [PATCH v2 05/25] devtool: ide-sdk: auto-write image debug settings to bbappend AdrianF
2026-08-30 21:48 ` [PATCH v2 06/25] oe-selftest: devtool ide-sdk: adapt tests for auto-written image debug settings AdrianF
2026-08-30 21:48 ` [PATCH v2 07/25] oeqa: QemuTarget: set use_slirp when slirp is in runqemuparams AdrianF
2026-08-30 21:48 ` [PATCH v2 08/25] devtool: ide-sdk: support runqemu slirp AdrianF
2026-08-30 21:48 ` [PATCH v2 09/25] devtool: ide-sdk: auto-disable ssh host key checking for loopback targets AdrianF
2026-08-30 21:48 ` [PATCH v2 10/25] oe-selftest: devtool ide-sdk: add slirp networking test AdrianF
2026-08-30 21:48 ` [PATCH v2 11/25] devtool: deploy-target: add --package/--file-glob filters AdrianF
2026-08-30 22:11 ` Patchtest results for " patchtest
2026-08-30 21:48 ` [PATCH v2 12/25] oe-selftest: devtool deploy-target: test " AdrianF
2026-08-30 21:48 ` [PATCH v2 13/25] devtool: ide-sdk: forward --package/--file-glob deploy filters AdrianF
2026-08-30 21:48 ` [PATCH v2 14/25] oe-selftest: devtool ide-sdk: cover --package filters AdrianF
2026-08-30 21:48 ` [PATCH v2 15/25] runqemu-extract-sdk: refactor in Python AdrianF
2026-09-03 11:36 ` [OE-core] " Richard Purdie
2026-08-30 21:48 ` [PATCH v2 16/25] runqemu-export-rootfs: " AdrianF
2026-08-30 21:48 ` [PATCH v2 17/25] devtool: deploy: split ssh deployment into a separate function AdrianF
2026-08-30 21:48 ` [PATCH v2 18/25] devtool: deploy-target: allow deploying directly into a local rootfs AdrianF
2026-08-30 21:48 ` [PATCH v2 19/25] oe-selftest: devtool deploy-target: test deploying into a local rootfs path AdrianF
2026-08-30 21:48 ` [PATCH v2 20/25] oe-selftest: tinfoil: test prepared task runner AdrianF
2026-08-30 21:48 ` [PATCH v2 21/25] devtool: ide-sdk: run do_install through BitBake AdrianF
2026-08-30 21:48 ` [PATCH v2 22/25] oe-selftest: devtool ide-sdk: test install task AdrianF
2026-08-30 21:48 ` [PATCH v2 23/25] devtool: ide-sdk: support NFS rootfs AdrianF
2026-08-30 21:48 ` [PATCH v2 24/25] oe-selftest: devtool ide-sdk: test NFS debug rootfs AdrianF
2026-08-30 21:48 ` [PATCH v2 25/25] devtool: ide-sdk: fix GDB loading stale libs instead of recipe's own build AdrianF
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.