* [yocto-autobuilder-helper][PATCH 00/11] Implement 'containers' jobs
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
This series contains the 'yocto-autobuilder-helper' patches to enable:
* building, testing and publishing of 'vcontainer-tarball'
- the tarball is checked to see if a newer version exists and if so
extracts the new version before testing
- vcontainer-tests, vdkr-tests and vpdmn-tests are run as individual
jobs
* building and pushing containers from:
- meta-virtualization
- meta-yocto-containers-demo
* containers are pushed when the push_containers boolean is set in the
AutoBuilder UI
- regstries are set with CONTAINER_REGISTRIES variable
* added getconfigdict() to scripts/utils to allow container names to mapped
from recipe name to a more commonly used upstream container name (e.g.
app-container-python -> python)
* authentication uses local user credentials (e.g. ~/.docker/config.json
or .../auth.json for podman)
* containers are tagged with PV_MAJOR, PV_MAJOR.PV_MINOR, DISTRO_CODENAME
and yocto-x.y release (yocto-x.y.z for release builds).
Additional features, such as container signing and attaching SLSA provenance
attestations will come in a future series, once this initial functionality
is agreed upon an stable.
This series depends on a related series for 'yocto-autobuilder2'.
The following changes since commit bfc6a42aa4df89878a1b0d950c7d0146ce20672c:
publish-artefacts: Update to include newer machines (2026-05-06 17:33:36 +0100)
are available in the Git repository at:
https://git.yoctoproject.org/yocto-autobuilder-helper contrib/timo/containers-upstream
https://git.yoctoproject.org/yocto-autobuilder-helper/log/?h=contrib/timo/containers-upstream
for you to fetch changes up to 601c281165450b2be14de46bb9601e82229ca5f6:
config.json: containers-library: add mosquitto and valkey demos (2026-05-07 12:53:00 -0700)
----------------------------------------------------------------
Tim Orling (11):
scripts/utils: fix stale extraction dir when tarball is updated
scripts/utils: warn and force re-download for HTTPS sources without
SHA256
config.json: add sha256sum for BUILDTOOLS URLs
scripts/utils: add getconfigdict() for dict-type config values
scripts: add vcontainer-tarball setup, integration, and publishing
config.json: add vcontainer-tarball build target
config.json: add 'containers-' build jobs
scripts: add run-vcontainer-tests for meta-virtualization
config.json: add vcontainer-tests, vdkr-tests, vpdmn-tests jobs
scripts: add container registry push, auth, tagging, runtime selection
config.json: containers-library: add mosquitto and valkey demos
config.json | 151 ++++++++++++++++++++++++++++++--
scripts/publish-artefacts | 5 ++
scripts/run-config | 147 +++++++++++++++++++++++++++++++
scripts/run-vcontainer-tests | 164 +++++++++++++++++++++++++++++++++++
scripts/shared-repo-unpack | 1 +
scripts/utils.py | 109 +++++++++++++++++------
6 files changed, 545 insertions(+), 32 deletions(-)
create mode 100755 scripts/run-vcontainer-tests
--
2.43.0
^ permalink raw reply [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 01/11] scripts/utils: fix stale extraction dir when tarball is updated
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Previously the entire download/cache-validation block in
setup_tools_tarball() was guarded by `if not os.path.exists(btdir)`.
Once the extraction directory existed from a prior build, every
subsequent call was a no-op: the cached tarball was never re-validated,
so a freshly-published SDK (e.g. vcontainer-tarball-latest) was silently
ignored and the stale btdir kept being used.
Fix by moving the lock/download block outside the btdir existence guard
so cache validation always runs. Track tarball_updated to know when the
cached download was actually replaced, then remove the stale btdir when
True so the fresh tarball is re-extracted.
Also add an mtime-based staleness check for local-path (cp) sources so
that a newer source file automatically invalidates the cached copy.
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 64 +++++++++++++++++++++++++++++++-----------------
1 file changed, 41 insertions(+), 23 deletions(-)
diff --git a/scripts/utils.py b/scripts/utils.py
index 88842f0..87acad6 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -481,31 +481,49 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
if ";" in bttarball:
bttarball, sha256 = bttarball.split(";")
btdir = os.path.abspath(btdir)
+ btdlpath = getconfig("BASE_SHAREDDIR", ourconfig) + "/cluster-downloads-cache/" + os.path.basename(bttarball)
+ btlock = btdlpath + ".lock"
+ if not os.path.exists(os.path.dirname(btdlpath)):
+ os.makedirs(os.path.dirname(btdlpath), exist_ok=True)
+ # Always run the cache-validation / download step so that a freshly
+ # published tarball is detected even when btdir already exists from a
+ # previous build. tarball_updated is set to True whenever the cached
+ # download is replaced, which triggers removal of the stale btdir.
+ tarball_updated = False
+ while True:
+ try:
+ with open(btlock, 'a+') as lf:
+ fileno = lf.fileno()
+ fcntl.flock(fileno, fcntl.LOCK_EX)
+ if sha256 and os.path.exists(btdlpath):
+ dl_sha256 = sha256_file(btdlpath)
+ if dl_sha256 != sha256:
+ os.unlink(btdlpath)
+ elif bttarball.startswith("/") and os.path.exists(btdlpath):
+ # For local-path sources (e.g. vcontainer-tarball-latest)
+ # invalidate the cached copy when the source is newer so
+ # that a freshly-published tarball is always picked up.
+ if os.path.getmtime(bttarball) > os.path.getmtime(btdlpath):
+ os.unlink(btdlpath)
+ os.unlink(btdlpath)
+ if not os.path.exists(btdlpath):
+ if bttarball.startswith("/"):
+ subprocess.check_call(["cp", bttarball, btdlpath])
+ else:
+ subprocess.check_call(["wget", "-O", btdlpath, bttarball])
+ os.chmod(btdlpath, 0o775)
+ tarball_updated = True
+ break
+ except OSError:
+ # We raced with someone else, try again
+ pass
+ # If the underlying tarball changed, remove any stale extraction
+ # directory so it is re-extracted below.
+ if tarball_updated and os.path.exists(btdir):
+ print("Removing stale %s extraction at %s" % (name, btdir))
+ subprocess.check_call(["rm", "-rf", btdir])
if not os.path.exists(btdir):
- btdlpath = getconfig("BASE_SHAREDDIR", ourconfig) + "/cluster-downloads-cache/" + os.path.basename(bttarball)
print("Extracting %s %s" % (name, bttarball))
- btlock = btdlpath + ".lock"
- if not os.path.exists(os.path.dirname(btdlpath)):
- os.makedirs(os.path.dirname(btdlpath), exist_ok=True)
- while True:
- try:
- with open(btlock, 'a+') as lf:
- fileno = lf.fileno()
- fcntl.flock(fileno, fcntl.LOCK_EX)
- if sha256 and os.path.exists(btdlpath):
- dl_sha256 = sha256_file(btdlpath)
- if dl_sha256 != sha256:
- os.unlink(btdlpath)
- if not os.path.exists(btdlpath):
- if bttarball.startswith("/"):
- subprocess.check_call(["cp", bttarball, btdlpath])
- else:
- subprocess.check_call(["wget", "-O", btdlpath, bttarball])
- os.chmod(btdlpath, 0o775)
- break
- except OSError:
- # We raced with someone else, try again
- pass
subprocess.check_call(["bash", btdlpath, "-d", btdir, "-y"])
enable_tools_tarball(btdir, name)
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 01/11] scripts/utils: fix stale extraction dir when tarball is updated
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Previously the entire download/cache-validation block in
setup_tools_tarball() was guarded by `if not os.path.exists(btdir)`.
Once the extraction directory existed from a prior build, every
subsequent call was a no-op: the cached tarball was never re-validated,
so a freshly-published SDK (e.g. vcontainer-tarball-latest) was silently
ignored and the stale btdir kept being used.
Fix by moving the lock/download block outside the btdir existence guard
so cache validation always runs. Track tarball_updated to know when the
cached download was actually replaced, then remove the stale btdir when
True so the fresh tarball is re-extracted.
Also add an mtime-based staleness check for local-path (cp) sources so
that a newer source file automatically invalidates the cached copy.
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 64 +++++++++++++++++++++++++++++++-----------------
1 file changed, 41 insertions(+), 23 deletions(-)
diff --git a/scripts/utils.py b/scripts/utils.py
index 88842f0..87acad6 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -481,31 +481,49 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
if ";" in bttarball:
bttarball, sha256 = bttarball.split(";")
btdir = os.path.abspath(btdir)
+ btdlpath = getconfig("BASE_SHAREDDIR", ourconfig) + "/cluster-downloads-cache/" + os.path.basename(bttarball)
+ btlock = btdlpath + ".lock"
+ if not os.path.exists(os.path.dirname(btdlpath)):
+ os.makedirs(os.path.dirname(btdlpath), exist_ok=True)
+ # Always run the cache-validation / download step so that a freshly
+ # published tarball is detected even when btdir already exists from a
+ # previous build. tarball_updated is set to True whenever the cached
+ # download is replaced, which triggers removal of the stale btdir.
+ tarball_updated = False
+ while True:
+ try:
+ with open(btlock, 'a+') as lf:
+ fileno = lf.fileno()
+ fcntl.flock(fileno, fcntl.LOCK_EX)
+ if sha256 and os.path.exists(btdlpath):
+ dl_sha256 = sha256_file(btdlpath)
+ if dl_sha256 != sha256:
+ os.unlink(btdlpath)
+ elif bttarball.startswith("/") and os.path.exists(btdlpath):
+ # For local-path sources (e.g. vcontainer-tarball-latest)
+ # invalidate the cached copy when the source is newer so
+ # that a freshly-published tarball is always picked up.
+ if os.path.getmtime(bttarball) > os.path.getmtime(btdlpath):
+ os.unlink(btdlpath)
+ os.unlink(btdlpath)
+ if not os.path.exists(btdlpath):
+ if bttarball.startswith("/"):
+ subprocess.check_call(["cp", bttarball, btdlpath])
+ else:
+ subprocess.check_call(["wget", "-O", btdlpath, bttarball])
+ os.chmod(btdlpath, 0o775)
+ tarball_updated = True
+ break
+ except OSError:
+ # We raced with someone else, try again
+ pass
+ # If the underlying tarball changed, remove any stale extraction
+ # directory so it is re-extracted below.
+ if tarball_updated and os.path.exists(btdir):
+ print("Removing stale %s extraction at %s" % (name, btdir))
+ subprocess.check_call(["rm", "-rf", btdir])
if not os.path.exists(btdir):
- btdlpath = getconfig("BASE_SHAREDDIR", ourconfig) + "/cluster-downloads-cache/" + os.path.basename(bttarball)
print("Extracting %s %s" % (name, bttarball))
- btlock = btdlpath + ".lock"
- if not os.path.exists(os.path.dirname(btdlpath)):
- os.makedirs(os.path.dirname(btdlpath), exist_ok=True)
- while True:
- try:
- with open(btlock, 'a+') as lf:
- fileno = lf.fileno()
- fcntl.flock(fileno, fcntl.LOCK_EX)
- if sha256 and os.path.exists(btdlpath):
- dl_sha256 = sha256_file(btdlpath)
- if dl_sha256 != sha256:
- os.unlink(btdlpath)
- if not os.path.exists(btdlpath):
- if bttarball.startswith("/"):
- subprocess.check_call(["cp", bttarball, btdlpath])
- else:
- subprocess.check_call(["wget", "-O", btdlpath, bttarball])
- os.chmod(btdlpath, 0o775)
- break
- except OSError:
- # We raced with someone else, try again
- pass
subprocess.check_call(["bash", btdlpath, "-d", btdir, "-y"])
enable_tools_tarball(btdir, name)
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 02/11] scripts/utils: warn and force re-download for HTTPS sources without SHA256
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Without a SHA256 checksum there is no way to verify that a cached HTTPS
download is still current. Rather than silently reusing a potentially
stale copy, delete the cached file and force a re-download each run,
and emit a clear WARNING telling the operator how to avoid the overhead
(by appending ;sha256=<hash> to the URL in their config).
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/scripts/utils.py b/scripts/utils.py
index 87acad6..ea905d9 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -505,6 +505,14 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
# that a freshly-published tarball is always picked up.
if os.path.getmtime(bttarball) > os.path.getmtime(btdlpath):
os.unlink(btdlpath)
+ elif not bttarball.startswith("/") and os.path.exists(btdlpath):
+ # HTTPS/FTP source with no SHA256: there is no way to
+ # verify the cached copy is current, so force a
+ # re-download every run. Add a sha256=<hash> suffix to
+ # the URL in your config to avoid this.
+ print("WARNING: no SHA256 provided for %s source %s; "
+ "forcing re-download to avoid using a stale cached copy"
+ % (name, bttarball))
os.unlink(btdlpath)
if not os.path.exists(btdlpath):
if bttarball.startswith("/"):
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 02/11] scripts/utils: warn and force re-download for HTTPS sources without SHA256
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Without a SHA256 checksum there is no way to verify that a cached HTTPS
download is still current. Rather than silently reusing a potentially
stale copy, delete the cached file and force a re-download each run,
and emit a clear WARNING telling the operator how to avoid the overhead
(by appending ;sha256=<hash> to the URL in their config).
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/scripts/utils.py b/scripts/utils.py
index 87acad6..ea905d9 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -505,6 +505,14 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
# that a freshly-published tarball is always picked up.
if os.path.getmtime(bttarball) > os.path.getmtime(btdlpath):
os.unlink(btdlpath)
+ elif not bttarball.startswith("/") and os.path.exists(btdlpath):
+ # HTTPS/FTP source with no SHA256: there is no way to
+ # verify the cached copy is current, so force a
+ # re-download every run. Add a sha256=<hash> suffix to
+ # the URL in your config to avoid this.
+ print("WARNING: no SHA256 provided for %s source %s; "
+ "forcing re-download to avoid using a stale cached copy"
+ % (name, bttarball))
os.unlink(btdlpath)
if not os.path.exists(btdlpath):
if bttarball.startswith("/"):
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 03/11] config.json: add sha256sum for BUILDTOOLS URLs
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
setup_tools_tarball() now checks the sha256sum of cached tools tarballs
to determine if an update is available and needs to be re-downloaded.
To prevent unnecessary downloads, add ;sha256= to the BUILDTOOLS URLs.
extratools/EXTRATOOLS_URL uses the same utils.setup_tools_tarball(), so
it also needs the ;sha256=.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/config.json b/config.json
index f0e220e..8d42bbe 100644
--- a/config.json
+++ b/config.json
@@ -7,10 +7,10 @@
"BUILD_HISTORY_REPO" : "ssh://git@push.yoctoproject.org/poky-buildhistory",
"BUILD_HISTORY_FORKPUSH" : {"openembedded-core-contrib:ross/mut" : "openembedded-core:master", "openembedded-core-contrib:abelloni/master-next": "openembedded-core:master", "openembedded-core:master-next" : "openembedded-core:master"},
- "BUILDTOOLS_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/x86_64-buildtools-extended-nativesdk-standalone-5.1.sh",
- "BUILDTOOLS_ARM_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/aarch64-buildtools-extended-nativesdk-standalone-5.1.sh",
- "BUILDTOOLS_MAKE_URL" : "https://downloads.yoctoproject.org/releases/yocto/yocto-5.0.4/buildtools/x86_64-buildtools-make-nativesdk-standalone-5.0.4.sh",
- "EXTRATOOLS_URL" : "https://downloads.yoctoproject.org/tools/buildtools/x86_64-buildtools-imagemagick-nativesdk-standalone-4.3+snapshot-5f2ba20f203114db9a3b11264467f8c23a05041d.sh",
+ "BUILDTOOLS_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/x86_64-buildtools-extended-nativesdk-standalone-5.1.sh;sha256=5af9d92898af17fcc2fca4d07607a59f41b1c39a0a4ff058f64d834b98ec7fd5",
+ "BUILDTOOLS_ARM_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/aarch64-buildtools-extended-nativesdk-standalone-5.1.sh;sha256=8074d582a60a5135fd5606b8326e749185d26f51aca27570447831310bcac187",
+ "BUILDTOOLS_MAKE_URL" : "https://downloads.yoctoproject.org/releases/yocto/yocto-5.0.4/buildtools/x86_64-buildtools-make-nativesdk-standalone-5.0.4.sh;sha256=2eb7a6c013113f4fdb87e800167606dc13af1bd8bbc1c9e2443b7be37fefd124",
+ "EXTRATOOLS_URL" : "https://downloads.yoctoproject.org/tools/buildtools/x86_64-buildtools-imagemagick-nativesdk-standalone-4.3+snapshot-5f2ba20f203114db9a3b11264467f8c23a05041d.sh;sha256=9cbff3a7cf524bdfa7779dce8afaf3453114d8017918d2927f723ea38a36ebdc",
"REPO_STASH_DIR" : "${BASE_HOMEDIR}/git/mirror",
"TRASH_DIR" : "${BASE_HOMEDIR}/git/trash",
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 03/11] config.json: add sha256sum for BUILDTOOLS URLs
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
setup_tools_tarball() now checks the sha256sum of cached tools tarballs
to determine if an update is available and needs to be re-downloaded.
To prevent unnecessary downloads, add ;sha256= to the BUILDTOOLS URLs.
extratools/EXTRATOOLS_URL uses the same utils.setup_tools_tarball(), so
it also needs the ;sha256=.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/config.json b/config.json
index f0e220e..8d42bbe 100644
--- a/config.json
+++ b/config.json
@@ -7,10 +7,10 @@
"BUILD_HISTORY_REPO" : "ssh://git@push.yoctoproject.org/poky-buildhistory",
"BUILD_HISTORY_FORKPUSH" : {"openembedded-core-contrib:ross/mut" : "openembedded-core:master", "openembedded-core-contrib:abelloni/master-next": "openembedded-core:master", "openembedded-core:master-next" : "openembedded-core:master"},
- "BUILDTOOLS_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/x86_64-buildtools-extended-nativesdk-standalone-5.1.sh",
- "BUILDTOOLS_ARM_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/aarch64-buildtools-extended-nativesdk-standalone-5.1.sh",
- "BUILDTOOLS_MAKE_URL" : "https://downloads.yoctoproject.org/releases/yocto/yocto-5.0.4/buildtools/x86_64-buildtools-make-nativesdk-standalone-5.0.4.sh",
- "EXTRATOOLS_URL" : "https://downloads.yoctoproject.org/tools/buildtools/x86_64-buildtools-imagemagick-nativesdk-standalone-4.3+snapshot-5f2ba20f203114db9a3b11264467f8c23a05041d.sh",
+ "BUILDTOOLS_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/x86_64-buildtools-extended-nativesdk-standalone-5.1.sh;sha256=5af9d92898af17fcc2fca4d07607a59f41b1c39a0a4ff058f64d834b98ec7fd5",
+ "BUILDTOOLS_ARM_URL" : "https://downloads.yoctoproject.org/releases/yocto/milestones/yocto-5.2_M2/buildtools/aarch64-buildtools-extended-nativesdk-standalone-5.1.sh;sha256=8074d582a60a5135fd5606b8326e749185d26f51aca27570447831310bcac187",
+ "BUILDTOOLS_MAKE_URL" : "https://downloads.yoctoproject.org/releases/yocto/yocto-5.0.4/buildtools/x86_64-buildtools-make-nativesdk-standalone-5.0.4.sh;sha256=2eb7a6c013113f4fdb87e800167606dc13af1bd8bbc1c9e2443b7be37fefd124",
+ "EXTRATOOLS_URL" : "https://downloads.yoctoproject.org/tools/buildtools/x86_64-buildtools-imagemagick-nativesdk-standalone-4.3+snapshot-5f2ba20f203114db9a3b11264467f8c23a05041d.sh;sha256=9cbff3a7cf524bdfa7779dce8afaf3453114d8017918d2927f723ea38a36ebdc",
"REPO_STASH_DIR" : "${BASE_HOMEDIR}/git/mirror",
"TRASH_DIR" : "${BASE_HOMEDIR}/git/trash",
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 04/11] scripts/utils: add getconfigdict() for dict-type config values
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Parallel to getconfiglist(), but for JSON object values. The merge
priority is defaults < target-level < step-level so that more-specific
keys win: a step can override individual entries in a target-level dict
without replacing the whole thing, and both levels refine the defaults.
Used by the upcoming CONTAINER_IMAGES support, where each entry maps a
Yocto recipe name (the on-disk OCI path stem) to an image name (the
name pushed to the container registry).
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/scripts/utils.py b/scripts/utils.py
index ea905d9..112ebc2 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -112,6 +112,21 @@ def getconfiglist(name, config, target, stepnum):
ret.extend(config['defaults'][name])
return expandresult(ret, config)
+# Get a build configuration dict, merging defaults < target < step so that
+# more-specific entries win (step-level keys override target-level, which
+# override defaults).
+def getconfigdict(name, config, target, stepnum):
+ ret = {}
+ step = "step" + str(stepnum)
+ if name in config['defaults']:
+ ret.update(config['defaults'][name])
+ if target in config['overrides']:
+ if name in config['overrides'][target]:
+ ret.update(config['overrides'][target][name])
+ if step in config['overrides'][target] and name in config['overrides'][target][step]:
+ ret.update(config['overrides'][target][step][name])
+ return expandresult(ret, config)
+
# Return only unique configuration values (identified with '=' in them)
def getconfiglistfilter(name, config, target, stepnum):
def merge(main, newvals):
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 04/11] scripts/utils: add getconfigdict() for dict-type config values
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Parallel to getconfiglist(), but for JSON object values. The merge
priority is defaults < target-level < step-level so that more-specific
keys win: a step can override individual entries in a target-level dict
without replacing the whole thing, and both levels refine the defaults.
Used by the upcoming CONTAINER_IMAGES support, where each entry maps a
Yocto recipe name (the on-disk OCI path stem) to an image name (the
name pushed to the container registry).
AI-Generated: Claude Cowork Sonnet 4.6
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/utils.py | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/scripts/utils.py b/scripts/utils.py
index ea905d9..112ebc2 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -112,6 +112,21 @@ def getconfiglist(name, config, target, stepnum):
ret.extend(config['defaults'][name])
return expandresult(ret, config)
+# Get a build configuration dict, merging defaults < target < step so that
+# more-specific entries win (step-level keys override target-level, which
+# override defaults).
+def getconfigdict(name, config, target, stepnum):
+ ret = {}
+ step = "step" + str(stepnum)
+ if name in config['defaults']:
+ ret.update(config['defaults'][name])
+ if target in config['overrides']:
+ if name in config['overrides'][target]:
+ ret.update(config['overrides'][target][name])
+ if step in config['overrides'][target] and name in config['overrides'][target][step]:
+ ret.update(config['overrides'][target][step][name])
+ return expandresult(ret, config)
+
# Return only unique configuration values (identified with '=' in them)
def getconfiglistfilter(name, config, target, stepnum):
def merge(main, newvals):
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 05/11] scripts: add vcontainer-tarball setup, integration, and publishing
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Introduce the vcontainer-tarball SDK plumbing used by container build
jobs. The vcontainer-tarball is a meta-virtualization-derived SDK
(modelled after buildtools-tarball) that ships the container build
toolchain so worker jobs do not need to rebuild it for every step.
* scripts/utils.py: add setup_vcontainer_tarball(), and add an
env_glob keyword argument to setup_tools_tarball() and
enable_tools_tarball() so the vcontainer-tarball can source its
specific environment-setup-ci file rather than the universal
glob.
* scripts/run-config: source the vcontainer-tarball environment
for build-targets / cmds / test-targets / plain-cmds steps,
gated by a new NOVCONTAINER step variable so individual steps
(such as the dashboard indexing step) can opt out independently
of NOBUILDTOOLS.
* scripts/shared-repo-unpack: invoke setup_vcontainer_tarball so
workers extract the SDK during unpack.
* scripts/publish-artefacts: publish the vcontainer-tarball
artefact so downstream test jobs can fetch a stable SDK.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 1 +
scripts/publish-artefacts | 5 +++++
scripts/run-config | 19 +++++++++++++++++++
scripts/shared-repo-unpack | 1 +
scripts/utils.py | 22 ++++++++++++++++++----
5 files changed, 44 insertions(+), 4 deletions(-)
diff --git a/config.json b/config.json
index 8d42bbe..ab0ad7c 100644
--- a/config.json
+++ b/config.json
@@ -1433,6 +1433,7 @@
"step3" : {
"shortname" : "Populate/update dashboard site",
"NOBUILDTOOLS" : true,
+ "NOVCONTAINER" : true,
"EXTRACMDS" : ["${SCRIPTSDIR}/run-dashboard-index ${HELPERBUILDDIR}/../"]
}
},
diff --git a/scripts/publish-artefacts b/scripts/publish-artefacts
index e56e131..0e820e9 100755
--- a/scripts/publish-artefacts
+++ b/scripts/publish-artefacts
@@ -146,5 +146,10 @@ case "$target" in
sha256sums $TMPDIR/deploy/images/qemux86-64
cp -R --no-dereference --preserve=links $TMPDIR/deploy/images/qemux86-64/*qemux86* $DEST/patchtest
;;
+ "vcontainer-tarball")
+ mkdir -p $DEST/vcontainer-tarball
+ sha256sums $TMPDIR/deploy/sdk
+ cp -R --no-dereference --preserve=links $TMPDIR/deploy/sdk/*vcontainer* $DEST/vcontainer-tarball
+ ;;
esac
diff --git a/scripts/run-config b/scripts/run-config
index e896234..0f5a26a 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -153,6 +153,25 @@ else:
if args.phase == "init" and args.stepname == "buildtools":
sys.exit(0)
+if jcfg:
+ vcontainer = utils.setup_vcontainer_tarball(ourconfig, args.workername, None, checkonly=True)
+ if vcontainer:
+ addentry("vcontainer", "Setup vcontainer tarball", "init")
+else:
+ # If we're executing a specific step, check whether vcontainer is disabled for it
+ vcontainer = True
+ if args.stepname in ("build-targets", "cmds", "test-targets", "plain-cmds"):
+ try:
+ vcontainer = not utils.getconfigvar("NOVCONTAINER", ourconfig, args.target, int(args.phase))
+ except ValueError:
+ # Not an integer step phase
+ pass
+
+ if vcontainer:
+ utils.setup_vcontainer_tarball(ourconfig, args.workername, args.builddir + "/../vcontainer-tarball")
+ if args.phase == "init" and args.stepname == "vcontainer":
+ sys.exit(0)
+
extratools = utils.getconfigvar("extratools", ourconfig, args.target)
if jcfg:
if extratools:
diff --git a/scripts/shared-repo-unpack b/scripts/shared-repo-unpack
index 797dec6..869b214 100755
--- a/scripts/shared-repo-unpack
+++ b/scripts/shared-repo-unpack
@@ -77,6 +77,7 @@ for repo in sorted(repos.keys()):
utils.flush()
utils.setup_buildtools_tarball(ourconfig, args.workername, args.abworkdir + "/buildtools")
+utils.setup_vcontainer_tarball(ourconfig, args.workername, args.abworkdir + "/vcontainer-tarball")
if "bitbake" not in repos:
sys.exit(0)
diff --git a/scripts/utils.py b/scripts/utils.py
index 112ebc2..b020a7b 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -456,8 +456,8 @@ def sha256_file(filename):
pass
return method.hexdigest()
-def enable_tools_tarball(btdir, name):
- btenv = glob.glob(btdir + "/environment-setup*")
+def enable_tools_tarball(btdir, name, env_glob="/environment-setup*"):
+ btenv = glob.glob(btdir + env_glob)
print("Using %s %s" % (name, btenv))
# We either parse or wrap all our execution calls, rock and a hard place :(
with open(btenv[0], "r") as f:
@@ -474,6 +474,20 @@ def enable_tools_tarball(btdir, name):
if line in os.environ:
del os.environ[line]
+def setup_vcontainer_tarball(ourconfig, workername, vcdir, checkonly=False):
+ vctarball = None
+ if "vcontainer" in ourconfig and workername:
+ vccfg = getconfig("vcontainer", ourconfig)
+ for entry in vccfg:
+ if fnmatch.fnmatch(workername, entry):
+ vctarball = vccfg[entry]
+ break
+
+ if checkonly:
+ return vctarball
+
+ setup_tools_tarball(ourconfig, vcdir, vctarball, name="vcontainer-tarball", env_glob="/environment-setup-ci")
+
def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
bttarball = None
if "buildtools" in ourconfig and workername:
@@ -488,7 +502,7 @@ def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
setup_tools_tarball(ourconfig, btdir, bttarball)
-def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
+def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools", env_glob="/environment-setup*"):
btenv = None
if bttarball:
@@ -548,7 +562,7 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
if not os.path.exists(btdir):
print("Extracting %s %s" % (name, bttarball))
subprocess.check_call(["bash", btdlpath, "-d", btdir, "-y"])
- enable_tools_tarball(btdir, name)
+ enable_tools_tarball(btdir, name, env_glob)
def get_string_from_version(version, milestone=None, rc=None):
""" Point releases finishing by 0 (e.g 4.0.0, 4.1.0) do no exists,
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 05/11] scripts: add vcontainer-tarball setup, integration, and publishing
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Introduce the vcontainer-tarball SDK plumbing used by container build
jobs. The vcontainer-tarball is a meta-virtualization-derived SDK
(modelled after buildtools-tarball) that ships the container build
toolchain so worker jobs do not need to rebuild it for every step.
* scripts/utils.py: add setup_vcontainer_tarball(), and add an
env_glob keyword argument to setup_tools_tarball() and
enable_tools_tarball() so the vcontainer-tarball can source its
specific environment-setup-ci file rather than the universal
glob.
* scripts/run-config: source the vcontainer-tarball environment
for build-targets / cmds / test-targets / plain-cmds steps,
gated by a new NOVCONTAINER step variable so individual steps
(such as the dashboard indexing step) can opt out independently
of NOBUILDTOOLS.
* scripts/shared-repo-unpack: invoke setup_vcontainer_tarball so
workers extract the SDK during unpack.
* scripts/publish-artefacts: publish the vcontainer-tarball
artefact so downstream test jobs can fetch a stable SDK.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 1 +
scripts/publish-artefacts | 5 +++++
scripts/run-config | 19 +++++++++++++++++++
scripts/shared-repo-unpack | 1 +
scripts/utils.py | 22 ++++++++++++++++++----
5 files changed, 44 insertions(+), 4 deletions(-)
diff --git a/config.json b/config.json
index 8d42bbe..ab0ad7c 100644
--- a/config.json
+++ b/config.json
@@ -1433,6 +1433,7 @@
"step3" : {
"shortname" : "Populate/update dashboard site",
"NOBUILDTOOLS" : true,
+ "NOVCONTAINER" : true,
"EXTRACMDS" : ["${SCRIPTSDIR}/run-dashboard-index ${HELPERBUILDDIR}/../"]
}
},
diff --git a/scripts/publish-artefacts b/scripts/publish-artefacts
index e56e131..0e820e9 100755
--- a/scripts/publish-artefacts
+++ b/scripts/publish-artefacts
@@ -146,5 +146,10 @@ case "$target" in
sha256sums $TMPDIR/deploy/images/qemux86-64
cp -R --no-dereference --preserve=links $TMPDIR/deploy/images/qemux86-64/*qemux86* $DEST/patchtest
;;
+ "vcontainer-tarball")
+ mkdir -p $DEST/vcontainer-tarball
+ sha256sums $TMPDIR/deploy/sdk
+ cp -R --no-dereference --preserve=links $TMPDIR/deploy/sdk/*vcontainer* $DEST/vcontainer-tarball
+ ;;
esac
diff --git a/scripts/run-config b/scripts/run-config
index e896234..0f5a26a 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -153,6 +153,25 @@ else:
if args.phase == "init" and args.stepname == "buildtools":
sys.exit(0)
+if jcfg:
+ vcontainer = utils.setup_vcontainer_tarball(ourconfig, args.workername, None, checkonly=True)
+ if vcontainer:
+ addentry("vcontainer", "Setup vcontainer tarball", "init")
+else:
+ # If we're executing a specific step, check whether vcontainer is disabled for it
+ vcontainer = True
+ if args.stepname in ("build-targets", "cmds", "test-targets", "plain-cmds"):
+ try:
+ vcontainer = not utils.getconfigvar("NOVCONTAINER", ourconfig, args.target, int(args.phase))
+ except ValueError:
+ # Not an integer step phase
+ pass
+
+ if vcontainer:
+ utils.setup_vcontainer_tarball(ourconfig, args.workername, args.builddir + "/../vcontainer-tarball")
+ if args.phase == "init" and args.stepname == "vcontainer":
+ sys.exit(0)
+
extratools = utils.getconfigvar("extratools", ourconfig, args.target)
if jcfg:
if extratools:
diff --git a/scripts/shared-repo-unpack b/scripts/shared-repo-unpack
index 797dec6..869b214 100755
--- a/scripts/shared-repo-unpack
+++ b/scripts/shared-repo-unpack
@@ -77,6 +77,7 @@ for repo in sorted(repos.keys()):
utils.flush()
utils.setup_buildtools_tarball(ourconfig, args.workername, args.abworkdir + "/buildtools")
+utils.setup_vcontainer_tarball(ourconfig, args.workername, args.abworkdir + "/vcontainer-tarball")
if "bitbake" not in repos:
sys.exit(0)
diff --git a/scripts/utils.py b/scripts/utils.py
index 112ebc2..b020a7b 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -456,8 +456,8 @@ def sha256_file(filename):
pass
return method.hexdigest()
-def enable_tools_tarball(btdir, name):
- btenv = glob.glob(btdir + "/environment-setup*")
+def enable_tools_tarball(btdir, name, env_glob="/environment-setup*"):
+ btenv = glob.glob(btdir + env_glob)
print("Using %s %s" % (name, btenv))
# We either parse or wrap all our execution calls, rock and a hard place :(
with open(btenv[0], "r") as f:
@@ -474,6 +474,20 @@ def enable_tools_tarball(btdir, name):
if line in os.environ:
del os.environ[line]
+def setup_vcontainer_tarball(ourconfig, workername, vcdir, checkonly=False):
+ vctarball = None
+ if "vcontainer" in ourconfig and workername:
+ vccfg = getconfig("vcontainer", ourconfig)
+ for entry in vccfg:
+ if fnmatch.fnmatch(workername, entry):
+ vctarball = vccfg[entry]
+ break
+
+ if checkonly:
+ return vctarball
+
+ setup_tools_tarball(ourconfig, vcdir, vctarball, name="vcontainer-tarball", env_glob="/environment-setup-ci")
+
def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
bttarball = None
if "buildtools" in ourconfig and workername:
@@ -488,7 +502,7 @@ def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
setup_tools_tarball(ourconfig, btdir, bttarball)
-def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
+def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools", env_glob="/environment-setup*"):
btenv = None
if bttarball:
@@ -548,7 +562,7 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
if not os.path.exists(btdir):
print("Extracting %s %s" % (name, bttarball))
subprocess.check_call(["bash", btdlpath, "-d", btdir, "-y"])
- enable_tools_tarball(btdir, name)
+ enable_tools_tarball(btdir, name, env_glob)
def get_string_from_version(version, milestone=None, rc=None):
""" Point releases finishing by 0 (e.g 4.0.0, 4.1.0) do no exists,
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 06/11] config.json: add vcontainer-tarball build target
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Add the vcontainer-tarball build target which produces the
meta-virtualization SDK used by downstream container build/test
jobs. Modelled after the buildtools-tarball target.
The target uses EXTRACMDS to reset BBMULTICONFIG for the worker
shell, places the extravars in the per-step config, and includes
a publish-artefacts step so the resulting SDK tarball is staged
for reuse by container-tests and other consumers.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/config.json b/config.json
index ab0ad7c..9e6898d 100644
--- a/config.json
+++ b/config.json
@@ -1869,6 +1869,32 @@
},
"toaster" : {
"EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests ${HELPERBUILDDIR} ${HELPERBUILDDIR}/../layers/bitbake"]
+ },
+ "vcontainer-tarball": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Build vcontainer-tarballs",
+ "BBTARGETS" : "vcontainer-tarball",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'",
+ "BBMULTICONFIG = 'vruntime-aarch64 vruntime-x86-64'",
+ "INIT_MANAGER = 'systemd'"
+ ],
+ "EXTRACMDS" : ["sed -i '/vruntime-aarch64 vruntime-x86-64/d' ${HELPERBUILDDIR}/conf/auto.conf"]
+ },
+ "step2" : {
+ "shortname" : "Publish vcontainer SDK for test reuse",
+ "EXTRAPLAINCMDS" : [
+ "install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
+ ]
+ }
}
},
"repo-defaults" : {
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 06/11] config.json: add vcontainer-tarball build target
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Add the vcontainer-tarball build target which produces the
meta-virtualization SDK used by downstream container build/test
jobs. Modelled after the buildtools-tarball target.
The target uses EXTRACMDS to reset BBMULTICONFIG for the worker
shell, places the extravars in the per-step config, and includes
a publish-artefacts step so the resulting SDK tarball is staged
for reuse by container-tests and other consumers.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/config.json b/config.json
index ab0ad7c..9e6898d 100644
--- a/config.json
+++ b/config.json
@@ -1869,6 +1869,32 @@
},
"toaster" : {
"EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests ${HELPERBUILDDIR} ${HELPERBUILDDIR}/../layers/bitbake"]
+ },
+ "vcontainer-tarball": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Build vcontainer-tarballs",
+ "BBTARGETS" : "vcontainer-tarball",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'",
+ "BBMULTICONFIG = 'vruntime-aarch64 vruntime-x86-64'",
+ "INIT_MANAGER = 'systemd'"
+ ],
+ "EXTRACMDS" : ["sed -i '/vruntime-aarch64 vruntime-x86-64/d' ${HELPERBUILDDIR}/conf/auto.conf"]
+ },
+ "step2" : {
+ "shortname" : "Publish vcontainer SDK for test reuse",
+ "EXTRAPLAINCMDS" : [
+ "install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
+ ]
+ }
}
},
"repo-defaults" : {
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 06/11] config.json: add vcontainer-tarball build target
2026-05-08 2:00 ` tim.orling
(?)
@ 2026-05-11 8:48 ` Paul Barker
2026-05-11 17:49 ` Tim Orling
-1 siblings, 1 reply; 34+ messages in thread
From: Paul Barker @ 2026-05-11 8:48 UTC (permalink / raw)
To: tim.orling, yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 2192 bytes --]
On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> From: Tim Orling <tim.orling@konsulko.com>
>
> Add the vcontainer-tarball build target which produces the
> meta-virtualization SDK used by downstream container build/test
> jobs. Modelled after the buildtools-tarball target.
>
> The target uses EXTRACMDS to reset BBMULTICONFIG for the worker
> shell, places the extravars in the per-step config, and includes
> a publish-artefacts step so the resulting SDK tarball is staged
> for reuse by container-tests and other consumers.
>
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
> config.json | 26 ++++++++++++++++++++++++++
> 1 file changed, 26 insertions(+)
>
> diff --git a/config.json b/config.json
> index ab0ad7c..9e6898d 100644
> --- a/config.json
> +++ b/config.json
> @@ -1869,6 +1869,32 @@
> },
> "toaster" : {
> "EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests ${HELPERBUILDDIR} ${HELPERBUILDDIR}/../layers/bitbake"]
> + },
> + "vcontainer-tarball": {
> + "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
> + "ADDLAYER" : [
> + "${BUILDDIR}/../meta-openembedded/meta-oe",
> + "${BUILDDIR}/../meta-openembedded/meta-python",
> + "${BUILDDIR}/../meta-openembedded/meta-networking",
> + "${BUILDDIR}/../meta-openembedded/meta-filesystems",
> + "${BUILDDIR}/../meta-virtualization"
> + ],
> + "step1" : {
> + "shortname" : "Build vcontainer-tarballs",
> + "BBTARGETS" : "vcontainer-tarball",
> + "extravars" : [
> + "DISTRO_FEATURES:append = ' virtualization vcontainer'",
> + "BBMULTICONFIG = 'vruntime-aarch64 vruntime-x86-64'",
> + "INIT_MANAGER = 'systemd'"
> + ],
> + "EXTRACMDS" : ["sed -i '/vruntime-aarch64 vruntime-x86-64/d' ${HELPERBUILDDIR}/conf/auto.conf"]
What does this sed command achieve? EXTRACMDS run after building
BBTARGETS.
Best regards,
--
Paul Barker
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 06/11] config.json: add vcontainer-tarball build target
2026-05-11 8:48 ` Paul Barker
@ 2026-05-11 17:49 ` Tim Orling
0 siblings, 0 replies; 34+ messages in thread
From: Tim Orling @ 2026-05-11 17:49 UTC (permalink / raw)
To: Paul Barker; +Cc: yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]
On Mon, May 11, 2026 at 1:48 AM Paul Barker <paul@pbarker.dev> wrote:
> On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> > From: Tim Orling <tim.orling@konsulko.com>
> >
> > Add the vcontainer-tarball build target which produces the
> > meta-virtualization SDK used by downstream container build/test
> > jobs. Modelled after the buildtools-tarball target.
> >
> > The target uses EXTRACMDS to reset BBMULTICONFIG for the worker
> > shell, places the extravars in the per-step config, and includes
> > a publish-artefacts step so the resulting SDK tarball is staged
> > for reuse by container-tests and other consumers.
> >
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> > ---
> > config.json | 26 ++++++++++++++++++++++++++
> > 1 file changed, 26 insertions(+)
> >
> > diff --git a/config.json b/config.json
> > index ab0ad7c..9e6898d 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1869,6 +1869,32 @@
> > },
> > "toaster" : {
> > "EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests
> ${HELPERBUILDDIR} ${HELPERBUILDDIR}/../layers/bitbake"]
> > + },
> > + "vcontainer-tarball": {
> > + "NEEDREPOS" : ["bitbake", "meta-openembedded",
> "meta-virtualization"],
> > + "ADDLAYER" : [
> > + "${BUILDDIR}/../meta-openembedded/meta-oe",
> > + "${BUILDDIR}/../meta-openembedded/meta-python",
> > + "${BUILDDIR}/../meta-openembedded/meta-networking",
> > + "${BUILDDIR}/../meta-openembedded/meta-filesystems",
> > + "${BUILDDIR}/../meta-virtualization"
> > + ],
> > + "step1" : {
> > + "shortname" : "Build vcontainer-tarballs",
> > + "BBTARGETS" : "vcontainer-tarball",
> > + "extravars" : [
> > + "DISTRO_FEATURES:append = ' virtualization
> vcontainer'",
> > + "BBMULTICONFIG = 'vruntime-aarch64
> vruntime-x86-64'",
> > + "INIT_MANAGER = 'systemd'"
> > + ],
> > + "EXTRACMDS" : ["sed -i '/vruntime-aarch64
> vruntime-x86-64/d' ${HELPERBUILDDIR}/conf/auto.conf"]
>
> What does this sed command achieve? EXTRACMDS run after building
> BBTARGETS.
>
>
Describing the reason was accidentally removed while refactoring and
slimming down the number of commits.
Original squashed commit message:
"
Since meta-virtualization is the first layer removed by 'remove-layers', our
BBMULTICONFIGs become invalid immediately after layer removal and throw an
error.
In scripts/run-config, EXTRACMDS runs after BBTARGETS, but before
'remove-layers'.
Use 'sed' to reset BBMULTICONFIG to ''.
"
The error in the 'remove-layers' step can be summarized as:
bb.parse.ParseError: ParseError at
/home/pokybuild/yocto-worker/vcontainer-tarball/build/layers/openembedded-core/meta/conf/bitbake.conf:824:
Could not include required file conf/multiconfig/vruntime-aarch64.conf
Best regards,
>
> --
> Paul Barker
>
>
[-- Attachment #2: Type: text/html, Size: 4599 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 07/11] config.json: add 'containers-' build jobs
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Add 'containers-' build jobs that build container images on top
of the vcontainer-tarball SDK:
* containers-meta-virt: the original app-container-* images that
ship in meta-virtualization itself (e.g. app-container-curl).
* containers-library: a layer for additional images modelled
after docker.io/library/*, sourced from meta-yocto-containers-demo
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/config.json b/config.json
index 9e6898d..86c1ffb 100644
--- a/config.json
+++ b/config.json
@@ -1895,6 +1895,49 @@
"install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
]
}
+ },
+ "containers-meta-virt": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Build 'base' container",
+ "BBTARGETS" : "container-base",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ]
+ },
+ "step2" : {
+ "shortname" : "Build 'curl' container",
+ "BBTARGETS" : "app-container-curl",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ]
+ }
+ },
+ "containers-library": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization", "meta-yocto-containers-demo"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-openembedded/meta-webserver",
+ "${BUILDDIR}/../meta-virtualization",
+ "${BUILDDIR}/../meta-yocto-containers-demo"
+ ],
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ],
+ "step1" : {
+ "shortname" : "Build 'python' container",
+ "BBTARGETS" : "app-container-python"
+ }
}
},
"repo-defaults" : {
@@ -2018,6 +2061,12 @@
"revision" : "HEAD",
"no-layer-add" : true
},
+ "meta-yocto-containers-demo" : {
+ "url" : "https://github.com/moto-timo/meta-yocto-containers-demo.git",
+ "branch" : "master",
+ "revision" : "HEAD",
+ "no-layer-add" : true
+ },
"auto-upgrade-helper" : {
"url" : "git://git.yoctoproject.org/auto-upgrade-helper",
"branch" : "master",
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 07/11] config.json: add 'containers-' build jobs
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Add 'containers-' build jobs that build container images on top
of the vcontainer-tarball SDK:
* containers-meta-virt: the original app-container-* images that
ship in meta-virtualization itself (e.g. app-container-curl).
* containers-library: a layer for additional images modelled
after docker.io/library/*, sourced from meta-yocto-containers-demo
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/config.json b/config.json
index 9e6898d..86c1ffb 100644
--- a/config.json
+++ b/config.json
@@ -1895,6 +1895,49 @@
"install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
]
}
+ },
+ "containers-meta-virt": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Build 'base' container",
+ "BBTARGETS" : "container-base",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ]
+ },
+ "step2" : {
+ "shortname" : "Build 'curl' container",
+ "BBTARGETS" : "app-container-curl",
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ]
+ }
+ },
+ "containers-library": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization", "meta-yocto-containers-demo"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-openembedded/meta-webserver",
+ "${BUILDDIR}/../meta-virtualization",
+ "${BUILDDIR}/../meta-yocto-containers-demo"
+ ],
+ "extravars" : [
+ "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+ ],
+ "step1" : {
+ "shortname" : "Build 'python' container",
+ "BBTARGETS" : "app-container-python"
+ }
}
},
"repo-defaults" : {
@@ -2018,6 +2061,12 @@
"revision" : "HEAD",
"no-layer-add" : true
},
+ "meta-yocto-containers-demo" : {
+ "url" : "https://github.com/moto-timo/meta-yocto-containers-demo.git",
+ "branch" : "master",
+ "revision" : "HEAD",
+ "no-layer-add" : true
+ },
"auto-upgrade-helper" : {
"url" : "git://git.yoctoproject.org/auto-upgrade-helper",
"branch" : "master",
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 07/11] config.json: add 'containers-' build jobs
2026-05-08 2:00 ` tim.orling
(?)
@ 2026-05-11 8:52 ` Paul Barker
2026-05-11 17:36 ` Tim Orling
-1 siblings, 1 reply; 34+ messages in thread
From: Paul Barker @ 2026-05-11 8:52 UTC (permalink / raw)
To: tim.orling, yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 761 bytes --]
On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> From: Tim Orling <tim.orling@konsulko.com>
>
> Add 'containers-' build jobs that build container images on top
> of the vcontainer-tarball SDK:
>
> * containers-meta-virt: the original app-container-* images that
> ship in meta-virtualization itself (e.g. app-container-curl).
> * containers-library: a layer for additional images modelled
> after docker.io/library/*, sourced from meta-yocto-containers-demo
I think we should combine these into one build job.
What's the plan for meta-yocto-containers-demo? Should we host that on
git.yoctoproject.org, or will the contents eventually be merged into
meta-virtualization?
Best regards,
--
Paul Barker
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [yocto-autobuilder-helper][PATCH 07/11] config.json: add 'containers-' build jobs
2026-05-11 8:52 ` Paul Barker
@ 2026-05-11 17:36 ` Tim Orling
0 siblings, 0 replies; 34+ messages in thread
From: Tim Orling @ 2026-05-11 17:36 UTC (permalink / raw)
To: Paul Barker; +Cc: yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]
On Mon, May 11, 2026 at 1:52 AM Paul Barker <paul@pbarker.dev> wrote:
> On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> > From: Tim Orling <tim.orling@konsulko.com>
> >
> > Add 'containers-' build jobs that build container images on top
> > of the vcontainer-tarball SDK:
> >
> > * containers-meta-virt: the original app-container-* images that
> > ship in meta-virtualization itself (e.g. app-container-curl).
> > * containers-library: a layer for additional images modelled
> > after docker.io/library/*, sourced from meta-yocto-containers-demo
>
> I think we should combine these into one build job.
>
For now, that is impractical unless we use a branch of meta-virt that I can
control (because the container recipes are in flux). Or I would have to
continue running builds with a custom yocto-autobuilder-helper and a custom
meta-virt branch which defeats the purpose of pushing the containers to the
official registry.yocto.io and quay.io/yocto.
Let me put it another way: the container recipes in meta-virt are a simpler
demonstration. The container recipes in meta-yocto-containers-demo are
heading towards being fully signed/attested and hardened. So to me it made
sense to keep the jobs separate.
>
> What's the plan for meta-yocto-containers-demo? Should we host that on
> git.yoctoproject.org, or will the contents eventually be merged into
> meta-virtualization?
>
I haven't discussed this with Bruce, but I think it makes sense to have the
official list in meta-virtualization, when they are ready. The other main
reason for the meta-yocto-container-demo layer is for things like the
slsa-provenance.bbclass before submitting that to oe-core.
> Best regards,
>
> --
> Paul Barker
>
>
[-- Attachment #2: Type: text/html, Size: 2902 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 08/11] scripts: add run-vcontainer-tests for meta-virtualization
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Add scripts/run-vcontainer-tests, the test runner used by the
vcontainer test jobs. It sources the vcontainer-tarball SDK,
discovers the meta-virtualization pytest suite, and runs a
configurable set of suites (vdkr, vpdmn, memres) against the
checked-out layers. Suites can be selected per-step so the
top-level 'vcontainer-tests' job runs the container engine
agnostic tests:
- tests/test_container_cross_install.py
- tests/test_container_registry_script.py
- tests/test_vcontainer_auth_config.py
- tests/test_multiarch_oci.py
- tests/test_multilayer_oci.py
The 'vdkr-tests' and 'vpdmn-tests' jobs run only their respective
suites (including memres for each container engine):
- tests/test_vdkr.py
- tests/test_vdkr_registry.py
and
- tests/test_vpdmn.py
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/run-vcontainer-tests | 165 +++++++++++++++++++++++++++++++++++
1 file changed, 165 insertions(+)
create mode 100755 scripts/run-vcontainer-tests
diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
new file mode 100755
index 0000000..cbb5544
--- /dev/null
+++ b/scripts/run-vcontainer-tests
@@ -0,0 +1,165 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Run meta-virtualization pytest test suites against the vcontainer
+# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
+# step.
+#
+# Arguments:
+# $1 - suite name: one of "vcontainer", "vdkr", "vpdmn"
+# $2 - bitbake build directory (${BUILDDIR})
+# $3 - path to the meta-virtualization layer
+#
+# Optional environment variables:
+# RESULTS_DIR - directory to copy pytest artefacts (junit xml / log) to
+# VCONTAINER_EXTRACT_DIR - where to extract the standalone SDK tarball
+# (default: ${builddir}/vcontainer-test-extracted)
+# TEST_OCI_IMAGE - path to an OCI image directory (enables vdkr/vpdmn
+# import tests)
+# VDKR_ARCH - target architecture for vdkr/vpdmn tests (default: x86_64)
+#
+# The script is intentionally conservative: any pytest tests that cannot run
+# in the CI environment (those marked "slow", "network", "boot") are skipped
+# are skipped so that the autobuilder step completes without needing network
+# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
+# before invocation.
+#
+# It is assumed that /dev/kvm is writable by the CI user running the tests,
+# since the performance is significantly faster with 'memres'.
+#
+
+set -e
+set -u
+set -o pipefail
+set -x
+
+if [ $# -lt 3 ]; then
+ echo "Usage: $0 <suite> <builddir> <meta-virtualization-dir>" >&2
+ echo " suite: vcontainer | vdkr | vpdmn" >&2
+ exit 2
+fi
+
+suite="$1"
+builddir=$(realpath "$2")
+metavirtdir=$(realpath "$3")
+
+if [ ! -d "$metavirtdir/tests" ]; then
+ echo "ERROR: meta-virtualization tests directory not found at $metavirtdir/tests" >&2
+ exit 1
+fi
+
+# Locate the vcontainer standalone SDK tarball. Prefer an externally-built
+# SDK passed via VCONTAINER_SDK (the autobuilder -tests jobs share the SDK
+# produced by the separate vcontainer-tarball builder), and fall back to
+# looking in the local build's deploy/sdk directory when running stand-alone.
+sdk_tarball=""
+if [ -n "${VCONTAINER_SDK:-}" ]; then
+ if [ -f "$VCONTAINER_SDK" ]; then
+ sdk_tarball="$VCONTAINER_SDK"
+ else
+ echo "ERROR: VCONTAINER_SDK=$VCONTAINER_SDK is set but not a file" >&2
+ exit 1
+ fi
+fi
+if [ -z "$sdk_tarball" ]; then
+ sdk_tarball="$builddir/tmp/deploy/sdk/vcontainer-standalone.sh"
+ if [ ! -f "$sdk_tarball" ]; then
+ # Try to find any matching tarball in case naming changed (e.g. versioned)
+ alt=$(ls -1 "$builddir"/tmp/deploy/sdk/vcontainer-*.sh 2>/dev/null | head -n1 || true)
+ if [ -n "$alt" ]; then
+ sdk_tarball="$alt"
+ else
+ echo "ERROR: vcontainer standalone SDK not found." >&2
+ echo " Set VCONTAINER_SDK to an existing SDK installer, or" >&2
+ echo " build vcontainer-tarball so $builddir/tmp/deploy/sdk/vcontainer-standalone.sh exists." >&2
+ exit 1
+ fi
+ fi
+fi
+
+extract_dir="${VCONTAINER_EXTRACT_DIR:-$builddir/vcontainer-test-extracted}"
+rm -rf "$extract_dir"
+mkdir -p "$(dirname "$extract_dir")"
+
+# Self-extracting installer (silent, -y agrees to license, -d picks dir)
+"$sdk_tarball" -d "$extract_dir" -y
+
+# Prepare a Python venv so we don't pollute the worker's system packages.
+python3 -m venv "$builddir/meta-virt-test-venv"
+# shellcheck disable=SC1091
+source "$builddir/meta-virt-test-venv/bin/activate"
+# Avoid warnings by upgrading pip; install pytest/pexpect into the venv via pip.
+python3 -m pip install --quiet --upgrade pip setuptools wheel
+python3 -m pip install --quiet --upgrade pytest pytest-timeout pexpect
+
+# Default marker filter excludes long running / infrastructure dependent tests.
+marker_filter="${META_VIRT_PYTEST_MARKERS:-not slow and not network and not boot and not incus and not k3s}"
+
+# Per-suite test file selection. Uses -k/-m for fine-grained filtering and
+# keeps the CLI small for logging clarity.
+case "$suite" in
+ vdkr)
+ test_files=(
+ "tests/test_vdkr.py"
+ "tests/test_vdkr_registry.py"
+ )
+ ;;
+ vpdmn)
+ test_files=(
+ "tests/test_vpdmn.py"
+ )
+ ;;
+ vcontainer)
+ # Broad vcontainer/bbclass/tooling coverage that doesn't require the
+ # vdkr/vpdmn CLI harness to be running.
+ test_files=(
+ "tests/test_container_cross_install.py"
+ "tests/test_container_registry_script.py"
+ "tests/test_vcontainer_auth_config.py"
+ "tests/test_multiarch_oci.py"
+ "tests/test_multilayer_oci.py"
+ )
+ ;;
+ *)
+ echo "ERROR: unknown suite '$suite' (expected vcontainer|vdkr|vpdmn)" >&2
+ exit 2
+ ;;
+esac
+
+pytest_args=(
+ -v
+ --tb=short
+ -m "$marker_filter"
+ --vdkr-dir "$extract_dir"
+ --junitxml="$builddir/pytest-$suite-results.xml"
+)
+
+# Allow tests that consume an OCI image (import/save/load) to find one.
+if [ -n "${TEST_OCI_IMAGE:-}" ] && [ -d "${TEST_OCI_IMAGE}" ]; then
+ pytest_args+=(--oci-image "$TEST_OCI_IMAGE")
+fi
+
+# Pass architecture through when set in the environment (default is x86_64).
+if [ -n "${VDKR_ARCH:-}" ]; then
+ pytest_args+=(--arch "$VDKR_ARCH")
+fi
+
+cd "$metavirtdir"
+# Don't let a single failing test kill the whole step - collect the junit
+# report, then surface the exit code via the junit file + exit status.
+set +e
+python3 -m pytest "${pytest_args[@]}" "${test_files[@]}"
+rc=$?
+set -e
+
+# Copy artefacts to the results dir if one was provided.
+if [ -n "${RESULTS_DIR:-}" ]; then
+ mkdir -p "$RESULTS_DIR"
+ cp -f "$builddir/pytest-$suite-results.xml" "$RESULTS_DIR/" 2>/dev/null || true
+ if [ -f /tmp/pytest-vcontainer.log ]; then
+ cp -f /tmp/pytest-vcontainer.log "$RESULTS_DIR/pytest-$suite.log" || true
+ fi
+fi
+
+exit $rc
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 08/11] scripts: add run-vcontainer-tests for meta-virtualization
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Add scripts/run-vcontainer-tests, the test runner used by the
vcontainer test jobs. It sources the vcontainer-tarball SDK,
discovers the meta-virtualization pytest suite, and runs a
configurable set of suites (vdkr, vpdmn, memres) against the
checked-out layers. Suites can be selected per-step so the
top-level 'vcontainer-tests' job runs the container engine
agnostic tests:
- tests/test_container_cross_install.py
- tests/test_container_registry_script.py
- tests/test_vcontainer_auth_config.py
- tests/test_multiarch_oci.py
- tests/test_multilayer_oci.py
The 'vdkr-tests' and 'vpdmn-tests' jobs run only their respective
suites (including memres for each container engine):
- tests/test_vdkr.py
- tests/test_vdkr_registry.py
and
- tests/test_vpdmn.py
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
scripts/run-vcontainer-tests | 165 +++++++++++++++++++++++++++++++++++
1 file changed, 165 insertions(+)
create mode 100755 scripts/run-vcontainer-tests
diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
new file mode 100755
index 0000000..cbb5544
--- /dev/null
+++ b/scripts/run-vcontainer-tests
@@ -0,0 +1,165 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Run meta-virtualization pytest test suites against the vcontainer
+# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
+# step.
+#
+# Arguments:
+# $1 - suite name: one of "vcontainer", "vdkr", "vpdmn"
+# $2 - bitbake build directory (${BUILDDIR})
+# $3 - path to the meta-virtualization layer
+#
+# Optional environment variables:
+# RESULTS_DIR - directory to copy pytest artefacts (junit xml / log) to
+# VCONTAINER_EXTRACT_DIR - where to extract the standalone SDK tarball
+# (default: ${builddir}/vcontainer-test-extracted)
+# TEST_OCI_IMAGE - path to an OCI image directory (enables vdkr/vpdmn
+# import tests)
+# VDKR_ARCH - target architecture for vdkr/vpdmn tests (default: x86_64)
+#
+# The script is intentionally conservative: any pytest tests that cannot run
+# in the CI environment (those marked "slow", "network", "boot") are skipped
+# are skipped so that the autobuilder step completes without needing network
+# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
+# before invocation.
+#
+# It is assumed that /dev/kvm is writable by the CI user running the tests,
+# since the performance is significantly faster with 'memres'.
+#
+
+set -e
+set -u
+set -o pipefail
+set -x
+
+if [ $# -lt 3 ]; then
+ echo "Usage: $0 <suite> <builddir> <meta-virtualization-dir>" >&2
+ echo " suite: vcontainer | vdkr | vpdmn" >&2
+ exit 2
+fi
+
+suite="$1"
+builddir=$(realpath "$2")
+metavirtdir=$(realpath "$3")
+
+if [ ! -d "$metavirtdir/tests" ]; then
+ echo "ERROR: meta-virtualization tests directory not found at $metavirtdir/tests" >&2
+ exit 1
+fi
+
+# Locate the vcontainer standalone SDK tarball. Prefer an externally-built
+# SDK passed via VCONTAINER_SDK (the autobuilder -tests jobs share the SDK
+# produced by the separate vcontainer-tarball builder), and fall back to
+# looking in the local build's deploy/sdk directory when running stand-alone.
+sdk_tarball=""
+if [ -n "${VCONTAINER_SDK:-}" ]; then
+ if [ -f "$VCONTAINER_SDK" ]; then
+ sdk_tarball="$VCONTAINER_SDK"
+ else
+ echo "ERROR: VCONTAINER_SDK=$VCONTAINER_SDK is set but not a file" >&2
+ exit 1
+ fi
+fi
+if [ -z "$sdk_tarball" ]; then
+ sdk_tarball="$builddir/tmp/deploy/sdk/vcontainer-standalone.sh"
+ if [ ! -f "$sdk_tarball" ]; then
+ # Try to find any matching tarball in case naming changed (e.g. versioned)
+ alt=$(ls -1 "$builddir"/tmp/deploy/sdk/vcontainer-*.sh 2>/dev/null | head -n1 || true)
+ if [ -n "$alt" ]; then
+ sdk_tarball="$alt"
+ else
+ echo "ERROR: vcontainer standalone SDK not found." >&2
+ echo " Set VCONTAINER_SDK to an existing SDK installer, or" >&2
+ echo " build vcontainer-tarball so $builddir/tmp/deploy/sdk/vcontainer-standalone.sh exists." >&2
+ exit 1
+ fi
+ fi
+fi
+
+extract_dir="${VCONTAINER_EXTRACT_DIR:-$builddir/vcontainer-test-extracted}"
+rm -rf "$extract_dir"
+mkdir -p "$(dirname "$extract_dir")"
+
+# Self-extracting installer (silent, -y agrees to license, -d picks dir)
+"$sdk_tarball" -d "$extract_dir" -y
+
+# Prepare a Python venv so we don't pollute the worker's system packages.
+python3 -m venv "$builddir/meta-virt-test-venv"
+# shellcheck disable=SC1091
+source "$builddir/meta-virt-test-venv/bin/activate"
+# Avoid warnings by upgrading pip; install pytest/pexpect into the venv via pip.
+python3 -m pip install --quiet --upgrade pip setuptools wheel
+python3 -m pip install --quiet --upgrade pytest pytest-timeout pexpect
+
+# Default marker filter excludes long running / infrastructure dependent tests.
+marker_filter="${META_VIRT_PYTEST_MARKERS:-not slow and not network and not boot and not incus and not k3s}"
+
+# Per-suite test file selection. Uses -k/-m for fine-grained filtering and
+# keeps the CLI small for logging clarity.
+case "$suite" in
+ vdkr)
+ test_files=(
+ "tests/test_vdkr.py"
+ "tests/test_vdkr_registry.py"
+ )
+ ;;
+ vpdmn)
+ test_files=(
+ "tests/test_vpdmn.py"
+ )
+ ;;
+ vcontainer)
+ # Broad vcontainer/bbclass/tooling coverage that doesn't require the
+ # vdkr/vpdmn CLI harness to be running.
+ test_files=(
+ "tests/test_container_cross_install.py"
+ "tests/test_container_registry_script.py"
+ "tests/test_vcontainer_auth_config.py"
+ "tests/test_multiarch_oci.py"
+ "tests/test_multilayer_oci.py"
+ )
+ ;;
+ *)
+ echo "ERROR: unknown suite '$suite' (expected vcontainer|vdkr|vpdmn)" >&2
+ exit 2
+ ;;
+esac
+
+pytest_args=(
+ -v
+ --tb=short
+ -m "$marker_filter"
+ --vdkr-dir "$extract_dir"
+ --junitxml="$builddir/pytest-$suite-results.xml"
+)
+
+# Allow tests that consume an OCI image (import/save/load) to find one.
+if [ -n "${TEST_OCI_IMAGE:-}" ] && [ -d "${TEST_OCI_IMAGE}" ]; then
+ pytest_args+=(--oci-image "$TEST_OCI_IMAGE")
+fi
+
+# Pass architecture through when set in the environment (default is x86_64).
+if [ -n "${VDKR_ARCH:-}" ]; then
+ pytest_args+=(--arch "$VDKR_ARCH")
+fi
+
+cd "$metavirtdir"
+# Don't let a single failing test kill the whole step - collect the junit
+# report, then surface the exit code via the junit file + exit status.
+set +e
+python3 -m pytest "${pytest_args[@]}" "${test_files[@]}"
+rc=$?
+set -e
+
+# Copy artefacts to the results dir if one was provided.
+if [ -n "${RESULTS_DIR:-}" ]; then
+ mkdir -p "$RESULTS_DIR"
+ cp -f "$builddir/pytest-$suite-results.xml" "$RESULTS_DIR/" 2>/dev/null || true
+ if [ -f /tmp/pytest-vcontainer.log ]; then
+ cp -f /tmp/pytest-vcontainer.log "$RESULTS_DIR/pytest-$suite.log" || true
+ fi
+fi
+
+exit $rc
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 09/11] config.json: add vcontainer-tests, vdkr-tests, vpdmn-tests jobs
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Add three pytest job overrides that run the meta-virtualization test
suites against the vcontainer standalone SDK published by the
vcontainer-tarball builder.
Each job runs the new scripts/run-vcontainer-tests helper via
EXTRACMDS (we need BUILDDIR variable so EXTRAPLAINCMDS is not
sufficient) with both NOBUILDTOOLS and NOVCONTAINER set, so the
workers don't spend time setting up either tarball: the test runner
gets the published SDK installer through VCONTAINER_SDK and manages
its own pytest venv.
vcontainer-tests covers the broader bbclass/tooling tests (and so
brings in meta-openembedded layers it transitively exercises);
vdkr-tests and vpdmn-tests cover their respective CLI harnesses.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 42 +++++++++++++++++++++++++++++++++++-
scripts/run-vcontainer-tests | 7 +++---
2 files changed, 44 insertions(+), 5 deletions(-)
diff --git a/config.json b/config.json
index 86c1ffb..dda5b12 100644
--- a/config.json
+++ b/config.json
@@ -1891,7 +1891,7 @@
},
"step2" : {
"shortname" : "Publish vcontainer SDK for test reuse",
- "EXTRAPLAINCMDS" : [
+ "EXTRACMDS" : [
"install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
]
}
@@ -1938,6 +1938,46 @@
"shortname" : "Build 'python' container",
"BBTARGETS" : "app-container-python"
}
+ },
+ "vcontainer-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Run vcontainer pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vcontainer ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
+ },
+ "vdkr-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "step1" : {
+ "shortname" : "Run vdkr pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vdkr ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
+ },
+ "vpdmn-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "step1" : {
+ "shortname" : "Run vpdmn pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vpdmn ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
}
},
"repo-defaults" : {
diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
index cbb5544..1394c7c 100755
--- a/scripts/run-vcontainer-tests
+++ b/scripts/run-vcontainer-tests
@@ -3,7 +3,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
# Run meta-virtualization pytest test suites against the vcontainer
-# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
+# standalone SDK (vdkr/vpdmn) that was built in a previous bitbake
# step.
#
# Arguments:
@@ -21,9 +21,8 @@
#
# The script is intentionally conservative: any pytest tests that cannot run
# in the CI environment (those marked "slow", "network", "boot") are skipped
-# are skipped so that the autobuilder step completes without needing network
-# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
-# before invocation.
+# so that the autobuilder step completes without needing network access. Those
+# can be re-enabled by exporting META_VIRT_PYTEST_MARKERS before invocation.
#
# It is assumed that /dev/kvm is writable by the CI user running the tests,
# since the performance is significantly faster with 'memres'.
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 09/11] config.json: add vcontainer-tests, vdkr-tests, vpdmn-tests jobs
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Add three pytest job overrides that run the meta-virtualization test
suites against the vcontainer standalone SDK published by the
vcontainer-tarball builder.
Each job runs the new scripts/run-vcontainer-tests helper via
EXTRACMDS (we need BUILDDIR variable so EXTRAPLAINCMDS is not
sufficient) with both NOBUILDTOOLS and NOVCONTAINER set, so the
workers don't spend time setting up either tarball: the test runner
gets the published SDK installer through VCONTAINER_SDK and manages
its own pytest venv.
vcontainer-tests covers the broader bbclass/tooling tests (and so
brings in meta-openembedded layers it transitively exercises);
vdkr-tests and vpdmn-tests cover their respective CLI harnesses.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 42 +++++++++++++++++++++++++++++++++++-
scripts/run-vcontainer-tests | 7 +++---
2 files changed, 44 insertions(+), 5 deletions(-)
diff --git a/config.json b/config.json
index 86c1ffb..dda5b12 100644
--- a/config.json
+++ b/config.json
@@ -1891,7 +1891,7 @@
},
"step2" : {
"shortname" : "Publish vcontainer SDK for test reuse",
- "EXTRAPLAINCMDS" : [
+ "EXTRACMDS" : [
"install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
]
}
@@ -1938,6 +1938,46 @@
"shortname" : "Build 'python' container",
"BBTARGETS" : "app-container-python"
}
+ },
+ "vcontainer-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "ADDLAYER" : [
+ "${BUILDDIR}/../meta-openembedded/meta-oe",
+ "${BUILDDIR}/../meta-openembedded/meta-python",
+ "${BUILDDIR}/../meta-openembedded/meta-networking",
+ "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+ "${BUILDDIR}/../meta-virtualization"
+ ],
+ "step1" : {
+ "shortname" : "Run vcontainer pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vcontainer ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
+ },
+ "vdkr-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "step1" : {
+ "shortname" : "Run vdkr pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vdkr ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
+ },
+ "vpdmn-tests": {
+ "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+ "step1" : {
+ "shortname" : "Run vpdmn pytest suite",
+ "NOBUILDTOOLS" : 1,
+ "NOVCONTAINER" : 1,
+ "EXTRACMDS" : [
+ "VCONTAINER_SDK=${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh RESULTS_DIR=${HELPERRESULTSDIR} ${SCRIPTSDIR}/run-vcontainer-tests vpdmn ${BUILDDIR} ${BUILDDIR}/../meta-virtualization"
+ ]
+ }
}
},
"repo-defaults" : {
diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
index cbb5544..1394c7c 100755
--- a/scripts/run-vcontainer-tests
+++ b/scripts/run-vcontainer-tests
@@ -3,7 +3,7 @@
# SPDX-License-Identifier: GPL-2.0-only
#
# Run meta-virtualization pytest test suites against the vcontainer
-# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
+# standalone SDK (vdkr/vpdmn) that was built in a previous bitbake
# step.
#
# Arguments:
@@ -21,9 +21,8 @@
#
# The script is intentionally conservative: any pytest tests that cannot run
# in the CI environment (those marked "slow", "network", "boot") are skipped
-# are skipped so that the autobuilder step completes without needing network
-# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
-# before invocation.
+# so that the autobuilder step completes without needing network access. Those
+# can be re-enabled by exporting META_VIRT_PYTEST_MARKERS before invocation.
#
# It is assumed that /dev/kvm is writable by the CI user running the tests,
# since the performance is significantly faster with 'memres'.
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 09/11] config.json: add vcontainer-tests, vdkr-tests, vpdmn-tests jobs
2026-05-08 2:00 ` tim.orling
(?)
@ 2026-05-11 8:57 ` Paul Barker
2026-05-11 17:52 ` Tim Orling
-1 siblings, 1 reply; 34+ messages in thread
From: Paul Barker @ 2026-05-11 8:57 UTC (permalink / raw)
To: tim.orling, yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 2975 bytes --]
On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> From: Tim Orling <tim.orling@konsulko.com>
>
> Add three pytest job overrides that run the meta-virtualization test
> suites against the vcontainer standalone SDK published by the
> vcontainer-tarball builder.
I think this would be better as one job with three steps.
>
> Each job runs the new scripts/run-vcontainer-tests helper via
> EXTRACMDS (we need BUILDDIR variable so EXTRAPLAINCMDS is not
> sufficient) with both NOBUILDTOOLS and NOVCONTAINER set, so the
> workers don't spend time setting up either tarball: the test runner
> gets the published SDK installer through VCONTAINER_SDK and manages
> its own pytest venv.
>
> vcontainer-tests covers the broader bbclass/tooling tests (and so
> brings in meta-openembedded layers it transitively exercises);
> vdkr-tests and vpdmn-tests cover their respective CLI harnesses.
>
> AI-Generated: Claude Cowork Opus 4.7
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
> config.json | 42 +++++++++++++++++++++++++++++++++++-
> scripts/run-vcontainer-tests | 7 +++---
> 2 files changed, 44 insertions(+), 5 deletions(-)
>
> diff --git a/config.json b/config.json
> index 86c1ffb..dda5b12 100644
> --- a/config.json
> +++ b/config.json
> @@ -1891,7 +1891,7 @@
> },
> "step2" : {
> "shortname" : "Publish vcontainer SDK for test reuse",
> - "EXTRAPLAINCMDS" : [
> + "EXTRACMDS" : [
This fixup should be squashed into the earlier patch in this series that
added this line.
[snip]
> diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
> index cbb5544..1394c7c 100755
> --- a/scripts/run-vcontainer-tests
> +++ b/scripts/run-vcontainer-tests
> @@ -3,7 +3,7 @@
> # SPDX-License-Identifier: GPL-2.0-only
> #
> # Run meta-virtualization pytest test suites against the vcontainer
> -# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
> +# standalone SDK (vdkr/vpdmn) that was built in a previous bitbake
> # step.
> #
> # Arguments:
> @@ -21,9 +21,8 @@
> #
> # The script is intentionally conservative: any pytest tests that cannot run
> # in the CI environment (those marked "slow", "network", "boot") are skipped
> -# are skipped so that the autobuilder step completes without needing network
> -# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
> -# before invocation.
> +# so that the autobuilder step completes without needing network access. Those
> +# can be re-enabled by exporting META_VIRT_PYTEST_MARKERS before invocation.
> #
> # It is assumed that /dev/kvm is writable by the CI user running the tests,
> # since the performance is significantly faster with 'memres'.
As above, these fixups need squashing into the patch that added
run-vcontainer-tests.
Best regards,
--
Paul Barker
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 09/11] config.json: add vcontainer-tests, vdkr-tests, vpdmn-tests jobs
2026-05-11 8:57 ` Paul Barker
@ 2026-05-11 17:52 ` Tim Orling
0 siblings, 0 replies; 34+ messages in thread
From: Tim Orling @ 2026-05-11 17:52 UTC (permalink / raw)
To: Paul Barker; +Cc: yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 3365 bytes --]
On Mon, May 11, 2026 at 1:57 AM Paul Barker <paul@pbarker.dev> wrote:
> On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> > From: Tim Orling <tim.orling@konsulko.com>
> >
> > Add three pytest job overrides that run the meta-virtualization test
> > suites against the vcontainer standalone SDK published by the
> > vcontainer-tarball builder.
>
> I think this would be better as one job with three steps.
>
>
I considered that. No strong arguments either way.
> >
> > Each job runs the new scripts/run-vcontainer-tests helper via
> > EXTRACMDS (we need BUILDDIR variable so EXTRAPLAINCMDS is not
> > sufficient) with both NOBUILDTOOLS and NOVCONTAINER set, so the
> > workers don't spend time setting up either tarball: the test runner
> > gets the published SDK installer through VCONTAINER_SDK and manages
> > its own pytest venv.
> >
> > vcontainer-tests covers the broader bbclass/tooling tests (and so
> > brings in meta-openembedded layers it transitively exercises);
> > vdkr-tests and vpdmn-tests cover their respective CLI harnesses.
> >
> > AI-Generated: Claude Cowork Opus 4.7
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> > ---
> > config.json | 42 +++++++++++++++++++++++++++++++++++-
> > scripts/run-vcontainer-tests | 7 +++---
> > 2 files changed, 44 insertions(+), 5 deletions(-)
> >
> > diff --git a/config.json b/config.json
> > index 86c1ffb..dda5b12 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1891,7 +1891,7 @@
> > },
> > "step2" : {
> > "shortname" : "Publish vcontainer SDK for test reuse",
> > - "EXTRAPLAINCMDS" : [
> > + "EXTRACMDS" : [
>
> This fixup should be squashed into the earlier patch in this series that
> added this line.
>
>
Agreed. Oversight. I'll fix it in V2.
> [snip]
>
> > diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
> > index cbb5544..1394c7c 100755
> > --- a/scripts/run-vcontainer-tests
> > +++ b/scripts/run-vcontainer-tests
> > @@ -3,7 +3,7 @@
> > # SPDX-License-Identifier: GPL-2.0-only
> > #
> > # Run meta-virtualization pytest test suites against the vcontainer
> > -# standalone SDK (vdkr/vpdmn) that was built by the previous bitbake
> > +# standalone SDK (vdkr/vpdmn) that was built in a previous bitbake
> > # step.
> > #
> > # Arguments:
> > @@ -21,9 +21,8 @@
> > #
> > # The script is intentionally conservative: any pytest tests that
> cannot run
> > # in the CI environment (those marked "slow", "network", "boot") are
> skipped
> > -# are skipped so that the autobuilder step completes without needing
> network
> > -# access. Those can be re-enabled by exporting META_VIRT_PYTEST_MARKERS
> > -# before invocation.
> > +# so that the autobuilder step completes without needing network
> access. Those
> > +# can be re-enabled by exporting META_VIRT_PYTEST_MARKERS before
> invocation.
> > #
> > # It is assumed that /dev/kvm is writable by the CI user running the
> tests,
> > # since the performance is significantly faster with 'memres'.
>
> As above, these fixups need squashing into the patch that added
> run-vcontainer-tests.
>
>
Agreed. Oversight. I'll fix it in V2.
> Best regards,
>
> --
> Paul Barker
>
>
[-- Attachment #2: Type: text/html, Size: 4908 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 10/11] scripts: add container registry push, auth, tagging, runtime selection
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Add the push-containers infrastructure that drives the
post-build steps for the 'containers-' jobs. After each build
step the runtime container store is harvested and pushed to
one or more registries with derived per-step tags.
* config.json: add CONTAINER_REGISTRIES, CONTAINER_AUTH_CONFIG,
CONTAINER_RUNTIME, CONTAINER_TAG_CMDS and
CONTAINER_VERSION_RECIPE configuration knobs. Tag
app-container-python with python3 PV via
CONTAINER_VERSION_RECIPE.
* scripts/run-config: drive push-containers as a post-step
action. Tags are generated from recipe and distro metadata
(yocto- tag uses major.minor on snapshots and full PV on
releases) with CONTAINER_VERSION_RECIPE allowing a step to
source PV from a different recipe than the image itself.
* Registry auth is staged via .../config.json or podman
.../auth.json using CONTAINER_AUTH_CONFIG, replacing an
interactive login that could hang. CONTAINER_RUNTIME picks
between vdkr (Docker-compatible) and vpdmn (Podman) runtimes.
* Robustness: skip gracefully when no registries are configured,
fix the OCI directory path, handle memres already running,
and avoid hanging when memres has not yet come up.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 17 +++++-
scripts/run-config | 128 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 143 insertions(+), 2 deletions(-)
diff --git a/config.json b/config.json
index dda5b12..7cdf91a 100644
--- a/config.json
+++ b/config.json
@@ -1,4 +1,4 @@
-{
+ {
"BASE_HOMEDIR" : "/home/pokybuild",
"BASE_SHAREDDIR" : "/srv/autobuilder/autobuilder.yocto.io",
"BASE_PUBLISHDIR" : "/srv/autobuilder/downloads.yoctoproject.org",
@@ -40,6 +40,10 @@
"SDKEXTRAS" : ["SSTATE_MIRRORS += '\\", "file://.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH'", "BB_HASHSERVE = 'auto'", "BB_HASHSERVE_UPSTREAM = '${AUTOBUILDER_HASHSERV}'"],
"BUILDINFO" : false,
"BUILDHISTORY" : false,
+ "CONTAINER_RUNTIME" : "vdkr",
+ "CONTAINER_REGISTRIES" : [],
+ "CONTAINER_TAGS" : ["latest"],
+ "CONTAINER_TAG_CMDS" : [],
"BUILDINFOVARS" : ["INHERIT += 'image-buildinfo'", "IMAGE_BUILDINFO_VARS:append = ' IMAGE_BASENAME IMAGE_NAME'"],
"WRITECONFIG" : true,
"SENDERRORS" : true,
@@ -1908,6 +1912,7 @@
"step1" : {
"shortname" : "Build 'base' container",
"BBTARGETS" : "container-base",
+ "CONTAINER_IMAGES" : {"container-base": "base"},
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
]
@@ -1915,6 +1920,7 @@
"step2" : {
"shortname" : "Build 'curl' container",
"BBTARGETS" : "app-container-curl",
+ "CONTAINER_IMAGES" : {"app-container-curl": "curl"},
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
]
@@ -1934,9 +1940,16 @@
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
],
+ "CONTAINER_TAG_CMDS" : [
+ "_PV_MAJOR=$(echo $_PV | cut -d. -f1)",
+ "_PV_MAJOR_MINOR=$(echo $_PV | cut -d. -f1,2)",
+ "_EXTRA_TAGS=\"$_PV_MAJOR $_PV_MAJOR_MINOR\""
+ ],
"step1" : {
"shortname" : "Build 'python' container",
- "BBTARGETS" : "app-container-python"
+ "BBTARGETS" : "app-container-python",
+ "CONTAINER_IMAGES" : {"app-container-python": "python"},
+ "CONTAINER_VERSION_RECIPE" : "python3"
}
},
"vcontainer-tests": {
diff --git a/scripts/run-config b/scripts/run-config
index 0f5a26a..48e0b85 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -198,6 +198,7 @@ utils.mkdir(args.builddir)
revision = "unknown"
report = utils.ErrorReport(ourconfig, args.target, args.builddir, properties['branch_oecore'], revision)
+push_containers = properties.get("push_containers", False)
errordir = utils.errorreportdir(args.builddir)
utils.mkdir(errordir)
@@ -321,6 +322,133 @@ def handle_stepnum(stepnum):
hp.printheader("Step %s/%s: Running bitbake %s" % (stepnum, maxsteps, sanitytargets))
bitbakecmd(args.builddir, "bitbake %s -k" % (sanitytargets), report, stepnum, args.stepname)
+ # Push container images to registries when push_containers is enabled
+ container_images = utils.getconfigdict("CONTAINER_IMAGES", ourconfig, args.target, stepnum)
+ if container_images and push_containers:
+ if jcfg:
+ addstepentry("push-containers", "Push containers", shortdesc, desc, str(container_images), str(stepnum))
+ elif args.stepname == "push-containers":
+ runtime = utils.getconfigvar("CONTAINER_RUNTIME", ourconfig, args.target, stepnum) or "vdkr"
+ registries = utils.getconfiglist("CONTAINER_REGISTRIES", ourconfig, args.target, stepnum)
+ if not registries:
+ hp.printheader("Step %s/%s: push-containers skipped — CONTAINER_REGISTRIES is empty, no containers pushed" % (stepnum, maxsteps))
+ else:
+ static_tags = utils.getconfiglist("CONTAINER_TAGS", ourconfig, args.target, stepnum)
+ auth_config = utils.getconfigvar("CONTAINER_AUTH_CONFIG", ourconfig, args.target, stepnum)
+ if not auth_config:
+ if runtime == "vpdmn":
+ auth_config = "${HOME}/.config/containers/auth.json"
+ else:
+ auth_config = "${HOME}/.docker/config.json"
+ hp.printheader("Step %s/%s: Pushing container images %s" % (stepnum, maxsteps, list(container_images.keys())))
+ script = [
+ "set -e",
+ "test -w /dev/kvm || { echo 'ERROR: /dev/kvm is not writable, cannot push containers'; exit 1; }",
+ # Always bring up a fresh memres VM in the foreground.
+ #
+ # 'memres status' only checks that the QEMU PID in daemon.pid
+ # is alive (see daemon_is_running()/daemon_status() in
+ # meta-virtualization's vrunner.sh); it returns 0 as soon as
+ # QEMU forks, so a hung/partially-booted VM from a previous
+ # run — or a VM in mid-boot — is reported as healthy. The
+ # subsequent 'login'/'vimport'/'push' commands then hang on
+ # the unresponsive daemon socket.
+ #
+ # 'memres restart' is synchronous: it does stop+start and
+ # runs a PING/PONG readiness probe against the daemon socket
+ # (120s timeout), exiting non-zero if the VM never answers.
+ # Running it in the foreground gives us a trustworthy ready
+ # signal via its exit code, so we can drop the status-poll
+ # loop entirely.
+ #
+ # Install an EXIT trap first so we always tear the daemon
+ # down, even if bitbake -e / vimport / push fails mid-step
+ # under 'set -e'. The trap is armed before the restart so
+ # a restart failure also triggers cleanup.
+ #
+ # Registry auth is staged into the guest at VM boot via
+ # the global '--config' flag — vrunner.sh's setup_auth_share()
+ # copies $AUTH_CONFIG onto a read-only 9p share, and
+ # vdkr-init.sh / vpdmn-init.sh's install_auth_config()
+ # installs it at /root/.docker/config.json (vdkr) or
+ # /run/containers/0/auth.json (vpdmn) inside the guest.
+ # Subsequent 'push' calls use those creds directly, so no
+ # explicit 'login' step is needed. Calling 'login' would
+ # actually hang under the autobuilder (no PTY): when the
+ # memres daemon is running, vcontainer-common.sh dispatches
+ # login via '--daemon-interactive' and blocks reading the
+ # password from stdin (see login case in vcontainer-common.sh).
+ "trap '%s-$(arch) memres stop 2>/dev/null || true' EXIT" % runtime,
+ "%s-$(arch) --config %s memres restart </dev/null" % (runtime, auth_config),
+ ]
+ tag_cmds = utils.getconfiglist("CONTAINER_TAG_CMDS", ourconfig, args.target, stepnum)
+ version_recipe = utils.getconfigvar("CONTAINER_VERSION_RECIPE", ourconfig, args.target, stepnum)
+ for recipe, image in container_images.items():
+ # Extract version metadata from the recipe and distro via
+ # bitbake -e. Steps that need additional derived tags (e.g.
+ # major, major.minor) populate _EXTRA_TAGS via
+ # CONTAINER_TAG_CMDS in their step config.
+ #
+ # PV is sanitized with 'sed s/+.*//' to drop Yocto's
+ # '+git<sha>' suffix on AUTOREV/dev recipes — Docker
+ # reference format does not allow '+' in tags, and the
+ # base PV is what consumers expect.
+ #
+ # DISTRO_VERSION needs context-sensitive handling. Poky's
+ # DISTRO_VERSION resolves to '${PV}+snapshot-${METADATA_REVISION}'
+ # off a tag and just '${PV}' on a release tag. The '+' in
+ # the snapshot form is illegal in a Docker tag, but more
+ # importantly the patch level on a snapshot build (e.g.
+ # '6.0.99' between 6.0 and 6.1) is a moving target that
+ # doesn't correspond to any real release — only the
+ # major.minor line is meaningful. So:
+ # - snapshot build (DISTRO_VERSION contains '+') → tag
+ # with major.minor only, e.g. 'yocto-6.0'.
+ # - release-tag build (no '+') → tag with the full
+ # version, e.g. 'yocto-5.0.5' from the yocto-5.0.5 tag.
+ script += [
+ "_BBENV=$(bitbake -e %s 2>/dev/null) || true" % recipe,
+ "_PV=$(echo \"$_BBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+ "_DISTRO_CODENAME=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_CODENAME=/{ print $2; exit }')",
+ "_DISTRO_VERSION_RAW=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_VERSION=/{ print $2; exit }')",
+ "case \"$_DISTRO_VERSION_RAW\" in",
+ " *+*) _DISTRO_VERSION=$(echo \"${_DISTRO_VERSION_RAW%%+*}\" | cut -d. -f1,2) ;;",
+ " *) _DISTRO_VERSION=\"$_DISTRO_VERSION_RAW\" ;;",
+ "esac",
+ "_DEPLOY_DIR_IMAGE=$(echo \"$_BBENV\" | awk -F'\"' '/^DEPLOY_DIR_IMAGE=/{ print $2; exit }')",
+ "_EXTRA_TAGS=\"\"",
+ ]
+ if version_recipe:
+ # When the image recipe's PV is a wrapper-style
+ # placeholder (e.g. app-container-python_1.0.0.bb,
+ # whose 1.0.0 is meaningless to a downstream user),
+ # CONTAINER_VERSION_RECIPE points at the recipe whose
+ # PV is actually meaningful for the resulting tag —
+ # typically the language runtime or app being packaged
+ # (e.g. python3 -> 3.14.x). Override _PV from that
+ # recipe; image-recipe state still drives
+ # DEPLOY_DIR_IMAGE and DISTRO_* since those are
+ # environment-wide.
+ script += [
+ "_VBBENV=$(bitbake -e %s 2>/dev/null) || true" % version_recipe,
+ "_PV=$(echo \"$_VBBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+ ]
+ script += tag_cmds
+ script.append(
+ "_TAGS=\"%s $_PV $_DISTRO_CODENAME yocto-$_DISTRO_VERSION $_EXTRA_TAGS\"" % " ".join(static_tags)
+ )
+ for registry in registries:
+ # No per-registry 'login': credentials were staged into
+ # the guest by '--config' on 'memres restart' above.
+ script += [
+ "for _tag in $_TAGS; do",
+ " %s-$(arch) vimport ${_DEPLOY_DIR_IMAGE}/%s-latest-oci %s/%s:${_tag}" % (runtime, recipe, registry, image),
+ " %s-$(arch) push %s/%s:${_tag}" % (runtime, registry, image),
+ "done",
+ ]
+ # Tear-down is handled by the EXIT trap installed above.
+ bitbakecmd(args.builddir, "\n".join(script), report, stepnum, args.stepname)
+
# Run any extra commands specified
cmds = utils.getconfiglist("EXTRACMDS", ourconfig, args.target, stepnum)
if jcfg:
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 10/11] scripts: add container registry push, auth, tagging, runtime selection
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Add the push-containers infrastructure that drives the
post-build steps for the 'containers-' jobs. After each build
step the runtime container store is harvested and pushed to
one or more registries with derived per-step tags.
* config.json: add CONTAINER_REGISTRIES, CONTAINER_AUTH_CONFIG,
CONTAINER_RUNTIME, CONTAINER_TAG_CMDS and
CONTAINER_VERSION_RECIPE configuration knobs. Tag
app-container-python with python3 PV via
CONTAINER_VERSION_RECIPE.
* scripts/run-config: drive push-containers as a post-step
action. Tags are generated from recipe and distro metadata
(yocto- tag uses major.minor on snapshots and full PV on
releases) with CONTAINER_VERSION_RECIPE allowing a step to
source PV from a different recipe than the image itself.
* Registry auth is staged via .../config.json or podman
.../auth.json using CONTAINER_AUTH_CONFIG, replacing an
interactive login that could hang. CONTAINER_RUNTIME picks
between vdkr (Docker-compatible) and vpdmn (Podman) runtimes.
* Robustness: skip gracefully when no registries are configured,
fix the OCI directory path, handle memres already running,
and avoid hanging when memres has not yet come up.
AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 17 +++++-
scripts/run-config | 128 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 143 insertions(+), 2 deletions(-)
diff --git a/config.json b/config.json
index dda5b12..7cdf91a 100644
--- a/config.json
+++ b/config.json
@@ -1,4 +1,4 @@
-{
+ {
"BASE_HOMEDIR" : "/home/pokybuild",
"BASE_SHAREDDIR" : "/srv/autobuilder/autobuilder.yocto.io",
"BASE_PUBLISHDIR" : "/srv/autobuilder/downloads.yoctoproject.org",
@@ -40,6 +40,10 @@
"SDKEXTRAS" : ["SSTATE_MIRRORS += '\\", "file://.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH'", "BB_HASHSERVE = 'auto'", "BB_HASHSERVE_UPSTREAM = '${AUTOBUILDER_HASHSERV}'"],
"BUILDINFO" : false,
"BUILDHISTORY" : false,
+ "CONTAINER_RUNTIME" : "vdkr",
+ "CONTAINER_REGISTRIES" : [],
+ "CONTAINER_TAGS" : ["latest"],
+ "CONTAINER_TAG_CMDS" : [],
"BUILDINFOVARS" : ["INHERIT += 'image-buildinfo'", "IMAGE_BUILDINFO_VARS:append = ' IMAGE_BASENAME IMAGE_NAME'"],
"WRITECONFIG" : true,
"SENDERRORS" : true,
@@ -1908,6 +1912,7 @@
"step1" : {
"shortname" : "Build 'base' container",
"BBTARGETS" : "container-base",
+ "CONTAINER_IMAGES" : {"container-base": "base"},
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
]
@@ -1915,6 +1920,7 @@
"step2" : {
"shortname" : "Build 'curl' container",
"BBTARGETS" : "app-container-curl",
+ "CONTAINER_IMAGES" : {"app-container-curl": "curl"},
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
]
@@ -1934,9 +1940,16 @@
"extravars" : [
"DISTRO_FEATURES:append = ' virtualization vcontainer'"
],
+ "CONTAINER_TAG_CMDS" : [
+ "_PV_MAJOR=$(echo $_PV | cut -d. -f1)",
+ "_PV_MAJOR_MINOR=$(echo $_PV | cut -d. -f1,2)",
+ "_EXTRA_TAGS=\"$_PV_MAJOR $_PV_MAJOR_MINOR\""
+ ],
"step1" : {
"shortname" : "Build 'python' container",
- "BBTARGETS" : "app-container-python"
+ "BBTARGETS" : "app-container-python",
+ "CONTAINER_IMAGES" : {"app-container-python": "python"},
+ "CONTAINER_VERSION_RECIPE" : "python3"
}
},
"vcontainer-tests": {
diff --git a/scripts/run-config b/scripts/run-config
index 0f5a26a..48e0b85 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -198,6 +198,7 @@ utils.mkdir(args.builddir)
revision = "unknown"
report = utils.ErrorReport(ourconfig, args.target, args.builddir, properties['branch_oecore'], revision)
+push_containers = properties.get("push_containers", False)
errordir = utils.errorreportdir(args.builddir)
utils.mkdir(errordir)
@@ -321,6 +322,133 @@ def handle_stepnum(stepnum):
hp.printheader("Step %s/%s: Running bitbake %s" % (stepnum, maxsteps, sanitytargets))
bitbakecmd(args.builddir, "bitbake %s -k" % (sanitytargets), report, stepnum, args.stepname)
+ # Push container images to registries when push_containers is enabled
+ container_images = utils.getconfigdict("CONTAINER_IMAGES", ourconfig, args.target, stepnum)
+ if container_images and push_containers:
+ if jcfg:
+ addstepentry("push-containers", "Push containers", shortdesc, desc, str(container_images), str(stepnum))
+ elif args.stepname == "push-containers":
+ runtime = utils.getconfigvar("CONTAINER_RUNTIME", ourconfig, args.target, stepnum) or "vdkr"
+ registries = utils.getconfiglist("CONTAINER_REGISTRIES", ourconfig, args.target, stepnum)
+ if not registries:
+ hp.printheader("Step %s/%s: push-containers skipped — CONTAINER_REGISTRIES is empty, no containers pushed" % (stepnum, maxsteps))
+ else:
+ static_tags = utils.getconfiglist("CONTAINER_TAGS", ourconfig, args.target, stepnum)
+ auth_config = utils.getconfigvar("CONTAINER_AUTH_CONFIG", ourconfig, args.target, stepnum)
+ if not auth_config:
+ if runtime == "vpdmn":
+ auth_config = "${HOME}/.config/containers/auth.json"
+ else:
+ auth_config = "${HOME}/.docker/config.json"
+ hp.printheader("Step %s/%s: Pushing container images %s" % (stepnum, maxsteps, list(container_images.keys())))
+ script = [
+ "set -e",
+ "test -w /dev/kvm || { echo 'ERROR: /dev/kvm is not writable, cannot push containers'; exit 1; }",
+ # Always bring up a fresh memres VM in the foreground.
+ #
+ # 'memres status' only checks that the QEMU PID in daemon.pid
+ # is alive (see daemon_is_running()/daemon_status() in
+ # meta-virtualization's vrunner.sh); it returns 0 as soon as
+ # QEMU forks, so a hung/partially-booted VM from a previous
+ # run — or a VM in mid-boot — is reported as healthy. The
+ # subsequent 'login'/'vimport'/'push' commands then hang on
+ # the unresponsive daemon socket.
+ #
+ # 'memres restart' is synchronous: it does stop+start and
+ # runs a PING/PONG readiness probe against the daemon socket
+ # (120s timeout), exiting non-zero if the VM never answers.
+ # Running it in the foreground gives us a trustworthy ready
+ # signal via its exit code, so we can drop the status-poll
+ # loop entirely.
+ #
+ # Install an EXIT trap first so we always tear the daemon
+ # down, even if bitbake -e / vimport / push fails mid-step
+ # under 'set -e'. The trap is armed before the restart so
+ # a restart failure also triggers cleanup.
+ #
+ # Registry auth is staged into the guest at VM boot via
+ # the global '--config' flag — vrunner.sh's setup_auth_share()
+ # copies $AUTH_CONFIG onto a read-only 9p share, and
+ # vdkr-init.sh / vpdmn-init.sh's install_auth_config()
+ # installs it at /root/.docker/config.json (vdkr) or
+ # /run/containers/0/auth.json (vpdmn) inside the guest.
+ # Subsequent 'push' calls use those creds directly, so no
+ # explicit 'login' step is needed. Calling 'login' would
+ # actually hang under the autobuilder (no PTY): when the
+ # memres daemon is running, vcontainer-common.sh dispatches
+ # login via '--daemon-interactive' and blocks reading the
+ # password from stdin (see login case in vcontainer-common.sh).
+ "trap '%s-$(arch) memres stop 2>/dev/null || true' EXIT" % runtime,
+ "%s-$(arch) --config %s memres restart </dev/null" % (runtime, auth_config),
+ ]
+ tag_cmds = utils.getconfiglist("CONTAINER_TAG_CMDS", ourconfig, args.target, stepnum)
+ version_recipe = utils.getconfigvar("CONTAINER_VERSION_RECIPE", ourconfig, args.target, stepnum)
+ for recipe, image in container_images.items():
+ # Extract version metadata from the recipe and distro via
+ # bitbake -e. Steps that need additional derived tags (e.g.
+ # major, major.minor) populate _EXTRA_TAGS via
+ # CONTAINER_TAG_CMDS in their step config.
+ #
+ # PV is sanitized with 'sed s/+.*//' to drop Yocto's
+ # '+git<sha>' suffix on AUTOREV/dev recipes — Docker
+ # reference format does not allow '+' in tags, and the
+ # base PV is what consumers expect.
+ #
+ # DISTRO_VERSION needs context-sensitive handling. Poky's
+ # DISTRO_VERSION resolves to '${PV}+snapshot-${METADATA_REVISION}'
+ # off a tag and just '${PV}' on a release tag. The '+' in
+ # the snapshot form is illegal in a Docker tag, but more
+ # importantly the patch level on a snapshot build (e.g.
+ # '6.0.99' between 6.0 and 6.1) is a moving target that
+ # doesn't correspond to any real release — only the
+ # major.minor line is meaningful. So:
+ # - snapshot build (DISTRO_VERSION contains '+') → tag
+ # with major.minor only, e.g. 'yocto-6.0'.
+ # - release-tag build (no '+') → tag with the full
+ # version, e.g. 'yocto-5.0.5' from the yocto-5.0.5 tag.
+ script += [
+ "_BBENV=$(bitbake -e %s 2>/dev/null) || true" % recipe,
+ "_PV=$(echo \"$_BBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+ "_DISTRO_CODENAME=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_CODENAME=/{ print $2; exit }')",
+ "_DISTRO_VERSION_RAW=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_VERSION=/{ print $2; exit }')",
+ "case \"$_DISTRO_VERSION_RAW\" in",
+ " *+*) _DISTRO_VERSION=$(echo \"${_DISTRO_VERSION_RAW%%+*}\" | cut -d. -f1,2) ;;",
+ " *) _DISTRO_VERSION=\"$_DISTRO_VERSION_RAW\" ;;",
+ "esac",
+ "_DEPLOY_DIR_IMAGE=$(echo \"$_BBENV\" | awk -F'\"' '/^DEPLOY_DIR_IMAGE=/{ print $2; exit }')",
+ "_EXTRA_TAGS=\"\"",
+ ]
+ if version_recipe:
+ # When the image recipe's PV is a wrapper-style
+ # placeholder (e.g. app-container-python_1.0.0.bb,
+ # whose 1.0.0 is meaningless to a downstream user),
+ # CONTAINER_VERSION_RECIPE points at the recipe whose
+ # PV is actually meaningful for the resulting tag —
+ # typically the language runtime or app being packaged
+ # (e.g. python3 -> 3.14.x). Override _PV from that
+ # recipe; image-recipe state still drives
+ # DEPLOY_DIR_IMAGE and DISTRO_* since those are
+ # environment-wide.
+ script += [
+ "_VBBENV=$(bitbake -e %s 2>/dev/null) || true" % version_recipe,
+ "_PV=$(echo \"$_VBBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+ ]
+ script += tag_cmds
+ script.append(
+ "_TAGS=\"%s $_PV $_DISTRO_CODENAME yocto-$_DISTRO_VERSION $_EXTRA_TAGS\"" % " ".join(static_tags)
+ )
+ for registry in registries:
+ # No per-registry 'login': credentials were staged into
+ # the guest by '--config' on 'memres restart' above.
+ script += [
+ "for _tag in $_TAGS; do",
+ " %s-$(arch) vimport ${_DEPLOY_DIR_IMAGE}/%s-latest-oci %s/%s:${_tag}" % (runtime, recipe, registry, image),
+ " %s-$(arch) push %s/%s:${_tag}" % (runtime, registry, image),
+ "done",
+ ]
+ # Tear-down is handled by the EXIT trap installed above.
+ bitbakecmd(args.builddir, "\n".join(script), report, stepnum, args.stepname)
+
# Run any extra commands specified
cmds = utils.getconfiglist("EXTRACMDS", ourconfig, args.target, stepnum)
if jcfg:
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
2026-05-08 2:00 ` tim.orling
@ 2026-05-08 2:00 ` tim.orling
-1 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 1:25 UTC (permalink / raw)
To: o=yocto-patches, yocto
From: Tim Orling <tim.orling@konsulko.com>
Extend containers-library with two additional demo images sourced
from meta-yocto-containers-demo:
* app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
* app-container-valkey: Valkey (Redis-compatible) key/value datastore.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/config.json b/config.json
index 7cdf91a..a9a2e9c 100644
--- a/config.json
+++ b/config.json
@@ -1950,6 +1950,18 @@
"BBTARGETS" : "app-container-python",
"CONTAINER_IMAGES" : {"app-container-python": "python"},
"CONTAINER_VERSION_RECIPE" : "python3"
+ },
+ "step2" : {
+ "shortname" : "Build 'mosquitto' container",
+ "BBTARGETS" : "app-container-mosquitto",
+ "CONTAINER_IMAGES" : {"app-container-mosquitto": "mosquitto"},
+ "CONTAINER_VERSION_RECIPE" : "mosquitto"
+ },
+ "step3" : {
+ "shortname" : "Build 'valkey' container",
+ "BBTARGETS" : "app-container-valkey",
+ "CONTAINER_IMAGES" : {"app-container-valkey": "valkey"},
+ "CONTAINER_VERSION_RECIPE" : "valkey"
}
},
"vcontainer-tests": {
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
@ 2026-05-08 2:00 ` tim.orling
0 siblings, 0 replies; 34+ messages in thread
From: tim.orling @ 2026-05-08 2:00 UTC (permalink / raw)
To: yocto-patches
From: Tim Orling <tim.orling@konsulko.com>
Extend containers-library with two additional demo images sourced
from meta-yocto-containers-demo:
* app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
* app-container-valkey: Valkey (Redis-compatible) key/value datastore.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
config.json | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/config.json b/config.json
index 7cdf91a..a9a2e9c 100644
--- a/config.json
+++ b/config.json
@@ -1950,6 +1950,18 @@
"BBTARGETS" : "app-container-python",
"CONTAINER_IMAGES" : {"app-container-python": "python"},
"CONTAINER_VERSION_RECIPE" : "python3"
+ },
+ "step2" : {
+ "shortname" : "Build 'mosquitto' container",
+ "BBTARGETS" : "app-container-mosquitto",
+ "CONTAINER_IMAGES" : {"app-container-mosquitto": "mosquitto"},
+ "CONTAINER_VERSION_RECIPE" : "mosquitto"
+ },
+ "step3" : {
+ "shortname" : "Build 'valkey' container",
+ "BBTARGETS" : "app-container-valkey",
+ "CONTAINER_IMAGES" : {"app-container-valkey": "valkey"},
+ "CONTAINER_VERSION_RECIPE" : "valkey"
}
},
"vcontainer-tests": {
--
2.43.0
^ permalink raw reply related [flat|nested] 34+ messages in thread* Re: [yocto] [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
2026-05-08 2:00 ` tim.orling
(?)
@ 2026-05-08 7:14 ` Atiksh Sharma
-1 siblings, 0 replies; 34+ messages in thread
From: Atiksh Sharma @ 2026-05-08 7:14 UTC (permalink / raw)
To: yocto, tim.orling
[-- Attachment #1: Type: text/plain, Size: 2063 bytes --]
CAN YOU JUST STOP SPAMMING?
On Fri, May 8, 2026 at 6:56 AM Tim Orling via lists.yoctoproject.org
<tim.orling=konsulko.com@lists.yoctoproject.org> wrote:
> From: Tim Orling <tim.orling@konsulko.com>
>
> Extend containers-library with two additional demo images sourced
> from meta-yocto-containers-demo:
>
> * app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
> * app-container-valkey: Valkey (Redis-compatible) key/value datastore.
>
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
> config.json | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/config.json b/config.json
> index 7cdf91a..a9a2e9c 100644
> --- a/config.json
> +++ b/config.json
> @@ -1950,6 +1950,18 @@
> "BBTARGETS" : "app-container-python",
> "CONTAINER_IMAGES" : {"app-container-python": "python"},
> "CONTAINER_VERSION_RECIPE" : "python3"
> + },
> + "step2" : {
> + "shortname" : "Build 'mosquitto' container",
> + "BBTARGETS" : "app-container-mosquitto",
> + "CONTAINER_IMAGES" : {"app-container-mosquitto":
> "mosquitto"},
> + "CONTAINER_VERSION_RECIPE" : "mosquitto"
> + },
> + "step3" : {
> + "shortname" : "Build 'valkey' container",
> + "BBTARGETS" : "app-container-valkey",
> + "CONTAINER_IMAGES" : {"app-container-valkey": "valkey"},
> + "CONTAINER_VERSION_RECIPE" : "valkey"
> }
> },
> "vcontainer-tests": {
> --
> 2.43.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#66512):
> https://lists.yoctoproject.org/g/yocto/message/66512
> Mute This Topic: https://lists.yoctoproject.org/mt/119207998/8395439
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> atsharma623@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
[-- Attachment #2: Type: text/html, Size: 3500 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
2026-05-08 2:00 ` tim.orling
(?)
(?)
@ 2026-05-11 9:00 ` Paul Barker
2026-05-11 18:04 ` Tim Orling
-1 siblings, 1 reply; 34+ messages in thread
From: Paul Barker @ 2026-05-11 9:00 UTC (permalink / raw)
To: tim.orling, yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]
On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> From: Tim Orling <tim.orling@konsulko.com>
>
> Extend containers-library with two additional demo images sourced
> from meta-yocto-containers-demo:
>
> * app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
> * app-container-valkey: Valkey (Redis-compatible) key/value datastore.
>
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
> config.json | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/config.json b/config.json
> index 7cdf91a..a9a2e9c 100644
> --- a/config.json
> +++ b/config.json
> @@ -1950,6 +1950,18 @@
> "BBTARGETS" : "app-container-python",
> "CONTAINER_IMAGES" : {"app-container-python": "python"},
> "CONTAINER_VERSION_RECIPE" : "python3"
> + },
> + "step2" : {
> + "shortname" : "Build 'mosquitto' container",
> + "BBTARGETS" : "app-container-mosquitto",
> + "CONTAINER_IMAGES" : {"app-container-mosquitto": "mosquitto"},
> + "CONTAINER_VERSION_RECIPE" : "mosquitto"
> + },
> + "step3" : {
> + "shortname" : "Build 'valkey' container",
> + "BBTARGETS" : "app-container-valkey",
> + "CONTAINER_IMAGES" : {"app-container-valkey": "valkey"},
> + "CONTAINER_VERSION_RECIPE" : "valkey"
This makes me wonder: can we build multiple app container images with
one bitbake command? If so, can we combine these into one step?
Best regards,
--
Paul Barker
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
2026-05-11 9:00 ` Paul Barker
@ 2026-05-11 18:04 ` Tim Orling
2026-05-11 18:24 ` Tim Orling
0 siblings, 1 reply; 34+ messages in thread
From: Tim Orling @ 2026-05-11 18:04 UTC (permalink / raw)
To: Paul Barker; +Cc: yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 2637 bytes --]
On Mon, May 11, 2026 at 2:00 AM Paul Barker <paul@pbarker.dev> wrote:
> On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
> > From: Tim Orling <tim.orling@konsulko.com>
> >
> > Extend containers-library with two additional demo images sourced
> > from meta-yocto-containers-demo:
> >
> > * app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
> > * app-container-valkey: Valkey (Redis-compatible) key/value datastore.
> >
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> > ---
> > config.json | 12 ++++++++++++
> > 1 file changed, 12 insertions(+)
> >
> > diff --git a/config.json b/config.json
> > index 7cdf91a..a9a2e9c 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1950,6 +1950,18 @@
> > "BBTARGETS" : "app-container-python",
> > "CONTAINER_IMAGES" : {"app-container-python": "python"},
> > "CONTAINER_VERSION_RECIPE" : "python3"
> > + },
> > + "step2" : {
> > + "shortname" : "Build 'mosquitto' container",
> > + "BBTARGETS" : "app-container-mosquitto",
> > + "CONTAINER_IMAGES" : {"app-container-mosquitto":
> "mosquitto"},
> > + "CONTAINER_VERSION_RECIPE" : "mosquitto"
> > + },
> > + "step3" : {
> > + "shortname" : "Build 'valkey' container",
> > + "BBTARGETS" : "app-container-valkey",
> > + "CONTAINER_IMAGES" : {"app-container-valkey": "valkey"},
> > + "CONTAINER_VERSION_RECIPE" : "valkey"
>
> This makes me wonder: can we build multiple app container images with
> one bitbake command? If so, can we combine these into one step?
>
The getconfigdict() that was added to scripts/utils.py might make the JSON
definition of CONTAINER_IMAGES awkward as more containers are added, but it
"should" work.
I had considered using a list of recipe names and building them in one
step, rather than a long 'bitbake <recipe 1> <recipe 2> ... <recipe N>'
command. This first attempt was kept simple.
This would also impact pushing containers, as pushing is currently appended
as a step after 'build targets' when the push_containers boolean is true.
Container tooling needs to push one container at a time, which will become
more complicated with signing and attaching SLSA attestations. This might
be another use case for a list of container recipes that can be iterated
through.
I'm open to another design if folks have better ideas for how the AB works.
> Best regards,
>
> --
> Paul Barker
>
>
[-- Attachment #2: Type: text/html, Size: 3966 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread* Re: [yocto-autobuilder-helper][PATCH 11/11] config.json: containers-library: add mosquitto and valkey demos
2026-05-11 18:04 ` Tim Orling
@ 2026-05-11 18:24 ` Tim Orling
0 siblings, 0 replies; 34+ messages in thread
From: Tim Orling @ 2026-05-11 18:24 UTC (permalink / raw)
To: Paul Barker; +Cc: yocto-patches, yocto
[-- Attachment #1: Type: text/plain, Size: 3273 bytes --]
On Mon, May 11, 2026 at 11:04 AM Tim Orling <tim.orling@konsulko.com> wrote:
>
>
> On Mon, May 11, 2026 at 2:00 AM Paul Barker <paul@pbarker.dev> wrote:
>
>> On Thu, 2026-05-07 at 18:25 -0700, tim.orling@konsulko.com wrote:
>> > From: Tim Orling <tim.orling@konsulko.com>
>> >
>> > Extend containers-library with two additional demo images sourced
>> > from meta-yocto-containers-demo:
>> >
>> > * app-container-mosquitto: the Eclipse Mosquitto MQTT broker.
>> > * app-container-valkey: Valkey (Redis-compatible) key/value datastore.
>> >
>> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
>> > ---
>> > config.json | 12 ++++++++++++
>> > 1 file changed, 12 insertions(+)
>> >
>> > diff --git a/config.json b/config.json
>> > index 7cdf91a..a9a2e9c 100644
>> > --- a/config.json
>> > +++ b/config.json
>> > @@ -1950,6 +1950,18 @@
>> > "BBTARGETS" : "app-container-python",
>> > "CONTAINER_IMAGES" : {"app-container-python":
>> "python"},
>> > "CONTAINER_VERSION_RECIPE" : "python3"
>> > + },
>> > + "step2" : {
>> > + "shortname" : "Build 'mosquitto' container",
>> > + "BBTARGETS" : "app-container-mosquitto",
>> > + "CONTAINER_IMAGES" : {"app-container-mosquitto":
>> "mosquitto"},
>> > + "CONTAINER_VERSION_RECIPE" : "mosquitto"
>> > + },
>> > + "step3" : {
>> > + "shortname" : "Build 'valkey' container",
>> > + "BBTARGETS" : "app-container-valkey",
>> > + "CONTAINER_IMAGES" : {"app-container-valkey":
>> "valkey"},
>> > + "CONTAINER_VERSION_RECIPE" : "valkey"
>>
>> This makes me wonder: can we build multiple app container images with
>> one bitbake command? If so, can we combine these into one step?
>>
>
> The getconfigdict() that was added to scripts/utils.py might make the JSON
> definition of CONTAINER_IMAGES awkward as more containers are added, but it
> "should" work.
>
> I had considered using a list of recipe names and building them in one
> step, rather than a long 'bitbake <recipe 1> <recipe 2> ... <recipe N>'
> command. This first attempt was kept simple.
>
> This would also impact pushing containers, as pushing is currently
> appended as a step after 'build targets' when the push_containers boolean
> is true. Container tooling needs to push one container at a time, which
> will become more complicated with signing and attaching SLSA
> attestations. This might be another use case for a list of container
> recipes that can be iterated through.
>
> I'm open to another design if folks have better ideas for how the AB works.
>
>
Capturing some additional thoughts I had on IRC:
do we have any existing examples where a step iterates over a list in the
AB?
11:10 AM
I guess all the container recipes could be in BBTARGETS... but that list
could get quite long as more container recipes are added...
11:13 AM
maybe follow the ptest-packagelists.inc pattern
11:15 AM
which then begs for BBCLASSEXTEND = "${@' '.join(['mcextend:'+x for x in
d.getVar('PTESTS').split()])}" ;)
>
>> Best regards,
>>
>> --
>> Paul Barker
>>
>>
[-- Attachment #2: Type: text/html, Size: 5114 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread