[yocto-autobuilder2][PATCH v3 0/6] Implement 'containers' jobs

[yocto-autobuilder2][PATCH v3 0/6] Implement 'containers' jobs

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

This series builds on top of the patches already merged from v2.

This series contains the 'yocto-autobuilder-helper' patches to enable:
* building, testing and publishing of 'vcontainer-tarball'
  - vcontainer-tests job
* building and pushing containers from:
  - meta-virtualization
* containers are pushed when the push_containers boolean is set in the
  AutoBuilder UI
  - regstries are set with CONTAINER_REGISTRIES variable
* authentication uses local user credentials (e.g. ~/.docker/config.json
  or .../auth.json for podman)
* containers are tagged with PV_MAJOR, PV_MAJOR.PV_MINOR, DISTRO_CODENAME
  and yocto-x.y release (yocto-x.y.z for release builds).

Additional features, such as container signing and attaching SLSA provenance
attestations will come in a future series, once this initial functionality
is agreed upon an stable.

Concerns about the possible infinite loop around the tarball extraction
"while True, try, expect OSError" code [1] are addressed separately [2],
since the original patch has been merged.

This series relies on the resolution of [3] and [4].

[1] https://lore.kernel.org/yocto-patches/02f0c35f16a8a51bbe0e89145353008f67180a84.camel@pbarker.dev/
[2] https://lore.kernel.org/yocto-patches/20260606011918.61582-1-tim.orling@konsulko.com/
[3] https://lore.kernel.org/yocto-meta-virtualization/aiJDIinJWZhO4Jid@gmail.com/T/#t
[4] https://lore.kernel.org/yocto-meta-virtualization/20260604001931.96550-1-tim.orling@konsulko.com/T/#u

Changes in v2:
* Simplify by  merging vdkr-tests and vpdmn-tests into vcontainer-tests
* Simplify by building containers from meta-virtualization:
- Depends on resolution of: https://lists.yoctoproject.org/g/meta-virtualization/message/9826
* Workaround for recent (since Friday May 29, 2026) errors:
  Error: reading blob sha256:<hash>: file integrity checksum failed for "<file>"
  - This is probably related to either sstate changes or recent changes in vcontainer-common...

Changes in v3:
* vcontainer-tarball is installed in jobs that set "vcontainer" similar to "extratools", instead
  of the more global "buildtools" behavior.
* The (rather large amount of) code for pushing containers in run-config is broken out into
  a dedicated run-push-containers script.
* Workaround for pushing containers is moved to run-push-containers. The most recent
  behavior (on top of meta-virt master-next) showed a different error pattern:
  Error: reading blob sha256:<hash>: EOF
  The workaround simply removes all container images from the VM's container-registry before
  importing or pushing new container images. The commit message was reworded accordingly.
* Rather than installing latest versions of pip, setuptools and wheel, silence the pip
  warning with PIP_DISABLE_PIP_VERSION_CHECK=1
* Rather than installing the latest versions of pytest, pytest-timeout and pexpect, install
  from meta-virtualization/tests/requirements.txt (pending).
* Refactor run-vcontainer-tests script to use optargs instead of a mix of positional args
  and env vars.

The following changes since commit 07887d0ace65c873aa929ef31cae52c4c1a8e5c5:

  script/utils: Remove prefix from sha256 parameter if present (2026-06-05 23:41:35 +0100)

are available in the Git repository at:

  https://git.yoctoproject.org/yocto-autobuilder-helper contrib/timo/containers-upstream-v3
  https://git.yoctoproject.org/yocto-autobuilder-helper/log/?h=contrib/timo/containers-upstream-v3

for you to fetch changes up to c21fe441965ef9fd801338f017109592dba8b9e0:

  scripts/run-push-containers: remove all images before push (2026-06-05 18:20:07 -0700)

----------------------------------------------------------------

Tim Orling (6):
  scripts: add vcontainer-tarball setup, integration, and publishing
  config.json: add vcontainer-tarball build target
  scripts: add run-vcontainer-tests for meta-virtualization
  scripts: add container registry push, auth, tagging, runtime selection
  config.json: add 'containers-library' build job
  scripts/run-push-containers: remove all images before push

 config.json                  | 118 +++++++++++++++++++
 scripts/publish-artefacts    |   5 +
 scripts/run-config           |  23 ++++
 scripts/run-push-containers  | 167 +++++++++++++++++++++++++++
 scripts/run-vcontainer-tests | 212 +++++++++++++++++++++++++++++++++++
 scripts/utils.py             |  20 +++-
 6 files changed, 541 insertions(+), 4 deletions(-)
 create mode 100755 scripts/run-push-containers
 create mode 100755 scripts/run-vcontainer-tests

-- 
2.43.0

[yocto-autobuilder2][PATCH v3 1/6] scripts: add vcontainer-tarball setup, integration, and publishing

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Introduce the vcontainer-tarball SDK plumbing used by container build
jobs. The vcontainer-tarball is a meta-virtualization-derived SDK
(modelled after buildtools-tarball) that ships the container build
toolchain so worker jobs do not need to rebuild it for every step.

  * scripts/utils.py: add setup_vcontainer_tarball(), and add an
    env_glob keyword argument to setup_tools_tarball() and
    enable_tools_tarball() so the vcontainer-tarball can source its
    specific environment-setup-ci file rather than the universal
    glob.
  * scripts/run-config: source the vcontainer-tarball environment
    for build-targets when "vcontainer" parameter is passed in,
    similar to "extratools" behavior.
  * scripts/publish-artefacts: publish the vcontainer-tarball
    artefact so downstream test jobs can fetch a stable SDK.

AI-Generated: Claude Cowork Opus 4.8
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 scripts/publish-artefacts |  5 +++++
 scripts/run-config        | 11 +++++++++++
 scripts/utils.py          | 20 ++++++++++++++++----
 3 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/scripts/publish-artefacts b/scripts/publish-artefacts
index e56e131..0e820e9 100755
--- a/scripts/publish-artefacts
+++ b/scripts/publish-artefacts
@@ -146,5 +146,10 @@ case "$target" in
         sha256sums $TMPDIR/deploy/images/qemux86-64
         cp -R --no-dereference --preserve=links $TMPDIR/deploy/images/qemux86-64/*qemux86* $DEST/patchtest
         ;;
+    "vcontainer-tarball")
+        mkdir -p $DEST/vcontainer-tarball
+        sha256sums $TMPDIR/deploy/sdk
+        cp -R --no-dereference --preserve=links $TMPDIR/deploy/sdk/*vcontainer* $DEST/vcontainer-tarball
+        ;;
 esac
 
diff --git a/scripts/run-config b/scripts/run-config
index e896234..90a5996 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -153,6 +153,17 @@ else:
     if args.phase == "init" and args.stepname == "buildtools":
         sys.exit(0)
 
+vcontainer = utils.getconfigvar("vcontainer", ourconfig, args.target)
+if jcfg:
+    if vcontainer:
+        addentry("vcontainer", "Setup vcontainer tarball", "init")
+elif vcontainer:
+    # vcontainer is opt-in per target via the "vcontainer" config variable,
+    # so this is a no-op for targets which don't set it
+    utils.setup_vcontainer_tarball(ourconfig, args.target, args.builddir + "/../vcontainer-tarball")
+    if args.phase == "init" and args.stepname == "vcontainer":
+        sys.exit(0)
+
 extratools = utils.getconfigvar("extratools", ourconfig, args.target)
 if jcfg:
     if extratools:
diff --git a/scripts/utils.py b/scripts/utils.py
index a4dd12e..bddc715 100644
--- a/scripts/utils.py
+++ b/scripts/utils.py
@@ -456,8 +456,8 @@ def sha256_file(filename):
             pass
     return method.hexdigest()
 
-def enable_tools_tarball(btdir, name):
-    btenv = glob.glob(btdir + "/environment-setup*")
+def enable_tools_tarball(btdir, name, env_glob="/environment-setup*"):
+    btenv = glob.glob(btdir + env_glob)
     print("Using %s %s" % (name, btenv))
     # We either parse or wrap all our execution calls, rock and a hard place :(
     with open(btenv[0], "r") as f:
@@ -474,6 +474,18 @@ def enable_tools_tarball(btdir, name):
                 if line in os.environ:
                     del os.environ[line]
 
+# Unlike buildtools (a host/worker property, keyed by worker name globs),
+# the vcontainer-tarball is only needed by specific jobs (e.g.
+# containers-library), so it is keyed off the target/builder via a
+# per-target "vcontainer" config variable, following the extratools pattern.
+def setup_vcontainer_tarball(ourconfig, target, vcdir, checkonly=False):
+    vctarball = getconfigvar("vcontainer", ourconfig, target) or None
+
+    if checkonly:
+        return vctarball
+
+    setup_tools_tarball(ourconfig, vcdir, vctarball, name="vcontainer-tarball", env_glob="/environment-setup-ci")
+
 def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
     bttarball = None
     if "buildtools" in ourconfig and workername:
@@ -488,7 +500,7 @@ def setup_buildtools_tarball(ourconfig, workername, btdir, checkonly=False):
 
     setup_tools_tarball(ourconfig, btdir, bttarball)
 
-def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
+def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools", env_glob="/environment-setup*"):
 
     btenv = None
     if bttarball:
@@ -557,7 +569,7 @@ def setup_tools_tarball(ourconfig, btdir, bttarball, name="buildtools"):
         if not os.path.exists(btdir):
             print("Extracting %s %s" % (name, bttarball))
             subprocess.check_call(["bash", btdlpath, "-d", btdir, "-y"])
-        enable_tools_tarball(btdir, name)
+        enable_tools_tarball(btdir, name, env_glob)
 
 def get_string_from_version(version, milestone=None, rc=None):
     """ Point releases finishing by 0 (e.g 4.0.0, 4.1.0) do no exists,
-- 
2.43.0

[yocto-autobuilder2][PATCH v3 2/6] config.json: add vcontainer-tarball build target

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Add the vcontainer-tarball build target which produces the
meta-virtualization SDK used by downstream container build
jobs. Modelled after the buildtools-tarball target.

The target uses EXTRACMDS to reset BBMULTICONFIG for the worker
shell, places the extravars in the per-step config, and includes
a publish-artefacts step so the resulting SDK tarball is staged
for reuse by container-tests and other consumers.

Since meta-virtualization is the first layer removed by
'remove-layers', our BBMULTICONFIGs become invalid immediately
after layer removal and throw an error.

In scripts/run-config, EXTRACMDS runs after BBTARGETS, but
before 'remove-layers'. Use 'sed' to reset BBMULTICONFIG to ''.

The error in the 'remove-layers' step can be summarized as:

bb.parse.ParseError: ParseError at /home/pokybuild/yocto-worker/vcontainer-tarball/build/layers/openembedded-core/meta/conf/bitbake.conf:824: Could not include required file conf/multiconfig/vruntime-aarch64.conf

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 config.json | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/config.json b/config.json
index 4904439..2971fe9 100644
--- a/config.json
+++ b/config.json
@@ -1827,6 +1827,32 @@
         },
         "toaster" : {
             "EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests ${HELPERBUILDDIR} ${HELPERBUILDDIR}/../layers/bitbake"]
+        },
+        "vcontainer-tarball": {
+            "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+            "ADDLAYER" : [
+                "${BUILDDIR}/../meta-openembedded/meta-oe",
+                "${BUILDDIR}/../meta-openembedded/meta-python",
+                "${BUILDDIR}/../meta-openembedded/meta-networking",
+                "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+                "${BUILDDIR}/../meta-virtualization"
+            ],
+            "step1" : {
+                "shortname" : "Build vcontainer-tarballs",
+                "BBTARGETS" : "vcontainer-tarball",
+                "extravars" : [
+                    "DISTRO_FEATURES:append = ' virtualization vcontainer'",
+                    "BBMULTICONFIG = 'vruntime-aarch64 vruntime-x86-64'",
+                    "INIT_MANAGER = 'systemd'"
+                ],
+                "EXTRACMDS" : ["sed -i '/vruntime-aarch64 vruntime-x86-64/d' ${HELPERBUILDDIR}/conf/auto.conf"]
+            },
+            "step2" : {
+                "shortname" : "Publish vcontainer SDK for test reuse",
+                "EXTRACMDS" : [
+                    "install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
+                ]
+            }
         }
     },
     "repo-defaults" : {
-- 
2.43.0

[yocto-autobuilder2][PATCH v3 3/6] scripts: add run-vcontainer-tests for meta-virtualization

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Add scripts/run-vcontainer-tests, the test runner used by the
vcontainers-test job. It sources the vcontainer-tarball SDK,
discovers the meta-virtualization pytest suite, and runs a
configurable set of suites (vdkr, vpdmn, memres) against the
checked-out layers. Suites can be selected per-step so the
top-level 'vcontainer-tests' job runs the container engine
agnostic tests:

- tests/test_container_cross_install.py
- tests/test_container_registry_script.py
- tests/test_vcontainer_auth_config.py
- tests/test_multiarch_oci.py
- tests/test_multilayer_oci.py

The 'vdkr-tests' and 'vpdmn-tests' jobs run only their respective
suites (including memres for each container engine):

- tests/test_vdkr.py
- tests/test_vdkr_registry.py

and

- tests/test_vpdmn.py

The purpose of this script is to test the just built vcontainer-tarball,
so that it can be considered "known-good" and be published for use by
container building jobs that need the vcontainer-tarball installed.

AI-Generated: Claude Cowork Opus 4.7
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 config.json                  |  31 +++++
 scripts/run-vcontainer-tests | 212 +++++++++++++++++++++++++++++++++++
 2 files changed, 243 insertions(+)
 create mode 100755 scripts/run-vcontainer-tests

diff --git a/config.json b/config.json
index 2971fe9..3fccd83 100644
--- a/config.json
+++ b/config.json
@@ -1853,6 +1853,37 @@
                     "install -d ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest && install -m 0755 ${BUILDDIR}/tmp/deploy/sdk/vcontainer-standalone.sh ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new && mv -f ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh.new ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh"
                 ]
             }
+        },
+        "vcontainer-tests": {
+            "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+            "ADDLAYER" : [
+                "${BUILDDIR}/../meta-openembedded/meta-oe",
+                "${BUILDDIR}/../meta-openembedded/meta-python",
+                "${BUILDDIR}/../meta-openembedded/meta-networking",
+                "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+                "${BUILDDIR}/../meta-virtualization"
+            ],
+            "step1" : {
+                "shortname" : "Run vcontainer pytest suite",
+                "NOBUILDTOOLS" : 1,
+                "EXTRACMDS" : [
+                    "${SCRIPTSDIR}/run-vcontainer-tests -s vcontainer -b ${BUILDDIR} -m ${BUILDDIR}/../meta-virtualization -S ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh -r ${HELPERRESULTSDIR}"
+                ]
+            },
+            "step2" : {
+                "shortname" : "Run vdkr pytest suite",
+                "NOBUILDTOOLS" : 1,
+                "EXTRACMDS" : [
+                    "${SCRIPTSDIR}/run-vcontainer-tests -s vdkr -b ${BUILDDIR} -m ${BUILDDIR}/../meta-virtualization -S ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh -r ${HELPERRESULTSDIR}"
+                ]
+            },
+            "step3" : {
+                "shortname" : "Run vpdmn pytest suite",
+                "NOBUILDTOOLS" : 1,
+                "EXTRACMDS" : [
+                    "${SCRIPTSDIR}/run-vcontainer-tests -s vpdmn -b ${BUILDDIR} -m ${BUILDDIR}/../meta-virtualization -S ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh -r ${HELPERRESULTSDIR}"
+                ]
+            }
         }
     },
     "repo-defaults" : {
diff --git a/scripts/run-vcontainer-tests b/scripts/run-vcontainer-tests
new file mode 100755
index 0000000..a7dbab3
--- /dev/null
+++ b/scripts/run-vcontainer-tests
@@ -0,0 +1,212 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Run meta-virtualization pytest test suites against the vcontainer
+# standalone SDK (vdkr/vpdmn) that was built in a previous bitbake
+# step.
+#
+# Usage:
+#   run-vcontainer-tests -s <suite> -b <builddir> -m <metavirtdir> [options]
+#
+# Required:
+#   -s <suite>        suite name: one of "vcontainer", "vdkr", "vpdmn"
+#   -b <builddir>     bitbake build directory (${BUILDDIR})
+#   -m <metavirtdir>  path to the meta-virtualization layer
+#
+# Optional:
+#   -S <sdk>          path to the vcontainer standalone SDK installer. The
+#                     autobuilder -tests jobs share the SDK produced by the
+#                     separate vcontainer-tarball builder. When unset, falls
+#                     back to looking in <builddir>/tmp/deploy/sdk.
+#   -e <extractdir>   where to extract the standalone SDK tarball
+#                     (default: <builddir>/vcontainer-test-extracted)
+#   -i <oci_image>    path to an OCI image directory (enables vdkr/vpdmn
+#                     import tests)
+#   -a <arch>         target architecture for vdkr/vpdmn tests (default: x86_64)
+#   -r <resultsdir>   directory to copy pytest artefacts (junit xml / log) to
+#   -k <markers>      pytest marker filter (default excludes long running /
+#                     infrastructure dependent tests)
+#   -h                show this help and exit
+#
+# The script is intentionally conservative: any pytest tests that cannot run
+# in the CI environment (those marked "slow", "network", "boot") are skipped
+# so that the autobuilder step completes without needing network access. Those
+# can be re-enabled via -k before invocation.
+#
+# It is assumed that /dev/kvm is writable by the CI user running the tests,
+# since the performance is significantly faster with 'memres'.
+#
+
+set -e
+set -u
+set -o pipefail
+set -x
+
+usage() {
+    cat >&2 <<'EOF'
+Usage: run-vcontainer-tests -s <suite> -b <builddir> -m <metavirtdir> [options]
+
+Required:
+  -s <suite>        vcontainer | vdkr | vpdmn
+  -b <builddir>     bitbake build directory
+  -m <metavirtdir>  path to the meta-virtualization layer
+
+Optional:
+  -S <sdk>          path to the vcontainer standalone SDK installer
+  -e <extractdir>   where to extract the standalone SDK tarball
+  -i <oci_image>    path to an OCI image directory
+  -a <arch>         target architecture (default: x86_64)
+  -r <resultsdir>   directory to copy pytest artefacts to
+  -k <markers>      pytest marker filter
+  -h                show this help and exit
+EOF
+}
+
+suite=""
+builddir=""
+metavirtdir=""
+sdk_tarball=""
+extract_dir=""
+oci_image=""
+arch=""
+results_dir=""
+marker_filter="not slow and not network and not boot and not incus and not k3s"
+
+while getopts ":s:b:m:S:e:i:a:r:k:h" opt; do
+    case "$opt" in
+        s) suite="$OPTARG" ;;
+        b) builddir="$OPTARG" ;;
+        m) metavirtdir="$OPTARG" ;;
+        S) sdk_tarball="$OPTARG" ;;
+        e) extract_dir="$OPTARG" ;;
+        i) oci_image="$OPTARG" ;;
+        a) arch="$OPTARG" ;;
+        r) results_dir="$OPTARG" ;;
+        k) marker_filter="$OPTARG" ;;
+        h) usage; exit 0 ;;
+        :) echo "ERROR: option -$OPTARG requires an argument" >&2; usage; exit 2 ;;
+        \?) echo "ERROR: unknown option -$OPTARG" >&2; usage; exit 2 ;;
+    esac
+done
+
+if [ -z "$suite" ] || [ -z "$builddir" ] || [ -z "$metavirtdir" ]; then
+    echo "ERROR: -s, -b and -m are required" >&2
+    usage
+    exit 2
+fi
+
+builddir=$(realpath "$builddir")
+metavirtdir=$(realpath "$metavirtdir")
+
+if [ ! -d "$metavirtdir/tests" ]; then
+    echo "ERROR: meta-virtualization tests directory not found at $metavirtdir/tests" >&2
+    exit 1
+fi
+
+# Locate the vcontainer standalone SDK tarball. Prefer an explicitly-provided
+# SDK (-S), and fall back to looking in the local build's
+# deploy/sdk directory when running stand-alone.
+if [ -n "$sdk_tarball" ]; then
+    if [ ! -f "$sdk_tarball" ]; then
+        echo "ERROR: SDK installer '$sdk_tarball' is set but not a file" >&2
+        exit 1
+    fi
+else
+    sdk_tarball="$builddir/tmp/deploy/sdk/vcontainer-standalone.sh"
+    if [ ! -f "$sdk_tarball" ]; then
+        # Try to find any matching tarball in case naming changed (e.g. versioned)
+        alt=$(ls -1 "$builddir"/tmp/deploy/sdk/vcontainer-*.sh 2>/dev/null | head -n1 || true)
+        if [ -n "$alt" ]; then
+            sdk_tarball="$alt"
+        else
+            echo "ERROR: vcontainer standalone SDK not found." >&2
+            echo "       Pass -S with an existing SDK installer, or" >&2
+            echo "       build vcontainer-tarball so $builddir/tmp/deploy/sdk/vcontainer-standalone.sh exists." >&2
+            exit 1
+        fi
+    fi
+fi
+
+extract_dir="${extract_dir:-$builddir/vcontainer-test-extracted}"
+rm -rf "$extract_dir"
+mkdir -p "$(dirname "$extract_dir")"
+
+# Self-extracting installer (silent, -y agrees to license, -d picks dir)
+"$sdk_tarball" -d "$extract_dir" -y
+
+# Prepare a Python venv so we don't pollute the worker's system packages.
+python3 -m venv "$builddir/meta-virt-test-venv"
+# shellcheck disable=SC1091
+source "$builddir/meta-virt-test-venv/bin/activate"
+# Avoid warnings by upgrading pip; install pytest/pexpect into the venv via pip.
+export PIP_DISABLE_PIP_VERSION_CHECK=1
+python3 -m pip install --quiet -r "$metavirtdir/tests/requirements.txt"
+
+# Per-suite test file selection. Uses -k/-m for fine-grained filtering and
+# keeps the CLI small for logging clarity.
+case "$suite" in
+    vdkr)
+        test_files=(
+            "tests/test_vdkr.py"
+            "tests/test_vdkr_registry.py"
+        )
+        ;;
+    vpdmn)
+        test_files=(
+            "tests/test_vpdmn.py"
+        )
+        ;;
+    vcontainer)
+        # Broad vcontainer/bbclass/tooling coverage that doesn't require the
+        # vdkr/vpdmn CLI harness to be running.
+        test_files=(
+            "tests/test_container_cross_install.py"
+            "tests/test_container_registry_script.py"
+            "tests/test_vcontainer_auth_config.py"
+            "tests/test_multiarch_oci.py"
+            "tests/test_multilayer_oci.py"
+        )
+        ;;
+    *)
+        echo "ERROR: unknown suite '$suite' (expected vcontainer|vdkr|vpdmn)" >&2
+        exit 2
+        ;;
+esac
+
+pytest_args=(
+    -v
+    --tb=short
+    -m "$marker_filter"
+    --vdkr-dir "$extract_dir"
+    --junitxml="$builddir/pytest-$suite-results.xml"
+)
+
+# Allow tests that consume an OCI image (import/save/load) to find one.
+if [ -n "$oci_image" ] && [ -d "$oci_image" ]; then
+    pytest_args+=(--oci-image "$oci_image")
+fi
+
+# Pass architecture through when set (default is x86_64).
+if [ -n "$arch" ]; then
+    pytest_args+=(--arch "$arch")
+fi
+
+cd "$metavirtdir"
+# Don't let a single failing test kill the whole step - collect the junit
+# report, then surface the exit code via the junit file + exit status.
+set +e
+python3 -m pytest "${pytest_args[@]}" "${test_files[@]}"
+rc=$?
+set -e
+
+# Copy artefacts to the results dir if one was provided.
+if [ -n "$results_dir" ]; then
+    mkdir -p "$results_dir"
+    cp -f "$builddir/pytest-$suite-results.xml" "$results_dir/" 2>/dev/null || true
+    if [ -f /tmp/pytest-vcontainer.log ]; then
+        cp -f /tmp/pytest-vcontainer.log "$results_dir/pytest-$suite.log" || true
+    fi
+fi
+
+exit $rc
-- 
2.43.0

[yocto-autobuilder2][PATCH v3 4/6] scripts: add container registry push, auth, tagging, runtime selection

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Add the push-containers infrastructure that drives the
post-build steps for the 'containers-library' job. After
each build step the runtime container store is harvested
and pushed to one or more registries with derived per-step
tags.

* config.json: add CONTAINER_REGISTRIES, CONTAINER_AUTH_CONFIG,
  CONTAINER_RUNTIME, CONTAINER_TAG_CMDS, CONTAINER_VERSION_RECIPE
  and CONTAINER_IMAGE_MAP configuration knobs.
* scripts/run-config: drive push-containers as a post-step
  action (by running scripts/run-push-containers).
* scripts/run-push-containers: Tags are generated from recipe and
  distro metadata (yocto- tag uses major.minor on snapshots and full
  PV on releases) with CONTAINER_VERSION_RECIPE allowing a step to
  source PV from a different recipe than the image itself.
* Registry auth is staged via .../config.json or podman
  .../auth.json using CONTAINER_AUTH_CONFIG, replacing an
  interactive login that could hang. CONTAINER_RUNTIME picks
  between vdkr (Docker-compatible) and vpdmn (Podman) runtimes.
* Robustness: skip gracefully when no registries are configured,
  fix the OCI directory path, handle memres already running,
  and avoid hanging when memres has not yet come up.

AI-Generated: Claude Cowork Opus 4.8
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 config.json                 |   5 ++
 scripts/run-config          |  12 +++
 scripts/run-push-containers | 165 ++++++++++++++++++++++++++++++++++++
 3 files changed, 182 insertions(+)
 create mode 100755 scripts/run-push-containers

diff --git a/config.json b/config.json
index 3fccd83..e1ea01a 100644
--- a/config.json
+++ b/config.json
@@ -43,6 +43,11 @@
         "BUILDINFOVARS" : ["INHERIT += 'image-buildinfo'", "IMAGE_BUILDINFO_VARS:append = ' IMAGE_BASENAME IMAGE_NAME'"],
         "WRITECONFIG" : true,
         "SENDERRORS" : true,
+        "CONTAINER_RUNTIME" : "vpdmn",
+        "CONTAINER_REGISTRIES" : [],
+        "CONTAINER_TAGS" : ["latest"],
+        "CONTAINER_TAG_CMDS" : [],
+        "CONTAINER_IMAGE_MAP" : {},
         "extravars" : [
             "SANITY_TESTED_DISTROS = ''",
             "BB_HASHSERVE = '${AUTOBUILDER_HASHSERV}'",
diff --git a/scripts/run-config b/scripts/run-config
index 90a5996..31db212 100755
--- a/scripts/run-config
+++ b/scripts/run-config
@@ -195,6 +195,8 @@ utils.mkdir(errordir)
 
 errorlogs = set()
 
+push_containers = properties.get("push_containers", False)
+
 def log_file_contents(filename, builddir, stepnum, stepname):
     logfile = logname(builddir, stepnum, stepname)
     with open(logfile, "a") as outf, open(filename, "r") as f:
@@ -313,6 +315,16 @@ def handle_stepnum(stepnum):
             hp.printheader("Step %s/%s: Running bitbake %s" % (stepnum, maxsteps, sanitytargets))
             bitbakecmd(args.builddir, "bitbake %s -k" % (sanitytargets), report, stepnum, args.stepname)
 
+    # Push container images to registries when push_containers is enabled.
+    # The push logic itself lives in scripts/run-push-containers.
+    container_images = utils.getconfigdict("CONTAINER_IMAGE_MAP", ourconfig, args.target, stepnum)
+    if container_images and push_containers:
+        if jcfg:
+            addstepentry("push-containers", "Push containers", shortdesc, desc, str(container_images), str(stepnum))
+        elif args.stepname == "push-containers":
+            hp.printheader("Step %s/%s: Pushing container images %s" % (stepnum, maxsteps, list(container_images.keys())))
+            bitbakecmd(args.builddir, "%s/run-push-containers %s %s" % (scriptsdir, args.target, stepnum), report, stepnum, args.stepname)
+
     # Run any extra commands specified
     cmds = utils.getconfiglist("EXTRACMDS", ourconfig, args.target, stepnum)
     if jcfg:
diff --git a/scripts/run-push-containers b/scripts/run-push-containers
new file mode 100755
index 0000000..cdef497
--- /dev/null
+++ b/scripts/run-push-containers
@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Push container images built by a target step to the configured registries.
+#
+# Invoked by run-config for 'push-containers' steps with the OE build
+# environment already sourced (bitbake and the vcontainer runtime wrappers
+# must be on PATH). All configuration comes from the target/step config:
+#
+#   CONTAINER_IMAGE_MAP      - dict of image recipe -> registry image name
+#   CONTAINER_RUNTIME        - vpdmn (default) or vdkr
+#   CONTAINER_REGISTRIES     - list of registries to push to
+#   CONTAINER_TAGS           - list of static tags (e.g. latest)
+#   CONTAINER_TAG_CMDS       - extra shell to populate _EXTRA_TAGS
+#   CONTAINER_VERSION_RECIPE - recipe whose PV provides the version tag
+#   CONTAINER_AUTH_CONFIG    - registry auth file staged into the guest
+#
+
+import subprocess
+import sys
+
+import utils
+
+parser = utils.ArgParser(description='Pushes container images for a target step to the configured registries.')
+
+parser.add_argument('target',
+                    help="The target build name")
+parser.add_argument('stepnum',
+                    type=int,
+                    help="The step number within the target")
+
+args = parser.parse_args()
+
+ourconfig = utils.loadconfig()
+
+container_images = utils.getconfigdict("CONTAINER_IMAGE_MAP", ourconfig, args.target, args.stepnum)
+if not container_images:
+    print("No CONTAINER_IMAGE_MAP for %s step %s, nothing to push" % (args.target, args.stepnum))
+    sys.exit(0)
+
+registries = utils.getconfiglist("CONTAINER_REGISTRIES", ourconfig, args.target, args.stepnum)
+if not registries:
+    utils.printheader("push-containers skipped — CONTAINER_REGISTRIES is empty, no containers pushed")
+    sys.exit(0)
+
+runtime = utils.getconfigvar("CONTAINER_RUNTIME", ourconfig, args.target, args.stepnum) or "vpdmn"
+static_tags = utils.getconfiglist("CONTAINER_TAGS", ourconfig, args.target, args.stepnum)
+auth_config = utils.getconfigvar("CONTAINER_AUTH_CONFIG", ourconfig, args.target, args.stepnum)
+if not auth_config:
+    if runtime == "vpdmn":
+        auth_config = "${HOME}/.config/containers/auth.json"
+    else:
+        auth_config = "${HOME}/.docker/config.json"
+
+utils.printheader("Pushing container images %s" % list(container_images.keys()))
+
+script = [
+    "set -e",
+    "test -w /dev/kvm || { echo 'ERROR: /dev/kvm is not writable, cannot push containers'; exit 1; }",
+    # Always bring up a fresh memres VM in the foreground.
+    #
+    # 'memres status' only checks that the QEMU PID in daemon.pid
+    # is alive (see daemon_is_running()/daemon_status() in
+    # meta-virtualization's vrunner.sh); it returns 0 as soon as
+    # QEMU forks, so a hung/partially-booted VM from a previous
+    # run — or a VM in mid-boot — is reported as healthy. The
+    # subsequent 'login'/'vimport'/'push' commands then hang on
+    # the unresponsive daemon socket.
+    #
+    # 'memres restart' is synchronous: it does stop+start and
+    # runs a PING/PONG readiness probe against the daemon socket
+    # (120s timeout), exiting non-zero if the VM never answers.
+    # Running it in the foreground gives us a trustworthy ready
+    # signal via its exit code, so we can drop the status-poll
+    # loop entirely.
+    #
+    # Install an EXIT trap first so we always tear the daemon
+    # down, even if bitbake -e / vimport / push fails mid-step
+    # under 'set -e'. The trap is armed before the restart so
+    # a restart failure also triggers cleanup.
+    #
+    # Registry auth is staged into the guest at VM boot via
+    # the global '--config' flag — vrunner.sh's setup_auth_share()
+    # copies $AUTH_CONFIG onto a read-only 9p share, and
+    # vdkr-init.sh / vpdmn-init.sh's install_auth_config()
+    # installs it at /root/.docker/config.json (vdkr) or
+    # /run/containers/0/auth.json (vpdmn) inside the guest.
+    # Subsequent 'push' calls use those creds directly, so no
+    # explicit 'login' step is needed. Calling 'login' would
+    # actually hang under the autobuilder (no PTY): when the
+    # memres daemon is running, vcontainer-common.sh dispatches
+    # login via '--daemon-interactive' and blocks reading the
+    # password from stdin (see login case in vcontainer-common.sh).
+    "trap '%s-$(arch) memres stop 2>/dev/null || true' EXIT" % runtime,
+    "%s-$(arch) --config %s memres restart </dev/null" % (runtime, auth_config),
+]
+tag_cmds = utils.getconfiglist("CONTAINER_TAG_CMDS", ourconfig, args.target, args.stepnum)
+version_recipe = utils.getconfigvar("CONTAINER_VERSION_RECIPE", ourconfig, args.target, args.stepnum)
+for recipe, image in container_images.items():
+    # Extract version metadata from the recipe and distro via
+    # bitbake -e. Steps that need additional derived tags (e.g.
+    # major, major.minor) populate _EXTRA_TAGS via
+    # CONTAINER_TAG_CMDS in their step config.
+    #
+    # PV is sanitized with 'sed s/+.*//' to drop Yocto's
+    # '+git<sha>' suffix on AUTOREV/dev recipes — Docker
+    # reference format does not allow '+' in tags, and the
+    # base PV is what consumers expect.
+    #
+    # DISTRO_VERSION needs context-sensitive handling. Poky's
+    # DISTRO_VERSION resolves to '${PV}+snapshot-${METADATA_REVISION}'
+    # off a tag and just '${PV}' on a release tag. The '+' in
+    # the snapshot form is illegal in a Docker tag, but more
+    # importantly the patch level on a snapshot build (e.g.
+    # '6.0.99' between 6.0 and 6.1) is a moving target that
+    # doesn't correspond to any real release — only the
+    # major.minor line is meaningful. So:
+    #   - snapshot build (DISTRO_VERSION contains '+') → tag
+    #     with major.minor only, e.g. 'yocto-6.0'.
+    #   - release-tag build (no '+') → tag with the full
+    #     version, e.g. 'yocto-5.0.5' from the yocto-5.0.5 tag.
+    script += [
+        "_BBENV=$(bitbake -e %s 2>/dev/null) || true" % recipe,
+        "_PV=$(echo \"$_BBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+        "_DISTRO_CODENAME=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_CODENAME=/{ print $2; exit }')",
+        "_DISTRO_VERSION_RAW=$(echo \"$_BBENV\" | awk -F'\"' '/^DISTRO_VERSION=/{ print $2; exit }')",
+        "case \"$_DISTRO_VERSION_RAW\" in",
+        "    *+*) _DISTRO_VERSION=$(echo \"${_DISTRO_VERSION_RAW%%+*}\" | cut -d. -f1,2) ;;",
+        "    *)   _DISTRO_VERSION=\"$_DISTRO_VERSION_RAW\" ;;",
+        "esac",
+        "_DEPLOY_DIR_IMAGE=$(echo \"$_BBENV\" | awk -F'\"' '/^DEPLOY_DIR_IMAGE=/{ print $2; exit }')",
+        "_EXTRA_TAGS=\"\"",
+    ]
+    if version_recipe:
+        # When the image recipe's PV is a wrapper-style
+        # placeholder (e.g. app-container-python_1.0.0.bb,
+        # whose 1.0.0 is meaningless to a downstream user),
+        # CONTAINER_VERSION_RECIPE points at the recipe whose
+        # PV is actually meaningful for the resulting tag —
+        # typically the language runtime or app being packaged
+        # (e.g. python3 -> 3.14.x). Override _PV from that
+        # recipe; image-recipe state still drives
+        # DEPLOY_DIR_IMAGE and DISTRO_* since those are
+        # environment-wide.
+        script += [
+            "_VBBENV=$(bitbake -e %s 2>/dev/null) || true" % version_recipe,
+            "_PV=$(echo \"$_VBBENV\" | awk -F'\"' '/^PV=/{ print $2; exit }' | sed 's/+.*//')",
+        ]
+    script += tag_cmds
+    script.append(
+        "_TAGS=\"%s $_PV $_DISTRO_CODENAME yocto-$_DISTRO_VERSION $_EXTRA_TAGS\"" % " ".join(static_tags)
+    )
+    for registry in registries:
+        # No per-registry 'login': credentials were staged into
+        # the guest by '--config' on 'memres restart' above.
+        script += [
+            "for _tag in $_TAGS; do",
+            "    %s-$(arch) vimport ${_DEPLOY_DIR_IMAGE}/%s-latest-oci %s/%s:${_tag}" % (runtime, recipe, registry, image),
+            "    %s-$(arch) push %s/%s:${_tag}" % (runtime, registry, image),
+            "done",
+        ]
+# Tear-down is handled by the EXIT trap installed above.
+utils.flush()
+sys.exit(subprocess.call(["/bin/bash", "-c", "\n".join(script)]))
-- 
2.43.0

[yocto-autobuilder2][PATCH v3 5/6] config.json: add 'containers-library' build job

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Add 'containers-library' build job that builds container images on top
of the vcontainer-tarball SDK:

  * original container images from the intial "container-cross-install"
    branch of  meta-virtualization
    - container-base
    - app-container-curl
  * additional images modelled after docker.io/library/*
    - app-container-python
    - app-container-mosquitto
    - app-container-valkey
    - app-container-nginx
  * Tag containers with versions based on the recipe to which they are
    mapped, e.g. python:3, python:3.14, python:3.14.5. Also tag with
    DISTRO_CODENAME and DISTRO_VERSION, e.g. wrynose and yocto-6.0.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 config.json | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/config.json b/config.json
index e1ea01a..18a2b6b 100644
--- a/config.json
+++ b/config.json
@@ -1889,6 +1889,62 @@
                     "${SCRIPTSDIR}/run-vcontainer-tests -s vpdmn -b ${BUILDDIR} -m ${BUILDDIR}/../meta-virtualization -S ${BASE_SHAREDDIR}/pub/vcontainer-tarball-latest/vcontainer-standalone.sh -r ${HELPERRESULTSDIR}"
                 ]
             }
+        },
+        "containers-library": {
+            "NEEDREPOS" : ["bitbake", "meta-openembedded", "meta-virtualization"],
+            "ADDLAYER" : [
+                "${BUILDDIR}/../meta-openembedded/meta-oe",
+                "${BUILDDIR}/../meta-openembedded/meta-python",
+                "${BUILDDIR}/../meta-openembedded/meta-networking",
+                "${BUILDDIR}/../meta-openembedded/meta-filesystems",
+                "${BUILDDIR}/../meta-openembedded/meta-webserver",
+                "${BUILDDIR}/../meta-virtualization"
+            ],
+            "extravars" : [
+                "DISTRO_FEATURES:append = ' virtualization vcontainer'"
+            ],
+            "vcontainer" : "${VCONTAINER_TARBALL_URL}",
+            "CONTAINER_TAG_CMDS" : [
+                "_PV_MAJOR=$(echo $_PV | cut -d. -f1)",
+                "_PV_MAJOR_MINOR=$(echo $_PV | cut -d. -f1,2)",
+                "_EXTRA_TAGS=\"$_PV_MAJOR $_PV_MAJOR_MINOR\""
+            ],
+            "step1" : {
+                "shortname" : "Build 'base' container",
+                "BBTARGETS" : "container-base",
+                "CONTAINER_IMAGE_MAP" : {"container-base": "base"},
+                "CONTAINER_VERSION_RECIPE" : "base-files"
+            },
+            "step2" : {
+                "shortname" : "Build 'curl' container",
+                "BBTARGETS" : "app-container-curl",
+                "CONTAINER_IMAGE_MAP" : {"app-container-curl": "curl"},
+                "CONTAINER_VERSION_RECIPE" : "curl"
+            },
+            "step3" : {
+                "shortname" : "Build 'python' container",
+                "BBTARGETS" : "app-container-python",
+                "CONTAINER_IMAGE_MAP" : {"app-container-python": "python"},
+                "CONTAINER_VERSION_RECIPE" : "python3"
+            },
+            "step4" : {
+                "shortname" : "Build 'mosquitto' container",
+                "BBTARGETS" : "app-container-mosquitto",
+                "CONTAINER_IMAGE_MAP" : {"app-container-mosquitto": "mosquitto"},
+                "CONTAINER_VERSION_RECIPE" : "mosquitto"
+            },
+            "step5" : {
+                "shortname" : "Build 'valkey' container",
+                "BBTARGETS" : "app-container-valkey",
+                "CONTAINER_IMAGE_MAP" : {"app-container-valkey": "valkey"},
+                "CONTAINER_VERSION_RECIPE" : "valkey"
+            },
+            "step6" : {
+                "shortname" : "Build 'nginx' container",
+                "BBTARGETS" : "app-container-nginx",
+                "CONTAINER_IMAGE_MAP" : {"app-container-nginx": "nginx"},
+                "CONTAINER_VERSION_RECIPE" : "nginx"
+            }
         }
     },
     "repo-defaults" : {
-- 
2.43.0

[yocto-autobuilder2][PATCH v3 6/6] scripts/run-push-containers: remove all images before push

[tim.orling]

From: Tim Orling <tim.orling@konsulko.com>

Workaround for errors like:
Error: reading blob sha256:<hash>: file integrity checksum failed for "<file>"
Error: reading blob sha256:<hash>: EOF

The first time the blob (e.g. bin/bash.bash) is copied, it succeeds. All subsequent
tries fail. Might possibly be an issue with sstate or a recent change to vcontainer-common
to allow for multiarch containers.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 scripts/run-push-containers | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/run-push-containers b/scripts/run-push-containers
index cdef497..4f1ac53 100755
--- a/scripts/run-push-containers
+++ b/scripts/run-push-containers
@@ -94,6 +94,8 @@ script = [
     # password from stdin (see login case in vcontainer-common.sh).
     "trap '%s-$(arch) memres stop 2>/dev/null || true' EXIT" % runtime,
     "%s-$(arch) --config %s memres restart </dev/null" % (runtime, auth_config),
+    # workaround for 'Error: reading blob sha256:<hash>: file integrity checksum failed for "<file>"'
+    "%s-$(arch) image rm --all 2>/dev/null" % (runtime),
 ]
 tag_cmds = utils.getconfiglist("CONTAINER_TAG_CMDS", ourconfig, args.target, args.stepnum)
 version_recipe = utils.getconfigvar("CONTAINER_VERSION_RECIPE", ourconfig, args.target, args.stepnum)
-- 
2.43.0