[meta-virtualization][PATCH 0/7] Container improvements

[meta-virtualization][PATCH 0/7] Container improvements

[Tim Orling]

This series:
  * Adds a class to create/run containers with a non-root user
  * Adds new containers:
    - app-container-python
    - app-containter-mosquitto
    - app-container-valkey
    - app-container-nginx
  * Modifies app-container-curl to be more like the upstream
    experience (and more like the above containers)
  * Allows meta-webserver/recipes-http to be parsed for
    vcontainer distro so we can build multiarch containers
    for app-container-nginx, etc.

Each of these containers was built in a MACHINE=qemuarm64
environment as well as mc:container-amd64+mc:container-arm64
multiarch environment.

The resulting containers were tested with simple command line
usage compared to Docker provided equivalents to ensure the
same expected behavior.

Tim Orling (7):
  classes: add container-nonroot-user.bbclass
  recipes-containers/images: add app-container-python
  recipes-containers/images: add app-container-mosquitto
  recipes-containers/images: add app-container-valkey
  recipes-containers/images: add app-container-nginx
  vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd
  app-container-curl: use multilayer mode; container-nonroot-user

 classes/container-nonroot-user.bbclass        | 68 ++++++++++++++++
 conf/distro/include/vcontainer-bbmask.inc     |  2 +-
 conf/layer.conf                               |  1 +
 .../images/app-container-curl.bb              | 29 ++++++-
 .../images/app-container-mosquitto.bb         | 46 +++++++++++
 .../images/app-container-nginx.bb             | 77 +++++++++++++++++++
 .../images/app-container-python.bb            | 57 ++++++++++++++
 .../images/app-container-valkey.bb            | 61 +++++++++++++++
 8 files changed, 336 insertions(+), 5 deletions(-)
 create mode 100644 classes/container-nonroot-user.bbclass
 rename {recipes-demo => recipes-containers}/images/app-container-curl.bb (58%)
 create mode 100644 recipes-containers/images/app-container-mosquitto.bb
 create mode 100644 recipes-containers/images/app-container-nginx.bb
 create mode 100644 recipes-containers/images/app-container-python.bb
 create mode 100644 recipes-containers/images/app-container-valkey.bb

-- 
2.54.0

[meta-virtualization][PATCH 1/7] classes: add container-nonroot-user.bbclass

[Tim Orling]

For secure and production environments, we want to run containers as a
non-root user. Some applications, such as Python, require a $HOME
directory with proper permissions. Because OCI_LAYERS :directories:
copies with 'cp -a --no-preserve=ownership', we need a fixup function
to create the proper permissions and ownership in a new raw layer.

The behavior here is inspired by dhi.io/python:3 (Docker Hardened Image)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 classes/container-nonroot-user.bbclass | 68 ++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
 create mode 100644 classes/container-nonroot-user.bbclass

diff --git a/classes/container-nonroot-user.bbclass b/classes/container-nonroot-user.bbclass
new file mode 100644
index 00000000..5139ce8b
--- /dev/null
+++ b/classes/container-nonroot-user.bbclass
@@ -0,0 +1,68 @@
+# For secure and production environments, we want to run containers as a
+# non-root user. Some applications, such as Python, require a $HOME
+# directory with proper permissions. Because OCI_LAYERS :directories:
+# copies with 'cp -a --no-preserve=ownership', we need a fixup function
+# to create the proper permissions and ownership in a new raw layer.
+
+# The behavior here is inspired by dhi.io/python:3 (Docker Hardened Image)
+
+inherit extrausers
+
+NONROOT_USER ?= "nonroot"
+NONROOT_UID ?= "65532"
+NONROOT_GID ?= "65532"
+
+# ---------------------------------------------------------------------------
+# Create the unprivileged "nonroot" user (uid 65532, group 65532)
+# ---------------------------------------------------------------------------
+EXTRA_USERS_PARAMS = "\
+    groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
+    useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} -d /home/${NONROOT_USER}  ${NONROOT_USER}; \
+"
+
+# Allow a container to choose to run as 'root'
+OCI_IMAGE_RUNTIME_UID ?= "${NONROOT_UID}"
+OCI_IMAGE_ENV_VARS = "HOME=/home/${NONROOT_USER}"
+
+# Make sure we can write to e.g. /home/nonroot/.python_history
+# using :directories: in OCI_LAYERS does not preserve permissions
+fakeroot fix_oci_home_perms() {
+    cd ${IMGDEPLOYDIR}
+    image_name="${IMAGE_NAME}${IMAGE_NAME_SUFFIX}-oci"
+    layer_tar="${WORKDIR}/oci-home-fix-layer.tar"
+
+    rm -f "$layer_tar"
+
+    python3 - "$layer_tar" <<'PYEOF'
+import sys, tarfile, time
+
+layer_tar = sys.argv[1]
+mtime = int(time.time())
+
+# (path, mode, uid, gid)
+entries = [
+    ("home",         0o755, 0,     0),
+    ("home/${NONROOT_USER}", 0o700, ${NONROOT_UID}, ${NONROOT_GID}),
+]
+
+with tarfile.open(layer_tar, "w") as tar:
+    for name, mode, uid, gid in entries:
+        info = tarfile.TarInfo(name=name)
+        info.type  = tarfile.DIRTYPE
+        info.mode  = mode
+        info.uid   = uid
+        info.gid   = gid
+        info.uname = ""   # numeric-only; let umoci canonicalize
+        info.gname = ""
+        info.mtime = mtime
+        tar.addfile(info)
+PYEOF
+
+    umoci raw add-layer --image "$image_name:${OCI_IMAGE_TAG}" "$layer_tar"
+    rm -f "$layer_tar"
+
+    rm -f "$image_name.tar" "$image_name-dir.tar"
+    ( cd "$image_name" && tar -cf "../$image_name.tar" "." )
+    tar -cf "$image_name-dir.tar" "$image_name"
+}
+do_image_oci[postfuncs] += "fix_oci_home_perms"
-- 
2.54.0

[meta-virtualization][PATCH 2/7] recipes-containers/images: add app-container-python

[Tim Orling]

Add OCI container image recipe for Python to use as a base for
other Python app containers. The image uses multi-layer mode with
separate base, terminal and python layers.

Add ncurses-terminfo-base to a "terminal" layer to avoid warnings in the
REPL:
  "Cannot read termcap database;
  using dumb terminal settings."

Add coreutils to "python" layer to provide /usr/bin/env needed by
python3-idle in python3-modules.

Inherit container-nonroot-user and run a `nonroot` user by default.
Set PACKAGECONFIG:pn-app-container-python = "dev" in local.conf or
distro/image config to run as 'root' and include 'pip'.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-python.bb            | 57 +++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 recipes-containers/images/app-container-python.bb

diff --git a/recipes-containers/images/app-container-python.bb b/recipes-containers/images/app-container-python.bb
new file mode 100644
index 00000000..a93f1b0f
--- /dev/null
+++ b/recipes-containers/images/app-container-python.bb
@@ -0,0 +1,57 @@
+SUMMARY = "Base python3 container image"
+DESCRIPTION = "OCI container image running Python with non-root user. \
+\
+In "dev" mode, can optionally run as 'root' and add 'pip' to allow \
+developers to simply run 'pip install' on top of this container (Not \
+advised for production/hardened use)."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - adds python3-pip to the python layer (enables `pip install` at runtime)
+#   - runs the container as root (UID 0) so pip can write to site-packages
+# Enable with: PACKAGECONFIG:pn-app-container-python = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    terminal:packages:ncurses-terminfo-base \
+    python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG', 'dev', '+python3-pip', '', d)} \
+"
+
+# In 'dev' mode, override the nonroot UID inherited from container-nonroot-user
+# so the container runs as root (required for `pip install`).
+OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
+
+# Use CMD so `docker run image /bin/sh` works as expected
+OCI_IMAGE_CMD = "python3"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# IMAGE_INSTALL triggers package builds via do_rootfs recrdeptask.
+# Even for multi-layer mode, list packages here to ensure they're built.
+# The PM will install them directly to layers from DEPLOY_DIR_IPK.
+# Note: IMAGE_ROOTFS is still created but ignored for packages layers.
+IMAGE_INSTALL = "base-files base-passwd netbase"
+IMAGE_INSTALL += "ncurses-terminfo-base"
+IMAGE_INSTALL += "python3 coreutils"
+IMAGE_INSTALL += "${@bb.utils.contains('PACKAGECONFIG', 'dev', 'python3-pip', '', d)}"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+# Note: No ROOTFS_POSTPROCESS_COMMAND needed - IMAGE_ROOTFS is empty
+# and PM handles installation directly to OCI layers
-- 
2.54.0

Re: [meta-virtualization][PATCH 2/7] recipes-containers/images: add app-container-python

[Paul Barker]

[-- Attachment #1: Type: text/plain, Size: 4451 bytes --]

On Fri, 2026-05-29 at 18:31 -0700, Tim Orling via lists.yoctoproject.org
wrote:
> Add OCI container image recipe for Python to use as a base for
> other Python app containers. The image uses multi-layer mode with
> separate base, terminal and python layers.
> 
> Add ncurses-terminfo-base to a "terminal" layer to avoid warnings in the
> REPL:
>   "Cannot read termcap database;
>   using dumb terminal settings."
> 
> Add coreutils to "python" layer to provide /usr/bin/env needed by
> python3-idle in python3-modules.
> 
> Inherit container-nonroot-user and run a `nonroot` user by default.
> Set PACKAGECONFIG:pn-app-container-python = "dev" in local.conf or
> distro/image config to run as 'root' and include 'pip'.
> 
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
>  .../images/app-container-python.bb            | 57 +++++++++++++++++++
>  1 file changed, 57 insertions(+)
>  create mode 100644 recipes-containers/images/app-container-python.bb
> 
> diff --git a/recipes-containers/images/app-container-python.bb b/recipes-containers/images/app-container-python.bb
> new file mode 100644
> index 00000000..a93f1b0f
> --- /dev/null
> +++ b/recipes-containers/images/app-container-python.bb
> @@ -0,0 +1,57 @@
> +SUMMARY = "Base python3 container image"
> +DESCRIPTION = "OCI container image running Python with non-root user. \
> +\
> +In "dev" mode, can optionally run as 'root' and add 'pip' to allow \
> +developers to simply run 'pip install' on top of this container (Not \
> +advised for production/hardened use)."
> +LICENSE = "MIT"
> +LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
> +
> +# Multi-layer mode: create explicit layers instead of single rootfs layer
> +OCI_LAYER_MODE = "multi"
> +
> +# Optional 'dev' mode:
> +#   - adds python3-pip to the python layer (enables `pip install` at runtime)
> +#   - runs the container as root (UID 0) so pip can write to site-packages
> +# Enable with: PACKAGECONFIG:pn-app-container-python = "dev"
> +PACKAGECONFIG ??= ""
> +PACKAGECONFIG[dev] = ""
> +
> +# Define layers: each layer contains specific packages
> +# Format: "name:type:content" where content uses + as delimiter for multiple items
> +OCI_LAYERS = "\
> +    base:packages:base-files+base-passwd+netbase \

Hi Tim,

I wonder if we should define the base layer contents in image-oci to
ensure that it is consistent across recipes.

E.g. in image-oci.bbclass:

    OCI_BASE_LAYER = "base:packages:base-files+base-passwd+netbase"

Then in the recipe:

    OCI_LAYERS = "\
        ${OCI_BASE_LAYER} \
        ...
    "

That gives consistency without forcing all OCI images to use the base
layer definition if it isn't relevant to them.

> +    terminal:packages:ncurses-terminfo-base \
> +    python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG', 'dev', '+python3-pip', '', d)} \
> +"
> +
> +# In 'dev' mode, override the nonroot UID inherited from container-nonroot-user
> +# so the container runs as root (required for `pip install`).
> +OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
> +
> +# Use CMD so `docker run image /bin/sh` works as expected
> +OCI_IMAGE_CMD = "python3"
> +
> +IMAGE_FSTYPES = "container oci"
> +inherit image
> +inherit image-oci
> +inherit container-nonroot-user
> +
> +IMAGE_FEATURES = ""
> +IMAGE_LINGUAS = ""
> +NO_RECOMMENDATIONS = "1"
> +
> +# IMAGE_INSTALL triggers package builds via do_rootfs recrdeptask.
> +# Even for multi-layer mode, list packages here to ensure they're built.
> +# The PM will install them directly to layers from DEPLOY_DIR_IPK.
> +# Note: IMAGE_ROOTFS is still created but ignored for packages layers.
> +IMAGE_INSTALL = "base-files base-passwd netbase"

Maybe we also need an OCI_BASE_PACKAGES to go with OCI_BASE_LAYER.
Pretty much every image is going to need these three packages installed.

> +IMAGE_INSTALL += "ncurses-terminfo-base"
> +IMAGE_INSTALL += "python3 coreutils"
> +IMAGE_INSTALL += "${@bb.utils.contains('PACKAGECONFIG', 'dev', 'python3-pip', '', d)}"
> +
> +# Allow build with or without a specific kernel
> +IMAGE_CONTAINER_NO_DUMMY = "1"
> +
> +# Note: No ROOTFS_POSTPROCESS_COMMAND needed - IMAGE_ROOTFS is empty
> +# and PM handles installation directly to OCI layers

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]

Re: [meta-virtualization][PATCH 2/7] recipes-containers/images: add app-container-python

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 5778 bytes --]

On Tue, Jun 2, 2026 at 6:02 AM Paul Barker via lists.yoctoproject.org <paul=
pbarker.dev@lists.yoctoproject.org> wrote:

> On Fri, 2026-05-29 at 18:31 -0700, Tim Orling via lists.yoctoproject.org
> wrote:
> > Add OCI container image recipe for Python to use as a base for
> > other Python app containers. The image uses multi-layer mode with
> > separate base, terminal and python layers.
> >
> > Add ncurses-terminfo-base to a "terminal" layer to avoid warnings in the
> > REPL:
> >   "Cannot read termcap database;
> >   using dumb terminal settings."
> >
> > Add coreutils to "python" layer to provide /usr/bin/env needed by
> > python3-idle in python3-modules.
> >
> > Inherit container-nonroot-user and run a `nonroot` user by default.
> > Set PACKAGECONFIG:pn-app-container-python = "dev" in local.conf or
> > distro/image config to run as 'root' and include 'pip'.
> >
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> > ---
> >  .../images/app-container-python.bb            | 57 +++++++++++++++++++
> >  1 file changed, 57 insertions(+)
> >  create mode 100644 recipes-containers/images/app-container-python.bb
> >
> > diff --git a/recipes-containers/images/app-container-python.bb
> b/recipes-containers/images/app-container-python.bb
> > new file mode 100644
> > index 00000000..a93f1b0f
> > --- /dev/null
> > +++ b/recipes-containers/images/app-container-python.bb
> > @@ -0,0 +1,57 @@
> > +SUMMARY = "Base python3 container image"
> > +DESCRIPTION = "OCI container image running Python with non-root user. \
> > +\
> > +In "dev" mode, can optionally run as 'root' and add 'pip' to allow \
> > +developers to simply run 'pip install' on top of this container (Not \
> > +advised for production/hardened use)."
> > +LICENSE = "MIT"
> > +LIC_FILES_CHKSUM =
> "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
> > +
> > +# Multi-layer mode: create explicit layers instead of single rootfs
> layer
> > +OCI_LAYER_MODE = "multi"
> > +
> > +# Optional 'dev' mode:
> > +#   - adds python3-pip to the python layer (enables `pip install` at
> runtime)
> > +#   - runs the container as root (UID 0) so pip can write to
> site-packages
> > +# Enable with: PACKAGECONFIG:pn-app-container-python = "dev"
> > +PACKAGECONFIG ??= ""
> > +PACKAGECONFIG[dev] = ""
> > +
> > +# Define layers: each layer contains specific packages
> > +# Format: "name:type:content" where content uses + as delimiter for
> multiple items
> > +OCI_LAYERS = "\
> > +    base:packages:base-files+base-passwd+netbase \
>
> Hi Tim,
>
> I wonder if we should define the base layer contents in image-oci to
> ensure that it is consistent across recipes.
>
> E.g. in image-oci.bbclass:
>
>     OCI_BASE_LAYER = "base:packages:base-files+base-passwd+netbase"
>
> Then in the recipe:
>
>     OCI_LAYERS = "\
>         ${OCI_BASE_LAYER} \
>         ...
>     "
>
> That gives consistency without forcing all OCI images to use the base
> layer definition if it isn't relevant to them.
>

We can't (and shouldn't) enforce using packages at the base like that,
since specifying layers as Tim is using is simply one way of doing it.

It could just as easily be another image, or one of the other techniques.


>
> > +    terminal:packages:ncurses-terminfo-base \
> > +
> python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG',
> 'dev', '+python3-pip', '', d)} \
> > +"
> > +
> > +# In 'dev' mode, override the nonroot UID inherited from
> container-nonroot-user
> > +# so the container runs as root (required for `pip install`).
> > +OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev',
> '0', '${NONROOT_UID}', d)}"
> > +
> > +# Use CMD so `docker run image /bin/sh` works as expected
> > +OCI_IMAGE_CMD = "python3"
> > +
> > +IMAGE_FSTYPES = "container oci"
> > +inherit image
> > +inherit image-oci
> > +inherit container-nonroot-user
> > +
> > +IMAGE_FEATURES = ""
> > +IMAGE_LINGUAS = ""
> > +NO_RECOMMENDATIONS = "1"
> > +
> > +# IMAGE_INSTALL triggers package builds via do_rootfs recrdeptask.
> > +# Even for multi-layer mode, list packages here to ensure they're built.
> > +# The PM will install them directly to layers from DEPLOY_DIR_IPK.
> > +# Note: IMAGE_ROOTFS is still created but ignored for packages layers.
> > +IMAGE_INSTALL = "base-files base-passwd netbase"
>
> Maybe we also need an OCI_BASE_PACKAGES to go with OCI_BASE_LAYER.
> Pretty much every image is going to need these three packages installed.
>

Again, we shouldn't be overly prescriptive here.

Factoring things out at this point is premature.

Bruce



>
> > +IMAGE_INSTALL += "ncurses-terminfo-base"
> > +IMAGE_INSTALL += "python3 coreutils"
> > +IMAGE_INSTALL += "${@bb.utils.contains('PACKAGECONFIG', 'dev',
> 'python3-pip', '', d)}"
> > +
> > +# Allow build with or without a specific kernel
> > +IMAGE_CONTAINER_NO_DUMMY = "1"
> > +
> > +# Note: No ROOTFS_POSTPROCESS_COMMAND needed - IMAGE_ROOTFS is empty
> > +# and PM handles installation directly to OCI layers
>
> Best regards,
>
> --
> Paul Barker
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#9842):
> https://lists.yoctoproject.org/g/meta-virtualization/message/9842
> Mute This Topic: https://lists.yoctoproject.org/mt/119557386/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [
> bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 9182 bytes --]

[meta-virtualization][PATCH 3/7] recipes-containers/images: add app-container-mosquitto

[Tim Orling]

Add OCI container image recipe for the Eclipse Mosquitto MQTT broker.
The image uses multi-layer mode with separate base and mosquitto layers,
exposes standard MQTT (1883) and WebSocket (9001) ports, and launches
mosquitto with its default config file as the entrypoint.

Inherit container-nonroot-user to run as 'nonroot' with UID 65532.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-mosquitto.bb         | 46 +++++++++++++++++++
 1 file changed, 46 insertions(+)
 create mode 100644 recipes-containers/images/app-container-mosquitto.bb

diff --git a/recipes-containers/images/app-container-mosquitto.bb b/recipes-containers/images/app-container-mosquitto.bb
new file mode 100644
index 00000000..37e34a47
--- /dev/null
+++ b/recipes-containers/images/app-container-mosquitto.bb
@@ -0,0 +1,46 @@
+SUMMARY = "Mosquitto MQTT broker container image"
+DESCRIPTION = "OCI container running the Eclipse Mosquitto MQTT broker \
+with standard MQTT (1883) and WebSocket (9001) listeners enabled."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    mosquitto:packages:mosquitto \
+"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+IMAGE_INSTALL = " \
+    base-files \
+    base-passwd \
+    netbase \
+    mosquitto \
+"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+# Workaround /var/volatile for now
+ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
+rootfs_fixup_var_volatile () {
+    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
+}
+
+OCI_IMAGE_ENTRYPOINT = "${sbindir}/mosquitto"
+OCI_IMAGE_ENTRYPOINT_ARGS = "-c '${sysconfdir}/mosquitto/mosquitto.conf'"
+OCI_IMAGE_PORTS = "1883/tcp 9001/tcp"
+OCI_IMAGE_TAG = "latest"
-- 
2.54.0

[meta-virtualization][PATCH 4/7] recipes-containers/images: add app-container-valkey

[Tim Orling]

Add OCI container image recipe for the Valkey in-memory key-value
datastore. The image uses multi-layer mode with separate base and
valkey layers, exposes the standard Valkey port (6379), and launches
valkey-server with its default config file as the entrypoint.

The stock valkey.conf shipped by meta-oe is tuned for a host install
(daemonize yes, syslog-enabled yes, bind 127.0.0.1). Override those at
launch so the server stays in the foreground as PID 1, logs to stdout,
and is reachable from outside the container.

Inherit container-nonroot-user to run as 'nonroot' with UID 65532 by
default. Can optionally set PACKAGECONFIG:pn-app-container-valkey = "dev"
in local.conf or a distro/image config to run as root.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-valkey.bb            | 61 +++++++++++++++++++
 1 file changed, 61 insertions(+)
 create mode 100644 recipes-containers/images/app-container-valkey.bb

diff --git a/recipes-containers/images/app-container-valkey.bb b/recipes-containers/images/app-container-valkey.bb
new file mode 100644
index 00000000..b3da2efd
--- /dev/null
+++ b/recipes-containers/images/app-container-valkey.bb
@@ -0,0 +1,61 @@
+SUMMARY = "Valkey key-value store container image"
+DESCRIPTION = "OCI container running the Valkey in-memory key-value \
+datastore, a flexible distributed datastore that supports both caching \
+and beyond caching workloads."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-valkey = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    valkey:packages:valkey \
+"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+IMAGE_INSTALL = " \
+    base-files \
+    base-passwd \
+    netbase \
+    valkey \
+"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+# Workaround /var/volatile for now
+ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
+rootfs_fixup_var_volatile () {
+    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
+}
+
+OCI_IMAGE_ENTRYPOINT = "${bindir}/valkey-server"
+# The stock valkey.conf shipped by meta-oe is tuned for a host install
+# (daemonize yes, syslog-enabled yes, bind 127.0.0.1). Override those at
+# launch so the server stays in the foreground as PID 1, logs to stdout,
+# and is reachable from outside the container.
+OCI_IMAGE_ENTRYPOINT_ARGS = "'${sysconfdir}/valkey/valkey.conf' \
+    --daemonize no \
+    --syslog-enabled no \
+    --bind '0.0.0.0 -::*' \
+    --protected-mode no"
+OCI_IMAGE_PORTS = "6379/tcp"
+OCI_IMAGE_TAG = "latest"
-- 
2.54.0

[meta-virtualization][PATCH 5/7] recipes-containers/images: add app-container-nginx

[Tim Orling]

Add OCI container image recipe for the NGINX web server. The image
uses multi-layer mode with separate base, nginx packages, nginx
runtime directories, and nginx log file layers. Exposes the standard
HTTP port (80) and launches nginx with 'daemon off' so it stays in
the foreground as PID 1 and logs to stderr.

Add ROOTFS_POSTPROCESS fixups to create the runtime directories
nginx expects: /var/volatile/{tmp,log}, /var/log/nginx (resolved
explicitly to guarantee inclusion in the container layer regardless
of /var/log symlink ordering), and /run/nginx for nginx's compiled-in
temp paths (client_body_temp, proxy_temp, etc.) which are not created
by any package. Also create empty /var/log/nginx/{access,error}.log
to avoid do_image_oci warnings.

Inherit container-nonroot-user with NONROOT_USER = "nginx" to run with
UID 65532 by default. Set PACKAGECONFIG:pn-app-container-nginx = "dev"
in local.conf or distro/image config to run as 'root'.

Add SKIP_RECIPE and comment to layer.conf since nginx requires
meta-webserver.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 conf/layer.conf                               |  1 +
 .../images/app-container-nginx.bb             | 77 +++++++++++++++++++
 2 files changed, 78 insertions(+)
 create mode 100644 recipes-containers/images/app-container-nginx.bb

diff --git a/conf/layer.conf b/conf/layer.conf
index 2a4a4c91..6ea8ccf8 100644
--- a/conf/layer.conf
+++ b/conf/layer.conf
@@ -33,6 +33,7 @@ LAYERDEPENDS_virtualization-layer = " \
 # webserver:
 # - naigos requires apache2
 # - cockpit-machines requires cockpit
+# - app-container-nginx requires nginx
 LAYERRECOMMENDS_virtualization-layer = " \
     webserver \
     selinux \
diff --git a/recipes-containers/images/app-container-nginx.bb b/recipes-containers/images/app-container-nginx.bb
new file mode 100644
index 00000000..09844376
--- /dev/null
+++ b/recipes-containers/images/app-container-nginx.bb
@@ -0,0 +1,77 @@
+SUMMARY = "Base NGINX container image for development"
+DESCRIPTION = "OCI container with NGINX web server."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-nginx = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+NONROOT_USER = "nginx"
+
+OCI_IMAGE_APP_RECIPE = "nginx"
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    nginx:packages:nginx \
+    nginx-dirs:directories:${localstatedir}/log/nginx+/run/nginx+${localstatedir}/volatile/tmp+${localstatedir}/volatile/log \
+    nginx-files:files:${localstatedir}/log/nginx/access.log+${localstatedir}/log/nginx/error.log \
+"
+# Use CMD so `docker run image /bin/sh` works as expected
+OCI_IMAGE_CMD = ""
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+IMAGE_INSTALL = " \
+       base-files \
+       base-passwd \
+       netbase \
+       nginx \
+"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+# Workaround /var/volatile for now
+ROOTFS_POSTPROCESS_COMMAND:append = " rootfs_fixup_var_volatile ; "
+rootfs_fixup_var_volatile () {
+    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log/nginx
+
+    # Fix do_image_oci warnings
+    # OCI: File not found in IMAGE_ROOTFS: /var/log/nginx/access.log
+    touch ${IMAGE_ROOTFS}/${localstatedir}/volatile/log/nginx/access.log
+    touch ${IMAGE_ROOTFS}/${localstatedir}/volatile/log/nginx/error.log
+
+    # nginx opens the compiled-in error_log path before reading -c config.
+    # /var/log is typically a symlink to /var/volatile/log in a Yocto rootfs,
+    # so create the target path explicitly to guarantee the directory lands in
+    # the container layer regardless of symlink resolution order.
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log/nginx
+
+    # nginx's compiled-in temp paths (client_body_temp, proxy_temp, etc.) all
+    # live under /run/nginx, which is not created by any package.
+    install -m 755 -d ${IMAGE_ROOTFS}/run/nginx
+}
+
+OCI_IMAGE_ENTRYPOINT = "/usr/sbin/nginx"
+OCI_IMAGE_ENTRYPOINT_ARGS = "-g 'daemon off; error_log stderr notice;'"
+OCI_IMAGE_PORTS = "80/tcp"
+OCI_IMAGE_TAG = "latest"
+
+SKIP_RECIPE[app-container-nginx] ?= "${@bb.utils.contains('BBFILE_COLLECTIONS', 'webserver', '', 'Depends on meta-webserver which is not included', d)}"
-- 
2.54.0

[meta-virtualization][PATCH 6/7] vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd

[Tim Orling]

Allow us to build nginx, apache2, etc. multiarch containers.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 conf/distro/include/vcontainer-bbmask.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conf/distro/include/vcontainer-bbmask.inc b/conf/distro/include/vcontainer-bbmask.inc
index ac74464f..077255cb 100644
--- a/conf/distro/include/vcontainer-bbmask.inc
+++ b/conf/distro/include/vcontainer-bbmask.inc
@@ -93,6 +93,7 @@ BBMASK += "meta-networking/recipes-(?!filter|support)"
 BBMASK += "meta-openstack/recipes-dbs/postgresql/"
 BBMASK += "meta-oe/dynamic-layers/networking-layer/recipes-core/"
 BBMASK += "meta-openstack/recipes-extended/libvirt/"
+BBMASK += "meta-webserver/recipes-(?!httpd)"
 
 # ---------------------------------------------------------------------------
 # Entire layers with 0 recipes in the container image dependency graph.
@@ -101,7 +102,6 @@ BBMASK += "meta-openstack/recipes-extended/libvirt/"
 # ---------------------------------------------------------------------------
 BBMASK += "meta-filesystems/"
 BBMASK += "meta-python/"
-BBMASK += "meta-webserver/"
 # Warning suppression for these fully-masked layers is in meta-virt-host.conf
 # (BBFILE_PATTERN_IGNORE_EMPTY) because BitBake checks the base datastore,
 # not per-multiconfig datastores.
-- 
2.54.0

[meta-virtualization][PATCH 7/7] app-container-curl: use multilayer mode; container-nonroot-user

[Tim Orling]

* Move to recipes-containers/images to show maintenance intent
* Switch to multilayer mode to more like the other "library"/"official"
  container recipes.
* Change OCI_IMAGE_TAG to "latest" for similar reasons.
* Change OCI_IMAGE_ENTRYPOINT_ARGS to "--help" to be more like upstream
  containers.
* Install ca-certificates to enable handling https:// sites
* Inherit container-nonroot-user to run as 'nonroot' with UID 65532 by
  default. Set PACKAGECONFIG:pn-app-container-curl = "dev" in local.conf
  or distro/image config to run as `root` and include a CONTAINER_SHELL.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-curl.bb              | 29 ++++++++++++++++---
 1 file changed, 25 insertions(+), 4 deletions(-)
 rename {recipes-demo => recipes-containers}/images/app-container-curl.bb (58%)

diff --git a/recipes-demo/images/app-container-curl.bb b/recipes-containers/images/app-container-curl.bb
similarity index 58%
rename from recipes-demo/images/app-container-curl.bb
rename to recipes-containers/images/app-container-curl.bb
index ddeb3022..34204fb9 100644
--- a/recipes-demo/images/app-container-curl.bb
+++ b/recipes-containers/images/app-container-curl.bb
@@ -2,9 +2,31 @@ SUMMARY = "Curl Application container image"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
 
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-curl = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
+    curl:packages:curl+ca-certificates \
+"
+
+# In 'dev' mode, override the nonroot UID inherited from container-nonroot-user
+# so the container runs as root.
+OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
+
 IMAGE_FSTYPES = "container oci"
 inherit image
 inherit image-oci
+inherit container-nonroot-user
 
 IMAGE_FEATURES = ""
 IMAGE_LINGUAS = ""
@@ -39,8 +61,7 @@ rootfs_fixup_var_volatile () {
 }
 
 OCI_IMAGE_ENTRYPOINT = "curl"
-OCI_IMAGE_TAG = "easy"
-OCI_IMAGE_ENTRYPOINT_ARGS = "http://localhost:80"
-CONTAINER_SHELL = "busybox"
+OCI_IMAGE_TAG = "latest"
+OCI_IMAGE_ENTRYPOINT_ARGS = "--help"
 
-IMAGE_INSTALL:append = " curl"
+IMAGE_INSTALL:append = " curl ca-certificates"
-- 
2.54.0

Re: [meta-virtualization][PATCH 0/7] Container improvements

[Bruce Ashfield]

Just ack'ing the series.

I'm working through issues with the recipe bumps right now, but
will get to this shortly and have a closer look.

Bruce

In message: [meta-virtualization][PATCH 0/7] Container improvements
on 29/05/2026 Tim Orling via lists.yoctoproject.org wrote:

> This series:
>   * Adds a class to create/run containers with a non-root user
>   * Adds new containers:
>     - app-container-python
>     - app-containter-mosquitto
>     - app-container-valkey
>     - app-container-nginx
>   * Modifies app-container-curl to be more like the upstream
>     experience (and more like the above containers)
>   * Allows meta-webserver/recipes-http to be parsed for
>     vcontainer distro so we can build multiarch containers
>     for app-container-nginx, etc.
> 
> Each of these containers was built in a MACHINE=qemuarm64
> environment as well as mc:container-amd64+mc:container-arm64
> multiarch environment.
> 
> The resulting containers were tested with simple command line
> usage compared to Docker provided equivalents to ensure the
> same expected behavior.
> 
> Tim Orling (7):
>   classes: add container-nonroot-user.bbclass
>   recipes-containers/images: add app-container-python
>   recipes-containers/images: add app-container-mosquitto
>   recipes-containers/images: add app-container-valkey
>   recipes-containers/images: add app-container-nginx
>   vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd
>   app-container-curl: use multilayer mode; container-nonroot-user
> 
>  classes/container-nonroot-user.bbclass        | 68 ++++++++++++++++
>  conf/distro/include/vcontainer-bbmask.inc     |  2 +-
>  conf/layer.conf                               |  1 +
>  .../images/app-container-curl.bb              | 29 ++++++-
>  .../images/app-container-mosquitto.bb         | 46 +++++++++++
>  .../images/app-container-nginx.bb             | 77 +++++++++++++++++++
>  .../images/app-container-python.bb            | 57 ++++++++++++++
>  .../images/app-container-valkey.bb            | 61 +++++++++++++++
>  8 files changed, 336 insertions(+), 5 deletions(-)
>  create mode 100644 classes/container-nonroot-user.bbclass
>  rename {recipes-demo => recipes-containers}/images/app-container-curl.bb (58%)
>  create mode 100644 recipes-containers/images/app-container-mosquitto.bb
>  create mode 100644 recipes-containers/images/app-container-nginx.bb
>  create mode 100644 recipes-containers/images/app-container-python.bb
>  create mode 100644 recipes-containers/images/app-container-valkey.bb
> 
> -- 
> 2.54.0
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#9826): https://lists.yoctoproject.org/g/meta-virtualization/message/9826
> Mute This Topic: https://lists.yoctoproject.org/mt/119557384/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 

Re: [meta-virtualization][PATCH 1/7] classes: add container-nonroot-user.bbclass

[Bruce Ashfield]

Hi Tim,

A few things on this one — none of them blocking the intent, but worth a
small follow-up either as a fixup in this series or a v2.

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> For secure and production environments, we want to run containers as a
> non-root user. Some applications, such as Python, require a $HOME
> directory with proper permissions. Because OCI_LAYERS :directories:
> copies with 'cp -a --no-preserve=ownership', we need a fixup function
> to create the proper permissions and ownership in a new raw layer.
>
> The behavior here is inspired by dhi.io/python:3 (Docker Hardened Image)
>
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>

The non-root + DHI-style image is a real gap in what we ship today;
glad to see it filled.

> +inherit extrausers
> +
> +NONROOT_USER ?= "nonroot"
> +NONROOT_UID ?= "65532"
> +NONROOT_GID ?= "65532"
> +
> +# ---------------------------------------------------------------------------
> +# Create the unprivileged "nonroot" user (uid 65532, group 65532)
> +# ---------------------------------------------------------------------------
> +EXTRA_USERS_PARAMS = "\
> +    groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
> +    useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} -d /home/${NONROOT_USER}  ${NONROOT_USER}; \
> +"

This hard-assigns EXTRA_USERS_PARAMS rather than appending. Any image
that inherits container-nonroot-user AND wants its own extrausers (e.g.
a recipe that needs an additional service account) loses whatever else
was set, and the order of inheritance silently determines who wins.

Suggest:

  EXTRA_USERS_PARAMS += " \
      groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
      useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} \
              -d /home/${NONROOT_USER} ${NONROOT_USER}; \
  "

That way the class composes cleanly with anyone else who sets
EXTRA_USERS_PARAMS earlier.

> +# Make sure we can write to e.g. /home/nonroot/.python_history
> +# using :directories: in OCI_LAYERS does not preserve permissions
> +fakeroot fix_oci_home_perms() {
> +    cd ${IMGDEPLOYDIR}
> +    image_name="${IMAGE_NAME}${IMAGE_NAME_SUFFIX}-oci"
> +    layer_tar="${WORKDIR}/oci-home-fix-layer.tar"
> +
> +    rm -f "$layer_tar"
> +
> +    python3 - "$layer_tar" <<'PYEOF'
> +import sys, tarfile, time
> +
> +layer_tar = sys.argv[1]
> +mtime = int(time.time())
> +
> +# (path, mode, uid, gid)
> +entries = [
> +    ("home",         0o755, 0,     0),
> +    ("home/${NONROOT_USER}", 0o700, ${NONROOT_UID}, ${NONROOT_GID}),
> +]

Two things about the Python heredoc:

1. The 'PYEOF' delimiter uses single quotes, which would normally suppress
   shell expansion inside the heredoc — but bitbake expands ${NONROOT_USER}
   etc. at parse time *before* the shell ever sees the body, so the
   expansion does happen, just via a different path than the reader expects.
   Worth a one-line comment so the next person doesn't try to "fix" it.

2. While the values are simple here (alphanumeric user, integer uid/gid)
   it's fine, but if someone ever sets NONROOT_USER to something with a
   quote or backslash, the Python source is no longer well-formed. Since
   this is a meta-virt-internal class, low risk — but a comment
   ("NONROOT_USER must be a bare identifier") would protect it.

> +    umoci raw add-layer --image "$image_name:${OCI_IMAGE_TAG}" "$layer_tar"
> +    rm -f "$layer_tar"
> +
> +    rm -f "$image_name.tar" "$image_name-dir.tar"
> +    ( cd "$image_name" && tar -cf "../$image_name.tar" "." )
> +    tar -cf "$image_name-dir.tar" "$image_name"
> +}
> +do_image_oci[postfuncs] += "fix_oci_home_perms"

The repackaging step at the bottom assumes do_image_oci's output layout
(image_name.tar / image_name-dir.tar) is stable. If image-oci.bbclass
ever changes its packaging naming, this silently produces a broken
tarball — there's no error checking against the assumption.

A defensive option: stat the expected files before rm -f / tar -cf, and
bbwarn if either is missing rather than recreating from a possibly-empty
directory. Or, if there's a helper in image-oci.bbclass that already
does the repackaging, call that instead of rebuilding by hand.

Otherwise the series looks good — I'll keep going through 2/7..7/7 and
respond per-patch as I have specific notes.

Bruce

Re: [meta-virtualization][PATCH 1/7] classes: add container-nonroot-user.bbclass

[Tim Orling]

On 2026-06-12 09:54:17-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> A few things on this one — none of them blocking the intent, but worth a
> small follow-up either as a fixup in this series or a v2.
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > For secure and production environments, we want to run containers as a
> > non-root user. Some applications, such as Python, require a $HOME
> > directory with proper permissions. Because OCI_LAYERS :directories:
> > copies with 'cp -a --no-preserve=ownership', we need a fixup function
> > to create the proper permissions and ownership in a new raw layer.
> >
> > The behavior here is inspired by dhi.io/python:3 (Docker Hardened Image)
> >
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> 
> The non-root + DHI-style image is a real gap in what we ship today;
> glad to see it filled.
> 
> > +inherit extrausers
> > +
> > +NONROOT_USER ?= "nonroot"
> > +NONROOT_UID ?= "65532"
> > +NONROOT_GID ?= "65532"
> > +
> > +# ---------------------------------------------------------------------------
> > +# Create the unprivileged "nonroot" user (uid 65532, group 65532)
> > +# ---------------------------------------------------------------------------
> > +EXTRA_USERS_PARAMS = "\
> > +    groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
> > +    useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} -d /home/${NONROOT_USER}  ${NONROOT_USER}; \
> > +"
> 
> This hard-assigns EXTRA_USERS_PARAMS rather than appending. Any image
> that inherits container-nonroot-user AND wants its own extrausers (e.g.
> a recipe that needs an additional service account) loses whatever else
> was set, and the order of inheritance silently determines who wins.
> 
> Suggest:
> 
>   EXTRA_USERS_PARAMS += " \
>       groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
>       useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} \
>               -d /home/${NONROOT_USER} ${NONROOT_USER}; \
>   "
> 
> That way the class composes cleanly with anyone else who sets
> EXTRA_USERS_PARAMS earlier.
> 

Good catch and suggestion. Applied in v2.

> > +# Make sure we can write to e.g. /home/nonroot/.python_history
> > +# using :directories: in OCI_LAYERS does not preserve permissions
> > +fakeroot fix_oci_home_perms() {
> > +    cd ${IMGDEPLOYDIR}
> > +    image_name="${IMAGE_NAME}${IMAGE_NAME_SUFFIX}-oci"
> > +    layer_tar="${WORKDIR}/oci-home-fix-layer.tar"
> > +
> > +    rm -f "$layer_tar"
> > +
> > +    python3 - "$layer_tar" <<'PYEOF'
> > +import sys, tarfile, time
> > +
> > +layer_tar = sys.argv[1]
> > +mtime = int(time.time())
> > +
> > +# (path, mode, uid, gid)
> > +entries = [
> > +    ("home",         0o755, 0,     0),
> > +    ("home/${NONROOT_USER}", 0o700, ${NONROOT_UID}, ${NONROOT_GID}),
> > +]
> 
> Two things about the Python heredoc:
> 
> 1. The 'PYEOF' delimiter uses single quotes, which would normally suppress
>    shell expansion inside the heredoc — but bitbake expands ${NONROOT_USER}
>    etc. at parse time *before* the shell ever sees the body, so the
>    expansion does happen, just via a different path than the reader expects.
>    Worth a one-line comment so the next person doesn't try to "fix" it.
> 
> 2. While the values are simple here (alphanumeric user, integer uid/gid)
>    it's fine, but if someone ever sets NONROOT_USER to something with a
>    quote or backslash, the Python source is no longer well-formed. Since
>    this is a meta-virt-internal class, low risk — but a comment
>    ("NONROOT_USER must be a bare identifier") would protect it.
> 

Added comments for 1. and 2. in v2.

> > +    umoci raw add-layer --image "$image_name:${OCI_IMAGE_TAG}" "$layer_tar"
> > +    rm -f "$layer_tar"
> > +
> > +    rm -f "$image_name.tar" "$image_name-dir.tar"
> > +    ( cd "$image_name" && tar -cf "../$image_name.tar" "." )
> > +    tar -cf "$image_name-dir.tar" "$image_name"
> > +}
> > +do_image_oci[postfuncs] += "fix_oci_home_perms"
> 
> The repackaging step at the bottom assumes do_image_oci's output layout
> (image_name.tar / image_name-dir.tar) is stable. If image-oci.bbclass
> ever changes its packaging naming, this silently produces a broken
> tarball — there's no error checking against the assumption.
> 
> A defensive option: stat the expected files before rm -f / tar -cf, and
> bbwarn if either is missing rather than recreating from a possibly-empty
> directory. Or, if there's a helper in image-oci.bbclass that already
> does the repackaging, call that instead of rebuilding by hand.
> 

There is no helper in image-oci.bbclass, the repackaging is handled in
do_image_oci(). Added logic gated on OCI_IMAGE_TAR_OUTPUT to detect the
presence of $image_name/index.json and throw bbfatal if it is not found.
We could bbwarn, but I think we should fail loudly.

> Otherwise the series looks good — I'll keep going through 2/7..7/7 and
> respond per-patch as I have specific notes.
> 
> Bruce

Re: [meta-virtualization][PATCH 2/7] recipes-containers/images: add app-container-python

[Bruce Ashfield]

Hi Tim,

Just a couple of comments ...

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> Add OCI container image recipe for Python to use as a base for
> other Python app containers. The image uses multi-layer mode with
> separate base, terminal and python layers.
[...]
> +OCI_LAYERS = "\
> +    base:packages:base-files+base-passwd+netbase \
> +    terminal:packages:ncurses-terminfo-base \
> +    python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG', 'dev', '+python3-pip', '', d)} \
> +"

I like this conditional. Putting the bb.utils.contains() directly inside
the layer's package list (instead of duplicating the OCI_LAYERS
declaration in two PACKAGECONFIG branches).

Worth calling out in our multi-layer mode docs as the recommended way
to do conditional packages-per-layer. I'll do a patch for that.

> +# IMAGE_INSTALL triggers package builds via do_rootfs recrdeptask.
> +# Even for multi-layer mode, list packages here to ensure they're built.
> +# The PM will install them directly to layers from DEPLOY_DIR_IPK.
> +# Note: IMAGE_ROOTFS is still created but ignored for packages layers.
> +IMAGE_INSTALL = "base-files base-passwd netbase"
> +IMAGE_INSTALL += "ncurses-terminfo-base"
> +IMAGE_INSTALL += "python3 coreutils"
> +IMAGE_INSTALL += "${@bb.utils.contains('PACKAGECONFIG', 'dev', 'python3-pip', '', d)}"

This is correct today but it doubles the source of truth. The packages
listed in OCI_LAYERS:packages: and the packages listed in IMAGE_INSTALL
have to be kept in sync.

If they drift (change OCI_LAYERS but forget IMAGE_INSTALL or vice
versa), the build silently breaks at layer assembly time when the
missing package isn't in DEPLOY_DIR_IPK.

Two paths I'd consider, either as part of this series or as a follow-up:

  a) Add a "# KEEP IN SYNC WITH OCI_LAYERS" comment so the next
     maintainer knows.

  b) Better: derive IMAGE_INSTALL from OCI_LAYERS:packages: layers
     inside image-oci.bbclass, so the recipe only sets it once. That
     fixes the problem for every multi-layer recipe, not just python.

I've staged b) on master-next (shortly), if you can do a) .. or maybe
it isn't needed at all now.

We can do the same for all the other similar recipes in the series.

One more, also non-blocking — the other 4 new image recipes in this
series (mosquitto, valkey, nginx, curl) all carry the
rootfs_fixup_var_volatile postprocess to create /var/volatile/{tmp,log}.

This one doesn't. Intentional because python doesn't touch /var/log? Or
worth adding for consistency in case someone runs as 'dev' and pip
wants a writable spot?

Bruce

Re: [meta-virtualization][PATCH 2/7] recipes-containers/images: add app-container-python

[Tim Orling]

On 2026-06-12 10:57:38-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> Just a couple of comments ...
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > Add OCI container image recipe for Python to use as a base for
> > other Python app containers. The image uses multi-layer mode with
> > separate base, terminal and python layers.
> 
> [...]
> 
> > +OCI_LAYERS = "\
> > +    base:packages:base-files+base-passwd+netbase \
> > +    terminal:packages:ncurses-terminfo-base \
> > +    python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG', 'dev', '+python3-pip', '', d)} \
> > +"
> 
> I like this conditional. Putting the bb.utils.contains() directly inside
> the layer's package list (instead of duplicating the OCI_LAYERS
> declaration in two PACKAGECONFIG branches).
> 
> Worth calling out in our multi-layer mode docs as the recommended way
> to do conditional packages-per-layer. I'll do a patch for that.

Just capturing that conditional packages-per-layer docs merged in:
27e41b91 docs/container-bundling: refresh multi-layer mode section

> 
> > +# IMAGE_INSTALL triggers package builds via do_rootfs recrdeptask.
> > +# Even for multi-layer mode, list packages here to ensure they're built.
> > +# The PM will install them directly to layers from DEPLOY_DIR_IPK.
> > +# Note: IMAGE_ROOTFS is still created but ignored for packages layers.
> > +IMAGE_INSTALL = "base-files base-passwd netbase"
> > +IMAGE_INSTALL += "ncurses-terminfo-base"
> > +IMAGE_INSTALL += "python3 coreutils"
> > +IMAGE_INSTALL += "${@bb.utils.contains('PACKAGECONFIG', 'dev', 'python3-pip', '', d)}"
> 
> This is correct today but it doubles the source of truth. The packages
> listed in OCI_LAYERS:packages: and the packages listed in IMAGE_INSTALL
> have to be kept in sync.
> 
> If they drift (change OCI_LAYERS but forget IMAGE_INSTALL or vice
> versa), the build silently breaks at layer assembly time when the
> missing package isn't in DEPLOY_DIR_IPK.
> 
> Two paths I'd consider, either as part of this series or as a follow-up:
> 
>   a) Add a "# KEEP IN SYNC WITH OCI_LAYERS" comment so the next
>      maintainer knows.
> 
>   b) Better: derive IMAGE_INSTALL from OCI_LAYERS:packages: layers
>      inside image-oci.bbclass, so the recipe only sets it once. That
>      fixes the problem for every multi-layer recipe, not just python.
> 
> I've staged b) on master-next (shortly), if you can do a) .. or maybe
> it isn't needed at all now.

Option b) was merged in:
5470cd0a image-oci: auto-derive IMAGE_INSTALL from OCI_LAYERS packages layers

I don't think a) is needed anymore, since b) is proven to be working

Refactored to remove the redundant/brittle IMAGE_INSTALL in v2.

> 
> We can do the same for all the other similar recipes in the series.

Other similar recipes addressed in v2.

> 
> One more, also non-blocking — the other 4 new image recipes in this
> series (mosquitto, valkey, nginx, curl) all carry the
> rootfs_fixup_var_volatile postprocess to create /var/volatile/{tmp,log}.
> 
> This one doesn't. Intentional because python doesn't touch /var/log? Or
> worth adding for consistency in case someone runs as 'dev' and pip
> wants a writable spot?

Added rootfs_fixup_var_volatile via 'inherit container-volatile-fixup'
in v2.

> 
> Bruce

Re: [meta-virtualization][PATCH 3/7] recipes-containers/images: add app-container-mosquitto

[Bruce Ashfield]

Hi Tim,

A few comments — most are series-level patterns that show up here for
the first time. I won't repeat them later to not waste our time

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> Add OCI container image recipe for the Eclipse Mosquitto MQTT broker.
> The image uses multi-layer mode with separate base and mosquitto layers,
> exposes standard MQTT (1883) and WebSocket (9001) ports, and launches
> mosquitto with its default config file as the entrypoint.
>
> Inherit container-nonroot-user to run as 'nonroot' with UID 65532.

> +OCI_LAYERS = "\
> +    base:packages:base-files+base-passwd+netbase \
> +    mosquitto:packages:mosquitto \
> +"
[...]
> +IMAGE_INSTALL = " \
> +    base-files \
> +    base-passwd \
> +    netbase \
> +    mosquitto \
> +"

Same point as 2/7 — image-oci.bbclass now folds packages: layers into
IMAGE_INSTALL automatically (I pushed to master-next). When you respin,
this block can go.

> +# Workaround /var/volatile for now
> +ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
> +rootfs_fixup_var_volatile () {
> +    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
> +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
> +}

This same function appears in 3/7 (mosquitto), 4/7 (valkey), 5/7
(nginx), and 7/7 (curl) — four near-identical copies. Worth factoring
into a small helper bbclass (e.g. container-volatile-fixup.bbclass) that
each recipe can inherit, or rolling it into container-nonroot-user.bbclass
since every recipe that uses the fixup also inherits the nonroot class.
Not blocking — but it's a smell that's easy to silence in v2.

> +OCI_IMAGE_ENTRYPOINT = "${sbindir}/mosquitto"
> +OCI_IMAGE_ENTRYPOINT_ARGS = "-c '${sysconfdir}/mosquitto/mosquitto.conf'"

Two questions about running mosquitto as our nonroot (uid 65532):

1. The mosquitto package usually creates a 'mosquitto' system user
   (low uid, varies by build) and ships a default config that
   references paths under /var/lib/mosquitto/ and /var/log/mosquitto/
   owned by that user. As nonroot we won't be the package's expected
   user. Does mosquitto -c on the stock config actually start cleanly
   for you, or did you need to tweak the conf?

   I assume it runs fie, since you've been testing it for a while

2. If persistence is enabled in the stock mosquitto.conf (persistence
   true; persistence_location /var/lib/mosquitto/), we need a
   writable /var/lib/mosquitto for our nonroot user — same flavour of
   gap the rootfs_fixup_var_volatile workaround addresses, just for
   a different path. If you've already validated that persistence is
   off in the stock conf (or that mosquitto degrades gracefully when
   the persistence dir isn't writable), a one-line comment in the
   recipe explaining the trade-off would save the next person from
   re-deriving it.

Bruce

> +OCI_IMAGE_PORTS = "1883/tcp 9001/tcp"
> +OCI_IMAGE_TAG = "latest"

Re: [meta-virtualization][PATCH 3/7] recipes-containers/images: add app-container-mosquitto

[Tim Orling]

On 2026-06-12 11:06:31-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> A few comments — most are series-level patterns that show up here for
> the first time. I won't repeat them later to not waste our time
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > Add OCI container image recipe for the Eclipse Mosquitto MQTT broker.
> > The image uses multi-layer mode with separate base and mosquitto layers,
> > exposes standard MQTT (1883) and WebSocket (9001) ports, and launches
> > mosquitto with its default config file as the entrypoint.
> >
> > Inherit container-nonroot-user to run as 'nonroot' with UID 65532.
> 
> 
> 
> > +OCI_LAYERS = "\
> > +    base:packages:base-files+base-passwd+netbase \
> > +    mosquitto:packages:mosquitto \
> > +"
> 
> [...]
> 
> > +IMAGE_INSTALL = " \
> > +    base-files \
> > +    base-passwd \
> > +    netbase \
> > +    mosquitto \
> > +"
> 
> Same point as 2/7 — image-oci.bbclass now folds packages: layers into
> IMAGE_INSTALL automatically (I pushed to master-next). When you respin,
> this block can go.

Removed redundant IMAGE_INSTALL in v2.

> 
> > +# Workaround /var/volatile for now
> > +ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
> > +rootfs_fixup_var_volatile () {
> > +    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
> > +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
> > +}
> 
> This same function appears in 3/7 (mosquitto), 4/7 (valkey), 5/7
> (nginx), and 7/7 (curl) — four near-identical copies. Worth factoring
> into a small helper bbclass (e.g. container-volatile-fixup.bbclass) that
> each recipe can inherit, or rolling it into container-nonroot-user.bbclass
> since every recipe that uses the fixup also inherits the nonroot class.
> Not blocking — but it's a smell that's easy to silence in v2.

Added container-volatile-fixup.bbclass in v2.

> 
> > +OCI_IMAGE_ENTRYPOINT = "${sbindir}/mosquitto"
> > +OCI_IMAGE_ENTRYPOINT_ARGS = "-c '${sysconfdir}/mosquitto/mosquitto.conf'"
> 
> Two questions about running mosquitto as our nonroot (uid 65532):
> 
> 1. The mosquitto package usually creates a 'mosquitto' system user
>    (low uid, varies by build) and ships a default config that
>    references paths under /var/lib/mosquitto/ and /var/log/mosquitto/
>    owned by that user. As nonroot we won't be the package's expected
>    user. Does mosquitto -c on the stock config actually start cleanly
>    for you, or did you need to tweak the conf?
> 
>    I assume it runs fie, since you've been testing it for a while

It turns out that in v1 we were not really running as the 'nonroot' user,
because of image-oci and container-nonroot-user both setting OCI_IMAGE_RUNTIME_UID
with ?= (which means the order of inheritance influenced the behavior). Part of
v2 changes image-oci to OCI_IMAGE_RUNTIME_UID ??= so container-nonroot-user can
'override' it (while still being a 'soft' ?= assignment).

It runs fine in both 'production' (default) and '-dev' flavors.

$ docker run --rm -it registry.yocto.io/library/mosquitto:2

1783090865: Info: running mosquitto as user: nonroot.
1783090865: mosquitto version 2.1.2 starting
1783090865: Config loaded from /etc/mosquitto/mosquitto.conf.
1783090865: Bridge support available.
1783090865: Persistence support available.
1783090865: TLS support available.
1783090865: TLS-PSK support available.
1783090865: Websockets support available.
1783090865: Starting in local only mode. Connections will only be possible from clients running on this machine.
1783090865: Create a configuration file which defines a listener to allow remote access.
1783090865: For more details see https://mosquitto.org/documentation/authentication-methods/
1783090865: Opening ipv4 listen socket on port 1883.
1783090865: Opening ipv6 listen socket on port 1883.
1783090865: mosquitto version 2.1.2 running

$ docker run --rm -it registry.yocto.io/library/mosquitto:2-dev

1783090933: Info: running mosquitto as user: mosquitto.
1783090933: mosquitto version 2.1.2 starting
1783090933: Config loaded from /etc/mosquitto/mosquitto.conf.
1783090933: Bridge support available.
1783090933: Persistence support available.
1783090933: TLS support available.
1783090933: TLS-PSK support available.
1783090933: Websockets support available.
1783090933: Starting in local only mode. Connections will only be possible from clients running on this machine.
1783090933: Create a configuration file which defines a listener to allow remote access.
1783090933: For more details see https://mosquitto.org/documentation/authentication-methods/
1783090933: Opening ipv4 listen socket on port 1883.
1783090933: Opening ipv6 listen socket on port 1883.
1783090933: mosquitto version 2.1.2 running

For '-dev' mode, the container starts as 'root' and setuids to 'mosquitto' user.

One of the other things I noticed in testing is that the 'nonroot' UID 65532
was never making it into /etc/passwd, because the base-passwd package is what
was providing /etc/passwd and the useradd/extrausers mechanism installs in
IMAGE_ROOTFS, which was not being brought into the container layers. This has
also been fixed in v2 of container-nonroot-user.bbclass.

The mosquitto.conf that is built by the recipe has all lines commented out.
In practice, end users will almost always want to provide their own mosquitto.conf.
For instance, they will probably want to require a password and access control (ACL).

Many real-world uses on target will probably want a multi-container approach, putting a
reverse proxy like nginx in front of mosquitto or running with docker-compose.

> 
> 2. If persistence is enabled in the stock mosquitto.conf (persistence
>    true; persistence_location /var/lib/mosquitto/), we need a
>    writable /var/lib/mosquitto for our nonroot user — same flavour of
>    gap the rootfs_fixup_var_volatile workaround addresses, just for
>    a different path. If you've already validated that persistence is
>    off in the stock conf (or that mosquitto degrades gracefully when
>    the persistence dir isn't writable), a one-line comment in the
>    recipe explaining the trade-off would save the next person from
>    re-deriving it.

The as-shipped mosquitto.conf has all lines commented out, so persistence is not enabled.

> 
> Bruce

Re: [meta-virtualization][PATCH 4/7] recipes-containers/images: add app-container-valkey

[Bruce Ashfield]

Hi Tim,

Two valkey-specific comments — the series-level points from 2/7 and 3/7
apply here too but I won't re-raise them.

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> Add OCI container image recipe for the Valkey in-memory key-value
> datastore. The image uses multi-layer mode with separate base and
> valkey layers, exposes the standard Valkey port (6379), and launches
> valkey-server with its default config file as the entrypoint.
[...]
> +OCI_IMAGE_ENTRYPOINT = "${bindir}/valkey-server"
> +# The stock valkey.conf shipped by meta-oe is tuned for a host install
> +# (daemonize yes, syslog-enabled yes, bind 127.0.0.1). Override those at
> +# launch so the server stays in the foreground as PID 1, logs to stdout,
> +# and is reachable from outside the container.
> +OCI_IMAGE_ENTRYPOINT_ARGS = "'${sysconfdir}/valkey/valkey.conf' \
> +    --daemonize no \
> +    --syslog-enabled no \
> +    --bind '0.0.0.0 -::*' \
> +    --protected-mode no"

The override-the-conf-at-launch trick is nice, I like it, and it's
documented in the comments.

What about a description of `--protected-mode no` ?

I quickly searched and found with protected-mode off and bind on all
interfaces, anyone who can reach the container gets unauthenticated
access — that's the right default for a base image people will pull
and customise, but a deployer who slaps this into production without
adding AUTH or namespace isolation gets a surprise. Worth a single
sentence in DESCRIPTION saying so explicitly:

Maybe this ?

  DESCRIPTION = "OCI container running the Valkey in-memory key-value \
  datastore, a flexible distributed datastore that supports both caching \
  and beyond caching workloads. This image runs with protected-mode \
  disabled and bound to all interfaces, intended as a base for \
  customisation — production deployments should enable requirepass / \
  ACLs or restrict the network namespace."

The second is the same persistence question I asked on mosquitto. The
stock meta-oe valkey.conf may or may not enable AOF / RDB snapshotting
(`appendonly yes` / `save <seconds> <writes>`). If either is on,
valkey-server tries to write to its `dir` (usually /var/lib/valkey).
Running as our nonroot uid 65532 against a /var/lib/valkey owned by the
valkey package user will fail-to-write or worse, silently lose data.

Did you confirm during testing whether the stock conf has persistence
on? If on, we want a /var/lib/valkey fixup similar to the
/var/volatile one. If off, a one-line comment saying so would save the
next person from re-deriving it.

Bruce

Re: [meta-virtualization][PATCH 4/7] recipes-containers/images: add app-container-valkey

[Tim Orling]

On 2026-06-12 11:11:24-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> Two valkey-specific comments — the series-level points from 2/7 and 3/7
> apply here too but I won't re-raise them.
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > Add OCI container image recipe for the Valkey in-memory key-value
> > datastore. The image uses multi-layer mode with separate base and
> > valkey layers, exposes the standard Valkey port (6379), and launches
> > valkey-server with its default config file as the entrypoint.
> 
> [...]
> 
> > +OCI_IMAGE_ENTRYPOINT = "${bindir}/valkey-server"
> > +# The stock valkey.conf shipped by meta-oe is tuned for a host install
> > +# (daemonize yes, syslog-enabled yes, bind 127.0.0.1). Override those at
> > +# launch so the server stays in the foreground as PID 1, logs to stdout,
> > +# and is reachable from outside the container.
> > +OCI_IMAGE_ENTRYPOINT_ARGS = "'${sysconfdir}/valkey/valkey.conf' \
> > +    --daemonize no \
> > +    --syslog-enabled no \
> > +    --bind '0.0.0.0 -::*' \
> > +    --protected-mode no"
> 
> The override-the-conf-at-launch trick is nice, I like it, and it's
> documented in the comments.
> 
> What about a description of `--protected-mode no` ?

Although I also like the override-the-conf-at-launch, on further review of
upstream containers, they use tini and an entrypoint.sh script to launch
with a .conf if passed in on the command line and otherwise no conf.

The whole reason I did the override-the-conf-at-launch was that the default
valkey.conf provided by our valkey recipe was not launching cleanly.

I'm a little concerned that by doing the overrides with OCI_IMAGE_ENTRYPOINT_ARGS
like this it will be more confusing for the end user to re-use the container.

> 
> I quickly searched and found with protected-mode off and bind on all
> interfaces, anyone who can reach the container gets unauthenticated
> access — that's the right default for a base image people will pull
> and customise, but a deployer who slaps this into production without
> adding AUTH or namespace isolation gets a surprise. Worth a single
> sentence in DESCRIPTION saying so explicitly:
> 
> Maybe this ?
> 
>   DESCRIPTION = "OCI container running the Valkey in-memory key-value \
>   datastore, a flexible distributed datastore that supports both caching \
>   and beyond caching workloads. This image runs with protected-mode \
>   disabled and bound to all interfaces, intended as a base for \
>   customisation — production deployments should enable requirepass / \
>   ACLs or restrict the network namespace."
> 
> The second is the same persistence question I asked on mosquitto. The
> stock meta-oe valkey.conf may or may not enable AOF / RDB snapshotting
> (`appendonly yes` / `save <seconds> <writes>`). If either is on,
> valkey-server tries to write to its `dir` (usually /var/lib/valkey).
> Running as our nonroot uid 65532 against a /var/lib/valkey owned by the
> valkey package user will fail-to-write or worse, silently lose data.
> 
> Did you confirm during testing whether the stock conf has persistence
> on? If on, we want a /var/lib/valkey fixup similar to the
> /var/volatile one. If off, a one-line comment saying so would save the
> next person from re-deriving it.
> 
> Bruce

Re: [meta-virtualization][PATCH 5/7] recipes-containers/images: add app-container-nginx

[Bruce Ashfield]

Hi Tim,

Most complex recipe in the series, and I like the the runtime-dir
handling

A few nginx-specific things, that came up when I was searching up
the runtime parts.

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> Add OCI container image recipe for the NGINX web server. The image
> uses multi-layer mode with separate base, nginx packages, nginx
> runtime directories, and nginx log file layers.
[...]
> +NONROOT_USER = "nginx"

The biggest open question here. The nginx recipe in meta-webserver
creates its own 'nginx' user (via useradd in inherit useradd or a
postinst), with a uid the package picks. Now we have two paths trying
to create user 'nginx':

  a) container-nonroot-user.bbclass adds it via extrausers as uid
     65532 / gid 65532 (whatever NONROOT_UID/GID are set to).

  b) The nginx package's own user-creation step adds it with the
     package's chosen uid.

Last one to run wins in /etc/passwd, but RPM/dpkg may also refuse a
duplicate-uid useradd outright. Did you see any "user already exists"
or uid-mismatch noise in do_rootfs? If not, the order happens to be
fine in your build but it's fragile.

Are these options for v2 ?

  1) Override NONROOT_UID to whatever the nginx package picks for the
     nginx user. Looks up the package's uid, set NONROOT_UID to match.

  2) Use NONROOT_USER = "nginxapp" or similar — a name that doesn't
     collide with the package's own — and tell nginx to run as that
     user via the conf file (`user nginxapp;`) instead of relying on
     the implicit uid match.

Not blocking but worth a note in the recipe about which approach you
chose and why, so we won't get cut and paste propagation without
a reason.

> +OCI_LAYERS = "\
> +    base:packages:base-files+base-passwd+netbase \
> +    nginx:packages:nginx \
> +    nginx-dirs:directories:${localstatedir}/log/nginx+/run/nginx+${localstatedir}/volatile/tmp+${localstatedir}/volatile/log \
> +    nginx-files:files:${localstatedir}/log/nginx/access.log+${localstatedir}/log/nginx/error.log \
> +"

Nice mix of packages + directories + files layer types in a single
recipe — this is exactly the scenario the multi-layer support was
built for, glad to see it landing in a real recipe.

> +    # nginx opens the compiled-in error_log path before reading -c config.
> +    # /var/log is typically a symlink to /var/volatile/log in a Yocto rootfs,
> +    # so create the target path explicitly to guarantee the directory lands in
> +    # the container layer regardless of symlink resolution order.
> +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log
> +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log/nginx
> +
> +    # nginx's compiled-in temp paths (client_body_temp, proxy_temp, etc.) all
> +    # live under /run/nginx, which is not created by any package.
> +    install -m 755 -d ${IMAGE_ROOTFS}/run/nginx

The /run/nginx catch is great — that's the kind of thing that bites
you at first-request time and you can't reproduce without the right
URL hitting the right module. Good comment too.

> +OCI_IMAGE_APP_RECIPE = "nginx"

Glad to see this in use. It's been sitting in image-oci.bbclass as
documentation-only / hook-point, and this is the first recipe that
actually sets it. Once we wire it up to auto-extract SRCREV/branch
for OCI labels (the "future versions may auto-extract" hook comment
in the bbclass), recipes that set it correctly will get the
provenance labels for free.

Bruce

Re: [meta-virtualization][PATCH 5/7] recipes-containers/images: add app-container-nginx

[Tim Orling]

On 2026-06-12 11:15:22-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> Most complex recipe in the series, and I like the the runtime-dir
> handling
> 
> A few nginx-specific things, that came up when I was searching up
> the runtime parts.
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > Add OCI container image recipe for the NGINX web server. The image
> > uses multi-layer mode with separate base, nginx packages, nginx
> > runtime directories, and nginx log file layers.
> 
> [...]
> 
> > +NONROOT_USER = "nginx"
> 
> The biggest open question here. The nginx recipe in meta-webserver
> creates its own 'nginx' user (via useradd in inherit useradd or a
> postinst), with a uid the package picks. Now we have two paths trying
> to create user 'nginx':
> 
>   a) container-nonroot-user.bbclass adds it via extrausers as uid
>      65532 / gid 65532 (whatever NONROOT_UID/GID are set to).
> 
>   b) The nginx package's own user-creation step adds it with the
>      package's chosen uid.
> 
> Last one to run wins in /etc/passwd, but RPM/dpkg may also refuse a
> duplicate-uid useradd outright. Did you see any "user already exists"
> or uid-mismatch noise in do_rootfs? If not, the order happens to be
> fine in your build but it's fragile.
> 
> Are these options for v2 ?
> 
>   1) Override NONROOT_UID to whatever the nginx package picks for the
>      nginx user. Looks up the package's uid, set NONROOT_UID to match.
> 
>   2) Use NONROOT_USER = "nginxapp" or similar — a name that doesn't
>      collide with the package's own — and tell nginx to run as that
>      user via the conf file (`user nginxapp;`) instead of relying on
>      the implicit uid match.
> 
> Not blocking but worth a note in the recipe about which approach you
> chose and why, so we won't get cut and paste propagation without
> a reason.

Previously, our container was not even running as 'nonroot' user because of
inheritance order of image-oci and container-nonroot-user, this has been fixed.

Now, in container-nonroot-user, the UID:GID of the 'nginx' user is
changed to the 65532 'nonroot' user values in the layer which provides
/etc/passwd. The reason for keeping the 'nginx' user name is that there
are a number of places where upstream nginx uses the 'nginx' user name
explicitly. This is why the dhi.io/nginx:1.30 container sets the
'nonroot' user to have the 'nginx' user name. See for instance:
https://github.com/docker-hardened-images/catalog/blob/main/image/nginx/debian-13/stable.yaml#L81

One other issue that presented itself once we are running as uid 65532
was directory ownership. This has been fixed with the introduction of the
NONROOT_OWNED_DIRS list variable which fixes the ownership in the same
'raw' layer as the '/home/<nonroot_username>' was being fixed.

> 
> > +OCI_LAYERS = "\
> > +    base:packages:base-files+base-passwd+netbase \
> > +    nginx:packages:nginx \
> > +    nginx-dirs:directories:${localstatedir}/log/nginx+/run/nginx+${localstatedir}/volatile/tmp+${localstatedir}/volatile/log \
> > +    nginx-files:files:${localstatedir}/log/nginx/access.log+${localstatedir}/log/nginx/error.log \
> > +"
> 
> Nice mix of packages + directories + files layer types in a single
> recipe — this is exactly the scenario the multi-layer support was
> built for, glad to see it landing in a real recipe.
> 
> > +    # nginx opens the compiled-in error_log path before reading -c config.
> > +    # /var/log is typically a symlink to /var/volatile/log in a Yocto rootfs,
> > +    # so create the target path explicitly to guarantee the directory lands in
> > +    # the container layer regardless of symlink resolution order.
> > +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log
> > +    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/log/nginx
> > +
> > +    # nginx's compiled-in temp paths (client_body_temp, proxy_temp, etc.) all
> > +    # live under /run/nginx, which is not created by any package.
> > +    install -m 755 -d ${IMAGE_ROOTFS}/run/nginx
> 
> The /run/nginx catch is great — that's the kind of thing that bites
> you at first-request time and you can't reproduce without the right
> URL hitting the right module. Good comment too.
> 
> > +OCI_IMAGE_APP_RECIPE = "nginx"
> 
> Glad to see this in use. It's been sitting in image-oci.bbclass as
> documentation-only / hook-point, and this is the first recipe that
> actually sets it. Once we wire it up to auto-extract SRCREV/branch
> for OCI labels (the "future versions may auto-extract" hook comment
> in the bbclass), recipes that set it correctly will get the
> provenance labels for free.
> 
> Bruce

Re: [meta-virtualization][PATCH 6/7] vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd

[Bruce Ashfield]

Hi Tim,

Small focused change, intent is clear. Two questions, one cleanup.

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> Allow us to build nginx, apache2, etc. multiarch containers.
[...]
> +BBMASK += "meta-webserver/recipes-(?!httpd)"
[...]
>  BBMASK += "meta-filesystems/"
>  BBMASK += "meta-python/"
> -BBMASK += "meta-webserver/"

The mask is right and the regex correctly leaves recipes-httpd
parseable while keeping everything else under meta-webserver out.

A couple of things worth a follow-up:

  1. recipes-httpd contains both nginx and apache2 (and a couple of
     smaller recipes — hiawatha, lighttpd, monkey iirc). With this
     change apache2 also becomes parseable for the vcontainer distro,
     even though only nginx is used by 5/7. Is the apache2 inclusion
     intentional (to enable an app-container-apache2 follow-up later),
     or accidental?

     If accidental, tightening the regex to recipes-httpd/nginx- might
     be safer — fewer recipes pulled into parse keeps the dep graph
     smaller and avoids surprise cascades from apache2's deps.

  2. The companion entry in meta-virt-host.conf:

         BBFILE_PATTERN_IGNORE_EMPTY_<meta-webserver> = "1"
         (or similar variable name — the exact form is what
         meta-virt-host.conf uses for the other fully-masked layers)

     ...was added because meta-webserver used to be a fully-masked
     layer producing the "no recipes in this layer" warning. With this
     change, meta-webserver does parse at least one recipe (nginx),
     so the IGNORE_EMPTY entry is no longer needed and should be
     removed in this same commit. Otherwise it sits as dead config.

  3. Did you run `bitbake -g <container-image>` after this change to
     confirm the dep graph didn't pick up anything heavy you didn't
     expect? recipes-httpd's deps tend to be fairly contained, but
     it's worth a one-time check so we know the BBMASK fence is doing
     what we think it is.

The change itself is fine to apply as-is; (2) is the only thing I'd
ask be folded in before merge.

Bruce

Re: [meta-virtualization][PATCH 6/7] vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd

[Tim Orling]

On 2026-06-12 11:19:27-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> Small focused change, intent is clear. Two questions, one cleanup.
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > Allow us to build nginx, apache2, etc. multiarch containers.
> 
> [...]
> 
> > +BBMASK += "meta-webserver/recipes-(?!httpd)"
> 
> [...]
> 
> >  BBMASK += "meta-filesystems/"
> >  BBMASK += "meta-python/"
> > -BBMASK += "meta-webserver/"
> 
> The mask is right and the regex correctly leaves recipes-httpd
> parseable while keeping everything else under meta-webserver out.
> 
> A couple of things worth a follow-up:
> 
>   1. recipes-httpd contains both nginx and apache2 (and a couple of
>      smaller recipes — hiawatha, lighttpd, monkey iirc). With this
>      change apache2 also becomes parseable for the vcontainer distro,
>      even though only nginx is used by 5/7. Is the apache2 inclusion
>      intentional (to enable an app-container-apache2 follow-up later),
>      or accidental?
> 
>      If accidental, tightening the regex to recipes-httpd/nginx- might
>      be safer — fewer recipes pulled into parse keeps the dep graph
>      smaller and avoids surprise cascades from apache2's deps.

I don't currently plan to support apache2 (I'd rather focus our energy on
Caddy). Tightened up the filter to (?!httpd/nginx) in v2.

> 
>   2. The companion entry in meta-virt-host.conf:
> 
>          BBFILE_PATTERN_IGNORE_EMPTY_<meta-webserver> = "1"
>          (or similar variable name — the exact form is what
>          meta-virt-host.conf uses for the other fully-masked layers)
> 
>      ...was added because meta-webserver used to be a fully-masked
>      layer producing the "no recipes in this layer" warning. With this
>      change, meta-webserver does parse at least one recipe (nginx),
>      so the IGNORE_EMPTY entry is no longer needed and should be
>      removed in this same commit. Otherwise it sits as dead config.

Removed that line in v2.
However, warnings are still produced in vruntime-bbmask-meta-oe.inc
Move the BBFILE_PATTERN_IGNORE_EMPTY_webserver = "1" there.

> 
>   3. Did you run `bitbake -g <container-image>` after this change to
>      confirm the dep graph didn't pick up anything heavy you didn't
>      expect? recipes-httpd's deps tend to be fairly contained, but
>      it's worth a one-time check so we know the BBMASK fence is doing
>      what we think it is.
> 
> The change itself is fine to apply as-is; (2) is the only thing I'd
> ask be folded in before merge.
> 
> Bruce

Re: [meta-virtualization][PATCH 7/7] app-container-curl: use multilayer mode; container-nonroot-user

[Bruce Ashfield]

Hi Tim,

Wrap-up patch, mostly aligns curl with the new conventions. One real
question, one observation.

On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> * Move to recipes-containers/images to show maintenance intent
> * Switch to multilayer mode to more like the other "library"/"official"
>   container recipes.
> * Change OCI_IMAGE_TAG to "latest" for similar reasons.
> * Change OCI_IMAGE_ENTRYPOINT_ARGS to "--help" to be more like upstream
>   containers.

The directory move right. no issues. recipes-demo was always
"here's an example", recipes-containers/images is "this is supported".
Same for "easy" → "latest"; matches Docker convention.

The "--help" default is a small but a win ... running the image with
no args now self-documents instead of trying to hit a probably-absent
localhost:80.

> +OCI_LAYERS = "\
> +    base:packages:base-files+base-passwd+netbase \
> +    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
> +    curl:packages:curl+ca-certificates \
> +"
[...]
> -CONTAINER_SHELL = "busybox"

This worries me. The old recipe had CONTAINER_SHELL = "busybox" at
the top level, so the variable always had a value. The new recipe
removes that line and only references ${CONTAINER_SHELL} inside the
conditional shell layer.

In non-dev mode that's fine because the whole bb.utils.contains
expression resolves to the empty string and the layer doesn't appear.

In dev mode the OCI_LAYERS line expands to:

    shell:packages:

…because nothing in the recipe (or in container-nonroot-user.bbclass)
sets CONTAINER_SHELL. image-oci.bbclass's validator parses this as a
three-field entry with empty content, so it doesn't bb.fatal — but no
packages get added to the auto-derived IMAGE_INSTALL, and the
resulting layer is empty. The container is silently shipped without
a shell, which is exactly the thing dev mode is supposed to provide.

Two ways to fix:

  a) Add a default in the recipe near the PACKAGECONFIG declaration:

         CONTAINER_SHELL ??= "busybox"

     Then anyone enabling dev gets busybox by default and the variable
     remains overridable.

  b) Push the default into container-nonroot-user.bbclass so every
     recipe that inherits it gets a consistent shell value.

I'd lean (a) — the class shouldn't carry a shell concept; that's a
per-recipe choice — but either works.

> +OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"

This dev-mode-overrides-uid pattern also appears in 2/7 (python). It's
clean here, and identical there — another candidate for the helper
class refactor whenever the rootfs_fixup_var_volatile factor-out
happens, since it's the same shape of "PACKAGECONFIG dev means root".

> +IMAGE_INSTALL:append = " curl ca-certificates"

Same series-level point as the earlier recipes — image-oci.bbclass
now auto-derives this from OCI_LAYERS in v2 / master-next, so the
line can go.

Bruce

Re: [meta-virtualization][PATCH 7/7] app-container-curl: use multilayer mode; container-nonroot-user

[Tim Orling]

On 2026-06-12 11:23:51-07:00, Bruce Ashfield wrote:
> Hi Tim,
> 
> Wrap-up patch, mostly aligns curl with the new conventions. One real
> question, one observation.
> 
> On Fri, May 29, 2026 at 18:31 -0700, Tim Orling wrote:
> 
> > * Move to recipes-containers/images to show maintenance intent
> > * Switch to multilayer mode to more like the other "library"/"official"
> >   container recipes.
> > * Change OCI_IMAGE_TAG to "latest" for similar reasons.
> > * Change OCI_IMAGE_ENTRYPOINT_ARGS to "--help" to be more like upstream
> >   containers.
> 
> The directory move right. no issues. recipes-demo was always
> "here's an example", recipes-containers/images is "this is supported".
> Same for "easy" → "latest"; matches Docker convention.
> 
> The "--help" default is a small but a win ... running the image with
> no args now self-documents instead of trying to hit a probably-absent
> localhost:80.
> 
> > +OCI_LAYERS = "\
> > +    base:packages:base-files+base-passwd+netbase \
> > +    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
> > +    curl:packages:curl+ca-certificates \
> > +"
> 
> [...]
> 
> > -CONTAINER_SHELL = "busybox"
> 
> This worries me. The old recipe had CONTAINER_SHELL = "busybox" at
> the top level, so the variable always had a value. The new recipe
> removes that line and only references ${CONTAINER_SHELL} inside the
> conditional shell layer.
> 
> In non-dev mode that's fine because the whole bb.utils.contains
> expression resolves to the empty string and the layer doesn't appear.
> 
> In dev mode the OCI_LAYERS line expands to:
> 
>     shell:packages:
> 
> …because nothing in the recipe (or in container-nonroot-user.bbclass)
> sets CONTAINER_SHELL. image-oci.bbclass's validator parses this as a
> three-field entry with empty content, so it doesn't bb.fatal — but no
> packages get added to the auto-derived IMAGE_INSTALL, and the
> resulting layer is empty. The container is silently shipped without
> a shell, which is exactly the thing dev mode is supposed to provide.
> 
> Two ways to fix:
> 
>   a) Add a default in the recipe near the PACKAGECONFIG declaration:
> 
>          CONTAINER_SHELL ??= "busybox"
> 
>      Then anyone enabling dev gets busybox by default and the variable
>      remains overridable.
> 
>   b) Push the default into container-nonroot-user.bbclass so every
>      recipe that inherits it gets a consistent shell value.
> 
> I'd lean (a) — the class shouldn't carry a shell concept; that's a
> per-recipe choice — but either works.
> 

Added classes/container-dev-mode.bbclass which sets CONTAINER_SHELL to
'busybox' when PACKAGECONFIG contains 'dev'. For hardened/production mode
it is recommended to add 'container-dummy-provides' to PACKAGE_EXTRA_ARCHS
and then the class falls back to 'busybox' if that is not set.

> > +OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
> 
> This dev-mode-overrides-uid pattern also appears in 2/7 (python). It's
> clean here, and identical there — another candidate for the helper
> class refactor whenever the rootfs_fixup_var_volatile factor-out
> happens, since it's the same shape of "PACKAGECONFIG dev means root".
> 

Good point and this was refactored into container-dev-mode.bbclass.

> > +IMAGE_INSTALL:append = " curl ca-certificates"
> 
> Same series-level point as the earlier recipes — image-oci.bbclass
> now auto-derives this from OCI_LAYERS in v2 / master-next, so the
> line can go.

IMAGE_INSTALL lines removed in v2.

> 
> Bruce

[PATCH v2 00/13] Container Improvements

[Tim Orling]

This series:
  * Sets OCI_IMAGE_RUNTIME_UID with ??= weak assignment in
    image-oci to allow container-nonroot-user to set a different
    default
  * Adds a class to create/run containers with a non-root user
  * Adds a class to create non-volatile var log and tmp directories
  * Adds a class to enable 'dev' mode where the container starts as
    the root user
  * Adds new containers:
    - app-container-python
    - app-containter-mosquitto
    - app-container-valkey
    - app-container-nginx
  * Modifies app-container-curl to be more like the upstream
    experience (and more like the above containers)
  * Allows mosquitto and libwebsockets to be parsed for
    vcontainer distro so we can build multiarch containers
    for app-container-mosquitto.
  * Allows meta-webserver/recipes-httpd/nginx to be parsed for
    vcontainer distro so we can build multiarch containers
    for app-container-nginx.
  * Adds a fix for buildbot-venv shadowing for environment-setup-ci
    script

The resulting containers were tested with simple command line
usage compared to Docker provided equivalents to ensure the
same expected behavior.

Changes in v2:

  There were a fair number of changes in v2 in reponse to review
  comments and observations during testing.

  * image-oci.bbclass:
    - set OCI_IMAGE_RUNTIME_UID with ??= to allow other classes
      to override
  * container-nonroot-user.bbclass:
    - set EXTRA_USERS_PARAMS with +=
    - added comments to clarify the PYEOF Python heredoc behavior
    - added logic to detect $image_name/index.json and fail if not
      found
  * container-volatile-fixup.bbclass: avoid repeating the same
    rootfs postfunc by adding a helper class
  * container-dev-mode.bbclass: avoid repeating the same logic
    to run container as root user in '-dev' mode
  * app-container-python (and other container image recipes):
    - drop now redundant IMAGE_INSTALL lines
    - simplify by inheriting container-volatile-fixup
    - inherit container-dev-mode
  * app-container-mosquitto:
    - verify '-dev' mode switches from root user to mosquitto user
    - verify "production" mode runs as 'nonroot' user
    - fix bbmask to allow mosquitto and required libwebsockets
  * app-container-valkey:
    - added tini and container-entrypoint.sh, like upstream containers
  * app-container-nginx:
    - fixup 'nonroot' user ownership of NONROOT_OWNED_DIRS
    - fix bbmask to allow nginx (but not other recipes-httpd)
  * fix for older distro hosts where the 'websockets' Python library is
    not new enough (>10.0) and on the AutoBuilder needs to use 'buildtools'.
    The PATH in environment-setup-ci now plays properly with such a set up
    by no longer added /usr/bin and /bin before existing $PATH.

Successful build https://autobuilder.yoctoproject.org/valkyrie/#/builders/117/builds/25

Tim Orling (13):
  image-oci: set OCI_IMAGE_RUNTIME_UID with ??=
  classes: add container-nonroot-user.bbclass
  classes: add container-volatile-fixup.bbclass
  classes: add container-dev-mode.bbclass
  recipes-containers/images: add app-container-python
  recipes-containers/images: add app-container-mosquitto
  vcontainer-bbmask.inc: allow 'mosquitto', 'libwebsockets'
  recipes-containers/images: add app-container-valkey
  recipes-containers/images: add app-container-nginx
  vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd/nginx
  app-container-curl: use multilayer mode; container-nonroot-user
  container-image-multiarch: add helper recipe
  vcontainer-tarball: fix buildbot-venv shadowing

 classes/container-dev-mode.bbclass            |  34 ++++
 classes/container-nonroot-user.bbclass        | 155 ++++++++++++++++++
 classes/container-volatile-fixup.bbclass      |   6 +
 classes/image-oci.bbclass                     |   3 +-
 conf/distro/include/vcontainer-bbmask.inc     |   6 +-
 conf/layer.conf                               |   1 +
 .../images/app-container-curl.bb              |  39 +++++
 .../images/app-container-mosquitto.bb         |  43 +++++
 .../images/app-container-nginx.bb             |  62 +++++++
 .../images/app-container-python.bb            |  45 +++++
 .../images/app-container-valkey.bb            |  71 ++++++++
 .../container-entrypoint.sh                   |  18 ++
 .../images/container-image-multiarch.bb       |  31 ++++
 .../vcontainer/vcontainer-tarball.bb          |   4 +-
 recipes-demo/images/app-container-curl.bb     |  46 ------
 15 files changed, 512 insertions(+), 52 deletions(-)
 create mode 100644 classes/container-dev-mode.bbclass
 create mode 100644 classes/container-nonroot-user.bbclass
 create mode 100644 classes/container-volatile-fixup.bbclass
 create mode 100644 recipes-containers/images/app-container-curl.bb
 create mode 100644 recipes-containers/images/app-container-mosquitto.bb
 create mode 100644 recipes-containers/images/app-container-nginx.bb
 create mode 100644 recipes-containers/images/app-container-python.bb
 create mode 100644 recipes-containers/images/app-container-valkey.bb
 create mode 100644 recipes-containers/images/app-container-valkey/container-entrypoint.sh
 create mode 100644 recipes-containers/images/container-image-multiarch.bb
 delete mode 100644 recipes-demo/images/app-container-curl.bb

This series can be found at 
https://github.com/moto-timo/meta-virtualization/tree/container-improvements-v2

-- 
2.54.0

[PATCH v2 01/13] image-oci: set OCI_IMAGE_RUNTIME_UID with ??=

[Tim Orling]

Allow other subsequently inherited classes, such as container-nonroot-user
to set OCI_IMAGE_RUNTIME_UID with ?=

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 classes/image-oci.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/classes/image-oci.bbclass b/classes/image-oci.bbclass
index 1ceadc7f..9914411b 100644
--- a/classes/image-oci.bbclass
+++ b/classes/image-oci.bbclass
@@ -94,7 +94,8 @@ OCI_IMAGE_AUTHOR ?= "${PATCH_GIT_USER_NAME}"
 OCI_IMAGE_AUTHOR_EMAIL ?= "${PATCH_GIT_USER_EMAIL}"
 
 OCI_IMAGE_TAG ?= "latest"
-OCI_IMAGE_RUNTIME_UID ?= ""
+# Allow other classes e.g. container-nonroot-user to set with ?=
+OCI_IMAGE_RUNTIME_UID ??= ""
 
 OCI_IMAGE_ARCH ?= "${@oe.go.map_arch(d.getVar('TARGET_ARCH'))}"
 OCI_IMAGE_SUBARCH ?= "${@oci_map_subarch(d.getVar('TARGET_ARCH'), d.getVar('TUNE_FEATURES'), d)}"
-- 
2.54.0

[PATCH v2 02/13] classes: add container-nonroot-user.bbclass

[Tim Orling]

For secure and production environments, we want to run containers as a
non-root user. Some applications, such as Python, require a $HOME
directory with proper permissions. Because OCI_LAYERS :directories:
copies with 'cp -a --no-preserve=ownership', we need a fixup function
to create the proper permissions and ownership in a new raw layer.

Process a list of NONROOT_OWNED_DIRS which will also be added as owned
by the NONROOT_USER.

Add oci_nonroot_inject_user() to  do_image_oci[prefuncs] to inject
'nonroot' user into '/etc/passwd' when a layer is present which provides
'/etc/passwd', since 'extrausers' acts on IMAGE_ROOTFS and not the
'base-passwd' package.

The behavior here is inspired by dhi.io/python:3 and by dhi.io/valkey:9
(Docker Hardened Images).

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 classes/container-nonroot-user.bbclass | 155 +++++++++++++++++++++++++
 1 file changed, 155 insertions(+)
 create mode 100644 classes/container-nonroot-user.bbclass

diff --git a/classes/container-nonroot-user.bbclass b/classes/container-nonroot-user.bbclass
new file mode 100644
index 00000000..09826536
--- /dev/null
+++ b/classes/container-nonroot-user.bbclass
@@ -0,0 +1,155 @@
+# For secure and production environments, we want to run containers as a
+# non-root user. Some applications, such as Python, require a $HOME
+# directory with proper permissions. Because OCI_LAYERS :directories:
+# copies with 'cp -a --no-preserve=ownership', we need a fixup function
+# to create the proper permissions and ownership in a new raw layer.
+
+# The behavior here is inspired by dhi.io/python:3 (Docker Hardened Image)
+
+inherit extrausers
+
+# NONROOT_USER must be a bare identifier (no quotes or backslash, etc.)
+NONROOT_USER ?= "nonroot"
+NONROOT_UID ?= "65532"
+NONROOT_GID ?= "65532"
+# Space-separated absolute paths to create in the image, owned by nonroot.
+NONROOT_OWNED_DIRS ?= ""
+
+# ---------------------------------------------------------------------------
+# Create the unprivileged "nonroot" user (uid 65532, group 65532)
+# ---------------------------------------------------------------------------
+EXTRA_USERS_PARAMS += "\
+    groupadd -g ${NONROOT_GID} ${NONROOT_USER}; \
+    useradd -m -u ${NONROOT_UID} -g ${NONROOT_GID} \
+            -d /home/${NONROOT_USER}  ${NONROOT_USER}; \
+"
+
+# Allow a container to choose to run as 'root'
+OCI_IMAGE_RUNTIME_UID ?= "${NONROOT_UID}"
+OCI_IMAGE_ENV_VARS = "HOME=/home/${NONROOT_USER}"
+
+# In multi-layer OCI mode the image is assembled from per-layer package
+# installs (oci_multilayer_install_packages in image-oci-umoci.inc), not from
+# IMAGE_ROOTFS. extrausers/EXTRA_USERS_PARAMS only edits IMAGE_ROOTFS, so the
+# nonroot account never reaches the image. Inject it into the layer rootfs that
+# ships /etc/passwd, before IMAGE_CMD:oci assembles the layers.
+python oci_nonroot_inject_user() {
+    import os
+
+    if (d.getVar('OCI_LAYER_MODE') or 'single') != 'multi':
+        return  # single-layer mode builds from IMAGE_ROOTFS; extrausers handles it
+
+    user = d.getVar('NONROOT_USER')
+    uid  = d.getVar('NONROOT_UID')
+    gid  = d.getVar('NONROOT_GID')
+    home = '/home/%s' % user
+
+    # shell field is cosmetic (runtime user is pinned numerically via
+    # OCI config.user); /bin/sh matches dhi.io passwd entries.
+    passwd_line = '%s:x:%s:%s:%s:%s:/bin/sh\n' % (user, uid, gid, user, home)
+    group_line  = '%s:x:%s:\n' % (user, gid)
+    shadow_line = '%s:!:::::::\n' % user
+
+    def append_once(path, line, key):
+        if not os.path.exists(path):
+            return False
+        with open(path) as f:
+            if any(l.startswith(key) for l in f):
+                return True            # already present (idempotent rebuild)
+        with open(path, 'a') as f:
+            f.write(line)
+        return True
+
+    key = user + ':'
+    count = int(d.getVar('OCI_LAYER_COUNT') or 0)
+    found = False
+    for i in range(1, count + 1):
+        rootfs = d.getVar('OCI_LAYER_%d_ROOTFS' % i)
+        if not rootfs:
+            continue
+        # add to every layer that carries /etc/passwd so the topmost wins too
+        if append_once(os.path.join(rootfs, 'etc/passwd'), passwd_line, key):
+            append_once(os.path.join(rootfs, 'etc/group'),  group_line,  key)
+            append_once(os.path.join(rootfs, 'etc/shadow'), shadow_line, key)  # optional
+            found = True
+
+    if not found:
+        bb.warn("container-nonroot-user: no layer ships /etc/passwd; '%s' not "
+                "added — is base-passwd in a packages: layer?" % user)
+}
+
+# Must run AFTER oci_multilayer_install_packages populates OCI_LAYER_*_ROOTFS.
+do_image_oci[prefuncs] += "oci_nonroot_inject_user"
+
+# Make sure we can write to e.g. /home/nonroot/.python_history
+# using :directories: in OCI_LAYERS does not preserve permissions.
+fakeroot fix_oci_home_perms() {
+    cd ${IMGDEPLOYDIR}
+    image_name="${IMAGE_NAME}${IMAGE_NAME_SUFFIX}-oci"
+    layer_tar="${WORKDIR}/oci-home-fix-layer.tar"
+
+    rm -f "$layer_tar"
+
+    # BitBake expands ${NONROOT_USER} etc. at parse time *before*
+    # shell sees the body, so single quoted 'PYEOF' is okay.
+    python3 - "$layer_tar" <<'PYEOF'
+import sys, tarfile, time
+
+layer_tar = sys.argv[1]
+mtime = int(time.time())
+
+uid = ${NONROOT_UID}
+gid = ${NONROOT_GID}
+
+# (path, mode, uid, gid)  -- paths are tar-relative, no leading slash
+entries = [
+    ("home",         0o755, 0,     0),
+    ("home/${NONROOT_USER}", 0o700, uid, gid),
+]
+
+seen = {entry[0] for entry in entries}
+for nonrootdir in "${NONROOT_OWNED_DIRS}".split():
+    parts = nonrootdir.strip("/").split("/")
+    for i, _ in enumerate(parts):
+        path = "/".join(parts[:i+1])
+        if path in seen:
+            continue
+        seen.add(path)
+        leaf = (i == len(parts) - 1)
+        # leaf -> nonroot-owned; parents -> root:root, just to scaffold the path
+        entries.append((path, 0o755, uid if leaf else 0, gid if leaf else 0))
+
+with tarfile.open(layer_tar, "w") as tar:
+    for name, mode, uid, gid in entries:
+        info = tarfile.TarInfo(name=name)
+        info.type  = tarfile.DIRTYPE
+        info.mode  = mode
+        info.uid   = uid
+        info.gid   = gid
+        info.uname = ""   # numeric-only; let umoci canonicalize
+        info.gname = ""
+        info.mtime = mtime
+        tar.addfile(info)
+PYEOF
+
+    umoci raw add-layer --image "$image_name:${OCI_IMAGE_TAG}" "$layer_tar"
+    rm -f "$layer_tar"
+
+    # Adding the raw layer mutates the OCI image directory, so the tar outputs
+    # produced by do_image_oci are now stale and must be rebuilt. The image
+    # *directory* ($image_name) is the source of truth; the .tar files are
+    # derived from it and only exist when OCI_IMAGE_TAR_OUTPUT is set, so mirror
+    # that gating (see image-oci-umoci.inc) instead of rebuilding unconditionally.
+    # Guard the assumption: an OCI layout always has an index.json, so its
+    # absence means the packaging layout in image-oci.bbclass has changed (or
+    # the dir is empty). Fail loudly rather than emit a silently-broken tarball.
+    if [ -n "${OCI_IMAGE_TAR_OUTPUT}" ]; then
+        if [ ! -f "$image_name/index.json" ]; then
+            bbfatal "fix_oci_home_perms: '$image_name' is not an OCI image layout (no index.json) in ${IMGDEPLOYDIR}; image-oci packaging layout may have changed"
+        fi
+        rm -f "$image_name.tar" "$image_name-dir.tar"
+        ( cd "$image_name" && tar -cf "../$image_name.tar" "." )
+        tar -cf "$image_name-dir.tar" "$image_name"
+    fi
+}
+do_image_oci[postfuncs] += "fix_oci_home_perms"
-- 
2.54.0

[PATCH v2 03/13] classes: add container-volatile-fixup.bbclass

[Tim Orling]

Add a ROOTFS_POSTPROCESS_COMMAND to fixup 'log' and 'tmp' in
/var/volatile.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 classes/container-volatile-fixup.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 classes/container-volatile-fixup.bbclass

diff --git a/classes/container-volatile-fixup.bbclass b/classes/container-volatile-fixup.bbclass
new file mode 100644
index 00000000..1b8a4c1e
--- /dev/null
+++ b/classes/container-volatile-fixup.bbclass
@@ -0,0 +1,6 @@
+# Workaround /var/volatile for now
+ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
+rootfs_fixup_var_volatile () {
+    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
+    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
+}
-- 
2.54.0

[PATCH v2 04/13] classes: add container-dev-mode.bbclass

[Tim Orling]

Shared 'dev' mode support for containers-library app images.

Enable per recipe with:
   PACKAGECONFIG:pn-<recipe> = "dev"

 'dev' mode wants a shell; default 'production' intent does not, for
 security purposes. Recipes still choose for themselves where
 CONTAINER_SHELL actually gets consumed (e.g. an OCI_LAYERS 'shell' layer
 gated on PACKAGECONFIG 'dev') and whether 'dev' should also relax
 OCI_IMAGE_RUNTIME_UID to run as root.

Each recipe should set the following, we most likely do not want to set
at the class level.
PACKAGECONFIG ??= ""
PACKAGECONFIG[dev] = ""

Inspired by the -dev tagged Docker Hardened Images on dhi.io

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 classes/container-dev-mode.bbclass | 34 ++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
 create mode 100644 classes/container-dev-mode.bbclass

diff --git a/classes/container-dev-mode.bbclass b/classes/container-dev-mode.bbclass
new file mode 100644
index 00000000..cb96fa90
--- /dev/null
+++ b/classes/container-dev-mode.bbclass
@@ -0,0 +1,34 @@
+# Shared 'dev' mode support for containers-library app images.
+#
+# Enable per recipe with:
+#   PACKAGECONFIG:pn-<recipe> = "dev"
+#
+# 'dev' mode wants a shell; default 'production' intent does not, for
+# security purposes. Recipes still choose for themselves where
+# CONTAINER_SHELL actually gets consumed (e.g. an OCI_LAYERS 'shell' layer
+# gated on PACKAGECONFIG 'dev') and whether 'dev' should also relax
+# OCI_IMAGE_RUNTIME_UID to run as root.
+#
+# Each recipe should set the following, we most likely do not want to set
+# at the class level.
+# PACKAGECONFIG ??= ""
+# PACKAGECONFIG[dev] = ""
+
+# In 'dev' mode, override the nonroot UID inherited from container-nonroot-user
+# so the container runs as root.
+OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
+
+# 'dev' always wins and gets a real shell (busybox), regardless of whether
+# production intent (below) was also configured, e.g. in local.conf.
+#
+# For production, if the following is configured in local.conf (or the
+# distro):
+#      PACKAGE_EXTRA_ARCHS:append = " container-dummy-provides"
+#
+# it has been explicitly indicated that we don't want or need a shell, so
+# we'll add the dummy provides instead of busybox.
+#
+# This is required, since there are postinstall scripts in base-files and
+# base-passwd that reference /bin/sh and we'll get a rootfs error if
+# there's no shell or no dummy provider.
+CONTAINER_SHELL ?= "${@bb.utils.contains('PACKAGECONFIG', 'dev', 'busybox', bb.utils.contains('PACKAGE_EXTRA_ARCHS', 'container-dummy-provides', 'container-dummy-provides', 'busybox', d), d)}"
-- 
2.54.0

[PATCH v2 05/13] recipes-containers/images: add app-container-python

[Tim Orling]

From: Tim Orling <ticotimo@gmail.com>

Add OCI container image recipe for Python to use as a base for
other Python app containers. The image uses multi-layer mode with
separate base, terminal and python layers.

Add ncurses-terminfo-base to a "terminal" layer to avoid warnings in the
REPL:
  "Cannot read termcap database;
  using dumb terminal settings."

Add coreutils to "python" layer to provide /usr/bin/env needed by
python3-idle in python3-modules.

Inherit container-nonroot-user and run a `nonroot` user by default.
Set PACKAGECONFIG:pn-app-container-python = "dev" in local.conf or
distro/image config to run as 'root' and include 'pip'.

Inherit container-volatile-fixup to ensure /var/volatile/{tmp,log}
are available, if needed.

Inherit container-dev-mode to optionally run as UID '0' root user
and provide a shell.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-python.bb            | 45 +++++++++++++++++++
 1 file changed, 45 insertions(+)
 create mode 100644 recipes-containers/images/app-container-python.bb

diff --git a/recipes-containers/images/app-container-python.bb b/recipes-containers/images/app-container-python.bb
new file mode 100644
index 00000000..19a64814
--- /dev/null
+++ b/recipes-containers/images/app-container-python.bb
@@ -0,0 +1,45 @@
+SUMMARY = "Base python3 container image"
+DESCRIPTION = "OCI container image running Python with non-root user. \
+\
+In 'dev' mode, can optionally run as 'root' and add 'pip' to allow \
+developers to simply run 'pip install' on top of this container (Not \
+advised for production/hardened use)."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - adds a shell to the container
+#   - adds python3-pip to the python layer (enables `pip install` at runtime)
+#   - runs the container as root (UID 0) so pip can write to site-packages
+# Enable with: PACKAGECONFIG:pn-app-container-python = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+inherit container-dev-mode
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
+    terminal:packages:ncurses-terminfo-base \
+    python:packages:python3+coreutils${@bb.utils.contains('PACKAGECONFIG', 'dev', '+python3-pip', '', d)} \
+"
+
+# Use CMD so `docker run image /bin/sh` works as expected
+OCI_IMAGE_CMD = "python3"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+inherit container-volatile-fixup
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
-- 
2.54.0

[PATCH v2 06/13] recipes-containers/images: add app-container-mosquitto

[Tim Orling]

Add OCI container image recipe for the Eclipse Mosquitto MQTT broker.
The image uses multi-layer mode with separate base and mosquitto layers,
exposes standard MQTT (1883) and WebSocket (9001) ports, and launches
mosquitto with its default config file as the entrypoint.

Inherit container-nonroot-user to run as 'nonroot' with UID 65532.

Inherit container-volatile-fixup to fix 'log' and 'tmp' in
/var/volatile.

Inherit container-dev-mode to optionally run as UID '0' root user
and provide a shell.

Relies on image-oci to auto-derive IMAGE_INSTALL from OCI_LAYERS
packages: layers.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-mosquitto.bb         | 43 +++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 recipes-containers/images/app-container-mosquitto.bb

diff --git a/recipes-containers/images/app-container-mosquitto.bb b/recipes-containers/images/app-container-mosquitto.bb
new file mode 100644
index 00000000..9a74dcd8
--- /dev/null
+++ b/recipes-containers/images/app-container-mosquitto.bb
@@ -0,0 +1,43 @@
+SUMMARY = "Mosquitto MQTT broker container image"
+DESCRIPTION = "OCI container running the Eclipse Mosquitto MQTT broker \
+with standard MQTT (1883) and WebSocket (9001) listeners enabled."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - adds a shell to the container
+#   - adds python3-pip to the python layer (enables `pip install` at runtime)
+#   - runs the container as root (UID 0) so pip can write to site-packages
+# Enable with: PACKAGECONFIG:pn-app-container-python = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+inherit container-dev-mode
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
+    mosquitto:packages:mosquitto \
+"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+inherit container-volatile-fixup
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+OCI_IMAGE_ENTRYPOINT = "${sbindir}/mosquitto"
+OCI_IMAGE_ENTRYPOINT_ARGS = "-c '${sysconfdir}/mosquitto/mosquitto.conf'"
+OCI_IMAGE_PORTS = "1883/tcp 9001/tcp"
+OCI_IMAGE_TAG = "latest"
-- 
2.54.0

[PATCH v2 07/13] vcontainer-bbmask.inc: allow 'mosquitto', 'libwebsockets'

[Tim Orling]

Allow meta-networking/recipes-connectivity/mosquitto and required
dependency meta-oe/recipes-connectivity/libwebsockets

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 conf/distro/include/vcontainer-bbmask.inc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/conf/distro/include/vcontainer-bbmask.inc b/conf/distro/include/vcontainer-bbmask.inc
index ac74464f..f79ea9b9 100644
--- a/conf/distro/include/vcontainer-bbmask.inc
+++ b/conf/distro/include/vcontainer-bbmask.inc
@@ -88,8 +88,8 @@ BBMASK += "meta-virtualization/recipes-networking/passt/"
 # ---------------------------------------------------------------------------
 # meta-oe / meta-networking: mask heavyweight categories
 # ---------------------------------------------------------------------------
-BBMASK += "meta-oe/recipes-(?!devtools|extended|support)"
-BBMASK += "meta-networking/recipes-(?!filter|support)"
+BBMASK += "meta-oe/recipes-(?!devtools|extended|support|connectivity/libwebsockets)"
+BBMASK += "meta-networking/recipes-(?!filter|support|connectivity/mosquitto)"
 BBMASK += "meta-openstack/recipes-dbs/postgresql/"
 BBMASK += "meta-oe/dynamic-layers/networking-layer/recipes-core/"
 BBMASK += "meta-openstack/recipes-extended/libvirt/"
-- 
2.54.0

[PATCH v2 08/13] recipes-containers/images: add app-container-valkey

[Tim Orling]

From: Tim Orling <ticotimo@gmail.com>

Add OCI container image recipe for the Valkey in-memory key-value
datastore. The image uses multi-layer mode with separate base and
valkey layers, exposes the standard Valkey port (6379), and launches
valkey-server with its default config file as the entrypoint.

The stock valkey.conf shipped by meta-oe is tuned for a host install
(daemonize yes, syslog-enabled yes, bind 127.0.0.1). In most use cases
the user will want to provide their own valkey.conf. In that vein,
provide container-entrypoint.sh which will use /path/to/valkey.conf
passed in on the command line. If no .conf is passed in, we default
to the same behavior as upstream containers (no config).

Add container-entrypoint.sh script based on upstream [1]

Add OCI_IMAGE_WORKINGDIR = "/data" to avoid issues writing .rdb on save
since 'valkey' will default to '/' for the working dir.

Add NONROOT_OWNED_DIRS = "/data /var/lib/valkey /var/log/valkey /run/valkey"
as the 'nonroot' user needs permissions on these directories.

Inherit container-nonroot-user to run as 'nonroot' with UID 65532 by
default. Can optionally set PACKAGECONFIG:pn-app-container-valkey = "dev"
in local.conf or a distro/image config to run as 'root'.

Inherit container-volatile-fixup to fix 'log' and 'tmp' in
/var/volatile.

Inherit container-dev-mode to optionally run as UID '0' root user.
Since we use 'container-entrypoint.sh' which has #!/bin/sh we
provide a shell for both '-dev' and 'production' container flavors,
which matches upstream Docker Hardened Images behavior.

Relies on image-oci to auto-derive IMAGE_INSTALL from OCI_LAYERS
:packages: layers.

[1] https://github.com/valkey-io/valkey-container/blob/mainline/9.1/debian/docker-entrypoint.sh

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-valkey.bb            | 71 +++++++++++++++++++
 .../container-entrypoint.sh                   | 18 +++++
 2 files changed, 89 insertions(+)
 create mode 100644 recipes-containers/images/app-container-valkey.bb
 create mode 100644 recipes-containers/images/app-container-valkey/container-entrypoint.sh

diff --git a/recipes-containers/images/app-container-valkey.bb b/recipes-containers/images/app-container-valkey.bb
new file mode 100644
index 00000000..10c239e2
--- /dev/null
+++ b/recipes-containers/images/app-container-valkey.bb
@@ -0,0 +1,71 @@
+SUMMARY = "Valkey key-value store container image"
+DESCRIPTION = "OCI container running the Valkey in-memory key-value \
+datastore, a flexible distributed datastore that supports both caching \
+and beyond caching workloads."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-valkey = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+inherit container-dev-mode
+
+# Unlike the other app-container-* recipes, valkey always ships a real shell,
+# in 'dev' and production alike. container-entrypoint.sh is a #!/bin/sh
+# script (upstream-compatible arg handling + umask hardening; see the file
+# for details) and requires an interpreter to boot at all — 'no shell in
+# production' isn't available here without dropping that script and its
+# behavior. Matches DHI's own 'hardened' valkey image (debian-13/9.1.yaml),
+# which ships bash for the exact same reason rather than go shell-less.
+CONTAINER_SHELL = "busybox"
+
+#  image.bbclass intentionally sets do_fetch, do_unpack and do_install to noexec.
+#  We do not want to abuse that isolation, so instead get the file from local
+#  tree and install in rootfs postprocess.
+ROOTFS_POSTPROCESS_COMMAND:append = " rootfs_install_entrypoint_sh ; "
+rootfs_install_entrypoint_sh () {
+    install -m 0755 ${THISDIR}/${BPN}/container-entrypoint.sh ${IMAGE_ROOTFS}/${bindir}/
+}
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    shell:packages:${CONTAINER_SHELL} \
+    valkey:packages:valkey+tini \
+    entrypoint:files:${bindir}/container-entrypoint.sh \
+"
+
+# In 'dev' mode, override the nonroot UID inherited from container-nonroot-user
+OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG', 'dev', '0', '${NONROOT_UID}', d)}"
+
+# The 'nonroot' user needs permissions on the following directories
+NONROOT_OWNED_DIRS = "/data /var/lib/valkey /var/log/valkey /run/valkey"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+inherit container-volatile-fixup
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+# The stock valkey.conf shipped by meta-oe is tuned for a host install
+# (daemonize yes, syslog-enabled yes, bind 127.0.0.1). Most users will
+# want to create their own valkey.conf and pass it in to the
+# container-entrypoint.sh script
+OCI_IMAGE_ENTRYPOINT = "${bindir}/docker-init -- ${bindir}/container-entrypoint.sh"
+OCI_IMAGE_CMD = "${bindir}/valkey-server"
+OCI_IMAGE_PORTS = "6379/tcp"
+OCI_IMAGE_TAG = "latest"
+OCI_IMAGE_WORKINGDIR = "/data"
diff --git a/recipes-containers/images/app-container-valkey/container-entrypoint.sh b/recipes-containers/images/app-container-valkey/container-entrypoint.sh
new file mode 100644
index 00000000..36cdf93a
--- /dev/null
+++ b/recipes-containers/images/app-container-valkey/container-entrypoint.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+# Based on https://github.com/valkey-io/valkey-container/blob/mainline/docker-entrypoint.sh
+# SPDX-License-Identifier: BSD-3-Clause
+set -e
+
+# first arg is `-f` or `--some-option`
+# or first arg is `something.conf`
+if [ "${1#-}" != "$1" ] || [ "${1%.conf}" != "$1" ]; then
+    set -- valkey-server "$@"
+fi
+
+# set an appropriate umask (if one isn't set already)
+um="$(umask)"
+if [ "$um" = '0022' ]; then
+    umask 0077
+fi
+
+exec "$@"
-- 
2.54.0

[PATCH v2 09/13] recipes-containers/images: add app-container-nginx

[Tim Orling]

From: Tim Orling <ticotimo@gmail.com>

Add OCI container image recipe for the NGINX web server. The image
uses multi-layer mode with separate base, nginx packages, nginx
runtime directories, and nginx log file layers. Exposes the standard
HTTP port (80) and launches nginx with 'daemon off' so it stays in
the foreground as PID 1 and logs to stderr.

Add ROOTFS_POSTPROCESS fixups to create the runtime directories
nginx expects: /var/volatile/{tmp,log}, /var/log/nginx (resolved
explicitly to guarantee inclusion in the container layer regardless
of /var/log symlink ordering), and /run/nginx for nginx's compiled-in
temp paths (client_body_temp, proxy_temp, etc.) which are not created
by any package. Also create empty /var/log/nginx/{access,error}.log
to avoid do_image_oci warnings.

Since nginx uses user 'nginx' by name and not UID, we also need to
change ownership of several files/directories for the container to be
functional.

Inherit container-nonroot-user with NONROOT_USER = "nginx" to run with
UID 65532 by default. Set PACKAGECONFIG:pn-app-container-nginx = "dev"
in local.conf or distro/image config to run as 'root'.

Add NONROOT_OWNED_DIRS = "/run/nginx, /var/log/nginx, /var/cache/nginx,
/usr/share/nginx/html" as the 'nonroot' user needs permissions on these
directories.

Inherit container-volatile-fixup to fix 'log' and 'tmp' in
/var/volatile.

Relies on image-oci to auto-derive IMAGE_INSTALL from OCI_LAYERS
:packages: layers.

Inherit container-dev-mode to optionally run as UID '0' root user
and provide a shell.

Add SKIP_RECIPE and comment to layer.conf since nginx requires
meta-webserver.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 conf/layer.conf                               |  1 +
 .../images/app-container-nginx.bb             | 62 +++++++++++++++++++
 2 files changed, 63 insertions(+)
 create mode 100644 recipes-containers/images/app-container-nginx.bb

diff --git a/conf/layer.conf b/conf/layer.conf
index 2a4a4c91..6ea8ccf8 100644
--- a/conf/layer.conf
+++ b/conf/layer.conf
@@ -33,6 +33,7 @@ LAYERDEPENDS_virtualization-layer = " \
 # webserver:
 # - naigos requires apache2
 # - cockpit-machines requires cockpit
+# - app-container-nginx requires nginx
 LAYERRECOMMENDS_virtualization-layer = " \
     webserver \
     selinux \
diff --git a/recipes-containers/images/app-container-nginx.bb b/recipes-containers/images/app-container-nginx.bb
new file mode 100644
index 00000000..f57c7887
--- /dev/null
+++ b/recipes-containers/images/app-container-nginx.bb
@@ -0,0 +1,62 @@
+SUMMARY = "Base NGINX container image for development"
+DESCRIPTION = "OCI container with NGINX web server."
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - adds a shell to the container
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-nginx = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+inherit container-dev-mode
+NONROOT_USER = "nginx"
+
+OCI_IMAGE_APP_RECIPE = "nginx"
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
+    nginx:packages:nginx \
+    nginx-dirs:directories:${localstatedir}/log/nginx+/run/nginx+${localstatedir}/volatile/tmp+${localstatedir}/volatile/log \
+"
+
+# nginx runs as the nonroot user (uid 65532) and must own the dirs it writes.
+# The OCI_LAYERS 'directories:'/'files:' types create paths but drop ownership
+# (cp -a --no-preserve=ownership), so they land root-owned and nginx can't
+# write them. NONROOT_OWNED_DIRS re-creates them in the nonroot raw layer with
+# correct ownership. Mirrors dhi.io/nginx (debian-13/stable.yaml): run, log,
+# cache and the default html dir.
+NONROOT_OWNED_DIRS = "\
+    /run/nginx \
+    ${localstatedir}/log/nginx \
+    ${localstatedir}/cache/nginx \
+    ${datadir}/nginx/html \
+"
+# Use CMD so `docker run image /bin/sh` works as expected
+OCI_IMAGE_CMD = ""
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+inherit container-volatile-fixup
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+OCI_IMAGE_ENTRYPOINT = "/usr/sbin/nginx"
+OCI_IMAGE_ENTRYPOINT_ARGS = "-g 'daemon off; error_log stderr notice;'"
+OCI_IMAGE_PORTS = "80/tcp"
+OCI_IMAGE_TAG = "latest"
+
+SKIP_RECIPE[app-container-nginx] ?= "${@bb.utils.contains('BBFILE_COLLECTIONS', 'webserver', '', 'Depends on meta-webserver which is not included', d)}"
-- 
2.54.0

[PATCH v2 10/13] vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd/nginx

[Tim Orling]

Allow us to build nginx multiarch containers.
If we choose to support apache2 we can add it in the future.

While it seems that BBFILE_PATTERN_IGNORE_EMTPY_webserver = "1"
in conf/distro/include/meta-virt-host.conf is no longer
needed, we will see warnings from vruntime-* multiconfigs if
we remove it.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 conf/distro/include/vcontainer-bbmask.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/conf/distro/include/vcontainer-bbmask.inc b/conf/distro/include/vcontainer-bbmask.inc
index f79ea9b9..0c9b8b02 100644
--- a/conf/distro/include/vcontainer-bbmask.inc
+++ b/conf/distro/include/vcontainer-bbmask.inc
@@ -93,6 +93,7 @@ BBMASK += "meta-networking/recipes-(?!filter|support|connectivity/mosquitto)"
 BBMASK += "meta-openstack/recipes-dbs/postgresql/"
 BBMASK += "meta-oe/dynamic-layers/networking-layer/recipes-core/"
 BBMASK += "meta-openstack/recipes-extended/libvirt/"
+BBMASK += "meta-webserver/recipes-(?!httpd/nginx)"
 
 # ---------------------------------------------------------------------------
 # Entire layers with 0 recipes in the container image dependency graph.
@@ -101,7 +102,6 @@ BBMASK += "meta-openstack/recipes-extended/libvirt/"
 # ---------------------------------------------------------------------------
 BBMASK += "meta-filesystems/"
 BBMASK += "meta-python/"
-BBMASK += "meta-webserver/"
 # Warning suppression for these fully-masked layers is in meta-virt-host.conf
 # (BBFILE_PATTERN_IGNORE_EMPTY) because BitBake checks the base datastore,
 # not per-multiconfig datastores.
-- 
2.54.0

[PATCH v2 11/13] app-container-curl: use multilayer mode; container-nonroot-user

[Tim Orling]

From: Tim Orling <ticotimo@gmail.com>

* Move to recipes-containers/images to show maintenance intent
* Switch to multilayer mode to more like the other "library"/"official"
  container recipes.
* Change OCI_IMAGE_TAG to "latest" for similar reasons.
* Change OCI_IMAGE_ENTRYPOINT_ARGS to "--help" to be more like upstream
  containers.
* Install ca-certificates to enable handling https:// sites
* Inherit container-nonroot-user to run as 'nonroot' with UID 65532 by
  default.
* Inherit container-volatile-fixup to fix 'log' and 'tmp' in
  /var/volatile.
* Inherit container-dev-mode to optionally build a "-dev" image.
  Set PACKAGECONFIG:pn-app-container-curl = "dev" in local.conf or
  distro/image config to run as `root` and include a CONTAINER_SHELL.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/app-container-curl.bb              | 39 ++++++++++++++++
 recipes-demo/images/app-container-curl.bb     | 46 -------------------
 2 files changed, 39 insertions(+), 46 deletions(-)
 create mode 100644 recipes-containers/images/app-container-curl.bb
 delete mode 100644 recipes-demo/images/app-container-curl.bb

diff --git a/recipes-containers/images/app-container-curl.bb b/recipes-containers/images/app-container-curl.bb
new file mode 100644
index 00000000..bfa3e708
--- /dev/null
+++ b/recipes-containers/images/app-container-curl.bb
@@ -0,0 +1,39 @@
+SUMMARY = "Curl Application container image"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+# Multi-layer mode: create explicit layers instead of single rootfs layer
+OCI_LAYER_MODE = "multi"
+
+# Optional 'dev' mode:
+#   - adds a shell to the container
+#   - runs the container as root (UID 0)
+# Enable with: PACKAGECONFIG:pn-app-container-curl = "dev"
+PACKAGECONFIG ??= ""
+PACKAGECONFIG[dev] = ""
+inherit container-dev-mode
+
+# Define layers: each layer contains specific packages
+# Format: "name:type:content" where content uses + as delimiter for multiple items
+OCI_LAYERS = "\
+    base:packages:base-files+base-passwd+netbase \
+    ${@bb.utils.contains('PACKAGECONFIG', 'dev', 'shell:packages:${CONTAINER_SHELL}', '', d)} \
+    curl:packages:curl+ca-certificates \
+"
+
+IMAGE_FSTYPES = "container oci"
+inherit image
+inherit image-oci
+inherit container-nonroot-user
+inherit container-volatile-fixup
+
+IMAGE_FEATURES = ""
+IMAGE_LINGUAS = ""
+NO_RECOMMENDATIONS = "1"
+
+# Allow build with or without a specific kernel
+IMAGE_CONTAINER_NO_DUMMY = "1"
+
+OCI_IMAGE_ENTRYPOINT = "curl"
+OCI_IMAGE_TAG = "latest"
+OCI_IMAGE_ENTRYPOINT_ARGS = "--help"
diff --git a/recipes-demo/images/app-container-curl.bb b/recipes-demo/images/app-container-curl.bb
deleted file mode 100644
index ddeb3022..00000000
--- a/recipes-demo/images/app-container-curl.bb
+++ /dev/null
@@ -1,46 +0,0 @@
-SUMMARY = "Curl Application container image"
-LICENSE = "MIT"
-LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
-
-IMAGE_FSTYPES = "container oci"
-inherit image
-inherit image-oci
-
-IMAGE_FEATURES = ""
-IMAGE_LINGUAS = ""
-NO_RECOMMENDATIONS = "1"
-
-IMAGE_INSTALL = " \
-       base-files \
-       base-passwd \
-       netbase \
-       ${CONTAINER_SHELL} \
-"
-
-# If the following is configured in local.conf (or the distro):
-#      PACKAGE_EXTRA_ARCHS:append = " container-dummy-provides"
-# 
-# it has been explicitly # indicated that we don't want or need a shell, so we'll
-# add the dummy provides.
-# 
-# This is required, since there are postinstall scripts in base-files and base-passwd
-# that reference /bin/sh and we'll get a rootfs error if there's no shell or no dummy
-# provider.
-CONTAINER_SHELL ?= "${@bb.utils.contains('PACKAGE_EXTRA_ARCHS', 'container-dummy-provides', 'container-dummy-provides', 'busybox', d)}"
-
-# Allow build with or without a specific kernel
-IMAGE_CONTAINER_NO_DUMMY = "1"
-
-# Workaround /var/volatile for now
-ROOTFS_POSTPROCESS_COMMAND += "rootfs_fixup_var_volatile ; "
-rootfs_fixup_var_volatile () {
-    install -m 1777 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/tmp
-    install -m 755 -d ${IMAGE_ROOTFS}/${localstatedir}/volatile/log
-}
-
-OCI_IMAGE_ENTRYPOINT = "curl"
-OCI_IMAGE_TAG = "easy"
-OCI_IMAGE_ENTRYPOINT_ARGS = "http://localhost:80"
-CONTAINER_SHELL = "busybox"
-
-IMAGE_INSTALL:append = " curl"
-- 
2.54.0

[PATCH v2 12/13] container-image-multiarch: add helper recipe

[Tim Orling]

Similar to core-image-ptest.bb, this recipe uses mcextend to add
multiarch support for any recipe in CONTAINER_IMAGES variable.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../images/container-image-multiarch.bb       | 31 +++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 recipes-containers/images/container-image-multiarch.bb

diff --git a/recipes-containers/images/container-image-multiarch.bb b/recipes-containers/images/container-image-multiarch.bb
new file mode 100644
index 00000000..13c4ae24
--- /dev/null
+++ b/recipes-containers/images/container-image-multiarch.bb
@@ -0,0 +1,31 @@
+inherit features_check
+REQUIRED_DISTRO_FEATURES = "vcontainer"
+
+
+DESCRIPTION += "Enable building the ${MCNAME} multiarch image."
+SUMMARY ?= "${MCNAME} multiarch image."
+HOMEPAGE ?= "https://www.yoctoproject.org/"
+LICENSE ?= "MIT"
+LIC_FILES_CHKSUM ?= "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+
+CONTAINER_IMAGES ?= "\
+    container-base \
+    app-container-curl \
+    app-container-mosquitto \
+    app-container-nginx \
+    app-container-python \
+    app-container-valkey \
+"
+
+inherit oci-multiarch
+
+BBCLASSEXTEND = "${@' '.join(['mcextend:'+x for x in d.getVar('CONTAINER_IMAGES').split()])}"
+
+OCI_MULTIARCH_RECIPE = "${MCNAME}"
+OCI_MULTIARCH_PLATFORMS = "aarch64 x86_64"
+
+python () {
+    if not d.getVar("MCNAME"):
+        raise bb.parse.SkipRecipe("No class extension set")
+}
-- 
2.54.0

[PATCH v2 13/13] vcontainer-tarball: fix buildbot-venv shadowing

[Tim Orling]

Prepend the SDK dirs (vdkr/vpdmn wrappers + native qemu) but keep \$PATH
ahead of the host /usr/bin:/bin. The CI parser sources this into the
push-containers step where bitbake also runs, and the worker's buildbot-venv
is activated via \$PATH; putting /usr/bin first shadowed the venv python and
made bitbake pick up the host websockets 9.1 (< 10) -> hashserv failure.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 recipes-containers/vcontainer/vcontainer-tarball.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/recipes-containers/vcontainer/vcontainer-tarball.bb b/recipes-containers/vcontainer/vcontainer-tarball.bb
index f943cfd0..522cb0ba 100644
--- a/recipes-containers/vcontainer/vcontainer-tarball.bb
+++ b/recipes-containers/vcontainer/vcontainer-tarball.bb
@@ -450,8 +450,8 @@ ENVEOF
 # SDK relocation rewrites these paths at install time.
 export VCONTAINER_DIR="${SDKPATH}"
 export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"
-export PATH="${SDKPATH}:${SDKPATHNATIVE}/usr/bin:/usr/bin:/bin:\$PATH"
-# Clean up - unset to avoid confusing other Yocto tools' >> $script
+export PATH="${SDKPATH}:${SDKPATHNATIVE}/usr/bin:\$PATH"
+# Clean up - unset to avoid confusing other Yocto tools'
 unset OECORE_NATIVE_SYSROOT
 CISCRIPT
     chmod 755 $ci_script
-- 
2.54.0

Re: [meta-virtualization] [PATCH v2 00/13] Container Improvements

[Bruce Ashfield]

Thanks Tim,

I'm out of the office for the next 1.5 weeks. so it will likely be
after that before I look into this in any detail.

Bruce


On Mon, Jul 6, 2026 at 1:13 PM Tim Orling via lists.yoctoproject.org
<ticotimo=gmail.com@lists.yoctoproject.org> wrote:
>
> This series:
>   * Sets OCI_IMAGE_RUNTIME_UID with ??= weak assignment in
>     image-oci to allow container-nonroot-user to set a different
>     default
>   * Adds a class to create/run containers with a non-root user
>   * Adds a class to create non-volatile var log and tmp directories
>   * Adds a class to enable 'dev' mode where the container starts as
>     the root user
>   * Adds new containers:
>     - app-container-python
>     - app-containter-mosquitto
>     - app-container-valkey
>     - app-container-nginx
>   * Modifies app-container-curl to be more like the upstream
>     experience (and more like the above containers)
>   * Allows mosquitto and libwebsockets to be parsed for
>     vcontainer distro so we can build multiarch containers
>     for app-container-mosquitto.
>   * Allows meta-webserver/recipes-httpd/nginx to be parsed for
>     vcontainer distro so we can build multiarch containers
>     for app-container-nginx.
>   * Adds a fix for buildbot-venv shadowing for environment-setup-ci
>     script
>
> The resulting containers were tested with simple command line
> usage compared to Docker provided equivalents to ensure the
> same expected behavior.
>
> Changes in v2:
>
>   There were a fair number of changes in v2 in reponse to review
>   comments and observations during testing.
>
>   * image-oci.bbclass:
>     - set OCI_IMAGE_RUNTIME_UID with ??= to allow other classes
>       to override
>   * container-nonroot-user.bbclass:
>     - set EXTRA_USERS_PARAMS with +=
>     - added comments to clarify the PYEOF Python heredoc behavior
>     - added logic to detect $image_name/index.json and fail if not
>       found
>   * container-volatile-fixup.bbclass: avoid repeating the same
>     rootfs postfunc by adding a helper class
>   * container-dev-mode.bbclass: avoid repeating the same logic
>     to run container as root user in '-dev' mode
>   * app-container-python (and other container image recipes):
>     - drop now redundant IMAGE_INSTALL lines
>     - simplify by inheriting container-volatile-fixup
>     - inherit container-dev-mode
>   * app-container-mosquitto:
>     - verify '-dev' mode switches from root user to mosquitto user
>     - verify "production" mode runs as 'nonroot' user
>     - fix bbmask to allow mosquitto and required libwebsockets
>   * app-container-valkey:
>     - added tini and container-entrypoint.sh, like upstream containers
>   * app-container-nginx:
>     - fixup 'nonroot' user ownership of NONROOT_OWNED_DIRS
>     - fix bbmask to allow nginx (but not other recipes-httpd)
>   * fix for older distro hosts where the 'websockets' Python library is
>     not new enough (>10.0) and on the AutoBuilder needs to use 'buildtools'.
>     The PATH in environment-setup-ci now plays properly with such a set up
>     by no longer added /usr/bin and /bin before existing $PATH.
>
> Successful build https://autobuilder.yoctoproject.org/valkyrie/#/builders/117/builds/25
>
> Tim Orling (13):
>   image-oci: set OCI_IMAGE_RUNTIME_UID with ??=
>   classes: add container-nonroot-user.bbclass
>   classes: add container-volatile-fixup.bbclass
>   classes: add container-dev-mode.bbclass
>   recipes-containers/images: add app-container-python
>   recipes-containers/images: add app-container-mosquitto
>   vcontainer-bbmask.inc: allow 'mosquitto', 'libwebsockets'
>   recipes-containers/images: add app-container-valkey
>   recipes-containers/images: add app-container-nginx
>   vcontainer-bbmask.inc: allow meta-webserver/recipes-httpd/nginx
>   app-container-curl: use multilayer mode; container-nonroot-user
>   container-image-multiarch: add helper recipe
>   vcontainer-tarball: fix buildbot-venv shadowing
>
>  classes/container-dev-mode.bbclass            |  34 ++++
>  classes/container-nonroot-user.bbclass        | 155 ++++++++++++++++++
>  classes/container-volatile-fixup.bbclass      |   6 +
>  classes/image-oci.bbclass                     |   3 +-
>  conf/distro/include/vcontainer-bbmask.inc     |   6 +-
>  conf/layer.conf                               |   1 +
>  .../images/app-container-curl.bb              |  39 +++++
>  .../images/app-container-mosquitto.bb         |  43 +++++
>  .../images/app-container-nginx.bb             |  62 +++++++
>  .../images/app-container-python.bb            |  45 +++++
>  .../images/app-container-valkey.bb            |  71 ++++++++
>  .../container-entrypoint.sh                   |  18 ++
>  .../images/container-image-multiarch.bb       |  31 ++++
>  .../vcontainer/vcontainer-tarball.bb          |   4 +-
>  recipes-demo/images/app-container-curl.bb     |  46 ------
>  15 files changed, 512 insertions(+), 52 deletions(-)
>  create mode 100644 classes/container-dev-mode.bbclass
>  create mode 100644 classes/container-nonroot-user.bbclass
>  create mode 100644 classes/container-volatile-fixup.bbclass
>  create mode 100644 recipes-containers/images/app-container-curl.bb
>  create mode 100644 recipes-containers/images/app-container-mosquitto.bb
>  create mode 100644 recipes-containers/images/app-container-nginx.bb
>  create mode 100644 recipes-containers/images/app-container-python.bb
>  create mode 100644 recipes-containers/images/app-container-valkey.bb
>  create mode 100644 recipes-containers/images/app-container-valkey/container-entrypoint.sh
>  create mode 100644 recipes-containers/images/container-image-multiarch.bb
>  delete mode 100644 recipes-demo/images/app-container-curl.bb
>
> This series can be found at
> https://github.com/moto-timo/meta-virtualization/tree/container-improvements-v2
>
> --
> 2.54.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#9924): https://lists.yoctoproject.org/g/meta-virtualization/message/9924
> Mute This Topic: https://lists.yoctoproject.org/mt/120143346/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] [PATCH v2 00/13] Container Improvements

[Bruce Ashfield]

Hi Tim,

Thanks for the substantial rework. Reviewed the whole series against the
v1 comments, and the two new pieces you factored out (container-volatile-fixup
and container-dev-mode) look great. Series has been applied on master.

Here's my summary, just so we can have it archived and use it as a reference
later if something breaks:

v1 items — status
-----------------

Addressed:

  * EXTRA_USERS_PARAMS uses += now (was = in v1)
  * PYEOF single-quoted heredoc has the "bitbake pre-expansion" comment
  * NONROOT_USER bare-identifier requirement documented
  * fix_oci_home_perms has the index.json guard + OCI_IMAGE_TAR_OUTPUT
    gate + bbfatal if the layout ever changes
  * IMAGE_INSTALL is no longer duplicated against OCI_LAYERS packages
    across the image recipes (image-oci auto-derivation on master
    handles it)
  * rootfs_fixup_var_volatile is factored into
    container-volatile-fixup.bbclass and app-container-python inherits
    it (was missing in v1)
  * container-dev-mode.bbclass cleans up the '-dev' boilerplate that
    each recipe previously repeated
  * app-container-valkey's persistence paths (/data, /var/lib/valkey,
    /var/log/valkey, /run/valkey) are in NONROOT_OWNED_DIRS
  * The valkey security-caveats item is moot in v2 — the
    container-entrypoint.sh design supersedes the always-on
    protected-mode override I had asked to be flagged in DESCRIPTION

I did notice a few things on the way through, but it was more efficient
if I just did them myself.

All three are behavior-neutral cleanups:

1. app-container-valkey: drop redundant OCI_IMAGE_RUNTIME_UID

   The recipe carries a verbatim copy of the same
     OCI_IMAGE_RUNTIME_UID = "${@bb.utils.contains('PACKAGECONFIG',
                                'dev', '0', '${NONROOT_UID}', d)}"
   line that container-dev-mode.bbclass now supplies. Leftover from v1.

2. app-container-mosquitto: fix PACKAGECONFIG 'dev' comment

   The block above PACKAGECONFIG was copied from app-container-python
   and mentions python3-pip / site-packages /
   pn-app-container-python — none of which apply. Rewrote it per-recipe.
   nginx and curl in the same series already have the correct wording.

3. container-nonroot-user.bbclass: document + assert inherit order

   oci_nonroot_inject_user() is a do_image_oci prefunc that reads
   OCI_LAYER_*_ROOTFS. Those vars are populated by image-oci's OWN
   do_image_oci prefunc (oci_multilayer_install_packages), and prefunc
   execution follows inherit order. If a future recipe inherits
   container-nonroot-user before image-oci, injection runs against
   empty per-layer state, no layer ships /etc/passwd, and the only
   signal is a bb.warn that reviewers can miss. Added a header comment
   spelling out the requirement + an anonymous-python guard that
   raises bb.fatal at parse time. All five in-tree consumers already
   inherit in the right order, so this catches a future mistake rather
   than fixing a current one.

Also — the cover letter says "verify '-dev' mode switches from root
user to mosquitto user", but the recipe doesn't set
NONROOT_USER = "mosquitto" (unlike nginx which does set it to "nginx").
So either the cover letter is aspirational or something is happening
implicitly that I'm missing. Any changes we can do on top of the
series. I've put it on master-next for visibility.

Bruce