Re: [meta-virtualization][PATCH 1/2] image-oci: add OCI runtime bundle output

[Bruce Ashfield <bruce.ashfield@gmail.com>]

Hi Tanguy,

I don't want to take this as it stands, and I want to be clear about
why. It isn't the mechanics of the change, it's the architectural
one.

But this is a good topic to discuss more, since there may be another
way to get what you ar elooking for.

Sorry for the long reply, this is something I'm currently working
through as part of my OSSEU presentation (and next release planning)
that I'm doing this week.

image-oci exists to build OCI images. If a use case wants an
OCI-targeted container, the "sanctioned" way is a real image recipe
with the appropriate image type set (IMAGE_FSTYPES = "oci") and the
OCI_IMAGE_* variables expressing what that container is. The recipe
is where the container's identity lives and where it can be reasoned
about, audited, patched, and rebuilt.

I'll risk saying the obvious, since this will go in the list archive
.. and it is worth being clear about what the two outputs actually
are. The existing -oci-dir[.tar] artifact is an OCI Image Layout
(blobs + manifests). The patch's new .oci-bundle.tar is an OCI Runtime
Bundle (flat rootfs + config.json). Same container in two spec
representations, aimed at different consumers. Image Layout for
interchange (registries, skopeo, cross-arch manifests), Runtime Bundle
for direct exec by runc/crun. `umoci unpack --image <layout>:tag
<bundle-dir>` converts one to the other in a step. So the patch's
output is a strictly-lossier view (no layer sharing, no multi-arch, no
registry push) of the artifact we already ship, produced at build time
in a way the deploy side could have produced itself.

I can see some value in emitting the bundle directly. But that piece
is trivially scriptable downstream from an OCI container that's
already built: one `umoci unpack` at deploy time gets you there, 
and doesn't have to be baked into image-oci as a build-time output
to be reachable.

The part I object to more strongly is the modification-to-make-it-
runnable half. OCI_BUNDLE_POSTPROCESS_COMMAND (jq-mutate config.json,
or drop config.json in from SRC_URI) is hidden logic that mutates
container output bytes after the recipe has finished expressing what
the container is. That can't be a first-class output of the layer.
First-class means the recipe/variables describe the container end-to-
end, and someone reading the recipe can tell what will land in
config.json without also reading a post-hook shell function that
patches it afterward. Layered together with the 2/2 pattern
("require app-container.bb; OCI_IMAGE_BUNDLE_OUTPUT = 1"), it's a
way to define effectively a new container by appending post-hooks
onto an unrelated image recipe. Every image-oci build silently
becomes two products with the second one shaped by hidden logic.
That's the side door.

One clarification on the code comment you cite as prior
encouragement:

    "We could make this optional, since the bundle is directly
     runnable via runc"

That was less about promoting the bundle as a first-class shippable
artifact and more a note about reuse/sharing between image builds --
"we could keep it around instead of rebuilding it every time" reuse
angle. Reading it as a plan for exposing bundles as standalone-
runnable outputs via crun/runc goes further than intended.

That's the reasoning, now to the path to hopefully a solution ....

One question that would help me understand the motivation of what
you're after: is the goal here to produce multiple containers from a
single image construction, and the bundle+postprocess pair is the
plumbing you found that gets close to that? If so, the current patch
is still 1-image-to-1-bundle so it doesn't actually get you there
either, and that's a different discussion worth having on its own.

If the goal is just "1 image, 1 bundle, but with config.json tweaks
the current OCI_IMAGE_* variables can't express", then let me know the
missing knobs and we can look at adding them as first-class variables
in image-oci (the recipe-level path), rather than as a post-hook that
sits outside the image-config machinery.

If there's something clunky or otherwise not working in the image-oci
output, then that is also something to look into fixing. But if we
still require a recipe -> produce one output, then I'm missing why
we need a change like this.

Cheers,

Bruce