Re: [RFC] block: add an overlaybd image format driver

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 5321 bytes --]

On Tue, Sep 01, 2026 at 11:09:56AM +0800, Huiba Li wrote:
> Hi all,
> 
> Gentle ping on this RFC from Aug 18.

Hi Huiba,
Sorry your RFC was not seen. Kevin is on vacation and I will take a look
in the meantime. Please CC stefanha@redhat.com in future emails so it
comes to my attention.

Thanks!

Stefan

> 
> In short: overlaybd is an openly specified image format (CNCF-hosted;
> read-only blob stack + writable top, O(1) lookup regardless of chain
> depth) deployed at scale for container/VM image streaming. The RFC
> proposes an in-tree C block driver, starting with read-only support
> (index merge at open, raw + ZFile layers), writes handled by a
> qcow2/raw file on top.
> 
> I'd especially appreciate thoughts on the four questions in the RFC:
> 
>   1. Is a new in-tree format driver acceptable in principle?
>   2. Is read-only-first a reasonable initial scope (with iotests and a
>      docs/interop spec page)?
>   3. Remote blobs: compose over block/curl, or fetch in-driver?
>   4. CLI surface: point -drive at the OCI-style JSON config, or
>      per-layer options?
> 
> Question 1 is the main gate -- if in-principle feedback is easier to
> give than a full design review, that alone would help me decide
> whether to proceed with the first series.
> 
> Thanks,
> Huiba
> 
> 
> Huiba Li <lihuiba@gmail.com> 于2026年8月18日周二 17:38写道：
> >
> > Hi,
> >
> > I would like to propose adding a QEMU block driver for the
> > overlaybd image format, and I am looking for early feedback on
> > scope and design before sending patches.
> >
> > What overlaybd is
> > -----------------
> > Overlaybd is an open-source image format (https://containerd.github.io/overlaybd/)
> > hosted as a sub-project of containerd (CNCF graduated). A disk
> > image is a stack of read-only snapshot blobs plus a writable top
> > file, in a layout compatible with OCI image registries: a base
> > image is stored once and shared by every derivative image, and
> > data is fetched on demand rather than pulled in full before boot.
> >
> > Two properties distinguish it from qcow2/VHDX/VMDK-style chains:
> >
> >   * Lookup cost is O(1) in chain depth. At open time the per-file
> >     indices are merged into a single in-memory index (an
> >     extent-based log-structured merge tree), so a read never walks
> >     backing files, and per-file index caches do not multiply with
> >     snapshots.
> >
> >   * The merged index is small (about 300 KB average for 50+ GB
> >     images in our production data), and because extents are
> >     variable-length, writes land at 512-byte granularity with no
> >     cluster copy-on-write.
> >
> > The format is openly specified:
> >   https://github.com/containerd/overlaybd/tree/main/docs/specs/lsmt.md
> >   https://github.com/containerd/overlaybd/tree/main/docs/specs/zfile.md
> >
> > Reference implementation:
> >   https://github.com/containerd/overlaybd
> >
> > Overlaybd has been in production at Alibaba for years (Taobao,
> > Tmall, Alibaba Cloud, Function Compute) and is used by Azure AKS
> > Artifact Streaming, Databricks serverless compute, and several
> > microVM-based sandbox platforms. The design is documented in two
> > USENIX ATC papers (DADI, ATC'20; FaaSNet, ATC'21). I am one of the
> > overlaybd/DADI authors.
> >
> > Why a QEMU driver
> > -----------------
> > Today overlaybd images are served to VMs through a kernel blk dev
> > and a userspace daemon. That fits container workloads well, but
> > a native QEMU driver would let any QEMU-based stack (libvirt,
> > KubeVirt, OpenStack, plain qemu-system-*) boot overlaybd images
> > directly, with QEMU's usual amenities (backing files, block jobs,
> > throttling, migration) and no extra daemon on the host. This is
> > beneficial to agent sandboxes.
> >
> > Proposed implementation
> > -----------------------
> > A new block format driver, block/overlaybd.c, written in C from the
> > open spec (like the existing vmdk/vhdx drivers) rather than binding
> > the C++ reference implementation:
> >
> >   * v1: read-only support. Local layer files, index merge at open,
> >     raw and ZFile blobs (lz4/zstd are already optional QEMU
> >     dependencies). Writes can be served by a qcow2/raw file on top,
> >     exactly like booting a read-only backing image today.
> >   * v2: native writable layer, and lazy fetching of remote blobs —
> >     either composed over the existing curl driver or with a small
> >     built-in HTTP fetcher; guidance welcome.
> >
> > Questions
> > ---------
> >   1. Is a new in-tree format driver acceptable in principle, given
> >      the format is stable, openly specified, and deployed at scale?
> >   2. Is read-only-first a reasonable merge scope for the initial
> >      series (with iotests and a docs/interop/ spec page)?
> >   3. For remote blobs: compose over block/curl, or fetch in-driver
> >      with a local cache?
> >   4. CLI surface: point -drive at the OCI-style JSON image config
> >      (which lists the layer blobs), or expose per-layer options?
> >
> > I will write the iotests and the docs/interop format page, and add
> > myself to MAINTAINERS in the first series.
> >
> > Thanks for any comments,
> > Huiba Li
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]