Re: [PATCH v3] Add VIRTIO_F_DMB (Device Memory Buffer)

["Michael S. Tsirkin" <mst@redhat.com>]

On Sun, Aug 09, 2026 at 10:09:15PM -0400, Demi Marie Obenour wrote:
> On 8/9/26 17:48, Michael S. Tsirkin wrote:
> > On Tue, Aug 04, 2026 at 04:12:02PM +0000, Alexander Graf wrote:
> >> When a transport uses a shared bounce region for virtqueue memory today,
> >> that region is global and shared by multiple devices at once. Devices
> >> then contend over one small region, so a busy device can starve the
> >> others, and one device's emulation cannot be isolated or torn down on its
> >> own because its state lives in memory everyone else is using too.
> >>
> >> VIRTIO_F_DMB gives each device its own shared memory region, the Device
> >> Memory Buffer, that holds that device's virtqueues (Descriptor, Driver
> >> and Device Areas) and the buffers the descriptors reference. When the
> >> feature is negotiated, every address the driver places in a virtqueue is
> >> an offset into that region rather than a physical or bus address, so the
> >> device only ever touches memory the driver placed there. Because each
> >> device owns its region, its emulation can run in a separate host process
> >> and be terminated independently, and devices no longer contend over one
> >> shared region.
> >>
> >> The device reports the region's shmid through a transport-specific
> >> mechanism (dmb_shm_id in the PCI common configuration structure, DMBSHMId
> >> on MMIO), so the shmid does not collide with device-specific or future
> >> transport-specific shared memory regions. The driver locates the region
> >> through the existing Shared Memory Regions facility. VIRTIO_F_DMB refines
> >> VIRTIO_F_ACCESS_PLATFORM and is negotiated only together with it.
> >>
> >> To provide the confinement guarantee, the device bounds-checks every
> >> driver-supplied offset against the region and sets DEVICE_NEEDS_RESET on
> >> violation.
> >>
> >> The general requirement on shared memory regions forbids a device to
> >> expose a region used to control its operation or to stream data, and a
> >> Device Memory Buffer is used for both, so it is named as that
> >> requirement's exception.
> >>
> >> The Device Memory Buffer is memory the driver shares with the device on
> >> platforms that distinguish shared from private mappings, and the device
> >> provides coherency without cache maintenance by the driver. A device
> >> that requires the platform's barriers offers VIRTIO_F_ORDER_PLATFORM,
> >> and a driver accepts VIRTIO_F_DMB only together with it. Offset 0 in
> >> the region is reserved: a device that predates the feature reads a
> >> Descriptor Area address of 0 as a queue that is not in use, as the
> >> legacy QueuePFN register states.
> >>
> >> Signed-off-by: Alexander Graf <graf@amazon.com>
> >> ---
> >>
> >> Link: https://lore.kernel.org/virtio-comment/20260702040006.65669-1-graf@amazon.com/ [v1]
> >> Link: https://lore.kernel.org/virtio-comment/20260716215426.46037-1-graf@amazon.com/ [v2]
> >>
> >> v1 -> v2:
> >>  - Replace the fixed reserved shmid 2 and its registry with a shmid the
> >>    device reports per transport, as Stefan Hajnoczi suggested on v1.
> >>  - Spell out per-transport compatibility for dmb_shm_id and DMBSHMId.
> >>  - Set FAILED when the driver cannot find the region, rather than refuse
> >>    to negotiate.
> >>  - List VIRTIO_F_DMB under CCW's features reserved for future use.
> >>
> >> v2 -> v3:
> >>  - Require the region to be memory the driver shares with the device.
> >>  - Require coherency; couple VIRTIO_F_DMB to VIRTIO_F_ORDER_PLATFORM,
> >>    replacing v2's sentence on the areas' interaction with it.
> >>  - Reserve offset 0 in the region.
> >>  - Except the Device Memory Buffer from the shared-region prohibition.
> >>  - Allow a device to expose further shared memory regions.
> >>  - Move bit 44 into the queue and feature negotiation reserved list.
> >>  - Aim two cross-references at the Reserved Feature Bits chapter.
> >>
> >>
> >>  conformance.tex    |   2 +
> >>  content.tex        |  28 ++++++-
> >>  shared-mem.tex     | 199 ++++++++++++++++++++++++++++++++++++++++++++-
> >>  transport-ccw.tex  |   1 +
> >>  transport-mmio.tex |  15 ++++
> >>  transport-pci.tex  |  20 +++++
> >>  6 files changed, 262 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/conformance.tex b/conformance.tex
> >> index 9af31e2..cf4259d 100644
> >> --- a/conformance.tex
> >> +++ b/conformance.tex
> >> @@ -104,6 +104,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >>  \item \ref{drivernormative:General Initialization And Device Operation / Device Initialization}
> >>  \item \ref{drivernormative:General Initialization And Device Operation / Device Cleanup}
> >>  \item \ref{drivernormative:Reserved Feature Bits}
> >> +\item \ref{drivernormative:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}
> >>  \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities}
> >>  \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
> >>  \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device parts}
> >> @@ -191,6 +192,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >>  \item \ref{devicenormative:Basic Facilities of a Virtio Device / Packed Virtqueues / The Virtqueue Descriptor Table}
> >>  \item \ref{devicenormative:Basic Facilities of a Virtio Device / Packed Virtqueues / Scatter-Gather Support}
> >>  \item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions}
> >> +\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}
> >>  \item \ref{devicenormative:Reserved Feature Bits}
> >>  \item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities}
> >>  \item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects}
> >> diff --git a/content.tex b/content.tex
> >> index 243ce2d..8c5a844 100644
> >> --- a/content.tex
> >> +++ b/content.tex
> >> @@ -102,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B
> >>  \begin{description}
> >>  \item[0 to 23, 41, 42 and 50 to 127] Feature bits for the specific device type
> >>  
> >> -\item[24 to 40, and 43] Feature bits reserved for extensions to the queue and
> >> +\item[24 to 40, 43 and 44] Feature bits reserved for extensions to the queue and
> >>    feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits}
> >>  
> >> -\item[44 to 49, and 128 and above] Feature bits reserved for future extensions.
> >> +\item[45 to 49, and 128 and above] Feature bits reserved for future extensions.
> >>  \end{description}
> >>  
> >>  \begin{note}
> >> @@ -946,6 +946,15 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
> >>     suspend the device by set the SUSPEND bit to 1.
> >>     See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}.
> >>  
> >> +  \item[VIRTIO_F_DMB(44)] This feature indicates that the device offers a
> >> +  Device Memory Buffer: a shared memory region, owned by the device,
> >> +  that holds the virtqueues and the buffers they reference.  When this feature
> >> +  is negotiated, every address that the driver places in a virtqueue is an
> >> +  offset into the Device Memory Buffer region rather than a physical address
> >> +  or a bus address.  The device is thereby confined to memory the driver has
> >> +  published in that region.
> >> +  See \ref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}~\nameref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}.
> >> +
> >>  \end{description}
> >>  
> >>  \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
> >> @@ -982,6 +991,13 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
> >>  
> >>  A driver SHOULD accept VIRTIO_F_NOTIF_CONFIG_DATA if it is offered.
> >>  
> >> +A driver MUST NOT accept VIRTIO_F_DMB unless it also accepts
> >> +VIRTIO_F_ACCESS_PLATFORM.  If VIRTIO_F_ORDER_PLATFORM is offered, a driver
> >> +MUST NOT accept VIRTIO_F_DMB unless it also accepts VIRTIO_F_ORDER_PLATFORM.
> > 
> > Why is that? access platform is for DMA from device of which
> > there's none here. Both it and order platform are useful to disable
> > to speed up software configs.
> 
> I agree: combining VIRTIO_F_DMB and VIRTIO_F_ACCESS_PLATFORM shouldn't
> be combined.  I'd treat the combination as an error.

Nope. Just let the decision to devices. Again, there needs to be a
reason for such restrictions.  In fact, access platform is mostly "dma
is limited by a viommu". Since with dmb does not need a viommu to be
secure, it is in fact reasonable to have a device that offers both as
ways to be secure, accepts either VIRTIO_F_DMB or
VIRTIO_F_ACCESS_PLATFORM but fails if none negotiated.


> >> +For the driver requirements that apply once VIRTIO_F_DMB has been
> >> +negotiated, see
> >> +\ref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}~\nameref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}.
> >> +
> >>  \devicenormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
> >>  
> >>  A device MUST offer VIRTIO_F_VERSION_1.  A device MAY fail to operate further
> >> @@ -1011,6 +1027,14 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
> >>  and presents a PCI SR-IOV capability structure, otherwise
> >>  it MUST NOT offer VIRTIO_F_SR_IOV.
> >>  
> >> +A device that offers VIRTIO_F_DMB MUST also offer VIRTIO_F_ACCESS_PLATFORM.
> >> +A device that offers VIRTIO_F_DMB MUST also offer VIRTIO_F_ORDER_PLATFORM if
> >> +the device requires the driver to use memory barriers suitable for devices
> >> +described by the platform when accessing the Device Memory Buffer region.
> > 
> > This is really vague. On most platforms drivers can map
> > in any way they want. What is expected here? What barriers to
> > use will very much depend on that.
> > 
> > In particular, how do you expect this mapped? If cached what forces
> > the data out of the CPU cache?
> 
> My understanding is that VIRTIO_F_DMB is only a good choice for
> untrusted devices implemented in software.  Both the original use-case
> (protected VMs on Android) and my use-case (Spectrum VMs using untrusted
> virtio-vhost-user devices) fall into this category.  In this case,
> the DMB is just ordinary RAM, so VIRTIO_F_ORDER_PLATFORM isn't needed.
> 
> On the other hand, hardware devices shouldn't implement VIRTIO_F_DMB.
> This would require the CPU to read data over PCIe, which is slow.

The device can be on a quick bus with cache snooping.


> Instead, the driver should use a bounce buffer in system RAM, which
> provides the same level of security (assuming hardware DMA protection
> of some sort).

This is none of spec's business.

> I'd be fine with making VIRTIO_F_DMB and VIRTIO_F_ORDER_PLATFORM
> mutually exclusive.  Forbid devices from offering both, and require
> drivers to refuse to use a device that does.  Reconsider once a
> sensible use-case appears.


We should not deal with policy and use-cases. Just provide building
blocks.

> -- 
> Sincerely,
> Demi Marie Obenour (she/her/hers)