Re: [RFC PATCH 00/12] virtio: support devices that own their virtqueue memory

["Graf (AWS), Alexander" <graf@amazon.de>]

On 10.08.26 10:04, Michael S. Tsirkin wrote:
> On Mon, Aug 10, 2026 at 07:39:02AM +0000, Graf (AWS), Alexander wrote:
>> Hey Michael,
>>
>> Thanks a bunch for having a detailed and quick look!
>>
>> On 10.08.26 08:23, Michael S. Tsirkin wrote:
>>> On Sun, Aug 09, 2026 at 06:19:58PM +0000, Alexander Graf wrote:
>>>> Virtio drivers use guest memory to back virtqueues and their buffers.
>>>> That means a VMM needs to be able to map guest memory. That is ok in the
>>>> normal virt case. It gets icky with confidential computing (where we use
>>>> swiotlb as workaround) and it defeats the purpose of isolated vhost-user
>>>> backing devices, because they end up with full RAM access to the guest.
>>>>
>>>> So instead, I'm proposing an extension to virtio which allows it to give
>>>> each virtio device its own dedicated memory region to communicate with the
>>>> host, called DMB (Device Memory Buffer). A trusted hypervisor can force
>>>> DMB to be present, which then enables safer, more isolated and resilient
>>>> communication between guest and host.
>>>>
>>>> With DMB, the device provides a shared memory region that both parties
>>>> agree is the full memory map both have access to. All memory offsets
>>>> that previously would have been into guest RAM, are then offsets into
>>>> this shared memory buffer region. One nice property of this is that it
>>>> is a generic mechanism in the virtio transport layer, so higher level
>>>> drivers work unmodified.
>>>>
>>>> I was exploring to use swiotlb instead to create individual pools. But
>>>> that approach has multiple downsides:
>>>>
>>>> 1. Swiotlb is an OS primitive which is not available in all Operating
>>>> Systems. DMB however lives in the virtio transport layer, which means we
>>>> can add support for it in any OS independent of generic layers. This
>>>> helps with Windows support.
>>> How does it help, if you are going to put a pool in
>>> the driver, put a pool in the driver. Maybe with virtio mem to
>>> simplify allocation.
>>
>> I'm not sure I understand the suggestion :)
>>
>
> I'm not sure what the problem is for windows :)
>
> But if you want a chunk of contiguos memory that windows
> does not poke at without a driver, virtio mem is that :)
>
>>>> 2. We munge DMA space together. DMB provides a separate DMA space per
>>>> virtio device. This means we can for example implement a device in
>>>> vhost-user and give the implementing process only visibility to the DMB
>>>> region, not all of guest memory. That reduces the exposure the
>>>> vhost-user provider has, improving security.
>>>>
>>>> 3. Devices can opt-in. A hypervisor can choose to use standard virtio
>>>> semantics for self-implemented devices (e.g. NSM), while requiring DMB
>>>> for devices implemented by less trustworthy providers. The
>>>> non-trustworthy devices do not get any visibility into the trustworthy
>>>> ones, even with DMB in place for both.
>>> So I am not sure whether the implication is that it's purely a software
>>> construct. But if it is, can we extend virtio iommu to
>>> add a way to discover and enforce trust boundaries?
>>> And maybe translate offsets to BARs, if that is desired?
>>>
>>> It seems to be that the result would be that we don't need fiddly
>>> special casing in virtio ring specifically, and a lot of things like
>>> pre-mapped dma will begin to work.
>>
>> On thing I'm trying to avoid is dynamicity. Anything that dynamically
>> changes visibility or needs state tracking is something that can go
>> wrong. By keeping everything self-contained within the guest, I can
>> reason about what is visible and what is not easily. Especially for
>> confidential computing, IMHO static wins over dynamic in general.
>>
>> Or did I misunderstand your suggestion?
>
> I get this part. But we can absolutely make it static.
>
>
> Let's start by replicating your functionality with virtio-iommu.  So:
>
> -device is behind virtio-iommu
> -virtio-iommu tells guest "this device can only consume memory from that range,"
> -and maybe: ...and translation is 1:1 with this offset
>
> if guest does not acknowledge it will just fail?
>
>
> this covers the proposal here simply by using a dedicated range
> per device, right?
>
>
>
> But look what we can easily add later: share a range
> between devices, now you can move data between them with zero copies.
>
>
> Isn't that better?


I don't know if "Add device emulation for virtio-mem and virtio-iommu, 
including all of its state machine and tracking" is necessarily an 
improvement, but let me try and prototype it.


>
>
>> As to pure software construct: With CXL, you can implement the exact
>> same protocol on real hardware as well. All it takes is cache coherency
>> of the BAR (or whatever the transport uses) region.
>>
>>
>>
>> Alex
> Yes this is what I thought originally, that you intend to
> do it in hardware. But re-reading it, it begins to look like
> that's not really the case, it's only "theoretically possible"?


Today I only have backends in software. I think it would be a pretty 
cool feature even for actual hardware or for 
not-as-obvious-but-still-cache-coherent-software such as EL3/SMM. I 
don't have concrete plans for them, but I like generic building blocks 
over too targeted and tied to a single use case :)


Alex