Re: [PATCH v4 13/13] RFC: hw/virtio: start virtio-mem guest_memfd regions as shared

[Peter Xu <peterx@redhat.com>]

On Mon, May 04, 2026 at 04:30:19PM +0400, Marc-André Lureau wrote:
> In TDX guests, virtio-mem plug/unplug/re-plug fails because
> kvm_set_phys_mem() unconditionally sets KVM memory attributes to
> PRIVATE for all guest_memfd regions. On re-plug, the PRIVATE->PRIVATE
> transition is a no-op, so KVM doesn't re-AUG pages and the guest's
> TDG.MEM.PAGE.ACCEPT fails.

Know little on TDX, please bare with me..

I saw KVM does a seamcall to ADD or AUG whenever a new EPT pte is set, via
this path:

        __tdp_mmu_set_spte_atomic
        set_external_spte_present
        tdx_sept_set_private_spte <------

On unplug, I'm expecting with your prior patches, gmem pages will be
truncated properly, so they'll be all gone.

Then, qemu does replug -> guest gets that event, start access page -> EPT
violation, KVM resolving page fault with __tdp_mmu_set_spte_atomic() (per
above) and a new page -> triggering AUG (not ADD, since it's post-boot).

Could you elaborate here why AUG is missing in the first place?

Thanks,

> 
> Implement the "start-shared" approach: virtio-mem memory starts with
> shared KVM attributes. The guest converts shared->private on plug (via
> set_memory_encrypted -> MapGPA + ACCEPT), and back to shared on unplug
> (via set_memory_decrypted). This ensures every plug triggers a real
> SHARED->PRIVATE transition, causing KVM to AUG fresh pages.
> 
> Add RAM_GUEST_MEMFD_START_SHARED flag and set it during virtio-mem
> realize for guest_memfd-backed regions. Use
> ram_block_attributes_state_change() to properly update the attributes
> bitmap through the API. Skip setting PRIVATE in kvm_set_phys_mem()
> when the flag is set. On unplug, explicitly reset KVM attributes to
> shared on the host side to handle the case where the guest skips
> set_memory_decrypted().
> 
> See also virtio-comment "[PATCH RFC] virtio-mem: add shared/private memory property details".
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  include/system/memory.h |  6 ++++++
>  accel/kvm/kvm-all.c     |  3 ++-
>  hw/virtio/virtio-mem.c  | 27 ++++++++++++++++++++++++++-
>  3 files changed, 34 insertions(+), 2 deletions(-)
> 
> diff --git a/include/system/memory.h b/include/system/memory.h
> index 28a75dac4ae..9dbf67efe50 100644
> --- a/include/system/memory.h
> +++ b/include/system/memory.h
> @@ -277,6 +277,12 @@ typedef struct IOMMUTLBEvent {
>   */
>  #define RAM_PRIVATE (1 << 13)
>  
> +/*
> + * RAM with guest_memfd that should start with shared KVM memory
> + * attributes. The guest converts to private on use.
> + */
> +#define RAM_GUEST_MEMFD_START_SHARED (1 << 14)
> +
>  static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn,
>                                         IOMMUNotifierFlag flags,
>                                         hwaddr start, hwaddr end,
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 97463a683f4..c034e74c8e5 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -1737,7 +1737,8 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
>              abort();
>          }
>  
> -        if (memory_region_has_guest_memfd(mr)) {
> +        if (memory_region_has_guest_memfd(mr) &&
> +            !(mr->ram_block->flags & RAM_GUEST_MEMFD_START_SHARED)) {
>              err = kvm_set_memory_attributes_private(start_addr, slot_size);
>              if (err) {
>                  error_report("%s: failed to set memory attribute private: %s",
> diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c
> index 35e03ed7599..b46efe21126 100644
> --- a/hw/virtio/virtio-mem.c
> +++ b/hw/virtio/virtio-mem.c
> @@ -19,6 +19,7 @@
>  #include "system/memory.h"
>  #include "system/numa.h"
>  #include "system/system.h"
> +#include "system/kvm.h"
>  #include "system/ramblock.h"
>  #include "system/reset.h"
>  #include "system/runstate.h"
> @@ -479,6 +480,11 @@ static int virtio_mem_set_block_state(VirtIOMEM *vmem, uint64_t start_gpa,
>          if (vmem->dynamic_memslots) {
>              virtio_mem_deactivate_unplugged_memslots(vmem, offset, size);
>          }
> +        if (rb->flags & RAM_GUEST_MEMFD_START_SHARED) {
> +            kvm_set_memory_attributes_shared(start_gpa, size);
> +            ram_block_attributes_state_change(rb->attributes,
> +                                              offset, size, false);
> +        }
>          return 0;
>      }
>  
> @@ -606,10 +612,12 @@ static int virtio_mem_unplug_all(VirtIOMEM *vmem)
>      RAMBlock *rb = vmem->memdev->mr.ram_block;
>  
>      if (vmem->size) {
> +        uint64_t used = qemu_ram_get_used_length(rb);
> +
>          if (virtio_mem_is_busy()) {
>              return -EBUSY;
>          }
> -        if (ram_block_discard_range(rb, 0, qemu_ram_get_used_length(rb))) {
> +        if (ram_block_discard_range(rb, 0, used)) {
>              return -EBUSY;
>          }
>          virtio_mem_notify_unplug_all(vmem);
> @@ -622,6 +630,11 @@ static int virtio_mem_unplug_all(VirtIOMEM *vmem)
>          if (vmem->dynamic_memslots) {
>              virtio_mem_deactivate_unplugged_memslots(vmem, 0, region_size);
>          }
> +        if (rb->flags & RAM_GUEST_MEMFD_START_SHARED) {
> +            kvm_set_memory_attributes_shared(vmem->addr, used);
> +            ram_block_attributes_state_change(rb->attributes,
> +                                              0, used, false);
> +        }
>      }
>  
>      trace_virtio_mem_unplugged_all();
> @@ -859,6 +872,18 @@ static void virtio_mem_device_realize(DeviceState *dev, Error **errp)
>      rb = vmem->memdev->mr.ram_block;
>      page_size = qemu_ram_pagesize(rb);
>  
> +    /*
> +     * For CoCo VMs with guest_memfd, use the "start-shared" model:
> +     * memory starts as shared and the guest converts to private on
> +     * plug.
> +     */
> +    if (rb->flags & RAM_GUEST_MEMFD) {
> +        rb->flags |= RAM_GUEST_MEMFD_START_SHARED;
> +        ram_block_attributes_state_change(rb->attributes, 0,
> +                                          qemu_ram_get_used_length(rb),
> +                                          false);
> +    }
> +
>      if (virtio_mem_has_legacy_guests()) {
>          switch (vmem->unplugged_inaccessible) {
>          case ON_OFF_AUTO_AUTO:
> 
> -- 
> 2.54.0
> 
> 

-- 
Peter Xu