Re: [RFC PATCH v2 0/4] Add KVM API for confidential guest live migration

[Peter Xu <peterx@redhat.com>]

On Fri, Oct 02, 2026 at 10:57:46PM +0300, Artem Bityutskiy wrote:
> On Tue, 2026-09-29 at 17:05 -0400, Peter Xu wrote:
> > > Here is how I saw this, but I may be missing something (my excuse is that I
> > > am still new to the team and still learning).
> > > 
> > > 1. QEMU has a bitmap of shared pages in RAMBlockAttributes, so it can
> > >    distinguish shared pages.
> > > 2. In general, QEMU does not distinguish private vs unaccepted pages, so
> > >    unaccepted pages are treated as private pages.
> > 
> > Yes, the latter seems uncontroversial.
> > 
> > The 1st one is true, and it just reminded me if the conversion is
> > synchronous and one step requires the hypercall to QEMU, then indeed
> > background conversion can be avoided by some form of userspace locking.
> > 
> > Perhaps, a rwlock suites, each vCPU takes it for write whenever page
> > conversion requested from the guest (private <-> shared; nothing about
> > "accepted" that matters).  Then the migration threads, one or multiple,
> > take the read lock, lookup the bit, do MEM.EXPORT, unlock.
> > 
> > Then it seems fine in general, except that I donno if things can still go
> > wrong when there are multiple versions of "if this page is private or
> > shared".  Say, minimum of three?
> > 
> >   (a) QEMU maintains the bitmap in RAMBlockAttributes, each bit represents
> >       if the page is shared or private
> > 
> >   (b) KVM should maintain one, looks to me, kvm->mem_attr_array
> > 
> >   (c) Hardware / Firmware may maintain its own, in case of TDX, is that one
> >       bit on the SEPT pgtable?
> > 
> > They don't change together, AFAIU, they change in order, I believe
> > (c)->(a)->(b) if my above understanding is correct.
> 
> It looks like in terms of which layer saves the page type change first:
> 
> - Private->Shared: TDX -> KVM  -> QEMU
> - Shared->Private: KVM -> QEMU -> TDX
> 
> In both cases the TD initiates the change. This ends up with a TD exit,
> followed by KVM exiting to QEMU. QEMU calls kvm_convert_memory(), which
> calls back into KVM (KVM_SET_MEMORY_ATTRIBUTES). At this point SEPT did not
> change yet.
> 
> Private->Shared:
> - KVM first removes the page from SEPT, so TDX sees the change first.
> - KVM updates own data (kvm->mem_attr_array). So KVM "gets" the change
>   second.
> - QEMU updates RAMBlockAttributes, so QEMU "gets" the change last.
> 
> Shared->Private:
> - KVM removes the page from the shared EPT, updates own data
>   (kvm->mem_attr_array).
> - QEMU updates RAMBlockAttributes, discards backing storage.
> - Back to TD, which accepts the page. This causes an EPT violation, and
>   KVM adds the page to SEPT (TDH.MEM.PAGE.AUG).

Oh, this reminded me that ram_block_attributes_state_change() is done after
the ioctl(KVM_SET_MEMORY_ATTRIBUTES); I believe I didn't notice this detail
and assumed the other way round.  In that case, yes, KVM's page status will
always change before QEMU's.

> 
> > Then, what if they report different things?
> > 
> > Say, during migration the guest wants to convert a page from shared to
> > private.  (c) can be already done saying one page "private" now for TDX,
> > (a) tries to mark it "private" too, but now assuming page being accessed
> > (read lock held), it may be trying to take a write lock and sleep, which
> > means (b) will be "shared" so far.
> > 
> > So what happens is, QEMU thinks this page "shared" because the conversion
> > hasn't take place waiting for the write lock, however at least TDX may
> > think it already "private" instead.
> > 
> > Then QEMU logically can access HVA of that page, with (a)=shared,
> > (b)=shared, (c)=private.
> > 
> > Would it cause trouble?
> 
> (c) should see it as private only after KVM and QEMU do.
> 
> But I am not sure about the entire idea. Holding the read lock around the
> export ioctl means that a vCPU requesting a conversion waits until the
> export finishes. One export call may cover many MiB of crypto work, so the
> vCPU stalling may be significant, right?
> 
> Let's check the 2 cases.
> 
> Private -> Shared
> 
> QEMU calls the export ioctl for a page that it thinks is private, but
> meanwhile it became shared. In this case, if the semantics of the export
> ioctl is that such pages are skipped, we should be fine, right? QEMU will
> just handle this page during the next round.

Yes, this should be benign as long as conversion of page status set the
dirty bit; QEMU guarantees to clear the D-bit before reading the page, then
it must read it again later.

> 
> Shared -> Private
> 
> QEMU tries to migrate a shared page, which meanwhile became private. Reading
> it would result in zeros or some stale data, right? Would it help if QEMU
> used a lock-check_if_still_shared-copy-release, and the same lock around
> kvm_convert_memory()?

AFAIU, this should SIGBUS QEMU, which is the major issue, that should be
what happens in current linux tree, where now only INIT_SHARED is available
for fault() processing.

I also think that's the plan for afterwards, but still worth check kernel
tree with TDX migration integrated: kvm_gmem_fault_user_mapping() should
decide what happens..

-- 
Peter Xu