Re: [RFC PATCH v2 0/4] Add KVM API for confidential guest live migration

["Serge Hallyn (AMD)" <sergeh@kernel.org>]

On Mon, Aug 31, 2026 at 10:13:00AM +0300, Tony Lindgren wrote:
> Hi all,
> 
> As discussed in a recent PUCK call, Sean suggested we post what Intel is
> using for the KVM live migration API for TDX as an example to see if we
> can come up with APIs that are not vendor specific.
> 
> The goal of this patch series is to start a discussion about live migration
> kernel APIs for CoCo guests.
> 
> Tom, since you mentioned that AMD SEV-SNP and Intel TDX live migration
> sound similar, can you please take a look how the API might work for
> SEV-SNP?
> 
> For CoCo VMs, the guest memory and vCPU states are not accessible to the
> userspace or KVM for live migration. The memory and vCPU states need to be
> extracted into encrypted blobs on the source, and decrypted on the
> destination. Before live migration, an encryption key needs to be
> negotiated between the source and destination.
> 
> The layer handling the encryption for live migration is implementation
> specific. It can be the TDX module or Coconut-SVSM for example.
> 
> For CoCo VMs, the KVM_MEMORY_ENCRYPT_OP ioctl() has been used with vendor
> specific sub-commands. Adding more vendor specific sub-commands is an
> option also for live migration. However, depending on how similar the KVM
> needs are, it may be possible to have a common API.
> 
> For TDX, we're using a group of ioctl()s that might be possible to adapt
> also for other CoCo implementations.
> 
> Artem has put together a brief description below of the example API and the
> migration flow:

Hi Tony,

is there any pubically available qemu git branch or patchset to show
how you currently are driving this?

> Example API
> ===========
> 
> - KVM_CAP_LIVE_MIGRATION - if a VM supports live migration through this
>   uAPI.
> - KVM_MIGRATE_CMD - the main ioctl that drives the migration phases. Each
>   command takes vendor-specific flags and a buffer for the blob that travels
>   between the hosts.
>   - KVM_MIGRATE_SETUP - establish the migration session and transfer the
>     immutable VM state.
>   - KVM_MIGRATE_ITERATION - close a memory copy round.
>   - KVM_MIGRATE_STOP_AND_COPY - pause the VM and transfer the remaining VM
>     state.
>   - KVM_MIGRATE_END - complete the migration, or abort it.
> - KVM_EXPORT_MEMORY - export memory pages on the source host.
> - KVM_IMPORT_MEMORY - import memory pages on the destination host.
> - KVM_EXPORT_VCPU - export vCPU state on the source host.
> - KVM_IMPORT_VCPU - import vCPU state on the destination host.
> 
> Dirty page tracking does not add a new uAPI. Userspace keeps using
> KVM_GET_DIRTY_LOG and KVM_CLEAR_DIRTY_LOG.
> 
> Migration flow
> ==============
> 
>  Source host                                      Destination host
>  ===========                                      ================
> 
>  CMD(SETUP/SESSION)          <--- setup msgs ---> CMD(SETUP/SESSION)
>        |    (repeated)                                  |
>  CMD(SETUP/IMMUTABLE_STATE)  - immutable state -> CMD(SETUP/IMMUTABLE_STATE)
>        |                                                |
>  KVM_GET_DIRTY_LOG                                      |
>  KVM_EXPORT_MEMORY           --- memory data ---> KVM_IMPORT_MEMORY
>  CMD(ITERATION)              --- epoch token ---> CMD(ITERATION)
>        |    (repeat until convergence)                  |
>  CMD(STOP_AND_COPY/PAUSE)                               |
>  CMD(STOP_AND_COPY/TD_STATE) --- VM state ------> CMD(STOP_AND_COPY/TD_STATE)
>  KVM_EXPORT_VCPU             --- vCPU state ----> KVM_IMPORT_VCPU
>  KVM_EXPORT_MEMORY           -- final memory ---> KVM_IMPORT_MEMORY
>  CMD(ITERATION/DONE)         --- start token ---> CMD(ITERATION)
>        |                                                |
>  CMD(END)                                         CMD(END)
> 
> For the TDX implementation, the above map to the TDX module SEAMCALLs.
> 
> Regards,
> 
> Tony
> 
> Changes since v1 at [0] below:
> 
> - Drop KVM_MIGRATE_CMD sub-command PREPARE, SETUP sub-command has been
>   enough for TDX at least
> 
> - Rename KVM_MIGRATE_CMD sub-command KVM_MIGRATE_TOKEN to
>   KVM_MIGRATE_ITERATION
> 
> - Rename KVM_MIGRATE_CMD sub-command KVM_MIGRATE_SOURCE_BLACKOUT to
>   KVM_MIGRATE_STOP_AND_COPY
> 
> - Add x86 ioctl handling
> 
> [0] https://lore.kernel.org/kvm/20251006113524.1573116-1-tony.lindgren@linux.intel.com/
> 
> 
> Tony Lindgren (4):
>   Documentation: KVM: Add live migration API for confidential guests
>   KVM: x86: Add optional KVM_CAP_LIVE_MIGRATION and KVM_MIGRATE_CMD
>   KVM: x86: Add optional KVM_EXPORT_MEMORY and KVM_IMPORT_MEMORY
>   KVM: x86: Add optional KVM_EXPORT_VCPU and KVM_IMPORT_VCPU
> 
>  Documentation/virt/kvm/api.rst     | 205 +++++++++++++++++++++++++++++
>  arch/x86/include/asm/kvm-x86-ops.h |   6 +
>  arch/x86/include/asm/kvm_host.h    |   6 +
>  arch/x86/kvm/x86.c                 | 102 ++++++++++++++
>  include/uapi/linux/kvm.h           |  43 ++++++
>  5 files changed, 362 insertions(+)
> 
> 
> base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
> -- 
> 2.43.0
>