Kernel KVM virtualization development
 help / color / mirror / Atom feed
* [RFC PATCH v2 0/4] Add KVM API for confidential guest live migration
@ 2026-08-31  7:13 Tony Lindgren
  2026-08-31  7:13 ` [RFC PATCH v2 1/4] Documentation: KVM: Add live migration API for confidential guests Tony Lindgren
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Tony Lindgren @ 2026-08-31  7:13 UTC (permalink / raw)
  To: Paolo Bonzini, Sean Christopherson
  Cc: Peter Xu, Artem Bityutskiy, Fabiano Rosas, Jon Grimm,
	Pankaj Gupta, Tom Lendacky, Marc Zyngier, Oliver Upton,
	Steven Price, Anup Patel, Samuel Ortiz,
	Jakub Růžička, Jörg Rödel ,
	Vishal Annapurve, Elena Reshetova, Kai Huang, Kishen Maloor,
	Mika Westerberg, Peter Fang, Rick Edgecombe, Xiaoyao Li, Xu Yilun,
	kvm

Hi all,

As discussed in a recent PUCK call, Sean suggested we post what Intel is
using for the KVM live migration API for TDX as an example to see if we
can come up with APIs that are not vendor specific.

The goal of this patch series is to start a discussion about live migration
kernel APIs for CoCo guests.

Tom, since you mentioned that AMD SEV-SNP and Intel TDX live migration
sound similar, can you please take a look how the API might work for
SEV-SNP?

For CoCo VMs, the guest memory and vCPU states are not accessible to the
userspace or KVM for live migration. The memory and vCPU states need to be
extracted into encrypted blobs on the source, and decrypted on the
destination. Before live migration, an encryption key needs to be
negotiated between the source and destination.

The layer handling the encryption for live migration is implementation
specific. It can be the TDX module or Coconut-SVSM for example.

For CoCo VMs, the KVM_MEMORY_ENCRYPT_OP ioctl() has been used with vendor
specific sub-commands. Adding more vendor specific sub-commands is an
option also for live migration. However, depending on how similar the KVM
needs are, it may be possible to have a common API.

For TDX, we're using a group of ioctl()s that might be possible to adapt
also for other CoCo implementations.

Artem has put together a brief description below of the example API and the
migration flow:

Example API
===========

- KVM_CAP_LIVE_MIGRATION - if a VM supports live migration through this
  uAPI.
- KVM_MIGRATE_CMD - the main ioctl that drives the migration phases. Each
  command takes vendor-specific flags and a buffer for the blob that travels
  between the hosts.
  - KVM_MIGRATE_SETUP - establish the migration session and transfer the
    immutable VM state.
  - KVM_MIGRATE_ITERATION - close a memory copy round.
  - KVM_MIGRATE_STOP_AND_COPY - pause the VM and transfer the remaining VM
    state.
  - KVM_MIGRATE_END - complete the migration, or abort it.
- KVM_EXPORT_MEMORY - export memory pages on the source host.
- KVM_IMPORT_MEMORY - import memory pages on the destination host.
- KVM_EXPORT_VCPU - export vCPU state on the source host.
- KVM_IMPORT_VCPU - import vCPU state on the destination host.

Dirty page tracking does not add a new uAPI. Userspace keeps using
KVM_GET_DIRTY_LOG and KVM_CLEAR_DIRTY_LOG.

Migration flow
==============

 Source host                                      Destination host
 ===========                                      ================

 CMD(SETUP/SESSION)          <--- setup msgs ---> CMD(SETUP/SESSION)
       |    (repeated)                                  |
 CMD(SETUP/IMMUTABLE_STATE)  - immutable state -> CMD(SETUP/IMMUTABLE_STATE)
       |                                                |
 KVM_GET_DIRTY_LOG                                      |
 KVM_EXPORT_MEMORY           --- memory data ---> KVM_IMPORT_MEMORY
 CMD(ITERATION)              --- epoch token ---> CMD(ITERATION)
       |    (repeat until convergence)                  |
 CMD(STOP_AND_COPY/PAUSE)                               |
 CMD(STOP_AND_COPY/TD_STATE) --- VM state ------> CMD(STOP_AND_COPY/TD_STATE)
 KVM_EXPORT_VCPU             --- vCPU state ----> KVM_IMPORT_VCPU
 KVM_EXPORT_MEMORY           -- final memory ---> KVM_IMPORT_MEMORY
 CMD(ITERATION/DONE)         --- start token ---> CMD(ITERATION)
       |                                                |
 CMD(END)                                         CMD(END)

For the TDX implementation, the above map to the TDX module SEAMCALLs.

Regards,

Tony

Changes since v1 at [0] below:

- Drop KVM_MIGRATE_CMD sub-command PREPARE, SETUP sub-command has been
  enough for TDX at least

- Rename KVM_MIGRATE_CMD sub-command KVM_MIGRATE_TOKEN to
  KVM_MIGRATE_ITERATION

- Rename KVM_MIGRATE_CMD sub-command KVM_MIGRATE_SOURCE_BLACKOUT to
  KVM_MIGRATE_STOP_AND_COPY

- Add x86 ioctl handling

[0] https://lore.kernel.org/kvm/20251006113524.1573116-1-tony.lindgren@linux.intel.com/


Tony Lindgren (4):
  Documentation: KVM: Add live migration API for confidential guests
  KVM: x86: Add optional KVM_CAP_LIVE_MIGRATION and KVM_MIGRATE_CMD
  KVM: x86: Add optional KVM_EXPORT_MEMORY and KVM_IMPORT_MEMORY
  KVM: x86: Add optional KVM_EXPORT_VCPU and KVM_IMPORT_VCPU

 Documentation/virt/kvm/api.rst     | 205 +++++++++++++++++++++++++++++
 arch/x86/include/asm/kvm-x86-ops.h |   6 +
 arch/x86/include/asm/kvm_host.h    |   6 +
 arch/x86/kvm/x86.c                 | 102 ++++++++++++++
 include/uapi/linux/kvm.h           |  43 ++++++
 5 files changed, 362 insertions(+)


base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
-- 
2.43.0


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2026-09-01  6:13 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-31  7:13 [RFC PATCH v2 0/4] Add KVM API for confidential guest live migration Tony Lindgren
2026-08-31  7:13 ` [RFC PATCH v2 1/4] Documentation: KVM: Add live migration API for confidential guests Tony Lindgren
2026-08-31  7:20   ` sashiko-bot
2026-08-31  7:13 ` [RFC PATCH v2 2/4] KVM: x86: Add optional KVM_CAP_LIVE_MIGRATION and KVM_MIGRATE_CMD Tony Lindgren
2026-08-31  7:23   ` sashiko-bot
2026-09-01  6:03     ` Tony Lindgren
2026-08-31  7:13 ` [RFC PATCH v2 3/4] KVM: x86: Add optional KVM_EXPORT_MEMORY and KVM_IMPORT_MEMORY Tony Lindgren
2026-08-31  7:23   ` sashiko-bot
2026-09-01  6:10     ` Tony Lindgren
2026-08-31  7:13 ` [RFC PATCH v2 4/4] KVM: x86: Add optional KVM_EXPORT_VCPU and KVM_IMPORT_VCPU Tony Lindgren
2026-08-31  7:23   ` sashiko-bot
2026-09-01  6:12     ` Tony Lindgren

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox