Re: TDX migration model

["Serge Hallyn (AMD)" <sergeh@kernel.org>]

(Adding Tycho and Mahmoud, who will be working with me on this)

On Mon, Nov 09, 2026 at 09:45:17AM +0200, Tony Lindgren wrote:
> On Thu, Nov 05, 2026 at 06:00:41PM -0800, Kishen Maloor wrote:
> > On 11/5/26 8:42 AM, Serge E. Hallyn wrote:
> > > On Thu, Nov 05, 2026 at 05:59:01PM +0200, Artem Bityutskiy wrote:
> > >> On Thu, 2026-11-05 at 08:58 -0600, Serge Hallyn (AMD) wrote:
> > >>> ...
> > >>> So, looking at the section '### How the Setup Session Works', is it safe
> > >>> to assume that you are using the qemu migration fd for the message
> > >>> transfers?  So basically qemu loops over KVM_MIGRATE_SETUP calls, which
> > >>> ...
> > >> Yes, that's the flow. I’m not sure about the migration fd part, I believe we
> > >> pass the setup session data via ioctl buffers. I am sure Kishen will can
> > >> clarify.
> > > 
> > > If you mean pass via ioctl buffers from tdx->kvm-> back to qemu, which passes
> > > it over the migration fd to the other side, then that's what I expected.
> > > Othewise I'll be interested to the details :)
> > 
> > Hi Serge,
> > 
> > Your understanding is right. Migration setup requires a handshake of some sort
> > between the source and destination. So the messages KVM_MIGRATE_SETUP returns
> > are transferred over QEMU's migration channel to the other side and processed
> > by a matching call.
> 
> Serge, on the QEMU side, we have been enabling the return-path early for
> the key exchange before migration is active. Not sure if the key exchange
> can be done using a VMState, possibly could be if we can ensure it happens
> before the migration is active.

Key exchange requires bidirectional communication, so I don't think a
VMState would work, at least not on its own. If the destination's
info got send to the source out of band and delivered through migrate
command option, that could work, but it seems much nicer to me if it's
all done more transparently.  OTOH if this simplifies the qemu code
enough, maybe it's worth it.

> For the migration related commands, looks like we can just make the
> KVM_EXPORT_CMD and KVM_IMPORT_CMD subcommands a bit more finer grained and
> then drop the vendor flags until needed:
> 
> #define KVM_MIGRATE_EXCHANGE_KEYS	0 (New, was KVM_MIGRATE_SETUP and a flag)

We are currently looking at a couple of messages back and forth during the
setup negotiation, to accomplish both key exchange and verification of
attestation reports.  I was actually just using KVM_MIGRATE_SETUP with
flags for that.  Maybe KVM_MIGRATE_EXCHANGE_KEYS could be renamed to
KVM_MIGRATE_NEGOTIATE?  Flags aren't strictly needed, if the
KVM_MIGRATE_NEGOTIATE ioctl will be repeated depending on the return
value.

> #define KVM_MIGRATE_SETUP		1 (Migrates the immutable state)
> #define KVM_MIGRATE_ITERATION		2 (Start a new pre-copy round)
> #define KVM_MIGRATE_ITERATION_FINAL	3 (New, last pre-copy round before pause)
> #define KVM_MIGRATE_PAUSE		4 (New, split from old STOP_AND_COPY)
> #define KVM_MIGRATE_MUTABLE_STATE	5 (New, split from old STOP_AND_COPY)
> #define KVM_MIGRATE_START_DESTINATION	6 (New, split from old STOP_AND_COPY)
> #define KVM_MIGRATE_START_POSTCOPY	7 (New, could return -EOPNOTSUPP for now?)
> #define KVM_MIGRATE_ABORT		8
> #define KVM_MIGRATE_END			9
> 
> For the above PAUSE is four the source only. START_DESTINATION is for both
> source and destination as source exports a start token for the destination.
> START_POSTCOPY is not implemented but would be destination only when
> implemented. ABORT is allowed at any point.
> 
> Any other subcommands needed at this point?

Not absolutely certain, but it looks like this might be complete.

> Then for handling the results, the following seems to simplify things a bit:
> 
> - Results are copied back only on success, vendor error is returned as is
> 
> - Import is all or nothing, vendor code resumes on interruptions
> 
> - For memory transfer, a GFN can only be transferred once per round,
>   duplicate import attempt returns -EINVAL

Hm, does the platform independent code need to worry about this?

> Regards,
> 
> Tony

thanks,
-serge