Re: [RFC PATCH v2 2/4] KVM: x86: Add optional KVM_CAP_LIVE_MIGRATION and KVM_MIGRATE_CMD

[Kishen Maloor <kishen.maloor@intel.com>]

On 9/8/26 11:57 PM, Tony Lindgren wrote:
> On Tue, Sep 08, 2026 at 05:22:30PM -0700, Kishen Maloor wrote:
>> On 9/7/26 9:43 PM, Tony Lindgren wrote:
>>> On Mon, Sep 07, 2026 at 04:32:33PM +0300, Artem Bityutskiy wrote:
>>>> On Mon, 2026-09-07 at 15:15 +0200, Jörg Rödel wrote:
>>>>> The direction is always the same over a single live migration session, right?
>>>>> So it could be a setup flag, on the other hand having separate KVM_EXPORT_CMD
>>>>> and KVM_IMPORT_CMD seems to be a cleaner ABI.
>>>
>>> OK
>>
>>
>> Just sharing an alternate point of view:
>>
>> The roles are fixed over a migration session. A split ABI is certainly more 
>> self-describing, but it restates that invariant on every call, and therefore 
>> also permits it to be contradicted -- a failure mode that does not otherwise
>> exist. With a single KVM_MIGRATE_CMD and a per-session role recorded once (more 
>> on that below), there is no need for per-call policing: the role could be 
>> checked once when the session is established.
> 
> There are two occasions the role is set or changed. On starting the
> destination the incoming role needs to configured at least for TDX.
> And then after the migration, the role changes if re-migrated.

A destination TD needs a directive to not initialize the TD and its vCPUs.
It comes from its launch parameters (e.g. QEMU cmdline) which selects the
delayed_init path. That is a construction directive though, and it applies
only to a destination -- a source needs nothing at init. A session role is
symmetric and is what the transfer calls consume, so I don't think the two
need to be the same thing.

> I don't think there are other cases for role change, maybe cancelled
> migration could require that for some hardware possibly.

Architecturally, per-session scoping of migration roles should be
straightforward with any platform: each side asserts a role at the start of every 
migration session and vendor code will either accept or reject the stated role.

>> Along these lines: it raises a question of whether the MEMORY and VCPU
>> calls should be coalesced as well into KVM_MIGRATE_MEMORY and KVM_MIGRATE_VCPU.
>> As posted, direction is implicit for KVM_MIGRATE_CMD but encoded in the ioctl
>> number for those transfers, so collapsing them would at least make the uAPI
>> consistent about where direction comes from, and free two ioctls.
> 
> Using naming KVM_TRANSFER_MEMORY and KVM_TRANSFER_VCPU might be more
> descriptive?
> 
> Eventually these same commands could be used to save the state to disk
> for power management use.

Sure, and the save-to-disk case is a good argument for a more generic name.

> And going back to the dmaengine like analogy of what is being done..
> 
> The transfer direction flags could be KVM_TRANSFER_FROM_GUEST and
> KVM_TRANSFER_TO_GUEST?

FROM_GUEST/TO_GUEST still encodes direction per call, which is the open
question above. If direction is a per-session property, then KVM_TRANSFER_MEMORY
and KVM_TRANSFER_VCPU are sufficient on their own -- no direction flag, and no
separate export/import ioctls.

And if we settle on a per-session property, then SETUP could conceivably state a
role for a non-migration transfer session as well.

>   
>>>> For direction specifically, we could pass it to KVM on every call and let
>>>> KVM stay stateless about it, or KVM could record it once and remember it for
>>>> the rest of the session.
>>>
>>> Yes the "record and remember" is another option, it could be a sub-command
>>> something like KVM_MIGRATE_DIRECTION.
>>
>>
>> Agreed on record-and-remember, with a refinement on scope.
>>
>> A VM that was migrated in can later be migrated out, so the role is not a
>> property of the VM -- it has to be recorded per session. SETUP is the call
>> that starts a migration and runs ahead of all other migration calls, so its
>> arguments look like the natural place for userspace to state the role; a
>> separate sub-command would need its own scope rules relative to SETUP.
>> KVM can then ask the vendor layer whether the requested role is permitted for
>> this VM -- only it knows the confidential-VM state -- and on success record it
>> in generic KVM state, where it then selects the export or import callbacks.
> 
> The role can change, but it can be VM specific for starting the migration
> destination even before migration is started.
> 
> At least for TDX we need to specify direction for migration destination on
> init to prevent fully initializing the TD and vCPUs.

As mentioned above, that is a destination launch time directive that we needn't
conflate with a migration/transfer session role.