Re: [RFC PATCH v2 2/4] KVM: x86: Add optional KVM_CAP_LIVE_MIGRATION and KVM_MIGRATE_CMD

[Kishen Maloor <kishen.maloor@intel.com>]

On 9/15/26 10:09 PM, Tony Lindgren wrote:
> On Tue, Sep 15, 2026 at 08:53:18AM -0700, Kishen Maloor wrote:
>> On 9/14/26 9:44 PM, Tony Lindgren wrote:
>>> On Mon, Sep 14, 2026 at 05:14:32PM -0700, Kishen Maloor wrote:
>>>> On 9/10/26 9:23 PM, Tony Lindgren wrote:
>>>>> On Thu, Sep 10, 2026 at 06:40:04PM -0700, Kishen Maloor wrote:
>>>>>> On 9/9/26 11:33 PM, Tony Lindgren wrote:
>>>>>>> On Wed, Sep 09, 2026 at 06:11:11PM -0700, Kishen Maloor wrote:
>>>>>>> ...
>>>>>>>>
>>>>>>>> As mentioned above, that is a destination launch time directive that we needn't
>>>>>>>> conflate with a migration/transfer session role.
>>>>>>>
>>>>>>> It's still the same role though. Yes we can set it on init, but would
>>>>>>> be nice to have some generic way to do it for qemu -incoming.
>>>>>>
>>>>>> I'd separate these.
>>>>>>
>>>>>> On the role: it seems we agree on recording roles per-session. My only point
>>>>>> then is that a VM created through the delayed_init flow doesn't additionally
>>>>>> need a destination role recorded for it if SETUP will assert one when the
>>>>>> migration is kicked off.
>>>>>>
>>>>>> On generic plumbing for -incoming: is this about KVM_TDX_INIT_VM_F_DELAY_INIT?
>>>>>> A generic mechanism would make sense to me if the flag were consumed by generic
>>>>>> KVM code, but that isn't the case here. Userspace has to make a
>>>>>> vendor-specific VM-init call like KVM_TDX_INIT_VM anyway, with the flag passed
>>>>>> on that call. So I'm not sure what a generic version would add, unless you have
>>>>>> something else in mind.
>>>>>
>>>>> So we could add a SETUP subcommand SET_ROLE or SET_INCOMING.
>>>> It might be better to pass the role as a parameter of the SETUP call
>>>> rather than a separate SET_ROLE call.
>>>> A separate SET_ROLE would bring its own ordering rules relative to the
>>>> other SETUP sub-commands. 
>>>
>>> OK a flag for SETUP sounds good to me. SETUP is needed anyways for each
>>> migration session.
>>
>> To be clear, I was suggesting a field in kvm_migrate_cmd and not a flag to pass
>> the role as an argument to SETUP. As I mentioned in my last comments (right 
>> below), 'flags' in the current proposal carry the vendor-defined sub-command values,
>> so a generic role argument wouldn't belong there.
> 
> I was thinking 8 bits for common flags and 8 bits for vendor flags but
> yeah that can be a bit tight. Sorry if the SETUP above caused extra
> confusion.
> 
> So trying to summarize the common flags for the role and separate vendor
> flags:
> 
> struct kvm_migrate_cmd {
> 	__u16 command;
> 	__u16 flags;
> 	__u16 vflags;
> 	__u16 reserved;
> 	__u32 reserved;
> 	struct kvm_transfer_buffer buf;
> };
> 
> Is the above along the lines what you were thinking?

No, I was suggesting a 'role' field carved out of the 'reserved' space,
like this:

struct kvm_migrate_cmd {
      __u16 command;
      __u16 flags;
      __u8  role;     /* 0 = unset, 1 = source, 2 = destination */
      __u8  reserved[3];
      struct kvm_transfer_buffer buf;
};

We haven't defined any generic flags. Thus far in this proposal 'flags'
contains only vendor-defined sub-command values.

>   
>>>> The specific role (src or dst) still needs someplace to go, and flags is
>>>> already the sub-command selector. An option is to carve out room in
>>>> the __u32 reserved field to carry a role argument, something
>>>> like 0=unset, 1=src, 2=dest so KVM can verify that a role was indeed set.
>>>
>>> To me it seems that 0=src can be the natural default starting point, I
>>> don't think we need 0=unset.
>>
>> With 0=src, the field would only carry information when it's a destination,
>> thereby making it an is_dest boolean rather than a role. It would also mean any 
>> VM that never established a session still reads as a source, so a 
>> KVM_TRANSFER_MEMORY aimed at the wrong VM by buggy or rogue userspace would get 
>> dispatched to the export path instead of rejected outright. The generic 
>> dispatcher shouldn't have to rely on the vendor layer to catch that. Reserving 0 
>> for 'unset' costs nothing and lets KVM reject a session that never stated a 
>> role.
> 
> Ah OK, yes that would also tell "the hardware has been initialized to a
> certain migration role". That seems like a usable common feature.

Not quite. It tells us that userspace asserted a role for this VM's migration
session. Whether a TD was created for import is a separate, vendor-level detail.
The generic layer only needs the role to reject a session that never stated one,
and to pick the export or import callback. That callback then knows which side
it's on and can reject an incorrect role (e.g., if SETUP asserted dst for a src TD).

>  
>>>> It could further be written into a generic KVM struct (kvm_arch or kvm) which
>>>> could be queried on KVM_TRANSFER_MEMORY, etc. to identify the relevant
>>>> callback.
>>>
>>> Yeah eventually some generic place for it would be nice. But that's easy
>>> to add later on too.
>>
>> Sure, we don't have to decide now as we're still discussing the UAPI.
>> But we'd want this detail also settled sometime before we call the UAPI
>> complete as it determines whether the dispatch is generic.
> 
> Yes a shared place for the role would make some generic sanity checks
> easier.

Agreed. The field above is just an argument to SETUP. Where that role gets stored
for KVM's top-level dispatcher to check is the detail we can settle later.