Re: [PATCH 2/5] migration/postcopy: magic value for postcopy channel

[Fabiano Rosas <farosas@suse.de>]

Prasad Pandit <ppandit@redhat.com> writes:

> Hello Peter, Dan, Fabiano,
>
> Thank you so much for the detailed comments, I appreciate it.
>
> On Thu, 7 Nov 2024 at 17:41, Fabiano Rosas <farosas@suse.de> wrote:
>> The handshake will be a QEMU-only feature. Libvirt will then only start
>> the migration on src and QEMU will do the capabilities handling.
>>
> On Thu, 7 Nov 2024 at 18:03, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> Libvirt does what it does because it has had no other choice,
>> not because it was good or desirable.
>>
>> This kind of handshake really does not belong in libvirt. A number
>> of exposed migration protocol feature knobs should be considered
>> private to QEMU only.
>
> * Right, okay.
>
> * So then IIUC, libvirtd(8) would invoke migration by sending (without
> first checking with the destination libvirtd(8)) the migration command
> to the source QEMU and in qmp_migrate() before setting up the required
> connections, QEMU will add the feature negotiation (or handshake)
> step, right?

Yes, but there are still some points to figure out, as Peter mentioned,
such as how to handle capabilities for which there is a high chance that
libvirt does actually want to control, e.g. multifd. One approach is to
just continue to allow migrate-set-caps before qmp-migrate and take
those into account during handshake as well.

>
>> It has the very negative consequence that every time QEMU wants to
>> provide a new feature in migration, it needs to be plumbed up through
>> libvirt, and often applications above, and those 3rd party projects
>> need to be told when & where to use the new features. The 3rd party
>> developers have their own project dev priorities so may not get
>> around to enable the new migration features for years, if ever,
>> undermining the work of QEMU's migration maintainers.
>
> * True. I've seen the mismatch/disconnect between QEMU features and
> how libvirtd(8)/virsh(1) offers them to the end users. ex. What QEMU
> calls Multifd, virsh(1) calls --parallel-connections. Features like
> 'postcopy-preempt-channel' are implemented in QEMU, but no way for an
> end user to see/enable/disable it from virsh(1) side.
>
> * TBH, Multifd is such a misnomer, it could have been a plain simple
> --threads <N> option.
>     ex: virsh migrate --threads <N>: precopy migration with <N>
> threads, default <N> = 1.
>
>    Irrespective of the number of threads, the underlying migration
> mechanism/protocol remains the same. Threads only help to accelerate
> the rate of data transfer through multiple connections. We don't have
> to differentiate channels by sending magic values then.
>
> * Since we are thinking about correcting past wrongs, does having a
> unified migration protocol make sense? OR that is too ambitious a
> thing to think about? (just checking)
>
> * Meanwhile, @Fabiano Rosas If you have not started with the handshake
> or feature negotiation in QEMU, I'd try it on my side and we can
> discuss how the handshake should work. If you've already started with
> it, I'd be happy to help in some way as possible.
>

I'm putting together a prototype that we can iterate on. I'll let you
know as soon as I have something I can share.

> * Are we thinking about dynamically adjusting migration features based
> on their availability on both sides? Ex. say source says open 10
> parallel connections, but destination can do only 5, then source
> reports an error and terminates migration or continues with 5
> threads/connections OR say user does not mention parallel connections,
> but still we automatically start multiple threads because both ends
> support it? Just checking about dynamic adjustments, because earlier
> while discussing with Peter, he mentioned that we can not
> enable/disable user supplied options.
>
> Thank you.
> ---
>   - Prasad