Re: [PATCH RFC 0/2] migration/vl: new -incoming config:* for early migration parameters

[Peter Xu <peterx@redhat.com>]

On Fri, Jun 05, 2026 at 10:35:52AM +0300, Vladimir Sementsov-Ogievskiy wrote:
> On 04.06.26 21:01, Peter Xu wrote:
> > On Thu, Jun 04, 2026 at 01:00:10AM +0300, Vladimir Sementsov-Ogievskiy wrote:
> > > On 03.06.26 21:46, Peter Xu wrote:
> > > > > My general feeling wrt changes to the current command line is that
> > > > > any suggestion should be desgined with a nod towards a future scenario
> > > > > where QEMU is 100% configured with QMP. ie the full command line is
> > > > > 
> > > > >      qemu-system-x86_64 -qmp <address>
> > > > > 
> > > 
> > > [..]
> > > 
> > > > 
> > > > Requirements like this (allow migration parameters to be accessible during
> > > > early stage of QEMU) is going towards the other direction of the that idea.
> > > > That means even if we put all configs (caps/params) into QMP command
> > > > migrate[-incoming], libvirt will still need to manage these global setup
> > > > and making sure when invoking migrate[-incoming] QMP commands they match
> > > > with the globals.  Say, if one start QEMU with -incomingconfig:local=on
> > > > but then invoke "migrate-incoming,local=off" it's illegal.
> > > 
> > > Right.. And this show, that moving "local" from migrate-set-parameters to
> > > commandline is not a clear solution for the whole problem.
> > > 
> > > We don't need cmdline argument. We need two things:
> > > 
> > > 1. "local" must be set before initializing tap-device.
> > > 
> > > It not actually requires it being a cmdline parameter. Calling
> > > migrate-set-parameters before netdev_add is also OK.
> > 
> > Ohh is it ok?  I thought it was not OK so we look at this.
> > 
> > Say, this is what I see on init TAP device when without hotplug:
> > 
> > qemu_create_late_backends() -> net_init_clients()
> > 
> > Such happens before migration object initialization.
> 
> Yes, it doesn't work for TAP devices added through commandline. But
> if devices are added only throuhg QMP (isn't libvirt do so?), it
> should work.
> 
> > 
> > > 
> > > So, moving to cmdline solves this [1] point, but may be too restrictive.
> > > 
> > > 
> > > 2. "local" must not be changed after initializing tap-device.
> > > 
> > > And this one is not guaranteed anyway, with cmdline or with QMP.
> > > 
> > > ---
> > > 
> > > 
> > > So, in my series, if drop "incoming-fds" and rely on "local" instead, we
> > > actually want "local" be immutable after first read in tap initialization
> > > code.
> > > 
> > > Maybe, just implement this feature? So, in code it will look like:
> > > 
> > > set_migration_parameters(...) {
> > > 
> > > ...
> > > 
> > > if "local" value is changing and "local" is immutable:
> > >     fail
> > 
> > We can't do that, can we?
> > 
> > Imagine we need to further migrate this VM to another host, where we need
> > to turn "local" off after this incoming migration.. we can forbid only
> > during incoming phase and re-enable the mutability, but it seems too much.
> 
> Agree, right. I missed further outgoing migration.
> 
> > 
> > My understanding is such protection is fine but not strongly necessary.
> > IMHO we rely on a lot of things that admin needs to do right.  I hope this
> > isn't a major issue to offload that to admin to say the admin should always
> > do the right things.
> > 
> > We have a bunch of similar issues in QEMU IIUC, e.g. we have known issue
> > that some -device needs to be ordered in some way otherwise it'll stop
> > working.  We then need admin (or in this case libvirt) do the right thing
> > too.
> 
> So, possible way is:
> 
> 1. net devices are only added by QMP commands
> 2. migration parameter "local" is set before adding devices
> 
> So, we can omit "incoming-fds" and rely on "local" && "support-local-migration".
> 
> And we can check in .pre_incoming of TAP device, does current value of "local"
> equal to what it was at time of TAP initialization, and fail if it differs.
> 
> Still, having explicit incoming-fds looks safer, as it's more difficult to do
> a mistake.

If this will be the only outlier, then maybe yes, an extra option in a
special device backend isn't much of an issue.

The question is if there'll be more things relying on "local=on".  We may
not want to keep introducing per-device flags even if all of them can play
the same role as migration's "local".

> 
> With incoming-fds=true, user is sure, that new target vm will not try
> to connect/open resources, which are still used by running source vm,
> regardless of migration configuration correctness.
> 
> For example, if user accidentally set "local" _after_ creating TAP device,
> we can of-course check it in .pre_incoming and fail, but at the moment we
> already break networking in running guest, by attaching to same backend on
> target QEMU.
> 
> -- 
> Best regards,
> Vladimir
> 

-- 
Peter Xu