Re: [PATCH RFC 0/2] migration/vl: new -incoming config:* for early migration parameters

[Peter Xu <peterx@redhat.com>]

On Thu, Jun 04, 2026 at 09:18:26AM +0100, Daniel P. Berrangé wrote:
> On Wed, Jun 03, 2026 at 02:46:52PM -0400, Peter Xu wrote:
> > On Wed, Jun 03, 2026 at 07:00:07PM +0100, Daniel P. Berrangé wrote:
> > > On Wed, Jun 03, 2026 at 02:51:30PM -0300, Fabiano Rosas wrote:
> > > > Daniel P. Berrangé <berrange@redhat.com> writes:
> > > > 
> > > > > In that case they should definitely be independent command line
> > > > > options. The old "-incoming" design is already broken / limited
> > > > > in the non-'defer' case because it is hardcoded to use URI syntax
> > > > > which can't express all the address formats we accept in QAPI
> > > > > syntax. Adding more special cases onto -incoming just makes the
> > > > > bad situation even worse and is not a forward looking design.
> > > > >
> > > > > If we need the ability to specify migration parameters on the
> > > > > CLI, then as a starting point for the design, we should assume that
> > > > > -incoming does not exist, and design a complete solution from scratch
> > > > > that uses QAPI exclusively, both for addresses and configuration.
> > > > >
> > > > > As discussed before, IMHO the "migrate" and "migrate-incoming"
> > > > > commands need to accept both the address(s) and parameters/capabilities
> > > > > as inline data items rather than relying on pre-configured global state
> > > > > from the 'migrate-parameter' / 'migrate-capability' commands.
> > > > >
> > > > > If we did that modelling for 'migrate-incoming' then that modelling of
> > > > > command parmaeters could map directly to a new '-migrate-incoming'
> > > > > command line argument that accepted exactly the same data model.
> > > > >
> > > > 
> > > > Maybe a user creatable object would better fit this use case instead of
> > > > a new command. We could expose what is today MigrationParameters plus
> > > > the few compat options from migration_properties. It could work more or
> > > > less the same for the QMP migration commands, QMP set/get commands,
> > > > source and destination command lines, the compatibility use-case and the
> > > > debugging use-case.
> > > 
> > > If we can do something using "-object" that is useful both at command
> > > line time and in QMP runtime, then that would be interesting too.
> > > 
> > > My general feeling wrt changes to the current command line is that
> > > any suggestion should be desgined with a nod towards a future scenario
> > > where QEMU is 100% configured with QMP. ie the full command line is
> > > 
> > >    qemu-system-x86_64 -qmp <address>
> > > 
> > > In this world, the current -incoming argument is already largely
> > > unsatisfactory for anything other than "defer".  A model that uses
> > > -object would fit in well, as would a hypothetical -migrate-incoming
> > > that directly mapped to 'migrate-incoming' QMP including capabilities
> > > and parameters.
> > 
> > I don't think even the current -incoming config:* is a blocker or add too
> > much complexity to the full-QMP-based solution.
> > 
> > When it comes, we can simply map all -incoming config:* (JSON or not) to
> > QMP command migrate-set-parameters, what we need is teaching that QMP
> > handler to apply parameters to the temporary MigrationParameters rather
> > than migration object when the latter hasn't been initialized.
> > 
> > I was talking to Fabiano and he suggested me to mention one more thing I
> > said on the list.  It's a matter of whether we should still invest time on
> > supporting migration caps/parameters in QMP migrate/migrate-incoming
> > commands.
> > 
> > Requirements like this (allow migration parameters to be accessible during
> > early stage of QEMU) is going towards the other direction of the that idea.
> > That means even if we put all configs (caps/params) into QMP command
> > migrate[-incoming], libvirt will still need to manage these global setup
> > and making sure when invoking migrate[-incoming] QMP commands they match
> > with the globals.  Say, if one start QEMU with -incoming config:local=on
> > but then invoke "migrate-incoming,local=off" it's illegal.
> > 
> > Considering that it looks like there're solid use cases that we want to
> > support (after CPR's "mode" parameter, now "local"), I want to discuss
> > again whether we want to still spend effort supporting "allow migrate QMP
> > command to specify capabilities/parameters".  Ultimately, we still seem to
> > need these global parameters.
> 
> This is saying that incoming migration is a multi phase/stage task.
> 
> There is a "prepare" phase which sets up QEMU ready to receive an
> incoming migration.
> 
> There is a "running" phase where we are waiting for incoming connection
> and/or handling the migration data stream.
> 
> The current "migrate_incoming" overloads both tasks into the same QMP
> command such that they always happen at the same point in time.
> "-incoming defer" was a crude hack to partially separate them, allowing
> parameters/capabilities to be set before the real migrate_incoming
> QMP command is invoked.
> 
> We can address that by separating the tasks explicitly with a
> "migrate_prepare" command / -migrate-prepare CLI arg that sets all
> the parameters/capabilities atomically, and a "migrate_run" command
> that initiates the processing.

I'm OK with this, or IMHO we can also stick with -incoming treating it as
the preparation phase.

For most of the time, I would slightly prefer keeping old things working
but building new things on top.  Obsolete should only happen if very needed
and properly justified.

For this one, I hope the QAPI helpers I used should introduce minimum
overhead even if we're reusing an old paramaeter.. we can still consider
introducing another parmeter besides -incoming, but it'll work similarly
like what this patch does, then.

> 
> The "jobs" QAPI design has this explicit concept of different phases
> that a job can be in, and provides commands for lifecycle handling.

We have some rich discussion before on Jobs:

https://lore.kernel.org/qemu-devel/878qg1uhbd.fsf_-_@pond.sub.org/#t

IIRC the consensus was it's only about the interfacing that can be shared or
not, which doesn't seem to be a huge mount of code that we can reuse; what
we can reuse is minimum and we may need to overload the interface when
migration joins the party.

We're definitely open to adopt anything Jobs did right.  We don't
necessarily need to switch to Jobs until further justified, which won't be
a trivial task.

Thanks,

-- 
Peter Xu