Re: [RFC V1 0/6] Live update: cpr-transfer

[Steven Sistare <steven.sistare@oracle.com>]

On 8/16/2024 2:34 PM, Steven Sistare wrote:
> On 8/16/2024 11:59 AM, Peter Xu wrote:
>> On Fri, Aug 16, 2024 at 04:36:58PM +0100, Daniel P. Berrangé wrote:
>>> On Fri, Aug 16, 2024 at 11:23:01AM -0400, Peter Xu wrote:
>>>> On Fri, Aug 16, 2024 at 11:13:36AM -0400, Steven Sistare wrote:
>>>>> On 8/15/2024 4:28 PM, Peter Xu wrote:
>>>>>> On Sat, Jul 20, 2024 at 04:07:50PM -0400, Steven Sistare wrote:
>>>>>>>>> The new user-visible interfaces are:
>>>>>>>>>      * cpr-transfer (MigMode migration parameter)
>>>>>>>>>      * cpr-uri (migration parameter)
>>>>>>>>
>>>>>>>> I wonder whether this parameter can be avoided already, maybe we can let
>>>>>>>> cpr-transfer depend on unix socket in -incoming, then integrate fd sharing
>>>>>>>> in the same channel?
>>>>>>>
>>>>>>> You saw the answer in another thread, but I repeat it here for others benefit:
>>>>>>>
>>>>>>>     "CPR state cannot be sent over the normal migration channel, because devices
>>>>>>>      and backends are created prior to reading the channel, so this mode sends
>>>>>>>      CPR state over a second migration channel that is not visible to the user.
>>>>>>>      New QEMU reads the second channel prior to creating devices or backends."
>>>>>>
>>>>>> Today when looking again, I wonder about the other way round: can we make
>>>>>> the new parameter called "-incoming-cpr", working exactly the same as
>>>>>> "cpr-uri" qemu cmdline, but then after cpr is loaded it'll be automatically
>>>>>> be reused for migration incoming ports?
>>>>>>
>>>>>> After all, cpr needs to happen already with unix sockets.  Having separate
>>>>>> cmdline options grants user to make the other one to be non-unix, but that
>>>>>> doesn't seem to buy us anything.. then it seems easier to always reuse it,
>>>>>> and restrict cpr-transfer to only work with unix sockets for incoming too?
>>>>>
>>>>> This idea also occurred to me, but I dislike the loss of flexibility for
>>>>> the incoming socket type.  The exec URI in particular can do anything, and
>>>>> we would be eliminating it.
>>>>
>>>> Ah, I would be guessing that if Juan is still around then exec URI should
>>>> already been marked deprecated and prone to removal soon.. while I tend to
>>>> agree that exec does introduce some complexity meanwhile iiuc nobody uses
>>>> that in production systems.
>>>>
>>>> What's the exec use case you're picturing?  Would that mostly for debugging
>>>> purpose, and would that be easily replaceable with another tunnelling like
>>>> "ncat" or so?
>>>
>>> Conceptually "exec:" is a nice thing, but from a practical POV it
>>> introduces difficulties for QEMU. QEMU doesn't know if the exec'd
>>> command will provide a unidirectional channel or bidirectional
>>> channel, so has to assume the worst - unidirectional. It also can't
>>> know if it is safe to run the exec multiple times, or is only valid
>>> to run it once - so afgai nhas to assume once only.
>>>
>>> We could fix those by adding further flags in the migration address
>>> to indicate if its bi-directional & multi-channel safe.
>>>
>>> Technically "exec" is obsolete given "fd", but then that applies to
>>> literally all protocols. Implementing them in QEMU is a more user
>>> friendly thing.
>>>
>>> Exec was more compelling when QEMU's other protocols were less
>>> mature, lacking TLS for example, but I still find it interesting
>>> as a facility.
>>
>> Right, it's an interesting idea on its own.  It's just that when QEMU grows
>> into not only a tool anymore it adds burden on top as you discussed, in
>> which case we consider dropping things as wins (and we already started
>> doing so at least in migration, but iiuc it's not limited to migration).
>>
>> Again, it looks reasonable to drop because I think it's too easy to tool-up
>> the same "exec:" function with ncat or similar things.  E.g. kubevirt does
>> TLS even today without qemu's TLS, and AFAIU that's based on unix sockets
>> not exec, and it tunnels to the daemon for TLS encryption (which is prone
>> of removal, though).  So even that is not leveraged as we thought.
> 
> Also, the "fd" URI would not work.  We could not read from it once for cpr state,
> reopen it, and read again for migration state.
> 
> Nor multifd.

Am I wrong?

I still go back to my original statement: -incoming-cpr  has equal or greater
"specification complexity" than -cpr-uri.  It is not simpler, and comes with
restrictions.

- Steve