Re: [PATCH v3 0/7] migration/cpr: support vhost-vsock devices

[Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>]

On 7/14/26 6:03 PM, Vladimir Sementsov-Ogievskiy wrote:
> On 26.06.26 19:46, Andrey Drobyshev wrote:
>> v2 -> v3:
>>
>> Re-design device ownership hand-off (suggested by Dongli).  Do not rely
>> on SETUP migration notifiers, as this approach forces us to reorder
>> generic migration code (see v2 discussion).  Instead:
>>
>>    * .pre_save() releases ownership on the source (RESET_OWNER) once VM
>>      is stopped;
>>    * .realize() is adjusted to defer device ownership acquisition for an
>>      incoming CPR;
>>    * .post_load() actually claims the device (VHOST_SET_OWNER);
>>    * FAILED migration event callback re-aquires the ownership on the
>>      source after re_save released it.
>>
>> v2:https://lore.kernel.org/qemu-devel/57c9c9b3-d758-489b-95b8-d16c258b9b0c@virtuozzo.com
> 
> Hi!
>

Hello Vladimir!

> Two notes:
> 
> 1. Did you consider migrating needed FDs through main migration channel, without
> use of CPR, like I do in (not yet landed) "[PATCH v19 00/15] virtio-net: live-TAP local migration" [1]
>

AFAIU you're doing local same-host migration as a way to update QEMU
binary.  With this approach transferring FDs through the main channel is
indeed cleaner.  However in our own downstream we use enhanced in-place
memory preservation which is based entirely on cpr-exec migration mode.

So, to answer your question: yes, we considered it, and it might work
for cpr-transfer where we're also doing same-host migration.  But in
cpr-exec case there's only one QEMU process involved => we can't use
UNIX socket for migration transport => no SCM_RIGHTS FDs passing => FDs
should be passed via additional transport, no way around it.


> 2. I don't know how much vhost-vsock differs from vhost-net, but for vhost-net I remember
> that RESET_OWNER + SET_OWNER is effectively equal to simply recreating vhost device on target,
> because in kernel vhost device is hardly bound to the process itself, and can't be passed to
> another process. So, we can pass only "empty" FD, not the initialized vhost device.
> That's why in [1] I only pass tap-fds (and some additional state), but vhost fds are simply
> reopened on target (or passed by mgmt app).
> Does vhost-vsock work in a different way? Is there real sense in passing vhost fds here?
> 
> 
> [1] https://lore.kernel.org/qemu-devel/20260714154246.1242856-1-vsementsov@yandex-team.ru/
> 

I'm also not extremely keen on vhost-net internals, but as I understand
with vhost-net we have 2 FDs: TAP FD for the network endpoint and vhost
FD for the memory tables, vring addresses etc.  The TAP FD is
process-independent state - and we pass it via migration channel.  The
vhost FD is entirely per owning process - and we're able to close it on
source, reopen on target and bind to the passed TAP FD.

However with vhost-vsock there's only one FD.  It's used both for
plumbing and the endpoint.  At the very least, vhost device state holds
guest CID registration in the global vhost_vsock_hash.  If we go the
"recreate/reopen the device" path - CID is removed from the hash in
vhost_vsock_dev_release(), and then we're gonna get ECONNRESET for any
incoming packets => connections are dead.  We can't afford that.

Andrey