[PATCH v2 00/25] vhost-user-blk: live-backend local migration

[PATCH v2 00/25] vhost-user-blk: live-backend local migration

[Vladimir Sementsov-Ogievskiy]

Hi all!

v2: extremely simplified API: instead of 

- separate capability for migrating chardevs
- separate capability for migrating vhost-user-blk
- "support-local-migration" and "local-incoming" chardev options
- "local-incoming" option for vhost-user-blke

instead of all of this, v2 API is only one capability to
enable the whole feature.

Still, final look of this capability is not clear yet, the
discussion is in context of
"[PATCH v8 00/19] virtio-net: live-TAP local migration", here:
https://lore.kernel.org/qemu-devel/29aa1d66-9fa7-4e44-b0e3-2ca26e77accf@yandex-team.ru/

This series comes with temporary API in commit
"RFC qapi: add local-vhost-user-blk migration capability"

v2 is based on:
[PATCH v4 0/7] chardev: postpone connect
  (which in turn is based on [PATCH 0/2] remove deprecated 'reconnect' options)
[PATCH v3 00/23] vhost refactoring and fixes
and one commit from v8 of live-tap series:
[PATCH v8 14/19] migration: introduce .pre_incoming() vmsd handler

Based-on: <20250924133309.334631-1-vsementsov@yandex-team.ru>
Based-on: <20251015212051.1156334-1-vsementsov@yandex-team.ru>
Based-on: <20251015145808.1112843-1-vsementsov@yandex-team.ru>
Based-on: <20251015132136.1083972-15-vsementsov@yandex-team.ru>

The code is also may be found in my gitlab account, tag
up-vhost-user-blk-fd-migration-v2 :
https://gitlab.com/vsementsov/qemu/-/tree/up-vhost-user-blk-fd-migration-v2

-

Local migration of vhost-user-blk requires non-trivial actions
from management layer, it should provide a new connection for new
QEMU process and handle disk operation movement from one connection
to another.

Such switching, including reinitialization of vhost-user connection,
draining disk requests, etc, adds significant value to local migration
downtime.

This all leads to an idea: why not to just pass all we need from
old QEMU process to the new one (including open file descriptors),
and don't touch the backend at all? This way, the vhost user backend
server will not even know, that QEMU process is changed, as live
vhost-user connection is migrated.

So this series realize the idea. No requests are done to backend
during migration, instead all backend-related state and all related
file descriptors (vhost-user connection, guest/host notifiers,
inflight region) are passed to new process. Of course, migration
should go through unix socket.

Why not CPR-transfer?

1. In the new mode of local migration we need to pass not only
file descriptors, but additional parts of backend-related state,
which we don't want (or even can't) reinitialize in target process.
And it's a lot simpler to add new fields to common migration stream.
And why not to pass fds in the same stream?

2. No benefit of vhost-user connection fd passed to target in early
stage before device creation: we can't use it together with source
QEMU process anyway. So, we need a moment, when source qemu stops using
the fd, and target start doing it. And native place for this moment is
usual save/load of the device in migration process. And yes, we have to
deeply update initialization/starting of the device to not reinitialize
the backend, but just continue to work with it in a new QEMU process.

3. So, if we can't actually use fd, passed early before device creation,
no reason to care about:
- non-working QMP connection on target until "migrate" command on source
- additional migration channel
- implementing code to pass additional non-fd fields together with fds in CPR

However, the series doesn't conflict with CPR-transfer, as it's actually
a usual migration with some additional capabilities. The only
requirement is that main migration channel should be a unix socket.

Vladimir Sementsov-Ogievskiy (25):
  vhost: store busyloop_timeout into struct vhost_dev
  vhost: reorder logic in vhost_dev_init()
  vhost: rework vhost_virtqueue_init()
  vhost: add connect parameter to vhost_dev_init()
  vhost: split vhost_dev_connect() out of vhost_dev_init()
  vhost-user: support connect api
  vhost-user-blk: vhost_user_blk_connect() move connected check to
    caller
  vhost-user-blk: vhost_user_blk_connect(): call vhost_dev_connect()
  vhost-user-blk: rename vhost_user_blk_connect to vhost_user_blk_init
  vhost-user-blk: split vhost_user_blk_init()
  vhost-user-blk: move initial reconnect loop to separate function
  vhost-user-blk: move first vhost_user_blk_init() to _realize()
  vhost-user-blk: postpone connect to pre-incoming
  virtio: introduce .skip_vhost_migration_log() handler
  migration: introduce vmstate_event_notifier
  chardev: add .chr_get_client() handler
  vhost: add inflight region backend-transfer vmstate
  chardev: introduce backend-transfer vmstate for chardev
  vhost: support backend-transfer migration
  vhost-user: add vmstate
  virtio: support vhost backend migration
  virtio: support .needed for virtio vmsd
  RFC qapi: add local-vhost-user-blk migration capability
  vhost-user-blk: support vhost backend migration
  tests/functional: add test_x86_64_vhost_user_blk_fd_migration.py

 backends/cryptodev-vhost.c                    |   2 +-
 backends/vhost-user.c                         |   2 +-
 chardev/char-backend-transfer.c               |  52 +++
 chardev/char-socket.c                         |   7 +
 chardev/char.c                                |   6 +
 chardev/meson.build                           |   1 +
 hw/block/trace-events                         |   2 +
 hw/block/vhost-user-blk.c                     | 255 +++++++++++----
 hw/net/vhost_net.c                            |   2 +-
 hw/scsi/vhost-scsi.c                          |   2 +-
 hw/scsi/vhost-user-scsi.c                     |   2 +-
 hw/virtio/vdpa-dev.c                          |   3 +-
 hw/virtio/vhost-user-base.c                   |   2 +-
 hw/virtio/vhost-user-fs.c                     |   2 +-
 hw/virtio/vhost-user-scmi.c                   |   2 +-
 hw/virtio/vhost-user-vsock.c                  |   2 +-
 hw/virtio/vhost-user.c                        | 127 +++++++-
 hw/virtio/vhost-vsock.c                       |   2 +-
 hw/virtio/vhost.c                             | 266 +++++++++++----
 hw/virtio/virtio-bus.c                        |   2 +-
 hw/virtio/virtio.c                            |  77 ++++-
 include/chardev/char-backend-transfer.h       |  17 +
 include/chardev/char.h                        |   4 +
 include/hw/virtio/vhost-backend.h             |   2 +
 include/hw/virtio/vhost-user-blk.h            |   2 +
 include/hw/virtio/vhost-user.h                |   4 +
 include/hw/virtio/vhost.h                     |  19 +-
 include/hw/virtio/virtio.h                    |   4 +
 include/migration/vmstate.h                   |   7 +
 migration/meson.build                         |   1 +
 migration/options.c                           |   7 +
 migration/options.h                           |   1 +
 migration/vmstate-event-notifier.c            |  54 +++
 qapi/migration.json                           |  11 +-
 ...test_x86_64_vhost_user_blk_fd_migration.py | 307 ++++++++++++++++++
 35 files changed, 1115 insertions(+), 143 deletions(-)
 create mode 100644 chardev/char-backend-transfer.c
 create mode 100644 include/chardev/char-backend-transfer.h
 create mode 100644 migration/vmstate-event-notifier.c
 create mode 100644 tests/functional/test_x86_64_vhost_user_blk_fd_migration.py

-- 
2.48.1

[PATCH v2 01/25] vhost: store busyloop_timeout into struct vhost_dev

[Vladimir Sementsov-Ogievskiy]

We'll split vhost_dev_init() into _init() and _connect(), to be able
to postpone communication with backend, to support backend-transfer
migration of vhost-user-blk in future commit.

So, instead of passing it through parameters, store it in vhost_dev
structure.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c         | 11 +++++------
 include/hw/virtio/vhost.h |  1 +
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 7ba90c24db..9fc6e7ba65 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1501,8 +1501,7 @@ static void vhost_virtqueue_error_notifier(EventNotifier *n)
 }
 
 static int vhost_virtqueue_init(struct vhost_dev *dev,
-                                struct vhost_virtqueue *vq, int n,
-                                bool busyloop_timeout)
+                                struct vhost_virtqueue *vq, int n)
 {
     int vhost_vq_index = dev->vhost_ops->vhost_get_vq_index(dev, n);
     struct vhost_vring_file file = {
@@ -1539,8 +1538,8 @@ static int vhost_virtqueue_init(struct vhost_dev *dev,
                                    vhost_virtqueue_error_notifier);
     }
 
-    if (busyloop_timeout) {
-        r = vhost_virtqueue_set_busyloop_timeout(dev, n, busyloop_timeout);
+    if (dev->busyloop_timeout) {
+        r = vhost_virtqueue_set_busyloop_timeout(dev, n, dev->busyloop_timeout);
         if (r < 0) {
             VHOST_OPS_DEBUG(r, "Failed to set busyloop timeout");
             goto fail_err;
@@ -1628,6 +1627,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
 
     hdev->vdev = NULL;
     hdev->migration_blocker = NULL;
+    hdev->busyloop_timeout = busyloop_timeout;
 
     r = vhost_set_backend_type(hdev, backend_type);
     assert(r >= 0);
@@ -1650,8 +1650,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     }
 
     for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
-        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i,
-                                 busyloop_timeout);
+        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
         if (r < 0) {
             error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
             goto fail;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 1ba1af1d86..f1a7e7b971 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -105,6 +105,7 @@ struct vhost_dev {
     VIRTIO_DECLARE_FEATURES(_features);
     VIRTIO_DECLARE_FEATURES(acked_features);
 
+    uint32_t busyloop_timeout;
     uint64_t max_queues;
     uint64_t backend_cap;
     /* @started: is the vhost device started? */
-- 
2.48.1

[PATCH v2 02/25] vhost: reorder logic in vhost_dev_init()

[Vladimir Sementsov-Ogievskiy]

We are going to split vhost_dev_init() so that the first part will do
early initialization of QEMU structures, but don't communicate with
backend, and the second part will do backend communication. We need
this for future support for backend-transfer migration support for
vhost-user-blk (backend will not be available in the early
initialization point).

With this commit, we simply reorder the logic in vhost_dev_init()
in accordance with idea of further split.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c | 60 +++++++++++++++++++++++------------------------
 1 file changed, 30 insertions(+), 30 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 9fc6e7ba65..551d1687fc 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1637,26 +1637,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
         goto fail;
     }
 
-    r = hdev->vhost_ops->vhost_set_owner(hdev);
-    if (r < 0) {
-        error_setg_errno(errp, -r, "vhost_set_owner failed");
-        goto fail;
-    }
-
-    r = vhost_dev_init_features(hdev);
-    if (r < 0) {
-        error_setg_errno(errp, -r, "vhost_init_features failed");
-        goto fail;
-    }
-
-    for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
-        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
-        if (r < 0) {
-            error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
-            goto fail;
-        }
-    }
-
     hdev->memory_listener = (MemoryListener) {
         .name = "vhost",
         .begin = vhost_begin,
@@ -1677,6 +1657,36 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
         .region_del = vhost_iommu_region_del,
     };
 
+    hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
+    hdev->n_mem_sections = 0;
+    hdev->mem_sections = NULL;
+    hdev->log = NULL;
+    hdev->log_size = 0;
+    hdev->log_enabled = false;
+    hdev->started = false;
+    memory_listener_register(&hdev->memory_listener, &address_space_memory);
+    QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
+
+    r = hdev->vhost_ops->vhost_set_owner(hdev);
+    if (r < 0) {
+        error_setg_errno(errp, -r, "vhost_set_owner failed");
+        goto fail;
+    }
+
+    r = vhost_dev_init_features(hdev);
+    if (r < 0) {
+        error_setg_errno(errp, -r, "vhost_init_features failed");
+        goto fail;
+    }
+
+    for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
+        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
+        if (r < 0) {
+            error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
+            goto fail;
+        }
+    }
+
     if (hdev->migration_blocker == NULL) {
         if (!vhost_dev_has_feature_ex(hdev, VHOST_F_LOG_ALL)) {
             error_setg(&hdev->migration_blocker,
@@ -1694,16 +1704,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
         }
     }
 
-    hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
-    hdev->n_mem_sections = 0;
-    hdev->mem_sections = NULL;
-    hdev->log = NULL;
-    hdev->log_size = 0;
-    hdev->log_enabled = false;
-    hdev->started = false;
-    memory_listener_register(&hdev->memory_listener, &address_space_memory);
-    QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
-
     if (!check_memslots(hdev, errp)) {
         r = -EINVAL;
         goto fail;
-- 
2.48.1

[PATCH v2 03/25] vhost: rework vhost_virtqueue_init()

[Vladimir Sementsov-Ogievskiy]

We are going to split vhost_dev_init() so that the first part will do
early initialization of QEMU structures, but don't communicate with
backend, and the second part will do backend communication. We need
this for future support for backend-transfer migration support for
vhost-user-blk (backend will not be available in the early
initialization point).

With this commit, let's split vhost_virtqueue_init(). The whole function
is mostly about configuring the backend, so this logic will be postponed
until backend become available. The only thing to keep in early
initialization is attaching vhost_dev structure. Let's simply move it to
vhost_dev_init().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 551d1687fc..1998663461 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1500,9 +1500,9 @@ static void vhost_virtqueue_error_notifier(EventNotifier *n)
     }
 }
 
-static int vhost_virtqueue_init(struct vhost_dev *dev,
-                                struct vhost_virtqueue *vq, int n)
+static int vhost_virtqueue_connect(struct vhost_virtqueue *vq, int n)
 {
+    struct vhost_dev *dev = vq->dev;
     int vhost_vq_index = dev->vhost_ops->vhost_get_vq_index(dev, n);
     struct vhost_vring_file file = {
         .index = vhost_vq_index,
@@ -1519,8 +1519,6 @@ static int vhost_virtqueue_init(struct vhost_dev *dev,
         goto fail_call;
     }
 
-    vq->dev = dev;
-
     if (dev->vhost_ops->vhost_set_vring_err) {
         r = event_notifier_init(&vq->error_notifier, 0);
         if (r < 0) {
@@ -1629,6 +1627,10 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     hdev->migration_blocker = NULL;
     hdev->busyloop_timeout = busyloop_timeout;
 
+    for (i = 0; i < hdev->nvqs; ++i) {
+        hdev->vqs[i].dev = hdev;
+    }
+
     r = vhost_set_backend_type(hdev, backend_type);
     assert(r >= 0);
 
@@ -1680,7 +1682,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     }
 
     for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
-        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
+        r = vhost_virtqueue_connect(hdev->vqs + i, hdev->vq_index + i);
         if (r < 0) {
             error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
             goto fail;
-- 
2.48.1

[PATCH v2 04/25] vhost: add connect parameter to vhost_dev_init()

[Vladimir Sementsov-Ogievskiy]

We are going to split vhost_dev_init() so that the first part will do
early initialization of QEMU structures, but don't communicate with
backend, and the second part will do backend communication. We need
this for future support for backend-transfer migration support for
vhost-user-blk (backend will not be available in the early
initialization point).

With this commit we introduce boolean parameter for vhost_dev_init(),
so callers may chose, do they want "init + connect" (which is current
behavior, so all callers pass true), or caller may want "only init",
and call vhost_dev_connect() later. vhost_dev_connect(), as well
as support for connect=false will be added in further commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 backends/cryptodev-vhost.c        |  2 +-
 backends/vhost-user.c             |  2 +-
 hw/block/vhost-user-blk.c         |  2 +-
 hw/net/vhost_net.c                |  2 +-
 hw/scsi/vhost-scsi.c              |  2 +-
 hw/scsi/vhost-user-scsi.c         |  2 +-
 hw/virtio/vdpa-dev.c              |  3 ++-
 hw/virtio/vhost-user-base.c       |  2 +-
 hw/virtio/vhost-user-fs.c         |  2 +-
 hw/virtio/vhost-user-scmi.c       |  2 +-
 hw/virtio/vhost-user-vsock.c      |  2 +-
 hw/virtio/vhost-vsock.c           |  2 +-
 hw/virtio/vhost.c                 | 11 ++++++++++-
 include/hw/virtio/vhost-backend.h |  2 ++
 include/hw/virtio/vhost.h         |  3 ++-
 15 files changed, 27 insertions(+), 14 deletions(-)

diff --git a/backends/cryptodev-vhost.c b/backends/cryptodev-vhost.c
index c6069f4e5b..b4dafb4062 100644
--- a/backends/cryptodev-vhost.c
+++ b/backends/cryptodev-vhost.c
@@ -66,7 +66,7 @@ cryptodev_vhost_init(
     crypto->dev.vq_index = crypto->cc->queue_index * crypto->dev.nvqs;
 
     r = vhost_dev_init(&crypto->dev, options->opaque, options->backend_type, 0,
-                       &local_err);
+                       true, &local_err);
     if (r < 0) {
         error_report_err(local_err);
         goto fail;
diff --git a/backends/vhost-user.c b/backends/vhost-user.c
index 42845329e7..e65ba7b648 100644
--- a/backends/vhost-user.c
+++ b/backends/vhost-user.c
@@ -37,7 +37,7 @@ vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
     b->dev.vqs = g_new0(struct vhost_virtqueue, nvqs);
 
     ret = vhost_dev_init(&b->dev, &b->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
-                         errp);
+                         true, errp);
     if (ret < 0) {
         return -1;
     }
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index a5daed4346..a92426f18c 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -365,7 +365,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
 
     s->vhost_user.supports_config = true;
     ret = vhost_dev_init(&s->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
-                         errp);
+                         true, errp);
     if (ret < 0) {
         return ret;
     }
diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
index 323d117735..c4526974fb 100644
--- a/hw/net/vhost_net.c
+++ b/hw/net/vhost_net.c
@@ -274,7 +274,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
 
     r = vhost_dev_init(&net->dev, options->opaque,
                        options->backend_type, options->busyloop_timeout,
-                       &local_err);
+                       true, &local_err);
     if (r < 0) {
         error_report_err(local_err);
         goto fail;
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index d694a25fe2..d187c705d8 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -278,7 +278,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
     vsc->dev.vq_index = 0;
 
     ret = vhost_dev_init(&vsc->dev, (void *)(uintptr_t)vhostfd,
-                         VHOST_BACKEND_TYPE_KERNEL, 0, errp);
+                         VHOST_BACKEND_TYPE_KERNEL, 0, true, errp);
     if (ret < 0) {
         /*
          * vhost_dev_init calls vhost_dev_cleanup on error, which closes
diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
index 0c80a271d8..e121f2e259 100644
--- a/hw/scsi/vhost-user-scsi.c
+++ b/hw/scsi/vhost-user-scsi.c
@@ -161,7 +161,7 @@ static int vhost_user_scsi_connect(DeviceState *dev, Error **errp)
     vsc->dev.vq_index = 0;
 
     ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
-                         errp);
+                         true, errp);
     if (ret < 0) {
         return ret;
     }
diff --git a/hw/virtio/vdpa-dev.c b/hw/virtio/vdpa-dev.c
index e1a2ff433d..b6b4ee7d38 100644
--- a/hw/virtio/vdpa-dev.c
+++ b/hw/virtio/vdpa-dev.c
@@ -116,7 +116,8 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp)
     v->vdpa.shared->device_fd = v->vhostfd;
     v->vdpa.shared->iova_range = iova_range;
 
-    ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, NULL);
+    ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, true,
+                         NULL);
     if (ret < 0) {
         error_setg(errp, "vhost-vdpa-device: vhost initialization failed: %s",
                    strerror(-ret));
diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
index cf311c3bfc..0768231a88 100644
--- a/hw/virtio/vhost-user-base.c
+++ b/hw/virtio/vhost-user-base.c
@@ -334,7 +334,7 @@ static void vub_device_realize(DeviceState *dev, Error **errp)
 
     /* connect to backend */
     ret = vhost_dev_init(&vub->vhost_dev, &vub->vhost_user,
-                         VHOST_BACKEND_TYPE_USER, 0, errp);
+                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
 
     if (ret < 0) {
         do_vhost_user_cleanup(vdev, vub);
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index e77c69eb12..2a8eead90b 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -256,7 +256,7 @@ static void vuf_device_realize(DeviceState *dev, Error **errp)
     fs->vhost_dev.nvqs = 1 + fs->conf.num_request_queues;
     fs->vhost_dev.vqs = g_new0(struct vhost_virtqueue, fs->vhost_dev.nvqs);
     ret = vhost_dev_init(&fs->vhost_dev, &fs->vhost_user,
-                         VHOST_BACKEND_TYPE_USER, 0, errp);
+                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
     if (ret < 0) {
         goto err_virtio;
     }
diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
index f9264c4374..40e567c18a 100644
--- a/hw/virtio/vhost-user-scmi.c
+++ b/hw/virtio/vhost-user-scmi.c
@@ -253,7 +253,7 @@ static void vu_scmi_device_realize(DeviceState *dev, Error **errp)
     scmi->vhost_dev.vqs = g_new0(struct vhost_virtqueue, scmi->vhost_dev.nvqs);
 
     ret = vhost_dev_init(&scmi->vhost_dev, &scmi->vhost_user,
-                         VHOST_BACKEND_TYPE_USER, 0, errp);
+                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
     if (ret < 0) {
         error_setg_errno(errp, -ret,
                          "vhost-user-scmi: vhost_dev_init() failed");
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 993c287348..b630af0fe7 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -115,7 +115,7 @@ static void vuv_device_realize(DeviceState *dev, Error **errp)
     vhost_dev_set_config_notifier(&vvc->vhost_dev, &vsock_ops);
 
     ret = vhost_dev_init(&vvc->vhost_dev, &vsock->vhost_user,
-                         VHOST_BACKEND_TYPE_USER, 0, errp);
+                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
     if (ret < 0) {
         goto err_virtio;
     }
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index 107d88babe..3a4b2d924d 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -166,7 +166,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp)
     vhost_vsock_common_realize(vdev);
 
     ret = vhost_dev_init(&vvc->vhost_dev, (void *)(uintptr_t)vhostfd,
-                         VHOST_BACKEND_TYPE_KERNEL, 0, errp);
+                         VHOST_BACKEND_TYPE_KERNEL, 0, true, errp);
     if (ret < 0) {
         /*
          * vhostfd is closed by vhost_dev_cleanup, which is called
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 1998663461..f733e98b4a 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1617,7 +1617,7 @@ static bool check_memslots(struct vhost_dev *hdev, Error **errp)
 
 int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    VhostBackendType backend_type, uint32_t busyloop_timeout,
-                   Error **errp)
+                   bool connect, Error **errp)
 {
     int i, r, n_initialized_vqs = 0;
 
@@ -1634,6 +1634,15 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     r = vhost_set_backend_type(hdev, backend_type);
     assert(r >= 0);
 
+    /*
+     * Postponed connect only supported for devices with
+     * .vhost_backend_connect handler
+     */
+    assert(connect || hdev->vhost_ops->vhost_backend_connect);
+
+    /* TDDO: support connect=false */
+    assert(connect);
+
     r = hdev->vhost_ops->vhost_backend_init(hdev, opaque, errp);
     if (r < 0) {
         goto fail;
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index ff94fa1734..d3f055f95e 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -53,6 +53,7 @@ struct vhost_virtqueue;
 
 typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque,
                                   Error **errp);
+typedef int (*vhost_backend_connect)(struct vhost_dev *dev, Error **errp);
 typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
 typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
 
@@ -167,6 +168,7 @@ typedef int (*vhost_check_device_state_op)(struct vhost_dev *dev, Error **errp);
 typedef struct VhostOps {
     VhostBackendType backend_type;
     vhost_backend_init vhost_backend_init;
+    vhost_backend_connect vhost_backend_connect;
     vhost_backend_cleanup vhost_backend_cleanup;
     vhost_backend_memslots_limit vhost_backend_memslots_limit;
     vhost_backend_no_private_memslots_op vhost_backend_no_private_memslots;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index f1a7e7b971..74ed24232e 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -155,7 +155,8 @@ struct vhost_net {
  */
 int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    VhostBackendType backend_type,
-                   uint32_t busyloop_timeout, Error **errp);
+                   uint32_t busyloop_timeout,
+                   bool connect, Error **errp);
 
 /**
  * vhost_dev_cleanup() - tear down and cleanup vhost interface
-- 
2.48.1

[PATCH v2 05/25] vhost: split vhost_dev_connect() out of vhost_dev_init()

[Vladimir Sementsov-Ogievskiy]

Split vhost_dev_init() so that the first part will do early
initialization of QEMU structures, but don't communicate with backend,
and the second part will do backend communication.

We need this for future support for backend-transfer migration support
for vhost-user-blk (backend will not be available in the early
initialization point).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c         | 27 +++++++++++++++++++++------
 include/hw/virtio/vhost.h |  2 ++
 2 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index f733e98b4a..09d00e4d98 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1619,7 +1619,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    VhostBackendType backend_type, uint32_t busyloop_timeout,
                    bool connect, Error **errp)
 {
-    int i, r, n_initialized_vqs = 0;
+    int i, r;
 
     trace_vhost_dev_init_in(hdev);
 
@@ -1640,9 +1640,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
      */
     assert(connect || hdev->vhost_ops->vhost_backend_connect);
 
-    /* TDDO: support connect=false */
-    assert(connect);
-
     r = hdev->vhost_ops->vhost_backend_init(hdev, opaque, errp);
     if (r < 0) {
         goto fail;
@@ -1678,6 +1675,26 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
     memory_listener_register(&hdev->memory_listener, &address_space_memory);
     QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
 
+    trace_vhost_dev_init_out(hdev);
+
+    return connect ? vhost_dev_connect(hdev, errp) : 0;
+
+fail:
+    vhost_dev_cleanup(hdev);
+    return r;
+}
+
+int vhost_dev_connect(struct vhost_dev *hdev, Error **errp)
+{
+    int i, r, n_initialized_vqs = 0;
+
+    if (hdev->vhost_ops->vhost_backend_connect) {
+        r = hdev->vhost_ops->vhost_backend_connect(hdev, errp);
+        if (r < 0) {
+            goto fail;
+        }
+    }
+
     r = hdev->vhost_ops->vhost_set_owner(hdev);
     if (r < 0) {
         error_setg_errno(errp, -r, "vhost_set_owner failed");
@@ -1720,8 +1737,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
         goto fail;
     }
 
-    trace_vhost_dev_init_out(hdev);
-
     return 0;
 
 fail:
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 74ed24232e..af46d4b5f2 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -158,6 +158,8 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
                    uint32_t busyloop_timeout,
                    bool connect, Error **errp);
 
+int vhost_dev_connect(struct vhost_dev *hdev, Error **errp);
+
 /**
  * vhost_dev_cleanup() - tear down and cleanup vhost interface
  * @hdev: the common vhost_dev structure
-- 
2.48.1

[PATCH v2 06/25] vhost-user: support connect api

[Vladimir Sementsov-Ogievskiy]

Memory allocation and connecting of structures remain ins _init(),
communication with backend goes to _connect().

We need this for further support of backend-transfer migration of
vhost-user-blk, as we'll need to postpone (or not do) initial
communication to backend.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost-user.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e45b74eddd..c5cb5ed528 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -2270,21 +2270,12 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
     return 0;
 }
 
-static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
-                                   Error **errp)
+static int vhost_user_backend_connect(struct vhost_dev *dev, Error **errp)
 {
     uint64_t features, ram_slots;
-    struct vhost_user *u;
-    VhostUserState *vus = (VhostUserState *) opaque;
+    struct vhost_user *u = dev->opaque;
     int err;
 
-    assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
-
-    u = g_new0(struct vhost_user, 1);
-    u->user = vus;
-    u->dev = dev;
-    dev->opaque = u;
-
     err = vhost_user_get_features(dev, &features);
     if (err < 0) {
         error_setg_errno(errp, -err, "vhost_backend_init failed");
@@ -2292,7 +2283,7 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
     }
 
     if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) {
-        bool supports_f_config = vus->supports_config ||
+        bool supports_f_config = u->user->supports_config ||
             (dev->config_ops && dev->config_ops->vhost_dev_config_notifier);
         uint64_t protocol_features;
 
@@ -2408,6 +2399,22 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
     return 0;
 }
 
+static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
+                                   Error **errp)
+{
+    struct vhost_user *u;
+    VhostUserState *vus = (VhostUserState *) opaque;
+
+    assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
+
+    u = g_new0(struct vhost_user, 1);
+    u->user = vus;
+    u->dev = dev;
+    dev->opaque = u;
+
+    return 0;
+}
+
 static int vhost_user_backend_cleanup(struct vhost_dev *dev)
 {
     struct vhost_user *u;
@@ -3133,6 +3140,7 @@ void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status)
 const VhostOps user_ops = {
         .backend_type = VHOST_BACKEND_TYPE_USER,
         .vhost_backend_init = vhost_user_backend_init,
+        .vhost_backend_connect = vhost_user_backend_connect,
         .vhost_backend_cleanup = vhost_user_backend_cleanup,
         .vhost_backend_memslots_limit = vhost_user_memslots_limit,
         .vhost_backend_no_private_memslots = vhost_user_no_private_memslots,
-- 
2.48.1

[PATCH v2 07/25] vhost-user-blk: vhost_user_blk_connect() move connected check to caller

[Vladimir Sementsov-Ogievskiy]

vhost_user_blk_connect() has two callers:

- vhost_user_blk_realize_connect(), which directly set .connected = false
before call

- vhost_user_blk_event(), where we want this check

Move the check to the only caller which needs it, to simplify further
refactoring.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index a92426f18c..57214a69cd 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -352,9 +352,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
 
     trace_vhost_user_blk_connect_in(vdev);
 
-    if (s->connected) {
-        return 0;
-    }
+    assert(!s->connected);
 
     s->dev.num_queues = s->num_queues;
     s->dev.nvqs = s->num_queues;
@@ -411,10 +409,12 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event)
 
     switch (event) {
     case CHR_EVENT_OPENED:
-        if (vhost_user_blk_connect(dev, &local_err) < 0) {
-            error_report_err(local_err);
-            qemu_chr_fe_disconnect(&s->chardev);
-            return;
+        if (!s->connected) {
+            if (vhost_user_blk_connect(dev, &local_err) < 0) {
+                error_report_err(local_err);
+                qemu_chr_fe_disconnect(&s->chardev);
+                return;
+            }
         }
         break;
     case CHR_EVENT_CLOSED:
-- 
2.48.1

[PATCH v2 08/25] vhost-user-blk: vhost_user_blk_connect(): call vhost_dev_connect()

[Vladimir Sementsov-Ogievskiy]

Call vhost_dev_connect() directly, to simplify further refactoring.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 57214a69cd..f2ecf81e4d 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -363,7 +363,12 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
 
     s->vhost_user.supports_config = true;
     ret = vhost_dev_init(&s->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
-                         true, errp);
+                         false, errp);
+    if (ret < 0) {
+        return ret;
+    }
+
+    ret = vhost_dev_connect(&s->dev, errp);
     if (ret < 0) {
         return ret;
     }
-- 
2.48.1

[PATCH v2 09/25] vhost-user-blk: rename vhost_user_blk_connect to vhost_user_blk_init

[Vladimir Sementsov-Ogievskiy]

The function does both vhost_dev_init() and vhost_dev_connect().
Following interface of vhost_dev_init(), and preparing to further
refactoring, let's rename to _init() and add boolean "connect"
parameter.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/trace-events     |  2 ++
 hw/block/vhost-user-blk.c | 13 ++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/hw/block/trace-events b/hw/block/trace-events
index dbaa5ca6cb..9f00412a99 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -63,6 +63,8 @@ vhost_user_blk_start_in(void *vdev) "vdev %p"
 vhost_user_blk_start_out(void *vdev) "vdev %p"
 vhost_user_blk_stop_in(void *vdev) "vdev %p"
 vhost_user_blk_stop_out(void *vdev) "vdev %p"
+vhost_user_blk_init_in(void *vdev) "vdev %p"
+vhost_user_blk_init_out(void *vdev) "vdev %p"
 vhost_user_blk_connect_in(void *vdev) "vdev %p"
 vhost_user_blk_connect_out(void *vdev) "vdev %p"
 vhost_user_blk_device_realize_in(void *vdev) "vdev %p"
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index f2ecf81e4d..c31c265a0e 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -344,13 +344,16 @@ static void vhost_user_blk_reset(VirtIODevice *vdev)
     vhost_dev_free_inflight(s->inflight);
 }
 
-static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
+static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VHostUserBlk *s = VHOST_USER_BLK(vdev);
     int ret = 0;
 
-    trace_vhost_user_blk_connect_in(vdev);
+    trace_vhost_user_blk_init_in(vdev);
+
+    /* TODO: implement support for connect=false */
+    assert(connect);
 
     assert(!s->connected);
 
@@ -380,7 +383,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
         ret = vhost_user_blk_start(vdev, errp);
     }
 
-    trace_vhost_user_blk_connect_out(vdev);
+    trace_vhost_user_blk_init_out(vdev);
 
     return ret;
 }
@@ -415,7 +418,7 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event)
     switch (event) {
     case CHR_EVENT_OPENED:
         if (!s->connected) {
-            if (vhost_user_blk_connect(dev, &local_err) < 0) {
+            if (vhost_user_blk_init(dev, true, &local_err) < 0) {
                 error_report_err(local_err);
                 qemu_chr_fe_disconnect(&s->chardev);
                 return;
@@ -447,7 +450,7 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
         return ret;
     }
 
-    ret = vhost_user_blk_connect(dev, errp);
+    ret = vhost_user_blk_init(dev, true, errp);
     if (ret < 0) {
         qemu_chr_fe_disconnect(&s->chardev);
         return ret;
-- 
2.48.1

[PATCH v2 10/25] vhost-user-blk: split vhost_user_blk_init()

[Vladimir Sementsov-Ogievskiy]

Split it into _init() and _connect() part, following pattern of
vhost_dev_init / vhost_dev_connect.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index c31c265a0e..9c727c3977 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -58,6 +58,7 @@ static const int user_feature_bits[] = {
 };
 
 static void vhost_user_blk_event(void *opaque, QEMUChrEvent event);
+static int vhost_user_blk_connect(DeviceState *dev, Error **errp);
 
 static void vhost_user_blk_update_config(VirtIODevice *vdev, uint8_t *config)
 {
@@ -352,9 +353,6 @@ static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
 
     trace_vhost_user_blk_init_in(vdev);
 
-    /* TODO: implement support for connect=false */
-    assert(connect);
-
     assert(!s->connected);
 
     s->dev.num_queues = s->num_queues;
@@ -371,6 +369,29 @@ static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
         return ret;
     }
 
+    if (connect) {
+        ret = vhost_user_blk_connect(dev, errp);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
+    trace_vhost_user_blk_init_out(vdev);
+
+    return 0;
+}
+
+static int vhost_user_blk_connect(DeviceState *dev,
+                                  Error **errp)
+{
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserBlk *s = VHOST_USER_BLK(vdev);
+    int ret = 0;
+
+    trace_vhost_user_blk_connect_in(vdev);
+
+    assert(!s->connected);
+
     ret = vhost_dev_connect(&s->dev, errp);
     if (ret < 0) {
         return ret;
-- 
2.48.1

[PATCH v2 11/25] vhost-user-blk: move initial reconnect loop to separate function

[Vladimir Sementsov-Ogievskiy]

Simplify _realize function, and prepare to further changes.

While being here, also rename virtio_err: label to more generic
fail:, virtio_err doesn't improve readability here.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 54 +++++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 22 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 9c727c3977..36e32229ad 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -489,14 +489,40 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
     return 0;
 }
 
-static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
+static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
 {
     ERRP_GUARD();
+    DeviceState *dev = DEVICE(s);
+    int ret, retries = VU_REALIZE_CONN_RETRIES;
+
+    assert(!*errp);
+    do {
+        if (*errp) {
+            error_prepend(errp, "Reconnecting after error: ");
+            error_report_err(*errp);
+            *errp = NULL;
+        }
+        ret = vhost_user_blk_realize_connect(s, errp);
+    } while (ret < 0 && retries--);
+
+    if (ret < 0) {
+        return ret;
+    }
+
+    /* we're fully initialized, now we can operate, so add the handler */
+    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
+                             vhost_user_blk_event, NULL, (void *)dev,
+                             NULL, true);
+
+    return 0;
+}
+
+static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
+{
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VHostUserBlk *s = VHOST_USER_BLK(vdev);
     size_t config_size;
-    int retries;
-    int i, ret;
+    int i;
 
     trace_vhost_user_blk_device_realize_in(vdev);
 
@@ -540,31 +566,15 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
     s->inflight = g_new0(struct vhost_inflight, 1);
     s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
 
-    retries = VU_REALIZE_CONN_RETRIES;
-    assert(!*errp);
-    do {
-        if (*errp) {
-            error_prepend(errp, "Reconnecting after error: ");
-            error_report_err(*errp);
-            *errp = NULL;
-        }
-        ret = vhost_user_blk_realize_connect(s, errp);
-    } while (ret < 0 && retries--);
-
-    if (ret < 0) {
-        goto virtio_err;
+    if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
+        goto fail;
     }
 
-    /* we're fully initialized, now we can operate, so add the handler */
-    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
-                             vhost_user_blk_event, NULL, (void *)dev,
-                             NULL, true);
-
     trace_vhost_user_blk_device_realize_out(vdev);
 
     return;
 
-virtio_err:
+fail:
     g_free(s->vhost_vqs);
     s->vhost_vqs = NULL;
     g_free(s->inflight);
-- 
2.48.1

[PATCH v2 12/25] vhost-user-blk: move first vhost_user_blk_init() to _realize()

[Vladimir Sementsov-Ogievskiy]

We'll need to postpone further connecting/reconnecting logic to the
later point to support backend-transfer migration for vhost-user-blk.
For now, move first call to vhost_user_blk_init() to _realize() (this
call will not be postponed). To support this, we also have to move
re-initialization to vhost_user_blk_realize_connect_loop().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 36e32229ad..af4a97b8e4 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -464,14 +464,12 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
     DeviceState *dev = DEVICE(s);
     int ret;
 
-    s->connected = false;
-
     ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
     if (ret < 0) {
         return ret;
     }
 
-    ret = vhost_user_blk_init(dev, true, errp);
+    ret = vhost_user_blk_connect(dev, errp);
     if (ret < 0) {
         qemu_chr_fe_disconnect(&s->chardev);
         return ret;
@@ -501,7 +499,16 @@ static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
             error_prepend(errp, "Reconnecting after error: ");
             error_report_err(*errp);
             *errp = NULL;
+
+            s->connected = false;
+
+            ret = vhost_user_blk_init(dev, false, errp);
+            if (ret < 0) {
+                /* No reason to retry initialization */
+                return ret;
+            }
         }
+
         ret = vhost_user_blk_realize_connect(s, errp);
     } while (ret < 0 && retries--);
 
@@ -566,6 +573,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
     s->inflight = g_new0(struct vhost_inflight, 1);
     s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
 
+    if (vhost_user_blk_init(dev, false, errp) < 0) {
+        goto fail;
+    }
+
     if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
         goto fail;
     }
-- 
2.48.1

[PATCH v2 13/25] vhost-user-blk: postpone connect to pre-incoming

[Vladimir Sementsov-Ogievskiy]

That's a preparation for further backend-transfer migration for
vhost-user-blk. At initialization time we don't know will
user enable backent-transfer (by setting migration parameter) or
not. At time of pre-incoming, we know all migration parameters and
capabilities. So, now, let's postpone connecting up to pre-incoming
for incoming migration.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index af4a97b8e4..ffdd600526 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -577,8 +577,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
         goto fail;
     }
 
-    if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
-        goto fail;
+    if (!runstate_check(RUN_STATE_INMIGRATE)) {
+        if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
+            goto fail;
+        }
     }
 
     trace_vhost_user_blk_device_realize_out(vdev);
@@ -636,10 +638,18 @@ static struct vhost_dev *vhost_user_blk_get_vhost(VirtIODevice *vdev)
     return &s->dev;
 }
 
+static bool vhost_user_blk_pre_incoming(void *opaque, Error **errp)
+{
+    VHostUserBlk *s = VHOST_USER_BLK(opaque);
+
+    return vhost_user_blk_realize_connect(s, errp) == 0;
+}
+
 static const VMStateDescription vmstate_vhost_user_blk = {
     .name = "vhost-user-blk",
     .minimum_version_id = 1,
     .version_id = 1,
+    .pre_incoming = vhost_user_blk_pre_incoming,
     .fields = (const VMStateField[]) {
         VMSTATE_VIRTIO_DEVICE,
         VMSTATE_END_OF_LIST()
@@ -647,7 +657,7 @@ static const VMStateDescription vmstate_vhost_user_blk = {
 };
 
 static const Property vhost_user_blk_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev),
+    DEFINE_PROP_CHR_NO_CONNECT("chardev", VHostUserBlk, chardev),
     DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
                        VHOST_USER_BLK_AUTO_NUM_QUEUES),
     DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128),
-- 
2.48.1

[PATCH v2 14/25] virtio: introduce .skip_vhost_migration_log() handler

[Vladimir Sementsov-Ogievskiy]

For vhost user backend migration we'll need to disable memory
logging on the device. Let's prepare a corresponding handler for
the device.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c          | 10 ++++++++++
 include/hw/virtio/virtio.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 09d00e4d98..3e5192ec23 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1138,6 +1138,16 @@ static int vhost_migration_log(MemoryListener *listener, bool enable)
     struct vhost_dev *dev = container_of(listener, struct vhost_dev,
                                          memory_listener);
     int r;
+
+    if (dev->vdev) {
+        VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(dev->vdev);
+
+        if (vdc->skip_vhost_migration_log &&
+            vdc->skip_vhost_migration_log(dev->vdev)) {
+            return 0;
+        }
+    }
+
     if (enable == dev->log_enabled) {
         return 0;
     }
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 7db8046766..620ee4e389 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -238,6 +238,8 @@ struct VirtioDeviceClass {
     /* May be called even when vdev->vhost_started is false */
     struct vhost_dev *(*get_vhost)(VirtIODevice *vdev);
     void (*toggle_device_iotlb)(VirtIODevice *vdev);
+
+    bool (*skip_vhost_migration_log)(VirtIODevice *vdev);
 };
 
 void virtio_instance_init_common(Object *proxy_obj, void *data,
-- 
2.48.1

[PATCH v2 15/25] migration: introduce vmstate_event_notifier

[Vladimir Sementsov-Ogievskiy]

This will be used to support backend-transfer migration for
vhost-user-blk, we'll migrate event notifier fds through
migration stream, to avoid extra contact with backend.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 include/migration/vmstate.h        |  7 ++++
 migration/meson.build              |  1 +
 migration/vmstate-event-notifier.c | 54 ++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+)
 create mode 100644 migration/vmstate-event-notifier.c

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index f243518fb5..7f1f1c166a 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -1294,4 +1294,11 @@ void vmstate_register_ram_global(struct MemoryRegion *memory);
 
 bool vmstate_check_only_migratable(const VMStateDescription *vmsd);
 
+extern const VMStateDescription vmstate_event_notifier;
+
+#define VMSTATE_EVENT_NOTIFIER(_field, _struct) \
+    VMSTATE_STRUCT(_field, _struct, 0, vmstate_event_notifier, \
+                   EventNotifier)
+
+
 #endif
diff --git a/migration/meson.build b/migration/meson.build
index 16909d54c5..b5341ae0cb 100644
--- a/migration/meson.build
+++ b/migration/meson.build
@@ -5,6 +5,7 @@ migration_files = files(
   'xbzrle.c',
   'vmstate-types.c',
   'vmstate.c',
+  'vmstate-event-notifier.c',
   'qemu-file.c',
   'yank_functions.c',
 )
diff --git a/migration/vmstate-event-notifier.c b/migration/vmstate-event-notifier.c
new file mode 100644
index 0000000000..2076eec961
--- /dev/null
+++ b/migration/vmstate-event-notifier.c
@@ -0,0 +1,54 @@
+/*
+ * Event notifier migration support
+ * Copyright (c) Yandex Technologies LLC, 2025
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/event_notifier.h"
+#include "migration/vmstate.h"
+
+static int event_notifier_pre_save(void *opaque)
+{
+    struct EventNotifier *e = opaque;
+
+    if (!e->initialized || e->rfd != e->wfd) {
+        return -1;
+    }
+
+    return 0;
+}
+
+static int event_notifier_pre_load(void *opaque)
+{
+    struct EventNotifier *e = opaque;
+
+    if (e->initialized) {
+        return -1;
+    }
+
+    return 0;
+}
+
+static int event_notifier_post_load(void *opaque, int version_id)
+{
+    struct EventNotifier *e = opaque;
+
+    if (e->rfd < 0) {
+        return -1;
+    }
+
+    e->wfd = e->rfd;
+    e->initialized = true;
+
+    return 0;
+}
+
+const VMStateDescription vmstate_event_notifier = {
+    .name = "event-notifier",
+    .pre_save = event_notifier_pre_save,
+    .pre_load = event_notifier_pre_load,
+    .post_load = event_notifier_post_load,
+    .fields = (const VMStateField[]){VMSTATE_FD(rfd, EventNotifier),
+                                     VMSTATE_END_OF_LIST()},
+};
-- 
2.48.1

[PATCH v2 16/25] chardev: add .chr_get_client() handler

[Vladimir Sementsov-Ogievskiy]

A pair for .char_add_client(), to be used to support backend-transfer
migration of chardev attached to vhost-user-blk in following commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 chardev/char-socket.c  | 7 +++++++
 chardev/char.c         | 6 ++++++
 include/chardev/char.h | 4 ++++
 3 files changed, 17 insertions(+)

diff --git a/chardev/char-socket.c b/chardev/char-socket.c
index 0a5738c158..51bb9d0a2d 100644
--- a/chardev/char-socket.c
+++ b/chardev/char-socket.c
@@ -929,6 +929,12 @@ static int tcp_chr_new_client(Chardev *chr, QIOChannelSocket *sioc)
     return 0;
 }
 
+static int tcp_chr_get_client(Chardev *chr)
+{
+    SocketChardev *s = SOCKET_CHARDEV(chr);
+
+    return s->sioc->fd;
+}
 
 static int tcp_chr_add_client(Chardev *chr, int fd)
 {
@@ -1591,6 +1597,7 @@ static void char_socket_class_init(ObjectClass *oc, const void *data)
     cc->get_msgfds = tcp_get_msgfds;
     cc->set_msgfds = tcp_set_msgfds;
     cc->chr_add_client = tcp_chr_add_client;
+    cc->chr_get_client = tcp_chr_get_client;
     cc->chr_add_watch = tcp_chr_add_watch;
     cc->chr_update_read_handler = tcp_chr_update_read_handler;
 
diff --git a/chardev/char.c b/chardev/char.c
index b68d44e394..3c2ed972f1 100644
--- a/chardev/char.c
+++ b/chardev/char.c
@@ -247,6 +247,12 @@ int qemu_chr_add_client(Chardev *s, int fd)
         CHARDEV_GET_CLASS(s)->chr_add_client(s, fd) : -1;
 }
 
+int qemu_chr_get_client(Chardev *s)
+{
+    return CHARDEV_GET_CLASS(s)->chr_get_client ?
+        CHARDEV_GET_CLASS(s)->chr_get_client(s) : -1;
+}
+
 static bool qemu_char_open(Chardev *chr, ChardevBackend *backend,
                            const char *default_filename, Error **errp)
 {
diff --git a/include/chardev/char.h b/include/chardev/char.h
index d2e01f0f9c..671f64cb7e 100644
--- a/include/chardev/char.h
+++ b/include/chardev/char.h
@@ -216,6 +216,7 @@ void qemu_chr_be_update_read_handlers(Chardev *s,
 void qemu_chr_be_event(Chardev *s, QEMUChrEvent event);
 
 int qemu_chr_add_client(Chardev *s, int fd);
+int qemu_chr_get_client(Chardev *s);
 Chardev *qemu_chr_find(const char *name);
 
 bool qemu_chr_has_feature(Chardev *chr,
@@ -316,6 +317,9 @@ struct ChardevClass {
     /* accept the given fd */
     int (*chr_add_client)(Chardev *chr, int fd);
 
+    /* get fd back */
+    int (*chr_get_client)(Chardev *chr);
+
     /* wait for a connection */
     int (*chr_wait_connected)(Chardev *chr, Error **errp);
 
-- 
2.48.1

[PATCH v2 17/25] vhost: add inflight region backend-transfer vmstate

[Vladimir Sementsov-Ogievskiy]

Prepare for future backend-transfer migration of vhost-user-blk.
Among other things we'll need to transfer the inflight region, let's
prepare for this.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c         | 26 ++++++++++++++++++++++++++
 include/hw/virtio/vhost.h |  5 +++++
 2 files changed, 31 insertions(+)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 3e5192ec23..63036f8214 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1975,6 +1975,32 @@ void vhost_get_features_ex(struct vhost_dev *hdev,
     }
 }
 
+static int vhost_inflight_backend_transfer_post_load(void *opaque,
+                                                     int version_id)
+{
+    struct vhost_inflight *inflight = opaque;
+
+    inflight->addr = mmap(0, inflight->size, PROT_READ | PROT_WRITE,
+                             MAP_SHARED, inflight->fd, inflight->offset);
+    if (inflight->addr == MAP_FAILED) {
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
+const VMStateDescription vmstate_backend_transfer_vhost_inflight = {
+    .name = "vhost-inflight",
+    .post_load = vhost_inflight_backend_transfer_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_FD(fd, struct vhost_inflight),
+        VMSTATE_UINT64(size, struct vhost_inflight),
+        VMSTATE_UINT64(offset, struct vhost_inflight),
+        VMSTATE_UINT16(queue_size, struct vhost_inflight),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
                            const uint64_t *features)
 {
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index af46d4b5f2..94a0c75fc8 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -587,4 +587,9 @@ static inline int vhost_load_backend_state(struct vhost_dev *dev, QEMUFile *f,
 }
 #endif
 
+extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
+#define VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(_field, _state) \
+    VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
+                           struct vhost_inflight)
+
 #endif
-- 
2.48.1

[PATCH v2 18/25] chardev: introduce backend-transfer vmstate for chardev

[Vladimir Sementsov-Ogievskiy]

We'll need to transfer the chardev attached to vhost-user-blk, to
support backend-transfer migration for vhost-user-blk. So, prepare
chardev vmsd now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 chardev/char-backend-transfer.c         | 52 +++++++++++++++++++++++++
 chardev/meson.build                     |  1 +
 include/chardev/char-backend-transfer.h | 17 ++++++++
 3 files changed, 70 insertions(+)
 create mode 100644 chardev/char-backend-transfer.c
 create mode 100644 include/chardev/char-backend-transfer.h

diff --git a/chardev/char-backend-transfer.c b/chardev/char-backend-transfer.c
new file mode 100644
index 0000000000..f1a399c7fa
--- /dev/null
+++ b/chardev/char-backend-transfer.c
@@ -0,0 +1,52 @@
+/*
+ * Event notifier migration support
+ * Copyright (c) Yandex Technologies LLC, 2025
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "chardev/char-fe.h"
+#include "migration/vmstate.h"
+
+typedef struct CharBackendTransferTmp {
+    CharBackend *parent;
+    int fd;
+} CharBackendTransferTmp;
+
+static int char_backend_transfer_pre_save(void *opaque)
+{
+    CharBackendTransferTmp *tmp = opaque;
+
+    tmp->fd = qemu_chr_get_client(tmp->parent->chr);
+    if (tmp->fd < 0) {
+        return -1;
+    }
+
+    return 0;
+}
+
+static int char_backend_transfer_post_load(void *opaque, int version_id)
+{
+    CharBackendTransferTmp *tmp = opaque;
+
+    return qemu_chr_add_client(tmp->parent->chr, tmp->fd);
+}
+
+const VMStateDescription vmstate_backend_transfer_char_tmp = {
+    .name = "backend-transfer-char-tmp",
+    .pre_save = char_backend_transfer_pre_save,
+    .post_load = char_backend_transfer_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_FD(fd, CharBackendTransferTmp),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
+const VMStateDescription vmstate_backend_transfer_char = {
+    .name = "backend-transfer-char",
+    .fields = (const VMStateField[]) {
+        VMSTATE_WITH_TMP(CharBackend, CharBackendTransferTmp,
+                         vmstate_backend_transfer_char_tmp),
+        VMSTATE_END_OF_LIST()
+    },
+};
diff --git a/chardev/meson.build b/chardev/meson.build
index 56ee39ac0b..239c8cd072 100644
--- a/chardev/meson.build
+++ b/chardev/meson.build
@@ -31,6 +31,7 @@ chardev_ss = chardev_ss.apply({})
 
 system_ss.add(files(
     'char-hmp-cmds.c',
+    'char-backend-transfer.c',
     'msmouse.c',
     'wctablet.c',
     'testdev.c'))
diff --git a/include/chardev/char-backend-transfer.h b/include/chardev/char-backend-transfer.h
new file mode 100644
index 0000000000..2c3da5f836
--- /dev/null
+++ b/include/chardev/char-backend-transfer.h
@@ -0,0 +1,17 @@
+/*
+ * Event notifier migration support
+ * Copyright (c) Yandex Technologies LLC, 2025
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef CHAR_BACKEND_TRANSFER_H
+#define CHAR_BACKEND_TRANSFER_H
+
+#include "qemu/typedefs.h"
+
+extern const VMStateDescription vmstate_backend_transfer_char;
+#define VMSTATE_BACKEND_TRANSFER_CHARDEV(_field, _state)                       \
+    VMSTATE_STRUCT(_field, _state, 0, vmstate_backend_transfer_char,           \
+                   CharBackend)
+
+#endif
-- 
2.48.1

[PATCH v2 19/25] vhost: support backend-transfer migration

[Vladimir Sementsov-Ogievskiy]

Introduce vhost_dev.backend_transfer field,

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost.c         | 121 +++++++++++++++++++++++++++++++++-----
 include/hw/virtio/vhost.h |   7 +++
 2 files changed, 113 insertions(+), 15 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index 63036f8214..c46203eb9c 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -1325,6 +1325,8 @@ out:
     return ret;
 }
 
+static void vhost_virtqueue_error_notifier(EventNotifier *n);
+
 int vhost_virtqueue_start(struct vhost_dev *dev,
                           struct VirtIODevice *vdev,
                           struct vhost_virtqueue *vq,
@@ -1350,7 +1352,13 @@ int vhost_virtqueue_start(struct vhost_dev *dev,
         return r;
     }
 
-    vq->num = state.num = virtio_queue_get_num(vdev, idx);
+    vq->num = virtio_queue_get_num(vdev, idx);
+
+    if (dev->backend_transfer) {
+        return 0;
+    }
+
+    state.num = vq->num;
     r = dev->vhost_ops->vhost_set_vring_num(dev, &state);
     if (r) {
         VHOST_OPS_DEBUG(r, "vhost_set_vring_num failed");
@@ -1428,6 +1436,10 @@ static int do_vhost_virtqueue_stop(struct vhost_dev *dev,
 
     trace_vhost_virtque_stop_in(dev, vdev->name, idx);
 
+    if (dev->backend_transfer) {
+        return 0;
+    }
+
     if (virtio_queue_get_desc_addr(vdev, idx) == 0) {
         /* Don't stop the virtqueue which might have not been started */
         return 0;
@@ -1565,10 +1577,14 @@ fail_call:
 
 static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
 {
-    event_notifier_cleanup(&vq->masked_notifier);
+    if (!vq->dev->backend_transfer) {
+        event_notifier_cleanup(&vq->masked_notifier);
+    }
     if (vq->dev->vhost_ops->vhost_set_vring_err) {
         event_notifier_set_handler(&vq->error_notifier, NULL);
-        event_notifier_cleanup(&vq->error_notifier);
+        if (!vq->dev->backend_transfer) {
+            event_notifier_cleanup(&vq->error_notifier);
+        }
     }
 }
 
@@ -1635,6 +1651,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
 
     hdev->vdev = NULL;
     hdev->migration_blocker = NULL;
+    hdev->_features_wait_incoming = true;
     hdev->busyloop_timeout = busyloop_timeout;
 
     for (i = 0; i < hdev->nvqs; ++i) {
@@ -1717,6 +1734,8 @@ int vhost_dev_connect(struct vhost_dev *hdev, Error **errp)
         goto fail;
     }
 
+    hdev->_features_wait_incoming = false;
+
     for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
         r = vhost_virtqueue_connect(hdev->vqs + i, hdev->vq_index + i);
         if (r < 0) {
@@ -1808,8 +1827,11 @@ void vhost_dev_disable_notifiers_nvqs(struct vhost_dev *hdev,
      */
     memory_region_transaction_commit();
 
-    for (i = 0; i < nvqs; ++i) {
-        virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i);
+    if (!hdev->backend_transfer) {
+        for (i = 0; i < nvqs; ++i) {
+            virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus),
+                                             hdev->vq_index + i);
+        }
     }
     virtio_device_release_ioeventfd(vdev);
 }
@@ -1967,6 +1989,11 @@ void vhost_get_features_ex(struct vhost_dev *hdev,
 {
     const int *bit = feature_bits;
 
+    if (hdev->_features_wait_incoming) {
+        /* Excessive set is enough for early initialization. */
+        return;
+    }
+
     while (*bit != VHOST_INVALID_FEATURE_BIT) {
         if (!vhost_dev_has_feature_ex(hdev, *bit)) {
             virtio_clear_feature_ex(features, *bit);
@@ -2001,6 +2028,54 @@ const VMStateDescription vmstate_backend_transfer_vhost_inflight = {
     }
 };
 
+const VMStateDescription vmstate_vhost_virtqueue = {
+    .name = "vhost-virtqueue",
+    .fields = (const VMStateField[]) {
+        VMSTATE_EVENT_NOTIFIER(error_notifier, struct vhost_virtqueue),
+        VMSTATE_EVENT_NOTIFIER(masked_notifier, struct vhost_virtqueue),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
+static int vhost_dev_post_load(void *opaque, int version_id)
+{
+    struct vhost_dev *hdev = opaque;
+    Error *err = NULL;
+    int i;
+
+    if (!check_memslots(hdev, &err)) {
+        error_report_err(err);
+        return -EINVAL;
+    }
+
+    hdev->_features_wait_incoming = false;
+
+    if (hdev->vhost_ops->vhost_set_vring_err) {
+        for (i = 0; i < hdev->nvqs; ++i) {
+            event_notifier_set_handler(&hdev->vqs[i].error_notifier,
+                                       vhost_virtqueue_error_notifier);
+        }
+    }
+
+
+    return 0;
+}
+
+const VMStateDescription vmstate_vhost_dev = {
+    .name = "vhost-dev",
+    .post_load = vhost_dev_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT64(_features, struct vhost_dev),
+        VMSTATE_UINT64(max_queues, struct vhost_dev),
+        VMSTATE_UINT32_EQUAL(nvqs, struct vhost_dev, NULL),
+        VMSTATE_STRUCT_VARRAY_POINTER_UINT32(vqs, struct vhost_dev,
+                                             nvqs,
+                                             vmstate_vhost_virtqueue,
+                                             struct vhost_virtqueue),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
                            const uint64_t *features)
 {
@@ -2127,19 +2202,24 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
     hdev->started = true;
     hdev->vdev = vdev;
 
-    r = vhost_dev_set_features(hdev, hdev->log_enabled);
-    if (r < 0) {
-        goto fail_features;
+    if (!hdev->backend_transfer) {
+        r = vhost_dev_set_features(hdev, hdev->log_enabled);
+        if (r < 0) {
+            warn_report("%s %d", __func__, __LINE__);
+            goto fail_features;
+        }
     }
 
     if (vhost_dev_has_iommu(hdev)) {
         memory_listener_register(&hdev->iommu_listener, vdev->dma_as);
     }
 
-    r = hdev->vhost_ops->vhost_set_mem_table(hdev, hdev->mem);
-    if (r < 0) {
-        VHOST_OPS_DEBUG(r, "vhost_set_mem_table failed");
-        goto fail_mem;
+    if (!hdev->backend_transfer) {
+        r = hdev->vhost_ops->vhost_set_mem_table(hdev, hdev->mem);
+        if (r < 0) {
+            VHOST_OPS_DEBUG(r, "vhost_set_mem_table failed");
+            goto fail_mem;
+        }
     }
     for (i = 0; i < hdev->nvqs; ++i) {
         r = vhost_virtqueue_start(hdev,
@@ -2179,13 +2259,13 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
         }
         vhost_dev_elect_mem_logger(hdev, true);
     }
-    if (vrings) {
+    if (vrings && !hdev->backend_transfer) {
         r = vhost_dev_set_vring_enable(hdev, true);
         if (r) {
             goto fail_log;
         }
     }
-    if (hdev->vhost_ops->vhost_dev_start) {
+    if (hdev->vhost_ops->vhost_dev_start && !hdev->backend_transfer) {
         r = hdev->vhost_ops->vhost_dev_start(hdev, true);
         if (r) {
             goto fail_start;
@@ -2207,6 +2287,8 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
     }
     vhost_start_config_intr(hdev);
 
+    hdev->backend_transfer = false;
+
     trace_vhost_dev_start_out(hdev, vdev->name);
     return 0;
 fail_iotlb:
@@ -2262,9 +2344,18 @@ static int do_vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
     if (hdev->vhost_ops->vhost_dev_start) {
         hdev->vhost_ops->vhost_dev_start(hdev, false);
     }
-    if (vrings) {
+    if (vrings && !hdev->backend_transfer) {
         vhost_dev_set_vring_enable(hdev, false);
     }
+
+    if (hdev->backend_transfer) {
+        for (i = 0; i < hdev->nvqs; ++i) {
+            struct vhost_virtqueue *vq = hdev->vqs + i;
+
+            event_notifier_set_handler(&vq->error_notifier, NULL);
+        }
+    }
+
     for (i = 0; i < hdev->nvqs; ++i) {
         rc |= do_vhost_virtqueue_stop(hdev,
                                       vdev,
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 94a0c75fc8..55ad822848 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -105,6 +105,9 @@ struct vhost_dev {
     VIRTIO_DECLARE_FEATURES(_features);
     VIRTIO_DECLARE_FEATURES(acked_features);
 
+    bool _features_wait_incoming;
+    bool backend_transfer;
+
     uint32_t busyloop_timeout;
     uint64_t max_queues;
     uint64_t backend_cap;
@@ -592,4 +595,8 @@ extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
     VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
                            struct vhost_inflight)
 
+extern const VMStateDescription vmstate_vhost_dev;
+#define VMSTATE_BACKEND_TRANSFER_VHOST(_field, _state) \
+    VMSTATE_STRUCT(_field, _state, 0, vmstate_vhost_dev, struct vhost_dev)
+
 #endif
-- 
2.48.1

[PATCH v2 20/25] vhost-user: add vmstate

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/vhost-user.c         | 95 ++++++++++++++++++++++++++++++++++
 include/hw/virtio/vhost-user.h |  4 ++
 2 files changed, 99 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index c5cb5ed528..a820214188 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -28,6 +28,8 @@
 #include "system/runstate.h"
 #include "system/cryptodev.h"
 #include "migration/postcopy-ram.h"
+#include "migration/qemu-file-types.h"
+#include "migration/qemu-file.h"
 #include "trace.h"
 #include "system/ramblock.h"
 
@@ -3137,6 +3139,99 @@ void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status)
         qmp_decode_protocols(u->protocol_features);
 }
 
+typedef struct VhostUserMigTmp {
+    struct vhost_dev *parent;
+    bool has_backend_channel;
+    int backend_fd;
+    uint32_t memory_slots;
+    uint64_t protocol_features;
+} VhostUserMigTmp;
+
+static int vhost_user_tmp_pre_save(void *opaque)
+{
+    VhostUserMigTmp *tmp = opaque;
+    struct vhost_dev *dev = tmp->parent;
+    struct vhost_user *u = dev->opaque;
+    QIOChannelSocket *sioc = u->backend_sioc;
+
+    if (sioc && sioc->fd < 0) {
+        return -EINVAL;
+    }
+
+    tmp->backend_fd = sioc ? sioc->fd : -1;
+    tmp->has_backend_channel = !!sioc;
+    tmp->memory_slots = u->user->memory_slots;
+    tmp->protocol_features = u->protocol_features;
+
+    return 0;
+}
+
+static int vhost_user_tmp_post_load(void *opaque, int version_id)
+{
+    struct VhostUserMigTmp *tmp = opaque;
+    struct vhost_dev *dev = tmp->parent;
+    struct vhost_user *u = dev->opaque;
+    Error *local_err = NULL;
+
+    if (tmp->has_backend_channel) {
+        u->backend_sioc = qio_channel_socket_new_fd(tmp->backend_fd,
+                                                    &local_err);
+        if (!u->backend_sioc) {
+            error_report_err(local_err);
+            return -EINVAL;
+        }
+        u->backend_src = qio_channel_add_watch_source(
+            QIO_CHANNEL(u->backend_sioc), G_IO_IN | G_IO_HUP,
+            backend_read, u->dev, NULL, NULL);
+    }
+
+    u->user->memory_slots = tmp->memory_slots;
+    u->protocol_features = tmp->protocol_features;
+
+    return 0;
+}
+
+static bool vhost_user_tmp_test_fd(void *opaque, int version_id)
+{
+    struct VhostUserMigTmp *tmp = opaque;
+
+    return tmp->has_backend_channel;
+}
+
+static const VMStateDescription vmstate_vhost_user_blk_tmp = {
+    .name = "vhost-user-blk-tmp",
+    .pre_save = vhost_user_tmp_pre_save,
+    .post_load = vhost_user_tmp_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_UINT64(protocol_features, VhostUserMigTmp),
+        VMSTATE_UINT32(memory_slots, VhostUserMigTmp),
+        VMSTATE_BOOL(has_backend_channel, VhostUserMigTmp),
+        VMSTATE_FD_TEST(backend_fd, VhostUserMigTmp, vhost_user_tmp_test_fd),
+        VMSTATE_END_OF_LIST()
+   },
+};
+
+static int vhost_user_post_load(void *opaque, int version_id)
+{
+    struct vhost_dev *dev = opaque;
+    struct vhost_user *u = dev->opaque;
+
+    u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
+    postcopy_add_notifier(&u->postcopy_notifier);
+
+    return 0;
+}
+
+const VMStateDescription vmstate_vhost_user = {
+    .name = "vhost-user",
+    .post_load = vhost_user_post_load,
+    .fields = (const VMStateField[]) {
+        VMSTATE_WITH_TMP(struct vhost_dev, VhostUserMigTmp,
+                         vmstate_vhost_user_blk_tmp),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VhostOps user_ops = {
         .backend_type = VHOST_BACKEND_TYPE_USER,
         .vhost_backend_init = vhost_user_backend_init,
diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
index 36d96296a3..fb89268de2 100644
--- a/include/hw/virtio/vhost-user.h
+++ b/include/hw/virtio/vhost-user.h
@@ -114,4 +114,8 @@ void vhost_user_async_close(DeviceState *d,
 
 void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status);
 
+extern const VMStateDescription vmstate_vhost_user;
+#define VMSTATE_BACKEND_TRANSFER_VHOST_USER(_field, _state) \
+    VMSTATE_STRUCT(_field, _state, 0, vmstate_vhost_user, struct vhost_dev)
+
 #endif
-- 
2.48.1

[PATCH v2 21/25] virtio: support vhost backend migration

[Vladimir Sementsov-Ogievskiy]

Add logic to transfer virtio notifiers through migration channel
for vhost backend migration case.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/virtio-bus.c     |  2 +-
 hw/virtio/virtio.c         | 73 ++++++++++++++++++++++++++++++++++++--
 include/hw/virtio/virtio.h |  2 ++
 3 files changed, 74 insertions(+), 3 deletions(-)

diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index 9b545acda3..577693b6c9 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -291,7 +291,7 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
         return -ENOSYS;
     }
 
-    if (assign) {
+    if (assign && !virtio_is_vhost_backend_transfer(vdev)) {
         r = event_notifier_init(notifier, 1);
         if (r < 0) {
             error_report("%s: unable to init event notifier: %s (%d)",
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 4184aff75c..bf361811d0 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -26,6 +26,7 @@
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/vhost.h"
 #include "migration/qemu-file-types.h"
+#include "migration/qemu-file.h"
 #include "qemu/atomic.h"
 #include "hw/virtio/virtio-bus.h"
 #include "hw/qdev-properties.h"
@@ -3032,6 +3033,7 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f)
     uint32_t guest_features_lo = (vdev->guest_features & 0xffffffff);
     int i;
     Error *local_err = NULL;
+    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
 
     if (k->save_config) {
         k->save_config(qbus->parent, f);
@@ -3065,11 +3067,23 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f)
          */
         qemu_put_be64(f, vdev->vq[i].vring.desc);
         qemu_put_be16s(f, &vdev->vq[i].last_avail_idx);
+
+        if (migrating_backend) {
+            qemu_file_put_fd(f,
+                             event_notifier_get_fd(&vdev->vq[i].host_notifier));
+            qemu_file_put_fd(
+                f, event_notifier_get_fd(&vdev->vq[i].guest_notifier));
+        }
+
         if (k->save_queue) {
             k->save_queue(qbus->parent, i, f);
         }
     }
 
+    if (migrating_backend) {
+        qemu_file_put_fd(f, event_notifier_get_fd(&vdev->config_notifier));
+    }
+
     if (vdc->save != NULL) {
         vdc->save(vdev, f);
     }
@@ -3295,6 +3309,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
     VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
     VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
     Error *local_err = NULL;
+    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
 
     /*
      * We poison the endianness to ensure it does not get used before
@@ -3364,6 +3379,13 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
         vdev->vq[i].signalled_used_valid = false;
         vdev->vq[i].notification = true;
 
+        if (migrating_backend) {
+            event_notifier_init_fd(&vdev->vq[i].host_notifier,
+                                   qemu_file_get_fd(f));
+            event_notifier_init_fd(&vdev->vq[i].guest_notifier,
+                                   qemu_file_get_fd(f));
+        }
+
         if (!vdev->vq[i].vring.desc && vdev->vq[i].last_avail_idx) {
             error_report("VQ %d address 0x0 "
                          "inconsistent with Host index 0x%x",
@@ -3377,6 +3399,10 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
         }
     }
 
+    if (migrating_backend) {
+        event_notifier_init_fd(&vdev->config_notifier, qemu_file_get_fd(f));
+    }
+
     virtio_notify_vector(vdev, VIRTIO_NO_VECTOR);
 
     if (vdc->load != NULL) {
@@ -3394,6 +3420,19 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
         }
     }
 
+    if (migrating_backend) {
+        /*
+         * On vhost backend migration, device do load host_features from
+         * migration stream. So update host_features.
+         */
+        vdev->host_features = vdc->get_features(vdev, vdev->host_features,
+                                                &local_err);
+        if (local_err) {
+            error_report_err(local_err);
+            return -EINVAL;
+        }
+    }
+
     /* Subsections */
     ret = vmstate_load_state(f, &vmstate_virtio, vdev, 1, &local_err);
     if (ret) {
@@ -3447,6 +3486,18 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
                 continue;
             }
 
+            if (migrating_backend) {
+                /*
+                 * Indices are not synced prior to backend migration (as we
+                 * don't stop vrings by GET_VRING_BASE). No reason to sync them
+                 * now, and do any checks.
+                 */
+                vdev->vq[i].used_idx = 0;
+                vdev->vq[i].shadow_avail_idx = 0;
+                vdev->vq[i].inuse = 0;
+                continue;
+            }
+
             nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
             /* Check it isn't doing strange things with descriptor numbers. */
             if (nheads > vdev->vq[i].vring.num) {
@@ -3815,8 +3866,9 @@ int virtio_queue_set_guest_notifier(VirtIODevice *vdev, int n, bool assign,
     EventNotifierHandler *read_fn = is_config ?
         virtio_config_guest_notifier_read :
         virtio_queue_guest_notifier_read;
+    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
 
-    if (assign) {
+    if (assign && !migrating_backend) {
         int r = event_notifier_init(notifier, 0);
         if (r < 0) {
             return r;
@@ -3826,7 +3878,7 @@ int virtio_queue_set_guest_notifier(VirtIODevice *vdev, int n, bool assign,
     event_notifier_set_handler(notifier,
                                (assign && !with_irqfd) ? read_fn : NULL);
 
-    if (!assign) {
+    if (!assign && !migrating_backend) {
         /* Test and clear notifier before closing it,*/
         /* in case poll callback didn't have time to run. */
         read_fn(notifier);
@@ -4445,6 +4497,23 @@ done:
     return element;
 }
 
+bool virtio_is_vhost_backend_transfer(VirtIODevice *vdev)
+{
+    VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
+    struct vhost_dev *hdev;
+
+    if (!vdc->get_vhost) {
+        return false;
+    }
+
+    hdev = vdc->get_vhost(vdev);
+    if (!hdev) {
+        return false;
+    }
+
+    return hdev->backend_transfer;
+}
+
 static const TypeInfo virtio_device_info = {
     .name = TYPE_VIRTIO_DEVICE,
     .parent = TYPE_DEVICE,
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index 620ee4e389..8e2d3019cd 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -242,6 +242,8 @@ struct VirtioDeviceClass {
     bool (*skip_vhost_migration_log)(VirtIODevice *vdev);
 };
 
+bool virtio_is_vhost_backend_transfer(VirtIODevice *vdev);
+
 void virtio_instance_init_common(Object *proxy_obj, void *data,
                                  size_t vdev_size, const char *vdev_name);
 
-- 
2.48.1

[PATCH v2 22/25] virtio: support .needed for virtio vmsd

[Vladimir Sementsov-Ogievskiy]

Virtio devices now don't use .needed field for their VMSD, but
we are going to use it for vhost-user-blk. So, let's handle this
field in generic code. Nothing is changed for existing vmstates.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/virtio/virtio.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index bf361811d0..aa36438e69 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3088,7 +3088,7 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f)
         vdc->save(vdev, f);
     }
 
-    if (vdc->vmsd) {
+    if (vdc->vmsd && vmstate_section_needed(vdc->vmsd, vdev)) {
         int ret = vmstate_save_state(f, vdc->vmsd, vdev, NULL, &local_err);
         if (ret) {
             error_report_err(local_err);
@@ -3412,7 +3412,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
         }
     }
 
-    if (vdc->vmsd) {
+    if (vdc->vmsd && vmstate_section_needed(vdc->vmsd, vdev)) {
         ret = vmstate_load_state(f, vdc->vmsd, vdev, version_id, &local_err);
         if (ret) {
             error_report_err(local_err);
-- 
2.48.1

[PATCH v2 23/25] RFC qapi: add local-vhost-user-blk migration capability

[Vladimir Sementsov-Ogievskiy]

That's an old way to enable backend-transfer feature for
vhost-user-blk.

I just kept it unchanged since v1 of the series.

At least, it should be migration-parameter instead of
capability.

But currently, there is discussion on final API in
context of "[PATCH v8 00/19] virtio-net: live-TAP local migration", here:
https://lore.kernel.org/qemu-devel/29aa1d66-9fa7-4e44-b0e3-2ca26e77accf@yandex-team.ru/

So, consider this one as temporary patch, which will be replaced
by some other API.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 migration/options.c |  7 +++++++
 migration/options.h |  1 +
 qapi/migration.json | 11 +++++++++--
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/migration/options.c b/migration/options.c
index 5183112775..3a013b9898 100644
--- a/migration/options.c
+++ b/migration/options.c
@@ -262,6 +262,13 @@ bool migrate_mapped_ram(void)
     return s->capabilities[MIGRATION_CAPABILITY_MAPPED_RAM];
 }
 
+bool migrate_local_vhost_user_blk(void)
+{
+    MigrationState *s = migrate_get_current();
+
+    return s->capabilities[MIGRATION_CAPABILITY_LOCAL_VHOST_USER_BLK];
+}
+
 bool migrate_ignore_shared(void)
 {
     MigrationState *s = migrate_get_current();
diff --git a/migration/options.h b/migration/options.h
index 82d839709e..95a3f482bc 100644
--- a/migration/options.h
+++ b/migration/options.h
@@ -30,6 +30,7 @@ bool migrate_colo(void);
 bool migrate_dirty_bitmaps(void);
 bool migrate_events(void);
 bool migrate_mapped_ram(void);
+bool migrate_local_vhost_user_blk(void);
 bool migrate_ignore_shared(void);
 bool migrate_late_block_activate(void);
 bool migrate_multifd(void);
diff --git a/qapi/migration.json b/qapi/migration.json
index be0f3fcc12..d29a4915b4 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -517,9 +517,15 @@
 #     each RAM page.  Requires a migration URI that supports seeking,
 #     such as a file.  (since 9.0)
 #
+# @local-vhost-user-blk: Migrate vhost-user-blk locally, keeping
+#     backend alive. Open file descriptors and backend-related state
+#     are migrated. Only may be used when migration channel is UNIX
+#     domain socket. (since 10.2)
+#
 # Features:
 #
-# @unstable: Members @x-colo and @x-ignore-shared are experimental.
+# @unstable: Members @x-colo, @x-ignore-shared,
+#     @local-vhost-user-blk are experimental.
 # @deprecated: Member @zero-blocks is deprecated as being part of
 #     block migration which was already removed.
 #
@@ -536,7 +542,8 @@
            { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
            'validate-uuid', 'background-snapshot',
            'zero-copy-send', 'postcopy-preempt', 'switchover-ack',
-           'dirty-limit', 'mapped-ram'] }
+           'dirty-limit', 'mapped-ram',
+           { 'name': 'local-vhost-user-blk', 'features': [ 'unstable' ] } ] }
 
 ##
 # @MigrationCapabilityStatus:
-- 
2.48.1

[PATCH v2 24/25] vhost-user-blk: support vhost backend migration

[Vladimir Sementsov-Ogievskiy]

Opt-out backend initialization code, and instead get the state
from migration channel (including inflight region).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 hw/block/vhost-user-blk.c          | 129 ++++++++++++++++++++++++-----
 include/hw/virtio/vhost-user-blk.h |   2 +
 include/hw/virtio/vhost.h          |   3 +-
 3 files changed, 111 insertions(+), 23 deletions(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index ffdd600526..a8fd90480a 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -17,6 +17,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qapi-types-run-state.h"
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "qemu/cutils.h"
@@ -31,7 +32,13 @@
 #include "hw/virtio/virtio-access.h"
 #include "system/system.h"
 #include "system/runstate.h"
+#include "chardev/char-backend-transfer.h"
 #include "trace.h"
+#include "migration/qemu-file.h"
+#include "migration/migration.h"
+#include "migration/options.h"
+#include "qemu/event_notifier.h"
+#include <sys/mman.h>
 
 static const int user_feature_bits[] = {
     VIRTIO_BLK_F_SIZE_MAX,
@@ -160,32 +167,35 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Error **errp)
 
     s->dev.acked_features = vdev->guest_features;
 
-    ret = vhost_dev_prepare_inflight(&s->dev, vdev);
-    if (ret < 0) {
-        error_setg_errno(errp, -ret, "Error setting inflight format");
-        goto err_guest_notifiers;
-    }
-
-    if (!s->inflight->addr) {
-        ret = vhost_dev_get_inflight(&s->dev, s->queue_size, s->inflight);
+    if (!s->dev.backend_transfer) {
+        ret = vhost_dev_prepare_inflight(&s->dev, vdev);
         if (ret < 0) {
-            error_setg_errno(errp, -ret, "Error getting inflight");
+            error_setg_errno(errp, -ret, "Error setting inflight format");
             goto err_guest_notifiers;
         }
-    }
 
-    ret = vhost_dev_set_inflight(&s->dev, s->inflight);
-    if (ret < 0) {
-        error_setg_errno(errp, -ret, "Error setting inflight");
-        goto err_guest_notifiers;
-    }
+        if (!s->inflight->addr) {
+            ret = vhost_dev_get_inflight(&s->dev, s->queue_size, s->inflight);
+            if (ret < 0) {
+                error_setg_errno(errp, -ret, "Error getting inflight");
+                goto err_guest_notifiers;
+            }
+        }
 
-    /* guest_notifier_mask/pending not used yet, so just unmask
-     * everything here. virtio-pci will do the right thing by
-     * enabling/disabling irqfd.
-     */
-    for (i = 0; i < s->dev.nvqs; i++) {
-        vhost_virtqueue_mask(&s->dev, vdev, i, false);
+        ret = vhost_dev_set_inflight(&s->dev, s->inflight);
+        if (ret < 0) {
+            error_setg_errno(errp, -ret, "Error setting inflight");
+            goto err_guest_notifiers;
+        }
+
+        /*
+         * guest_notifier_mask/pending not used yet, so just unmask
+         * everything here. virtio-pci will do the right thing by
+         * enabling/disabling irqfd.
+         */
+        for (i = 0; i < s->dev.nvqs; i++) {
+            vhost_virtqueue_mask(&s->dev, vdev, i, false);
+        }
     }
 
     s->dev.vq_index_end = s->dev.nvqs;
@@ -232,6 +242,10 @@ static int vhost_user_blk_stop(VirtIODevice *vdev)
     force_stop = s->skip_get_vring_base_on_force_shutdown &&
                  qemu_force_shutdown_requested();
 
+    s->dev.backend_transfer = s->dev.backend_transfer ||
+        (runstate_check(RUN_STATE_FINISH_MIGRATE) &&
+         migrate_local_vhost_user_blk());
+
     ret = force_stop ? vhost_dev_force_stop(&s->dev, vdev, true) :
                        vhost_dev_stop(&s->dev, vdev, true);
 
@@ -391,6 +405,7 @@ static int vhost_user_blk_connect(DeviceState *dev,
     trace_vhost_user_blk_connect_in(vdev);
 
     assert(!s->connected);
+    assert(!s->dev.backend_transfer);
 
     ret = vhost_dev_connect(&s->dev, errp);
     if (ret < 0) {
@@ -464,6 +479,9 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
     DeviceState *dev = DEVICE(s);
     int ret;
 
+    assert(!s->connected);
+    assert(!s->dev.backend_transfer);
+
     ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
     if (ret < 0) {
         return ret;
@@ -642,7 +660,13 @@ static bool vhost_user_blk_pre_incoming(void *opaque, Error **errp)
 {
     VHostUserBlk *s = VHOST_USER_BLK(opaque);
 
-    return vhost_user_blk_realize_connect(s, errp) == 0;
+    s->dev.backend_transfer = migrate_local_vhost_user_blk();
+
+    if (!s->dev.backend_transfer) {
+        return vhost_user_blk_realize_connect_loop(s, errp) >= 0;
+    }
+
+    return true;
 }
 
 static const VMStateDescription vmstate_vhost_user_blk = {
@@ -656,6 +680,64 @@ static const VMStateDescription vmstate_vhost_user_blk = {
     },
 };
 
+static bool vhost_user_needed(void *opaque)
+{
+    return migrate_local_vhost_user_blk();
+}
+
+static const VMStateDescription vmstate_vhost_user_blk_device = {
+    .name = "vhost-user-blk-device",
+    .version_id = 1,
+    .needed = vhost_user_needed,
+    .fields = (const VMStateField[]) {
+        VMSTATE_BACKEND_TRANSFER_CHARDEV(chardev, VHostUserBlk),
+        VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(inflight, VHostUserBlk),
+        VMSTATE_BACKEND_TRANSFER_VHOST_USER(dev, VHostUserBlk),
+        VMSTATE_BACKEND_TRANSFER_VHOST(dev, VHostUserBlk),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
+static int vhost_user_blk_post_load(VirtIODevice *vdev)
+{
+    VHostUserBlk *s = VHOST_USER_BLK(vdev);
+    struct vhost_dev *hdev = vhost_user_blk_get_vhost(vdev);
+    DeviceState *dev = &s->parent_obj.parent_obj;
+
+    if (!hdev->backend_transfer) {
+        return 0;
+    }
+
+    s->connected = true;
+
+    memcpy(&s->blkcfg, vdev->config, vdev->config_len);
+
+    if (virtio_device_started(vdev, vdev->status)) {
+        int ret;
+        ret = vhost_user_blk_start(vdev, NULL);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
+    /* we're fully initialized, now we can operate, so add the handler */
+    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
+                             vhost_user_blk_event, NULL, (void *)dev,
+                             NULL, true);
+
+    return 0;
+}
+
+static bool vhost_user_blk_skip_migration_log(VirtIODevice *vdev)
+{
+    /*
+     * Note that hdev->migrating_backend is false at this moment,
+     * as logging is being setup during outging migration setup stage,
+     * which is far before vm stop.
+     */
+    return migrate_local_vhost_user_blk();
+}
+
 static const Property vhost_user_blk_properties[] = {
     DEFINE_PROP_CHR_NO_CONNECT("chardev", VHostUserBlk, chardev),
     DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
@@ -688,6 +770,9 @@ static void vhost_user_blk_class_init(ObjectClass *klass, const void *data)
     vdc->set_status = vhost_user_blk_set_status;
     vdc->reset = vhost_user_blk_reset;
     vdc->get_vhost = vhost_user_blk_get_vhost;
+    vdc->vmsd = &vmstate_vhost_user_blk_device;
+    vdc->post_load = vhost_user_blk_post_load,
+    vdc->skip_vhost_migration_log = vhost_user_blk_skip_migration_log;
 }
 
 static const TypeInfo vhost_user_blk_info = {
diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h
index a10f785672..b06f55fd6f 100644
--- a/include/hw/virtio/vhost-user-blk.h
+++ b/include/hw/virtio/vhost-user-blk.h
@@ -52,6 +52,8 @@ struct VHostUserBlk {
     bool started_vu;
 
     bool skip_get_vring_base_on_force_shutdown;
+
+    bool incoming_backend;
 };
 
 #endif
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index 55ad822848..13ca2c319f 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -592,7 +592,8 @@ static inline int vhost_load_backend_state(struct vhost_dev *dev, QEMUFile *f,
 
 extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
 #define VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(_field, _state) \
-    VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
+    VMSTATE_STRUCT_POINTER(_field, _state, \
+                           vmstate_backend_transfer_vhost_inflight, \
                            struct vhost_inflight)
 
 extern const VMStateDescription vmstate_vhost_dev;
-- 
2.48.1

[PATCH v2 25/25] tests/functional: add test_x86_64_vhost_user_blk_fd_migration.py

[Vladimir Sementsov-Ogievskiy]

Introduce a simple test to check that local migration of vhost-user-blk
device with passing open fds through unix socket works, and the disk
is still working on target.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
---
 ...test_x86_64_vhost_user_blk_fd_migration.py | 307 ++++++++++++++++++
 1 file changed, 307 insertions(+)
 create mode 100644 tests/functional/test_x86_64_vhost_user_blk_fd_migration.py

diff --git a/tests/functional/test_x86_64_vhost_user_blk_fd_migration.py b/tests/functional/test_x86_64_vhost_user_blk_fd_migration.py
new file mode 100644
index 0000000000..0523a71f15
--- /dev/null
+++ b/tests/functional/test_x86_64_vhost_user_blk_fd_migration.py
@@ -0,0 +1,307 @@
+#!/usr/bin/env python3
+#
+# Functional test that tests vhost-user-blk local migration
+# with fd passing
+#
+# Copyright (c) Yandex
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import time
+import subprocess
+
+from qemu_test import (
+    LinuxKernelTest,
+    Asset,
+    exec_command_and_wait_for_pattern,
+)
+
+
+def wait_migration_finish(source_vm, target_vm):
+    migr_events = (
+        ("MIGRATION", {"data": {"status": "completed"}}),
+        ("MIGRATION", {"data": {"status": "failed"}}),
+    )
+
+    source_e = source_vm.events_wait(migr_events)["data"]
+    target_e = target_vm.events_wait(migr_events)["data"]
+
+    source_s = source_vm.cmd("query-status")["status"]
+    target_s = target_vm.cmd("query-status")["status"]
+
+    assert (
+        source_e["status"] == "completed"
+        and target_e["status"] == "completed"
+        and source_s == "postmigrate"
+        and target_s == "paused"
+    ), f"""Migration failed:
+    SRC status: {source_s}
+    SRC event: {source_e}
+    TGT status: {target_s}
+    TGT event:{target_e}"""
+
+
+class VhostUserBlkFdMigration(LinuxKernelTest):
+
+    ASSET_KERNEL = Asset(
+        (
+            "https://archives.fedoraproject.org/"
+            "pub/archive/fedora/linux/releases"
+            "/31/Server/x86_64/os/images/pxeboot/vmlinuz"
+        ),
+        "d4738d03dbbe083ca610d0821d0a8f1488bebbdccef54ce33e3adb35fda00129",
+    )
+
+    ASSET_INITRD = Asset(
+        (
+            "https://archives.fedoraproject.org/"
+            "pub/archive/fedora/linux/releases"
+            "/31/Server/x86_64/os/images/pxeboot/initrd.img"
+        ),
+        "277cd6c7adf77c7e63d73bbb2cded8ef9e2d3a2f100000e92ff1f8396513cd8b",
+    )
+
+    DATA1 = "TEST_DATA_BEFORE_MIGRATION_12345"
+    DATA2 = "TEST_DATA_AFTER_MIGRATION_54321"
+
+    def write_data(self, data, vm) -> None:
+        exec_command_and_wait_for_pattern(
+            self,
+            f'echo "{data}" | ' "dd of=/dev/vda bs=512 count=1 oflag=direct",
+            "# ",
+            vm=vm,
+        )
+
+    def read_data(self, data, vm) -> None:
+        exec_command_and_wait_for_pattern(
+            self,
+            "dd if=/dev/vda bs=512 count=1 iflag=direct 2>/dev/null",
+            data,
+            vm=vm,
+        )
+
+    def setUp(self):
+        super().setUp()
+        self.vhost_procs = []
+
+    def tearDown(self):
+        # Cleanup vhost-user server processes
+        for proc in self.vhost_procs:
+            if proc:
+                try:
+                    proc.terminate()
+                    proc.wait(timeout=5)
+                except subprocess.TimeoutExpired:
+                    proc.kill()
+                    proc.wait()
+                except:
+                    pass
+
+        super().tearDown()
+
+    def create_test_image(self):
+        """Create a temporary test image for vhost-user-blk"""
+        img_path = self.scratch_file("disk.img")
+
+        # Create 64MB image
+        with open(img_path, "wb") as f:
+            f.write(b"\0" * (64 * 1024 * 1024))
+
+        return img_path
+
+    def start_vhost_user_server(self, socket_path, img_path):
+        """Start vhost-user-blk server using contrib/vhost-user-blk"""
+        # Find vhost-user-blk binary
+        vub_binary = self.build_file(
+            "contrib", "vhost-user-blk", "vhost-user-blk"
+        )
+
+        if not os.path.isfile(vub_binary) or not os.access(vub_binary, os.X_OK):
+            self.skipTest("vhost-user-blk binary not found")
+
+        # assert that our further waiting would be correct
+        self.assertFalse(os.path.exists(socket_path))
+
+        cmd = [vub_binary, "-s", socket_path, "-b", img_path]
+        self.log.info(f'Starting vhost-user server: {" ".join(cmd)}')
+        proc = subprocess.Popen(
+            cmd, stderr=subprocess.PIPE, text=True, preexec_fn=os.setsid
+        )
+        self.vhost_procs.append(proc)
+
+        # Wait for socket to be created
+        for _ in range(100):  # 10 seconds timeout
+            time.sleep(0.1)
+
+            # Check if process is still running
+            if proc.poll() is not None:
+                self.fail(f"vhost-user server failed: {proc.stderr}")
+
+            if os.path.exists(socket_path):
+                return
+
+        self.fail(f"vhost-user socket {socket_path} was not created")
+
+    def setup_shared_memory(self):
+        shm_path = f"/dev/shm/qemu_test_{os.getpid()}"
+
+        try:
+            with open(shm_path, "wb") as f:
+                f.write(b"\0" * (1024 * 1024 * 1024))  # 1GB
+        except Exception as e:
+            self.fail(f"Failed to create shared memory file: {e}")
+
+        return shm_path
+
+    def prepare_and_launch_vm(
+        self,
+        shm_path,
+        vhost_socket,
+        incoming=False,
+        vm=None,
+        use_backend_transfer=True,
+    ):
+        if not vm:
+            vm = self.vm
+
+        vm.add_args("-accel", "kvm")
+        vm.add_args("-device", "pcie-pci-bridge,id=pci.1,bus=pcie.0")
+        vm.add_args("-m", "1G")
+        vm.add_args("-append", "console=ttyS0 rd.rescue")
+
+        vm.add_args(
+            "-object",
+            f"memory-backend-file,id=ram0,size=1G,mem-path={shm_path},share=on",
+        )
+        vm.add_args("-machine", "memory-backend=ram0")
+
+        vm.add_args("-kernel", self.ASSET_KERNEL.fetch())
+        vm.add_args("-initrd", self.ASSET_INITRD.fetch())
+
+        vm.add_args("-S")
+
+        if incoming:
+            vm.add_args("-incoming", "defer")
+
+        vm.set_console()
+
+        vm_s = "target" if incoming else "source"
+        self.log.info(f"Launching {vm_s} VM")
+        vm.launch()
+
+        self.set_migration_capabilities(vm, use_backend_transfer)
+        self.add_vhost_user_blk_device(vm, vhost_socket)
+
+    def add_vhost_user_blk_device(self, vm, socket_path):
+        # Add chardev
+        chardev_params = {
+            "id": "chardev-virtio-disk0",
+            "backend": {
+                "type": "socket",
+                "data": {
+                    "addr": {"type": "unix", "data": {"path": socket_path}},
+                    "server": False,
+                    "reconnect-ms": 20,
+                },
+            },
+        }
+
+        vm.cmd("chardev-add", chardev_params)
+
+        # Add device
+        device_params = {
+            "id": "virtio-disk0",
+            "driver": "vhost-user-blk-pci",
+            "chardev": "chardev-virtio-disk0",
+            "num-queues": 1,
+            "bus": "pci.1",
+            "config-wce": False,
+            "bootindex": 1,
+            "disable-legacy": "off",
+        }
+
+        vm.cmd("device_add", device_params)
+
+    def set_migration_capabilities(self, vm, use_backend_transfer=True):
+        capabilities = [
+            {"capability": "events", "state": True},
+            {"capability": "x-ignore-shared", "state": True},
+        ]
+
+        if use_backend_transfer:
+            capabilities.append(
+                {"capability": "local-vhost-user-blk", "state": True}
+            )
+
+        vm.cmd("migrate-set-capabilities", {"capabilities": capabilities})
+
+    def do_test_vhost_user_blk_fd_migration(self, use_backend_transfer=True):
+        self.require_accelerator("kvm")
+        self.set_machine("q35")
+
+        socket_dir = self.socket_dir()
+        vhost_socket = os.path.join(socket_dir.name, "vhost-user-blk.sock")
+        migration_socket = os.path.join(socket_dir.name, "migration.sock")
+
+        img_path = self.create_test_image()
+        shm_path = self.setup_shared_memory()
+
+        self.start_vhost_user_server(vhost_socket, img_path)
+
+        if not use_backend_transfer:
+            target_vhost_socket = os.path.join(
+                socket_dir.name, "vhost-user-blk-target.sock"
+            )
+            self.start_vhost_user_server(target_vhost_socket, img_path)
+        else:
+            target_vhost_socket = vhost_socket
+
+        self.prepare_and_launch_vm(
+            shm_path, vhost_socket, use_backend_transfer=use_backend_transfer
+        )
+        self.vm.cmd("cont")
+        self.wait_for_console_pattern("Entering emergency mode.")
+        self.wait_for_console_pattern("# ")
+
+        self.write_data(self.DATA1, self.vm)
+        self.read_data(self.DATA1, self.vm)
+
+        target_vm = self.get_vm(name="target")
+        self.prepare_and_launch_vm(
+            shm_path,
+            target_vhost_socket,
+            incoming=True,
+            vm=target_vm,
+            use_backend_transfer=use_backend_transfer,
+        )
+
+        target_vm.cmd("migrate-incoming", {"uri": f"unix:{migration_socket}"})
+
+        self.log.info("Starting migration")
+        self.vm.cmd("migrate", {"uri": f"unix:{migration_socket}"})
+
+        self.log.info("Waiting for migration completion")
+        wait_migration_finish(self.vm, target_vm)
+
+        target_vm.cmd("cont")
+        self.vm.shutdown()
+
+        self.log.info("Verifying disk on target VM after migration")
+        self.read_data(self.DATA1, target_vm)
+        self.write_data(self.DATA2, target_vm)
+        self.read_data(self.DATA2, target_vm)
+
+        target_vm.shutdown()
+
+    def test_vhost_user_blk_fd_migration_backend_transfer(self):
+        """Test vhost-user-blk migration with backend-transfer (fd passing)"""
+        self.do_test_vhost_user_blk_fd_migration(use_backend_transfer=True)
+
+    def test_vhost_user_blk_fd_migration_regular(self):
+        """Test vhost-user-blk migration without backend-transfer"""
+        self.do_test_vhost_user_blk_fd_migration(use_backend_transfer=False)
+
+
+if __name__ == "__main__":
+    LinuxKernelTest.main()
-- 
2.48.1

Re: [PATCH v2 16/25] chardev: add .chr_get_client() handler

[Daniel P. Berrangé]

On Thu, Oct 16, 2025 at 02:40:53PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> A pair for .char_add_client(), to be used to support backend-transfer
> migration of chardev attached to vhost-user-blk in following commits.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  chardev/char-socket.c  | 7 +++++++
>  chardev/char.c         | 6 ++++++
>  include/chardev/char.h | 4 ++++
>  3 files changed, 17 insertions(+)
> 
> diff --git a/chardev/char-socket.c b/chardev/char-socket.c
> index 0a5738c158..51bb9d0a2d 100644
> --- a/chardev/char-socket.c
> +++ b/chardev/char-socket.c
> @@ -929,6 +929,12 @@ static int tcp_chr_new_client(Chardev *chr, QIOChannelSocket *sioc)
>      return 0;
>  }
>  
> +static int tcp_chr_get_client(Chardev *chr)
> +{
> +    SocketChardev *s = SOCKET_CHARDEV(chr);
> +
> +    return s->sioc->fd;
> +}

This will crash on a NULL s->sioc pointer when the chardev
is not connected. More generally it feels wrong to be exposing
internal impl details of the socket chardev in this way.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 18/25] chardev: introduce backend-transfer vmstate for chardev

[Daniel P. Berrangé]

On Thu, Oct 16, 2025 at 02:40:55PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> We'll need to transfer the chardev attached to vhost-user-blk, to
> support backend-transfer migration for vhost-user-blk. So, prepare
> chardev vmsd now.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  chardev/char-backend-transfer.c         | 52 +++++++++++++++++++++++++
>  chardev/meson.build                     |  1 +
>  include/chardev/char-backend-transfer.h | 17 ++++++++
>  3 files changed, 70 insertions(+)
>  create mode 100644 chardev/char-backend-transfer.c
>  create mode 100644 include/chardev/char-backend-transfer.h
> 
> diff --git a/chardev/char-backend-transfer.c b/chardev/char-backend-transfer.c
> new file mode 100644
> index 0000000000..f1a399c7fa
> --- /dev/null
> +++ b/chardev/char-backend-transfer.c
> @@ -0,0 +1,52 @@
> +/*
> + * Event notifier migration support
> + * Copyright (c) Yandex Technologies LLC, 2025
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +#include "chardev/char-fe.h"
> +#include "migration/vmstate.h"
> +
> +typedef struct CharBackendTransferTmp {
> +    CharBackend *parent;
> +    int fd;
> +} CharBackendTransferTmp;
> +
> +static int char_backend_transfer_pre_save(void *opaque)
> +{
> +    CharBackendTransferTmp *tmp = opaque;
> +
> +    tmp->fd = qemu_chr_get_client(tmp->parent->chr);
> +    if (tmp->fd < 0) {
> +        return -1;
> +    }
> +
> +    return 0;
> +}
> +
> +static int char_backend_transfer_post_load(void *opaque, int version_id)
> +{
> +    CharBackendTransferTmp *tmp = opaque;
> +
> +    return qemu_chr_add_client(tmp->parent->chr, tmp->fd);
> +}
> +
> +const VMStateDescription vmstate_backend_transfer_char_tmp = {
> +    .name = "backend-transfer-char-tmp",
> +    .pre_save = char_backend_transfer_pre_save,
> +    .post_load = char_backend_transfer_post_load,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_FD(fd, CharBackendTransferTmp),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
> +const VMStateDescription vmstate_backend_transfer_char = {
> +    .name = "backend-transfer-char",
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_WITH_TMP(CharBackend, CharBackendTransferTmp,
> +                         vmstate_backend_transfer_char_tmp),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};

On the one hand this code is positioned as if it were a generic
mechanism for transferring chardev vmstate, but on the oother
hand it is hardcoded to only work with the socket chardev and
is only able to transfer an FD, no other state.

Socket chardevs can involve telnet, tls or websocket protocol
layering all of which have considerable state that is being
ignored by this.

IMHO each chardev backend needs to be directly responsible
for handling vmstate, and it needs to be able to raise an
error if there is state which cannot be transferred. This
would avoid creatin of the undesirable qemu_chr_get_client
method as public API.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v2 18/25] chardev: introduce backend-transfer vmstate for chardev

[Vladimir Sementsov-Ogievskiy]

On 16.10.25 14:55, Daniel P. Berrangé wrote:
> On Thu, Oct 16, 2025 at 02:40:55PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>> We'll need to transfer the chardev attached to vhost-user-blk, to
>> support backend-transfer migration for vhost-user-blk. So, prepare
>> chardev vmsd now.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>> ---
>>   chardev/char-backend-transfer.c         | 52 +++++++++++++++++++++++++
>>   chardev/meson.build                     |  1 +
>>   include/chardev/char-backend-transfer.h | 17 ++++++++
>>   3 files changed, 70 insertions(+)
>>   create mode 100644 chardev/char-backend-transfer.c
>>   create mode 100644 include/chardev/char-backend-transfer.h
>>
>> diff --git a/chardev/char-backend-transfer.c b/chardev/char-backend-transfer.c
>> new file mode 100644
>> index 0000000000..f1a399c7fa
>> --- /dev/null
>> +++ b/chardev/char-backend-transfer.c
>> @@ -0,0 +1,52 @@
>> +/*
>> + * Event notifier migration support
>> + * Copyright (c) Yandex Technologies LLC, 2025
>> + * SPDX-License-Identifier: GPL-2.0-or-later
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "chardev/char-fe.h"
>> +#include "migration/vmstate.h"
>> +
>> +typedef struct CharBackendTransferTmp {
>> +    CharBackend *parent;
>> +    int fd;
>> +} CharBackendTransferTmp;
>> +
>> +static int char_backend_transfer_pre_save(void *opaque)
>> +{
>> +    CharBackendTransferTmp *tmp = opaque;
>> +
>> +    tmp->fd = qemu_chr_get_client(tmp->parent->chr);
>> +    if (tmp->fd < 0) {
>> +        return -1;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static int char_backend_transfer_post_load(void *opaque, int version_id)
>> +{
>> +    CharBackendTransferTmp *tmp = opaque;
>> +
>> +    return qemu_chr_add_client(tmp->parent->chr, tmp->fd);
>> +}
>> +
>> +const VMStateDescription vmstate_backend_transfer_char_tmp = {
>> +    .name = "backend-transfer-char-tmp",
>> +    .pre_save = char_backend_transfer_pre_save,
>> +    .post_load = char_backend_transfer_post_load,
>> +    .fields = (const VMStateField[]) {
>> +        VMSTATE_FD(fd, CharBackendTransferTmp),
>> +        VMSTATE_END_OF_LIST()
>> +    },
>> +};
>> +
>> +const VMStateDescription vmstate_backend_transfer_char = {
>> +    .name = "backend-transfer-char",
>> +    .fields = (const VMStateField[]) {
>> +        VMSTATE_WITH_TMP(CharBackend, CharBackendTransferTmp,
>> +                         vmstate_backend_transfer_char_tmp),
>> +        VMSTATE_END_OF_LIST()
>> +    },
>> +};
> 
> On the one hand this code is positioned as if it were a generic
> mechanism for transferring chardev vmstate, but on the oother
> hand it is hardcoded to only work with the socket chardev and
> is only able to transfer an FD, no other state.
> 
> Socket chardevs can involve telnet, tls or websocket protocol
> layering all of which have considerable state that is being
> ignored by this.
> 
> IMHO each chardev backend needs to be directly responsible
> for handling vmstate, and it needs to be able to raise an
> error if there is state which cannot be transferred. This
> would avoid creatin of the undesirable qemu_chr_get_client
> method as public API.
> 

Ow, that's right, me ashamed. I implemented it in a wrong layer.
Will rework, thanks!

-- 
Best regards,
Vladimir

Re: [PATCH v2 15/25] migration: introduce vmstate_event_notifier

[Peter Xu]

On Thu, Oct 16, 2025 at 02:40:52PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> This will be used to support backend-transfer migration for
> vhost-user-blk, we'll migrate event notifier fds through
> migration stream, to avoid extra contact with backend.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>

Acked-by: Peter Xu <peterx@redhat.com>

Note: we recently merged _errp versions of all below three hooks.  You can
also switch to that if all of them can fail.  You can keep my A-b if you
switch over.

> ---
>  include/migration/vmstate.h        |  7 ++++
>  migration/meson.build              |  1 +
>  migration/vmstate-event-notifier.c | 54 ++++++++++++++++++++++++++++++
>  3 files changed, 62 insertions(+)
>  create mode 100644 migration/vmstate-event-notifier.c
> 
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index f243518fb5..7f1f1c166a 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -1294,4 +1294,11 @@ void vmstate_register_ram_global(struct MemoryRegion *memory);
>  
>  bool vmstate_check_only_migratable(const VMStateDescription *vmsd);
>  
> +extern const VMStateDescription vmstate_event_notifier;
> +
> +#define VMSTATE_EVENT_NOTIFIER(_field, _struct) \
> +    VMSTATE_STRUCT(_field, _struct, 0, vmstate_event_notifier, \
> +                   EventNotifier)
> +
> +
>  #endif
> diff --git a/migration/meson.build b/migration/meson.build
> index 16909d54c5..b5341ae0cb 100644
> --- a/migration/meson.build
> +++ b/migration/meson.build
> @@ -5,6 +5,7 @@ migration_files = files(
>    'xbzrle.c',
>    'vmstate-types.c',
>    'vmstate.c',
> +  'vmstate-event-notifier.c',
>    'qemu-file.c',
>    'yank_functions.c',
>  )
> diff --git a/migration/vmstate-event-notifier.c b/migration/vmstate-event-notifier.c
> new file mode 100644
> index 0000000000..2076eec961
> --- /dev/null
> +++ b/migration/vmstate-event-notifier.c
> @@ -0,0 +1,54 @@
> +/*
> + * Event notifier migration support
> + * Copyright (c) Yandex Technologies LLC, 2025
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/event_notifier.h"
> +#include "migration/vmstate.h"
> +
> +static int event_notifier_pre_save(void *opaque)
> +{
> +    struct EventNotifier *e = opaque;
> +
> +    if (!e->initialized || e->rfd != e->wfd) {
> +        return -1;
> +    }
> +
> +    return 0;
> +}
> +
> +static int event_notifier_pre_load(void *opaque)
> +{
> +    struct EventNotifier *e = opaque;
> +
> +    if (e->initialized) {
> +        return -1;
> +    }
> +
> +    return 0;
> +}
> +
> +static int event_notifier_post_load(void *opaque, int version_id)
> +{
> +    struct EventNotifier *e = opaque;
> +
> +    if (e->rfd < 0) {
> +        return -1;
> +    }
> +
> +    e->wfd = e->rfd;
> +    e->initialized = true;
> +
> +    return 0;
> +}
> +
> +const VMStateDescription vmstate_event_notifier = {
> +    .name = "event-notifier",
> +    .pre_save = event_notifier_pre_save,
> +    .pre_load = event_notifier_pre_load,
> +    .post_load = event_notifier_post_load,
> +    .fields = (const VMStateField[]){VMSTATE_FD(rfd, EventNotifier),
> +                                     VMSTATE_END_OF_LIST()},
> +};
> -- 
> 2.48.1
> 

-- 
Peter Xu

Re: [PATCH v2 15/25] migration: introduce vmstate_event_notifier

[Vladimir Sementsov-Ogievskiy]

On 16.10.25 21:59, Peter Xu wrote:
> On Thu, Oct 16, 2025 at 02:40:52PM +0300, Vladimir Sementsov-Ogievskiy wrote:
>> This will be used to support backend-transfer migration for
>> vhost-user-blk, we'll migrate event notifier fds through
>> migration stream, to avoid extra contact with backend.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> 
> Acked-by: Peter Xu <peterx@redhat.com>
> 
> Note: we recently merged _errp versions of all below three hooks.  You can
> also switch to that if all of them can fail.  You can keep my A-b if you
> switch over.

O, yes, will do, thanks.



-- 
Best regards,
Vladimir

Re: [PATCH v2 01/25] vhost: store busyloop_timeout into struct vhost_dev

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:50 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> We'll split vhost_dev_init() into _init() and _connect(), to be able
> to postpone communication with backend, to support backend-transfer
> migration of vhost-user-blk in future commit.
>
> So, instead of passing it through parameters, store it in vhost_dev
> structure.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c         | 11 +++++------
>  include/hw/virtio/vhost.h |  1 +
>  2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 7ba90c24db..9fc6e7ba65 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1501,8 +1501,7 @@ static void vhost_virtqueue_error_notifier(EventNotifier *n)
>  }
>
>  static int vhost_virtqueue_init(struct vhost_dev *dev,
> -                                struct vhost_virtqueue *vq, int n,
> -                                bool busyloop_timeout)
> +                                struct vhost_virtqueue *vq, int n)
>  {
>      int vhost_vq_index = dev->vhost_ops->vhost_get_vq_index(dev, n);
>      struct vhost_vring_file file = {
> @@ -1539,8 +1538,8 @@ static int vhost_virtqueue_init(struct vhost_dev *dev,
>                                     vhost_virtqueue_error_notifier);
>      }
>
> -    if (busyloop_timeout) {
> -        r = vhost_virtqueue_set_busyloop_timeout(dev, n, busyloop_timeout);
> +    if (dev->busyloop_timeout) {
> +        r = vhost_virtqueue_set_busyloop_timeout(dev, n, dev->busyloop_timeout);
>          if (r < 0) {
>              VHOST_OPS_DEBUG(r, "Failed to set busyloop timeout");
>              goto fail_err;
> @@ -1628,6 +1627,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>
>      hdev->vdev = NULL;
>      hdev->migration_blocker = NULL;
> +    hdev->busyloop_timeout = busyloop_timeout;
>
>      r = vhost_set_backend_type(hdev, backend_type);
>      assert(r >= 0);
> @@ -1650,8 +1650,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      }
>
>      for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
> -        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i,
> -                                 busyloop_timeout);
> +        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
>          if (r < 0) {
>              error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
>              goto fail;
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 1ba1af1d86..f1a7e7b971 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -105,6 +105,7 @@ struct vhost_dev {
>      VIRTIO_DECLARE_FEATURES(_features);
>      VIRTIO_DECLARE_FEATURES(acked_features);
>
> +    uint32_t busyloop_timeout;
>      uint64_t max_queues;
>      uint64_t backend_cap;
>      /* @started: is the vhost device started? */
> --
> 2.48.1
>
>

Re: [PATCH v2 02/25] vhost: reorder logic in vhost_dev_init()

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:47 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> We are going to split vhost_dev_init() so that the first part will do
> early initialization of QEMU structures, but don't communicate with
> backend, and the second part will do backend communication. We need
> this for future support for backend-transfer migration support for
> vhost-user-blk (backend will not be available in the early
> initialization point).
>
> With this commit, we simply reorder the logic in vhost_dev_init()
> in accordance with idea of further split.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c | 60 +++++++++++++++++++++++------------------------
>  1 file changed, 30 insertions(+), 30 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 9fc6e7ba65..551d1687fc 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1637,26 +1637,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>          goto fail;
>      }
>
> -    r = hdev->vhost_ops->vhost_set_owner(hdev);
> -    if (r < 0) {
> -        error_setg_errno(errp, -r, "vhost_set_owner failed");
> -        goto fail;
> -    }
> -
> -    r = vhost_dev_init_features(hdev);
> -    if (r < 0) {
> -        error_setg_errno(errp, -r, "vhost_init_features failed");
> -        goto fail;
> -    }
> -
> -    for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
> -        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
> -        if (r < 0) {
> -            error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
> -            goto fail;
> -        }
> -    }
> -
>      hdev->memory_listener = (MemoryListener) {
>          .name = "vhost",
>          .begin = vhost_begin,
> @@ -1677,6 +1657,36 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>          .region_del = vhost_iommu_region_del,
>      };
>
> +    hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
> +    hdev->n_mem_sections = 0;
> +    hdev->mem_sections = NULL;
> +    hdev->log = NULL;
> +    hdev->log_size = 0;
> +    hdev->log_enabled = false;
> +    hdev->started = false;
> +    memory_listener_register(&hdev->memory_listener, &address_space_memory);
> +    QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
> +
> +    r = hdev->vhost_ops->vhost_set_owner(hdev);
> +    if (r < 0) {
> +        error_setg_errno(errp, -r, "vhost_set_owner failed");
> +        goto fail;
> +    }
> +
> +    r = vhost_dev_init_features(hdev);
> +    if (r < 0) {
> +        error_setg_errno(errp, -r, "vhost_init_features failed");
> +        goto fail;
> +    }
> +
> +    for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
> +        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
> +        if (r < 0) {
> +            error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
> +            goto fail;
> +        }
> +    }
> +
>      if (hdev->migration_blocker == NULL) {
>          if (!vhost_dev_has_feature_ex(hdev, VHOST_F_LOG_ALL)) {
>              error_setg(&hdev->migration_blocker,
> @@ -1694,16 +1704,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>          }
>      }
>
> -    hdev->mem = g_malloc0(offsetof(struct vhost_memory, regions));
> -    hdev->n_mem_sections = 0;
> -    hdev->mem_sections = NULL;
> -    hdev->log = NULL;
> -    hdev->log_size = 0;
> -    hdev->log_enabled = false;
> -    hdev->started = false;
> -    memory_listener_register(&hdev->memory_listener, &address_space_memory);
> -    QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
> -
>      if (!check_memslots(hdev, errp)) {
>          r = -EINVAL;
>          goto fail;
> --
> 2.48.1
>
>

Re: [PATCH v2 03/25] vhost: rework vhost_virtqueue_init()

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:48 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> We are going to split vhost_dev_init() so that the first part will do
> early initialization of QEMU structures, but don't communicate with
> backend, and the second part will do backend communication. We need
> this for future support for backend-transfer migration support for
> vhost-user-blk (backend will not be available in the early
> initialization point).
>
> With this commit, let's split vhost_virtqueue_init(). The whole function
> is mostly about configuring the backend, so this logic will be postponed

nit: "until the backend becomes"

> until backend become available. The only thing to keep in early
> initialization is attaching vhost_dev structure. Let's simply move it to
> vhost_dev_init().
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c | 12 +++++++-----
>  1 file changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 551d1687fc..1998663461 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1500,9 +1500,9 @@ static void vhost_virtqueue_error_notifier(EventNotifier *n)
>      }
>  }
>
> -static int vhost_virtqueue_init(struct vhost_dev *dev,
> -                                struct vhost_virtqueue *vq, int n)
> +static int vhost_virtqueue_connect(struct vhost_virtqueue *vq, int n)
>  {
> +    struct vhost_dev *dev = vq->dev;
>      int vhost_vq_index = dev->vhost_ops->vhost_get_vq_index(dev, n);
>      struct vhost_vring_file file = {
>          .index = vhost_vq_index,
> @@ -1519,8 +1519,6 @@ static int vhost_virtqueue_init(struct vhost_dev *dev,
>          goto fail_call;
>      }
>
> -    vq->dev = dev;
> -
>      if (dev->vhost_ops->vhost_set_vring_err) {
>          r = event_notifier_init(&vq->error_notifier, 0);
>          if (r < 0) {
> @@ -1629,6 +1627,10 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      hdev->migration_blocker = NULL;
>      hdev->busyloop_timeout = busyloop_timeout;
>
> +    for (i = 0; i < hdev->nvqs; ++i) {
> +        hdev->vqs[i].dev = hdev;
> +    }
> +
>      r = vhost_set_backend_type(hdev, backend_type);
>      assert(r >= 0);
>
> @@ -1680,7 +1682,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      }
>
>      for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
> -        r = vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i);
> +        r = vhost_virtqueue_connect(hdev->vqs + i, hdev->vq_index + i);
>          if (r < 0) {
>              error_setg_errno(errp, -r, "Failed to initialize virtqueue %d", i);
>              goto fail;
> --
> 2.48.1
>
>

Re: [PATCH v2 04/25] vhost: add connect parameter to vhost_dev_init()

[Raphael Norwitz]

Looks ok, just a comment typo which is deleted in a subsequent commit.

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:47 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> We are going to split vhost_dev_init() so that the first part will do
> early initialization of QEMU structures, but don't communicate with
> backend, and the second part will do backend communication. We need
> this for future support for backend-transfer migration support for
> vhost-user-blk (backend will not be available in the early
> initialization point).
>
> With this commit we introduce boolean parameter for vhost_dev_init(),
> so callers may chose, do they want "init + connect" (which is current
> behavior, so all callers pass true), or caller may want "only init",
> and call vhost_dev_connect() later. vhost_dev_connect(), as well
> as support for connect=false will be added in further commits.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  backends/cryptodev-vhost.c        |  2 +-
>  backends/vhost-user.c             |  2 +-
>  hw/block/vhost-user-blk.c         |  2 +-
>  hw/net/vhost_net.c                |  2 +-
>  hw/scsi/vhost-scsi.c              |  2 +-
>  hw/scsi/vhost-user-scsi.c         |  2 +-
>  hw/virtio/vdpa-dev.c              |  3 ++-
>  hw/virtio/vhost-user-base.c       |  2 +-
>  hw/virtio/vhost-user-fs.c         |  2 +-
>  hw/virtio/vhost-user-scmi.c       |  2 +-
>  hw/virtio/vhost-user-vsock.c      |  2 +-
>  hw/virtio/vhost-vsock.c           |  2 +-
>  hw/virtio/vhost.c                 | 11 ++++++++++-
>  include/hw/virtio/vhost-backend.h |  2 ++
>  include/hw/virtio/vhost.h         |  3 ++-
>  15 files changed, 27 insertions(+), 14 deletions(-)
>
> diff --git a/backends/cryptodev-vhost.c b/backends/cryptodev-vhost.c
> index c6069f4e5b..b4dafb4062 100644
> --- a/backends/cryptodev-vhost.c
> +++ b/backends/cryptodev-vhost.c
> @@ -66,7 +66,7 @@ cryptodev_vhost_init(
>      crypto->dev.vq_index = crypto->cc->queue_index * crypto->dev.nvqs;
>
>      r = vhost_dev_init(&crypto->dev, options->opaque, options->backend_type, 0,
> -                       &local_err);
> +                       true, &local_err);
>      if (r < 0) {
>          error_report_err(local_err);
>          goto fail;
> diff --git a/backends/vhost-user.c b/backends/vhost-user.c
> index 42845329e7..e65ba7b648 100644
> --- a/backends/vhost-user.c
> +++ b/backends/vhost-user.c
> @@ -37,7 +37,7 @@ vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
>      b->dev.vqs = g_new0(struct vhost_virtqueue, nvqs);
>
>      ret = vhost_dev_init(&b->dev, &b->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
> -                         errp);
> +                         true, errp);
>      if (ret < 0) {
>          return -1;
>      }
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index a5daed4346..a92426f18c 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -365,7 +365,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
>
>      s->vhost_user.supports_config = true;
>      ret = vhost_dev_init(&s->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
> -                         errp);
> +                         true, errp);
>      if (ret < 0) {
>          return ret;
>      }
> diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c
> index 323d117735..c4526974fb 100644
> --- a/hw/net/vhost_net.c
> +++ b/hw/net/vhost_net.c
> @@ -274,7 +274,7 @@ struct vhost_net *vhost_net_init(VhostNetOptions *options)
>
>      r = vhost_dev_init(&net->dev, options->opaque,
>                         options->backend_type, options->busyloop_timeout,
> -                       &local_err);
> +                       true, &local_err);
>      if (r < 0) {
>          error_report_err(local_err);
>          goto fail;
> diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
> index d694a25fe2..d187c705d8 100644
> --- a/hw/scsi/vhost-scsi.c
> +++ b/hw/scsi/vhost-scsi.c
> @@ -278,7 +278,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
>      vsc->dev.vq_index = 0;
>
>      ret = vhost_dev_init(&vsc->dev, (void *)(uintptr_t)vhostfd,
> -                         VHOST_BACKEND_TYPE_KERNEL, 0, errp);
> +                         VHOST_BACKEND_TYPE_KERNEL, 0, true, errp);
>      if (ret < 0) {
>          /*
>           * vhost_dev_init calls vhost_dev_cleanup on error, which closes
> diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
> index 0c80a271d8..e121f2e259 100644
> --- a/hw/scsi/vhost-user-scsi.c
> +++ b/hw/scsi/vhost-user-scsi.c
> @@ -161,7 +161,7 @@ static int vhost_user_scsi_connect(DeviceState *dev, Error **errp)
>      vsc->dev.vq_index = 0;
>
>      ret = vhost_dev_init(&vsc->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
> -                         errp);
> +                         true, errp);
>      if (ret < 0) {
>          return ret;
>      }
> diff --git a/hw/virtio/vdpa-dev.c b/hw/virtio/vdpa-dev.c
> index e1a2ff433d..b6b4ee7d38 100644
> --- a/hw/virtio/vdpa-dev.c
> +++ b/hw/virtio/vdpa-dev.c
> @@ -116,7 +116,8 @@ static void vhost_vdpa_device_realize(DeviceState *dev, Error **errp)
>      v->vdpa.shared->device_fd = v->vhostfd;
>      v->vdpa.shared->iova_range = iova_range;
>
> -    ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, NULL);
> +    ret = vhost_dev_init(&v->dev, &v->vdpa, VHOST_BACKEND_TYPE_VDPA, 0, true,
> +                         NULL);
>      if (ret < 0) {
>          error_setg(errp, "vhost-vdpa-device: vhost initialization failed: %s",
>                     strerror(-ret));
> diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
> index cf311c3bfc..0768231a88 100644
> --- a/hw/virtio/vhost-user-base.c
> +++ b/hw/virtio/vhost-user-base.c
> @@ -334,7 +334,7 @@ static void vub_device_realize(DeviceState *dev, Error **errp)
>
>      /* connect to backend */
>      ret = vhost_dev_init(&vub->vhost_dev, &vub->vhost_user,
> -                         VHOST_BACKEND_TYPE_USER, 0, errp);
> +                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
>
>      if (ret < 0) {
>          do_vhost_user_cleanup(vdev, vub);
> diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
> index e77c69eb12..2a8eead90b 100644
> --- a/hw/virtio/vhost-user-fs.c
> +++ b/hw/virtio/vhost-user-fs.c
> @@ -256,7 +256,7 @@ static void vuf_device_realize(DeviceState *dev, Error **errp)
>      fs->vhost_dev.nvqs = 1 + fs->conf.num_request_queues;
>      fs->vhost_dev.vqs = g_new0(struct vhost_virtqueue, fs->vhost_dev.nvqs);
>      ret = vhost_dev_init(&fs->vhost_dev, &fs->vhost_user,
> -                         VHOST_BACKEND_TYPE_USER, 0, errp);
> +                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
>      if (ret < 0) {
>          goto err_virtio;
>      }
> diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
> index f9264c4374..40e567c18a 100644
> --- a/hw/virtio/vhost-user-scmi.c
> +++ b/hw/virtio/vhost-user-scmi.c
> @@ -253,7 +253,7 @@ static void vu_scmi_device_realize(DeviceState *dev, Error **errp)
>      scmi->vhost_dev.vqs = g_new0(struct vhost_virtqueue, scmi->vhost_dev.nvqs);
>
>      ret = vhost_dev_init(&scmi->vhost_dev, &scmi->vhost_user,
> -                         VHOST_BACKEND_TYPE_USER, 0, errp);
> +                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
>      if (ret < 0) {
>          error_setg_errno(errp, -ret,
>                           "vhost-user-scmi: vhost_dev_init() failed");
> diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
> index 993c287348..b630af0fe7 100644
> --- a/hw/virtio/vhost-user-vsock.c
> +++ b/hw/virtio/vhost-user-vsock.c
> @@ -115,7 +115,7 @@ static void vuv_device_realize(DeviceState *dev, Error **errp)
>      vhost_dev_set_config_notifier(&vvc->vhost_dev, &vsock_ops);
>
>      ret = vhost_dev_init(&vvc->vhost_dev, &vsock->vhost_user,
> -                         VHOST_BACKEND_TYPE_USER, 0, errp);
> +                         VHOST_BACKEND_TYPE_USER, 0, true, errp);
>      if (ret < 0) {
>          goto err_virtio;
>      }
> diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
> index 107d88babe..3a4b2d924d 100644
> --- a/hw/virtio/vhost-vsock.c
> +++ b/hw/virtio/vhost-vsock.c
> @@ -166,7 +166,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp)
>      vhost_vsock_common_realize(vdev);
>
>      ret = vhost_dev_init(&vvc->vhost_dev, (void *)(uintptr_t)vhostfd,
> -                         VHOST_BACKEND_TYPE_KERNEL, 0, errp);
> +                         VHOST_BACKEND_TYPE_KERNEL, 0, true, errp);
>      if (ret < 0) {
>          /*
>           * vhostfd is closed by vhost_dev_cleanup, which is called
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 1998663461..f733e98b4a 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1617,7 +1617,7 @@ static bool check_memslots(struct vhost_dev *hdev, Error **errp)
>
>  int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>                     VhostBackendType backend_type, uint32_t busyloop_timeout,
> -                   Error **errp)
> +                   bool connect, Error **errp)
>  {
>      int i, r, n_initialized_vqs = 0;
>
> @@ -1634,6 +1634,15 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      r = vhost_set_backend_type(hdev, backend_type);
>      assert(r >= 0);
>
> +    /*
> +     * Postponed connect only supported for devices with
> +     * .vhost_backend_connect handler
> +     */
> +    assert(connect || hdev->vhost_ops->vhost_backend_connect);
> +

nit: TODO

> +    /* TDDO: support connect=false */
> +    assert(connect);
> +
>      r = hdev->vhost_ops->vhost_backend_init(hdev, opaque, errp);
>      if (r < 0) {
>          goto fail;
> diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
> index ff94fa1734..d3f055f95e 100644
> --- a/include/hw/virtio/vhost-backend.h
> +++ b/include/hw/virtio/vhost-backend.h
> @@ -53,6 +53,7 @@ struct vhost_virtqueue;
>
>  typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque,
>                                    Error **errp);
> +typedef int (*vhost_backend_connect)(struct vhost_dev *dev, Error **errp);
>  typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev);
>  typedef int (*vhost_backend_memslots_limit)(struct vhost_dev *dev);
>
> @@ -167,6 +168,7 @@ typedef int (*vhost_check_device_state_op)(struct vhost_dev *dev, Error **errp);
>  typedef struct VhostOps {
>      VhostBackendType backend_type;
>      vhost_backend_init vhost_backend_init;
> +    vhost_backend_connect vhost_backend_connect;
>      vhost_backend_cleanup vhost_backend_cleanup;
>      vhost_backend_memslots_limit vhost_backend_memslots_limit;
>      vhost_backend_no_private_memslots_op vhost_backend_no_private_memslots;
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index f1a7e7b971..74ed24232e 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -155,7 +155,8 @@ struct vhost_net {
>   */
>  int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>                     VhostBackendType backend_type,
> -                   uint32_t busyloop_timeout, Error **errp);
> +                   uint32_t busyloop_timeout,
> +                   bool connect, Error **errp);
>
>  /**
>   * vhost_dev_cleanup() - tear down and cleanup vhost interface
> --
> 2.48.1
>
>

Re: [PATCH v2 05/25] vhost: split vhost_dev_connect() out of vhost_dev_init()

[Raphael Norwitz]

Just a tracepoint suggestion. Otherwise LGTM.

On Thu, Oct 16, 2025 at 7:46 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Split vhost_dev_init() so that the first part will do early
> initialization of QEMU structures, but don't communicate with backend,
> and the second part will do backend communication.
>
> We need this for future support for backend-transfer migration support
> for vhost-user-blk (backend will not be available in the early
> initialization point).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c         | 27 +++++++++++++++++++++------
>  include/hw/virtio/vhost.h |  2 ++
>  2 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index f733e98b4a..09d00e4d98 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1619,7 +1619,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>                     VhostBackendType backend_type, uint32_t busyloop_timeout,
>                     bool connect, Error **errp)
>  {
> -    int i, r, n_initialized_vqs = 0;
> +    int i, r;
>
>      trace_vhost_dev_init_in(hdev);
>
> @@ -1640,9 +1640,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>       */
>      assert(connect || hdev->vhost_ops->vhost_backend_connect);
>
> -    /* TDDO: support connect=false */
> -    assert(connect);
> -
>      r = hdev->vhost_ops->vhost_backend_init(hdev, opaque, errp);
>      if (r < 0) {
>          goto fail;
> @@ -1678,6 +1675,26 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>      memory_listener_register(&hdev->memory_listener, &address_space_memory);
>      QLIST_INSERT_HEAD(&vhost_devices, hdev, entry);
>
> +    trace_vhost_dev_init_out(hdev);
> +
> +    return connect ? vhost_dev_connect(hdev, errp) : 0;
> +
> +fail:
> +    vhost_dev_cleanup(hdev);
> +    return r;
> +}
> +
> +int vhost_dev_connect(struct vhost_dev *hdev, Error **errp)
> +{
> +    int i, r, n_initialized_vqs = 0;
> +

Let's add tracepoints for connect here?

> +    if (hdev->vhost_ops->vhost_backend_connect) {
> +        r = hdev->vhost_ops->vhost_backend_connect(hdev, errp);
> +        if (r < 0) {
> +            goto fail;
> +        }
> +    }
> +
>      r = hdev->vhost_ops->vhost_set_owner(hdev);
>      if (r < 0) {
>          error_setg_errno(errp, -r, "vhost_set_owner failed");
> @@ -1720,8 +1737,6 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>          goto fail;
>      }
>
> -    trace_vhost_dev_init_out(hdev);
> -
>      return 0;
>
>  fail:
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 74ed24232e..af46d4b5f2 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -158,6 +158,8 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>                     uint32_t busyloop_timeout,
>                     bool connect, Error **errp);
>
> +int vhost_dev_connect(struct vhost_dev *hdev, Error **errp);
> +
>  /**
>   * vhost_dev_cleanup() - tear down and cleanup vhost interface
>   * @hdev: the common vhost_dev structure
> --
> 2.48.1
>
>

Re: [PATCH v2 06/25] vhost-user: support connect api

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:47 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Memory allocation and connecting of structures remain ins _init(),
> communication with backend goes to _connect().
>
> We need this for further support of backend-transfer migration of
> vhost-user-blk, as we'll need to postpone (or not do) initial
> communication to backend.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost-user.c | 32 ++++++++++++++++++++------------
>  1 file changed, 20 insertions(+), 12 deletions(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index e45b74eddd..c5cb5ed528 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -2270,21 +2270,12 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>      return 0;
>  }
>
> -static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
> -                                   Error **errp)
> +static int vhost_user_backend_connect(struct vhost_dev *dev, Error **errp)
>  {
>      uint64_t features, ram_slots;
> -    struct vhost_user *u;
> -    VhostUserState *vus = (VhostUserState *) opaque;
> +    struct vhost_user *u = dev->opaque;
>      int err;
>
> -    assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
> -
> -    u = g_new0(struct vhost_user, 1);
> -    u->user = vus;
> -    u->dev = dev;
> -    dev->opaque = u;
> -
>      err = vhost_user_get_features(dev, &features);
>      if (err < 0) {
>          error_setg_errno(errp, -err, "vhost_backend_init failed");
> @@ -2292,7 +2283,7 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>      }
>
>      if (virtio_has_feature(features, VHOST_USER_F_PROTOCOL_FEATURES)) {
> -        bool supports_f_config = vus->supports_config ||
> +        bool supports_f_config = u->user->supports_config ||
>              (dev->config_ops && dev->config_ops->vhost_dev_config_notifier);
>          uint64_t protocol_features;
>
> @@ -2408,6 +2399,22 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>      return 0;
>  }
>
> +static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
> +                                   Error **errp)
> +{
> +    struct vhost_user *u;
> +    VhostUserState *vus = (VhostUserState *) opaque;
> +
> +    assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER);
> +
> +    u = g_new0(struct vhost_user, 1);
> +    u->user = vus;
> +    u->dev = dev;
> +    dev->opaque = u;
> +
> +    return 0;
> +}
> +
>  static int vhost_user_backend_cleanup(struct vhost_dev *dev)
>  {
>      struct vhost_user *u;
> @@ -3133,6 +3140,7 @@ void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status)
>  const VhostOps user_ops = {
>          .backend_type = VHOST_BACKEND_TYPE_USER,
>          .vhost_backend_init = vhost_user_backend_init,
> +        .vhost_backend_connect = vhost_user_backend_connect,
>          .vhost_backend_cleanup = vhost_user_backend_cleanup,
>          .vhost_backend_memslots_limit = vhost_user_memslots_limit,
>          .vhost_backend_no_private_memslots = vhost_user_no_private_memslots,
> --
> 2.48.1
>
>

Re: [PATCH v2 07/25] vhost-user-blk: vhost_user_blk_connect() move connected check to caller

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:47 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> vhost_user_blk_connect() has two callers:
>
> - vhost_user_blk_realize_connect(), which directly set .connected = false
> before call
>
> - vhost_user_blk_event(), where we want this check
>
> Move the check to the only caller which needs it, to simplify further
> refactoring.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index a92426f18c..57214a69cd 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -352,9 +352,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
>
>      trace_vhost_user_blk_connect_in(vdev);
>
> -    if (s->connected) {
> -        return 0;
> -    }
> +    assert(!s->connected);
>
>      s->dev.num_queues = s->num_queues;
>      s->dev.nvqs = s->num_queues;
> @@ -411,10 +409,12 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event)
>
>      switch (event) {
>      case CHR_EVENT_OPENED:
> -        if (vhost_user_blk_connect(dev, &local_err) < 0) {
> -            error_report_err(local_err);
> -            qemu_chr_fe_disconnect(&s->chardev);
> -            return;
> +        if (!s->connected) {
> +            if (vhost_user_blk_connect(dev, &local_err) < 0) {
> +                error_report_err(local_err);
> +                qemu_chr_fe_disconnect(&s->chardev);
> +                return;
> +            }
>          }
>          break;
>      case CHR_EVENT_CLOSED:
> --
> 2.48.1
>
>

Re: [PATCH v2 08/25] vhost-user-blk: vhost_user_blk_connect(): call vhost_dev_connect()

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:46 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Call vhost_dev_connect() directly, to simplify further refactoring.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index 57214a69cd..f2ecf81e4d 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -363,7 +363,12 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
>
>      s->vhost_user.supports_config = true;
>      ret = vhost_dev_init(&s->dev, &s->vhost_user, VHOST_BACKEND_TYPE_USER, 0,
> -                         true, errp);
> +                         false, errp);
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    ret = vhost_dev_connect(&s->dev, errp);
>      if (ret < 0) {
>          return ret;
>      }
> --
> 2.48.1
>
>

Re: [PATCH v2 09/25] vhost-user-blk: rename vhost_user_blk_connect to vhost_user_blk_init

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:44 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> The function does both vhost_dev_init() and vhost_dev_connect().
> Following interface of vhost_dev_init(), and preparing to further
> refactoring, let's rename to _init() and add boolean "connect"
> parameter.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/trace-events     |  2 ++
>  hw/block/vhost-user-blk.c | 13 ++++++++-----
>  2 files changed, 10 insertions(+), 5 deletions(-)
>
> diff --git a/hw/block/trace-events b/hw/block/trace-events
> index dbaa5ca6cb..9f00412a99 100644
> --- a/hw/block/trace-events
> +++ b/hw/block/trace-events
> @@ -63,6 +63,8 @@ vhost_user_blk_start_in(void *vdev) "vdev %p"
>  vhost_user_blk_start_out(void *vdev) "vdev %p"
>  vhost_user_blk_stop_in(void *vdev) "vdev %p"
>  vhost_user_blk_stop_out(void *vdev) "vdev %p"
> +vhost_user_blk_init_in(void *vdev) "vdev %p"
> +vhost_user_blk_init_out(void *vdev) "vdev %p"
>  vhost_user_blk_connect_in(void *vdev) "vdev %p"
>  vhost_user_blk_connect_out(void *vdev) "vdev %p"
>  vhost_user_blk_device_realize_in(void *vdev) "vdev %p"
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index f2ecf81e4d..c31c265a0e 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -344,13 +344,16 @@ static void vhost_user_blk_reset(VirtIODevice *vdev)
>      vhost_dev_free_inflight(s->inflight);
>  }
>
> -static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
> +static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
>  {
>      VirtIODevice *vdev = VIRTIO_DEVICE(dev);
>      VHostUserBlk *s = VHOST_USER_BLK(vdev);
>      int ret = 0;
>
> -    trace_vhost_user_blk_connect_in(vdev);
> +    trace_vhost_user_blk_init_in(vdev);
> +
> +    /* TODO: implement support for connect=false */
> +    assert(connect);
>
>      assert(!s->connected);
>
> @@ -380,7 +383,7 @@ static int vhost_user_blk_connect(DeviceState *dev, Error **errp)
>          ret = vhost_user_blk_start(vdev, errp);
>      }
>
> -    trace_vhost_user_blk_connect_out(vdev);
> +    trace_vhost_user_blk_init_out(vdev);
>
>      return ret;
>  }
> @@ -415,7 +418,7 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event)
>      switch (event) {
>      case CHR_EVENT_OPENED:
>          if (!s->connected) {
> -            if (vhost_user_blk_connect(dev, &local_err) < 0) {
> +            if (vhost_user_blk_init(dev, true, &local_err) < 0) {
>                  error_report_err(local_err);
>                  qemu_chr_fe_disconnect(&s->chardev);
>                  return;
> @@ -447,7 +450,7 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>          return ret;
>      }
>
> -    ret = vhost_user_blk_connect(dev, errp);
> +    ret = vhost_user_blk_init(dev, true, errp);
>      if (ret < 0) {
>          qemu_chr_fe_disconnect(&s->chardev);
>          return ret;
> --
> 2.48.1
>
>

Re: [PATCH v2 10/25] vhost-user-blk: split vhost_user_blk_init()

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:46 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Split it into _init() and _connect() part, following pattern of
> vhost_dev_init / vhost_dev_connect.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 27 ++++++++++++++++++++++++---
>  1 file changed, 24 insertions(+), 3 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index c31c265a0e..9c727c3977 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -58,6 +58,7 @@ static const int user_feature_bits[] = {
>  };
>
>  static void vhost_user_blk_event(void *opaque, QEMUChrEvent event);
> +static int vhost_user_blk_connect(DeviceState *dev, Error **errp);
>
>  static void vhost_user_blk_update_config(VirtIODevice *vdev, uint8_t *config)
>  {
> @@ -352,9 +353,6 @@ static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
>
>      trace_vhost_user_blk_init_in(vdev);
>
> -    /* TODO: implement support for connect=false */
> -    assert(connect);
> -
>      assert(!s->connected);
>
>      s->dev.num_queues = s->num_queues;
> @@ -371,6 +369,29 @@ static int vhost_user_blk_init(DeviceState *dev, bool connect, Error **errp)
>          return ret;
>      }
>
> +    if (connect) {
> +        ret = vhost_user_blk_connect(dev, errp);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +    }
> +
> +    trace_vhost_user_blk_init_out(vdev);
> +
> +    return 0;
> +}
> +
> +static int vhost_user_blk_connect(DeviceState *dev,
> +                                  Error **errp)
> +{
> +    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> +    VHostUserBlk *s = VHOST_USER_BLK(vdev);
> +    int ret = 0;
> +
> +    trace_vhost_user_blk_connect_in(vdev);
> +
> +    assert(!s->connected);
> +
>      ret = vhost_dev_connect(&s->dev, errp);
>      if (ret < 0) {
>          return ret;
> --
> 2.48.1
>
>

Re: [PATCH v2 11/25] vhost-user-blk: move initial reconnect loop to separate function

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:46 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Simplify _realize function, and prepare to further changes.
>
> While being here, also rename virtio_err: label to more generic
> fail:, virtio_err doesn't improve readability here.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 54 +++++++++++++++++++++++----------------
>  1 file changed, 32 insertions(+), 22 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index 9c727c3977..36e32229ad 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -489,14 +489,40 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>      return 0;
>  }
>
> -static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
> +static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
>  {
>      ERRP_GUARD();
> +    DeviceState *dev = DEVICE(s);
> +    int ret, retries = VU_REALIZE_CONN_RETRIES;
> +
> +    assert(!*errp);
> +    do {
> +        if (*errp) {
> +            error_prepend(errp, "Reconnecting after error: ");
> +            error_report_err(*errp);
> +            *errp = NULL;
> +        }
> +        ret = vhost_user_blk_realize_connect(s, errp);
> +    } while (ret < 0 && retries--);
> +
> +    if (ret < 0) {
> +        return ret;
> +    }
> +
> +    /* we're fully initialized, now we can operate, so add the handler */
> +    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
> +                             vhost_user_blk_event, NULL, (void *)dev,
> +                             NULL, true);
> +
> +    return 0;
> +}
> +
> +static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
> +{
>      VirtIODevice *vdev = VIRTIO_DEVICE(dev);
>      VHostUserBlk *s = VHOST_USER_BLK(vdev);
>      size_t config_size;
> -    int retries;
> -    int i, ret;
> +    int i;
>
>      trace_vhost_user_blk_device_realize_in(vdev);
>
> @@ -540,31 +566,15 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>      s->inflight = g_new0(struct vhost_inflight, 1);
>      s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
>
> -    retries = VU_REALIZE_CONN_RETRIES;
> -    assert(!*errp);
> -    do {
> -        if (*errp) {
> -            error_prepend(errp, "Reconnecting after error: ");
> -            error_report_err(*errp);
> -            *errp = NULL;
> -        }
> -        ret = vhost_user_blk_realize_connect(s, errp);
> -    } while (ret < 0 && retries--);
> -
> -    if (ret < 0) {
> -        goto virtio_err;
> +    if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
> +        goto fail;
>      }
>
> -    /* we're fully initialized, now we can operate, so add the handler */
> -    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
> -                             vhost_user_blk_event, NULL, (void *)dev,
> -                             NULL, true);
> -
>      trace_vhost_user_blk_device_realize_out(vdev);
>
>      return;
>
> -virtio_err:
> +fail:
>      g_free(s->vhost_vqs);
>      s->vhost_vqs = NULL;
>      g_free(s->inflight);
> --
> 2.48.1
>
>

Re: [PATCH v2 12/25] vhost-user-blk: move first vhost_user_blk_init() to _realize()

[Raphael Norwitz]

On Thu, Oct 16, 2025 at 7:48 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> We'll need to postpone further connecting/reconnecting logic to the
> later point to support backend-transfer migration for vhost-user-blk.
> For now, move first call to vhost_user_blk_init() to _realize() (this
> call will not be postponed). To support this, we also have to move
> re-initialization to vhost_user_blk_realize_connect_loop().
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 17 ++++++++++++++---
>  1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index 36e32229ad..af4a97b8e4 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -464,14 +464,12 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>      DeviceState *dev = DEVICE(s);
>      int ret;
>
> -    s->connected = false;
> -
>      ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
>      if (ret < 0) {
>          return ret;
>      }
>
> -    ret = vhost_user_blk_init(dev, true, errp);
> +    ret = vhost_user_blk_connect(dev, errp);
>      if (ret < 0) {
>          qemu_chr_fe_disconnect(&s->chardev);
>          return ret;
> @@ -501,7 +499,16 @@ static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
>              error_prepend(errp, "Reconnecting after error: ");
>              error_report_err(*errp);
>              *errp = NULL;
> +
> +            s->connected = false;
> +
> +            ret = vhost_user_blk_init(dev, false, errp);
> +            if (ret < 0) {
> +                /* No reason to retry initialization */
> +                return ret;
> +            }
>          }
> +
>          ret = vhost_user_blk_realize_connect(s, errp);
>      } while (ret < 0 && retries--);
>
> @@ -566,6 +573,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>      s->inflight = g_new0(struct vhost_inflight, 1);
>      s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
>

Why call vhost_user_blk_init() here if we call it in
host_user_blk_realize_connect_loop()?

> +    if (vhost_user_blk_init(dev, false, errp) < 0) {
> +        goto fail;
> +    }
> +
>      if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
>          goto fail;
>      }
> --
> 2.48.1
>
>

Re: [PATCH v2 13/25] vhost-user-blk: postpone connect to pre-incoming

[Raphael Norwitz]

Acked-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:46 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> That's a preparation for further backend-transfer migration for
> vhost-user-blk. At initialization time we don't know will
> user enable backent-transfer (by setting migration parameter) or
> not. At time of pre-incoming, we know all migration parameters and
> capabilities. So, now, let's postpone connecting up to pre-incoming
> for incoming migration.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c | 16 +++++++++++++---
>  1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index af4a97b8e4..ffdd600526 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -577,8 +577,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>          goto fail;
>      }
>
> -    if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
> -        goto fail;
> +    if (!runstate_check(RUN_STATE_INMIGRATE)) {
> +        if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
> +            goto fail;
> +        }
>      }
>
>      trace_vhost_user_blk_device_realize_out(vdev);
> @@ -636,10 +638,18 @@ static struct vhost_dev *vhost_user_blk_get_vhost(VirtIODevice *vdev)
>      return &s->dev;
>  }
>
> +static bool vhost_user_blk_pre_incoming(void *opaque, Error **errp)
> +{
> +    VHostUserBlk *s = VHOST_USER_BLK(opaque);
> +
> +    return vhost_user_blk_realize_connect(s, errp) == 0;
> +}
> +
>  static const VMStateDescription vmstate_vhost_user_blk = {
>      .name = "vhost-user-blk",
>      .minimum_version_id = 1,
>      .version_id = 1,
> +    .pre_incoming = vhost_user_blk_pre_incoming,
>      .fields = (const VMStateField[]) {
>          VMSTATE_VIRTIO_DEVICE,
>          VMSTATE_END_OF_LIST()
> @@ -647,7 +657,7 @@ static const VMStateDescription vmstate_vhost_user_blk = {
>  };
>
>  static const Property vhost_user_blk_properties[] = {
> -    DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev),
> +    DEFINE_PROP_CHR_NO_CONNECT("chardev", VHostUserBlk, chardev),
>      DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
>                         VHOST_USER_BLK_AUTO_NUM_QUEUES),
>      DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128),
> --
> 2.48.1
>
>

Re: [PATCH v2 14/25] virtio: introduce .skip_vhost_migration_log() handler

[Raphael Norwitz]

Reviewed-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:49 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> For vhost user backend migration we'll need to disable memory
> logging on the device. Let's prepare a corresponding handler for
> the device.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c          | 10 ++++++++++
>  include/hw/virtio/virtio.h |  2 ++
>  2 files changed, 12 insertions(+)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 09d00e4d98..3e5192ec23 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1138,6 +1138,16 @@ static int vhost_migration_log(MemoryListener *listener, bool enable)
>      struct vhost_dev *dev = container_of(listener, struct vhost_dev,
>                                           memory_listener);
>      int r;
> +
> +    if (dev->vdev) {
> +        VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(dev->vdev);
> +
> +        if (vdc->skip_vhost_migration_log &&
> +            vdc->skip_vhost_migration_log(dev->vdev)) {
> +            return 0;
> +        }
> +    }
> +
>      if (enable == dev->log_enabled) {
>          return 0;
>      }
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 7db8046766..620ee4e389 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -238,6 +238,8 @@ struct VirtioDeviceClass {
>      /* May be called even when vdev->vhost_started is false */
>      struct vhost_dev *(*get_vhost)(VirtIODevice *vdev);
>      void (*toggle_device_iotlb)(VirtIODevice *vdev);
> +
> +    bool (*skip_vhost_migration_log)(VirtIODevice *vdev);
>  };
>
>  void virtio_instance_init_common(Object *proxy_obj, void *data,
> --
> 2.48.1
>
>

Re: [PATCH v2 17/25] vhost: add inflight region backend-transfer vmstate

[Raphael Norwitz]

Acked-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:49 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Prepare for future backend-transfer migration of vhost-user-blk.
> Among other things we'll need to transfer the inflight region, let's
> prepare for this.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c         | 26 ++++++++++++++++++++++++++
>  include/hw/virtio/vhost.h |  5 +++++
>  2 files changed, 31 insertions(+)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 3e5192ec23..63036f8214 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1975,6 +1975,32 @@ void vhost_get_features_ex(struct vhost_dev *hdev,
>      }
>  }
>
> +static int vhost_inflight_backend_transfer_post_load(void *opaque,
> +                                                     int version_id)
> +{
> +    struct vhost_inflight *inflight = opaque;
> +
> +    inflight->addr = mmap(0, inflight->size, PROT_READ | PROT_WRITE,
> +                             MAP_SHARED, inflight->fd, inflight->offset);
> +    if (inflight->addr == MAP_FAILED) {
> +        return -EINVAL;
> +    }
> +
> +    return 0;
> +}
> +
> +const VMStateDescription vmstate_backend_transfer_vhost_inflight = {
> +    .name = "vhost-inflight",
> +    .post_load = vhost_inflight_backend_transfer_post_load,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_FD(fd, struct vhost_inflight),
> +        VMSTATE_UINT64(size, struct vhost_inflight),
> +        VMSTATE_UINT64(offset, struct vhost_inflight),
> +        VMSTATE_UINT16(queue_size, struct vhost_inflight),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
>                             const uint64_t *features)
>  {
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index af46d4b5f2..94a0c75fc8 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -587,4 +587,9 @@ static inline int vhost_load_backend_state(struct vhost_dev *dev, QEMUFile *f,
>  }
>  #endif
>
> +extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
> +#define VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(_field, _state) \
> +    VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
> +                           struct vhost_inflight)
> +
>  #endif
> --
> 2.48.1
>
>

Re: [PATCH v2 19/25] vhost: support backend-transfer migration

[Raphael Norwitz]

Overall looks good. Just a nit.

Acked-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>

On Thu, Oct 16, 2025 at 7:44 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Introduce vhost_dev.backend_transfer field,
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost.c         | 121 +++++++++++++++++++++++++++++++++-----
>  include/hw/virtio/vhost.h |   7 +++
>  2 files changed, 113 insertions(+), 15 deletions(-)
>
> diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
> index 63036f8214..c46203eb9c 100644
> --- a/hw/virtio/vhost.c
> +++ b/hw/virtio/vhost.c
> @@ -1325,6 +1325,8 @@ out:
>      return ret;
>  }
>
> +static void vhost_virtqueue_error_notifier(EventNotifier *n);
> +
>  int vhost_virtqueue_start(struct vhost_dev *dev,
>                            struct VirtIODevice *vdev,
>                            struct vhost_virtqueue *vq,
> @@ -1350,7 +1352,13 @@ int vhost_virtqueue_start(struct vhost_dev *dev,
>          return r;
>      }
>
> -    vq->num = state.num = virtio_queue_get_num(vdev, idx);
> +    vq->num = virtio_queue_get_num(vdev, idx);
> +
> +    if (dev->backend_transfer) {
> +        return 0;
> +    }
> +
> +    state.num = vq->num;
>      r = dev->vhost_ops->vhost_set_vring_num(dev, &state);
>      if (r) {
>          VHOST_OPS_DEBUG(r, "vhost_set_vring_num failed");
> @@ -1428,6 +1436,10 @@ static int do_vhost_virtqueue_stop(struct vhost_dev *dev,
>
>      trace_vhost_virtque_stop_in(dev, vdev->name, idx);
>
> +    if (dev->backend_transfer) {
> +        return 0;
> +    }
> +
>      if (virtio_queue_get_desc_addr(vdev, idx) == 0) {
>          /* Don't stop the virtqueue which might have not been started */
>          return 0;
> @@ -1565,10 +1577,14 @@ fail_call:
>
>  static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
>  {
> -    event_notifier_cleanup(&vq->masked_notifier);
> +    if (!vq->dev->backend_transfer) {
> +        event_notifier_cleanup(&vq->masked_notifier);
> +    }
>      if (vq->dev->vhost_ops->vhost_set_vring_err) {
>          event_notifier_set_handler(&vq->error_notifier, NULL);
> -        event_notifier_cleanup(&vq->error_notifier);
> +        if (!vq->dev->backend_transfer) {
> +            event_notifier_cleanup(&vq->error_notifier);
> +        }
>      }
>  }
>
> @@ -1635,6 +1651,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
>
>      hdev->vdev = NULL;
>      hdev->migration_blocker = NULL;
> +    hdev->_features_wait_incoming = true;
>      hdev->busyloop_timeout = busyloop_timeout;
>
>      for (i = 0; i < hdev->nvqs; ++i) {
> @@ -1717,6 +1734,8 @@ int vhost_dev_connect(struct vhost_dev *hdev, Error **errp)
>          goto fail;
>      }
>
> +    hdev->_features_wait_incoming = false;
> +
>      for (i = 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) {
>          r = vhost_virtqueue_connect(hdev->vqs + i, hdev->vq_index + i);
>          if (r < 0) {
> @@ -1808,8 +1827,11 @@ void vhost_dev_disable_notifiers_nvqs(struct vhost_dev *hdev,
>       */
>      memory_region_transaction_commit();
>
> -    for (i = 0; i < nvqs; ++i) {
> -        virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index + i);
> +    if (!hdev->backend_transfer) {
> +        for (i = 0; i < nvqs; ++i) {
> +            virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus),
> +                                             hdev->vq_index + i);
> +        }
>      }
>      virtio_device_release_ioeventfd(vdev);
>  }
> @@ -1967,6 +1989,11 @@ void vhost_get_features_ex(struct vhost_dev *hdev,
>  {
>      const int *bit = feature_bits;
>
> +    if (hdev->_features_wait_incoming) {
> +        /* Excessive set is enough for early initialization. */
> +        return;
> +    }
> +
>      while (*bit != VHOST_INVALID_FEATURE_BIT) {
>          if (!vhost_dev_has_feature_ex(hdev, *bit)) {
>              virtio_clear_feature_ex(features, *bit);
> @@ -2001,6 +2028,54 @@ const VMStateDescription vmstate_backend_transfer_vhost_inflight = {
>      }
>  };
>
> +const VMStateDescription vmstate_vhost_virtqueue = {
> +    .name = "vhost-virtqueue",
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_EVENT_NOTIFIER(error_notifier, struct vhost_virtqueue),
> +        VMSTATE_EVENT_NOTIFIER(masked_notifier, struct vhost_virtqueue),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
> +static int vhost_dev_post_load(void *opaque, int version_id)
> +{
> +    struct vhost_dev *hdev = opaque;
> +    Error *err = NULL;
> +    int i;
> +
> +    if (!check_memslots(hdev, &err)) {
> +        error_report_err(err);
> +        return -EINVAL;
> +    }
> +
> +    hdev->_features_wait_incoming = false;
> +
> +    if (hdev->vhost_ops->vhost_set_vring_err) {
> +        for (i = 0; i < hdev->nvqs; ++i) {
> +            event_notifier_set_handler(&hdev->vqs[i].error_notifier,
> +                                       vhost_virtqueue_error_notifier);
> +        }
> +    }
> +

nit: spurious newline

> +
> +    return 0;
> +}
> +
> +const VMStateDescription vmstate_vhost_dev = {
> +    .name = "vhost-dev",
> +    .post_load = vhost_dev_post_load,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_UINT64(_features, struct vhost_dev),
> +        VMSTATE_UINT64(max_queues, struct vhost_dev),
> +        VMSTATE_UINT32_EQUAL(nvqs, struct vhost_dev, NULL),
> +        VMSTATE_STRUCT_VARRAY_POINTER_UINT32(vqs, struct vhost_dev,
> +                                             nvqs,
> +                                             vmstate_vhost_virtqueue,
> +                                             struct vhost_virtqueue),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
>  void vhost_ack_features_ex(struct vhost_dev *hdev, const int *feature_bits,
>                             const uint64_t *features)
>  {
> @@ -2127,19 +2202,24 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
>      hdev->started = true;
>      hdev->vdev = vdev;
>
> -    r = vhost_dev_set_features(hdev, hdev->log_enabled);
> -    if (r < 0) {
> -        goto fail_features;
> +    if (!hdev->backend_transfer) {
> +        r = vhost_dev_set_features(hdev, hdev->log_enabled);
> +        if (r < 0) {
> +            warn_report("%s %d", __func__, __LINE__);
> +            goto fail_features;
> +        }
>      }
>
>      if (vhost_dev_has_iommu(hdev)) {
>          memory_listener_register(&hdev->iommu_listener, vdev->dma_as);
>      }
>
> -    r = hdev->vhost_ops->vhost_set_mem_table(hdev, hdev->mem);
> -    if (r < 0) {
> -        VHOST_OPS_DEBUG(r, "vhost_set_mem_table failed");
> -        goto fail_mem;
> +    if (!hdev->backend_transfer) {
> +        r = hdev->vhost_ops->vhost_set_mem_table(hdev, hdev->mem);
> +        if (r < 0) {
> +            VHOST_OPS_DEBUG(r, "vhost_set_mem_table failed");
> +            goto fail_mem;
> +        }
>      }
>      for (i = 0; i < hdev->nvqs; ++i) {
>          r = vhost_virtqueue_start(hdev,
> @@ -2179,13 +2259,13 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
>          }
>          vhost_dev_elect_mem_logger(hdev, true);
>      }
> -    if (vrings) {
> +    if (vrings && !hdev->backend_transfer) {
>          r = vhost_dev_set_vring_enable(hdev, true);
>          if (r) {
>              goto fail_log;
>          }
>      }
> -    if (hdev->vhost_ops->vhost_dev_start) {
> +    if (hdev->vhost_ops->vhost_dev_start && !hdev->backend_transfer) {
>          r = hdev->vhost_ops->vhost_dev_start(hdev, true);
>          if (r) {
>              goto fail_start;
> @@ -2207,6 +2287,8 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev, bool vrings)
>      }
>      vhost_start_config_intr(hdev);
>
> +    hdev->backend_transfer = false;
> +
>      trace_vhost_dev_start_out(hdev, vdev->name);
>      return 0;
>  fail_iotlb:
> @@ -2262,9 +2344,18 @@ static int do_vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev,
>      if (hdev->vhost_ops->vhost_dev_start) {
>          hdev->vhost_ops->vhost_dev_start(hdev, false);
>      }
> -    if (vrings) {
> +    if (vrings && !hdev->backend_transfer) {
>          vhost_dev_set_vring_enable(hdev, false);
>      }
> +
> +    if (hdev->backend_transfer) {
> +        for (i = 0; i < hdev->nvqs; ++i) {
> +            struct vhost_virtqueue *vq = hdev->vqs + i;
> +
> +            event_notifier_set_handler(&vq->error_notifier, NULL);
> +        }
> +    }
> +
>      for (i = 0; i < hdev->nvqs; ++i) {
>          rc |= do_vhost_virtqueue_stop(hdev,
>                                        vdev,
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 94a0c75fc8..55ad822848 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -105,6 +105,9 @@ struct vhost_dev {
>      VIRTIO_DECLARE_FEATURES(_features);
>      VIRTIO_DECLARE_FEATURES(acked_features);
>
> +    bool _features_wait_incoming;
> +    bool backend_transfer;
> +
>      uint32_t busyloop_timeout;
>      uint64_t max_queues;
>      uint64_t backend_cap;
> @@ -592,4 +595,8 @@ extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
>      VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
>                             struct vhost_inflight)
>
> +extern const VMStateDescription vmstate_vhost_dev;
> +#define VMSTATE_BACKEND_TRANSFER_VHOST(_field, _state) \
> +    VMSTATE_STRUCT(_field, _state, 0, vmstate_vhost_dev, struct vhost_dev)
> +
>  #endif
> --
> 2.48.1
>
>

Re: [PATCH v2 20/25] vhost-user: add vmstate

[Raphael Norwitz]

For now a naming nit and a question.

On Thu, Oct 16, 2025 at 7:44 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/vhost-user.c         | 95 ++++++++++++++++++++++++++++++++++
>  include/hw/virtio/vhost-user.h |  4 ++
>  2 files changed, 99 insertions(+)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index c5cb5ed528..a820214188 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -28,6 +28,8 @@
>  #include "system/runstate.h"
>  #include "system/cryptodev.h"
>  #include "migration/postcopy-ram.h"
> +#include "migration/qemu-file-types.h"
> +#include "migration/qemu-file.h"
>  #include "trace.h"
>  #include "system/ramblock.h"
>
> @@ -3137,6 +3139,99 @@ void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status)
>          qmp_decode_protocols(u->protocol_features);
>  }
>
> +typedef struct VhostUserMigTmp {
> +    struct vhost_dev *parent;
> +    bool has_backend_channel;
> +    int backend_fd;
> +    uint32_t memory_slots;
> +    uint64_t protocol_features;
> +} VhostUserMigTmp;
> +
> +static int vhost_user_tmp_pre_save(void *opaque)
> +{
> +    VhostUserMigTmp *tmp = opaque;
> +    struct vhost_dev *dev = tmp->parent;
> +    struct vhost_user *u = dev->opaque;
> +    QIOChannelSocket *sioc = u->backend_sioc;
> +
> +    if (sioc && sioc->fd < 0) {
> +        return -EINVAL;
> +    }
> +
> +    tmp->backend_fd = sioc ? sioc->fd : -1;
> +    tmp->has_backend_channel = !!sioc;
> +    tmp->memory_slots = u->user->memory_slots;
> +    tmp->protocol_features = u->protocol_features;
> +
> +    return 0;
> +}
> +
> +static int vhost_user_tmp_post_load(void *opaque, int version_id)
> +{
> +    struct VhostUserMigTmp *tmp = opaque;
> +    struct vhost_dev *dev = tmp->parent;
> +    struct vhost_user *u = dev->opaque;
> +    Error *local_err = NULL;
> +
> +    if (tmp->has_backend_channel) {
> +        u->backend_sioc = qio_channel_socket_new_fd(tmp->backend_fd,
> +                                                    &local_err);
> +        if (!u->backend_sioc) {
> +            error_report_err(local_err);
> +            return -EINVAL;
> +        }
> +        u->backend_src = qio_channel_add_watch_source(
> +            QIO_CHANNEL(u->backend_sioc), G_IO_IN | G_IO_HUP,
> +            backend_read, u->dev, NULL, NULL);
> +    }
> +
> +    u->user->memory_slots = tmp->memory_slots;
> +    u->protocol_features = tmp->protocol_features;
> +
> +    return 0;
> +}
> +
> +static bool vhost_user_tmp_test_fd(void *opaque, int version_id)
> +{
> +    struct VhostUserMigTmp *tmp = opaque;
> +
> +    return tmp->has_backend_channel;
> +}
> +

Will this be vhost-user-blk specific? It should probably be something
like vhost_user_backend_transfer_tmp?

> +static const VMStateDescription vmstate_vhost_user_blk_tmp = {
> +    .name = "vhost-user-blk-tmp",
> +    .pre_save = vhost_user_tmp_pre_save,
> +    .post_load = vhost_user_tmp_post_load,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_UINT64(protocol_features, VhostUserMigTmp),
> +        VMSTATE_UINT32(memory_slots, VhostUserMigTmp),
> +        VMSTATE_BOOL(has_backend_channel, VhostUserMigTmp),
> +        VMSTATE_FD_TEST(backend_fd, VhostUserMigTmp, vhost_user_tmp_test_fd),
> +        VMSTATE_END_OF_LIST()
> +   },
> +};
> +
> +static int vhost_user_post_load(void *opaque, int version_id)
> +{
> +    struct vhost_dev *dev = opaque;
> +    struct vhost_user *u = dev->opaque;
> +
> +    u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
> +    postcopy_add_notifier(&u->postcopy_notifier);
> +
> +    return 0;
> +}
> +

Why do we need a second post_load() callback here? Why can't
u->postcopy_notifier.notify be set in vhost_user_tmp_post_load()?


> +const VMStateDescription vmstate_vhost_user = {
> +    .name = "vhost-user",
> +    .post_load = vhost_user_post_load,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_WITH_TMP(struct vhost_dev, VhostUserMigTmp,
> +                         vmstate_vhost_user_blk_tmp),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  const VhostOps user_ops = {
>          .backend_type = VHOST_BACKEND_TYPE_USER,
>          .vhost_backend_init = vhost_user_backend_init,
> diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
> index 36d96296a3..fb89268de2 100644
> --- a/include/hw/virtio/vhost-user.h
> +++ b/include/hw/virtio/vhost-user.h
> @@ -114,4 +114,8 @@ void vhost_user_async_close(DeviceState *d,
>
>  void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status);
>
> +extern const VMStateDescription vmstate_vhost_user;
> +#define VMSTATE_BACKEND_TRANSFER_VHOST_USER(_field, _state) \
> +    VMSTATE_STRUCT(_field, _state, 0, vmstate_vhost_user, struct vhost_dev)
> +
>  #endif
> --
> 2.48.1
>
>

Re: [PATCH v2 21/25] virtio: support vhost backend migration

[Raphael Norwitz]

Acked-by: Raphael Norwitz <raphael.s.norwitz@gmail.com>


On Thu, Oct 16, 2025 at 7:47 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Add logic to transfer virtio notifiers through migration channel
> for vhost backend migration case.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/virtio/virtio-bus.c     |  2 +-
>  hw/virtio/virtio.c         | 73 ++++++++++++++++++++++++++++++++++++--
>  include/hw/virtio/virtio.h |  2 ++
>  3 files changed, 74 insertions(+), 3 deletions(-)
>
> diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
> index 9b545acda3..577693b6c9 100644
> --- a/hw/virtio/virtio-bus.c
> +++ b/hw/virtio/virtio-bus.c
> @@ -291,7 +291,7 @@ int virtio_bus_set_host_notifier(VirtioBusState *bus, int n, bool assign)
>          return -ENOSYS;
>      }
>
> -    if (assign) {
> +    if (assign && !virtio_is_vhost_backend_transfer(vdev)) {
>          r = event_notifier_init(notifier, 1);
>          if (r < 0) {
>              error_report("%s: unable to init event notifier: %s (%d)",
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 4184aff75c..bf361811d0 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -26,6 +26,7 @@
>  #include "hw/virtio/virtio.h"
>  #include "hw/virtio/vhost.h"
>  #include "migration/qemu-file-types.h"
> +#include "migration/qemu-file.h"
>  #include "qemu/atomic.h"
>  #include "hw/virtio/virtio-bus.h"
>  #include "hw/qdev-properties.h"
> @@ -3032,6 +3033,7 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f)
>      uint32_t guest_features_lo = (vdev->guest_features & 0xffffffff);
>      int i;
>      Error *local_err = NULL;
> +    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
>
>      if (k->save_config) {
>          k->save_config(qbus->parent, f);
> @@ -3065,11 +3067,23 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f)
>           */
>          qemu_put_be64(f, vdev->vq[i].vring.desc);
>          qemu_put_be16s(f, &vdev->vq[i].last_avail_idx);
> +
> +        if (migrating_backend) {
> +            qemu_file_put_fd(f,
> +                             event_notifier_get_fd(&vdev->vq[i].host_notifier));
> +            qemu_file_put_fd(
> +                f, event_notifier_get_fd(&vdev->vq[i].guest_notifier));
> +        }
> +
>          if (k->save_queue) {
>              k->save_queue(qbus->parent, i, f);
>          }
>      }
>
> +    if (migrating_backend) {
> +        qemu_file_put_fd(f, event_notifier_get_fd(&vdev->config_notifier));
> +    }
> +
>      if (vdc->save != NULL) {
>          vdc->save(vdev, f);
>      }
> @@ -3295,6 +3309,7 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>      VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
>      VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
>      Error *local_err = NULL;
> +    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
>
>      /*
>       * We poison the endianness to ensure it does not get used before
> @@ -3364,6 +3379,13 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>          vdev->vq[i].signalled_used_valid = false;
>          vdev->vq[i].notification = true;
>
> +        if (migrating_backend) {
> +            event_notifier_init_fd(&vdev->vq[i].host_notifier,
> +                                   qemu_file_get_fd(f));
> +            event_notifier_init_fd(&vdev->vq[i].guest_notifier,
> +                                   qemu_file_get_fd(f));
> +        }
> +
>          if (!vdev->vq[i].vring.desc && vdev->vq[i].last_avail_idx) {
>              error_report("VQ %d address 0x0 "
>                           "inconsistent with Host index 0x%x",
> @@ -3377,6 +3399,10 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>          }
>      }
>
> +    if (migrating_backend) {
> +        event_notifier_init_fd(&vdev->config_notifier, qemu_file_get_fd(f));
> +    }
> +
>      virtio_notify_vector(vdev, VIRTIO_NO_VECTOR);
>
>      if (vdc->load != NULL) {
> @@ -3394,6 +3420,19 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>          }
>      }
>
> +    if (migrating_backend) {
> +        /*
> +         * On vhost backend migration, device do load host_features from
> +         * migration stream. So update host_features.
> +         */
> +        vdev->host_features = vdc->get_features(vdev, vdev->host_features,
> +                                                &local_err);
> +        if (local_err) {
> +            error_report_err(local_err);
> +            return -EINVAL;
> +        }
> +    }
> +
>      /* Subsections */
>      ret = vmstate_load_state(f, &vmstate_virtio, vdev, 1, &local_err);
>      if (ret) {
> @@ -3447,6 +3486,18 @@ virtio_load(VirtIODevice *vdev, QEMUFile *f, int version_id)
>                  continue;
>              }
>
> +            if (migrating_backend) {
> +                /*
> +                 * Indices are not synced prior to backend migration (as we
> +                 * don't stop vrings by GET_VRING_BASE). No reason to sync them
> +                 * now, and do any checks.
> +                 */
> +                vdev->vq[i].used_idx = 0;
> +                vdev->vq[i].shadow_avail_idx = 0;
> +                vdev->vq[i].inuse = 0;
> +                continue;
> +            }
> +
>              nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
>              /* Check it isn't doing strange things with descriptor numbers. */
>              if (nheads > vdev->vq[i].vring.num) {
> @@ -3815,8 +3866,9 @@ int virtio_queue_set_guest_notifier(VirtIODevice *vdev, int n, bool assign,
>      EventNotifierHandler *read_fn = is_config ?
>          virtio_config_guest_notifier_read :
>          virtio_queue_guest_notifier_read;
> +    bool migrating_backend = virtio_is_vhost_backend_transfer(vdev);
>
> -    if (assign) {
> +    if (assign && !migrating_backend) {
>          int r = event_notifier_init(notifier, 0);
>          if (r < 0) {
>              return r;
> @@ -3826,7 +3878,7 @@ int virtio_queue_set_guest_notifier(VirtIODevice *vdev, int n, bool assign,
>      event_notifier_set_handler(notifier,
>                                 (assign && !with_irqfd) ? read_fn : NULL);
>
> -    if (!assign) {
> +    if (!assign && !migrating_backend) {
>          /* Test and clear notifier before closing it,*/
>          /* in case poll callback didn't have time to run. */
>          read_fn(notifier);
> @@ -4445,6 +4497,23 @@ done:
>      return element;
>  }
>
> +bool virtio_is_vhost_backend_transfer(VirtIODevice *vdev)
> +{
> +    VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev);
> +    struct vhost_dev *hdev;
> +
> +    if (!vdc->get_vhost) {
> +        return false;
> +    }
> +
> +    hdev = vdc->get_vhost(vdev);
> +    if (!hdev) {
> +        return false;
> +    }
> +
> +    return hdev->backend_transfer;
> +}
> +
>  static const TypeInfo virtio_device_info = {
>      .name = TYPE_VIRTIO_DEVICE,
>      .parent = TYPE_DEVICE,
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index 620ee4e389..8e2d3019cd 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -242,6 +242,8 @@ struct VirtioDeviceClass {
>      bool (*skip_vhost_migration_log)(VirtIODevice *vdev);
>  };
>
> +bool virtio_is_vhost_backend_transfer(VirtIODevice *vdev);
> +
>  void virtio_instance_init_common(Object *proxy_obj, void *data,
>                                   size_t vdev_size, const char *vdev_name);
>
> --
> 2.48.1
>
>

Re: [PATCH v2 24/25] vhost-user-blk: support vhost backend migration

[Raphael Norwitz]

Overall looks ok. A couple comments

On Thu, Oct 16, 2025 at 7:49 AM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> Opt-out backend initialization code, and instead get the state
> from migration channel (including inflight region).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> ---
>  hw/block/vhost-user-blk.c          | 129 ++++++++++++++++++++++++-----
>  include/hw/virtio/vhost-user-blk.h |   2 +
>  include/hw/virtio/vhost.h          |   3 +-
>  3 files changed, 111 insertions(+), 23 deletions(-)
>
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index ffdd600526..a8fd90480a 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -17,6 +17,7 @@
>   */
>
>  #include "qemu/osdep.h"
> +#include "qapi-types-run-state.h"
>  #include "qapi/error.h"
>  #include "qemu/error-report.h"
>  #include "qemu/cutils.h"
> @@ -31,7 +32,13 @@
>  #include "hw/virtio/virtio-access.h"
>  #include "system/system.h"
>  #include "system/runstate.h"
> +#include "chardev/char-backend-transfer.h"
>  #include "trace.h"
> +#include "migration/qemu-file.h"
> +#include "migration/migration.h"
> +#include "migration/options.h"
> +#include "qemu/event_notifier.h"
> +#include <sys/mman.h>
>
>  static const int user_feature_bits[] = {
>      VIRTIO_BLK_F_SIZE_MAX,
> @@ -160,32 +167,35 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Error **errp)
>
>      s->dev.acked_features = vdev->guest_features;
>
> -    ret = vhost_dev_prepare_inflight(&s->dev, vdev);
> -    if (ret < 0) {
> -        error_setg_errno(errp, -ret, "Error setting inflight format");
> -        goto err_guest_notifiers;
> -    }
> -
> -    if (!s->inflight->addr) {
> -        ret = vhost_dev_get_inflight(&s->dev, s->queue_size, s->inflight);
> +    if (!s->dev.backend_transfer) {
> +        ret = vhost_dev_prepare_inflight(&s->dev, vdev);
>          if (ret < 0) {
> -            error_setg_errno(errp, -ret, "Error getting inflight");
> +            error_setg_errno(errp, -ret, "Error setting inflight format");
>              goto err_guest_notifiers;
>          }
> -    }
>
> -    ret = vhost_dev_set_inflight(&s->dev, s->inflight);
> -    if (ret < 0) {
> -        error_setg_errno(errp, -ret, "Error setting inflight");
> -        goto err_guest_notifiers;
> -    }
> +        if (!s->inflight->addr) {
> +            ret = vhost_dev_get_inflight(&s->dev, s->queue_size, s->inflight);
> +            if (ret < 0) {
> +                error_setg_errno(errp, -ret, "Error getting inflight");
> +                goto err_guest_notifiers;
> +            }
> +        }
>
> -    /* guest_notifier_mask/pending not used yet, so just unmask
> -     * everything here. virtio-pci will do the right thing by
> -     * enabling/disabling irqfd.
> -     */
> -    for (i = 0; i < s->dev.nvqs; i++) {
> -        vhost_virtqueue_mask(&s->dev, vdev, i, false);
> +        ret = vhost_dev_set_inflight(&s->dev, s->inflight);
> +        if (ret < 0) {
> +            error_setg_errno(errp, -ret, "Error setting inflight");
> +            goto err_guest_notifiers;
> +        }
> +
> +        /*
> +         * guest_notifier_mask/pending not used yet, so just unmask
> +         * everything here. virtio-pci will do the right thing by
> +         * enabling/disabling irqfd.
> +         */
> +        for (i = 0; i < s->dev.nvqs; i++) {
> +            vhost_virtqueue_mask(&s->dev, vdev, i, false);
> +        }
>      }
>
>      s->dev.vq_index_end = s->dev.nvqs;
> @@ -232,6 +242,10 @@ static int vhost_user_blk_stop(VirtIODevice *vdev)
>      force_stop = s->skip_get_vring_base_on_force_shutdown &&
>                   qemu_force_shutdown_requested();
>
> +    s->dev.backend_transfer = s->dev.backend_transfer ||
> +        (runstate_check(RUN_STATE_FINISH_MIGRATE) &&
> +         migrate_local_vhost_user_blk());
> +
>      ret = force_stop ? vhost_dev_force_stop(&s->dev, vdev, true) :
>                         vhost_dev_stop(&s->dev, vdev, true);
>
> @@ -391,6 +405,7 @@ static int vhost_user_blk_connect(DeviceState *dev,
>      trace_vhost_user_blk_connect_in(vdev);
>
>      assert(!s->connected);
> +    assert(!s->dev.backend_transfer);
>
>      ret = vhost_dev_connect(&s->dev, errp);
>      if (ret < 0) {
> @@ -464,6 +479,9 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>      DeviceState *dev = DEVICE(s);
>      int ret;
>
> +    assert(!s->connected);
> +    assert(!s->dev.backend_transfer);
> +
>      ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
>      if (ret < 0) {
>          return ret;
> @@ -642,7 +660,13 @@ static bool vhost_user_blk_pre_incoming(void *opaque, Error **errp)
>  {
>      VHostUserBlk *s = VHOST_USER_BLK(opaque);
>
> -    return vhost_user_blk_realize_connect(s, errp) == 0;
> +    s->dev.backend_transfer = migrate_local_vhost_user_blk();
> +
> +    if (!s->dev.backend_transfer) {
> +        return vhost_user_blk_realize_connect_loop(s, errp) >= 0;
> +    }
> +
> +    return true;
>  }
>
>  static const VMStateDescription vmstate_vhost_user_blk = {
> @@ -656,6 +680,64 @@ static const VMStateDescription vmstate_vhost_user_blk = {
>      },
>  };
>

Rename vhost_user_blk_needed()?

> +static bool vhost_user_needed(void *opaque)
> +{
> +    return migrate_local_vhost_user_blk();
> +}
> +
> +static const VMStateDescription vmstate_vhost_user_blk_device = {
> +    .name = "vhost-user-blk-device",
> +    .version_id = 1,
> +    .needed = vhost_user_needed,
> +    .fields = (const VMStateField[]) {
> +        VMSTATE_BACKEND_TRANSFER_CHARDEV(chardev, VHostUserBlk),
> +        VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(inflight, VHostUserBlk),
> +        VMSTATE_BACKEND_TRANSFER_VHOST_USER(dev, VHostUserBlk),
> +        VMSTATE_BACKEND_TRANSFER_VHOST(dev, VHostUserBlk),
> +        VMSTATE_END_OF_LIST()
> +    },
> +};
> +
> +static int vhost_user_blk_post_load(VirtIODevice *vdev)
> +{
> +    VHostUserBlk *s = VHOST_USER_BLK(vdev);
> +    struct vhost_dev *hdev = vhost_user_blk_get_vhost(vdev);
> +    DeviceState *dev = &s->parent_obj.parent_obj;
> +
> +    if (!hdev->backend_transfer) {
> +        return 0;
> +    }
> +
> +    s->connected = true;
> +
> +    memcpy(&s->blkcfg, vdev->config, vdev->config_len);
> +
> +    if (virtio_device_started(vdev, vdev->status)) {
> +        int ret;
> +        ret = vhost_user_blk_start(vdev, NULL);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +    }
> +
> +    /* we're fully initialized, now we can operate, so add the handler */
> +    qemu_chr_fe_set_handlers(&s->chardev,  NULL, NULL,
> +                             vhost_user_blk_event, NULL, (void *)dev,
> +                             NULL, true);
> +
> +    return 0;
> +}
> +
> +static bool vhost_user_blk_skip_migration_log(VirtIODevice *vdev)
> +{
> +    /*
> +     * Note that hdev->migrating_backend is false at this moment,
> +     * as logging is being setup during outging migration setup stage,
> +     * which is far before vm stop.
> +     */
> +    return migrate_local_vhost_user_blk();
> +}
> +
>  static const Property vhost_user_blk_properties[] = {
>      DEFINE_PROP_CHR_NO_CONNECT("chardev", VHostUserBlk, chardev),
>      DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
> @@ -688,6 +770,9 @@ static void vhost_user_blk_class_init(ObjectClass *klass, const void *data)
>      vdc->set_status = vhost_user_blk_set_status;
>      vdc->reset = vhost_user_blk_reset;
>      vdc->get_vhost = vhost_user_blk_get_vhost;
> +    vdc->vmsd = &vmstate_vhost_user_blk_device;
> +    vdc->post_load = vhost_user_blk_post_load,
> +    vdc->skip_vhost_migration_log = vhost_user_blk_skip_migration_log;
>  }
>
>  static const TypeInfo vhost_user_blk_info = {
> diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h
> index a10f785672..b06f55fd6f 100644
> --- a/include/hw/virtio/vhost-user-blk.h
> +++ b/include/hw/virtio/vhost-user-blk.h
> @@ -52,6 +52,8 @@ struct VHostUserBlk {
>      bool started_vu;
>
>      bool skip_get_vring_base_on_force_shutdown;

Why do we need incoming_backend? Looks like it is unused.


> +
> +    bool incoming_backend;
>  };
>
>  #endif
> diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
> index 55ad822848..13ca2c319f 100644
> --- a/include/hw/virtio/vhost.h
> +++ b/include/hw/virtio/vhost.h
> @@ -592,7 +592,8 @@ static inline int vhost_load_backend_state(struct vhost_dev *dev, QEMUFile *f,
>
>  extern const VMStateDescription vmstate_backend_transfer_vhost_inflight;
>  #define VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(_field, _state) \
> -    VMSTATE_STRUCT_POINTER(_field, _state, vmstate_inflight, \
> +    VMSTATE_STRUCT_POINTER(_field, _state, \
> +                           vmstate_backend_transfer_vhost_inflight, \
>                             struct vhost_inflight)
>
>  extern const VMStateDescription vmstate_vhost_dev;
> --
> 2.48.1
>
>

Re: [PATCH v2 12/25] vhost-user-blk: move first vhost_user_blk_init() to _realize()

[Vladimir Sementsov-Ogievskiy]

On 21.10.25 02:35, Raphael Norwitz wrote:
> On Thu, Oct 16, 2025 at 7:48 AM Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru> wrote:
>>
>> We'll need to postpone further connecting/reconnecting logic to the
>> later point to support backend-transfer migration for vhost-user-blk.
>> For now, move first call to vhost_user_blk_init() to _realize() (this
>> call will not be postponed). To support this, we also have to move
>> re-initialization to vhost_user_blk_realize_connect_loop().
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>> ---
>>   hw/block/vhost-user-blk.c | 17 ++++++++++++++---
>>   1 file changed, 14 insertions(+), 3 deletions(-)
>>
>> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
>> index 36e32229ad..af4a97b8e4 100644
>> --- a/hw/block/vhost-user-blk.c
>> +++ b/hw/block/vhost-user-blk.c
>> @@ -464,14 +464,12 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>>       DeviceState *dev = DEVICE(s);
>>       int ret;
>>
>> -    s->connected = false;
>> -
>>       ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
>>       if (ret < 0) {
>>           return ret;
>>       }
>>
>> -    ret = vhost_user_blk_init(dev, true, errp);
>> +    ret = vhost_user_blk_connect(dev, errp);
>>       if (ret < 0) {
>>           qemu_chr_fe_disconnect(&s->chardev);
>>           return ret;
>> @@ -501,7 +499,16 @@ static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
>>               error_prepend(errp, "Reconnecting after error: ");
>>               error_report_err(*errp);
>>               *errp = NULL;
>> +
>> +            s->connected = false;
>> +
>> +            ret = vhost_user_blk_init(dev, false, errp);
>> +            if (ret < 0) {
>> +                /* No reason to retry initialization */
>> +                return ret;
>> +            }
>>           }
>> +
>>           ret = vhost_user_blk_realize_connect(s, errp);
>>       } while (ret < 0 && retries--);
>>
>> @@ -566,6 +573,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>>       s->inflight = g_new0(struct vhost_inflight, 1);
>>       s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
>>
> 
> Why call vhost_user_blk_init() here if we call it in
> host_user_blk_realize_connect_loop()?

To be able to postpone the whole realize-connect-loop to the later
point (not in realize) in further commits.

So this first init will stay in realize, for early initialization of the device.

> 
>> +    if (vhost_user_blk_init(dev, false, errp) < 0) {
>> +        goto fail;
>> +    }
>> +
>>       if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
>>           goto fail;
>>       }
>> --
>> 2.48.1
>>
>>


-- 
Best regards,
Vladimir

Re: [PATCH v2 20/25] vhost-user: add vmstate

[Vladimir Sementsov-Ogievskiy]

On 21.10.25 02:51, Raphael Norwitz wrote:
> For now a naming nit and a question.
> 
> On Thu, Oct 16, 2025 at 7:44 AM Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru> wrote:
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>> ---
>>   hw/virtio/vhost-user.c         | 95 ++++++++++++++++++++++++++++++++++
>>   include/hw/virtio/vhost-user.h |  4 ++
>>   2 files changed, 99 insertions(+)
>>
>> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>> index c5cb5ed528..a820214188 100644
>> --- a/hw/virtio/vhost-user.c
>> +++ b/hw/virtio/vhost-user.c
>> @@ -28,6 +28,8 @@
>>   #include "system/runstate.h"
>>   #include "system/cryptodev.h"
>>   #include "migration/postcopy-ram.h"
>> +#include "migration/qemu-file-types.h"
>> +#include "migration/qemu-file.h"
>>   #include "trace.h"
>>   #include "system/ramblock.h"
>>
>> @@ -3137,6 +3139,99 @@ void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status)
>>           qmp_decode_protocols(u->protocol_features);
>>   }
>>
>> +typedef struct VhostUserMigTmp {
>> +    struct vhost_dev *parent;
>> +    bool has_backend_channel;
>> +    int backend_fd;
>> +    uint32_t memory_slots;
>> +    uint64_t protocol_features;
>> +} VhostUserMigTmp;
>> +
>> +static int vhost_user_tmp_pre_save(void *opaque)
>> +{
>> +    VhostUserMigTmp *tmp = opaque;
>> +    struct vhost_dev *dev = tmp->parent;
>> +    struct vhost_user *u = dev->opaque;
>> +    QIOChannelSocket *sioc = u->backend_sioc;
>> +
>> +    if (sioc && sioc->fd < 0) {
>> +        return -EINVAL;
>> +    }
>> +
>> +    tmp->backend_fd = sioc ? sioc->fd : -1;
>> +    tmp->has_backend_channel = !!sioc;
>> +    tmp->memory_slots = u->user->memory_slots;
>> +    tmp->protocol_features = u->protocol_features;
>> +
>> +    return 0;
>> +}
>> +
>> +static int vhost_user_tmp_post_load(void *opaque, int version_id)
>> +{
>> +    struct VhostUserMigTmp *tmp = opaque;
>> +    struct vhost_dev *dev = tmp->parent;
>> +    struct vhost_user *u = dev->opaque;
>> +    Error *local_err = NULL;
>> +
>> +    if (tmp->has_backend_channel) {
>> +        u->backend_sioc = qio_channel_socket_new_fd(tmp->backend_fd,
>> +                                                    &local_err);
>> +        if (!u->backend_sioc) {
>> +            error_report_err(local_err);
>> +            return -EINVAL;
>> +        }
>> +        u->backend_src = qio_channel_add_watch_source(
>> +            QIO_CHANNEL(u->backend_sioc), G_IO_IN | G_IO_HUP,
>> +            backend_read, u->dev, NULL, NULL);
>> +    }
>> +
>> +    u->user->memory_slots = tmp->memory_slots;
>> +    u->protocol_features = tmp->protocol_features;
>> +
>> +    return 0;
>> +}
>> +
>> +static bool vhost_user_tmp_test_fd(void *opaque, int version_id)
>> +{
>> +    struct VhostUserMigTmp *tmp = opaque;
>> +
>> +    return tmp->has_backend_channel;
>> +}
>> +
> 
> Will this be vhost-user-blk specific? It should probably be something
> like vhost_user_backend_transfer_tmp?
> 

Oh right. a mistake.

>> +static const VMStateDescription vmstate_vhost_user_blk_tmp = {
>> +    .name = "vhost-user-blk-tmp",
>> +    .pre_save = vhost_user_tmp_pre_save,
>> +    .post_load = vhost_user_tmp_post_load,
>> +    .fields = (const VMStateField[]) {
>> +        VMSTATE_UINT64(protocol_features, VhostUserMigTmp),
>> +        VMSTATE_UINT32(memory_slots, VhostUserMigTmp),
>> +        VMSTATE_BOOL(has_backend_channel, VhostUserMigTmp),
>> +        VMSTATE_FD_TEST(backend_fd, VhostUserMigTmp, vhost_user_tmp_test_fd),
>> +        VMSTATE_END_OF_LIST()
>> +   },
>> +};
>> +
>> +static int vhost_user_post_load(void *opaque, int version_id)
>> +{
>> +    struct vhost_dev *dev = opaque;
>> +    struct vhost_user *u = dev->opaque;
>> +
>> +    u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
>> +    postcopy_add_notifier(&u->postcopy_notifier);
>> +
>> +    return 0;
>> +}
>> +
> 
> Why do we need a second post_load() callback here? Why can't
> u->postcopy_notifier.notify be set in vhost_user_tmp_post_load()?

Hmm.. Yes, looks strange, I don't remember. Will try to merge.

> 
> 
>> +const VMStateDescription vmstate_vhost_user = {
>> +    .name = "vhost-user",
>> +    .post_load = vhost_user_post_load,
>> +    .fields = (const VMStateField[]) {
>> +        VMSTATE_WITH_TMP(struct vhost_dev, VhostUserMigTmp,
>> +                         vmstate_vhost_user_blk_tmp),
>> +        VMSTATE_END_OF_LIST()
>> +    }
>> +};
>> +
>>   const VhostOps user_ops = {
>>           .backend_type = VHOST_BACKEND_TYPE_USER,
>>           .vhost_backend_init = vhost_user_backend_init,
>> diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
>> index 36d96296a3..fb89268de2 100644
>> --- a/include/hw/virtio/vhost-user.h
>> +++ b/include/hw/virtio/vhost-user.h
>> @@ -114,4 +114,8 @@ void vhost_user_async_close(DeviceState *d,
>>
>>   void vhost_user_qmp_status(struct vhost_dev *dev, VirtioStatus *status);
>>
>> +extern const VMStateDescription vmstate_vhost_user;
>> +#define VMSTATE_BACKEND_TRANSFER_VHOST_USER(_field, _state) \
>> +    VMSTATE_STRUCT(_field, _state, 0, vmstate_vhost_user, struct vhost_dev)
>> +
>>   #endif
>> --
>> 2.48.1
>>
>>


-- 
Best regards,
Vladimir

Re: [PATCH v2 24/25] vhost-user-blk: support vhost backend migration

[Vladimir Sementsov-Ogievskiy]

On 21.10.25 02:53, Raphael Norwitz wrote:
> Overall looks ok. A couple comments
> 
> On Thu, Oct 16, 2025 at 7:49 AM Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru> wrote:
>>
>> Opt-out backend initialization code, and instead get the state
>> from migration channel (including inflight region).
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>> ---
>>   hw/block/vhost-user-blk.c          | 129 ++++++++++++++++++++++++-----
>>   include/hw/virtio/vhost-user-blk.h |   2 +
>>   include/hw/virtio/vhost.h          |   3 +-
>>   3 files changed, 111 insertions(+), 23 deletions(-)
>>
>> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
>> index ffdd600526..a8fd90480a 100644
>> --- a/hw/block/vhost-user-blk.c
>> +++ b/hw/block/vhost-user-blk.c
>> @@ -17,6 +17,7 @@
>>    */

[..]

>> @@ -656,6 +680,64 @@ static const VMStateDescription vmstate_vhost_user_blk = {
>>       },
>>   };
>>
> 
> Rename vhost_user_blk_needed()?

Yes, will fix

> 
>> +static bool vhost_user_needed(void *opaque)
>> +{
>> +    return migrate_local_vhost_user_blk();
>> +}
>> +
>> +static const VMStateDescription vmstate_vhost_user_blk_device = {
>> +    .name = "vhost-user-blk-device",
>> +    .version_id = 1,
>> +    .needed = vhost_user_needed,
>> +    .fields = (const VMStateField[]) {
>> +        VMSTATE_BACKEND_TRANSFER_CHARDEV(chardev, VHostUserBlk),
>> +        VMSTATE_BACKEND_TRANSFER_VHOST_INFLIGHT(inflight, VHostUserBlk),
>> +        VMSTATE_BACKEND_TRANSFER_VHOST_USER(dev, VHostUserBlk),
>> +        VMSTATE_BACKEND_TRANSFER_VHOST(dev, VHostUserBlk),
>> +        VMSTATE_END_OF_LIST()
>> +    },
>> +};

[..]

>> diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h
>> index a10f785672..b06f55fd6f 100644
>> --- a/include/hw/virtio/vhost-user-blk.h
>> +++ b/include/hw/virtio/vhost-user-blk.h
>> @@ -52,6 +52,8 @@ struct VHostUserBlk {
>>       bool started_vu;
>>
>>       bool skip_get_vring_base_on_force_shutdown;
> 
> Why do we need incoming_backend? Looks like it is unused.
> 

Oops right. Forget to delete, it was used in v1.

> 
>> +
>> +    bool incoming_backend;
>>   };
>>
>>   #endif


Thanks a lot for reviewing!

-- 
Best regards,
Vladimir

Re: [PATCH v2 12/25] vhost-user-blk: move first vhost_user_blk_init() to _realize()

[Raphael Norwitz]

ACK on your point. One more question about setting s->connected = false.

On Tue, Oct 21, 2025 at 12:29 PM Vladimir Sementsov-Ogievskiy
<vsementsov@yandex-team.ru> wrote:
>
> On 21.10.25 02:35, Raphael Norwitz wrote:
> > On Thu, Oct 16, 2025 at 7:48 AM Vladimir Sementsov-Ogievskiy
> > <vsementsov@yandex-team.ru> wrote:
> >>
> >> We'll need to postpone further connecting/reconnecting logic to the
> >> later point to support backend-transfer migration for vhost-user-blk.
> >> For now, move first call to vhost_user_blk_init() to _realize() (this
> >> call will not be postponed). To support this, we also have to move
> >> re-initialization to vhost_user_blk_realize_connect_loop().
> >>
> >> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> >> ---
> >>   hw/block/vhost-user-blk.c | 17 ++++++++++++++---
> >>   1 file changed, 14 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> >> index 36e32229ad..af4a97b8e4 100644
> >> --- a/hw/block/vhost-user-blk.c
> >> +++ b/hw/block/vhost-user-blk.c
> >> @@ -464,14 +464,12 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
> >>       DeviceState *dev = DEVICE(s);
> >>       int ret;
> >>
> >> -    s->connected = false;
> >> -
> >>       ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
> >>       if (ret < 0) {
> >>           return ret;
> >>       }
> >>
> >> -    ret = vhost_user_blk_init(dev, true, errp);
> >> +    ret = vhost_user_blk_connect(dev, errp);
> >>       if (ret < 0) {
> >>           qemu_chr_fe_disconnect(&s->chardev);
> >>           return ret;
> >> @@ -501,7 +499,16 @@ static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
> >>               error_prepend(errp, "Reconnecting after error: ");
> >>               error_report_err(*errp);
> >>               *errp = NULL;
> >> +

Having removed setting s->connected = false from
vhost_user_blk_realize_connect() we will now only set s->connected =
false here in the if (*errp) {} error path. Shouldn't we also set
s->connected = false outside the error path here or in
vhost_user_blk_device_realize()?

> >> +            s->connected = false;
> >> +
> >> +            ret = vhost_user_blk_init(dev, false, errp);
> >> +            if (ret < 0) {
> >> +                /* No reason to retry initialization */
> >> +                return ret;
> >> +            }
> >>           }
> >> +
> >>           ret = vhost_user_blk_realize_connect(s, errp);
> >>       } while (ret < 0 && retries--);
> >>
> >> @@ -566,6 +573,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
> >>       s->inflight = g_new0(struct vhost_inflight, 1);
> >>       s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
> >>
> >
> > Why call vhost_user_blk_init() here if we call it in
> > host_user_blk_realize_connect_loop()?
>
> To be able to postpone the whole realize-connect-loop to the later
> point (not in realize) in further commits.
>
> So this first init will stay in realize, for early initialization of the device.
>

Makes sense - I missed that vhost_user_blk_init() is only called in
the error path.

> >
> >> +    if (vhost_user_blk_init(dev, false, errp) < 0) {
> >> +        goto fail;
> >> +    }
> >> +
> >>       if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
> >>           goto fail;
> >>       }
> >> --
> >> 2.48.1
> >>
> >>
>
>
> --
> Best regards,
> Vladimir

Re: [PATCH v2 12/25] vhost-user-blk: move first vhost_user_blk_init() to _realize()

[Vladimir Sementsov-Ogievskiy]

On 21.10.25 21:06, Raphael Norwitz wrote:
> ACK on your point. One more question about setting s->connected = false.
> 
> On Tue, Oct 21, 2025 at 12:29 PM Vladimir Sementsov-Ogievskiy
> <vsementsov@yandex-team.ru> wrote:
>>
>> On 21.10.25 02:35, Raphael Norwitz wrote:
>>> On Thu, Oct 16, 2025 at 7:48 AM Vladimir Sementsov-Ogievskiy
>>> <vsementsov@yandex-team.ru> wrote:
>>>>
>>>> We'll need to postpone further connecting/reconnecting logic to the
>>>> later point to support backend-transfer migration for vhost-user-blk.
>>>> For now, move first call to vhost_user_blk_init() to _realize() (this
>>>> call will not be postponed). To support this, we also have to move
>>>> re-initialization to vhost_user_blk_realize_connect_loop().
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
>>>> ---
>>>>    hw/block/vhost-user-blk.c | 17 ++++++++++++++---
>>>>    1 file changed, 14 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
>>>> index 36e32229ad..af4a97b8e4 100644
>>>> --- a/hw/block/vhost-user-blk.c
>>>> +++ b/hw/block/vhost-user-blk.c
>>>> @@ -464,14 +464,12 @@ static int vhost_user_blk_realize_connect(VHostUserBlk *s, Error **errp)
>>>>        DeviceState *dev = DEVICE(s);
>>>>        int ret;
>>>>
>>>> -    s->connected = false;
>>>> -
>>>>        ret = qemu_chr_fe_wait_connected(&s->chardev, errp);
>>>>        if (ret < 0) {
>>>>            return ret;
>>>>        }
>>>>
>>>> -    ret = vhost_user_blk_init(dev, true, errp);
>>>> +    ret = vhost_user_blk_connect(dev, errp);
>>>>        if (ret < 0) {
>>>>            qemu_chr_fe_disconnect(&s->chardev);
>>>>            return ret;
>>>> @@ -501,7 +499,16 @@ static int vhost_user_blk_realize_connect_loop(VHostUserBlk *s, Error **errp)
>>>>                error_prepend(errp, "Reconnecting after error: ");
>>>>                error_report_err(*errp);
>>>>                *errp = NULL;
>>>> +
> 
> Having removed setting s->connected = false from
> vhost_user_blk_realize_connect() we will now only set s->connected =
> false here in the if (*errp) {} error path. Shouldn't we also set
> s->connected = false outside the error path here or in
> vhost_user_blk_device_realize()?
> 

On success path we are calling vhost_user_blk_realize_connect() for the first
time, so s->connected was never set to true at this time, it must be false
(due to zero-initialization of the structure). We can add an assertion.

>>>> +            s->connected = false;
>>>> +
>>>> +            ret = vhost_user_blk_init(dev, false, errp);
>>>> +            if (ret < 0) {
>>>> +                /* No reason to retry initialization */
>>>> +                return ret;
>>>> +            }
>>>>            }
>>>> +
>>>>            ret = vhost_user_blk_realize_connect(s, errp);
>>>>        } while (ret < 0 && retries--);
>>>>
>>>> @@ -566,6 +573,10 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>>>>        s->inflight = g_new0(struct vhost_inflight, 1);
>>>>        s->vhost_vqs = g_new0(struct vhost_virtqueue, s->num_queues);
>>>>
>>>
>>> Why call vhost_user_blk_init() here if we call it in
>>> host_user_blk_realize_connect_loop()?
>>
>> To be able to postpone the whole realize-connect-loop to the later
>> point (not in realize) in further commits.
>>
>> So this first init will stay in realize, for early initialization of the device.
>>
> 
> Makes sense - I missed that vhost_user_blk_init() is only called in
> the error path.
> 
>>>
>>>> +    if (vhost_user_blk_init(dev, false, errp) < 0) {
>>>> +        goto fail;
>>>> +    }
>>>> +
>>>>        if (vhost_user_blk_realize_connect_loop(s, errp) < 0) {
>>>>            goto fail;
>>>>        }
>>>> --
>>>> 2.48.1
>>>>
>>>>
>>
>>
>> --
>> Best regards,
>> Vladimir


-- 
Best regards,
Vladimir