[PATCH RFC 00/15] vhost-user: isolated memory

[PATCH RFC 00/15] vhost-user: isolated memory

[ConKite]

This patch series implements a memory isolation mode in vhost-user. The
purpose of this mode is to provide the option of additional security by
eliminating direct access of guest memory by vhost-user devices.
At a high level this works by:

1. Adding qdev and qapi properties required to enable isolation mode for
   various devices.
2. Allocating an isolation memory region in an anonymous file and mapping it
   to host memory.  This isolation region will hold the bounce buffers and
   vrings necessary to move data.
3. Using a vhost-iova-tree to allocate and track the mapping between
   guest regions and their corresponding bounce buffers in the isolation
   memory.
4. Creating shadow virtqueues to intercept request notifications.  As
   kick and call events are received by an svq, it copies buffer
   contents and descriptors between isolation and guest memory before
   notifying the backend or guest.   

Note: This project is currently in a partially functional state. For
example, in testing vhost-user-input and a keyboard, keystrokes may
register correctly or be double counted depending on the combination
of backend daemon and guest OS used. However, the desire is to make
this work public at this stage for comment on the overall
approach.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
ConKite (2):
      vhost-user: Consolidate chardev property definitions
      vhost-user: Add memory-isolation qdev property to vhost-user devices

Connor Kite (13):
      backends/cryptodev-vhost-user: add memory isolation bool
      net/vhost-user: add memory isolation
      vhost-user: add memory_isolation to VhostUserState
      util/iova-tree: g_tree_foreach wrapper
      hw/virtio: iova_tree_foreach wrapper
      hw/virtio/vhost-shadow-virtqueue: used handler
      hw/virtio/vhost-shadow-virtqueue: specified vring placement
      hw/virtio/vhost-shadow-virtqueue: range boundary in translation
      hw/virtio/vhost-user: create isolation region
      hw/virtio/vhost-user: send isolation regions to device
      hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts
      hw/virtio/vhost-user: handle data movement with shadow vqs
      hw/virtio/vhost-user: shadow vq cleanup

 backends/cryptodev-vhost-user.c      |  24 +-
 backends/vhost-user.c                |   4 +-
 hw/block/vhost-user-blk.c            |   4 +-
 hw/display/vhost-user-gpu.c          |   4 +-
 hw/scsi/vhost-user-scsi.c            |   4 +-
 hw/virtio/vhost-iova-tree.c          |  15 ++
 hw/virtio/vhost-iova-tree.h          |   3 +
 hw/virtio/vhost-shadow-virtqueue.c   |  30 ++-
 hw/virtio/vhost-shadow-virtqueue.h   |  17 ++
 hw/virtio/vhost-stub.c               |   3 +-
 hw/virtio/vhost-user-base.c          |  13 +-
 hw/virtio/vhost-user-fs.c            |   4 +-
 hw/virtio/vhost-user-gpio.c          |   4 -
 hw/virtio/vhost-user-i2c.c           |   5 -
 hw/virtio/vhost-user-input.c         |   5 -
 hw/virtio/vhost-user-rng.c           |   5 -
 hw/virtio/vhost-user-rtc.c           |   4 -
 hw/virtio/vhost-user-scmi.c          |   4 +-
 hw/virtio/vhost-user-snd.c           |   1 -
 hw/virtio/vhost-user-spi.c           |   5 -
 hw/virtio/vhost-user-test-device.c   |   1 -
 hw/virtio/vhost-user-vsock.c         |   4 +-
 hw/virtio/vhost-user.c               | 441 ++++++++++++++++++++++++++++++++++-
 include/hw/virtio/vhost-user-base.h  |   1 +
 include/hw/virtio/vhost-user-blk.h   |   1 +
 include/hw/virtio/vhost-user-fs.h    |   1 +
 include/hw/virtio/vhost-user-scmi.h  |   1 +
 include/hw/virtio/vhost-user-vsock.h |   1 +
 include/hw/virtio/vhost-user.h       |   6 +-
 include/hw/virtio/virtio-gpu.h       |   1 +
 include/hw/virtio/virtio-scsi.h      |   1 +
 include/qemu/iova-tree.h             |  13 ++
 include/system/vhost-user-backend.h  |   3 +-
 net/passt.c                          |  16 +-
 net/vhost-user.c                     |  11 +-
 qapi/net.json                        |  15 +-
 qapi/qom.json                        |   6 +-
 util/iova-tree.c                     |   5 +
 38 files changed, 622 insertions(+), 64 deletions(-)
---
base-commit: 006a22cb26998998385b104db1ff9466ef2f3153
change-id: 20260604-vhost-user-isolated-memory-070ed4833ee7

Best regards,
-- 
Connor Kite <connorkite@gmail.com>

[PATCH RFC 01/15] vhost-user: Consolidate chardev property definitions

[ConKite]

Move chardev qdev property definitions from children into
VHostUserBase. This reduces code duplication and allows children
to simply inherit the property during initialization.

Signed-off-by: Connor Kite <connorkite@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/virtio/vhost-user-base.c        | 9 +++++++++
 hw/virtio/vhost-user-gpio.c        | 4 ----
 hw/virtio/vhost-user-i2c.c         | 5 -----
 hw/virtio/vhost-user-input.c       | 5 -----
 hw/virtio/vhost-user-rng.c         | 5 -----
 hw/virtio/vhost-user-rtc.c         | 4 ----
 hw/virtio/vhost-user-snd.c         | 1 -
 hw/virtio/vhost-user-spi.c         | 5 -----
 hw/virtio/vhost-user-test-device.c | 1 -
 9 files changed, 9 insertions(+), 30 deletions(-)

diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
index 478ec68f09..90bd2903db 100644
--- a/hw/virtio/vhost-user-base.c
+++ b/hw/virtio/vhost-user-base.c
@@ -407,9 +407,16 @@ static void vub_device_unrealize(DeviceState *dev)
     do_vhost_user_cleanup(vdev, vub);
 }
 
+/*Define common qdev properties.  Inherited by all children*/
+static const Property vub_properties[] = {
+    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
+};
+
+
 static void vub_class_init(ObjectClass *klass, const void *data)
 {
     VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass);
+    DeviceClass *dc = DEVICE_CLASS(klass);
 
     vdc->realize = vub_device_realize;
     vdc->unrealize = vub_device_unrealize;
@@ -417,6 +424,8 @@ static void vub_class_init(ObjectClass *klass, const void *data)
     vdc->get_config = vub_get_config;
     vdc->set_config = vub_set_config;
     vdc->set_status = vub_set_status;
+
+    device_class_set_props(dc, vub_properties);
 }
 
 static const TypeInfo vub_types[] = {
diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c
index d473f87077..33b8752282 100644
--- a/hw/virtio/vhost-user-gpio.c
+++ b/hw/virtio/vhost-user-gpio.c
@@ -14,9 +14,6 @@
 #include "standard-headers/linux/virtio_ids.h"
 #include "standard-headers/linux/virtio_gpio.h"
 
-static const Property vgpio_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
 
 static void vgpio_realize(DeviceState *dev, Error **errp)
 {
@@ -42,7 +39,6 @@ static void vu_gpio_class_init(ObjectClass *klass, const void *data)
     VHostUserBaseClass *vubc = VHOST_USER_BASE_CLASS(klass);
 
     dc->vmsd = &vu_gpio_vmstate;
-    device_class_set_props(dc, vgpio_properties);
     device_class_set_parent_realize(dc, vgpio_realize,
                                     &vubc->parent_realize);
     set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
diff --git a/hw/virtio/vhost-user-i2c.c b/hw/virtio/vhost-user-i2c.c
index 152b1f6740..3f5427b5b2 100644
--- a/hw/virtio/vhost-user-i2c.c
+++ b/hw/virtio/vhost-user-i2c.c
@@ -14,10 +14,6 @@
 #include "qemu/error-report.h"
 #include "standard-headers/linux/virtio_ids.h"
 
-static const Property vi2c_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
-
 static void vi2c_realize(DeviceState *dev, Error **errp)
 {
     VHostUserBase *vub = VHOST_USER_BASE(dev);
@@ -42,7 +38,6 @@ static void vu_i2c_class_init(ObjectClass *klass, const void *data)
     VHostUserBaseClass *vubc = VHOST_USER_BASE_CLASS(klass);
 
     dc->vmsd = &vu_i2c_vmstate;
-    device_class_set_props(dc, vi2c_properties);
     device_class_set_parent_realize(dc, vi2c_realize,
                                     &vubc->parent_realize);
     set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
diff --git a/hw/virtio/vhost-user-input.c b/hw/virtio/vhost-user-input.c
index 5cfc5bbb56..8fe6349ffc 100644
--- a/hw/virtio/vhost-user-input.c
+++ b/hw/virtio/vhost-user-input.c
@@ -7,10 +7,6 @@
 #include "qemu/osdep.h"
 #include "hw/virtio/virtio-input.h"
 
-static const Property vinput_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
-
 static void vinput_realize(DeviceState *dev, Error **errp)
 {
     VHostUserBase *vub = VHOST_USER_BASE(dev);
@@ -36,7 +32,6 @@ static void vhost_input_class_init(ObjectClass *klass, const void *data)
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_vhost_input;
-    device_class_set_props(dc, vinput_properties);
     device_class_set_parent_realize(dc, vinput_realize,
                                     &vubc->parent_realize);
     set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
diff --git a/hw/virtio/vhost-user-rng.c b/hw/virtio/vhost-user-rng.c
index 106c8f211a..664aee403d 100644
--- a/hw/virtio/vhost-user-rng.c
+++ b/hw/virtio/vhost-user-rng.c
@@ -20,10 +20,6 @@ static const VMStateDescription vu_rng_vmstate = {
     .unmigratable = 1,
 };
 
-static const Property vrng_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
-
 static void vu_rng_base_realize(DeviceState *dev, Error **errp)
 {
     VHostUserBase *vub = VHOST_USER_BASE(dev);
@@ -43,7 +39,6 @@ static void vu_rng_class_init(ObjectClass *klass, const void *data)
     VHostUserBaseClass *vubc = VHOST_USER_BASE_CLASS(klass);
 
     dc->vmsd = &vu_rng_vmstate;
-    device_class_set_props(dc, vrng_properties);
     device_class_set_parent_realize(dc, vu_rng_base_realize,
                                     &vubc->parent_realize);
 
diff --git a/hw/virtio/vhost-user-rtc.c b/hw/virtio/vhost-user-rtc.c
index 88b0c70b90..6d80cb6263 100644
--- a/hw/virtio/vhost-user-rtc.c
+++ b/hw/virtio/vhost-user-rtc.c
@@ -19,9 +19,6 @@ static const VMStateDescription vu_rtc_vmstate = {
     .unmigratable = 1,
 };
 
-static const Property vrtc_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
 
 static void vu_rtc_base_realize(DeviceState *dev, Error **errp)
 {
@@ -42,7 +39,6 @@ static void vu_rtc_class_init(ObjectClass *klass, const void *data)
     VHostUserBaseClass *vubc = VHOST_USER_BASE_CLASS(klass);
 
     dc->vmsd = &vu_rtc_vmstate;
-    device_class_set_props(dc, vrtc_properties);
     device_class_set_parent_realize(dc, vu_rtc_base_realize,
                                     &vubc->parent_realize);
 
diff --git a/hw/virtio/vhost-user-snd.c b/hw/virtio/vhost-user-snd.c
index 7129b77d9c..3663450aed 100644
--- a/hw/virtio/vhost-user-snd.c
+++ b/hw/virtio/vhost-user-snd.c
@@ -34,7 +34,6 @@ static const VMStateDescription vu_snd_vmstate = {
 };
 
 static const Property vsnd_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
     DEFINE_PROP_BIT64("controls", VHostUserBase,
                       parent_obj.host_features, VIRTIO_SND_F_CTLS, false),
 };
diff --git a/hw/virtio/vhost-user-spi.c b/hw/virtio/vhost-user-spi.c
index 707f96c250..fb6168d391 100644
--- a/hw/virtio/vhost-user-spi.c
+++ b/hw/virtio/vhost-user-spi.c
@@ -15,10 +15,6 @@
 #include "standard-headers/linux/virtio_ids.h"
 #include "standard-headers/linux/virtio_spi.h"
 
-static const Property vspi_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
-};
-
 static void vspi_realize(DeviceState *dev, Error **errp)
 {
     VHostUserBase *vub = VHOST_USER_BASE(dev);
@@ -44,7 +40,6 @@ static void vu_spi_class_init(ObjectClass *klass, const void *data)
     VHostUserBaseClass *vubc = VHOST_USER_BASE_CLASS(klass);
 
     dc->vmsd = &vu_spi_vmstate;
-    device_class_set_props(dc, vspi_properties);
     device_class_set_parent_realize(dc, vspi_realize,
                                     &vubc->parent_realize);
     set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
diff --git a/hw/virtio/vhost-user-test-device.c b/hw/virtio/vhost-user-test-device.c
index a2f963fdf6..91e23bf07a 100644
--- a/hw/virtio/vhost-user-test-device.c
+++ b/hw/virtio/vhost-user-test-device.c
@@ -30,7 +30,6 @@ static const VMStateDescription vud_vmstate = {
 };
 
 static const Property vud_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
     DEFINE_PROP_UINT16("virtio-id", VHostUserBase, virtio_id, 0),
     DEFINE_PROP_UINT32("vq_size", VHostUserBase, vq_size, 64),
     DEFINE_PROP_UINT32("num_vqs", VHostUserBase, num_vqs, 1),

-- 
2.43.0

[PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[ConKite]

In support of future work to add a memory isolation mode to vhost-user devices, a
qdev property, "memory-isolation", is added to all vhost-user devices. This is
simply a bool representing whether the mode is active or not.

Signed-off-by: Connor Kite <connorkite@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/vhost-user-blk.c            | 1 +
 hw/display/vhost-user-gpu.c          | 1 +
 hw/scsi/vhost-user-scsi.c            | 1 +
 hw/virtio/vhost-user-base.c          | 3 ++-
 hw/virtio/vhost-user-fs.c            | 1 +
 hw/virtio/vhost-user-scmi.c          | 1 +
 hw/virtio/vhost-user-vsock.c         | 1 +
 include/hw/virtio/vhost-user-base.h  | 1 +
 include/hw/virtio/vhost-user-blk.h   | 1 +
 include/hw/virtio/vhost-user-fs.h    | 1 +
 include/hw/virtio/vhost-user-scmi.h  | 1 +
 include/hw/virtio/vhost-user-vsock.h | 1 +
 include/hw/virtio/virtio-gpu.h       | 1 +
 include/hw/virtio/virtio-scsi.h      | 1 +
 14 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 2e5b3ae1b1..9830dc6c35 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -631,6 +631,7 @@ static const VMStateDescription vmstate_vhost_user_blk = {
 
 static const Property vhost_user_blk_properties[] = {
     DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VHostUserBlk, memory_isolation, false),
     DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
                        VHOST_USER_BLK_AUTO_NUM_QUEUES),
     DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128),
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 57360898ca..6d0ede672a 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -659,6 +659,7 @@ static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
 
 static const Property vhost_user_gpu_properties[] = {
     VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
+    DEFINE_PROP_BOOL("memory-isolation", VhostUserGPU, memory_isolation, false),
 };
 
 static void
diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
index 5070178dc2..c1a0393220 100644
--- a/hw/scsi/vhost-user-scsi.c
+++ b/hw/scsi/vhost-user-scsi.c
@@ -346,6 +346,7 @@ static void vhost_user_scsi_unrealize(DeviceState *dev)
 
 static const Property vhost_user_scsi_properties[] = {
     DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VirtIOSCSICommon, conf.memory_isolation, false),
     DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0),
     DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues,
                        VIRTIO_SCSI_AUTO_NUM_QUEUES),
diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
index 90bd2903db..d2a74a2860 100644
--- a/hw/virtio/vhost-user-base.c
+++ b/hw/virtio/vhost-user-base.c
@@ -409,7 +409,8 @@ static void vub_device_unrealize(DeviceState *dev)
 
 /*Define common qdev properties.  Inherited by all children*/
 static const Property vub_properties[] = {
-    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
+    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VHostUserBase, memory_isolation, false)
 };
 
 
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index 209993918a..3d980d814b 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -410,6 +410,7 @@ static const VMStateDescription vuf_backend_vmstate = {
 
 static const Property vuf_properties[] = {
     DEFINE_PROP_CHR("chardev", VHostUserFS, conf.chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VHostUserFS, conf.memory_isolation, false),
     DEFINE_PROP_STRING("tag", VHostUserFS, conf.tag),
     DEFINE_PROP_UINT16("num-request-queues", VHostUserFS,
                        conf.num_request_queues, 1),
diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
index 02dc088ea9..d514d89392 100644
--- a/hw/virtio/vhost-user-scmi.c
+++ b/hw/virtio/vhost-user-scmi.c
@@ -287,6 +287,7 @@ static const VMStateDescription vu_scmi_vmstate = {
 
 static const Property vu_scmi_properties[] = {
     DEFINE_PROP_CHR("chardev", VHostUserSCMI, chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VHostUserSCMI, memory_isolation, false),
 };
 
 static void vu_scmi_class_init(ObjectClass *klass, const void *data)
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index c2cd376e73..89aa789e22 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -154,6 +154,7 @@ static void vuv_device_unrealize(DeviceState *dev)
 
 static const Property vuv_properties[] = {
     DEFINE_PROP_CHR("chardev", VHostUserVSock, conf.chardev),
+    DEFINE_PROP_BOOL("memory-isolation", VHostUserVSock, conf.memory_isolation, false),
 };
 
 static void vuv_class_init(ObjectClass *klass, const void *data)
diff --git a/include/hw/virtio/vhost-user-base.h b/include/hw/virtio/vhost-user-base.h
index d5d2f081ae..29bc40736c 100644
--- a/include/hw/virtio/vhost-user-base.h
+++ b/include/hw/virtio/vhost-user-base.h
@@ -25,6 +25,7 @@ struct VHostUserBase {
     uint32_t num_vqs;
     uint32_t vq_size; /* can't exceed VIRTIO_QUEUE_MAX */
     uint32_t config_size;
+    bool memory_isolation;
     /* State tracking */
     VhostUserState vhost_user;
     struct vhost_virtqueue *vhost_vq;
diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h
index dee848cfd8..8126447a86 100644
--- a/include/hw/virtio/vhost-user-blk.h
+++ b/include/hw/virtio/vhost-user-blk.h
@@ -30,6 +30,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserBlk, VHOST_USER_BLK)
 struct VHostUserBlk {
     VirtIODevice parent_obj;
     CharFrontend chardev;
+    bool memory_isolation;
     int32_t bootindex;
     struct virtio_blk_config blkcfg;
     uint16_t num_queues;
diff --git a/include/hw/virtio/vhost-user-fs.h b/include/hw/virtio/vhost-user-fs.h
index 5699ea03a1..bf71990b9c 100644
--- a/include/hw/virtio/vhost-user-fs.h
+++ b/include/hw/virtio/vhost-user-fs.h
@@ -28,6 +28,7 @@ typedef struct {
     char *tag;
     uint16_t num_request_queues;
     uint16_t queue_size;
+    bool memory_isolation;
 } VHostUserFSConf;
 
 struct VHostUserFS {
diff --git a/include/hw/virtio/vhost-user-scmi.h b/include/hw/virtio/vhost-user-scmi.h
index 30ab0d7ab7..d346b42ed8 100644
--- a/include/hw/virtio/vhost-user-scmi.h
+++ b/include/hw/virtio/vhost-user-scmi.h
@@ -19,6 +19,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCMI, VHOST_USER_SCMI);
 struct VHostUserSCMI {
     VirtIODevice parent;
     CharFrontend chardev;
+    bool memory_isolation;
     struct vhost_virtqueue *vhost_vqs;
     struct vhost_dev vhost_dev;
     VhostUserState vhost_user;
diff --git a/include/hw/virtio/vhost-user-vsock.h b/include/hw/virtio/vhost-user-vsock.h
index 5db6938b8b..6288b26fa4 100644
--- a/include/hw/virtio/vhost-user-vsock.h
+++ b/include/hw/virtio/vhost-user-vsock.h
@@ -21,6 +21,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserVSock, VHOST_USER_VSOCK)
 
 typedef struct {
     CharFrontend chardev;
+    bool memory_isolation;
 } VHostUserVSockConf;
 
 struct VHostUserVSock {
diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
index b9bad27c97..707612c3a4 100644
--- a/include/hw/virtio/virtio-gpu.h
+++ b/include/hw/virtio/virtio-gpu.h
@@ -277,6 +277,7 @@ struct VhostUserGPU {
     CharFrontend vhost_chr;
     QemuDmaBuf *dmabuf[VIRTIO_GPU_MAX_SCANOUTS];
     bool backend_blocked;
+    bool memory_isolation;
 };
 
 #define MAX_SLOTS 4096
diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
index b6028bb5cd..47fdc3785c 100644
--- a/include/hw/virtio/virtio-scsi.h
+++ b/include/hw/virtio/virtio-scsi.h
@@ -59,6 +59,7 @@ struct VirtIOSCSIConf {
     char *vhostfd;
     char *wwpn;
     CharFrontend chardev;
+    bool memory_isolation;
     uint32_t boot_tpgt;
     IOThread *iothread;
     IOThreadVirtQueueMappingList *iothread_vq_mapping_list;

-- 
2.43.0

[PATCH RFC 03/15] backends/cryptodev-vhost-user: add memory isolation bool

[Connor Kite]

Add a memory_isolation property to CryptoDevBackendVhostUser and
add add it as an optional member of CryptodevVhostUserProperties
in qapi.

This is needed to be able to specify memory isolation for
cryptodev-vhost-user objects, and the bool will eventually be
passed to vhost_user_init() in a future patch.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 backends/cryptodev-vhost-user.c | 22 ++++++++++++++++++++++
 qapi/qom.json                   |  6 +++++-
 2 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c
index cc478d9902..e0547c5d40 100644
--- a/backends/cryptodev-vhost-user.c
+++ b/backends/cryptodev-vhost-user.c
@@ -49,6 +49,7 @@ struct CryptoDevBackendVhostUser {
     CharFrontend chr;
     char *chr_name;
     bool opened;
+    bool memory_isolation;
     CryptoDevBackendVhost *vhost_crypto[MAX_CRYPTO_QUEUE_NUM];
 };
 
@@ -392,6 +393,23 @@ static void cryptodev_vhost_user_finalize(Object *obj)
     g_free(s->chr_name);
 }
 
+static void cryptodev_vhost_user_set_mem_isolation(Object *obj, bool value,
+                                                   Error **errp)
+{
+    CryptoDevBackendVhostUser *s =
+                      CRYPTODEV_BACKEND_VHOST_USER(obj);
+
+    s->memory_isolation = value;
+}
+
+static bool cryptodev_vhost_user_get_mem_isolation(Object *obj, Error **errp)
+{
+    CryptoDevBackendVhostUser *s =
+                      CRYPTODEV_BACKEND_VHOST_USER(obj);
+
+    return s->memory_isolation;
+}
+
 static void
 cryptodev_vhost_user_class_init(ObjectClass *oc, const void *data)
 {
@@ -407,6 +425,10 @@ cryptodev_vhost_user_class_init(ObjectClass *oc, const void *data)
                                   cryptodev_vhost_user_get_chardev,
                                   cryptodev_vhost_user_set_chardev);
 
+    object_class_property_add_bool(oc, "memory-isolation",
+                                   cryptodev_vhost_user_get_mem_isolation,
+                                   cryptodev_vhost_user_set_mem_isolation);
+
 }
 
 static const TypeInfo cryptodev_vhost_user_info = {
diff --git a/qapi/qom.json b/qapi/qom.json
index c55776af7d..d7ca1b4203 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -350,11 +350,15 @@
 # @chardev: the name of a Unix domain socket character device that
 #     connects to the vhost-user server
 #
+# @memory-isolation: disables access from cryptodev to guest memory.
+#     (default: false)
+#
 # Since: 2.12
 ##
 { 'struct': 'CryptodevVhostUserProperties',
   'base': 'CryptodevBackendProperties',
-  'data': { 'chardev': 'str' },
+  'data': { 'chardev': 'str',
+            '*memory-isolation': 'bool' },
   'if': 'CONFIG_VHOST_CRYPTO' }
 
 ##

-- 
2.43.0

[PATCH RFC 04/15] net/vhost-user: add memory isolation

[Connor Kite]

Add a memory-isolation property to NetdevPassOptions and
NetDevVhostUserOptions in net.json.  This property will
be used for vhost-user devices to disable access to guest
memory from devices.

It will be passed into vhost_user_init() in a future patch.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 net/passt.c      |  7 +++++++
 net/vhost-user.c |  4 ++++
 qapi/net.json    | 15 +++++++++++----
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/net/passt.c b/net/passt.c
index 45440c399b..ce80186883 100644
--- a/net/passt.c
+++ b/net/passt.c
@@ -735,6 +735,13 @@ int net_init_passt(const Netdev *netdev, const char *name,
     s->pidfile = pidfile;
 
     if (netdev->u.passt.has_vhost_user && netdev->u.passt.vhost_user) {
+        bool memory_isolation G_GNUC_UNUSED = false;
+
+        if (netdev->u.passt.has_memory_isolation &&
+           netdev->u.passt.memory_isolation) {
+            memory_isolation = true;
+        }
+
         if (net_passt_vhost_user_init(s, errp) == -1) {
             qemu_del_net_client(nc);
             return -1;
diff --git a/net/vhost-user.c b/net/vhost-user.c
index 2d0fc49b4d..8fa303f901 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -459,6 +459,7 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
     int queues;
     const NetdevVhostUserOptions *vhost_user_opts;
     Chardev *chr;
+    bool memory_isolation G_GNUC_UNUSED;
 
     assert(netdev->type == NET_CLIENT_DRIVER_VHOST_USER);
     vhost_user_opts = &netdev->u.vhost_user;
@@ -476,5 +477,8 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
         return -1;
     }
 
+    memory_isolation = vhost_user_opts->has_memory_isolation ?
+                       vhost_user_opts->memory_isolation : false;
+
     return net_vhost_user_init(peer, "vhost_user", name, chr, queues);
 }
diff --git a/qapi/net.json b/qapi/net.json
index 1a6382825c..4588c7014b 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -213,6 +213,9 @@
 #
 # @param: parameter to pass to passt command
 #
+# @memory-isolation: enable memory isolation.  Only has effect if
+#     using vhost-user.  (default: false)
+#
 # Since: 10.1
 ##
 { 'struct': 'NetdevPasstOptions',
@@ -250,7 +253,8 @@
     '*ipv6':               'bool',
     '*tcp-ports':          ['PasstPortForward'],
     '*udp-ports':          ['PasstPortForward'],
-    '*param':              ['PasstParameter'] },
+    '*param':              ['PasstParameter'],
+    '*memory-isolation':   'bool'},
     'if': 'CONFIG_PASST' }
 
 ##
@@ -685,13 +689,16 @@
 # @queues: number of queues to be created for multiqueue vhost-user
 #     (default: 1) (Since 2.5)
 #
+# @memory-isolation: isolate guest memory from device (default: false)
+#
 # Since: 2.1
 ##
 { 'struct': 'NetdevVhostUserOptions',
   'data': {
-    'chardev':        'str',
-    '*vhostforce':    'bool',
-    '*queues':        'int' } }
+    'chardev':              'str',
+    '*vhostforce':          'bool',
+    '*queues':              'int',
+    '*memory-isolation':    'bool' } }
 
 ##
 # @NetdevVhostVDPAOptions:

-- 
2.43.0

[PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Connor Kite]

Add a memory_isolation bool to the VhostUserState struct. This
bool is set within vhost_user_init(), which takes a
memory_isolation bool as an argument.

Refactor all call locations of vhost_user_init to include the new
argument.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 backends/cryptodev-vhost-user.c     |  2 +-
 backends/vhost-user.c               |  4 ++--
 hw/block/vhost-user-blk.c           |  3 ++-
 hw/display/vhost-user-gpu.c         |  3 ++-
 hw/scsi/vhost-user-scsi.c           |  3 ++-
 hw/virtio/vhost-stub.c              |  3 ++-
 hw/virtio/vhost-user-base.c         |  3 ++-
 hw/virtio/vhost-user-fs.c           |  3 ++-
 hw/virtio/vhost-user-scmi.c         |  3 ++-
 hw/virtio/vhost-user-vsock.c        |  3 ++-
 hw/virtio/vhost-user.c              |  5 ++++-
 include/hw/virtio/vhost-user.h      |  6 +++++-
 include/system/vhost-user-backend.h |  3 ++-
 net/passt.c                         | 13 ++++++++-----
 net/vhost-user.c                    |  9 +++++----
 15 files changed, 43 insertions(+), 23 deletions(-)

diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c
index e0547c5d40..328d7fbdfe 100644
--- a/backends/cryptodev-vhost-user.c
+++ b/backends/cryptodev-vhost-user.c
@@ -214,7 +214,7 @@ static void cryptodev_vhost_user_init(
         }
     }
 
-    if (!vhost_user_init(&s->vhost_user, &s->chr, errp)) {
+    if (!vhost_user_init(&s->vhost_user, &s->chr, s->memory_isolation, errp)) {
         return;
     }
 
diff --git a/backends/vhost-user.c b/backends/vhost-user.c
index 380d825023..470eba1775 100644
--- a/backends/vhost-user.c
+++ b/backends/vhost-user.c
@@ -22,13 +22,13 @@
 
 int
 vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
-                            unsigned nvqs, Error **errp)
+                            unsigned nvqs, bool memory_isolation, Error **errp)
 {
     int ret;
 
     assert(!b->vdev && vdev);
 
-    if (!vhost_user_init(&b->vhost_user, &b->chr, errp)) {
+    if (!vhost_user_init(&b->vhost_user, &b->chr, memory_isolation, errp)) {
         return -1;
     }
 
diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
index 9830dc6c35..5c1a19eb50 100644
--- a/hw/block/vhost-user-blk.c
+++ b/hw/block/vhost-user-blk.c
@@ -505,7 +505,8 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!vhost_user_init(&s->vhost_user, &s->chardev, errp)) {
+    if (!vhost_user_init(&s->vhost_user, &s->chardev,
+                         s->memory_isolation, errp)) {
         return;
     }
 
diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
index 6d0ede672a..b44c989f58 100644
--- a/hw/display/vhost-user-gpu.c
+++ b/hw/display/vhost-user-gpu.c
@@ -625,7 +625,8 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
     VirtIODevice *vdev = VIRTIO_DEVICE(g);
 
     vhost_dev_set_config_notifier(&g->vhost->dev, &config_ops);
-    if (vhost_user_backend_dev_init(g->vhost, vdev, 2, errp) < 0) {
+    if (vhost_user_backend_dev_init(g->vhost, vdev, 2, g->memory_isolation,
+                                    errp) < 0) {
         return;
     }
 
diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
index c1a0393220..5a19182f2b 100644
--- a/hw/scsi/vhost-user-scsi.c
+++ b/hw/scsi/vhost-user-scsi.c
@@ -276,7 +276,8 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!vhost_user_init(&s->vhost_user, &vs->conf.chardev, errp)) {
+    if (!vhost_user_init(&s->vhost_user, &vs->conf.chardev,
+                         vs->conf.memory_isolation, errp)) {
         goto free_virtio;
     }
 
diff --git a/hw/virtio/vhost-stub.c b/hw/virtio/vhost-stub.c
index 7321dc9210..d1e1075a31 100644
--- a/hw/virtio/vhost-stub.c
+++ b/hw/virtio/vhost-stub.c
@@ -12,7 +12,8 @@ unsigned int vhost_get_free_memslots(void)
     return UINT_MAX;
 }
 
-bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp)
+bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
+                     bool memory_isolation, Error **errp)
 {
     return false;
 }
diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
index d2a74a2860..11864a3f08 100644
--- a/hw/virtio/vhost-user-base.c
+++ b/hw/virtio/vhost-user-base.c
@@ -317,7 +317,8 @@ static void vub_device_realize(DeviceState *dev, Error **errp)
         vub->vhost_user.supports_config = true;
     }
 
-    if (!vhost_user_init(&vub->vhost_user, &vub->chardev, errp)) {
+    if (!vhost_user_init(&vub->vhost_user, &vub->chardev,
+                         vub->memory_isolation, errp)) {
         return;
     }
 
diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
index 3d980d814b..07ff2a35e0 100644
--- a/hw/virtio/vhost-user-fs.c
+++ b/hw/virtio/vhost-user-fs.c
@@ -238,7 +238,8 @@ static void vuf_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!vhost_user_init(&fs->vhost_user, &fs->conf.chardev, errp)) {
+    if (!vhost_user_init(&fs->vhost_user, &fs->conf.chardev,
+                         fs->conf.memory_isolation, errp)) {
         return;
     }
 
diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
index d514d89392..2c1dbb723e 100644
--- a/hw/virtio/vhost-user-scmi.c
+++ b/hw/virtio/vhost-user-scmi.c
@@ -244,7 +244,8 @@ static void vu_scmi_device_realize(DeviceState *dev, Error **errp)
 
     vdev->host_features |= (1ULL << VIRTIO_SCMI_F_P2A_CHANNELS);
 
-    if (!vhost_user_init(&scmi->vhost_user, &scmi->chardev, errp)) {
+    if (!vhost_user_init(&scmi->vhost_user, &scmi->chardev,
+                         scmi->memory_isolation, errp)) {
         return;
     }
 
diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
index 89aa789e22..33ab3b69e6 100644
--- a/hw/virtio/vhost-user-vsock.c
+++ b/hw/virtio/vhost-user-vsock.c
@@ -106,7 +106,8 @@ static void vuv_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!vhost_user_init(&vsock->vhost_user, &vsock->conf.chardev, errp)) {
+    if (!vhost_user_init(&vsock->vhost_user, &vsock->conf.chardev,
+                         vsock->conf.memory_isolation, errp)) {
         return;
     }
 
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 517cc4ca71..f296b63fb9 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -3118,13 +3118,16 @@ static void vhost_user_state_destroy(gpointer data)
     vhost_user_host_notifier_remove(n, NULL, true);
 }
 
-bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp)
+bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
+                     bool memory_isolation,
+                     Error **errp)
 {
     if (user->chr) {
         error_setg(errp, "Cannot initialize vhost-user state");
         return false;
     }
     user->chr = chr;
+    user->memory_isolation = memory_isolation;
     user->memory_slots = 0;
     user->notifiers = g_ptr_array_new_full(VIRTIO_QUEUE_MAX / 4,
                                            &vhost_user_state_destroy);
diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
index 06c360af18..72b5787efe 100644
--- a/include/hw/virtio/vhost-user.h
+++ b/include/hw/virtio/vhost-user.h
@@ -66,6 +66,7 @@ typedef struct VhostUserHostNotifier {
  * @chr: the character backend for the socket
  * @notifiers: GPtrArray of @VhostUserHostnotifier
  * @memory_slots:
+ * @memory_isolation: determines whether data is shared or copied
  */
 typedef struct VhostUserState {
     CharFrontend *chr;
@@ -73,12 +74,14 @@ typedef struct VhostUserState {
     int memory_slots;
     bool supports_config;
     bool supports_inflight_migration;
+    bool memory_isolation;
 } VhostUserState;
 
 /**
  * vhost_user_init() - initialise shared vhost_user state
  * @user: allocated area for storing shared state
  * @chr: the chardev for the vhost socket
+ * @memory_isolation: disables device access to guest memory
  * @errp: error handle
  *
  * User can either directly g_new() space for the state or embed
@@ -87,7 +90,8 @@ typedef struct VhostUserState {
  *
  * Return: true on success, false on error while setting errp.
  */
-bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp);
+bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
+                     bool memory_isolation, Error **errp);
 
 /**
  * vhost_user_cleanup() - cleanup state
diff --git a/include/system/vhost-user-backend.h b/include/system/vhost-user-backend.h
index 3184c8e799..c23f689182 100644
--- a/include/system/vhost-user-backend.h
+++ b/include/system/vhost-user-backend.h
@@ -41,7 +41,8 @@ struct VhostUserBackend {
 };
 
 int vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
-                                unsigned nvqs, Error **errp);
+                                unsigned nvqs, bool memory_isolation,
+                                Error **errp);
 void vhost_user_backend_start(VhostUserBackend *b);
 int vhost_user_backend_stop(VhostUserBackend *b);
 
diff --git a/net/passt.c b/net/passt.c
index ce80186883..a3021f37ef 100644
--- a/net/passt.c
+++ b/net/passt.c
@@ -433,7 +433,9 @@ static void passt_vhost_user_event(void *opaque, QEMUChrEvent event)
     }
 }
 
-static int net_passt_vhost_user_init(NetPasstState *s, Error **errp)
+static int net_passt_vhost_user_init(NetPasstState *s,
+                                     bool memory_isolation,
+                                     Error **errp)
 {
     Chardev *chr;
     int sv[2];
@@ -457,7 +459,8 @@ static int net_passt_vhost_user_init(NetPasstState *s, Error **errp)
 
     s->vhost_user = g_new0(struct VhostUserState, 1);
     if (!qemu_chr_fe_init(&s->vhost_chr, chr, errp) ||
-        !vhost_user_init(s->vhost_user, &s->vhost_chr, errp)) {
+        !vhost_user_init(s->vhost_user, &s->vhost_chr, memory_isolation,
+                         errp)) {
         goto err;
     }
 
@@ -735,14 +738,14 @@ int net_init_passt(const Netdev *netdev, const char *name,
     s->pidfile = pidfile;
 
     if (netdev->u.passt.has_vhost_user && netdev->u.passt.vhost_user) {
-        bool memory_isolation G_GNUC_UNUSED = false;
+        bool memory_isolation = false;
 
         if (netdev->u.passt.has_memory_isolation &&
-           netdev->u.passt.memory_isolation) {
+            netdev->u.passt.memory_isolation) {
             memory_isolation = true;
         }
 
-        if (net_passt_vhost_user_init(s, errp) == -1) {
+        if (net_passt_vhost_user_init(s, memory_isolation, errp) == -1) {
             qemu_del_net_client(nc);
             return -1;
         }
diff --git a/net/vhost-user.c b/net/vhost-user.c
index 8fa303f901..722ab20228 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -370,7 +370,7 @@ static void net_vhost_user_event(void *opaque, QEMUChrEvent event)
 
 static int net_vhost_user_init(NetClientState *peer, const char *device,
                                const char *name, Chardev *chr,
-                               int queues)
+                               int queues, bool memory_isolation)
 {
     Error *err = NULL;
     NetClientState *nc, *nc0 = NULL;
@@ -390,7 +390,7 @@ static int net_vhost_user_init(NetClientState *peer, const char *device,
             nc0 = nc;
             s = DO_UPCAST(NetVhostUserState, nc, nc);
             if (!qemu_chr_fe_init(&s->chr, chr, &err) ||
-                !vhost_user_init(user, &s->chr, &err)) {
+                !vhost_user_init(user, &s->chr, memory_isolation, &err)) {
                 error_report_err(err);
                 goto err;
             }
@@ -459,7 +459,7 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
     int queues;
     const NetdevVhostUserOptions *vhost_user_opts;
     Chardev *chr;
-    bool memory_isolation G_GNUC_UNUSED;
+    bool memory_isolation;
 
     assert(netdev->type == NET_CLIENT_DRIVER_VHOST_USER);
     vhost_user_opts = &netdev->u.vhost_user;
@@ -480,5 +480,6 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
     memory_isolation = vhost_user_opts->has_memory_isolation ?
                        vhost_user_opts->memory_isolation : false;
 
-    return net_vhost_user_init(peer, "vhost_user", name, chr, queues);
+    return net_vhost_user_init(peer, "vhost_user", name, chr, queues,
+                               memory_isolation);
 }

-- 
2.43.0

[PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Connor Kite]

Add a wrapper to iova-tree to call g_tree_foreach.  Provies a means to
execute some function on each node of the iova-tree.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 include/qemu/iova-tree.h | 13 +++++++++++++
 util/iova-tree.c         |  5 +++++
 2 files changed, 18 insertions(+)

diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h
index 14e82a22d5..4d77f4dc57 100644
--- a/include/qemu/iova-tree.h
+++ b/include/qemu/iova-tree.h
@@ -161,4 +161,17 @@ int iova_tree_alloc_map(IOVATree *tree, DMAMap *map, hwaddr iova_begin,
  */
 void iova_tree_destroy(IOVATree *tree);
 
+/**
+ * iova_tree_foreach:
+ *
+ * @tree: the iova tree to traverse
+ * @func: the function called at every node. A true return value ends traversal
+ * @data: an argument passed to func
+ *
+ * Wrapper to call g_tree_foreach on underlying g_tree
+ *
+ * Return: None
+ */
+void iova_tree_foreach(IOVATree *tree, GTraverseFunc func, gpointer data);
+
 #endif
diff --git a/util/iova-tree.c b/util/iova-tree.c
index 5b0c95ff15..2239bb6ae1 100644
--- a/util/iova-tree.c
+++ b/util/iova-tree.c
@@ -303,3 +303,8 @@ int gpa_tree_insert(IOVATree *tree, const DMAMap *map)
 
     return IOVA_OK;
 }
+
+void iova_tree_foreach(IOVATree *tree, GTraverseFunc func, gpointer data)
+{
+    g_tree_foreach(tree->tree, func, data);
+}

-- 
2.43.0

[PATCH RFC 07/15] hw/virtio: iova_tree_foreach wrapper

[Connor Kite]

Add a wrapper to vhost-iova-tree that calls iova_tree_foreach on the
underlying iova_map.  This provides a means to execute some function on
each node of the tree.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-iova-tree.c | 15 +++++++++++++++
 hw/virtio/vhost-iova-tree.h |  3 +++
 2 files changed, 18 insertions(+)

diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c
index fa4147b773..d8206379bf 100644
--- a/hw/virtio/vhost-iova-tree.c
+++ b/hw/virtio/vhost-iova-tree.c
@@ -189,3 +189,18 @@ void vhost_iova_tree_remove_gpa(VhostIOVATree *iova_tree, DMAMap map)
     iova_tree_remove(iova_tree->gpa_iova_map, map);
     iova_tree_remove(iova_tree->iova_map, map);
 }
+
+
+/**
+ * Traverse through the iova tree in sorted order
+ *
+ * @tree: The VhostIOVATree
+ * @func: Function called at every node.  A false return value ends traversal
+ * @data: An argument passed to func
+ */
+void vhost_iova_tree_foreach(VhostIOVATree *tree,
+                             GTraverseFunc func,
+                             gpointer data)
+{
+    iova_tree_foreach(tree->iova_map, func, data);
+}
diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h
index 08f63b61cd..f3b7c48cb9 100644
--- a/hw/virtio/vhost-iova-tree.h
+++ b/hw/virtio/vhost-iova-tree.h
@@ -29,5 +29,8 @@ const DMAMap *vhost_iova_tree_find_gpa(const VhostIOVATree *iova_tree,
 int vhost_iova_tree_map_alloc_gpa(VhostIOVATree *iova_tree, DMAMap *map,
                                   hwaddr taddr);
 void vhost_iova_tree_remove_gpa(VhostIOVATree *iova_tree, DMAMap map);
+void vhost_iova_tree_foreach(VhostIOVATree *tree, GTraverseFunc func,
+                               gpointer data);
+
 
 #endif

-- 
2.43.0

[PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Connor Kite]

Adds a handler callback to shadow virtqueues in order to enable
non-default processing of used vring elements from the device.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
 hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index bcb7f2ffc7..eb86c1ee37 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -669,6 +669,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
                 break;
             }
 
+            if (svq->ops && svq->ops->used_handler) {
+                svq->ops->used_handler(svq, elem, svq->ops_opaque);
+            }
+
             if (unlikely(i >= svq->vring.num)) {
                 qemu_log_mask(LOG_GUEST_ERROR,
                          "More than %u used buffers obtained in a %u size SVQ",
diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index ec16a1e838..ccfeee36d7 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -56,8 +56,22 @@ typedef int (*VirtQueueAvailCallback)(VhostShadowVirtqueue *svq,
                                       VirtQueueElement *elem,
                                       void *vq_callback_opaque);
 
+/**
+ * Callback to handle a used buffer
+ *
+ * @svq: Shadow virtqueue
+ * @elem: Element placed in the queue by the device
+ * @vq_callback_opaque: Used to pass arguments to callback
+ *
+ * Returns 0 if the vq is running as expected
+ */
+typedef int (*VirtQueueUsedCallback)(VhostShadowVirtqueue *svq,
+                                      VirtQueueElement *elem,
+                                      void *vq_callback_opaque);
+
 typedef struct VhostShadowVirtqueueOps {
     VirtQueueAvailCallback avail_handler;
+    VirtQueueUsedCallback used_handler;
 } VhostShadowVirtqueueOps;
 
 /* Shadow virtqueue to relay notifications */

-- 
2.43.0

[PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Connor Kite]

By default svq vrings are placed in an anonymous memory map. As svqs
will be leveraged to enable memory isolation in vhost-user, it is useful
to be able to place the vrings in a shared isolation memory region.

Adds the option to specify vring placement by providing a vring base
address before starting the svq.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 22 +++++++++++++++-------
 hw/virtio/vhost-shadow-virtqueue.h |  3 +++
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index eb86c1ee37..9e3c359f50 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -857,14 +857,21 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
 
     svq->vring.num = virtio_queue_get_num(vdev, virtio_get_queue_index(vq));
     svq->num_free = svq->vring.num;
-    svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
-                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
-                           -1, 0);
     desc_size = sizeof(vring_desc_t) * svq->vring.num;
-    svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
-    svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
-                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
-                           -1, 0);
+    if (svq->base_addr == NULL) {
+        svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
+                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
+                            -1, 0);
+        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
+        svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
+                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
+                            -1, 0);
+    } else {
+        svq->vring.desc = (void *) svq->base_addr;
+        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
+        svq->vring.used = (void *)((char *)svq->base_addr +
+                          vhost_svq_driver_area_size(svq));
+    }
     svq->desc_state = g_new0(SVQDescState, svq->vring.num);
     if (virtio_vdev_has_feature(svq->vdev, VIRTIO_F_IN_ORDER)) {
         svq->batch_last.id = VIRTIO_RING_NOT_IN_BATCH;
@@ -929,6 +936,7 @@ VhostShadowVirtqueue *vhost_svq_new(const VhostShadowVirtqueueOps *ops,
     event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND);
     svq->ops = ops;
     svq->ops_opaque = ops_opaque;
+    svq->base_addr = NULL;
     return svq;
 }
 
diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
index ccfeee36d7..39f69e6455 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -148,6 +148,9 @@ typedef struct VhostShadowVirtqueue {
 
     /* Size of SVQ vring free descriptors */
     uint16_t num_free;
+
+    /* Location assigned to vrings if not in default anon memory map*/
+    hwaddr *base_addr;
 } VhostShadowVirtqueue;
 
 bool vhost_svq_valid_features(uint64_t features, Error **errp);

-- 
2.43.0

[PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Connor Kite]

iova-tree expects inclusive range sizing when maps are allocated or searched.
Currently, svqs use exclusive sizing when searching their
vhost-iova-tree for a match to the region to be translated.  This could
lead to errors if the region to be translated is at the edge of an iova
region.

Fix this by reducing `needle.size` by 1 in
vhost_svq_translate_addr to bring then it line with DMAMap and iova-tree
convention.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-shadow-virtqueue.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
index 9e3c359f50..20e5c7d2f1 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -104,14 +104,14 @@ static bool vhost_svq_translate_addr(const VhostShadowVirtqueue *svq,
             /* Search the GPA->IOVA tree */
             needle = (DMAMap) {
                 .translated_addr = gpas[i],
-                .size = iovec[i].iov_len,
+                .size = iovec[i].iov_len - 1,  /* Inclusive */
             };
             map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
         } else {
             /* Search the IOVA->HVA tree */
             needle = (DMAMap) {
                 .translated_addr = (hwaddr)(uintptr_t)iovec[i].iov_base,
-                .size = iovec[i].iov_len,
+                .size = iovec[i].iov_len - 1, /* Inclusive */
             };
             map = vhost_iova_tree_find_iova(svq->iova_tree, &needle);
         }

-- 
2.43.0

[PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

If memory isolation mode is active for the vhost-user device adds
features to:
- Gather the size required for bounce buffers and vrings in shared
  isolation region
- Allocate the required space in an anonymous file
- Create a vhost-iova-tree with space to map entire isolation region
- Map guest memory regions and shared vrings into the tree
- Release these resources upon backend cleanup

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-user.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 129 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index f296b63fb9..710cf966f8 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -18,6 +18,7 @@
 #include "hw/virtio/vhost-backend.h"
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/virtio-net.h"
+#include "hw/virtio/vhost-iova-tree.h"
 #include "chardev/char-fe.h"
 #include "io/channel-socket.h"
 #include "system/kvm.h"
@@ -25,6 +26,7 @@
 #include "qemu/main-loop.h"
 #include "qemu/uuid.h"
 #include "qemu/sockets.h"
+#include "qemu/memfd.h"
 #include "system/runstate.h"
 #include "system/cryptodev.h"
 #include "migration/postcopy-ram.h"
@@ -320,6 +322,13 @@ static VhostUserMsg m __attribute__ ((unused));
 /* The version of the protocol we support */
 #define VHOST_USER_VERSION    (0x1)
 
+typedef struct IsolationRegion {
+    uint64_t base_addr;
+    uint64_t vring_base_addr;
+    uint64_t size;
+    int iso_fd;
+} IsolationRegion;
+
 struct vhost_user {
     struct vhost_dev *dev;
     /* Shared between vhost devs of the same virtio device */
@@ -353,6 +362,10 @@ struct vhost_user {
      * by the backend (see @features).
      */
     uint64_t protocol_features;
+
+    /* Isolated memory data*/
+    struct IsolationRegion iso_memory;
+    VhostIOVATree *iso_iova_tree;
 };
 
 struct scrub_regions {
@@ -1109,6 +1122,121 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
     return 0;
 }
 
+/* TODO: Is there any notifier cleanup required here?*/
+static void cleanup_isolation_regions(struct vhost_dev *dev)
+{
+    struct vhost_user *u = dev->opaque;
+    if (u->iso_memory.base_addr) {
+        vhost_iova_tree_delete(u->iso_iova_tree);
+        u->iso_iova_tree = NULL;
+        memset(&u->iso_memory, 0, sizeof(IsolationRegion));
+        qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
+                         u->iso_memory.iso_fd);
+        u->iso_memory.base_addr = 0;
+    }
+}
+
+__attribute__((unused))
+static int init_isolation_regions(struct vhost_dev *dev,
+                                  VhostUserMsg *msg,
+                                  int *fds, size_t *fd_num)
+{
+    Error *err = NULL;
+    struct vhost_user *u = dev->opaque;
+    uint32_t nregions = dev->mem->nregions;
+    uint64_t buffer_reg_size = 0;
+    DMAMap newEntry = {
+        .perm = IOMMU_RW
+    };
+    g_autoptr(GArray) buffer_regions =
+        g_array_new(FALSE, TRUE, sizeof(DMAMap));
+
+    msg->hdr.request = VHOST_USER_SET_MEM_TABLE;
+
+    /* In case of reset, clear old regions*/
+    if (u->iso_memory.base_addr != 0) {
+        cleanup_isolation_regions(dev);
+        vhost_iova_tree_delete(u->iso_iova_tree);
+    }
+
+    /* Gather information for bounce buffers to be mapped */
+    for (int i = 0; i < nregions; i++) {
+        struct vhost_memory_region *dev_region = &dev->mem->regions[i];
+        hwaddr size = ROUND_UP(dev_region->memory_size,
+                      qemu_real_host_page_size());
+        newEntry.translated_addr = dev_region->guest_phys_addr;
+        newEntry.size = size - 1;
+        buffer_reg_size += size;
+        g_array_append_val(buffer_regions, newEntry);
+    }
+
+    int num;
+    size_t desc_size;
+    size_t avail_size;
+    size_t driver_area_size;
+    size_t device_area_size;
+    size_t total_vring_size = 0;
+    size_t total_mmap_size;
+
+    /* Get space required for all vrings */
+    for (int j = 0; j < dev->nvqs; j++) {
+        num = virtio_queue_get_num(dev->vdev, dev->vq_index + j);
+        desc_size = sizeof(vring_desc_t) * num;
+        avail_size = offsetof(vring_avail_t, ring[num]) +
+                        sizeof(uint16_t);
+        driver_area_size = ROUND_UP(desc_size + avail_size,
+                            qemu_real_host_page_size());
+        device_area_size = ROUND_UP(offsetof(vring_used_t, ring[num]) +
+                                    sizeof(uint16_t),
+                                    qemu_real_host_page_size());
+        total_vring_size += driver_area_size + device_area_size;
+    }
+
+    total_mmap_size = buffer_reg_size + total_vring_size;
+
+    /* Allocate and map an anonymous file to hold the isolation region */
+    u->iso_memory.base_addr = (uint64_t) qemu_memfd_alloc("iso_r",
+                              total_mmap_size,
+                              F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL,
+                              &u->iso_memory.iso_fd, &err);
+    u->iso_memory.size = total_mmap_size;
+
+    if (err) {
+        error_report_err(err);
+        cleanup_isolation_regions(dev);
+        return -1;
+    }
+
+    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
+                                                 total_mmap_size - 1));
+
+    /*
+     * Instantiates iova tree sized to map bounce buffers and vrings to the
+     * isolation region in host va.
+     */
+    u->iso_iova_tree = vhost_iova_tree_new(u->iso_memory.base_addr, last_addr);
+
+    assert(&u->iso_memory.iso_fd >= 0);
+    DMAMap *map;
+    DMAMap vring_map = {
+        .perm = IOMMU_RW,
+        .size = total_vring_size - 1,
+        /*vrings are allocated on tree first, so will be assigned base addr*/
+        .translated_addr = u->iso_memory.base_addr
+    };
+
+    vhost_iova_tree_map_alloc(u->iso_iova_tree, &vring_map,
+                              vring_map.translated_addr);
+    u->iso_memory.vring_base_addr = vring_map.iova;
+    for (int i = 0; i < buffer_regions->len; i++) {
+        map = &g_array_index(buffer_regions, DMAMap, i);
+        vhost_iova_tree_map_alloc_gpa(u->iso_iova_tree, map,
+                                      map->translated_addr);
+    }
+
+    return 0;
+}
+
 static int vhost_user_set_mem_table(struct vhost_dev *dev,
                                     struct vhost_memory *mem)
 {
@@ -2681,6 +2809,7 @@ static int vhost_user_backend_cleanup(struct vhost_dev *dev)
     g_free(u->region_rb_offset);
     u->region_rb_offset = NULL;
     u->region_rb_len = 0;
+    cleanup_isolation_regions(dev);
     g_free(u);
     dev->opaque = 0;
 

-- 
2.43.0

[PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Connor Kite]

Adds features to fill a vhost_user_set_mem_table message with the
addresses of isolation memory regions corresponding to bounce buffers
and vrings.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-user.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 79 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 710cf966f8..acabfb7f1c 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -625,6 +625,21 @@ static void vhost_user_fill_msg_region(struct vhost_dev *dev,
     dst->mmap_offset = mmap_offset;
 }
 
+static void vhost_user_fill_msg_region_iso(VhostUserMemoryRegion *dst,
+                                           const struct vhost_user *u,
+                                           const struct vhost_memory_region
+                                           *iova_reg)
+{
+    assert(u != NULL && dst != NULL && iova_reg != NULL);
+    uint64_t offset;
+
+    offset = iova_reg->userspace_addr - u->iso_memory.base_addr;
+    dst->userspace_addr = iova_reg->userspace_addr;
+    dst->memory_size = iova_reg->memory_size;
+    dst->guest_phys_addr = iova_reg->userspace_addr;
+    dst->mmap_offset = offset;
+}
+
 static int vhost_user_fill_set_mem_table_msg(struct vhost_user *u,
                                              struct vhost_dev *dev,
                                              VhostUserMsg *msg,
@@ -1136,7 +1151,33 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
     }
 }
 
-__attribute__((unused))
+struct iova_tree_traversal_args {
+    VhostUserMsg *msg;
+    struct vhost_user *u;
+    int *fds;
+    size_t *fd_num;
+};
+
+static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
+                                                    gpointer value,
+                                                    gpointer data)
+{
+    struct iova_tree_traversal_args *args = data;
+    struct vhost_memory_region msg_region;
+    VhostUserMemoryRegion region_buffer;
+    DMAMap *map = key;
+    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;
+
+    msg_region.guest_phys_addr = map->iova;
+    msg_region.memory_size = map->size + 1;
+    msg_region.userspace_addr = map->iova;
+    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
+    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;
+    (*args->fd_num)++;
+
+    return false;
+}
+
 static int init_isolation_regions(struct vhost_dev *dev,
                                   VhostUserMsg *msg,
                                   int *fds, size_t *fd_num)
@@ -1234,6 +1275,24 @@ static int init_isolation_regions(struct vhost_dev *dev,
                                       map->translated_addr);
     }
 
+    struct iova_tree_traversal_args args = {
+        .fd_num = fd_num,
+        .fds = fds,
+        .msg = msg,
+        .u = u
+    };
+
+    vhost_iova_tree_foreach(u->iso_iova_tree,
+                            vhost_user_iova_tree_traverse_funct, &args);
+
+    msg->payload.memory.nregions = *fd_num;
+
+    assert(*fd_num != 0);
+
+    msg->hdr.size = sizeof(msg->payload.memory.nregions);
+    msg->hdr.size += sizeof(msg->payload.memory.padding);
+    msg->hdr.size += *fd_num * sizeof(VhostUserMemoryRegion);
+
     return 0;
 }
 
@@ -1241,6 +1300,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
                                     struct vhost_memory *mem)
 {
     struct vhost_user *u = dev->opaque;
+    bool memory_isolation = u->user->memory_isolation;
     int fds[VHOST_MEMORY_BASELINE_NREGIONS];
     size_t fd_num = 0;
     bool do_postcopy = u->postcopy_listen && u->postcopy_fd.handler;
@@ -1268,6 +1328,24 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
         msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
     }
 
+    if (memory_isolation) {
+        ret = init_isolation_regions(dev, &msg, fds, &fd_num);
+        if (ret < 0) {
+            return ret;
+        }
+
+        ret = vhost_user_write(dev, &msg, fds, fd_num);
+        if (ret < 0) {
+            return ret;
+        }
+
+        if (reply_supported) {
+            return process_message_reply(dev, &msg);
+        }
+
+        return 0;
+    }
+
     if (config_mem_slots) {
         ret = vhost_user_add_remove_regions(dev, &msg, reply_supported, false);
         if (ret < 0) {

-- 
2.43.0

[PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Connor Kite]

Adds shadow virtqueues that will eventually be used to transfer data
between device and host via bounce buffers when isolation mode is
active.  The svqs are initalized, and eventfd assignments are
intercepted so that notifications come to svqs first before
the guest or backend receive them.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-user.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index acabfb7f1c..75858289a2 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -18,6 +18,7 @@
 #include "hw/virtio/vhost-backend.h"
 #include "hw/virtio/virtio.h"
 #include "hw/virtio/virtio-net.h"
+#include "hw/virtio/vhost-shadow-virtqueue.h"
 #include "hw/virtio/vhost-iova-tree.h"
 #include "chardev/char-fe.h"
 #include "io/channel-socket.h"
@@ -365,7 +366,9 @@ struct vhost_user {
 
     /* Isolated memory data*/
     struct IsolationRegion iso_memory;
+    GPtrArray *shadow_vqs;
     VhostIOVATree *iso_iova_tree;
+    bool svqs_allocated;
 };
 
 struct scrub_regions {
@@ -1693,6 +1696,24 @@ static int vhost_set_vring_file(struct vhost_dev *dev,
 static int vhost_user_set_vring_kick(struct vhost_dev *dev,
                                      struct vhost_vring_file *file)
 {
+    struct vhost_user *u = dev->opaque;
+    int svq_idx = file->index - dev->vq_index;
+    if (u->user->memory_isolation) {
+        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
+                                                      svq_idx);
+        vhost_svq_set_svq_kick_fd(svq, file->fd);
+
+        if (svq->hdev_kick.initialized == false) {
+            int r = event_notifier_init(&svq->hdev_kick, 0);
+            if (r) {
+                error_report("Failed to create kick event notifier");
+                return r;
+            }
+        }
+
+        file->fd = event_notifier_get_fd(&svq->hdev_kick);
+    }
+
     int ret = vhost_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
     if (ret < 0) {
         return ret;
@@ -1721,6 +1742,24 @@ static int vhost_user_set_vring_kick(struct vhost_dev *dev,
 static int vhost_user_set_vring_call(struct vhost_dev *dev,
                                      struct vhost_vring_file *file)
 {
+    struct vhost_user *u = dev->opaque;
+    int svq_idx = file->index - dev->vq_index;
+    if (u->user->memory_isolation) {
+        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
+                                                      svq_idx);
+        vhost_svq_set_svq_call_fd(svq, file->fd);
+
+        if (svq->hdev_call.initialized == false) {
+            int r = event_notifier_init(&svq->hdev_call, 0);
+            if (r) {
+                error_report("Failed to create call event notifier");
+                return r;
+            }
+        }
+
+        file->fd = event_notifier_get_fd(&svq->hdev_call);
+    }
+
     return vhost_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
 }
 
@@ -2715,6 +2754,17 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
     return 0;
 }
 
+static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
+{
+    /*Modified from vhost-vdpa*/
+    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
+    for (int i = 0; i < dev->nvqs; i++) {
+        VhostShadowVirtqueue *svq;
+        svq = vhost_svq_new(NULL, NULL);
+        g_ptr_array_add(u->shadow_vqs, svq);
+    }
+}
+
 static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
                                    Error **errp)
 {
@@ -2859,6 +2909,10 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
     u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
     postcopy_add_notifier(&u->postcopy_notifier);
 
+    if (vus->memory_isolation) {
+        vhost_user_init_svq(dev, u);
+    }
+
     return 0;
 }
 

-- 
2.43.0

[PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Connor Kite]

- Add start logic for shadow virtqueues, which sets vring addresses.
- Update logic for sending vring addresses to backend to point
  to the shadow vrings when isolation mode is active.
- Implement handlers for intercepted avail and used descriptors. These
  handlers copy buffer contents between bounce buffers in the isolation
  region and the buffers made available by the guest
- Implement logic to stop svqs

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-user.c | 167 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 166 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 75858289a2..e65f877f9a 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1299,6 +1299,97 @@ static int init_isolation_regions(struct vhost_dev *dev,
     return 0;
 }
 
+static int vhost_user_memory_lookup(struct vhost_dev *dev, hwaddr gpa,
+                                    hwaddr *hva)
+{
+    int i;
+    hwaddr offset;
+
+    for (i = 0; i < dev->mem->nregions; i++) {
+        struct vhost_memory_region *reg = dev->mem->regions + i;
+
+        if (gpa >= reg->guest_phys_addr &&
+            reg->guest_phys_addr + reg->memory_size > gpa) {
+            offset = gpa - reg->guest_phys_addr;
+            *hva = reg->userspace_addr + offset;
+            return 0;
+        }
+    }
+
+    return -EFAULT;
+}
+
+static int vhost_user_svq_handle_used(VhostShadowVirtqueue *svq,
+                                      VirtQueueElement *elem,
+                                      void *opaque)
+{
+    hwaddr hva;
+    int r;
+    struct vhost_dev *dev = opaque;
+
+    for (int i = 0; i < elem->in_num; i++) {
+        r = vhost_user_memory_lookup(dev, elem->in_addr[i], &hva);
+        if (r < 0) {
+            return r;
+        }
+
+        memcpy((void *) hva, elem->in_sg[i].iov_base, elem->in_sg[i].iov_len);
+        elem->in_sg[i].iov_base = (void *) hva;
+    }
+
+    for (int i = 0; i < elem->out_num; i++) {
+        r = vhost_user_memory_lookup(dev, elem->out_addr[i], &hva);
+        if (r < 0) {
+            return r;
+        }
+
+        memcpy((void *) hva, elem->out_sg[i].iov_base, elem->out_sg[i].iov_len);
+        elem->out_sg[i].iov_base = (void *) hva;
+    }
+
+    return 0;
+}
+
+static int vhost_user_svq_handle_avail(VhostShadowVirtqueue *svq,
+                                       VirtQueueElement *elem,
+                                       void *opaque)
+{
+    hwaddr offset;
+    const DMAMap *map;
+    DMAMap needle;
+    hwaddr *iova_base;
+
+    for (int i = 0; i < elem->out_num; i++) {
+        needle.translated_addr = elem->out_addr[i];
+        needle.size = elem->out_sg[i].iov_len - 1;
+        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
+        offset = needle.translated_addr - map->translated_addr;
+        iova_base = (void *)(map->iova + offset);
+
+        elem->out_sg[i].iov_base = iova_base;
+        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);
+    }
+
+    for (int i = 0; i < elem->in_num; i++) {
+        needle.translated_addr = elem->in_addr[i];
+        needle.size = elem->in_sg[i].iov_len - 1;
+        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
+        offset = needle.translated_addr - map->translated_addr;
+        iova_base = (void *)(map->iova + offset);
+
+        elem->in_sg[i].iov_base = iova_base;
+        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);
+    }
+
+    vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->out_addr,
+                  elem->in_sg, elem->in_num, elem->in_addr, elem);
+
+    return 0;
+}
+
+
+
+
 static int vhost_user_set_mem_table(struct vhost_dev *dev,
                                     struct vhost_memory *mem)
 {
@@ -1772,6 +1863,8 @@ static int vhost_user_set_vring_err(struct vhost_dev *dev,
 static int vhost_user_set_vring_addr(struct vhost_dev *dev,
                                      struct vhost_vring_addr *addr)
 {
+    struct vhost_user *u = dev->opaque;
+
     VhostUserMsg msg = {
         .hdr.request = VHOST_USER_SET_VRING_ADDR,
         .hdr.flags = VHOST_USER_VERSION,
@@ -1779,6 +1872,25 @@ static int vhost_user_set_vring_addr(struct vhost_dev *dev,
         .hdr.size = sizeof(msg.payload.addr),
     };
 
+    if (u->user->memory_isolation) {
+        if (!u->svqs_allocated) {
+            return 0;
+        }
+
+        int svq_idx = addr->index - dev->vq_index;
+        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
+                                                      svq_idx);
+
+        struct vhost_vring_addr svq_addr = {
+            .avail_user_addr = (uint64_t)(uintptr_t)svq->vring.avail,
+            .desc_user_addr = (uint64_t)(uintptr_t)svq->vring.desc,
+            .used_user_addr = (uint64_t)(uintptr_t)svq->vring.used,
+            .index = addr->index,
+        };
+
+        msg.payload.addr = svq_addr;
+    }
+
     /*
      * wait for a reply if logging is enabled to make sure
      * backend is actually logging changes
@@ -2754,13 +2866,18 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
     return 0;
 }
 
+static const VhostShadowVirtqueueOps vhost_user_svq_ops = {
+    .avail_handler = vhost_user_svq_handle_avail,
+    .used_handler = vhost_user_svq_handle_used
+};
+
 static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
 {
     /*Modified from vhost-vdpa*/
     u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
     for (int i = 0; i < dev->nvqs; i++) {
         VhostShadowVirtqueue *svq;
-        svq = vhost_svq_new(NULL, NULL);
+        svq = vhost_svq_new(&vhost_user_svq_ops, dev);
         g_ptr_array_add(u->shadow_vqs, svq);
     }
 }
@@ -3466,8 +3583,56 @@ void vhost_user_async_close(DeviceState *d,
     }
 }
 
+static bool vhost_user_svqs_start(struct vhost_dev *dev)
+{
+    struct vhost_user *u = dev->opaque;
+    uint64_t vring_base = u->iso_memory.vring_base_addr;
+    u->svqs_allocated = true;
+
+    for (int i = 0; i < u->shadow_vqs->len; i++) {
+        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
+        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
+        svq->base_addr = (hwaddr *) vring_base;
+        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);
+
+        struct vhost_vring_addr addr = {
+            .index = dev->vq_index + i,
+            .desc_user_addr = vring_base,
+            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
+                svq->vring.num,
+            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)
+        };
+
+        vhost_user_set_vring_addr(dev, &addr);
+
+        vring_base += vhost_svq_device_area_size(svq) +
+                      vhost_svq_driver_area_size(svq);
+    }
+
+    return false;
+}
+
+static void vhost_user_svqs_stop(struct vhost_dev *dev)
+{
+    struct vhost_user *u = dev->opaque;
+
+    for (int i = 0; i < u->shadow_vqs->len; i++) {
+        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
+    }
+}
+
+
 static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
 {
+    struct vhost_user *u = dev->opaque;
+    if (u->user->memory_isolation) {
+        if (started) {
+            vhost_user_svqs_start(dev);
+        } else {
+            vhost_user_svqs_stop(dev);
+        }
+    }
+
     if (!vhost_user_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_STATUS)) {
         return 0;
     }

-- 
2.43.0

[PATCH RFC 15/15] hw/virtio/vhost-user: shadow vq cleanup

[Connor Kite]

Implements functionality to cleanup svqs on cleanup of isolation
regions.

Signed-off-by: Connor Kite <connorkite@gmail.com>
---
 hw/virtio/vhost-user.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index e65f877f9a..bb4de78b95 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1140,6 +1140,15 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
     return 0;
 }
 
+static void vhost_user_svq_cleanup(struct vhost_user *u)
+{
+    for (int i = 0; i < u->shadow_vqs->len; i++) {
+        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
+    }
+
+    g_ptr_array_free(u->shadow_vqs, true);
+}
+
 /* TODO: Is there any notifier cleanup required here?*/
 static void cleanup_isolation_regions(struct vhost_dev *dev)
 {
@@ -1147,6 +1156,7 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
     if (u->iso_memory.base_addr) {
         vhost_iova_tree_delete(u->iso_iova_tree);
         u->iso_iova_tree = NULL;
+        vhost_user_svq_cleanup(u);
         memset(&u->iso_memory, 0, sizeof(IsolationRegion));
         qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
                          u->iso_memory.iso_fd);

-- 
2.43.0

Re: [PATCH RFC 03/15] backends/cryptodev-vhost-user: add memory isolation bool

[Markus Armbruster]

Connor Kite <connorkite@gmail.com> writes:

> Add a memory_isolation property to CryptoDevBackendVhostUser and
> add add it as an optional member of CryptodevVhostUserProperties
> in qapi.
>
> This is needed to be able to specify memory isolation for
> cryptodev-vhost-user objects, and the bool will eventually be
> passed to vhost_user_init() in a future patch.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>

[...]

> diff --git a/qapi/qom.json b/qapi/qom.json
> index c55776af7d..d7ca1b4203 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -350,11 +350,15 @@
>  # @chardev: the name of a Unix domain socket character device that
>  #     connects to the vhost-user server
>  #
> +# @memory-isolation: disables access from cryptodev to guest memory.
> +#     (default: false)

Want imperative mood and need "since", like this:

   # @memory-isolation: disable access from cryptodev to guest memory.
   #     (default: false) (since 11.2)

Pardon my ignorance...  What exactly is "cryptodev" here?  Is it code
running in another process?

Should the description answer my question?  Perhaps like this:

  # @memory-isolation: isolate guest memory.  Isolated guest memory
  #     cannot be accessed from uh, what exactly?

Fill in the blank.

> +#
>  # Since: 2.12
>  ##
>  { 'struct': 'CryptodevVhostUserProperties',
>    'base': 'CryptodevBackendProperties',
> -  'data': { 'chardev': 'str' },
> +  'data': { 'chardev': 'str',
> +            '*memory-isolation': 'bool' },
>    'if': 'CONFIG_VHOST_CRYPTO' }
>  
>  ##

Re: [PATCH RFC 01/15] vhost-user: Consolidate chardev property definitions

[Markus Armbruster]

This patch and the next one are From: ConKite, whereas the remainder are
From: Connor Kite, which is nicer.

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Akihiko Odaki]

On 2026/07/24 7:30, ConKite wrote:
> In support of future work to add a memory isolation mode to vhost-user devices, a
> qdev property, "memory-isolation", is added to all vhost-user devices. This is
> simply a bool representing whether the mode is active or not.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

This and the next patch add interfaces before implementation. Perhaps 
it's better reversed.

Regards,
Akihiko Odaki

> ---
>   hw/block/vhost-user-blk.c            | 1 +
>   hw/display/vhost-user-gpu.c          | 1 +
>   hw/scsi/vhost-user-scsi.c            | 1 +
>   hw/virtio/vhost-user-base.c          | 3 ++-
>   hw/virtio/vhost-user-fs.c            | 1 +
>   hw/virtio/vhost-user-scmi.c          | 1 +
>   hw/virtio/vhost-user-vsock.c         | 1 +
>   include/hw/virtio/vhost-user-base.h  | 1 +
>   include/hw/virtio/vhost-user-blk.h   | 1 +
>   include/hw/virtio/vhost-user-fs.h    | 1 +
>   include/hw/virtio/vhost-user-scmi.h  | 1 +
>   include/hw/virtio/vhost-user-vsock.h | 1 +
>   include/hw/virtio/virtio-gpu.h       | 1 +
>   include/hw/virtio/virtio-scsi.h      | 1 +
>   14 files changed, 15 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index 2e5b3ae1b1..9830dc6c35 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -631,6 +631,7 @@ static const VMStateDescription vmstate_vhost_user_blk = {
>   
>   static const Property vhost_user_blk_properties[] = {
>       DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserBlk, memory_isolation, false),
>       DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues,
>                          VHOST_USER_BLK_AUTO_NUM_QUEUES),
>       DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128),
> diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
> index 57360898ca..6d0ede672a 100644
> --- a/hw/display/vhost-user-gpu.c
> +++ b/hw/display/vhost-user-gpu.c
> @@ -659,6 +659,7 @@ static struct vhost_dev *vhost_user_gpu_get_vhost(VirtIODevice *vdev)
>   
>   static const Property vhost_user_gpu_properties[] = {
>       VIRTIO_GPU_BASE_PROPERTIES(VhostUserGPU, parent_obj.conf),
> +    DEFINE_PROP_BOOL("memory-isolation", VhostUserGPU, memory_isolation, false),
>   };
>   
>   static void
> diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
> index 5070178dc2..c1a0393220 100644
> --- a/hw/scsi/vhost-user-scsi.c
> +++ b/hw/scsi/vhost-user-scsi.c
> @@ -346,6 +346,7 @@ static void vhost_user_scsi_unrealize(DeviceState *dev)
>   
>   static const Property vhost_user_scsi_properties[] = {
>       DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VirtIOSCSICommon, conf.memory_isolation, false),
>       DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0),
>       DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues,
>                          VIRTIO_SCSI_AUTO_NUM_QUEUES),
> diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
> index 90bd2903db..d2a74a2860 100644
> --- a/hw/virtio/vhost-user-base.c
> +++ b/hw/virtio/vhost-user-base.c
> @@ -409,7 +409,8 @@ static void vub_device_unrealize(DeviceState *dev)
>   
>   /*Define common qdev properties.  Inherited by all children*/
>   static const Property vub_properties[] = {
> -    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
> +    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserBase, memory_isolation, false)
>   };
>   
>   
> diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
> index 209993918a..3d980d814b 100644
> --- a/hw/virtio/vhost-user-fs.c
> +++ b/hw/virtio/vhost-user-fs.c
> @@ -410,6 +410,7 @@ static const VMStateDescription vuf_backend_vmstate = {
>   
>   static const Property vuf_properties[] = {
>       DEFINE_PROP_CHR("chardev", VHostUserFS, conf.chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserFS, conf.memory_isolation, false),
>       DEFINE_PROP_STRING("tag", VHostUserFS, conf.tag),
>       DEFINE_PROP_UINT16("num-request-queues", VHostUserFS,
>                          conf.num_request_queues, 1),
> diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
> index 02dc088ea9..d514d89392 100644
> --- a/hw/virtio/vhost-user-scmi.c
> +++ b/hw/virtio/vhost-user-scmi.c
> @@ -287,6 +287,7 @@ static const VMStateDescription vu_scmi_vmstate = {
>   
>   static const Property vu_scmi_properties[] = {
>       DEFINE_PROP_CHR("chardev", VHostUserSCMI, chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserSCMI, memory_isolation, false),
>   };
>   
>   static void vu_scmi_class_init(ObjectClass *klass, const void *data)
> diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
> index c2cd376e73..89aa789e22 100644
> --- a/hw/virtio/vhost-user-vsock.c
> +++ b/hw/virtio/vhost-user-vsock.c
> @@ -154,6 +154,7 @@ static void vuv_device_unrealize(DeviceState *dev)
>   
>   static const Property vuv_properties[] = {
>       DEFINE_PROP_CHR("chardev", VHostUserVSock, conf.chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserVSock, conf.memory_isolation, false),
>   };
>   
>   static void vuv_class_init(ObjectClass *klass, const void *data)
> diff --git a/include/hw/virtio/vhost-user-base.h b/include/hw/virtio/vhost-user-base.h
> index d5d2f081ae..29bc40736c 100644
> --- a/include/hw/virtio/vhost-user-base.h
> +++ b/include/hw/virtio/vhost-user-base.h
> @@ -25,6 +25,7 @@ struct VHostUserBase {
>       uint32_t num_vqs;
>       uint32_t vq_size; /* can't exceed VIRTIO_QUEUE_MAX */
>       uint32_t config_size;
> +    bool memory_isolation;
>       /* State tracking */
>       VhostUserState vhost_user;
>       struct vhost_virtqueue *vhost_vq;
> diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h
> index dee848cfd8..8126447a86 100644
> --- a/include/hw/virtio/vhost-user-blk.h
> +++ b/include/hw/virtio/vhost-user-blk.h
> @@ -30,6 +30,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserBlk, VHOST_USER_BLK)
>   struct VHostUserBlk {
>       VirtIODevice parent_obj;
>       CharFrontend chardev;
> +    bool memory_isolation;
>       int32_t bootindex;
>       struct virtio_blk_config blkcfg;
>       uint16_t num_queues;
> diff --git a/include/hw/virtio/vhost-user-fs.h b/include/hw/virtio/vhost-user-fs.h
> index 5699ea03a1..bf71990b9c 100644
> --- a/include/hw/virtio/vhost-user-fs.h
> +++ b/include/hw/virtio/vhost-user-fs.h
> @@ -28,6 +28,7 @@ typedef struct {
>       char *tag;
>       uint16_t num_request_queues;
>       uint16_t queue_size;
> +    bool memory_isolation;
>   } VHostUserFSConf;
>   
>   struct VHostUserFS {
> diff --git a/include/hw/virtio/vhost-user-scmi.h b/include/hw/virtio/vhost-user-scmi.h
> index 30ab0d7ab7..d346b42ed8 100644
> --- a/include/hw/virtio/vhost-user-scmi.h
> +++ b/include/hw/virtio/vhost-user-scmi.h
> @@ -19,6 +19,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserSCMI, VHOST_USER_SCMI);
>   struct VHostUserSCMI {
>       VirtIODevice parent;
>       CharFrontend chardev;
> +    bool memory_isolation;
>       struct vhost_virtqueue *vhost_vqs;
>       struct vhost_dev vhost_dev;
>       VhostUserState vhost_user;
> diff --git a/include/hw/virtio/vhost-user-vsock.h b/include/hw/virtio/vhost-user-vsock.h
> index 5db6938b8b..6288b26fa4 100644
> --- a/include/hw/virtio/vhost-user-vsock.h
> +++ b/include/hw/virtio/vhost-user-vsock.h
> @@ -21,6 +21,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserVSock, VHOST_USER_VSOCK)
>   
>   typedef struct {
>       CharFrontend chardev;
> +    bool memory_isolation;
>   } VHostUserVSockConf;
>   
>   struct VHostUserVSock {
> diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
> index b9bad27c97..707612c3a4 100644
> --- a/include/hw/virtio/virtio-gpu.h
> +++ b/include/hw/virtio/virtio-gpu.h
> @@ -277,6 +277,7 @@ struct VhostUserGPU {
>       CharFrontend vhost_chr;
>       QemuDmaBuf *dmabuf[VIRTIO_GPU_MAX_SCANOUTS];
>       bool backend_blocked;
> +    bool memory_isolation;
>   };
>   
>   #define MAX_SLOTS 4096
> diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h
> index b6028bb5cd..47fdc3785c 100644
> --- a/include/hw/virtio/virtio-scsi.h
> +++ b/include/hw/virtio/virtio-scsi.h
> @@ -59,6 +59,7 @@ struct VirtIOSCSIConf {
>       char *vhostfd;
>       char *wwpn;
>       CharFrontend chardev;
> +    bool memory_isolation;
>       uint32_t boot_tpgt;
>       IOThread *iothread;
>       IOThreadVirtQueueMappingList *iothread_vq_mapping_list;
> 

Re: [PATCH RFC 03/15] backends/cryptodev-vhost-user: add memory isolation bool

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Add a memory_isolation property to CryptoDevBackendVhostUser and
> add add it as an optional member of CryptodevVhostUserProperties
> in qapi.
> 
> This is needed to be able to specify memory isolation for
> cryptodev-vhost-user objects, and the bool will eventually be
> passed to vhost_user_init() in a future patch.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   backends/cryptodev-vhost-user.c | 22 ++++++++++++++++++++++
>   qapi/qom.json                   |  6 +++++-
>   2 files changed, 27 insertions(+), 1 deletion(-)
> 
> diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c
> index cc478d9902..e0547c5d40 100644
> --- a/backends/cryptodev-vhost-user.c
> +++ b/backends/cryptodev-vhost-user.c
> @@ -49,6 +49,7 @@ struct CryptoDevBackendVhostUser {
>       CharFrontend chr;
>       char *chr_name;
>       bool opened;
> +    bool memory_isolation;
>       CryptoDevBackendVhost *vhost_crypto[MAX_CRYPTO_QUEUE_NUM];
>   };
>   
> @@ -392,6 +393,23 @@ static void cryptodev_vhost_user_finalize(Object *obj)
>       g_free(s->chr_name);
>   }
>   
> +static void cryptodev_vhost_user_set_mem_isolation(Object *obj, bool value,
> +                                                   Error **errp)
> +{
> +    CryptoDevBackendVhostUser *s =
> +                      CRYPTODEV_BACKEND_VHOST_USER(obj);
> +
> +    s->memory_isolation = value;

This setter should forbid setting once initialization finishes as done 
in cryptodev_vhost_user_set_chardev() since the value is copied into 
VhostUserState during initialization and setting this afterwards is 
ineffective.

Regards,
Akihiko Odaki

> +}
> +
> +static bool cryptodev_vhost_user_get_mem_isolation(Object *obj, Error **errp)
> +{
> +    CryptoDevBackendVhostUser *s =
> +                      CRYPTODEV_BACKEND_VHOST_USER(obj);
> +
> +    return s->memory_isolation;
> +}
> +
>   static void
>   cryptodev_vhost_user_class_init(ObjectClass *oc, const void *data)
>   {
> @@ -407,6 +425,10 @@ cryptodev_vhost_user_class_init(ObjectClass *oc, const void *data)
>                                     cryptodev_vhost_user_get_chardev,
>                                     cryptodev_vhost_user_set_chardev);
>   
> +    object_class_property_add_bool(oc, "memory-isolation",
> +                                   cryptodev_vhost_user_get_mem_isolation,
> +                                   cryptodev_vhost_user_set_mem_isolation);
> +
>   }
>   
>   static const TypeInfo cryptodev_vhost_user_info = {
> diff --git a/qapi/qom.json b/qapi/qom.json
> index c55776af7d..d7ca1b4203 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -350,11 +350,15 @@
>   # @chardev: the name of a Unix domain socket character device that
>   #     connects to the vhost-user server
>   #
> +# @memory-isolation: disables access from cryptodev to guest memory.
> +#     (default: false)
> +#
>   # Since: 2.12
>   ##
>   { 'struct': 'CryptodevVhostUserProperties',
>     'base': 'CryptodevBackendProperties',
> -  'data': { 'chardev': 'str' },
> +  'data': { 'chardev': 'str',
> +            '*memory-isolation': 'bool' },
>     'if': 'CONFIG_VHOST_CRYPTO' }
>   
>   ##
> 

Re: [PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Add a memory_isolation bool to the VhostUserState struct. This
> bool is set within vhost_user_init(), which takes a
> memory_isolation bool as an argument.
> 
> Refactor all call locations of vhost_user_init to include the new
> argument.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   backends/cryptodev-vhost-user.c     |  2 +-
>   backends/vhost-user.c               |  4 ++--
>   hw/block/vhost-user-blk.c           |  3 ++-
>   hw/display/vhost-user-gpu.c         |  3 ++-
>   hw/scsi/vhost-user-scsi.c           |  3 ++-
>   hw/virtio/vhost-stub.c              |  3 ++-
>   hw/virtio/vhost-user-base.c         |  3 ++-
>   hw/virtio/vhost-user-fs.c           |  3 ++-
>   hw/virtio/vhost-user-scmi.c         |  3 ++-
>   hw/virtio/vhost-user-vsock.c        |  3 ++-
>   hw/virtio/vhost-user.c              |  5 ++++-
>   include/hw/virtio/vhost-user.h      |  6 +++++-
>   include/system/vhost-user-backend.h |  3 ++-
>   net/passt.c                         | 13 ++++++++-----
>   net/vhost-user.c                    |  9 +++++----
>   15 files changed, 43 insertions(+), 23 deletions(-)
> 
> diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c
> index e0547c5d40..328d7fbdfe 100644
> --- a/backends/cryptodev-vhost-user.c
> +++ b/backends/cryptodev-vhost-user.c
> @@ -214,7 +214,7 @@ static void cryptodev_vhost_user_init(
>           }
>       }
>   
> -    if (!vhost_user_init(&s->vhost_user, &s->chr, errp)) {
> +    if (!vhost_user_init(&s->vhost_user, &s->chr, s->memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/backends/vhost-user.c b/backends/vhost-user.c
> index 380d825023..470eba1775 100644
> --- a/backends/vhost-user.c
> +++ b/backends/vhost-user.c
> @@ -22,13 +22,13 @@
>   
>   int
>   vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
> -                            unsigned nvqs, Error **errp)
> +                            unsigned nvqs, bool memory_isolation, Error **errp)
>   {
>       int ret;
>   
>       assert(!b->vdev && vdev);
>   
> -    if (!vhost_user_init(&b->vhost_user, &b->chr, errp)) {
> +    if (!vhost_user_init(&b->vhost_user, &b->chr, memory_isolation, errp)) {
>           return -1;
>       }
>   
> diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c
> index 9830dc6c35..5c1a19eb50 100644
> --- a/hw/block/vhost-user-blk.c
> +++ b/hw/block/vhost-user-blk.c
> @@ -505,7 +505,8 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp)
>           return;
>       }
>   
> -    if (!vhost_user_init(&s->vhost_user, &s->chardev, errp)) {
> +    if (!vhost_user_init(&s->vhost_user, &s->chardev,
> +                         s->memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/hw/display/vhost-user-gpu.c b/hw/display/vhost-user-gpu.c
> index 6d0ede672a..b44c989f58 100644
> --- a/hw/display/vhost-user-gpu.c
> +++ b/hw/display/vhost-user-gpu.c
> @@ -625,7 +625,8 @@ vhost_user_gpu_device_realize(DeviceState *qdev, Error **errp)
>       VirtIODevice *vdev = VIRTIO_DEVICE(g);
>   
>       vhost_dev_set_config_notifier(&g->vhost->dev, &config_ops);
> -    if (vhost_user_backend_dev_init(g->vhost, vdev, 2, errp) < 0) {
> +    if (vhost_user_backend_dev_init(g->vhost, vdev, 2, g->memory_isolation,
> +                                    errp) < 0) {
>           return;
>       }
>   
> diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c
> index c1a0393220..5a19182f2b 100644
> --- a/hw/scsi/vhost-user-scsi.c
> +++ b/hw/scsi/vhost-user-scsi.c
> @@ -276,7 +276,8 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp)
>           return;
>       }
>   
> -    if (!vhost_user_init(&s->vhost_user, &vs->conf.chardev, errp)) {
> +    if (!vhost_user_init(&s->vhost_user, &vs->conf.chardev,
> +                         vs->conf.memory_isolation, errp)) {
>           goto free_virtio;
>       }
>   
> diff --git a/hw/virtio/vhost-stub.c b/hw/virtio/vhost-stub.c
> index 7321dc9210..d1e1075a31 100644
> --- a/hw/virtio/vhost-stub.c
> +++ b/hw/virtio/vhost-stub.c
> @@ -12,7 +12,8 @@ unsigned int vhost_get_free_memslots(void)
>       return UINT_MAX;
>   }
>   
> -bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp)
> +bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
> +                     bool memory_isolation, Error **errp)
>   {
>       return false;
>   }
> diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
> index d2a74a2860..11864a3f08 100644
> --- a/hw/virtio/vhost-user-base.c
> +++ b/hw/virtio/vhost-user-base.c
> @@ -317,7 +317,8 @@ static void vub_device_realize(DeviceState *dev, Error **errp)
>           vub->vhost_user.supports_config = true;
>       }
>   
> -    if (!vhost_user_init(&vub->vhost_user, &vub->chardev, errp)) {
> +    if (!vhost_user_init(&vub->vhost_user, &vub->chardev,
> +                         vub->memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/hw/virtio/vhost-user-fs.c b/hw/virtio/vhost-user-fs.c
> index 3d980d814b..07ff2a35e0 100644
> --- a/hw/virtio/vhost-user-fs.c
> +++ b/hw/virtio/vhost-user-fs.c
> @@ -238,7 +238,8 @@ static void vuf_device_realize(DeviceState *dev, Error **errp)
>           return;
>       }
>   
> -    if (!vhost_user_init(&fs->vhost_user, &fs->conf.chardev, errp)) {
> +    if (!vhost_user_init(&fs->vhost_user, &fs->conf.chardev,
> +                         fs->conf.memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/hw/virtio/vhost-user-scmi.c b/hw/virtio/vhost-user-scmi.c
> index d514d89392..2c1dbb723e 100644
> --- a/hw/virtio/vhost-user-scmi.c
> +++ b/hw/virtio/vhost-user-scmi.c
> @@ -244,7 +244,8 @@ static void vu_scmi_device_realize(DeviceState *dev, Error **errp)
>   
>       vdev->host_features |= (1ULL << VIRTIO_SCMI_F_P2A_CHANNELS);
>   
> -    if (!vhost_user_init(&scmi->vhost_user, &scmi->chardev, errp)) {
> +    if (!vhost_user_init(&scmi->vhost_user, &scmi->chardev,
> +                         scmi->memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/hw/virtio/vhost-user-vsock.c b/hw/virtio/vhost-user-vsock.c
> index 89aa789e22..33ab3b69e6 100644
> --- a/hw/virtio/vhost-user-vsock.c
> +++ b/hw/virtio/vhost-user-vsock.c
> @@ -106,7 +106,8 @@ static void vuv_device_realize(DeviceState *dev, Error **errp)
>           return;
>       }
>   
> -    if (!vhost_user_init(&vsock->vhost_user, &vsock->conf.chardev, errp)) {
> +    if (!vhost_user_init(&vsock->vhost_user, &vsock->conf.chardev,
> +                         vsock->conf.memory_isolation, errp)) {
>           return;
>       }
>   
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 517cc4ca71..f296b63fb9 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -3118,13 +3118,16 @@ static void vhost_user_state_destroy(gpointer data)
>       vhost_user_host_notifier_remove(n, NULL, true);
>   }
>   
> -bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp)
> +bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
> +                     bool memory_isolation,
> +                     Error **errp)
>   {
>       if (user->chr) {
>           error_setg(errp, "Cannot initialize vhost-user state");
>           return false;
>       }
>       user->chr = chr;
> +    user->memory_isolation = memory_isolation;
>       user->memory_slots = 0;
>       user->notifiers = g_ptr_array_new_full(VIRTIO_QUEUE_MAX / 4,
>                                              &vhost_user_state_destroy);
> diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h
> index 06c360af18..72b5787efe 100644
> --- a/include/hw/virtio/vhost-user.h
> +++ b/include/hw/virtio/vhost-user.h
> @@ -66,6 +66,7 @@ typedef struct VhostUserHostNotifier {
>    * @chr: the character backend for the socket
>    * @notifiers: GPtrArray of @VhostUserHostnotifier
>    * @memory_slots:
> + * @memory_isolation: determines whether data is shared or copied
>    */
>   typedef struct VhostUserState {
>       CharFrontend *chr;
> @@ -73,12 +74,14 @@ typedef struct VhostUserState {
>       int memory_slots;
>       bool supports_config;
>       bool supports_inflight_migration;
> +    bool memory_isolation;
>   } VhostUserState;
>   
>   /**
>    * vhost_user_init() - initialise shared vhost_user state
>    * @user: allocated area for storing shared state
>    * @chr: the chardev for the vhost socket
> + * @memory_isolation: disables device access to guest memory
>    * @errp: error handle
>    *
>    * User can either directly g_new() space for the state or embed
> @@ -87,7 +90,8 @@ typedef struct VhostUserState {
>    *
>    * Return: true on success, false on error while setting errp.
>    */
> -bool vhost_user_init(VhostUserState *user, CharFrontend *chr, Error **errp);
> +bool vhost_user_init(VhostUserState *user, CharFrontend *chr,
> +                     bool memory_isolation, Error **errp);

The signature of the stub for !defined(CONFIG_VHOST_USER) is not 
updated, breaking builds.

Regards,
Akihiko Odaki

>   
>   /**
>    * vhost_user_cleanup() - cleanup state
> diff --git a/include/system/vhost-user-backend.h b/include/system/vhost-user-backend.h
> index 3184c8e799..c23f689182 100644
> --- a/include/system/vhost-user-backend.h
> +++ b/include/system/vhost-user-backend.h
> @@ -41,7 +41,8 @@ struct VhostUserBackend {
>   };
>   
>   int vhost_user_backend_dev_init(VhostUserBackend *b, VirtIODevice *vdev,
> -                                unsigned nvqs, Error **errp);
> +                                unsigned nvqs, bool memory_isolation,
> +                                Error **errp);
>   void vhost_user_backend_start(VhostUserBackend *b);
>   int vhost_user_backend_stop(VhostUserBackend *b);
>   
> diff --git a/net/passt.c b/net/passt.c
> index ce80186883..a3021f37ef 100644
> --- a/net/passt.c
> +++ b/net/passt.c
> @@ -433,7 +433,9 @@ static void passt_vhost_user_event(void *opaque, QEMUChrEvent event)
>       }
>   }
>   
> -static int net_passt_vhost_user_init(NetPasstState *s, Error **errp)
> +static int net_passt_vhost_user_init(NetPasstState *s,
> +                                     bool memory_isolation,
> +                                     Error **errp)
>   {
>       Chardev *chr;
>       int sv[2];
> @@ -457,7 +459,8 @@ static int net_passt_vhost_user_init(NetPasstState *s, Error **errp)
>   
>       s->vhost_user = g_new0(struct VhostUserState, 1);
>       if (!qemu_chr_fe_init(&s->vhost_chr, chr, errp) ||
> -        !vhost_user_init(s->vhost_user, &s->vhost_chr, errp)) {
> +        !vhost_user_init(s->vhost_user, &s->vhost_chr, memory_isolation,
> +                         errp)) {
>           goto err;
>       }
>   
> @@ -735,14 +738,14 @@ int net_init_passt(const Netdev *netdev, const char *name,
>       s->pidfile = pidfile;
>   
>       if (netdev->u.passt.has_vhost_user && netdev->u.passt.vhost_user) {
> -        bool memory_isolation G_GNUC_UNUSED = false;
> +        bool memory_isolation = false;
>   
>           if (netdev->u.passt.has_memory_isolation &&
> -           netdev->u.passt.memory_isolation) {
> +            netdev->u.passt.memory_isolation) {
>               memory_isolation = true;
>           }
>   
> -        if (net_passt_vhost_user_init(s, errp) == -1) {
> +        if (net_passt_vhost_user_init(s, memory_isolation, errp) == -1) {
>               qemu_del_net_client(nc);
>               return -1;
>           }
> diff --git a/net/vhost-user.c b/net/vhost-user.c
> index 8fa303f901..722ab20228 100644
> --- a/net/vhost-user.c
> +++ b/net/vhost-user.c
> @@ -370,7 +370,7 @@ static void net_vhost_user_event(void *opaque, QEMUChrEvent event)
>   
>   static int net_vhost_user_init(NetClientState *peer, const char *device,
>                                  const char *name, Chardev *chr,
> -                               int queues)
> +                               int queues, bool memory_isolation)
>   {
>       Error *err = NULL;
>       NetClientState *nc, *nc0 = NULL;
> @@ -390,7 +390,7 @@ static int net_vhost_user_init(NetClientState *peer, const char *device,
>               nc0 = nc;
>               s = DO_UPCAST(NetVhostUserState, nc, nc);
>               if (!qemu_chr_fe_init(&s->chr, chr, &err) ||
> -                !vhost_user_init(user, &s->chr, &err)) {
> +                !vhost_user_init(user, &s->chr, memory_isolation, &err)) {
>                   error_report_err(err);
>                   goto err;
>               }
> @@ -459,7 +459,7 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
>       int queues;
>       const NetdevVhostUserOptions *vhost_user_opts;
>       Chardev *chr;
> -    bool memory_isolation G_GNUC_UNUSED;
> +    bool memory_isolation;
>   
>       assert(netdev->type == NET_CLIENT_DRIVER_VHOST_USER);
>       vhost_user_opts = &netdev->u.vhost_user;
> @@ -480,5 +480,6 @@ int net_init_vhost_user(const Netdev *netdev, const char *name,
>       memory_isolation = vhost_user_opts->has_memory_isolation ?
>                          vhost_user_opts->memory_isolation : false;
>   
> -    return net_vhost_user_init(peer, "vhost_user", name, chr, queues);
> +    return net_vhost_user_init(peer, "vhost_user", name, chr, queues,
> +                               memory_isolation);
>   }
> 

Re: [PATCH RFC 07/15] hw/virtio: iova_tree_foreach wrapper

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Add a wrapper to vhost-iova-tree that calls iova_tree_foreach on the
> underlying iova_map.  This provides a means to execute some function on
> each node of the tree.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-iova-tree.c | 15 +++++++++++++++
>   hw/virtio/vhost-iova-tree.h |  3 +++
>   2 files changed, 18 insertions(+)
> 
> diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c
> index fa4147b773..d8206379bf 100644
> --- a/hw/virtio/vhost-iova-tree.c
> +++ b/hw/virtio/vhost-iova-tree.c
> @@ -189,3 +189,18 @@ void vhost_iova_tree_remove_gpa(VhostIOVATree *iova_tree, DMAMap map)
>       iova_tree_remove(iova_tree->gpa_iova_map, map);
>       iova_tree_remove(iova_tree->iova_map, map);
>   }
> +
> +
> +/**
> + * Traverse through the iova tree in sorted order
> + *
> + * @tree: The VhostIOVATree
> + * @func: Function called at every node.  A false return value ends traversal

GTraverseFunc stops traversal on true, but this says false ends traversal.

> + * @data: An argument passed to func
> + */
> +void vhost_iova_tree_foreach(VhostIOVATree *tree,
> +                             GTraverseFunc func,
> +                             gpointer data)
> +{
> +    iova_tree_foreach(tree->iova_map, func, data);
> +}
> diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h
> index 08f63b61cd..f3b7c48cb9 100644
> --- a/hw/virtio/vhost-iova-tree.h
> +++ b/hw/virtio/vhost-iova-tree.h
> @@ -29,5 +29,8 @@ const DMAMap *vhost_iova_tree_find_gpa(const VhostIOVATree *iova_tree,
>   int vhost_iova_tree_map_alloc_gpa(VhostIOVATree *iova_tree, DMAMap *map,
>                                     hwaddr taddr);
>   void vhost_iova_tree_remove_gpa(VhostIOVATree *iova_tree, DMAMap map);
> +void vhost_iova_tree_foreach(VhostIOVATree *tree, GTraverseFunc func,
> +                               gpointer data);
> +

Here is an extra blank line.

Regards,
Akihiko Odaki

>   
>   #endif
> 

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Adds a handler callback to shadow virtqueues in order to enable
> non-default processing of used vring elements from the device.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
>   hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
>   2 files changed, 18 insertions(+)
> 
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index bcb7f2ffc7..eb86c1ee37 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -669,6 +669,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
>                   break;
>               }
>   
> +            if (svq->ops && svq->ops->used_handler) {
> +                svq->ops->used_handler(svq, elem, svq->ops_opaque);

The error of the handler that may have left elem stale is ignored here.

Regards,
Akihiko Odaki

> +            }
> +
>               if (unlikely(i >= svq->vring.num)) {
>                   qemu_log_mask(LOG_GUEST_ERROR,
>                            "More than %u used buffers obtained in a %u size SVQ",
> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
> index ec16a1e838..ccfeee36d7 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.h
> +++ b/hw/virtio/vhost-shadow-virtqueue.h
> @@ -56,8 +56,22 @@ typedef int (*VirtQueueAvailCallback)(VhostShadowVirtqueue *svq,
>                                         VirtQueueElement *elem,
>                                         void *vq_callback_opaque);
>   
> +/**
> + * Callback to handle a used buffer
> + *
> + * @svq: Shadow virtqueue
> + * @elem: Element placed in the queue by the device
> + * @vq_callback_opaque: Used to pass arguments to callback
> + *
> + * Returns 0 if the vq is running as expected
> + */
> +typedef int (*VirtQueueUsedCallback)(VhostShadowVirtqueue *svq,
> +                                      VirtQueueElement *elem,
> +                                      void *vq_callback_opaque);
> +
>   typedef struct VhostShadowVirtqueueOps {
>       VirtQueueAvailCallback avail_handler;
> +    VirtQueueUsedCallback used_handler;
>   } VhostShadowVirtqueueOps;
>   
>   /* Shadow virtqueue to relay notifications */
> 

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> By default svq vrings are placed in an anonymous memory map. As svqs
> will be leveraged to enable memory isolation in vhost-user, it is useful
> to be able to place the vrings in a shared isolation memory region.
> 
> Adds the option to specify vring placement by providing a vring base
> address before starting the svq.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c | 22 +++++++++++++++-------
>   hw/virtio/vhost-shadow-virtqueue.h |  3 +++
>   2 files changed, 18 insertions(+), 7 deletions(-)
> 
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index eb86c1ee37..9e3c359f50 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -857,14 +857,21 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
>   
>       svq->vring.num = virtio_queue_get_num(vdev, virtio_get_queue_index(vq));
>       svq->num_free = svq->vring.num;
> -    svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
> -                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> -                           -1, 0);
>       desc_size = sizeof(vring_desc_t) * svq->vring.num;
> -    svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> -    svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
> -                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> -                           -1, 0);
> +    if (svq->base_addr == NULL) {
> +        svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
> +                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> +                            -1, 0);
> +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> +        svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
> +                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> +                            -1, 0);
> +    } else {
> +        svq->vring.desc = (void *) svq->base_addr;

Nit: please remove the whitespace between (void *) and svq->base_addr.

> +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> +        svq->vring.used = (void *)((char *)svq->base_addr +
> +                          vhost_svq_driver_area_size(svq));
> +    }

Unmapping svq here can lead to use-after-unmapping because the ring is 
not stopped yet. do_vhost_dev_stop() calls do_vhost_virtqueue_stop() 
only after calling hdev->vhost_ops->vhost_dev_start(hdev, false).

>       svq->desc_state = g_new0(SVQDescState, svq->vring.num);
>       if (virtio_vdev_has_feature(svq->vdev, VIRTIO_F_IN_ORDER)) {
>           svq->batch_last.id = VIRTIO_RING_NOT_IN_BATCH;
> @@ -929,6 +936,7 @@ VhostShadowVirtqueue *vhost_svq_new(const VhostShadowVirtqueueOps *ops,
>       event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND);
>       svq->ops = ops;
>       svq->ops_opaque = ops_opaque;
> +    svq->base_addr = NULL;
>       return svq;
>   }
>   
> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
> index ccfeee36d7..39f69e6455 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.h
> +++ b/hw/virtio/vhost-shadow-virtqueue.h
> @@ -148,6 +148,9 @@ typedef struct VhostShadowVirtqueue {
>   
>       /* Size of SVQ vring free descriptors */
>       uint16_t num_free;
> +
> +    /* Location assigned to vrings if not in default anon memory map*/

Nit: here please add a whitespace before */

Regards,
Akihiko Odaki

> +    hwaddr *base_addr;
>   } VhostShadowVirtqueue;
>   
>   bool vhost_svq_valid_features(uint64_t features, Error **errp);
> 

Re: [PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> iova-tree expects inclusive range sizing when maps are allocated or searched.
> Currently, svqs use exclusive sizing when searching their
> vhost-iova-tree for a match to the region to be translated.  This could
> lead to errors if the region to be translated is at the edge of an iova
> region.
> 
> Fix this by reducing `needle.size` by 1 in
> vhost_svq_translate_addr to bring then it line with DMAMap and iova-tree
> convention.

I think this patch is better to be extracted from this series since this 
series is complex and this patch is worth being landed earlier. You can 
chain this series to the patch with the Based-on: described in:
docs/devel/submitting-a-patch.rst

Regards,
Akihiko Odaki

> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index 9e3c359f50..20e5c7d2f1 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -104,14 +104,14 @@ static bool vhost_svq_translate_addr(const VhostShadowVirtqueue *svq,
>               /* Search the GPA->IOVA tree */
>               needle = (DMAMap) {
>                   .translated_addr = gpas[i],
> -                .size = iovec[i].iov_len,
> +                .size = iovec[i].iov_len - 1,  /* Inclusive */
>               };
>               map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
>           } else {
>               /* Search the IOVA->HVA tree */
>               needle = (DMAMap) {
>                   .translated_addr = (hwaddr)(uintptr_t)iovec[i].iov_base,
> -                .size = iovec[i].iov_len,
> +                .size = iovec[i].iov_len - 1, /* Inclusive */
>               };
>               map = vhost_iova_tree_find_iova(svq->iova_tree, &needle);
>           }
> 

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> If memory isolation mode is active for the vhost-user device adds
> features to:
> - Gather the size required for bounce buffers and vrings in shared
>    isolation region
> - Allocate the required space in an anonymous file
> - Create a vhost-iova-tree with space to map entire isolation region
> - Map guest memory regions and shared vrings into the tree
> - Release these resources upon backend cleanup
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 129 insertions(+)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index f296b63fb9..710cf966f8 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>   #include "hw/virtio/vhost-backend.h"
>   #include "hw/virtio/virtio.h"
>   #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-iova-tree.h"
>   #include "chardev/char-fe.h"
>   #include "io/channel-socket.h"
>   #include "system/kvm.h"
> @@ -25,6 +26,7 @@
>   #include "qemu/main-loop.h"
>   #include "qemu/uuid.h"
>   #include "qemu/sockets.h"
> +#include "qemu/memfd.h"
>   #include "system/runstate.h"
>   #include "system/cryptodev.h"
>   #include "migration/postcopy-ram.h"
> @@ -320,6 +322,13 @@ static VhostUserMsg m __attribute__ ((unused));
>   /* The version of the protocol we support */
>   #define VHOST_USER_VERSION    (0x1)
>   
> +typedef struct IsolationRegion {
> +    uint64_t base_addr;
> +    uint64_t vring_base_addr;
> +    uint64_t size;
> +    int iso_fd;
> +} IsolationRegion;
> +
>   struct vhost_user {
>       struct vhost_dev *dev;
>       /* Shared between vhost devs of the same virtio device */
> @@ -353,6 +362,10 @@ struct vhost_user {
>        * by the backend (see @features).
>        */
>       uint64_t protocol_features;
> +
> +    /* Isolated memory data*/
> +    struct IsolationRegion iso_memory;
> +    VhostIOVATree *iso_iova_tree;
>   };
>   
>   struct scrub_regions {
> @@ -1109,6 +1122,121 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
>       return 0;
>   }
>   
> +/* TODO: Is there any notifier cleanup required here?*/
> +static void cleanup_isolation_regions(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    if (u->iso_memory.base_addr) {
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +        u->iso_iova_tree = NULL;
> +        memset(&u->iso_memory, 0, sizeof(IsolationRegion));
> +        qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
> +                         u->iso_memory.iso_fd);

u->iso_memory is zeroed before they are read here.

> +        u->iso_memory.base_addr = 0;
> +    }
> +}
> +
> +__attribute__((unused))
> +static int init_isolation_regions(struct vhost_dev *dev,
> +                                  VhostUserMsg *msg,
> +                                  int *fds, size_t *fd_num)
> +{
> +    Error *err = NULL;
> +    struct vhost_user *u = dev->opaque;
> +    uint32_t nregions = dev->mem->nregions;
> +    uint64_t buffer_reg_size = 0;
> +    DMAMap newEntry = {

This should be new_entry as per: docs/devel/style.rst

> +        .perm = IOMMU_RW
> +    };
> +    g_autoptr(GArray) buffer_regions =
> +        g_array_new(FALSE, TRUE, sizeof(DMAMap));
> +
> +    msg->hdr.request = VHOST_USER_SET_MEM_TABLE;
> +
> +    /* In case of reset, clear old regions*/
> +    if (u->iso_memory.base_addr != 0) {
> +        cleanup_isolation_regions(dev);
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +    }

The check of u->iso_memory.base_addr is redundant; 
cleanup_isolation_regions() performs that check.

cleanup_isolation_regions() also calls vhost_iova_tree_delete(), so the 
duplicate call of the function leads to use-after-free and double-free.

> +
> +    /* Gather information for bounce buffers to be mapped */
> +    for (int i = 0; i < nregions; i++) {
> +        struct vhost_memory_region *dev_region = &dev->mem->regions[i];
> +        hwaddr size = ROUND_UP(dev_region->memory_size,
> +                      qemu_real_host_page_size());
> +        newEntry.translated_addr = dev_region->guest_phys_addr;
> +        newEntry.size = size - 1;
> +        buffer_reg_size += size;
> +        g_array_append_val(buffer_regions, newEntry);
> +    }
> +
> +    int num;
> +    size_t desc_size;
> +    size_t avail_size;
> +    size_t driver_area_size;
> +    size_t device_area_size;
> +    size_t total_vring_size = 0;
> +    size_t total_mmap_size;
> +
> +    /* Get space required for all vrings */
> +    for (int j = 0; j < dev->nvqs; j++) {
> +        num = virtio_queue_get_num(dev->vdev, dev->vq_index + j);
> +        desc_size = sizeof(vring_desc_t) * num;
> +        avail_size = offsetof(vring_avail_t, ring[num]) +
> +                        sizeof(uint16_t);
> +        driver_area_size = ROUND_UP(desc_size + avail_size,
> +                            qemu_real_host_page_size());
> +        device_area_size = ROUND_UP(offsetof(vring_used_t, ring[num]) +
> +                                    sizeof(uint16_t),
> +                                    qemu_real_host_page_size());
> +        total_vring_size += driver_area_size + device_area_size;
> +    }
> +
> +    total_mmap_size = buffer_reg_size + total_vring_size;
> +
> +    /* Allocate and map an anonymous file to hold the isolation region */
> +    u->iso_memory.base_addr = (uint64_t) qemu_memfd_alloc("iso_r",
> +                              total_mmap_size,
> +                              F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL,
> +                              &u->iso_memory.iso_fd, &err);
> +    u->iso_memory.size = total_mmap_size;
> +
> +    if (err) {
> +        error_report_err(err);
> +        cleanup_isolation_regions(dev);
> +        return -1;
> +    }
> +
> +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
> +                                                 total_mmap_size - 1));
> +
> +    /*
> +     * Instantiates iova tree sized to map bounce buffers and vrings to the
> +     * isolation region in host va.
> +     */
> +    u->iso_iova_tree = vhost_iova_tree_new(u->iso_memory.base_addr, last_addr);
> +
> +    assert(&u->iso_memory.iso_fd >= 0);
> +    DMAMap *map;
> +    DMAMap vring_map = {
> +        .perm = IOMMU_RW,
> +        .size = total_vring_size - 1,
> +        /*vrings are allocated on tree first, so will be assigned base addr*/
> +        .translated_addr = u->iso_memory.base_addr
> +    };
> +
> +    vhost_iova_tree_map_alloc(u->iso_iova_tree, &vring_map,
> +                              vring_map.translated_addr);
> +    u->iso_memory.vring_base_addr = vring_map.iova;
> +    for (int i = 0; i < buffer_regions->len; i++) {
> +        map = &g_array_index(buffer_regions, DMAMap, i);
> +        vhost_iova_tree_map_alloc_gpa(u->iso_iova_tree, map,
> +                                      map->translated_addr);

I wonder if it is fine to dismiss return values of 
vhost_iova_tree_map_alloc() and vhost_iova_tree_map_alloc_gpa().

Regards,
Akihiko Odaki

> +    }
> +
> +    return 0;
> +}
> +
>   static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                       struct vhost_memory *mem)
>   {
> @@ -2681,6 +2809,7 @@ static int vhost_user_backend_cleanup(struct vhost_dev *dev)
>       g_free(u->region_rb_offset);
>       u->region_rb_offset = NULL;
>       u->region_rb_len = 0;
> +    cleanup_isolation_regions(dev);
>       g_free(u);
>       dev->opaque = 0;
>   
> 

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Adds features to fill a vhost_user_set_mem_table message with the
> addresses of isolation memory regions corresponding to bounce buffers
> and vrings.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 79 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 710cf966f8..acabfb7f1c 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -625,6 +625,21 @@ static void vhost_user_fill_msg_region(struct vhost_dev *dev,
>       dst->mmap_offset = mmap_offset;
>   }
>   
> +static void vhost_user_fill_msg_region_iso(VhostUserMemoryRegion *dst,
> +                                           const struct vhost_user *u,
> +                                           const struct vhost_memory_region
> +                                           *iova_reg)
> +{
> +    assert(u != NULL && dst != NULL && iova_reg != NULL);
> +    uint64_t offset;
> +
> +    offset = iova_reg->userspace_addr - u->iso_memory.base_addr;
> +    dst->userspace_addr = iova_reg->userspace_addr;
> +    dst->memory_size = iova_reg->memory_size;
> +    dst->guest_phys_addr = iova_reg->userspace_addr;
> +    dst->mmap_offset = offset;
> +}
> +
>   static int vhost_user_fill_set_mem_table_msg(struct vhost_user *u,
>                                                struct vhost_dev *dev,
>                                                VhostUserMsg *msg,
> @@ -1136,7 +1151,33 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
>       }
>   }
>   
> -__attribute__((unused))
> +struct iova_tree_traversal_args {
> +    VhostUserMsg *msg;
> +    struct vhost_user *u;
> +    int *fds;
> +    size_t *fd_num;
> +};
> +
> +static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
> +                                                    gpointer value,
> +                                                    gpointer data)

It is a bit odd naming. Functions usually do not have "funct" or similar 
suffixes.

> +{
> +    struct iova_tree_traversal_args *args = data;
> +    struct vhost_memory_region msg_region;
> +    VhostUserMemoryRegion region_buffer;
> +    DMAMap *map = key;
> +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;

This may overrun the buffer because *args->fd_num is not checked as done 
in vhost_user_fill_set_mem_table_msg() for the non-isolation case.

> +
> +    msg_region.guest_phys_addr = map->iova;
> +    msg_region.memory_size = map->size + 1;
> +    msg_region.userspace_addr = map->iova;
> +    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
> +    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;
> +    (*args->fd_num)++;
> +
> +    return false;
> +}
> +
>   static int init_isolation_regions(struct vhost_dev *dev,
>                                     VhostUserMsg *msg,
>                                     int *fds, size_t *fd_num)
> @@ -1234,6 +1275,24 @@ static int init_isolation_regions(struct vhost_dev *dev,
>                                         map->translated_addr);
>       }
>   
> +    struct iova_tree_traversal_args args = {
> +        .fd_num = fd_num,
> +        .fds = fds,
> +        .msg = msg,
> +        .u = u
> +    };
> +
> +    vhost_iova_tree_foreach(u->iso_iova_tree,
> +                            vhost_user_iova_tree_traverse_funct, &args);
> +
> +    msg->payload.memory.nregions = *fd_num;
> +
> +    assert(*fd_num != 0);
> +
> +    msg->hdr.size = sizeof(msg->payload.memory.nregions);
> +    msg->hdr.size += sizeof(msg->payload.memory.padding);
> +    msg->hdr.size += *fd_num * sizeof(VhostUserMemoryRegion);
> +
>       return 0;
>   }
>   
> @@ -1241,6 +1300,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                       struct vhost_memory *mem)
>   {
>       struct vhost_user *u = dev->opaque;
> +    bool memory_isolation = u->user->memory_isolation;
>       int fds[VHOST_MEMORY_BASELINE_NREGIONS];

This needs one more element for vring.

>       size_t fd_num = 0;
>       bool do_postcopy = u->postcopy_listen && u->postcopy_fd.handler;
> @@ -1268,6 +1328,24 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
>           msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
>       }
>   
> +    if (memory_isolation) {
> +        ret = init_isolation_regions(dev, &msg, fds, &fd_num);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +
> +        ret = vhost_user_write(dev, &msg, fds, fd_num);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +
> +        if (reply_supported) {
> +            return process_message_reply(dev, &msg);
> +        }
> +
> +        return 0;
> +    }
> +

This does not cover the postcopy path.

It also allocates memory that is never sent for vq_index != 0. The 
isolation regions need to be shared by all vqs. Please see 
vhost_user_write() for the context.

Regards,
Akihiko Odaki

>       if (config_mem_slots) {
>           ret = vhost_user_add_remove_regions(dev, &msg, reply_supported, false);
>           if (ret < 0) {
> 

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Adds shadow virtqueues that will eventually be used to transfer data
> between device and host via bounce buffers when isolation mode is
> active.  The svqs are initalized, and eventfd assignments are
> intercepted so that notifications come to svqs first before
> the guest or backend receive them.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 54 insertions(+)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index acabfb7f1c..75858289a2 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>   #include "hw/virtio/vhost-backend.h"
>   #include "hw/virtio/virtio.h"
>   #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-shadow-virtqueue.h"
>   #include "hw/virtio/vhost-iova-tree.h"
>   #include "chardev/char-fe.h"
>   #include "io/channel-socket.h"
> @@ -365,7 +366,9 @@ struct vhost_user {
>   
>       /* Isolated memory data*/
>       struct IsolationRegion iso_memory;
> +    GPtrArray *shadow_vqs;
>       VhostIOVATree *iso_iova_tree;
> +    bool svqs_allocated;
>   };
>   
>   struct scrub_regions {
> @@ -1693,6 +1696,24 @@ static int vhost_set_vring_file(struct vhost_dev *dev,
>   static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>                                        struct vhost_vring_file *file)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +        vhost_svq_set_svq_kick_fd(svq, file->fd);

This is not undone in case of errors.

> +
> +        if (svq->hdev_kick.initialized == false) {

Simpler: if (!svq->hdev_kick.initialized)

Regards,
Akihiko Odaki

> +            int r = event_notifier_init(&svq->hdev_kick, 0);
> +            if (r) {
> +                error_report("Failed to create kick event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_kick);
> +    }
> +
>       int ret = vhost_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
>       if (ret < 0) {
>           return ret;
> @@ -1721,6 +1742,24 @@ static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>   static int vhost_user_set_vring_call(struct vhost_dev *dev,
>                                        struct vhost_vring_file *file)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +        vhost_svq_set_svq_call_fd(svq, file->fd);
> +
> +        if (svq->hdev_call.initialized == false) {
> +            int r = event_notifier_init(&svq->hdev_call, 0);
> +            if (r) {
> +                error_report("Failed to create call event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_call);
> +    }
> +
>       return vhost_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
>   }
>   
> @@ -2715,6 +2754,17 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>       return 0;
>   }
>   
> +static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
> +{
> +    /*Modified from vhost-vdpa*/
> +    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
> +    for (int i = 0; i < dev->nvqs; i++) {
> +        VhostShadowVirtqueue *svq;
> +        svq = vhost_svq_new(NULL, NULL);
> +        g_ptr_array_add(u->shadow_vqs, svq);
> +    }
> +}
> +
>   static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>                                      Error **errp)
>   {
> @@ -2859,6 +2909,10 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>       u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
>       postcopy_add_notifier(&u->postcopy_notifier);
>   
> +    if (vus->memory_isolation) {
> +        vhost_user_init_svq(dev, u);
> +    }
> +
>       return 0;
>   }
>   
> 

Re: [PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> - Add start logic for shadow virtqueues, which sets vring addresses.
> - Update logic for sending vring addresses to backend to point
>    to the shadow vrings when isolation mode is active.
> - Implement handlers for intercepted avail and used descriptors. These
>    handlers copy buffer contents between bounce buffers in the isolation
>    region and the buffers made available by the guest
> - Implement logic to stop svqs
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 167 ++++++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 166 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 75858289a2..e65f877f9a 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1299,6 +1299,97 @@ static int init_isolation_regions(struct vhost_dev *dev,
>       return 0;
>   }
>   
> +static int vhost_user_memory_lookup(struct vhost_dev *dev, hwaddr gpa,
> +                                    hwaddr *hva)
> +{
> +    int i;
> +    hwaddr offset;
> +
> +    for (i = 0; i < dev->mem->nregions; i++) {
> +        struct vhost_memory_region *reg = dev->mem->regions + i;
> +
> +        if (gpa >= reg->guest_phys_addr &&
> +            reg->guest_phys_addr + reg->memory_size > gpa) {
> +            offset = gpa - reg->guest_phys_addr;
> +            *hva = reg->userspace_addr + offset;
> +            return 0;
> +        }
> +    }
> +
> +    return -EFAULT;
> +}
> +
> +static int vhost_user_svq_handle_used(VhostShadowVirtqueue *svq,
> +                                      VirtQueueElement *elem,
> +                                      void *opaque)
> +{
> +    hwaddr hva;
> +    int r;
> +    struct vhost_dev *dev = opaque;
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->in_addr[i], &hva);
> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->in_sg[i].iov_base, elem->in_sg[i].iov_len);
> +        elem->in_sg[i].iov_base = (void *) hva;
> +    }
> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->out_addr[i], &hva);
> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->out_sg[i].iov_base, elem->out_sg[i].iov_len);
> +        elem->out_sg[i].iov_base = (void *) hva;
> +    }

This copies data in the reversed direction.

> +
> +    return 0;
> +}
> +
> +static int vhost_user_svq_handle_avail(VhostShadowVirtqueue *svq,
> +                                       VirtQueueElement *elem,
> +                                       void *opaque)
> +{
> +    hwaddr offset;
> +    const DMAMap *map;
> +    DMAMap needle;
> +    hwaddr *iova_base;

Please make it void *.

> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        needle.translated_addr = elem->out_addr[i];
> +        needle.size = elem->out_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);

An error check is missing here. This translation does not seem to 
support IOMMU.

> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->out_sg[i].iov_base = iova_base;

elem->out_sg shouldn't be overwritten since virtqueue_unpop() will use 
it to unmap the element.

> +        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);

This memcpy() is no-op. elem->out_sg[i].iov_base is already set to 
iova_base, so it's copying from iova_base to iova_base.

> +    }
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        needle.translated_addr = elem->in_addr[i];
> +        needle.size = elem->in_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->in_sg[i].iov_base = iova_base;
> +        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);
> +    }
> +
> +    vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->out_addr,
> +                  elem->in_sg, elem->in_num, elem->in_addr, elem);

An error is ignored here too.

> +
> +    return 0;
> +}
> +
> +
> +
> +
>   static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                       struct vhost_memory *mem)
>   {
> @@ -1772,6 +1863,8 @@ static int vhost_user_set_vring_err(struct vhost_dev *dev,
>   static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>                                        struct vhost_vring_addr *addr)
>   {
> +    struct vhost_user *u = dev->opaque;
> +
>       VhostUserMsg msg = {
>           .hdr.request = VHOST_USER_SET_VRING_ADDR,
>           .hdr.flags = VHOST_USER_VERSION,
> @@ -1779,6 +1872,25 @@ static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>           .hdr.size = sizeof(msg.payload.addr),
>       };
>   
> +    if (u->user->memory_isolation) {
> +        if (!u->svqs_allocated) {
> +            return 0;
> +        }
> +
> +        int svq_idx = addr->index - dev->vq_index;
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +
> +        struct vhost_vring_addr svq_addr = {
> +            .avail_user_addr = (uint64_t)(uintptr_t)svq->vring.avail,
> +            .desc_user_addr = (uint64_t)(uintptr_t)svq->vring.desc,
> +            .used_user_addr = (uint64_t)(uintptr_t)svq->vring.used,
> +            .index = addr->index,
> +        };
> +
> +        msg.payload.addr = svq_addr;
> +    }
> +
>       /*
>        * wait for a reply if logging is enabled to make sure
>        * backend is actually logging changes
> @@ -2754,13 +2866,18 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>       return 0;
>   }
>   
> +static const VhostShadowVirtqueueOps vhost_user_svq_ops = {
> +    .avail_handler = vhost_user_svq_handle_avail,
> +    .used_handler = vhost_user_svq_handle_used
> +};
> +
>   static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
>   {
>       /*Modified from vhost-vdpa*/
>       u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
>       for (int i = 0; i < dev->nvqs; i++) {
>           VhostShadowVirtqueue *svq;
> -        svq = vhost_svq_new(NULL, NULL);
> +        svq = vhost_svq_new(&vhost_user_svq_ops, dev);
>           g_ptr_array_add(u->shadow_vqs, svq);
>       }
>   }
> @@ -3466,8 +3583,56 @@ void vhost_user_async_close(DeviceState *d,
>       }
>   }
>   
> +static bool vhost_user_svqs_start(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    uint64_t vring_base = u->iso_memory.vring_base_addr;
> +    u->svqs_allocated = true;
> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
> +        svq->base_addr = (hwaddr *) vring_base;
> +        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);

Calling vhost_svq_start() here overwrites the shadow vring index 
configured in vhost_virtqueue_start().

> +
> +        struct vhost_vring_addr addr = {
> +            .index = dev->vq_index + i,
> +            .desc_user_addr = vring_base,
> +            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
> +                svq->vring.num,

This assumes the ring is split, but VIRTIO_F_RING_PACKED is not rejected.

> +            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)
> +        };
> +
> +        vhost_user_set_vring_addr(dev, &addr);


The initial ring startup is ordered incorrectly. Generic startup 
suppresses SET_VRING_ADDR, installs SET_VRING_KICK, and injects a kick; 
only the later backend-start hook creates the shadow ring and sends its 
address. SET_VRING_KICK starts the ring, so the backend can process 
before receiving valid addresses. The late SET_VRING_ADDR result is also 
ignored.

Regards,
Akihiko Odaki

> +
> +        vring_base += vhost_svq_device_area_size(svq) +
> +                      vhost_svq_driver_area_size(svq);
> +    }
> +
> +    return false;
> +}
> +
> +static void vhost_user_svqs_stop(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
> +    }
> +}
> +
> +
>   static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    if (u->user->memory_isolation) {
> +        if (started) {
> +            vhost_user_svqs_start(dev);
> +        } else {
> +            vhost_user_svqs_stop(dev);
> +        }
> +    }
> +
>       if (!vhost_user_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_STATUS)) {
>           return 0;
>       }
> 

Re: [PATCH RFC 15/15] hw/virtio/vhost-user: shadow vq cleanup

[Akihiko Odaki]

On 2026/07/24 7:30, Connor Kite wrote:
> Implements functionality to cleanup svqs on cleanup of isolation
> regions.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index e65f877f9a..bb4de78b95 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1140,6 +1140,15 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
>       return 0;
>   }
>   
> +static void vhost_user_svq_cleanup(struct vhost_user *u)
> +{
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
> +    }
> +
> +    g_ptr_array_free(u->shadow_vqs, true);
> +}
> +
>   /* TODO: Is there any notifier cleanup required here?*/
>   static void cleanup_isolation_regions(struct vhost_dev *dev)
>   {
> @@ -1147,6 +1156,7 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
>       if (u->iso_memory.base_addr) {
>           vhost_iova_tree_delete(u->iso_iova_tree);
>           u->iso_iova_tree = NULL;
> +        vhost_user_svq_cleanup(u);

Memory-table refresh frees live SVQs permanently. 
init_isolation_regions() uses cleanup for refreshes, but cleanup frees 
shadow_vqs; they are created only once during backend initialization.

Regards,
Akihiko Odaki

>           memset(&u->iso_memory, 0, sizeof(IsolationRegion));
>           qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
>                            u->iso_memory.iso_fd);
> 

Re: [PATCH RFC 00/15] vhost-user: isolated memory

[Demi Marie Obenour]

[-- Attachment #1.1: Type: text/plain, Size: 1959 bytes --]

On 7/23/26 18:29, ConKite wrote:
> This patch series implements a memory isolation mode in vhost-user. The
> purpose of this mode is to provide the option of additional security by
> eliminating direct access of guest memory by vhost-user devices.
> At a high level this works by:
> 
> 1. Adding qdev and qapi properties required to enable isolation mode for
>    various devices.
> 2. Allocating an isolation memory region in an anonymous file and mapping it
>    to host memory.  This isolation region will hold the bounce buffers and
>    vrings necessary to move data.
> 3. Using a vhost-iova-tree to allocate and track the mapping between
>    guest regions and their corresponding bounce buffers in the isolation
>    memory.
> 4. Creating shadow virtqueues to intercept request notifications.  As
>    kick and call events are received by an svq, it copies buffer
>    contents and descriptors between isolation and guest memory before
>    notifying the backend or guest.   
> 
> Note: This project is currently in a partially functional state. For
> example, in testing vhost-user-input and a keyboard, keystrokes may
> register correctly or be double counted depending on the combination
> of backend daemon and guest OS used. However, the desire is to make
> this work public at this stage for comment on the overall
> approach.

Thank you so much for the work!  I'm very excited about it.

I do have a few questions:

1. Is this dependent on the specific vhost-user device type?  In other
   words, does this code need to be specific to vhost-user-blk/net/etc
   devices?

2. What kind of performance penalty is there for isolation?  The use
   of shadow virtqueues seems relatively complex.

3. Does it make sense to have device-type-specific code to validate
   data sent back by the server?  This could protect drivers that
   trust virtio devices. 
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH RFC 00/15] vhost-user: isolated memory

[Akihiko Odaki]

On 2026/07/24 7:29, ConKite wrote:
> This patch series implements a memory isolation mode in vhost-user. The
> purpose of this mode is to provide the option of additional security by
> eliminating direct access of guest memory by vhost-user devices.
> At a high level this works by:
> 
> 1. Adding qdev and qapi properties required to enable isolation mode for
>     various devices.
> 2. Allocating an isolation memory region in an anonymous file and mapping it
>     to host memory.  This isolation region will hold the bounce buffers and
>     vrings necessary to move data.
> 3. Using a vhost-iova-tree to allocate and track the mapping between
>     guest regions and their corresponding bounce buffers in the isolation
>     memory.
> 4. Creating shadow virtqueues to intercept request notifications.  As
>     kick and call events are received by an svq, it copies buffer
>     contents and descriptors between isolation and guest memory before
>     notifying the backend or guest.
> 
> Note: This project is currently in a partially functional state. For
> example, in testing vhost-user-input and a keyboard, keystrokes may
> register correctly or be double counted depending on the combination
> of backend daemon and guest OS used. However, the desire is to make
> this work public at this stage for comment on the overall
> approach.

I guess this intends to achieve something similar to IOMMU and also 
Linux's swiotlb for the VMM (please correct me if I'm wrong):

 > Other edge case scenarios arise for bounce buffers. For example, when
 > IOMMU mappings are set up for a DMA operation to/from a device that is
 > considered “untrusted”, the device should be given access only to the
 > memory containing the data being transferred. But if that memory
 > occupies only part of an IOMMU granule, other parts of the granule may
 > contain unrelated kernel data. Since IOMMU access control is
 > per-granule, the untrusted device can gain access to the unrelated
 > kernel data. This problem is solved by bounce buffering the DMA
 > operation and ensuring that unused portions of the bounce buffers do
 > not contain any unrelated kernel data.

https://docs.kernel.org/next/core-api/swiotlb.html#usage-scenarios

It would be nice if the security property this feature provides is 
explicitly stated, ideally as documentation.

I also hope this could fix vhost-user for TCG:
https://gitlab.com/qemu-project/qemu/-/work_items/2092

Perhaps the memory isolation should be enabled by default for TCG.

Implementation-wise, there are rough edges as you stated. Some of them 
can be fixed by more closely following what vhost-user does for the 
non-isolated case and what vhost-vdpa does with shadow virtqueues.

Regards,
Akihiko Odaki

> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
> ConKite (2):
>        vhost-user: Consolidate chardev property definitions
>        vhost-user: Add memory-isolation qdev property to vhost-user devices
> 
> Connor Kite (13):
>        backends/cryptodev-vhost-user: add memory isolation bool
>        net/vhost-user: add memory isolation
>        vhost-user: add memory_isolation to VhostUserState
>        util/iova-tree: g_tree_foreach wrapper
>        hw/virtio: iova_tree_foreach wrapper
>        hw/virtio/vhost-shadow-virtqueue: used handler
>        hw/virtio/vhost-shadow-virtqueue: specified vring placement
>        hw/virtio/vhost-shadow-virtqueue: range boundary in translation
>        hw/virtio/vhost-user: create isolation region
>        hw/virtio/vhost-user: send isolation regions to device
>        hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts
>        hw/virtio/vhost-user: handle data movement with shadow vqs
>        hw/virtio/vhost-user: shadow vq cleanup
> 
>   backends/cryptodev-vhost-user.c      |  24 +-
>   backends/vhost-user.c                |   4 +-
>   hw/block/vhost-user-blk.c            |   4 +-
>   hw/display/vhost-user-gpu.c          |   4 +-
>   hw/scsi/vhost-user-scsi.c            |   4 +-
>   hw/virtio/vhost-iova-tree.c          |  15 ++
>   hw/virtio/vhost-iova-tree.h          |   3 +
>   hw/virtio/vhost-shadow-virtqueue.c   |  30 ++-
>   hw/virtio/vhost-shadow-virtqueue.h   |  17 ++
>   hw/virtio/vhost-stub.c               |   3 +-
>   hw/virtio/vhost-user-base.c          |  13 +-
>   hw/virtio/vhost-user-fs.c            |   4 +-
>   hw/virtio/vhost-user-gpio.c          |   4 -
>   hw/virtio/vhost-user-i2c.c           |   5 -
>   hw/virtio/vhost-user-input.c         |   5 -
>   hw/virtio/vhost-user-rng.c           |   5 -
>   hw/virtio/vhost-user-rtc.c           |   4 -
>   hw/virtio/vhost-user-scmi.c          |   4 +-
>   hw/virtio/vhost-user-snd.c           |   1 -
>   hw/virtio/vhost-user-spi.c           |   5 -
>   hw/virtio/vhost-user-test-device.c   |   1 -
>   hw/virtio/vhost-user-vsock.c         |   4 +-
>   hw/virtio/vhost-user.c               | 441 ++++++++++++++++++++++++++++++++++-
>   include/hw/virtio/vhost-user-base.h  |   1 +
>   include/hw/virtio/vhost-user-blk.h   |   1 +
>   include/hw/virtio/vhost-user-fs.h    |   1 +
>   include/hw/virtio/vhost-user-scmi.h  |   1 +
>   include/hw/virtio/vhost-user-vsock.h |   1 +
>   include/hw/virtio/vhost-user.h       |   6 +-
>   include/hw/virtio/virtio-gpu.h       |   1 +
>   include/hw/virtio/virtio-scsi.h      |   1 +
>   include/qemu/iova-tree.h             |  13 ++
>   include/system/vhost-user-backend.h  |   3 +-
>   net/passt.c                          |  16 +-
>   net/vhost-user.c                     |  11 +-
>   qapi/net.json                        |  15 +-
>   qapi/qom.json                        |   6 +-
>   util/iova-tree.c                     |   5 +
>   38 files changed, 622 insertions(+), 64 deletions(-)
> ---
> base-commit: 006a22cb26998998385b104db1ff9466ef2f3153
> change-id: 20260604-vhost-user-isolated-memory-070ed4833ee7
> 
> Best regards,

Re: [PATCH RFC 00/15] vhost-user: isolated memory

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1082 bytes --]

On Sat, Jul 25, 2026 at 12:45:08PM +0900, Akihiko Odaki wrote:
> I also hope this could fix vhost-user for TCG:
> https://gitlab.com/qemu-project/qemu/-/work_items/2092
> 
> Perhaps the memory isolation should be enabled by default for TCG.

If I understand the bug correctly, TCG needs the dirty bitmap to be
up-to-date and vhost-user's dirty logging mechanism does not suffice.
Memory isolation would keep the dirty bitmap up-to-date but copying data
between guest RAM and the isolated memory would add overhead.

For the TCG use case it should be possible to use just
vhost-shadow-virtqueue.c without isolated memory so that
vhost_svq_flush() -> virtqueue_fill() -> virtqueue_unmap_sg() ->
dma_memory_unmap() keeps the dirty bitmap updated. There is still an
overhead because the virtqueue kick/call is intercepted by QEMU, but
there is no memcpy.

The TCG use case is something that could be tackled once memory
isolation is finished. If no one has time to implement the pure svq
solution for TCG, then memory isolation could be used in any case.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 03/15] backends/cryptodev-vhost-user: add memory isolation bool

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]

On Fri, Jul 24, 2026 at 08:06:36AM +0200, Markus Armbruster wrote:
> Connor Kite <connorkite@gmail.com> writes:
>    # @memory-isolation: disable access from cryptodev to guest memory.
>    #     (default: false) (since 11.2)
> 
> Pardon my ignorance...  What exactly is "cryptodev" here?  Is it code
> running in another process?
> 
> Should the description answer my question?  Perhaps like this:
> 
>   # @memory-isolation: isolate guest memory.  Isolated guest memory
>   #     cannot be accessed from uh, what exactly?
> 
> Fill in the blank.

Maybe it's clearer to mention the potentially untrusted vhost-user
backend process rather than cryptodev (the virtio-crypto vhost-user
device)?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 04/15] net/vhost-user: add memory isolation

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

On Thu, Jul 23, 2026 at 03:30:03PM -0700, Connor Kite wrote:
> Add a memory-isolation property to NetdevPassOptions and
> NetDevVhostUserOptions in net.json.  This property will
> be used for vhost-user devices to disable access to guest
> memory from devices.
> 
> It will be passed into vhost_user_init() in a future patch.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  net/passt.c      |  7 +++++++
>  net/vhost-user.c |  4 ++++
>  qapi/net.json    | 15 +++++++++++----
>  3 files changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/net/passt.c b/net/passt.c
> index 45440c399b..ce80186883 100644
> --- a/net/passt.c
> +++ b/net/passt.c
> @@ -735,6 +735,13 @@ int net_init_passt(const Netdev *netdev, const char *name,
>      s->pidfile = pidfile;
>  
>      if (netdev->u.passt.has_vhost_user && netdev->u.passt.vhost_user) {
> +        bool memory_isolation G_GNUC_UNUSED = false;
> +
> +        if (netdev->u.passt.has_memory_isolation &&
> +           netdev->u.passt.memory_isolation) {
> +            memory_isolation = true;
> +        }

This is dead code (hence G_GNUC_UNUSED) and is harder to review because
we don't know how it will be used until later in the patch series. I
guess vhost_user_init() and net_passt_vhost_user_init() will gain a
memory_isolation argument in a future patch.

Linear reviews of the patch series are easiest for reviewers. Please
reorder this so that vhost_user_init() and net_pass_vhost_user_init()
are changed first. You can hardcode the argument to false in those
patches. This patch can be moved afterwards so that memory_isolation is
never dead code.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

On Thu, Jul 23, 2026 at 03:30:04PM -0700, Connor Kite wrote:
> @@ -73,12 +74,14 @@ typedef struct VhostUserState {
>      int memory_slots;
>      bool supports_config;
>      bool supports_inflight_migration;
> +    bool memory_isolation;
>  } VhostUserState;
>  
>  /**
>   * vhost_user_init() - initialise shared vhost_user state
>   * @user: allocated area for storing shared state
>   * @chr: the chardev for the vhost socket
> + * @memory_isolation: disables device access to guest memory

"Device" is confusing because it has several meanings. "vhost-user
back-end" would be clearer.

"Disables" is also a little confusing because the vhost-user back-end
still needs to access guest memory somehow. Memory access cannot be
completely disabled. Something like "restricts vhost-user back-end
access to guest memory via bounce buffers" could be clearer, but it's up
to you. We probably don't want to repeat the full explanation in every
comment about memory isolation, so it's fine to omit the details as long
as there is user-facing documentation somewhere that explains what the
feature does.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

On Thu, Jul 23, 2026 at 03:30:05PM -0700, Connor Kite wrote:
> Add a wrapper to iova-tree to call g_tree_foreach.  Provies a means to

Provies -> Provides

> execute some function on each node of the iova-tree.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  include/qemu/iova-tree.h | 13 +++++++++++++
>  util/iova-tree.c         |  5 +++++
>  2 files changed, 18 insertions(+)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 03/15] backends/cryptodev-vhost-user: add memory isolation bool

[Connor Kite]

[-- Attachment #1: Type: text/plain, Size: 552 bytes --]

On Mon, Jul 27, 2026 at 11:43 AM Stefan Hajnoczi <stefanha@redhat.com>
wrote:

> > Maybe it's clearer to mention the potentially untrusted vhost-user
> > backend process rather than cryptodev (the virtio-crypto vhost-user
> > device)?
>
> That difference in emphasis might indeed be clearer.  The main purpose of
this patch in the series is simply to provide a means of enabling the
isolation mode that is being implemented in vhost-user.  To Markus's point,
clarifying that this is limiting access from the backend process makes sense
.

[-- Attachment #2: Type: text/html, Size: 1100 bytes --]

Re: [PATCH RFC 00/15] vhost-user: isolated memory

[Demi Marie Obenour]

[-- Attachment #1.1: Type: text/plain, Size: 1601 bytes --]

On 7/23/26 18:29, ConKite wrote:
> This patch series implements a memory isolation mode in vhost-user. The
> purpose of this mode is to provide the option of additional security by
> eliminating direct access of guest memory by vhost-user devices.
> At a high level this works by:
> 
> 1. Adding qdev and qapi properties required to enable isolation mode for
>    various devices.
> 2. Allocating an isolation memory region in an anonymous file and mapping it
>    to host memory.  This isolation region will hold the bounce buffers and
>    vrings necessary to move data.
> 3. Using a vhost-iova-tree to allocate and track the mapping between
>    guest regions and their corresponding bounce buffers in the isolation
>    memory.
> 4. Creating shadow virtqueues to intercept request notifications.  As
>    kick and call events are received by an svq, it copies buffer
>    contents and descriptors between isolation and guest memory before
>    notifying the backend or guest.   
> 
> Note: This project is currently in a partially functional state. For
> example, in testing vhost-user-input and a keyboard, keystrokes may
> register correctly or be double counted depending on the combination
> of backend daemon and guest OS used. However, the desire is to make
> this work public at this stage for comment on the overall

Nice!

Does this approach work for a generic vhost-user device, as opposed to
one that knows it is a block/network/etc device?  Also, how much overhead
is there compared to the normal case?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH RFC 00/15] vhost-user: isolated memory

[Connor Kite]

On Tue, Jul 28, 2026 at 12:08 AM Demi Marie Obenour
<demiobenour@gmail.com> wrote:
> Does this approach work for a generic vhost-user device, as opposed to
> one that knows it is a block/network/etc device?  Also, how much overhead
> is there compared to the normal case?

The idea is for this approach to be agnostic to the specific device in use.

As noted on the cover letter, I am still working through some rough edges on
the implementation still, but when the project is more stable, I will be
benchmarking the effects of memory isolation on performance.

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2359 bytes --]

On Thu, Jul 23, 2026 at 03:30:07PM -0700, Connor Kite wrote:
> Adds a handler callback to shadow virtqueues in order to enable
> non-default processing of used vring elements from the device.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
>  hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index bcb7f2ffc7..eb86c1ee37 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -669,6 +669,10 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq,
>                  break;
>              }
>  
> +            if (svq->ops && svq->ops->used_handler) {
> +                svq->ops->used_handler(svq, elem, svq->ops_opaque);
> +            }
> +
>              if (unlikely(i >= svq->vring.num)) {
>                  qemu_log_mask(LOG_GUEST_ERROR,
>                           "More than %u used buffers obtained in a %u size SVQ",
> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
> index ec16a1e838..ccfeee36d7 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.h
> +++ b/hw/virtio/vhost-shadow-virtqueue.h
> @@ -56,8 +56,22 @@ typedef int (*VirtQueueAvailCallback)(VhostShadowVirtqueue *svq,
>                                        VirtQueueElement *elem,
>                                        void *vq_callback_opaque);
>  
> +/**
> + * Callback to handle a used buffer
> + *
> + * @svq: Shadow virtqueue
> + * @elem: Element placed in the queue by the device
> + * @vq_callback_opaque: Used to pass arguments to callback
> + *
> + * Returns 0 if the vq is running as expected

The doc comment should mention what non-0 look like. Are errors reported
as a negative errno?

> + */
> +typedef int (*VirtQueueUsedCallback)(VhostShadowVirtqueue *svq,
> +                                      VirtQueueElement *elem,
> +                                      void *vq_callback_opaque);
> +
>  typedef struct VhostShadowVirtqueueOps {
>      VirtQueueAvailCallback avail_handler;
> +    VirtQueueUsedCallback used_handler;
>  } VhostShadowVirtqueueOps;
>  
>  /* Shadow virtqueue to relay notifications */
> 
> -- 
> 2.43.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 3649 bytes --]

On Thu, Jul 23, 2026 at 03:30:08PM -0700, Connor Kite wrote:
> By default svq vrings are placed in an anonymous memory map. As svqs
> will be leveraged to enable memory isolation in vhost-user, it is useful
> to be able to place the vrings in a shared isolation memory region.
> 
> Adds the option to specify vring placement by providing a vring base
> address before starting the svq.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-shadow-virtqueue.c | 22 +++++++++++++++-------
>  hw/virtio/vhost-shadow-virtqueue.h |  3 +++
>  2 files changed, 18 insertions(+), 7 deletions(-)
> 
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index eb86c1ee37..9e3c359f50 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -857,14 +857,21 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODevice *vdev,
>  
>      svq->vring.num = virtio_queue_get_num(vdev, virtio_get_queue_index(vq));
>      svq->num_free = svq->vring.num;
> -    svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
> -                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> -                           -1, 0);
>      desc_size = sizeof(vring_desc_t) * svq->vring.num;
> -    svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> -    svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
> -                           PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> -                           -1, 0);
> +    if (svq->base_addr == NULL) {
> +        svq->vring.desc = mmap(NULL, vhost_svq_driver_area_size(svq),
> +                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> +                            -1, 0);
> +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> +        svq->vring.used = mmap(NULL, vhost_svq_device_area_size(svq),
> +                            PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS,
> +                            -1, 0);
> +    } else {
> +        svq->vring.desc = (void *) svq->base_addr;
> +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> +        svq->vring.used = (void *)((char *)svq->base_addr +
> +                          vhost_svq_driver_area_size(svq));
> +    }
>      svq->desc_state = g_new0(SVQDescState, svq->vring.num);
>      if (virtio_vdev_has_feature(svq->vdev, VIRTIO_F_IN_ORDER)) {
>          svq->batch_last.id = VIRTIO_RING_NOT_IN_BATCH;
> @@ -929,6 +936,7 @@ VhostShadowVirtqueue *vhost_svq_new(const VhostShadowVirtqueueOps *ops,
>      event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND);
>      svq->ops = ops;
>      svq->ops_opaque = ops_opaque;
> +    svq->base_addr = NULL;
>      return svq;
>  }
>  
> diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h
> index ccfeee36d7..39f69e6455 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.h
> +++ b/hw/virtio/vhost-shadow-virtqueue.h
> @@ -148,6 +148,9 @@ typedef struct VhostShadowVirtqueue {
>  
>      /* Size of SVQ vring free descriptors */
>      uint16_t num_free;
> +
> +    /* Location assigned to vrings if not in default anon memory map*/
> +    hwaddr *base_addr;

Why is base_addr a hwaddr? This is QEMU memory, not guest RAM. I
expected this to be void *.

A size needs to be at least documented here to reduce the chance of
memory bugs. Even better would be an interface like
vhost_svq_vring_total_size() so the caller can fetch the number of bytes
before allocating the memory.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

On Thu, Jul 23, 2026 at 03:30:09PM -0700, Connor Kite wrote:
> iova-tree expects inclusive range sizing when maps are allocated or searched.
> Currently, svqs use exclusive sizing when searching their
> vhost-iova-tree for a match to the region to be translated.  This could
> lead to errors if the region to be translated is at the edge of an iova
> region.
> 
> Fix this by reducing `needle.size` by 1 in
> vhost_svq_translate_addr to bring then it line with DMAMap and iova-tree
> convention.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-shadow-virtqueue.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 8880 bytes --]

On Thu, Jul 23, 2026 at 03:30:10PM -0700, Connor Kite wrote:
> If memory isolation mode is active for the vhost-user device adds
> features to:
> - Gather the size required for bounce buffers and vrings in shared
>   isolation region
> - Allocate the required space in an anonymous file
> - Create a vhost-iova-tree with space to map entire isolation region
> - Map guest memory regions and shared vrings into the tree
> - Release these resources upon backend cleanup
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-user.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 129 insertions(+)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index f296b63fb9..710cf966f8 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>  #include "hw/virtio/vhost-backend.h"
>  #include "hw/virtio/virtio.h"
>  #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-iova-tree.h"
>  #include "chardev/char-fe.h"
>  #include "io/channel-socket.h"
>  #include "system/kvm.h"
> @@ -25,6 +26,7 @@
>  #include "qemu/main-loop.h"
>  #include "qemu/uuid.h"
>  #include "qemu/sockets.h"
> +#include "qemu/memfd.h"
>  #include "system/runstate.h"
>  #include "system/cryptodev.h"
>  #include "migration/postcopy-ram.h"
> @@ -320,6 +322,13 @@ static VhostUserMsg m __attribute__ ((unused));
>  /* The version of the protocol we support */
>  #define VHOST_USER_VERSION    (0x1)
>  
> +typedef struct IsolationRegion {
> +    uint64_t base_addr;
> +    uint64_t vring_base_addr;
> +    uint64_t size;
> +    int iso_fd;
> +} IsolationRegion;

The purpose of the base_addr and vring_base_addr fields is not obvious.
I suggest adjusting the types, names, and adding comments to make the
purpose clearer:

  typedef struct {
      void *mem;           /* mapped shared memory */
      size_t size;
      uint64_t vring_iova;
      int fd;              /* shared memory fd */
  } IsolationRegion;

> +
>  struct vhost_user {
>      struct vhost_dev *dev;
>      /* Shared between vhost devs of the same virtio device */
> @@ -353,6 +362,10 @@ struct vhost_user {
>       * by the backend (see @features).
>       */
>      uint64_t protocol_features;
> +
> +    /* Isolated memory data*/
> +    struct IsolationRegion iso_memory;

struct is not necessary since there is a typedef:

  IsolationRegion iso_memory;

> +    VhostIOVATree *iso_iova_tree;

The IOVA tree seems to be closely used with IsolationRegion. Maybe this
field should move into IsolationRegion?

>  };
>  
>  struct scrub_regions {
> @@ -1109,6 +1122,121 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
>      return 0;
>  }
>  
> +/* TODO: Is there any notifier cleanup required here?*/
> +static void cleanup_isolation_regions(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    if (u->iso_memory.base_addr) {
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +        u->iso_iova_tree = NULL;
> +        memset(&u->iso_memory, 0, sizeof(IsolationRegion));
> +        qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
> +                         u->iso_memory.iso_fd);
> +        u->iso_memory.base_addr = 0;
> +    }
> +}
> +
> +__attribute__((unused))
> +static int init_isolation_regions(struct vhost_dev *dev,
> +                                  VhostUserMsg *msg,
> +                                  int *fds, size_t *fd_num)
> +{
> +    Error *err = NULL;
> +    struct vhost_user *u = dev->opaque;
> +    uint32_t nregions = dev->mem->nregions;
> +    uint64_t buffer_reg_size = 0;
> +    DMAMap newEntry = {
> +        .perm = IOMMU_RW
> +    };
> +    g_autoptr(GArray) buffer_regions =
> +        g_array_new(FALSE, TRUE, sizeof(DMAMap));
> +
> +    msg->hdr.request = VHOST_USER_SET_MEM_TABLE;
> +
> +    /* In case of reset, clear old regions*/
> +    if (u->iso_memory.base_addr != 0) {
> +        cleanup_isolation_regions(dev);
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +    }
> +
> +    /* Gather information for bounce buffers to be mapped */
> +    for (int i = 0; i < nregions; i++) {

nregions is uint32_t, so i should also be uint32_t to avoid
signed/unsigned comparisons.

> +        struct vhost_memory_region *dev_region = &dev->mem->regions[i];
> +        hwaddr size = ROUND_UP(dev_region->memory_size,
> +                      qemu_real_host_page_size());
> +        newEntry.translated_addr = dev_region->guest_phys_addr;
> +        newEntry.size = size - 1;
> +        buffer_reg_size += size;
> +        g_array_append_val(buffer_regions, newEntry);
> +    }
> +
> +    int num;
> +    size_t desc_size;
> +    size_t avail_size;
> +    size_t driver_area_size;
> +    size_t device_area_size;
> +    size_t total_vring_size = 0;
> +    size_t total_mmap_size;
> +
> +    /* Get space required for all vrings */
> +    for (int j = 0; j < dev->nvqs; j++) {
> +        num = virtio_queue_get_num(dev->vdev, dev->vq_index + j);
> +        desc_size = sizeof(vring_desc_t) * num;
> +        avail_size = offsetof(vring_avail_t, ring[num]) +
> +                        sizeof(uint16_t);
> +        driver_area_size = ROUND_UP(desc_size + avail_size,
> +                            qemu_real_host_page_size());
> +        device_area_size = ROUND_UP(offsetof(vring_used_t, ring[num]) +
> +                                    sizeof(uint16_t),
> +                                    qemu_real_host_page_size());
> +        total_vring_size += driver_area_size + device_area_size;

Please avoid duplicating the memory layout calculations. When packed
vring support is added to vhost-shadow-virtqueue.c this will become more
complex and it should be done in a single place. vhost-shadow-virtque.c
should expose an API for the size calculation.

> +    }
> +
> +    total_mmap_size = buffer_reg_size + total_vring_size;
> +
> +    /* Allocate and map an anonymous file to hold the isolation region */
> +    u->iso_memory.base_addr = (uint64_t) qemu_memfd_alloc("iso_r",

Please make the name unique (e.g. using dev->vdev->name).

> +                              total_mmap_size,
> +                              F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL,
> +                              &u->iso_memory.iso_fd, &err);
> +    u->iso_memory.size = total_mmap_size;
> +
> +    if (err) {
> +        error_report_err(err);
> +        cleanup_isolation_regions(dev);
> +        return -1;
> +    }
> +
> +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
> +                                                 total_mmap_size - 1));
> +
> +    /*
> +     * Instantiates iova tree sized to map bounce buffers and vrings to the
> +     * isolation region in host va.
> +     */
> +    u->iso_iova_tree = vhost_iova_tree_new(u->iso_memory.base_addr, last_addr);

Is it possible to use 0 as the IOVA base address so that QEMU's
addresses aren't leaked to the vhost-user back-end? It's good security
practice not to reveal memory addresses to the outside world because
that information can be used to defeat address space randomization or
infer memory addresses of other data structures.

> +
> +    assert(&u->iso_memory.iso_fd >= 0);

The dereference operator should not be used here, it's the iso_fd value
that is being tested.

> +    DMAMap *map;
> +    DMAMap vring_map = {
> +        .perm = IOMMU_RW,
> +        .size = total_vring_size - 1,
> +        /*vrings are allocated on tree first, so will be assigned base addr*/
> +        .translated_addr = u->iso_memory.base_addr

Why is this field assigned here, I think this field is used as the
output of vhost_iova_tree_map_alloc() rather than an input (e.g. see
vhost_vdpa_svq_map_rings())?

> +    };
> +
> +    vhost_iova_tree_map_alloc(u->iso_iova_tree, &vring_map,
> +                              vring_map.translated_addr);
> +    u->iso_memory.vring_base_addr = vring_map.iova;
> +    for (int i = 0; i < buffer_regions->len; i++) {
> +        map = &g_array_index(buffer_regions, DMAMap, i);
> +        vhost_iova_tree_map_alloc_gpa(u->iso_iova_tree, map,
> +                                      map->translated_addr);
> +    }
> +
> +    return 0;
> +}
> +
>  static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                      struct vhost_memory *mem)
>  {
> @@ -2681,6 +2809,7 @@ static int vhost_user_backend_cleanup(struct vhost_dev *dev)
>      g_free(u->region_rb_offset);
>      u->region_rb_offset = NULL;
>      u->region_rb_len = 0;
> +    cleanup_isolation_regions(dev);
>      g_free(u);
>      dev->opaque = 0;
>  
> 
> -- 
> 2.43.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Connor Kite]

On Mon, Jul 27, 2026 at 12:08 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Thu, Jul 23, 2026 at 03:30:05PM -0700, Connor Kite wrote:
> > Add a wrapper to iova-tree to call g_tree_foreach.  Provies a means to
>
> Provies -> Provides
>
> > execute some function on each node of the iova-tree.
> >
> > Signed-off-by: Connor Kite <connorkite@gmail.com>
> > ---
> >  include/qemu/iova-tree.h | 13 +++++++++++++
> >  util/iova-tree.c         |  5 +++++
> >  2 files changed, 18 insertions(+)
>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

This is fixed on my end now and will be included in the next rev of the series.

Thanks,
Connor

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Connor Kite]

On Fri, Jul 24, 2026 at 3:57 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
> On 2026/07/24 7:30, ConKite wrote:
> > In support of future work to add a memory isolation mode to vhost-user devices, a
> > qdev property, "memory-isolation", is added to all vhost-user devices. This is
> > simply a bool representing whether the mode is active or not.
> >
> > Signed-off-by: Connor Kite <connorkite@gmail.com>
> > Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
>
> This and the next patch add interfaces before implementation. Perhaps
> it's better reversed.

Agreed! I will move these for the next rev of the patch series.

Thanks,
Connor

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 5467 bytes --]

On Thu, Jul 23, 2026 at 03:30:11PM -0700, Connor Kite wrote:
> Adds features to fill a vhost_user_set_mem_table message with the
> addresses of isolation memory regions corresponding to bounce buffers
> and vrings.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-user.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 79 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 710cf966f8..acabfb7f1c 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -625,6 +625,21 @@ static void vhost_user_fill_msg_region(struct vhost_dev *dev,
>      dst->mmap_offset = mmap_offset;
>  }
>  
> +static void vhost_user_fill_msg_region_iso(VhostUserMemoryRegion *dst,
> +                                           const struct vhost_user *u,
> +                                           const struct vhost_memory_region
> +                                           *iova_reg)
> +{
> +    assert(u != NULL && dst != NULL && iova_reg != NULL);
> +    uint64_t offset;
> +
> +    offset = iova_reg->userspace_addr - u->iso_memory.base_addr;
> +    dst->userspace_addr = iova_reg->userspace_addr;
> +    dst->memory_size = iova_reg->memory_size;
> +    dst->guest_phys_addr = iova_reg->userspace_addr;
> +    dst->mmap_offset = offset;
> +}
> +
>  static int vhost_user_fill_set_mem_table_msg(struct vhost_user *u,
>                                               struct vhost_dev *dev,
>                                               VhostUserMsg *msg,
> @@ -1136,7 +1151,33 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
>      }
>  }
>  
> -__attribute__((unused))
> +struct iova_tree_traversal_args {

QEMU coding style:

  typedef struct {
      ...
  } IOVATreeTraversalArgs;

> +    VhostUserMsg *msg;
> +    struct vhost_user *u;
> +    int *fds;
> +    size_t *fd_num;

This field is also used to index into msg->payload.memory.regions[], so
"fd_num" is a misnomer. I suggest something like "region_idx" or just
"idx".

> +};
> +
> +static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
> +                                                    gpointer value,
> +                                                    gpointer data)
> +{
> +    struct iova_tree_traversal_args *args = data;
> +    struct vhost_memory_region msg_region;
> +    VhostUserMemoryRegion region_buffer;
> +    DMAMap *map = key;
> +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;
> +
> +    msg_region.guest_phys_addr = map->iova;
> +    msg_region.memory_size = map->size + 1;
> +    msg_region.userspace_addr = map->iova;
> +    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
> +    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;

I'm confused by this code. VhostUserMemoryRegion region_buffer is the
vhost-user protocol struct that is being filled in, but there is also a
struct vhost_memory_region msg_region from the Linux kernel headers?

msg_region and vhost_user_fill_msg_region_iso() make it harder to see
what is going on. Can you open code the region_buffer struct field
assignments instead?

> +    (*args->fd_num)++;
> +
> +    return false;
> +}
> +
>  static int init_isolation_regions(struct vhost_dev *dev,
>                                    VhostUserMsg *msg,
>                                    int *fds, size_t *fd_num)
> @@ -1234,6 +1275,24 @@ static int init_isolation_regions(struct vhost_dev *dev,
>                                        map->translated_addr);
>      }
>  
> +    struct iova_tree_traversal_args args = {
> +        .fd_num = fd_num,
> +        .fds = fds,
> +        .msg = msg,
> +        .u = u
> +    };
> +
> +    vhost_iova_tree_foreach(u->iso_iova_tree,
> +                            vhost_user_iova_tree_traverse_funct, &args);
> +
> +    msg->payload.memory.nregions = *fd_num;
> +
> +    assert(*fd_num != 0);
> +
> +    msg->hdr.size = sizeof(msg->payload.memory.nregions);
> +    msg->hdr.size += sizeof(msg->payload.memory.padding);
> +    msg->hdr.size += *fd_num * sizeof(VhostUserMemoryRegion);
> +
>      return 0;
>  }
>  
> @@ -1241,6 +1300,7 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                      struct vhost_memory *mem)
>  {
>      struct vhost_user *u = dev->opaque;
> +    bool memory_isolation = u->user->memory_isolation;
>      int fds[VHOST_MEMORY_BASELINE_NREGIONS];
>      size_t fd_num = 0;
>      bool do_postcopy = u->postcopy_listen && u->postcopy_fd.handler;
> @@ -1268,6 +1328,24 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
>          msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
>      }
>  
> +    if (memory_isolation) {
> +        ret = init_isolation_regions(dev, &msg, fds, &fd_num);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +
> +        ret = vhost_user_write(dev, &msg, fds, fd_num);
> +        if (ret < 0) {
> +            return ret;
> +        }
> +
> +        if (reply_supported) {
> +            return process_message_reply(dev, &msg);
> +        }
> +
> +        return 0;
> +    }
> +
>      if (config_mem_slots) {
>          ret = vhost_user_add_remove_regions(dev, &msg, reply_supported, false);
>          if (ret < 0) {
> 
> -- 
> 2.43.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 4762 bytes --]

On Thu, Jul 23, 2026 at 03:30:12PM -0700, Connor Kite wrote:
> Adds shadow virtqueues that will eventually be used to transfer data
> between device and host via bounce buffers when isolation mode is
> active.  The svqs are initalized, and eventfd assignments are
> intercepted so that notifications come to svqs first before
> the guest or backend receive them.
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-user.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 54 insertions(+)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index acabfb7f1c..75858289a2 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>  #include "hw/virtio/vhost-backend.h"
>  #include "hw/virtio/virtio.h"
>  #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-shadow-virtqueue.h"
>  #include "hw/virtio/vhost-iova-tree.h"
>  #include "chardev/char-fe.h"
>  #include "io/channel-socket.h"
> @@ -365,7 +366,9 @@ struct vhost_user {
>  
>      /* Isolated memory data*/
>      struct IsolationRegion iso_memory;
> +    GPtrArray *shadow_vqs;
>      VhostIOVATree *iso_iova_tree;
> +    bool svqs_allocated;
>  };
>  
>  struct scrub_regions {
> @@ -1693,6 +1696,24 @@ static int vhost_set_vring_file(struct vhost_dev *dev,
>  static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>                                       struct vhost_vring_file *file)
>  {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +        vhost_svq_set_svq_kick_fd(svq, file->fd);
> +
> +        if (svq->hdev_kick.initialized == false) {
> +            int r = event_notifier_init(&svq->hdev_kick, 0);

Where is event_notifier_cleanup() called?

> +            if (r) {
> +                error_report("Failed to create kick event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_kick);

Modifying the function argument is probably not expected but
vhost_virtqueue_start() doesn't use it after this call, so there is no
immediate problem. It would be safer to have a local struct
vhost_vring_file that can be modified without affecting the caller's
copy.

> +    }
> +
>      int ret = vhost_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
>      if (ret < 0) {
>          return ret;
> @@ -1721,6 +1742,24 @@ static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>  static int vhost_user_set_vring_call(struct vhost_dev *dev,
>                                       struct vhost_vring_file *file)
>  {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +        vhost_svq_set_svq_call_fd(svq, file->fd);
> +
> +        if (svq->hdev_call.initialized == false) {
> +            int r = event_notifier_init(&svq->hdev_call, 0);

Where is event_notifier_cleanup() called?

> +            if (r) {
> +                error_report("Failed to create call event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_call);

Same as above.

> +    }
> +
>      return vhost_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
>  }
>  
> @@ -2715,6 +2754,17 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>      return 0;
>  }
>  
> +static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
> +{
> +    /*Modified from vhost-vdpa*/
> +    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
> +    for (int i = 0; i < dev->nvqs; i++) {
> +        VhostShadowVirtqueue *svq;
> +        svq = vhost_svq_new(NULL, NULL);
> +        g_ptr_array_add(u->shadow_vqs, svq);
> +    }
> +}
> +
>  static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>                                     Error **errp)
>  {
> @@ -2859,6 +2909,10 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>      u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
>      postcopy_add_notifier(&u->postcopy_notifier);
>  
> +    if (vus->memory_isolation) {
> +        vhost_user_init_svq(dev, u);
> +    }
> +
>      return 0;
>  }
>  
> 
> -- 
> 2.43.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 9513 bytes --]

On Thu, Jul 23, 2026 at 03:30:13PM -0700, Connor Kite wrote:
> - Add start logic for shadow virtqueues, which sets vring addresses.
> - Update logic for sending vring addresses to backend to point
>   to the shadow vrings when isolation mode is active.
> - Implement handlers for intercepted avail and used descriptors. These
>   handlers copy buffer contents between bounce buffers in the isolation
>   region and the buffers made available by the guest
> - Implement logic to stop svqs
> 
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>  hw/virtio/vhost-user.c | 167 ++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 166 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 75858289a2..e65f877f9a 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1299,6 +1299,97 @@ static int init_isolation_regions(struct vhost_dev *dev,
>      return 0;
>  }
>  
> +static int vhost_user_memory_lookup(struct vhost_dev *dev, hwaddr gpa,
> +                                    hwaddr *hva)
> +{
> +    int i;
> +    hwaddr offset;
> +
> +    for (i = 0; i < dev->mem->nregions; i++) {
> +        struct vhost_memory_region *reg = dev->mem->regions + i;
> +
> +        if (gpa >= reg->guest_phys_addr &&
> +            reg->guest_phys_addr + reg->memory_size > gpa) {
> +            offset = gpa - reg->guest_phys_addr;
> +            *hva = reg->userspace_addr + offset;
> +            return 0;
> +        }
> +    }
> +
> +    return -EFAULT;
> +}
> +
> +static int vhost_user_svq_handle_used(VhostShadowVirtqueue *svq,
> +                                      VirtQueueElement *elem,
> +                                      void *opaque)
> +{
> +    hwaddr hva;
> +    int r;
> +    struct vhost_dev *dev = opaque;
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->in_addr[i], &hva);
> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->in_sg[i].iov_base, elem->in_sg[i].iov_len);
> +        elem->in_sg[i].iov_base = (void *) hva;
> +    }
> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->out_addr[i], &hva);
> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->out_sg[i].iov_base, elem->out_sg[i].iov_len);
> +        elem->out_sg[i].iov_base = (void *) hva;
> +    }
> +
> +    return 0;
> +}
> +
> +static int vhost_user_svq_handle_avail(VhostShadowVirtqueue *svq,
> +                                       VirtQueueElement *elem,
> +                                       void *opaque)
> +{
> +    hwaddr offset;
> +    const DMAMap *map;
> +    DMAMap needle;
> +    hwaddr *iova_base;
> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        needle.translated_addr = elem->out_addr[i];
> +        needle.size = elem->out_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);

Mapping failure must be handled.

> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->out_sg[i].iov_base = iova_base;
> +        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);

iova_base is an iova, not QEMU memory (HVA). Memcpy cannot be used with
IOVAs. The address of the mapped shared memory is needed as the
destination instead.

> +    }
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        needle.translated_addr = elem->in_addr[i];
> +        needle.size = elem->in_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);

Mapping failure must be handled.

> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->in_sg[i].iov_base = iova_base;
> +        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);

No memcpy is necessary before vhost_svq_add() since this is data that
will be read from the device upon I/O completion.

> +    }

elem->out_sg[] and elem->in_sg[] are modified in this function. Have we
lost the original I/O buffer memory address from the guest virtqueue?
This is a problem because they will be needed when completing the
request.

> +
> +    vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->out_addr,
> +                  elem->in_sg, elem->in_num, elem->in_addr, elem);
> +
> +    return 0;
> +}
> +
> +
> +
> +
>  static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                      struct vhost_memory *mem)
>  {
> @@ -1772,6 +1863,8 @@ static int vhost_user_set_vring_err(struct vhost_dev *dev,
>  static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>                                       struct vhost_vring_addr *addr)
>  {
> +    struct vhost_user *u = dev->opaque;
> +
>      VhostUserMsg msg = {
>          .hdr.request = VHOST_USER_SET_VRING_ADDR,
>          .hdr.flags = VHOST_USER_VERSION,
> @@ -1779,6 +1872,25 @@ static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>          .hdr.size = sizeof(msg.payload.addr),
>      };
>  
> +    if (u->user->memory_isolation) {
> +        if (!u->svqs_allocated) {
> +            return 0;
> +        }

Is the idea that this returns silently when called before
vhost_user_dev_start()? State make code harder to understand. It would
be cleaner to set up the vring addresses without relying on
svqs_allocated. What is the reason for deferring the vring address
setup to vhost_user_dev_start()?

> +
> +        int svq_idx = addr->index - dev->vq_index;
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +
> +        struct vhost_vring_addr svq_addr = {
> +            .avail_user_addr = (uint64_t)(uintptr_t)svq->vring.avail,
> +            .desc_user_addr = (uint64_t)(uintptr_t)svq->vring.desc,
> +            .used_user_addr = (uint64_t)(uintptr_t)svq->vring.used,
> +            .index = addr->index,
> +        };
> +
> +        msg.payload.addr = svq_addr;
> +    }
> +
>      /*
>       * wait for a reply if logging is enabled to make sure
>       * backend is actually logging changes
> @@ -2754,13 +2866,18 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>      return 0;
>  }
>  
> +static const VhostShadowVirtqueueOps vhost_user_svq_ops = {
> +    .avail_handler = vhost_user_svq_handle_avail,
> +    .used_handler = vhost_user_svq_handle_used
> +};
> +
>  static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
>  {
>      /*Modified from vhost-vdpa*/
>      u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
>      for (int i = 0; i < dev->nvqs; i++) {
>          VhostShadowVirtqueue *svq;
> -        svq = vhost_svq_new(NULL, NULL);
> +        svq = vhost_svq_new(&vhost_user_svq_ops, dev);
>          g_ptr_array_add(u->shadow_vqs, svq);
>      }
>  }
> @@ -3466,8 +3583,56 @@ void vhost_user_async_close(DeviceState *d,
>      }
>  }
>  
> +static bool vhost_user_svqs_start(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    uint64_t vring_base = u->iso_memory.vring_base_addr;
> +    u->svqs_allocated = true;

Where is this field cleared to false on reset?

> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
> +        svq->base_addr = (hwaddr *) vring_base;

Does this support multiple virtqueues, it looks like they will all use
the same base_addr?

> +        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);
> +
> +        struct vhost_vring_addr addr = {
> +            .index = dev->vq_index + i,
> +            .desc_user_addr = vring_base,
> +            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
> +                svq->vring.num,
> +            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)
> +        };
> +
> +        vhost_user_set_vring_addr(dev, &addr);

Also mentioned above in vhost_user_set_vring_addr():

Sending a vhost-user message here is strange since that should already
be done by vhost.c:vhost_virtqueue_start() ->
vhost_user_set_vring_addr().

> +
> +        vring_base += vhost_svq_device_area_size(svq) +
> +                      vhost_svq_driver_area_size(svq);
> +    }
> +
> +    return false;
> +}
> +
> +static void vhost_user_svqs_stop(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
> +    }
> +}
> +
> +
>  static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
>  {
> +    struct vhost_user *u = dev->opaque;
> +    if (u->user->memory_isolation) {
> +        if (started) {
> +            vhost_user_svqs_start(dev);
> +        } else {
> +            vhost_user_svqs_stop(dev);
> +        }
> +    }
> +
>      if (!vhost_user_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_STATUS)) {
>          return 0;
>      }
> 
> -- 
> 2.43.0
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Connor Kite]

On Fri, Jul 24, 2026 at 4:09 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
> The signature of the stub for !defined(CONFIG_VHOST_USER) is not
> updated, breaking builds.
>
> Regards,
> Akihiko Odaki

Good catch!  I have fixed this in passt.c, which I believe is the only
location affected.

On Mon, Jul 27, 2026 at 12:06 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Thu, Jul 23, 2026 at 03:30:04PM -0700, Connor Kite wrote:
> > @@ -73,12 +74,14 @@ typedef struct VhostUserState {
> >      int memory_slots;
> >      bool supports_config;
> >      bool supports_inflight_migration;
> > +    bool memory_isolation;
> >  } VhostUserState;
> >
> >  /**
> >   * vhost_user_init() - initialise shared vhost_user state
> >   * @user: allocated area for storing shared state
> >   * @chr: the chardev for the vhost socket
> > + * @memory_isolation: disables device access to guest memory
>
> "Device" is confusing because it has several meanings. "vhost-user
> back-end" would be clearer.
>
> "Disables" is also a little confusing because the vhost-user back-end
> still needs to access guest memory somehow. Memory access cannot be
> completely disabled. Something like "restricts vhost-user back-end
> access to guest memory via bounce buffers" could be clearer, but it's up
> to you. We probably don't want to repeat the full explanation in every
> comment about memory isolation, so it's fine to omit the details as long
> as there is user-facing documentation somewhere that explains what the
> feature does.

Perhaps for brevity it might make sense to say "@memory_isolation: restrict
direct access to guest memory from back-end".

Re: [PATCH RFC 07/15] hw/virtio: iova_tree_foreach wrapper

[Connor Kite]

On Fri, Jul 24, 2026 at 4:15 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
> GTraverseFunc stops traversal on true, but this says false ends traversal.
>

...

>
> Here is an extra blank line.
>
> Regards,
> Akihiko Odaki
>

Thanks!  I will fix both of these on the next rev.

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

On Fri, Jul 24, 2026 at 5:53 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
...
>
> u->iso_memory is zeroed before they are read here.
>
...
>
> This should be new_entry as per: docs/devel/style.rst
>
...
>
> The check of u->iso_memory.base_addr is redundant;
> cleanup_isolation_regions() performs that check.
>
> cleanup_isolation_regions() also calls vhost_iova_tree_delete(), so the
> duplicate call of the function leads to use-after-free and double-free.
>

Thanks!  Will fix these.

>
> I wonder if it is fine to dismiss return values of
> vhost_iova_tree_map_alloc() and vhost_iova_tree_map_alloc_gpa().
>

It probably makes sense to add a check here and return with the value on error

On Fri, Jul 24, 2026 at 5:53 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
> On 2026/07/24 7:30, Connor Kite wrote:
> > If memory isolation mode is active for the vhost-user device adds
> > features to:
> > - Gather the size required for bounce buffers and vrings in shared
> >    isolation region
> > - Allocate the required space in an anonymous file
> > - Create a vhost-iova-tree with space to map entire isolation region
> > - Map guest memory regions and shared vrings into the tree
> > - Release these resources upon backend cleanup
> >
> > Signed-off-by: Connor Kite <connorkite@gmail.com>
> > ---
> >   hw/virtio/vhost-user.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 129 insertions(+)
> >
> > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> > index f296b63fb9..710cf966f8 100644
> > --- a/hw/virtio/vhost-user.c
> > +++ b/hw/virtio/vhost-user.c
> > @@ -18,6 +18,7 @@
> >   #include "hw/virtio/vhost-backend.h"
> >   #include "hw/virtio/virtio.h"
> >   #include "hw/virtio/virtio-net.h"
> > +#include "hw/virtio/vhost-iova-tree.h"
> >   #include "chardev/char-fe.h"
> >   #include "io/channel-socket.h"
> >   #include "system/kvm.h"
> > @@ -25,6 +26,7 @@
> >   #include "qemu/main-loop.h"
> >   #include "qemu/uuid.h"
> >   #include "qemu/sockets.h"
> > +#include "qemu/memfd.h"
> >   #include "system/runstate.h"
> >   #include "system/cryptodev.h"
> >   #include "migration/postcopy-ram.h"
> > @@ -320,6 +322,13 @@ static VhostUserMsg m __attribute__ ((unused));
> >   /* The version of the protocol we support */
> >   #define VHOST_USER_VERSION    (0x1)
> >
> > +typedef struct IsolationRegion {
> > +    uint64_t base_addr;
> > +    uint64_t vring_base_addr;
> > +    uint64_t size;
> > +    int iso_fd;
> > +} IsolationRegion;
> > +
> >   struct vhost_user {
> >       struct vhost_dev *dev;
> >       /* Shared between vhost devs of the same virtio device */
> > @@ -353,6 +362,10 @@ struct vhost_user {
> >        * by the backend (see @features).
> >        */
> >       uint64_t protocol_features;
> > +
> > +    /* Isolated memory data*/
> > +    struct IsolationRegion iso_memory;
> > +    VhostIOVATree *iso_iova_tree;
> >   };
> >
> >   struct scrub_regions {
> > @@ -1109,6 +1122,121 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
> >       return 0;
> >   }
> >
> > +/* TODO: Is there any notifier cleanup required here?*/
> > +static void cleanup_isolation_regions(struct vhost_dev *dev)
> > +{
> > +    struct vhost_user *u = dev->opaque;
> > +    if (u->iso_memory.base_addr) {
> > +        vhost_iova_tree_delete(u->iso_iova_tree);
> > +        u->iso_iova_tree = NULL;
> > +        memset(&u->iso_memory, 0, sizeof(IsolationRegion));
> > +        qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
> > +                         u->iso_memory.iso_fd);
>
> u->iso_memory is zeroed before they are read here.
>
> > +        u->iso_memory.base_addr = 0;
> > +    }
> > +}
> > +
> > +__attribute__((unused))
> > +static int init_isolation_regions(struct vhost_dev *dev,
> > +                                  VhostUserMsg *msg,
> > +                                  int *fds, size_t *fd_num)
> > +{
> > +    Error *err = NULL;
> > +    struct vhost_user *u = dev->opaque;
> > +    uint32_t nregions = dev->mem->nregions;
> > +    uint64_t buffer_reg_size = 0;
> > +    DMAMap newEntry = {
>
> This should be new_entry as per: docs/devel/style.rst
>
> > +        .perm = IOMMU_RW
> > +    };
> > +    g_autoptr(GArray) buffer_regions =
> > +        g_array_new(FALSE, TRUE, sizeof(DMAMap));
> > +
> > +    msg->hdr.request = VHOST_USER_SET_MEM_TABLE;
> > +
> > +    /* In case of reset, clear old regions*/
> > +    if (u->iso_memory.base_addr != 0) {
> > +        cleanup_isolation_regions(dev);
> > +        vhost_iova_tree_delete(u->iso_iova_tree);
> > +    }
>
> The check of u->iso_memory.base_addr is redundant;
> cleanup_isolation_regions() performs that check.
>
> cleanup_isolation_regions() also calls vhost_iova_tree_delete(), so the
> duplicate call of the function leads to use-after-free and double-free.
>
> > +
> > +    /* Gather information for bounce buffers to be mapped */
> > +    for (int i = 0; i < nregions; i++) {
> > +        struct vhost_memory_region *dev_region = &dev->mem->regions[i];
> > +        hwaddr size = ROUND_UP(dev_region->memory_size,
> > +                      qemu_real_host_page_size());
> > +        newEntry.translated_addr = dev_region->guest_phys_addr;
> > +        newEntry.size = size - 1;
> > +        buffer_reg_size += size;
> > +        g_array_append_val(buffer_regions, newEntry);
> > +    }
> > +
> > +    int num;
> > +    size_t desc_size;
> > +    size_t avail_size;
> > +    size_t driver_area_size;
> > +    size_t device_area_size;
> > +    size_t total_vring_size = 0;
> > +    size_t total_mmap_size;
> > +
> > +    /* Get space required for all vrings */
> > +    for (int j = 0; j < dev->nvqs; j++) {
> > +        num = virtio_queue_get_num(dev->vdev, dev->vq_index + j);
> > +        desc_size = sizeof(vring_desc_t) * num;
> > +        avail_size = offsetof(vring_avail_t, ring[num]) +
> > +                        sizeof(uint16_t);
> > +        driver_area_size = ROUND_UP(desc_size + avail_size,
> > +                            qemu_real_host_page_size());
> > +        device_area_size = ROUND_UP(offsetof(vring_used_t, ring[num]) +
> > +                                    sizeof(uint16_t),
> > +                                    qemu_real_host_page_size());
> > +        total_vring_size += driver_area_size + device_area_size;
> > +    }
> > +
> > +    total_mmap_size = buffer_reg_size + total_vring_size;
> > +
> > +    /* Allocate and map an anonymous file to hold the isolation region */
> > +    u->iso_memory.base_addr = (uint64_t) qemu_memfd_alloc("iso_r",
> > +                              total_mmap_size,
> > +                              F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL,
> > +                              &u->iso_memory.iso_fd, &err);
> > +    u->iso_memory.size = total_mmap_size;
> > +
> > +    if (err) {
> > +        error_report_err(err);
> > +        cleanup_isolation_regions(dev);
> > +        return -1;
> > +    }
> > +
> > +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
> > +                                                 total_mmap_size - 1));
> > +
> > +    /*
> > +     * Instantiates iova tree sized to map bounce buffers and vrings to the
> > +     * isolation region in host va.
> > +     */
> > +    u->iso_iova_tree = vhost_iova_tree_new(u->iso_memory.base_addr, last_addr);
> > +
> > +    assert(&u->iso_memory.iso_fd >= 0);
> > +    DMAMap *map;
> > +    DMAMap vring_map = {
> > +        .perm = IOMMU_RW,
> > +        .size = total_vring_size - 1,
> > +        /*vrings are allocated on tree first, so will be assigned base addr*/
> > +        .translated_addr = u->iso_memory.base_addr
> > +    };
> > +
> > +    vhost_iova_tree_map_alloc(u->iso_iova_tree, &vring_map,
> > +                              vring_map.translated_addr);
> > +    u->iso_memory.vring_base_addr = vring_map.iova;
> > +    for (int i = 0; i < buffer_regions->len; i++) {
> > +        map = &g_array_index(buffer_regions, DMAMap, i);
> > +        vhost_iova_tree_map_alloc_gpa(u->iso_iova_tree, map,
> > +                                      map->translated_addr);
>
> I wonder if it is fine to dismiss return values of
> vhost_iova_tree_map_alloc() and vhost_iova_tree_map_alloc_gpa().
>
> Regards,
> Akihiko Odaki
>
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> >   static int vhost_user_set_mem_table(struct vhost_dev *dev,
> >                                       struct vhost_memory *mem)
> >   {
> > @@ -2681,6 +2809,7 @@ static int vhost_user_backend_cleanup(struct vhost_dev *dev)
> >       g_free(u->region_rb_offset);
> >       u->region_rb_offset = NULL;
> >       u->region_rb_len = 0;
> > +    cleanup_isolation_regions(dev);
> >       g_free(u);
> >       dev->opaque = 0;
> >
> >
>

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Hanna Czenczek]

On 24.07.26 00:30, ConKite wrote:
> In support of future work to add a memory isolation mode to vhost-user devices, a
> qdev property, "memory-isolation", is added to all vhost-user devices. This is
> simply a bool representing whether the mode is active or not.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>   hw/block/vhost-user-blk.c            | 1 +
>   hw/display/vhost-user-gpu.c          | 1 +
>   hw/scsi/vhost-user-scsi.c            | 1 +
>   hw/virtio/vhost-user-base.c          | 3 ++-
>   hw/virtio/vhost-user-fs.c            | 1 +
>   hw/virtio/vhost-user-scmi.c          | 1 +
>   hw/virtio/vhost-user-vsock.c         | 1 +
>   include/hw/virtio/vhost-user-base.h  | 1 +
>   include/hw/virtio/vhost-user-blk.h   | 1 +
>   include/hw/virtio/vhost-user-fs.h    | 1 +
>   include/hw/virtio/vhost-user-scmi.h  | 1 +
>   include/hw/virtio/vhost-user-vsock.h | 1 +
>   include/hw/virtio/virtio-gpu.h       | 1 +
>   include/hw/virtio/virtio-scsi.h      | 1 +
>   14 files changed, 15 insertions(+), 1 deletion(-)

Should `vhost_user_vga_inst_initfn()` set up an alias for 
"memory-isolation", like it does for "chardev" now?

There are four lines in here that the checkpatch script notes to be over 
80 characters in length, so they should be broken up.

[...]

> diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
> index 90bd2903db..d2a74a2860 100644
> --- a/hw/virtio/vhost-user-base.c
> +++ b/hw/virtio/vhost-user-base.c
> @@ -409,7 +409,8 @@ static void vub_device_unrealize(DeviceState *dev)
>   
>   /*Define common qdev properties.  Inherited by all children*/
>   static const Property vub_properties[] = {
> -    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
> +    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserBase, memory_isolation, false)

Nitpick: I would add a comma here. (This hunk shows specifically why 
it’s a good idea: So that future additions don’t have to change the line 
to add the comma.)

Hanna

Re: [PATCH RFC 04/15] net/vhost-user: add memory isolation

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Add a memory-isolation property to NetdevPassOptions and
> NetDevVhostUserOptions in net.json.  This property will
> be used for vhost-user devices to disable access to guest
> memory from devices.
>
> It will be passed into vhost_user_init() in a future patch.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   net/passt.c      |  7 +++++++
>   net/vhost-user.c |  4 ++++
>   qapi/net.json    | 15 +++++++++++----
>   3 files changed, 22 insertions(+), 4 deletions(-)

[...]

> diff --git a/qapi/net.json b/qapi/net.json
> index 1a6382825c..4588c7014b 100644
> --- a/qapi/net.json
> +++ b/qapi/net.json
> @@ -213,6 +213,9 @@
>   #
>   # @param: parameter to pass to passt command
>   #
> +# @memory-isolation: enable memory isolation.  Only has effect if
> +#     using vhost-user.  (default: false)
> +#
>   # Since: 10.1
>   ##
>   { 'struct': 'NetdevPasstOptions',
> @@ -250,7 +253,8 @@
>       '*ipv6':               'bool',
>       '*tcp-ports':          ['PasstPortForward'],
>       '*udp-ports':          ['PasstPortForward'],
> -    '*param':              ['PasstParameter'] },
> +    '*param':              ['PasstParameter'],
> +    '*memory-isolation':   'bool'},
>       'if': 'CONFIG_PASST' }

I have a gut feeling there ought to be a better way to add an option the 
only works for vhost-user, but maybe not. I certainly don’t have a 
constructive suggestions. :)

(Apart from that, my only comments would be the QAPI “since” field 
missing, as Markus noted on the previous patch, and the patch ordering 
Stefan has noted.)

Hanna

Re: [PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Add a memory_isolation bool to the VhostUserState struct. This
> bool is set within vhost_user_init(), which takes a
> memory_isolation bool as an argument.
>
> Refactor all call locations of vhost_user_init to include the new
> argument.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   backends/cryptodev-vhost-user.c     |  2 +-
>   backends/vhost-user.c               |  4 ++--
>   hw/block/vhost-user-blk.c           |  3 ++-
>   hw/display/vhost-user-gpu.c         |  3 ++-
>   hw/scsi/vhost-user-scsi.c           |  3 ++-
>   hw/virtio/vhost-stub.c              |  3 ++-
>   hw/virtio/vhost-user-base.c         |  3 ++-
>   hw/virtio/vhost-user-fs.c           |  3 ++-
>   hw/virtio/vhost-user-scmi.c         |  3 ++-
>   hw/virtio/vhost-user-vsock.c        |  3 ++-
>   hw/virtio/vhost-user.c              |  5 ++++-
>   include/hw/virtio/vhost-user.h      |  6 +++++-
>   include/system/vhost-user-backend.h |  3 ++-
>   net/passt.c                         | 13 ++++++++-----
>   net/vhost-user.c                    |  9 +++++----
>   15 files changed, 43 insertions(+), 23 deletions(-)

[...]

> diff --git a/net/passt.c b/net/passt.c
> index ce80186883..a3021f37ef 100644
> --- a/net/passt.c
> +++ b/net/passt.c

[...]

> @@ -735,14 +738,14 @@ int net_init_passt(const Netdev *netdev, const char *name,
>       s->pidfile = pidfile;
>   
>       if (netdev->u.passt.has_vhost_user && netdev->u.passt.vhost_user) {
> -        bool memory_isolation G_GNUC_UNUSED = false;
> +        bool memory_isolation = false;
>   
>           if (netdev->u.passt.has_memory_isolation &&
> -           netdev->u.passt.memory_isolation) {
> +            netdev->u.passt.memory_isolation) {
>               memory_isolation = true;
>           }

I think this part of the hunk should be squashed into HEAD^.

Hanna

> -        if (net_passt_vhost_user_init(s, errp) == -1) {
> +        if (net_passt_vhost_user_init(s, memory_isolation, errp) == -1) {
>               qemu_del_net_client(nc);
>               return -1;
>           }

Re: [PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Add a wrapper to iova-tree to call g_tree_foreach.  Provies a means to
> execute some function on each node of the iova-tree.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   include/qemu/iova-tree.h | 13 +++++++++++++
>   util/iova-tree.c         |  5 +++++
>   2 files changed, 18 insertions(+)
>
> diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h
> index 14e82a22d5..4d77f4dc57 100644
> --- a/include/qemu/iova-tree.h
> +++ b/include/qemu/iova-tree.h
> @@ -161,4 +161,17 @@ int iova_tree_alloc_map(IOVATree *tree, DMAMap *map, hwaddr iova_begin,
>    */
>   void iova_tree_destroy(IOVATree *tree);
>   
> +/**
> + * iova_tree_foreach:
> + *
> + * @tree: the iova tree to traverse
> + * @func: the function called at every node. A true return value ends traversal

Can you document roughly what the function looks like? Because the type 
(`GTraverseFunc`) is not defined in qemu. I know it has a boolean as a 
return type, and that one argument of it is going to be `gpointer data`, 
but surely it will also take a tree element.

Hanna

> + * @data: an argument passed to func
> + *
> + * Wrapper to call g_tree_foreach on underlying g_tree
> + *
> + * Return: None
> + */
> +void iova_tree_foreach(IOVATree *tree, GTraverseFunc func, gpointer data);
> +
>   #endif

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Adds a handler callback to shadow virtqueues in order to enable
> non-default processing of used vring elements from the device.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
>   hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
>   2 files changed, 18 insertions(+)

The avail handler replaces `vhost_svq_add_element()`. I find it a bit 
unintuitive that this handler does not replace `virtqueue_fill()`, i.e. 
that it is not a “mirror” of the avail handler; that the avail handler 
actually handles avail elements, whereas the used handler is “just” a 
callback on each used element. Was that intentional? (If so, I think the 
comment should be explicit about this choice.)

Hanna

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> By default svq vrings are placed in an anonymous memory map. As svqs
> will be leveraged to enable memory isolation in vhost-user, it is useful
> to be able to place the vrings in a shared isolation memory region.
>
> Adds the option to specify vring placement by providing a vring base
> address before starting the svq.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c | 22 +++++++++++++++-------
>   hw/virtio/vhost-shadow-virtqueue.h |  3 +++
>   2 files changed, 18 insertions(+), 7 deletions(-)

I would go beyond Stefan and say that I do really ask for explicit 
bounds checking on the area.

Besides that, `vhost_svq_stop()` still always calls `munmap()` on `desc` 
and `used` and that must absolutely not be done in case a fixed location 
was provided.

Also, it would be nicer to have a function to set the base address 
instead of having to modify the svq object directly.

Hanna

Re: [PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> iova-tree expects inclusive range sizing when maps are allocated or searched.
> Currently, svqs use exclusive sizing when searching their
> vhost-iova-tree for a match to the region to be translated.  This could
> lead to errors if the region to be translated is at the edge of an iova
> region.
>
> Fix this by reducing `needle.size` by 1 in
> vhost_svq_translate_addr to bring then it line with DMAMap and iova-tree
> convention.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-shadow-virtqueue.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c
> index 9e3c359f50..20e5c7d2f1 100644
> --- a/hw/virtio/vhost-shadow-virtqueue.c
> +++ b/hw/virtio/vhost-shadow-virtqueue.c
> @@ -104,14 +104,14 @@ static bool vhost_svq_translate_addr(const VhostShadowVirtqueue *svq,
>               /* Search the GPA->IOVA tree */
>               needle = (DMAMap) {
>                   .translated_addr = gpas[i],
> -                .size = iovec[i].iov_len,
> +                .size = iovec[i].iov_len - 1,  /* Inclusive */
>               };
>               map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
>           } else {
>               /* Search the IOVA->HVA tree */
>               needle = (DMAMap) {
>                   .translated_addr = (hwaddr)(uintptr_t)iovec[i].iov_base,
> -                .size = iovec[i].iov_len,
> +                .size = iovec[i].iov_len - 1, /* Inclusive */
>               };
>               map = vhost_iova_tree_find_iova(svq->iova_tree, &needle);
>           }

It’s not immediately obvious what is ensuring that `iov_len` can never 
be 0. Sure, it would be wrong and makes no sense, but that is why I 
think an `assert(iovec[i].iov_len > 0)` would be appropriate.

(Looks like `virtqueue_map_desc()` is what rejects zero length, but that 
is not really local to this code path, so not immediately obvious.)

Hanna

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> If memory isolation mode is active for the vhost-user device adds
> features to:
> - Gather the size required for bounce buffers and vrings in shared
>    isolation region
> - Allocate the required space in an anonymous file
> - Create a vhost-iova-tree with space to map entire isolation region
> - Map guest memory regions and shared vrings into the tree
> - Release these resources upon backend cleanup
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 129 insertions(+)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index f296b63fb9..710cf966f8 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>   #include "hw/virtio/vhost-backend.h"
>   #include "hw/virtio/virtio.h"
>   #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-iova-tree.h"
>   #include "chardev/char-fe.h"
>   #include "io/channel-socket.h"
>   #include "system/kvm.h"
> @@ -25,6 +26,7 @@
>   #include "qemu/main-loop.h"
>   #include "qemu/uuid.h"
>   #include "qemu/sockets.h"
> +#include "qemu/memfd.h"
>   #include "system/runstate.h"
>   #include "system/cryptodev.h"
>   #include "migration/postcopy-ram.h"
> @@ -320,6 +322,13 @@ static VhostUserMsg m __attribute__ ((unused));
>   /* The version of the protocol we support */
>   #define VHOST_USER_VERSION    (0x1)
>   
> +typedef struct IsolationRegion {
> +    uint64_t base_addr;
> +    uint64_t vring_base_addr;
> +    uint64_t size;
> +    int iso_fd;
> +} IsolationRegion;
> +

Going beyond Stefan, I would say please document each field with a 
proper doc comment, and ideally the whole struct, too.

>   struct vhost_user {
>       struct vhost_dev *dev;
>       /* Shared between vhost devs of the same virtio device */
> @@ -353,6 +362,10 @@ struct vhost_user {
>        * by the backend (see @features).
>        */
>       uint64_t protocol_features;
> +
> +    /* Isolated memory data*/

In general: Please put spaces after the leading /* and before the 
trailing */.

> +    struct IsolationRegion iso_memory;
> +    VhostIOVATree *iso_iova_tree;

Patch 13 adds more variables to this, so I think it may make sense to 
have a dedicated struct for these fields.

>   };
>   
>   struct scrub_regions {
> @@ -1109,6 +1122,121 @@ static int vhost_user_set_mem_table_postcopy(struct vhost_dev *dev,
>       return 0;
>   }
>   
> +/* TODO: Is there any notifier cleanup required here?*/
> +static void cleanup_isolation_regions(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    if (u->iso_memory.base_addr) {
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +        u->iso_iova_tree = NULL;
> +        memset(&u->iso_memory, 0, sizeof(IsolationRegion));
> +        qemu_memfd_free((gpointer) u->iso_memory.base_addr, u->iso_memory.size,
> +                         u->iso_memory.iso_fd);

As Akihiko said, the memset() should come after this, and also...

> +        u->iso_memory.base_addr = 0;

...this is just a subset of the memset().

> +    }
> +}
> +
> +__attribute__((unused))
> +static int init_isolation_regions(struct vhost_dev *dev,
> +                                  VhostUserMsg *msg,
> +                                  int *fds, size_t *fd_num)
> +{
> +    Error *err = NULL;
> +    struct vhost_user *u = dev->opaque;
> +    uint32_t nregions = dev->mem->nregions;
> +    uint64_t buffer_reg_size = 0;
> +    DMAMap newEntry = {
> +        .perm = IOMMU_RW
> +    };
> +    g_autoptr(GArray) buffer_regions =
> +        g_array_new(FALSE, TRUE, sizeof(DMAMap));
> +
> +    msg->hdr.request = VHOST_USER_SET_MEM_TABLE;
> +
> +    /* In case of reset, clear old regions*/
> +    if (u->iso_memory.base_addr != 0) {
> +        cleanup_isolation_regions(dev);
> +        vhost_iova_tree_delete(u->iso_iova_tree);
> +    }

(What Akihiko said)

> +
> +    /* Gather information for bounce buffers to be mapped */
> +    for (int i = 0; i < nregions; i++) {
> +        struct vhost_memory_region *dev_region = &dev->mem->regions[i];
> +        hwaddr size = ROUND_UP(dev_region->memory_size,
> +                      qemu_real_host_page_size());
> +        newEntry.translated_addr = dev_region->guest_phys_addr;
> +        newEntry.size = size - 1;
> +        buffer_reg_size += size;
> +        g_array_append_val(buffer_regions, newEntry);
> +    }
> +
> +    int num;
> +    size_t desc_size;
> +    size_t avail_size;
> +    size_t driver_area_size;
> +    size_t device_area_size;
> +    size_t total_vring_size = 0;
> +    size_t total_mmap_size;

Please do not mix variable declarations and normal code (style.rst calls 
it “Mixed declarations”).

> +
> +    /* Get space required for all vrings */
> +    for (int j = 0; j < dev->nvqs; j++) {

Why j here and i above and below?

> +        num = virtio_queue_get_num(dev->vdev, dev->vq_index + j);
> +        desc_size = sizeof(vring_desc_t) * num;
> +        avail_size = offsetof(vring_avail_t, ring[num]) +
> +                        sizeof(uint16_t);
> +        driver_area_size = ROUND_UP(desc_size + avail_size,
> +                            qemu_real_host_page_size());
> +        device_area_size = ROUND_UP(offsetof(vring_used_t, ring[num]) +
> +                                    sizeof(uint16_t),
> +                                    qemu_real_host_page_size());
> +        total_vring_size += driver_area_size + device_area_size;
> +    }
> +
> +    total_mmap_size = buffer_reg_size + total_vring_size;
> +
> +    /* Allocate and map an anonymous file to hold the isolation region */
> +    u->iso_memory.base_addr = (uint64_t) qemu_memfd_alloc("iso_r",
> +                              total_mmap_size,
> +                              F_SEAL_GROW | F_SEAL_SHRINK | F_SEAL_SEAL,
> +                              &u->iso_memory.iso_fd, &err);
> +    u->iso_memory.size = total_mmap_size;
> +
> +    if (err) {
> +        error_report_err(err);
> +        cleanup_isolation_regions(dev);
> +        return -1;
> +    }
> +
> +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
> +                                                 total_mmap_size - 1));

I would like a comment why you went for a 128-bit operation here. I 
assume it is because it checks for overflow, turning overflow into an 
assertion failure, and it can be assumed `qemu_memfd_alloc()` must 
naturally return a pointer such that adding the length of the allocated 
area to it (minus one) will never overflow?

The question is, do we even need to check for overflow then. Not that I 
mind it, in principle, I just find it non-obvious. (The more obvious 
check (imho) would be to just do a 64-bit operation and then 
assert(last_addr >= u->iso_memory.base_addr).)

> +
> +    /*
> +     * Instantiates iova tree sized to map bounce buffers and vrings to the
> +     * isolation region in host va.
> +     */
> +    u->iso_iova_tree = vhost_iova_tree_new(u->iso_memory.base_addr, last_addr);
> +
> +    assert(&u->iso_memory.iso_fd >= 0);
> +    DMAMap *map;
> +    DMAMap vring_map = {
> +        .perm = IOMMU_RW,
> +        .size = total_vring_size - 1,
> +        /*vrings are allocated on tree first, so will be assigned base addr*/
> +        .translated_addr = u->iso_memory.base_addr
> +    };
> +
> +    vhost_iova_tree_map_alloc(u->iso_iova_tree, &vring_map,
> +                              vring_map.translated_addr);
> +    u->iso_memory.vring_base_addr = vring_map.iova;
> +    for (int i = 0; i < buffer_regions->len; i++) {
> +        map = &g_array_index(buffer_regions, DMAMap, i);

God, I *really*, *really* hate this, and find it really disgusting that 
the documentation actually recommends doing this (`&g_array_index()`) 
instead of just offering a separate macro to get a reference.

And existing qemu code does it all over the place, too.

So I cannot really fault you for it.

Still. Too ugly for me to keep completely silent about it.

</rant>

Hanna

> +        vhost_iova_tree_map_alloc_gpa(u->iso_iova_tree, map,
> +                                      map->translated_addr);
> +    }
> +
> +    return 0;
> +}
> +
>   static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                       struct vhost_memory *mem)
>   {
> @@ -2681,6 +2809,7 @@ static int vhost_user_backend_cleanup(struct vhost_dev *dev)
>       g_free(u->region_rb_offset);
>       u->region_rb_offset = NULL;
>       u->region_rb_len = 0;
> +    cleanup_isolation_regions(dev);
>       g_free(u);
>       dev->opaque = 0;
>   
>

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Adds features to fill a vhost_user_set_mem_table message with the
> addresses of isolation memory regions corresponding to bounce buffers
> and vrings.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 79 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 710cf966f8..acabfb7f1c 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -625,6 +625,21 @@ static void vhost_user_fill_msg_region(struct vhost_dev *dev,
>       dst->mmap_offset = mmap_offset;
>   }
>   
> +static void vhost_user_fill_msg_region_iso(VhostUserMemoryRegion *dst,
> +                                           const struct vhost_user *u,
> +                                           const struct vhost_memory_region
> +                                           *iova_reg)
> +{
> +    assert(u != NULL && dst != NULL && iova_reg != NULL);
> +    uint64_t offset;
> +
> +    offset = iova_reg->userspace_addr - u->iso_memory.base_addr;
> +    dst->userspace_addr = iova_reg->userspace_addr;
> +    dst->memory_size = iova_reg->memory_size;
> +    dst->guest_phys_addr = iova_reg->userspace_addr;
> +    dst->mmap_offset = offset;
> +}
> +
>   static int vhost_user_fill_set_mem_table_msg(struct vhost_user *u,
>                                                struct vhost_dev *dev,
>                                                VhostUserMsg *msg,
> @@ -1136,7 +1151,33 @@ static void cleanup_isolation_regions(struct vhost_dev *dev)
>       }
>   }
>   
> -__attribute__((unused))
> +struct iova_tree_traversal_args {
> +    VhostUserMsg *msg;
> +    struct vhost_user *u;
> +    int *fds;
> +    size_t *fd_num;
> +};
> +
> +static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
> +                                                    gpointer value,
> +                                                    gpointer data)

This function name doesn’t really say what the function does, and more 
describes its type.

> +{
> +    struct iova_tree_traversal_args *args = data;
> +    struct vhost_memory_region msg_region;
> +    VhostUserMemoryRegion region_buffer;
> +    DMAMap *map = key;
> +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;

Generally: I find it quite hard to read code where general statements 
come after a list of declarations without an empty line, especially if 
there is an empty line later. Visually, this last line to me looks like 
another variable declaration.

> +
> +    msg_region.guest_phys_addr = map->iova;
> +    msg_region.memory_size = map->size + 1;
> +    msg_region.userspace_addr = map->iova;
> +    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
> +    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;

Why does `vhost_user_fill_msg_region_iso()` exist when this is the only 
caller, and we could just fill `regions[i]` here directly?

> +    (*args->fd_num)++;

As noted by Akihiko, there needs to be some bounds checking. At least 
against `VHOST_MEMORY_BASELINE_NREGIONS`, but even better against the 
array size of `args->msg->payload.memory.regions` and the caller should 
pass the size of `fds[]` via `args`.

(And I’m also with Stefan, something with `index` would be better. Maybe 
`region_idx`.)

> +
> +    return false;
> +}
> +
>   static int init_isolation_regions(struct vhost_dev *dev,
>                                     VhostUserMsg *msg,
>                                     int *fds, size_t *fd_num)
> @@ -1234,6 +1275,24 @@ static int init_isolation_regions(struct vhost_dev *dev,
>                                         map->translated_addr);
>       }
>   
> +    struct iova_tree_traversal_args args = {
> +        .fd_num = fd_num,
> +        .fds = fds,
> +        .msg = msg,
> +        .u = u
> +    };

Personal preference: I would like `*fd_num = 0` somewhere here, or 
`assert(*fd_num == 0)` before the iteration. Yes, the caller initializes 
it to zero, but it is just not obvious here, and it seems necessary for 
correct semantics.

(And the open question whether `fd_num` should be renamed here, too, or 
can keep that name. Not sure.)

> +
> +    vhost_iova_tree_foreach(u->iso_iova_tree,
> +                            vhost_user_iova_tree_traverse_funct, &args);
> +
> +    msg->payload.memory.nregions = *fd_num;
> +
> +    assert(*fd_num != 0);

Am I wrong or should `*fd_num == nregions`? Is it possible that 
`*fd_num` is less because regions end up joined? Can you add a comment 
on when they would differ?

Hanna

> +
> +    msg->hdr.size = sizeof(msg->payload.memory.nregions);
> +    msg->hdr.size += sizeof(msg->payload.memory.padding);
> +    msg->hdr.size += *fd_num * sizeof(VhostUserMemoryRegion);
> +
>       return 0;
>   }

[...]

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> Adds shadow virtqueues that will eventually be used to transfer data
> between device and host via bounce buffers when isolation mode is
> active.  The svqs are initalized, and eventfd assignments are
> intercepted so that notifications come to svqs first before
> the guest or backend receive them.
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 54 insertions(+)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index acabfb7f1c..75858289a2 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -18,6 +18,7 @@
>   #include "hw/virtio/vhost-backend.h"
>   #include "hw/virtio/virtio.h"
>   #include "hw/virtio/virtio-net.h"
> +#include "hw/virtio/vhost-shadow-virtqueue.h"
>   #include "hw/virtio/vhost-iova-tree.h"
>   #include "chardev/char-fe.h"
>   #include "io/channel-socket.h"
> @@ -365,7 +366,9 @@ struct vhost_user {
>   
>       /* Isolated memory data*/
>       struct IsolationRegion iso_memory;
> +    GPtrArray *shadow_vqs;
>       VhostIOVATree *iso_iova_tree;
> +    bool svqs_allocated;
>   };
>   
>   struct scrub_regions {
> @@ -1693,6 +1696,24 @@ static int vhost_set_vring_file(struct vhost_dev *dev,
>   static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>                                        struct vhost_vring_file *file)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);

Bounds checking via `vhost_user_get_vq_index()` would be nice. (Same below.)

> +        vhost_svq_set_svq_kick_fd(svq, file->fd);
> +
> +        if (svq->hdev_kick.initialized == false) {
> +            int r = event_notifier_init(&svq->hdev_kick, 0);
> +            if (r) {
> +                error_report("Failed to create kick event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_kick);

What if `file->fd` was -1, i.e. `VHOST_FILE_UNBIND`? Should we put a 
real FD here then or just continue with -1?

> +    }
> +
>       int ret = vhost_set_vring_file(dev, VHOST_USER_SET_VRING_KICK, file);
>       if (ret < 0) {
>           return ret;
> @@ -1721,6 +1742,24 @@ static int vhost_user_set_vring_kick(struct vhost_dev *dev,
>   static int vhost_user_set_vring_call(struct vhost_dev *dev,
>                                        struct vhost_vring_file *file)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    int svq_idx = file->index - dev->vq_index;
> +    if (u->user->memory_isolation) {
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);
> +        vhost_svq_set_svq_call_fd(svq, file->fd);
> +
> +        if (svq->hdev_call.initialized == false) {
> +            int r = event_notifier_init(&svq->hdev_call, 0);

Do we need to check if the call event FD is already active?

> +            if (r) {
> +                error_report("Failed to create call event notifier");
> +                return r;
> +            }
> +        }
> +
> +        file->fd = event_notifier_get_fd(&svq->hdev_call);
> +    }
> +
>       return vhost_set_vring_file(dev, VHOST_USER_SET_VRING_CALL, file);
>   }
>   
> @@ -2715,6 +2754,17 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>       return 0;
>   }
>   
> +static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
> +{
> +    /*Modified from vhost-vdpa*/
> +    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
> +    for (int i = 0; i < dev->nvqs; i++) {
> +        VhostShadowVirtqueue *svq;
> +        svq = vhost_svq_new(NULL, NULL);

Patch 15 adds clean-up for this; I would squash it into here, as far as 
possible. Ideally, a patch series can be stopped at any step and not 
break anything, so if an allocation is added, the accompanying freeing 
should come with it.

Hanna

> +        g_ptr_array_add(u->shadow_vqs, svq);
> +    }
> +}
> +
>   static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>                                      Error **errp)
>   {
> @@ -2859,6 +2909,10 @@ static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque,
>       u->postcopy_notifier.notify = vhost_user_postcopy_notifier;
>       postcopy_add_notifier(&u->postcopy_notifier);
>   
> +    if (vus->memory_isolation) {
> +        vhost_user_init_svq(dev, u);
> +    }
> +
>       return 0;
>   }
>   
>

Re: [PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Hanna Czenczek]

On 24.07.26 00:30, Connor Kite wrote:
> - Add start logic for shadow virtqueues, which sets vring addresses.
> - Update logic for sending vring addresses to backend to point
>    to the shadow vrings when isolation mode is active.
> - Implement handlers for intercepted avail and used descriptors. These
>    handlers copy buffer contents between bounce buffers in the isolation
>    region and the buffers made available by the guest
> - Implement logic to stop svqs
>
> Signed-off-by: Connor Kite <connorkite@gmail.com>
> ---
>   hw/virtio/vhost-user.c | 167 ++++++++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 166 insertions(+), 1 deletion(-)

In general, I find it strange to use two completely separate data 
structures for the address translation in either direction (avail/used). 
I suppose we could have a map of in-flight translations, adding entries 
in avail, and looking for them in used, removing them from the map.

I mean, sure, right now, we can calculate the addresses, because we 
“duplicate” the whole range of guest memory for each device, but that is 
clearly a waste and it would be better to use a smaller size in the 
future; and then each request will need to allocate from that smaller 
device memory, get a random address, and that will need such a map for 
tracking.

> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 75858289a2..e65f877f9a 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1299,6 +1299,97 @@ static int init_isolation_regions(struct vhost_dev *dev,
>       return 0;
>   }
>   
> +static int vhost_user_memory_lookup(struct vhost_dev *dev, hwaddr gpa,
> +                                    hwaddr *hva)
> +{
> +    int i;
> +    hwaddr offset;
> +
> +    for (i = 0; i < dev->mem->nregions; i++) {
> +        struct vhost_memory_region *reg = dev->mem->regions + i;
> +
> +        if (gpa >= reg->guest_phys_addr &&
> +            reg->guest_phys_addr + reg->memory_size > gpa) {
> +            offset = gpa - reg->guest_phys_addr;
> +            *hva = reg->userspace_addr + offset;
> +            return 0;
> +        }
> +    }
> +
> +    return -EFAULT;
> +}
> +
> +static int vhost_user_svq_handle_used(VhostShadowVirtqueue *svq,
> +                                      VirtQueueElement *elem,
> +                                      void *opaque)
> +{
> +    hwaddr hva;
> +    int r;
> +    struct vhost_dev *dev = opaque;
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->in_addr[i], &hva);

This function only looks for the starting address, so it is not 
guaranteed that the whole buffer is within one memory region.

> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->in_sg[i].iov_base, elem->in_sg[i].iov_len);
> +        elem->in_sg[i].iov_base = (void *) hva;
> +    }
> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        r = vhost_user_memory_lookup(dev, elem->out_addr[i], &hva);
> +        if (r < 0) {
> +            return r;
> +        }
> +
> +        memcpy((void *) hva, elem->out_sg[i].iov_base, elem->out_sg[i].iov_len);

I don’t think we need this `memcpy()` here, as the device may only write 
to IN buffers, not OUT buffers.

> +        elem->out_sg[i].iov_base = (void *) hva;
> +    }
> +
> +    return 0;
> +}
> +
> +static int vhost_user_svq_handle_avail(VhostShadowVirtqueue *svq,
> +                                       VirtQueueElement *elem,
> +                                       void *opaque)
> +{
> +    hwaddr offset;
> +    const DMAMap *map;
> +    DMAMap needle;
> +    hwaddr *iova_base;
> +
> +    for (int i = 0; i < elem->out_num; i++) {
> +        needle.translated_addr = elem->out_addr[i];
> +        needle.size = elem->out_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);

Is it guaranteed that the returned `map` will fully contain `needle`?

> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->out_sg[i].iov_base = iova_base;
> +        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);
> +    }
> +
> +    for (int i = 0; i < elem->in_num; i++) {
> +        needle.translated_addr = elem->in_addr[i];
> +        needle.size = elem->in_sg[i].iov_len - 1;
> +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
> +        offset = needle.translated_addr - map->translated_addr;
> +        iova_base = (void *)(map->iova + offset);
> +
> +        elem->in_sg[i].iov_base = iova_base;
> +        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);

I don’t think we need this `memcpy()` here as the device is to strictly 
write to IN buffers, not read from them.

> +    }
> +
> +    vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->out_addr,
> +                  elem->in_sg, elem->in_num, elem->in_addr, elem);

Might make sense to make vhost_svq_add_element() public and use it here.

> +
> +    return 0;
> +}
> +
> +
> +
> +
>   static int vhost_user_set_mem_table(struct vhost_dev *dev,
>                                       struct vhost_memory *mem)
>   {
> @@ -1772,6 +1863,8 @@ static int vhost_user_set_vring_err(struct vhost_dev *dev,
>   static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>                                        struct vhost_vring_addr *addr)
>   {
> +    struct vhost_user *u = dev->opaque;
> +
>       VhostUserMsg msg = {
>           .hdr.request = VHOST_USER_SET_VRING_ADDR,
>           .hdr.flags = VHOST_USER_VERSION,
> @@ -1779,6 +1872,25 @@ static int vhost_user_set_vring_addr(struct vhost_dev *dev,
>           .hdr.size = sizeof(msg.payload.addr),
>       };
>   
> +    if (u->user->memory_isolation) {
> +        if (!u->svqs_allocated) {
> +            return 0;
> +        }
> +
> +        int svq_idx = addr->index - dev->vq_index;
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> +                                                      svq_idx);

Again, bounds checking via `vhost_user_get_vq_index()` would be good.

> +
> +        struct vhost_vring_addr svq_addr = {
> +            .avail_user_addr = (uint64_t)(uintptr_t)svq->vring.avail,
> +            .desc_user_addr = (uint64_t)(uintptr_t)svq->vring.desc,
> +            .used_user_addr = (uint64_t)(uintptr_t)svq->vring.used,
> +            .index = addr->index,
> +        };
> +
> +        msg.payload.addr = svq_addr;
> +    }
> +
>       /*
>        * wait for a reply if logging is enabled to make sure
>        * backend is actually logging changes
> @@ -2754,13 +2866,18 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier,
>       return 0;
>   }
>   
> +static const VhostShadowVirtqueueOps vhost_user_svq_ops = {
> +    .avail_handler = vhost_user_svq_handle_avail,
> +    .used_handler = vhost_user_svq_handle_used
> +};
> +
>   static void vhost_user_init_svq(struct vhost_dev *dev, struct vhost_user *u)
>   {
>       /*Modified from vhost-vdpa*/
>       u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
>       for (int i = 0; i < dev->nvqs; i++) {
>           VhostShadowVirtqueue *svq;
> -        svq = vhost_svq_new(NULL, NULL);
> +        svq = vhost_svq_new(&vhost_user_svq_ops, dev);
>           g_ptr_array_add(u->shadow_vqs, svq);
>       }
>   }
> @@ -3466,8 +3583,56 @@ void vhost_user_async_close(DeviceState *d,
>       }
>   }
>   
> +static bool vhost_user_svqs_start(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +    uint64_t vring_base = u->iso_memory.vring_base_addr;
> +    u->svqs_allocated = true;

I don’t like setting this before it actually being the case very much. I 
understand that it is necessary for how you decided to set up 
`vhost_user_set_vring_addr()`, but there should at least be a comment 
(because reading this code, there is the obvious question: Why is this 
set when it is not true?); or the code should be different. I.e. having 
a `vhost_user_set_shadow_vring_addr()` function, maybe, which would not 
even take an `addr` argument and could be used both below and in 
`vhost_user_set_vring_addr()`.

> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
> +        svq->base_addr = (hwaddr *) vring_base;

I really do not like the lack of bounds checking here again. There 
should be something, somewhere (i.e. in the “Isolated memory data” 
portion, maybe in `struct IsolationRegion`), that tells how much memory 
has been allocated for the shadow virtqueues, and we need to check that 
we do not overflow.

> +        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);
> +
> +        struct vhost_vring_addr addr = {
> +            .index = dev->vq_index + i,
> +            .desc_user_addr = vring_base,
> +            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
> +                svq->vring.num,
> +            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)

All values set here are effectively ignored, right? Because 
`vhost_user_set_vring_addr()` just overwrites them. (Having a 
`vhost_user_set_shadow_vring_addr()` could maybe save us from having to 
set this up.)

Not that it’s currently wrong to set them this way, just asking.

Hanna

> +        };
> +
> +        vhost_user_set_vring_addr(dev, &addr);
> +
> +        vring_base += vhost_svq_device_area_size(svq) +
> +                      vhost_svq_driver_area_size(svq);
> +    }
> +
> +    return false;
> +}
> +
> +static void vhost_user_svqs_stop(struct vhost_dev *dev)
> +{
> +    struct vhost_user *u = dev->opaque;
> +
> +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> +        vhost_svq_stop(g_ptr_array_index(u->shadow_vqs, i));
> +    }
> +}
> +
> +
>   static int vhost_user_dev_start(struct vhost_dev *dev, bool started)
>   {
> +    struct vhost_user *u = dev->opaque;
> +    if (u->user->memory_isolation) {
> +        if (started) {
> +            vhost_user_svqs_start(dev);
> +        } else {
> +            vhost_user_svqs_stop(dev);
> +        }
> +    }
> +
>       if (!vhost_user_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_STATUS)) {
>           return 0;
>       }
>

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

On Tue, Jul 28, 2026 at 10:59 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >
> > +typedef struct IsolationRegion {
> > +    uint64_t base_addr;
> > +    uint64_t vring_base_addr;
> > +    uint64_t size;
> > +    int iso_fd;
> > +} IsolationRegion;
>
> The purpose of the base_addr and vring_base_addr fields is not obvious.
> I suggest adjusting the types, names, and adding comments to make the
> purpose clearer:
>
>   typedef struct {
>       void *mem;           /* mapped shared memory */
>       size_t size;
>       uint64_t vring_iova;
>       int fd;              /* shared memory fd */
>   } IsolationRegion;
>

Agreed on adding clarification.  Is there a reason to change from uint64_t
to void * for referencing the shared memory region address?  Most uses of
that variable expect uint64_t currently.  Here's what I am currently thinking:

typedef struct IsolationRegion {
    uint64_t shared_mem_addr; /* mapped shared memory */
    uint64_t vring_iova_addr; /* beginning of vring region in shared memory */
    size_t size;
    int fd; /* shared memory iova */
} IsolationRegion;

...
> > +
> > +    /* Isolated memory data*/
> > +    struct IsolationRegion iso_memory;
>
> struct is not necessary since there is a typedef:
>
>   IsolationRegion iso_memory;
>

Agreed!  This will be fixed.

> > +    VhostIOVATree *iso_iova_tree;
>
> The IOVA tree seems to be closely used with IsolationRegion. Maybe this
> field should move into IsolationRegion?
>

Agreed.  Since the IOVA tree is basically tracking mappings to the
isolation regions, these can comfortably be grouped together.

...
>
> nregions is uint32_t, so i should also be uint32_t to avoid
> signed/unsigned comparisons.
>

Agreed.  Will fix this.

...

>
> Please avoid duplicating the memory layout calculations. When packed
> vring support is added to vhost-shadow-virtqueue.c this will become more
> complex and it should be done in a single place. vhost-shadow-virtque.c
> should expose an API for the size calculation.
>

I think I originally did this because the svqs weren't initialized yet
at this stage.
However, I can cut down on the duplication with a new function in
vhost-shadow-virtqueue that makes a dummy svq with the correct num value
and calls the existing functions to get device and driver area sizes.
...

> Please make the name unique (e.g. using dev->vdev->name).
>

I will do this by prepending "iso_mem_" to dev->vdev->name.

...

>
> Is it possible to use 0 as the IOVA base address so that QEMU's
> addresses aren't leaked to the vhost-user back-end? It's good security
> practice not to reveal memory addresses to the outside world because
> that information can be used to defeat address space randomization or
> infer memory addresses of other data structures.
>

That should be possible.

> > +
> > +    assert(&u->iso_memory.iso_fd >= 0);
>
> The dereference operator should not be used here, it's the iso_fd value
> that is being tested.
>

Got it.  This will be fixed.

> > +    DMAMap *map;
> > +    DMAMap vring_map = {
> > +        .perm = IOMMU_RW,
> > +        .size = total_vring_size - 1,
> > +        /*vrings are allocated on tree first, so will be assigned base addr*/
> > +        .translated_addr = u->iso_memory.base_addr
>
> Why is this field assigned here, I think this field is used as the
> output of vhost_iova_tree_map_alloc() rather than an input (e.g. see
> vhost_vdpa_svq_map_rings())?
>

Yes, I suppose this doesn't need to be assigned here.  As it currently stands,
that value is just getting assigned back to itself in
vhost_iova_tree_map_alloc().  I'll fix this for the values in
`buffer_regions`, too

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

On Mon, Aug 3, 2026 at 6:25 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> > +typedef struct IsolationRegion {
> > +    uint64_t base_addr;
> > +    uint64_t vring_base_addr;
> > +    uint64_t size;
> > +    int iso_fd;
> > +} IsolationRegion;
> > +
> Going beyond Stefan, I would say please document each field with a
> proper doc comment, and ideally the whole struct, too.
>

Roger.  I will comment all fields, except for perhaps size, which seems
self-explanatory.

...

> In general: Please put spaces after the leading /* and before the
> trailing */.
>

Roger!  I believe I have this in a few places and will correct.

...

>
> Patch 13 adds more variables to this, so I think it may make sense to
> have a dedicated struct for these fields.
>

I agree that some of these make sense to be consolidated. However,
I think I will keep the svq array outside of the struct as the svqs don't live
inside of the isolation region.

I guess the alternative would be to have an `isolation_mode_ctx`
type of struct to hold all of the above.

>
> As Akihiko said, the memset() should come after this, and also...
>
> > +        u->iso_memory.base_addr = 0;
>
> ...this is just a subset of the memset().
>

Yes, the cleanup was not fully baked ahead of the RFC post.
This is currently fixed on my end and will make it into the next
rev.

...

> > +    int num;
> > +    size_t desc_size;
> > +    size_t avail_size;
> > +    size_t driver_area_size;
> > +    size_t device_area_size;
> > +    size_t total_vring_size = 0;
> > +    size_t total_mmap_size;
>
> Please do not mix variable declarations and normal code (style.rst calls
> it “Mixed declarations”).
>

Got it!  This will be fixed.

> > +
> > +    /* Get space required for all vrings */
> > +    for (int j = 0; j < dev->nvqs; j++) {
>
> Why j here and i above and below?
>

Most likely this was inside another loop on the initial implementation, and the
iteration variable name was not changed when the outer loop was removed.
I will change this for next rev.

...

> > +
> > +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
> > +                                                 total_mmap_size - 1));
>
> I would like a comment why you went for a 128-bit operation here. I
> assume it is because it checks for overflow, turning overflow into an
> assertion failure, and it can be assumed `qemu_memfd_alloc()` must
> naturally return a pointer such that adding the length of the allocated
> area to it (minus one) will never overflow?
>
> The question is, do we even need to check for overflow then. Not that I
> mind it, in principle, I just find it non-obvious. (The more obvious
> check (imho) would be to just do a 64-bit operation and then
> assert(last_addr >= u->iso_memory.base_addr).)
>

At the time I was concerned about potential overflow prior to application of
the -1 operand in case the last memory location was at 0xFFFFFFFFFFFFFFFF.
However, in that case, the -1 would cause the value to wrap back around before
any use of the value, so my concern was not needed.

There is a similar 128-bit op in the existing code in
vhost-shadow-virtqueue.c at
line 133.  To my eye, that might also be a good candidate for
simplification unless
I'm missing some other aspect of the use case.

...

> > +        map = &g_array_index(buffer_regions, DMAMap, i);
>
> God, I *really*, *really* hate this, and find it really disgusting that
> the documentation actually recommends doing this (`&g_array_index()`)
> instead of just offering a separate macro to get a reference.
>
> And existing qemu code does it all over the place, too.
>
> So I cannot really fault you for it.
>
> Still. Too ugly for me to keep completely silent about it.
>
> </rant>
>
> Hanna
>

Would you typically avoid using g_array / g_array_index?

Best,
Connor

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Connor Kite]

On Mon, Aug 3, 2026 at 5:21 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> Should `vhost_user_vga_inst_initfn()` set up an alias for
> "memory-isolation", like it does for "chardev" now?
>

I am not 100% sure but I can test to see.  If an alias is needed, I suspect
that would also apply to vhost-user-gpu-pci, right?

>
> There are four lines in here that the checkpatch script notes to be over
> 80 characters in length, so they should be broken up.
>

Got it.  This will be fixed for the next rev.

> [...]
>
> > diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
> > index 90bd2903db..d2a74a2860 100644
> > --- a/hw/virtio/vhost-user-base.c
> > +++ b/hw/virtio/vhost-user-base.c
> > @@ -409,7 +409,8 @@ static void vub_device_unrealize(DeviceState *dev)
> >
> >   /*Define common qdev properties.  Inherited by all children*/
> >   static const Property vub_properties[] = {
> > -    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
> > +    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
> > +    DEFINE_PROP_BOOL("memory-isolation", VHostUserBase, memory_isolation, false)
>
> Nitpick: I would add a comma here. (This hunk shows specifically why
> it’s a good idea: So that future additions don’t have to change the line
> to add the comma.)
>

Ah, I was wondering why I was seeing so many hanging commas .  That makes sense.
I will add the comma in.

Re: [PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Connor Kite]

On Mon, Aug 3, 2026 at 5:36 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
>
> Can you document roughly what the function looks like? Because the type
> (`GTraverseFunc`) is not defined in qemu. I know it has a boolean as a
> return type, and that one argument of it is going to be `gpointer data`,
> but surely it will also take a tree element.
>
> Hanna

I have updated the comment to describe the usage of GTraverseFunc as
below:

/**
* iova_tree_foreach:
*
* @tree: the iova tree to traverse
* @func: the function called at every node. A true return value ends traversal.
*              Arguments:
*                  gpointer key: key of current node
*                  gpointer value: value of current node
*                  gpointer data: data passed in at g_tree_foreach call
*              Return: gboolean
* @data: an argument passed to func
*
* Wrapper to call g_tree_foreach on the underlying g_tree. g_tree_foreach will
* traverse through every node of the tree, barring an early return. At each
* node, g_tree_foreach calls func with pointers to the node key, node value,
* and a pointer to data passed in by the caller.
*
* Return: None
*/

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Hanna Czenczek]

On 03.08.26 22:32, Connor Kite wrote:
> On Mon, Aug 3, 2026 at 5:21 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>> Should `vhost_user_vga_inst_initfn()` set up an alias for
>> "memory-isolation", like it does for "chardev" now?
>>
> I am not 100% sure but I can test to see.  If an alias is needed, I suspect
> that would also apply to vhost-user-gpu-pci, right?

Right, that makes sense. (If your testing confirms that it’s needed.)

Hanna

>> There are four lines in here that the checkpatch script notes to be over
>> 80 characters in length, so they should be broken up.
>>
> Got it.  This will be fixed for the next rev.
>
>> [...]
>>
>>> diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c
>>> index 90bd2903db..d2a74a2860 100644
>>> --- a/hw/virtio/vhost-user-base.c
>>> +++ b/hw/virtio/vhost-user-base.c
>>> @@ -409,7 +409,8 @@ static void vub_device_unrealize(DeviceState *dev)
>>>
>>>    /*Define common qdev properties.  Inherited by all children*/
>>>    static const Property vub_properties[] = {
>>> -    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev)
>>> +    DEFINE_PROP_CHR("chardev", VHostUserBase, chardev),
>>> +    DEFINE_PROP_BOOL("memory-isolation", VHostUserBase, memory_isolation, false)
>> Nitpick: I would add a comma here. (This hunk shows specifically why
>> it’s a good idea: So that future additions don’t have to change the line
>> to add the comma.)
>>
> Ah, I was wondering why I was seeing so many hanging commas .  That makes sense.
> I will add the comma in.
>

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Hanna Czenczek]

On 03.08.26 21:31, Connor Kite wrote:
> On Mon, Aug 3, 2026 at 6:25 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>>> +typedef struct IsolationRegion {
>>> +    uint64_t base_addr;
>>> +    uint64_t vring_base_addr;
>>> +    uint64_t size;
>>> +    int iso_fd;
>>> +} IsolationRegion;
>>> +
>> Going beyond Stefan, I would say please document each field with a
>> proper doc comment, and ideally the whole struct, too.
>>
> Roger.  I will comment all fields, except for perhaps size, which seems
> self-explanatory.
>
> ...
>
>> In general: Please put spaces after the leading /* and before the
>> trailing */.
>>
> Roger!  I believe I have this in a few places and will correct.
>
> ...
>
>> Patch 13 adds more variables to this, so I think it may make sense to
>> have a dedicated struct for these fields.
>>
> I agree that some of these make sense to be consolidated. However,
> I think I will keep the svq array outside of the struct as the svqs don't live
> inside of the isolation region.
>
> I guess the alternative would be to have an `isolation_mode_ctx`
> type of struct to hold all of the above.

Yes, that’s what I thought, to have an object that would hold everything 
that is used by the isolation code (and rather specific to the isolation 
code).

>> As Akihiko said, the memset() should come after this, and also...
>>
>>> +        u->iso_memory.base_addr = 0;
>> ...this is just a subset of the memset().
>>
> Yes, the cleanup was not fully baked ahead of the RFC post.
> This is currently fixed on my end and will make it into the next
> rev.
>
> ...
>
>>> +    int num;
>>> +    size_t desc_size;
>>> +    size_t avail_size;
>>> +    size_t driver_area_size;
>>> +    size_t device_area_size;
>>> +    size_t total_vring_size = 0;
>>> +    size_t total_mmap_size;
>> Please do not mix variable declarations and normal code (style.rst calls
>> it “Mixed declarations”).
>>
> Got it!  This will be fixed.
>
>>> +
>>> +    /* Get space required for all vrings */
>>> +    for (int j = 0; j < dev->nvqs; j++) {
>> Why j here and i above and below?
>>
> Most likely this was inside another loop on the initial implementation, and the
> iteration variable name was not changed when the outer loop was removed.
> I will change this for next rev.
>
> ...
>
>>> +
>>> +    uint64_t last_addr = int128_get64(int128_add(u->iso_memory.base_addr,
>>> +                                                 total_mmap_size - 1));
>> I would like a comment why you went for a 128-bit operation here. I
>> assume it is because it checks for overflow, turning overflow into an
>> assertion failure, and it can be assumed `qemu_memfd_alloc()` must
>> naturally return a pointer such that adding the length of the allocated
>> area to it (minus one) will never overflow?
>>
>> The question is, do we even need to check for overflow then. Not that I
>> mind it, in principle, I just find it non-obvious. (The more obvious
>> check (imho) would be to just do a 64-bit operation and then
>> assert(last_addr >= u->iso_memory.base_addr).)
>>
> At the time I was concerned about potential overflow prior to application of
> the -1 operand in case the last memory location was at 0xFFFFFFFFFFFFFFFF.
> However, in that case, the -1 would cause the value to wrap back around before
> any use of the value, so my concern was not needed.
>
> There is a similar 128-bit op in the existing code in
> vhost-shadow-virtqueue.c at
> line 133.  To my eye, that might also be a good candidate for
> simplification unless
> I'm missing some other aspect of the use case.

But that’s a guest-provided length, no? I believe the 128-bit operation 
there is done as a genuine check of invalid guest requests. Whereas 
here, `qemu_memfd_alloc()` already must not return a pointer such that 
adding the length of the allocated buffer would overflow (because it 
guarantees that the whole range from the pointer through the given 
length is valid, right).

> ...
>
>>> +        map = &g_array_index(buffer_regions, DMAMap, i);
>> God, I *really*, *really* hate this, and find it really disgusting that
>> the documentation actually recommends doing this (`&g_array_index()`)
>> instead of just offering a separate macro to get a reference.
>>
>> And existing qemu code does it all over the place, too.
>>
>> So I cannot really fault you for it.
>>
>> Still. Too ugly for me to keep completely silent about it.
>>
>> </rant>
>>
>> Hanna
>>
> Would you typically avoid using g_array / g_array_index?

I mean, in this case, I would have just used a plain C array because the 
size is pre-determined by `nregions`, right.

In general, *I* have not used GArray yet (no particular reason), which 
is why I had not come in contact with the `&g_array_index()` pattern 
yet, which had me stare in disbelief at first. (My main problem being 
that the name `g_array_index` is written in lowercase and thus indicates 
being a normal function, whereas macros in C are generally written in 
uppercase to allow at-a-glance distinction. So it pretends to be a 
normal function, but glib recommends a pattern of use 
(`&g_array_index()`) that would never work with a normal function, and 
that I find a dreadful decision by glib.)

Hanna

Re: [PATCH RFC 06/15] util/iova-tree: g_tree_foreach wrapper

[Hanna Czenczek]

On 03.08.26 23:13, Connor Kite wrote:
> On Mon, Aug 3, 2026 at 5:36 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>>
>> Can you document roughly what the function looks like? Because the type
>> (`GTraverseFunc`) is not defined in qemu. I know it has a boolean as a
>> return type, and that one argument of it is going to be `gpointer data`,
>> but surely it will also take a tree element.
>>
>> Hanna
> I have updated the comment to describe the usage of GTraverseFunc as
> below:
>
> /**
> * iova_tree_foreach:
> *
> * @tree: the iova tree to traverse
> * @func: the function called at every node. A true return value ends traversal.
> *              Arguments:
> *                  gpointer key: key of current node
> *                  gpointer value: value of current node
> *                  gpointer data: data passed in at g_tree_foreach call
> *              Return: gboolean
> * @data: an argument passed to func
> *
> * Wrapper to call g_tree_foreach on the underlying g_tree. g_tree_foreach will
> * traverse through every node of the tree, barring an early return. At each
> * node, g_tree_foreach calls func with pointers to the node key, node value,
> * and a pointer to data passed in by the caller.
> *
> * Return: None
> */

Awesome, thanks!

Hanna

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On Mon, Aug 03, 2026 at 11:26:29AM -0700, Connor Kite wrote:
> On Tue, Jul 28, 2026 at 10:59 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > >
> > > +typedef struct IsolationRegion {
> > > +    uint64_t base_addr;
> > > +    uint64_t vring_base_addr;
> > > +    uint64_t size;
> > > +    int iso_fd;
> > > +} IsolationRegion;
> >
> > The purpose of the base_addr and vring_base_addr fields is not obvious.
> > I suggest adjusting the types, names, and adding comments to make the
> > purpose clearer:
> >
> >   typedef struct {
> >       void *mem;           /* mapped shared memory */
> >       size_t size;
> >       uint64_t vring_iova;
> >       int fd;              /* shared memory fd */
> >   } IsolationRegion;
> >
> 
> Agreed on adding clarification.  Is there a reason to change from uint64_t
> to void * for referencing the shared memory region address?  Most uses of
> that variable expect uint64_t currently.  Here's what I am currently thinking:

The shared memory region address is a C pointer where the shared memory
is mmapped and the type for an arbitrary C pointer is void *. The casts
needed around qemu_memfd_alloc() and qemu_memfd_free() suggest that
uint64_t is not the natural type for this.

Keeping this as a C pointer helps differentiate the address from an
IOVA, making the code clearer.

Also, with the comments about base_addr below addressed, the
hwaddr/uint64_t uses will be reduced.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1156 bytes --]

On Mon, Aug 03, 2026 at 03:25:19PM +0200, Hanna Czenczek wrote:
> > +    for (int i = 0; i < buffer_regions->len; i++) {
> > +        map = &g_array_index(buffer_regions, DMAMap, i);
> 
> God, I *really*, *really* hate this, and find it really disgusting that the
> documentation actually recommends doing this (`&g_array_index()`) instead of
> just offering a separate macro to get a reference.
> 
> And existing qemu code does it all over the place, too.
> 
> So I cannot really fault you for it.
> 
> Still. Too ugly for me to keep completely silent about it.
> 
> </rant>

Do we need GArray here, maybe just allocate a C array with g_new()
instead?

The reason I didn't say anything was because I expected GArray to
perform bounds checking. Plain C arrays don't offer that. After reading
your comment and looking at the glib source, I realized it does not.
There is no way to handle an error and glib does not abort() when the
index is invalid.

GArray still has the advantage of allowing elements to be added
dynamically, but in this case we know the number of elements ahead of
time and don't need that feature.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH RFC 04/15] net/vhost-user: add memory isolation

[Connor Kite]

On Mon, Jul 27, 2026 at 11:48 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
>
> This is dead code (hence G_GNUC_UNUSED) and is harder to review because
> we don't know how it will be used until later in the patch series. I
> guess vhost_user_init() and net_passt_vhost_user_init() will gain a
> memory_isolation argument in a future patch.
>
> Linear reviews of the patch series are easiest for reviewers. Please
> reorder this so that vhost_user_init() and net_pass_vhost_user_init()
> are changed first. You can hardcode the argument to false in those
> patches. This patch can be moved afterwards so that memory_isolation is
> never dead code.

Now fixed for the next rev.  This and the previous patch are swapped,
with hardcoded values of memory_isolation used until the property is added
to net.json.

On Mon, Aug 3, 2026 at 5:27 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
>
> I have a gut feeling there ought to be a better way to add an option the
> only works for vhost-user, but maybe not. I certainly don’t have a
> constructive suggestions. :)
>
> (Apart from that, my only comments would be the QAPI “since” field
> missing, as Markus noted on the previous patch, and the patch ordering
> Stefan has noted.)
>
> Hanna
>

Added "Since 11.2" to both locations in net.json.

Re: [PATCH RFC 05/15] vhost-user: add memory_isolation to VhostUserState

[Connor Kite]

On Mon, Aug 3, 2026 at 5:29 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> > -           netdev->u.passt.memory_isolation) {
> > +            netdev->u.passt.memory_isolation) {
> >               memory_isolation = true;
> >           }
>
> I think this part of the hunk should be squashed into HEAD^.
>
> Hanna
>

And squashed!  This patch has also been swapped with the previous one to
avoid dead code.

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Connor Kite]

On Fri, Jul 24, 2026 at 4:29 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
>
> The error of the handler that may have left elem stale is ignored here.
>
> Regards,
> Akihiko Odaki

I have now added in a check that returns on error without setting the guest
notifier.  So essentially the call is ignored.  Maybe this could be expanded
on to handle errors more gracefully depending on the error code.

On Tue, Jul 28, 2026 at 8:06 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> >
> > +/**
> > + * Callback to handle a used buffer
> > + *
> > + * @svq: Shadow virtqueue
> > + * @elem: Element placed in the queue by the device
> > + * @vq_callback_opaque: Used to pass arguments to callback
> > + *
> > + * Returns 0 if the vq is running as expected
>
> The doc comment should mention what non-0 look like. Are errors reported
> as a negative errno?
>

Yes, a negative errno is expected on failure.  I have added clarification to the
comment.

On Mon, Aug 3, 2026 at 5:42 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> On 24.07.26 00:30, Connor Kite wrote:
> > Adds a handler callback to shadow virtqueues in order to enable
> > non-default processing of used vring elements from the device.
> >
> > Signed-off-by: Connor Kite <connorkite@gmail.com>
> > ---
> >   hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
> >   hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
> >   2 files changed, 18 insertions(+)
>
> The avail handler replaces `vhost_svq_add_element()`. I find it a bit
> unintuitive that this handler does not replace `virtqueue_fill()`, i.e.
> that it is not a “mirror” of the avail handler; that the avail handler
> actually handles avail elements, whereas the used handler is “just” a
> callback on each used element. Was that intentional? (If so, I think the
> comment should be explicit about this choice.)
>
> Hanna
>

I implemented it as a more narrowly-focused callback since virtqueue_fill
needs to be called regardless.  If I instead called virtqueue_fill
from the handler,
I would need to pass in the elem index and length via svq->ops_opaque,
which is a bit awkward as vhost_user defines ops_opaque when registering
the callback.  That's certainly not impossible to do, but this way felt cleaner.
For now, I will add a comment explaining that the callback should not be used
to add elements to the underlying vq's used ring.

Re: [PATCH RFC 02/15] vhost-user: Add memory-isolation qdev property to vhost-user devices

[Connor Kite]

On Tue, Aug 4, 2026 at 3:46 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> On 03.08.26 22:32, Connor Kite wrote:
> > On Mon, Aug 3, 2026 at 5:21 AM Hanna Czenczek <hreitz@redhat.com> wrote:
> >> Should `vhost_user_vga_inst_initfn()` set up an alias for
> >> "memory-isolation", like it does for "chardev" now?
> >>
> > I am not 100% sure but I can test to see.  If an alias is needed, I suspect
> > that would also apply to vhost-user-gpu-pci, right?
>
> Right, that makes sense. (If your testing confirms that it’s needed.)
>
> Hanna
>

Testing with both vhost-user-gpu-pcu and vhost-user-vga, the
memory_isolation property is getting
properly set in vhost_user without adding an alias.  In both cases,
dev->vdev refers to
a VhostUserGpu, which has the memory_isolation qdev property set as in
other vhost-user front-ends.

Unlike most vhost-user front-ends, vhost-user-gpu does not define a
"chardev" qdev property.  Maybe that
is why the "chardev" property needs to be aliased?  However, I don't
understand why "chardev" is not a qdev
property as vhost-user-gpu already makes use of many of them.

Best,
Connor

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Hanna Czenczek]

On 06.08.26 04:50, Connor Kite wrote:
> On Fri, Jul 24, 2026 at 4:29 AM Akihiko Odaki
> <odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>>
>> The error of the handler that may have left elem stale is ignored here.
>>
>> Regards,
>> Akihiko Odaki
> I have now added in a check that returns on error without setting the guest
> notifier.  So essentially the call is ignored.  Maybe this could be expanded
> on to handle errors more gracefully depending on the error code.
>
> On Tue, Jul 28, 2026 at 8:06 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>> +/**
>>> + * Callback to handle a used buffer
>>> + *
>>> + * @svq: Shadow virtqueue
>>> + * @elem: Element placed in the queue by the device
>>> + * @vq_callback_opaque: Used to pass arguments to callback
>>> + *
>>> + * Returns 0 if the vq is running as expected
>> The doc comment should mention what non-0 look like. Are errors reported
>> as a negative errno?
>>
> Yes, a negative errno is expected on failure.  I have added clarification to the
> comment.
>
> On Mon, Aug 3, 2026 at 5:42 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>> On 24.07.26 00:30, Connor Kite wrote:
>>> Adds a handler callback to shadow virtqueues in order to enable
>>> non-default processing of used vring elements from the device.
>>>
>>> Signed-off-by: Connor Kite <connorkite@gmail.com>
>>> ---
>>>    hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
>>>    hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
>>>    2 files changed, 18 insertions(+)
>> The avail handler replaces `vhost_svq_add_element()`. I find it a bit
>> unintuitive that this handler does not replace `virtqueue_fill()`, i.e.
>> that it is not a “mirror” of the avail handler; that the avail handler
>> actually handles avail elements, whereas the used handler is “just” a
>> callback on each used element. Was that intentional? (If so, I think the
>> comment should be explicit about this choice.)
>>
>> Hanna
>>
> I implemented it as a more narrowly-focused callback since virtqueue_fill
> needs to be called regardless.  If I instead called virtqueue_fill
> from the handler,
> I would need to pass in the elem index and length via svq->ops_opaque,
> which is a bit awkward as vhost_user defines ops_opaque when registering
> the callback.  That's certainly not impossible to do, but this way felt cleaner.
> For now, I will add a comment explaining that the callback should not be used
> to add elements to the underlying vq's used ring.

Then I personally wouldn’t call it a “handler” if it does not handle the 
element like the avail handler does. Maybe “callback”.

Hanna

Re: [PATCH RFC 08/15] hw/virtio/vhost-shadow-virtqueue: used handler

[Connor Kite]

On Thu, Aug 6, 2026 at 3:28 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> On 06.08.26 04:50, Connor Kite wrote:
> > On Fri, Jul 24, 2026 at 4:29 AM Akihiko Odaki
> > <odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
> >>
> >> The error of the handler that may have left elem stale is ignored here.
> >>
> >> Regards,
> >> Akihiko Odaki
> > I have now added in a check that returns on error without setting the guest
> > notifier.  So essentially the call is ignored.  Maybe this could be expanded
> > on to handle errors more gracefully depending on the error code.
> >
> > On Tue, Jul 28, 2026 at 8:06 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> >>> +/**
> >>> + * Callback to handle a used buffer
> >>> + *
> >>> + * @svq: Shadow virtqueue
> >>> + * @elem: Element placed in the queue by the device
> >>> + * @vq_callback_opaque: Used to pass arguments to callback
> >>> + *
> >>> + * Returns 0 if the vq is running as expected
> >> The doc comment should mention what non-0 look like. Are errors reported
> >> as a negative errno?
> >>
> > Yes, a negative errno is expected on failure.  I have added clarification to the
> > comment.
> >
> > On Mon, Aug 3, 2026 at 5:42 AM Hanna Czenczek <hreitz@redhat.com> wrote:
> >> On 24.07.26 00:30, Connor Kite wrote:
> >>> Adds a handler callback to shadow virtqueues in order to enable
> >>> non-default processing of used vring elements from the device.
> >>>
> >>> Signed-off-by: Connor Kite <connorkite@gmail.com>
> >>> ---
> >>>    hw/virtio/vhost-shadow-virtqueue.c |  4 ++++
> >>>    hw/virtio/vhost-shadow-virtqueue.h | 14 ++++++++++++++
> >>>    2 files changed, 18 insertions(+)
> >> The avail handler replaces `vhost_svq_add_element()`. I find it a bit
> >> unintuitive that this handler does not replace `virtqueue_fill()`, i.e.
> >> that it is not a “mirror” of the avail handler; that the avail handler
> >> actually handles avail elements, whereas the used handler is “just” a
> >> callback on each used element. Was that intentional? (If so, I think the
> >> comment should be explicit about this choice.)
> >>
> >> Hanna
> >>
> > I implemented it as a more narrowly-focused callback since virtqueue_fill
> > needs to be called regardless.  If I instead called virtqueue_fill
> > from the handler,
> > I would need to pass in the elem index and length via svq->ops_opaque,
> > which is a bit awkward as vhost_user defines ops_opaque when registering
> > the callback.  That's certainly not impossible to do, but this way felt cleaner.
> > For now, I will add a comment explaining that the callback should not be used
> > to add elements to the underlying vq's used ring.
>
> Then I personally wouldn’t call it a “handler” if it does not handle the
> element like the avail handler does. Maybe “callback”.
>
> Hanna
>

I see how that's more descriptive.  Updating the name.

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Connor Kite]

On Fri, Jul 24, 2026 at 5:19 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
...
>
> Nit: please remove the whitespace between (void *) and svq->base_addr.
>

Removing the whitespace!

> > +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
> > +        svq->vring.used = (void *)((char *)svq->base_addr +
> > +                          vhost_svq_driver_area_size(svq));
> > +    }
>
> Unmapping svq here can lead to use-after-unmapping because the ring is
> not stopped yet. do_vhost_dev_stop() calls do_vhost_virtqueue_stop()
> only after calling hdev->vhost_ops->vhost_dev_start(hdev, false).
>

I am afraid I don't quite understand, as this patch does not introduce
any unmapping.
Was this comment meant for a different patch?

...

> > +
> > +    /* Location assigned to vrings if not in default anon memory map*/
>
> Nit: here please add a whitespace before */
>

The whitespace is now added in.

Thanks!

Connor

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Akihiko Odaki]

On 2026/08/07 0:29, Connor Kite wrote:
> On Fri, Jul 24, 2026 at 5:19 AM Akihiko Odaki
> <odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
> ...
>>
>> Nit: please remove the whitespace between (void *) and svq->base_addr.
>>
> 
> Removing the whitespace!
> 
>>> +        svq->vring.avail = (void *)((char *)svq->vring.desc + desc_size);
>>> +        svq->vring.used = (void *)((char *)svq->base_addr +
>>> +                          vhost_svq_driver_area_size(svq));
>>> +    }
>>
>> Unmapping svq here can lead to use-after-unmapping because the ring is
>> not stopped yet. do_vhost_dev_stop() calls do_vhost_virtqueue_stop()
>> only after calling hdev->vhost_ops->vhost_dev_start(hdev, false).
>>
> 
> I am afraid I don't quite understand, as this patch does not introduce
> any unmapping.
> Was this comment meant for a different patch?

Yes, sorry, this comment was meant for "[PATCH RFC 14/15] 
hw/virtio/vhost-user: handle data movement with shadow vqs".

vhost_user_dev_start(dev, false) calls vhost_user_svqs_stop(), which 
calls vhost_svq_stop() and unmaps the shadow vring. However, 
do_vhost_dev_stop() calls do_vhost_virtqueue_stop() only after the 
callback returns. The backend vring therefore remains active until 
VHOST_USER_GET_VRING_BASE, so it may still access the unmapped memory.

Regards,
Akihiko Odaki

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Connor Kite]

On Tue, Jul 28, 2026 at 8:23 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> Why is base_addr a hwaddr? This is QEMU memory, not guest RAM. I
> expected this to be void *.
>

It doesn't need to be hwaddr.  I will change it to void *

> A size needs to be at least documented here to reduce the chance of
> memory bugs. Even better would be an interface like
> vhost_svq_vring_total_size() so the caller can fetch the number of bytes
> before allocating the memory.

I haved added in a vhost_svq_vring_total_size function to address this now.

Best,
Connor

Re: [PATCH RFC 09/15] hw/virtio/vhost-shadow-virtqueue: specified vring placement

[Connor Kite]

On Mon, Aug 3, 2026 at 5:48 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> On 24.07.26 00:30, Connor Kite wrote:
> > By default svq vrings are placed in an anonymous memory map. As svqs
> > will be leveraged to enable memory isolation in vhost-user, it is useful
> > to be able to place the vrings in a shared isolation memory region.
> >
> > Adds the option to specify vring placement by providing a vring base
> > address before starting the svq.
> >
> > Signed-off-by: Connor Kite <connorkite@gmail.com>
> > ---
> >   hw/virtio/vhost-shadow-virtqueue.c | 22 +++++++++++++++-------
> >   hw/virtio/vhost-shadow-virtqueue.h |  3 +++
> >   2 files changed, 18 insertions(+), 7 deletions(-)
>
> I would go beyond Stefan and say that I do really ask for explicit
> bounds checking on the area.
>
> Besides that, `vhost_svq_stop()` still always calls `munmap()` on `desc`
> and `used` and that must absolutely not be done in case a fixed location
> was provided.
>
> Also, it would be nicer to have a function to set the base address
> instead of having to modify the svq object directly.
>
> Hanna

Got it!  I now check that the last address in the vring area is not
less than the
first address.  vhost_svq_start now returns an int so that success/error can be
communicated to the caller.

I just moved some existing changes to vhost_svq_stop forward to this patch to
just memset the vring area to 0 if static location is used and vring
pointers are not
null.

vhost_svq_set_base_addr has been added in to remove direct adjustment to the svq
base address.

Best,
Connor

Re: [PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Connor Kite]

On Mon, Aug 3, 2026 at 5:56 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
>
> It’s not immediately obvious what is ensuring that `iov_len` can never
> be 0. Sure, it would be wrong and makes no sense, but that is why I
> think an `assert(iovec[i].iov_len > 0)` would be appropriate.
>
> (Looks like `virtqueue_map_desc()` is what rejects zero length, but that
> is not really local to this code path, so not immediately obvious.)
>
> Hanna
>

From the QEMU Coding Style guidelines on error-handling it looks like
errors that
could be initiated by a malfunctioning guest should not cause Qemu to
exit. Would it be
more appropriate to return false when zero-length is detected?  That follows
the existing error path returning up to vhost_handle_guest_kick, which
renders the svq
unresponsive to future kicks.

Best,
Connor

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

On Tue, Aug 4, 2026 at 4:03 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> >
> > I guess the alternative would be to have an `isolation_mode_ctx`
> > type of struct to hold all of the above.
>
> Yes, that’s what I thought, to have an object that would hold everything
> that is used by the isolation code (and rather specific to the isolation
> code).
>

You got it!  A struct to hold all context specific to isolation mode
will be in the next rev.

...

>
> I mean, in this case, I would have just used a plain C array because the
> size is pre-determined by `nregions`, right.
>
> In general, *I* have not used GArray yet (no particular reason), which
> is why I had not come in contact with the `&g_array_index()` pattern
> yet, which had me stare in disbelief at first. (My main problem being
> that the name `g_array_index` is written in lowercase and thus indicates
> being a normal function, whereas macros in C are generally written in
> uppercase to allow at-a-glance distinction. So it pretends to be a
> normal function, but glib recommends a pattern of use
> (`&g_array_index()`) that would never work with a normal function, and
> that I find a dreadful decision by glib.)
>
> Hanna
>

g_array ugliness is now avoided by using g_new0 with nregions to allocate
the memory.

Re: [PATCH RFC 11/15] hw/virtio/vhost-user: create isolation region

[Connor Kite]

On Tue, Aug 4, 2026 at 11:17 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Mon, Aug 03, 2026 at 11:26:29AM -0700, Connor Kite wrote:
> > On Tue, Jul 28, 2026 at 10:59 AM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > > >
> > > > +typedef struct IsolationRegion {
> > > > +    uint64_t base_addr;
> > > > +    uint64_t vring_base_addr;
> > > > +    uint64_t size;
> > > > +    int iso_fd;
> > > > +} IsolationRegion;
> > >
> > > The purpose of the base_addr and vring_base_addr fields is not obvious.
> > > I suggest adjusting the types, names, and adding comments to make the
> > > purpose clearer:
> > >
> > >   typedef struct {
> > >       void *mem;           /* mapped shared memory */
> > >       size_t size;
> > >       uint64_t vring_iova;
> > >       int fd;              /* shared memory fd */
> > >   } IsolationRegion;
> > >
> >
> > Agreed on adding clarification.  Is there a reason to change from uint64_t
> > to void * for referencing the shared memory region address?  Most uses of
> > that variable expect uint64_t currently.  Here's what I am currently thinking:
>
> The shared memory region address is a C pointer where the shared memory
> is mmapped and the type for an arbitrary C pointer is void *. The casts
> needed around qemu_memfd_alloc() and qemu_memfd_free() suggest that
> uint64_t is not the natural type for this.
>
> Keeping this as a C pointer helps differentiate the address from an
> IOVA, making the code clearer.
>
> Also, with the comments about base_addr below addressed, the
> hwaddr/uint64_t uses will be reduced.

Got it!  I will change here and plan to default to void * when representing host
addresses going forward.

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Connor Kite]

On Fri, Jul 24, 2026 at 6:33 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
> > +static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
> > +                                                    gpointer value,
> > +                                                    gpointer data)
>
> It is a bit odd naming. Functions usually do not have "funct" or similar
> suffixes.
>

I was mostly trying to convey that this was getting registered as a callback
to be called on each node of the iova_tree.  For more clarity, I am renaming it
vhost_user_iova_tree_fill_msg_reg.

> > +{
> > +    struct iova_tree_traversal_args *args = data;
> > +    struct vhost_memory_region msg_region;
> > +    VhostUserMemoryRegion region_buffer;
> > +    DMAMap *map = key;
> > +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;
>
> This may overrun the buffer because *args->fd_num is not checked as done
> in vhost_user_fill_set_mem_table_msg() for the non-isolation case.

Got it!  I have added a check here.

...

> >       int fds[VHOST_MEMORY_BASELINE_NREGIONS];
>
> This needs one more element for vring.
>

Good catch.  Fixed!

> >       size_t fd_num = 0;
> >       bool do_postcopy = u->postcopy_listen && u->postcopy_fd.handler;
> > @@ -1268,6 +1328,24 @@ static int vhost_user_set_mem_table(struct vhost_dev *dev,
> >           msg.hdr.flags |= VHOST_USER_NEED_REPLY_MASK;
> >       }
> >
> > +    if (memory_isolation) {
> > +        ret = init_isolation_regions(dev, &msg, fds, &fd_num);
> > +        if (ret < 0) {
> > +            return ret;
> > +        }
> > +
> > +        ret = vhost_user_write(dev, &msg, fds, fd_num);
> > +        if (ret < 0) {
> > +            return ret;
> > +        }
> > +
> > +        if (reply_supported) {
> > +            return process_message_reply(dev, &msg);
> > +        }
> > +
> > +        return 0;
> > +    }
> > +
>
> This does not cover the postcopy path.
>
> It also allocates memory that is never sent for vq_index != 0. The
> isolation regions need to be shared by all vqs. Please see
> vhost_user_write() for the context.
>
> Regards,
> Akihiko Odaki
>
> >       if (config_mem_slots) {
> >           ret = vhost_user_add_remove_regions(dev, &msg, reply_supported, false);
> >           if (ret < 0) {
> >
>

I am less familiar with postcopy, but I think it might make sense to
break out support
for that into a separate patch. For now, I will return -1 in the
postcopy path if isolation
is active.

I'll need to think about how to handle devices like virtio-net and
make some changes to
this and the previous patch so that:
1. I am not allocating a new shared region for each vhost_dev
2. The shared region has enough space for the svq vrings of all devices.

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Connor Kite]

On Tue, Jul 28, 2026 at 12:16 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> >
> > -__attribute__((unused))
> > +struct iova_tree_traversal_args {
>
> QEMU coding style:
>
>   typedef struct {
>       ...
>   } IOVATreeTraversalArgs;
>

...

Got it!  This is now fixed.

> > +    VhostUserMsg *msg;
> > +    struct vhost_user *u;
> > +    int *fds;
> > +    size_t *fd_num;
>
> This field is also used to index into msg->payload.memory.regions[], so
> "fd_num" is a misnomer. I suggest something like "region_idx" or just
> "idx".
>

I'll rename it to idx since it acts as an index to multiple arrays.

> > +};
> > +
> > +static gboolean vhost_user_iova_tree_traverse_funct(gpointer key,
> > +                                                    gpointer value,
> > +                                                    gpointer data)
> > +{
> > +    struct iova_tree_traversal_args *args = data;
> > +    struct vhost_memory_region msg_region;
> > +    VhostUserMemoryRegion region_buffer;
> > +    DMAMap *map = key;
> > +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;
> > +
> > +    msg_region.guest_phys_addr = map->iova;
> > +    msg_region.memory_size = map->size + 1;
> > +    msg_region.userspace_addr = map->iova;
> > +    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
> > +    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;
>
> I'm confused by this code. VhostUserMemoryRegion region_buffer is the
> vhost-user protocol struct that is being filled in, but there is also a
> struct vhost_memory_region msg_region from the Linux kernel headers?
>
> msg_region and vhost_user_fill_msg_region_iso() make it harder to see
> what is going on. Can you open code the region_buffer struct field
> assignments instead?
>

I've consolidated this code significantly now.  Will delete
vhost_user_fill_msg_region_iso as it's logic has been incorporated in
the traversal callback.

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Connor Kite]

On Mon, Aug 10, 2026 at 10:40 AM Connor Kite <connorkite@gmail.com> wrote:
>
> > >       int fds[VHOST_MEMORY_BASELINE_NREGIONS];
> >
> > This needs one more element for vring.
> >
>
> Good catch.  Fixed!
>

Actually, I'm not sure I can add an element here since the size of
VhostUserPayload.memory.regions is defined as fixed to
VHOST_MEMORY_BASELINE_NREGIONS, which is 8.  It seems
plausible that if this is a standard expected size, that adding another
region could violate some assumption in the back-end implementation.
However, I should be able to avoid the issue by combining the vring region
with one adjacent in the IOVA space when sending to the back-end.

Re: [PATCH RFC 12/15] hw/virtio/vhost-user: send isolation regions to device

[Connor Kite]

On Mon, Aug 3, 2026 at 6:46 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
>
> This function name doesn’t really say what the function does, and more
> describes its type.
>

I will change it to vhost_user_fill_msg_reg_from_tree since it is now doing
the filling directly rather than calling another function to do so.

> > +{
> > +    struct iova_tree_traversal_args *args = data;
> > +    struct vhost_memory_region msg_region;
> > +    VhostUserMemoryRegion region_buffer;
> > +    DMAMap *map = key;
> > +    args->fds[*args->fd_num] = args->u->iso_memory.iso_fd;
>
> Generally: I find it quite hard to read code where general statements
> come after a list of declarations without an empty line, especially if
> there is an empty line later. Visually, this last line to me looks like
> another variable declaration.
>

Got it!  The declarations and statements are now getting separated for clarity.

> > +
> > +    msg_region.guest_phys_addr = map->iova;
> > +    msg_region.memory_size = map->size + 1;
> > +    msg_region.userspace_addr = map->iova;
> > +    vhost_user_fill_msg_region_iso(&region_buffer, args->u, &msg_region);
> > +    args->msg->payload.memory.regions[*args->fd_num] = region_buffer;
>
> Why does `vhost_user_fill_msg_region_iso()` exist when this is the only
> caller, and we could just fill `regions[i]` here directly?
>
> > +    (*args->fd_num)++;
>

vhost_user_fill_msg_region_iso existed before the iova_tree was integrated in
the code base.  I agree that the traversal function can absorb it for
added clarity.

> As noted by Akihiko, there needs to be some bounds checking. At least
> against `VHOST_MEMORY_BASELINE_NREGIONS`, but even better against the
> array size of `args->msg->payload.memory.regions` and the caller should
> pass the size of `fds[]` via `args`.
>
> (And I’m also with Stefan, something with `index` would be better. Maybe
> `region_idx`.)
>

With the current state of the refactor, the size of fds should be the same as
dev->mem->nregions, so I will pass that value in via args and do the check.

The fd_num field is now called idx.

> > +
> > +    return false;
> > +}
> > +
> >   static int init_isolation_regions(struct vhost_dev *dev,
> >                                     VhostUserMsg *msg,
> >                                     int *fds, size_t *fd_num)
> > @@ -1234,6 +1275,24 @@ static int init_isolation_regions(struct vhost_dev *dev,
> >                                         map->translated_addr);
> >       }
> >
> > +    struct iova_tree_traversal_args args = {
> > +        .fd_num = fd_num,
> > +        .fds = fds,
> > +        .msg = msg,
> > +        .u = u
> > +    };
>
> Personal preference: I would like `*fd_num = 0` somewhere here, or
> `assert(*fd_num == 0)` before the iteration. Yes, the caller initializes
> it to zero, but it is just not obvious here, and it seems necessary for
> correct semantics.
>
> (And the open question whether `fd_num` should be renamed here, too, or
> can keep that name. Not sure.)
>

I will set *fd_num = 0 here.   I can see how it makes it easier to see
what is going on.

It may make sense to keep it named fd_num here.  Whereas its main use in the
traversal function is as an index, here and in vhost_user_set_mem_table it is
primarily providing a count of the number of elements in
msg->payload.memory.regions
that have been filled.

> > +
> > +    vhost_iova_tree_foreach(u->iso_iova_tree,
> > +                            vhost_user_iova_tree_traverse_funct, &args);
> > +
> > +    msg->payload.memory.nregions = *fd_num;
> > +
> > +    assert(*fd_num != 0);
>
> Am I wrong or should `*fd_num == nregions`? Is it possible that
> `*fd_num` is less because regions end up joined? Can you add a comment
> on when they would differ?
>
> Hanna

I updated this to check against nregions.  In the current revision, I
do not think
this assertion would be reached if there was an issue with allocating elements
on the iova-tree or filling out the message payloads, but the final sanity check
doesn't hurt.

Best,
Connor

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Connor Kite]

On Fri, Jul 24, 2026 at 6:45 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
>
> This is not undone in case of errors.
>

I've added event_notifier_cleanup calls to the errors here.  As Hanna mentioned,
I have svq_cleanup added in a later patch but should probably pull it
forward.  I
believe that currently doesn't clean up notifiers, so I will be sure
to do that as well.

> > +
> > +        if (svq->hdev_kick.initialized == false) {
>
> Simpler: if (!svq->hdev_kick.initialized)
>
> Regards,
> Akihiko Odaki
>

This is now simplified.

Thanks!
Connor

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Connor Kite]

On Tue, Jul 28, 2026 at 12:42 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> > +
> > +        if (svq->hdev_kick.initialized == false) {
> > +            int r = event_notifier_init(&svq->hdev_kick, 0);
>
> Where is event_notifier_cleanup() called?
>

I have added in to the error code paths when calling
vhost_user_set_vring_kick.

I left most cleanup to the end of the patch series, and it was underdeveloped at
the time this went out for RFC, with the intent of getting feedback on
the overall
approach.  In hindsight, it is probably better practice to implement
cleanup at the
same time as the init/start code, per Hanna's comment.

I will pull the svq cleanup code forward to this patch.

> > +            if (r) {
> > +                error_report("Failed to create kick event notifier");
> > +                return r;
> > +            }
> > +        }
> > +
> > +        file->fd = event_notifier_get_fd(&svq->hdev_kick);
>
> Modifying the function argument is probably not expected but
> vhost_virtqueue_start() doesn't use it after this call, so there is no
> immediate problem. It would be safer to have a local struct
> vhost_vring_file that can be modified without affecting the caller's
> copy.
>

That makes sense.  Swapped over to a local struct.

...

> > +
> > +        if (svq->hdev_call.initialized == false) {
> > +            int r = event_notifier_init(&svq->hdev_call, 0);
>
> Where is event_notifier_cleanup() called?
>

Same as when setting the kick.

> > +            if (r) {
> > +                error_report("Failed to create call event notifier");
> > +                return r;
> > +            }
> > +        }
> > +
> > +        file->fd = event_notifier_get_fd(&svq->hdev_call);
>
> Same as above.
>

Swapped over to a local struct as above.

Thanks!
Connor

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Connor Kite]

On Mon, Aug 3, 2026 at 6:52 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>
> > +    struct vhost_user *u = dev->opaque;
> > +    int svq_idx = file->index - dev->vq_index;
> > +    if (u->user->memory_isolation) {
> > +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
> > +                                                      svq_idx);
>
> Bounds checking via `vhost_user_get_vq_index()` would be nice. (Same below.)
>

Adding the bounds checking outside of the memory-isolation only code path, since
file->index wasn't being checked previously.

> > +        if (svq->hdev_kick.initialized == false) {
> > +            int r = event_notifier_init(&svq->hdev_kick, 0);
> > +            if (r) {
> > +                error_report("Failed to create kick event notifier");
> > +                return r;
> > +            }
> > +        }
> > +
> > +        file->fd = event_notifier_get_fd(&svq->hdev_kick);
>
> What if `file->fd` was -1, i.e. `VHOST_FILE_UNBIND`? Should we put a
> real FD here then or just continue with -1?
>

Good point!  I think we need to propagate the -1 to the backend device, and
run event_notifier_cleanup on the existing svq->hdev_kick.

...

> > +
> > +        if (svq->hdev_call.initialized == false) {
> > +            int r = event_notifier_init(&svq->hdev_call, 0);
>
> Do we need to check if the call event FD is already active?
>

Do you mean, do we need to call event_notifier_test_and_clear to see if it's
been set and potentially run the call handler at this stage?

If so, this might be where some of the intricacies of when things happen in
vhost-user vs in vhost would require some extra care, unless isolation mode
implementation moved up into vhost.

...

> > +    /*Modified from vhost-vdpa*/
> > +    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
> > +    for (int i = 0; i < dev->nvqs; i++) {
> > +        VhostShadowVirtqueue *svq;
> > +        svq = vhost_svq_new(NULL, NULL);
>
> Patch 15 adds clean-up for this; I would squash it into here, as far as
> possible. Ideally, a patch series can be stopped at any step and not
> break anything, so if an allocation is added, the accompanying freeing
> should come with it.
>
> Hanna
>

In hindsight I shouldn't have considered cleanup a separate stage in the
development.  Too easy to introduce bugs that way.  I'll move cleanup up
and comment in the Patch 15 thread.

Re: [PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Connor Kite]

On Fri, Jul 24, 2026 at 8:20 AM Akihiko Odaki
<odaki@rsg.ci.i.u-tokyo.ac.jp> wrote:
>
>
> This copies data in the reversed direction.
>

Good catch!  This will be fixed going forward.

...


> > +    DMAMap needle;
> > +    hwaddr *iova_base;
>
> Please make it void *.
>

Got it!

> > +
> > +    for (int i = 0; i < elem->out_num; i++) {
> > +        needle.translated_addr = elem->out_addr[i];
> > +        needle.size = elem->out_sg[i].iov_len - 1;
> > +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
>
> An error check is missing here. This translation does not seem to
> support IOMMU.
>

I have added in the error check.

Looking at vhost-vdpa, there is an
error when IOMMU is enabled at the same time as shadow virtqueues.
I'll need to dig into what, if any, work is needed to make svqs compatible
with IOMMU enabled.  For now, I can implement a similar check in
vhost_user_dev_start to that in vhost-vdpa.

> > +        offset = needle.translated_addr - map->translated_addr;
> > +        iova_base = (void *)(map->iova + offset);
> > +
> > +        elem->out_sg[i].iov_base = iova_base;
>
> elem->out_sg shouldn't be overwritten since virtqueue_unpop() will use
> it to unmap the element.
>
> > +        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);
>
> This memcpy() is no-op. elem->out_sg[i].iov_base is already set to
> iova_base, so it's copying from iova_base to iova_base.
>

Roger!  There were a number of unnecessary adjustments to elem in
both handlers which should now be fixed.  As the IOVA and hva of the
isolation region are no longer the same (see Patch 11), there are some
additional changes required here, but the memcpy issues should also
be fixed for the next rev.

> > +    }
> > +
> > +    for (int i = 0; i < elem->in_num; i++) {
> > +        needle.translated_addr = elem->in_addr[i];
> > +        needle.size = elem->in_sg[i].iov_len - 1;
> > +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
> > +        offset = needle.translated_addr - map->translated_addr;
> > +        iova_base = (void *)(map->iova + offset);
> > +
> > +        elem->in_sg[i].iov_base = iova_base;
> > +        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);
> > +    }
> > +
> > +    vhost_svq_add(svq, elem->out_sg, elem->out_num, elem->out_addr,
> > +                  elem->in_sg, elem->in_num, elem->in_addr, elem);
>
> An error is ignored here too.
>

This is added now.

> > +
> > +    return 0;
> > +}
> > +
> > +

...

> > +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> > +        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
> > +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
> > +        svq->base_addr = (hwaddr *) vring_base;
> > +        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);
>
> Calling vhost_svq_start() here overwrites the shadow vring index
> configured in vhost_virtqueue_start().
>

I might not understand what you are referring to here. If I understand
correctly,
vhost_virtqueue_start() does not touch any shadow vring indexes, so I
don't think
vhost_svq_start() could overwrite something configured there.

However, I see that the assignment to addr.index followed by
vhost_user_set_vring_addr
might create a disconnect between the vq index and that sent to the
backend.  It's possible
my ordering is just right as is, but I will confirm by assigning
virtio_get_queue_index(vq) to
addr.index

> > +
> > +        struct vhost_vring_addr addr = {
> > +            .index = dev->vq_index + i,
> > +            .desc_user_addr = vring_base,
> > +            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
> > +                svq->vring.num,
>
> This assumes the ring is split, but VIRTIO_F_RING_PACKED is not rejected.
>

Currently, shadow virtqueue does not support packed vrings, so I can't here.  It
is a good idea to add a check and reject VIRTIO_F_RING_PACKED.

> > +            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)
> > +        };
> > +
> > +        vhost_user_set_vring_addr(dev, &addr);
>
>
> The initial ring startup is ordered incorrectly. Generic startup
> suppresses SET_VRING_ADDR, installs SET_VRING_KICK, and injects a kick;
> only the later backend-start hook creates the shadow ring and sends its
> address. SET_VRING_KICK starts the ring, so the backend can process
> before receiving valid addresses. The late SET_VRING_ADDR result is also
> ignored.

I think to avoid this issue, I need to assign svq vring locations when the mem
table is getting set, rather than on device start, as the svq
implementation seems to
expect.

It is likely that the isolation mode implementation will end up moving
up to vhost rather than
vhost-user.  If that's the case, some of the complexity around
tracking order of operations should
be simplified.

Best,
Connor

Re: [PATCH RFC 14/15] hw/virtio/vhost-user: handle data movement with shadow vqs

[Connor Kite]

On Tue, Jul 28, 2026 at 1:57 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> > +
> > +    for (int i = 0; i < elem->out_num; i++) {
> > +        needle.translated_addr = elem->out_addr[i];
> > +        needle.size = elem->out_sg[i].iov_len - 1;
> > +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
>
> Mapping failure must be handled.
>

Got it!  This will be fixed.

> > +        offset = needle.translated_addr - map->translated_addr;
> > +        iova_base = (void *)(map->iova + offset);
> > +
> > +        elem->out_sg[i].iov_base = iova_base;
> > +        memcpy(iova_base, elem->out_sg[i].iov_base, needle.size + 1);
>
> iova_base is an iova, not QEMU memory (HVA). Memcpy cannot be used with
> IOVAs. The address of the mapped shared memory is needed as the
> destination instead.
>

This is a relic of how the IOVA space was originally really just the
hva of the shared
memory.   However, as you noted, that wasn't great for security.  Now that those
address spaces are separate, the call is adjusted.

> > +    }
> > +
> > +    for (int i = 0; i < elem->in_num; i++) {
> > +        needle.translated_addr = elem->in_addr[i];
> > +        needle.size = elem->in_sg[i].iov_len - 1;
> > +        map = vhost_iova_tree_find_gpa(svq->iova_tree, &needle);
>
> Mapping failure must be handled.
>

Fixed as above!

> > +        offset = needle.translated_addr - map->translated_addr;
> > +        iova_base = (void *)(map->iova + offset);
> > +
> > +        elem->in_sg[i].iov_base = iova_base;
> > +        memcpy(iova_base, elem->in_sg[i].iov_base, needle.size + 1);
>
> No memcpy is necessary before vhost_svq_add() since this is data that
> will be read from the device upon I/O completion.
>

Copy! It is removed.

> > +    }
>
> elem->out_sg[] and elem->in_sg[] are modified in this function. Have we
> lost the original I/O buffer memory address from the guest virtqueue?
> This is a problem because they will be needed when completing the
> request.
>

I don't believe they were lost.  I had complimentary operations on elem
in the two handlers.  However, my thought that these needed to be adjusted
was incorrect, so address fields in elem are no longer modified in either
handler.

...

> >
> > +    if (u->user->memory_isolation) {
> > +        if (!u->svqs_allocated) {
> > +            return 0;
> > +        }
>
> Is the idea that this returns silently when called before
> vhost_user_dev_start()? State make code harder to understand. It would
> be cleaner to set up the vring addresses without relying on
> svqs_allocated. What is the reason for deferring the vring address
> setup to vhost_user_dev_start()?
>

The current implementation follows a similar order of operations as
vhost-vdpa, in which vhost_svq_start is called by vhost_dev_start.
vhost_svq_start is where vrings get assigned their locations, so there
is no valid address to send before then.

As we discussed offline, this could potentially be fixed by calling
vhost_svq_start when the mem table is being set, though this
complexity among others is a good reason to move implementation
of the isolation mode up to vhost.

...

> >
> > +static bool vhost_user_svqs_start(struct vhost_dev *dev)
> > +{
> > +    struct vhost_user *u = dev->opaque;
> > +    uint64_t vring_base = u->iso_memory.vring_base_addr;
> > +    u->svqs_allocated = true;
>
> Where is this field cleared to false on reset?
>

Nowhere originally, but I have added it in vhost_user_svqs_stop.

> > +
> > +    for (int i = 0; i < u->shadow_vqs->len; i++) {
> > +        VirtQueue *vq = virtio_get_queue(dev->vdev, dev->vq_index + i);
> > +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs, i);
> > +        svq->base_addr = (hwaddr *) vring_base;
>
> Does this support multiple virtqueues, it looks like they will all use
> the same base_addr?
>

It does!  vring_base is incremented after the assignment is made
for each svq.  Having said that, I don't think this works with virtio-net
where the device is implemented as multiple vhost-devs.  Since
multiple vhost-devs would need to share the iso memory, I will need
to adjust the vring assignment accordingly.

> > +        vhost_svq_start(svq, dev->vdev, vq, u->iso_iova_tree);
> > +
> > +        struct vhost_vring_addr addr = {
> > +            .index = dev->vq_index + i,
> > +            .desc_user_addr = vring_base,
> > +            .avail_user_addr = vring_base + sizeof(vring_desc_t) *
> > +                svq->vring.num,
> > +            .used_user_addr = vring_base + vhost_svq_driver_area_size(svq)
> > +        };
> > +
> > +        vhost_user_set_vring_addr(dev, &addr);
>
> Also mentioned above in vhost_user_set_vring_addr():
>
> Sending a vhost-user message here is strange since that should already
> be done by vhost.c:vhost_virtqueue_start() ->
> vhost_user_set_vring_addr().
>

Same response as above.

Re: [PATCH RFC 10/15] hw/virtio/vhost-shadow-virtqueue: range boundary in translation

[Hanna Czenczek]

On 08.08.26 07:52, Connor Kite wrote:
> On Mon, Aug 3, 2026 at 5:56 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>>
>> It’s not immediately obvious what is ensuring that `iov_len` can never
>> be 0. Sure, it would be wrong and makes no sense, but that is why I
>> think an `assert(iovec[i].iov_len > 0)` would be appropriate.
>>
>> (Looks like `virtqueue_map_desc()` is what rejects zero length, but that
>> is not really local to this code path, so not immediately obvious.)
>>
>> Hanna
>>
>  From the QEMU Coding Style guidelines on error-handling it looks like
> errors that
> could be initiated by a malfunctioning guest should not cause Qemu to
> exit. Would it be
> more appropriate to return false when zero-length is detected?  That follows
> the existing error path returning up to vhost_handle_guest_kick, which
> renders the svq
> unresponsive to future kicks.

Sure, but it’s necessary only *if* a malfunctioning guest could cause 
this. As I said, it looks like `virtqueue_map_desc()` already rejects 
zero-length descriptors, so an assert should be sufficient.

Then again, if you can handle it benignly either way (without an 
assert), that’s always a good option.

Hanna

Re: [PATCH RFC 13/15] hw/virtio/vhost-user: add shadow virtqueues and eventfd intercepts

[Hanna Czenczek]

On 11.08.26 04:32, Connor Kite wrote:
> On Mon, Aug 3, 2026 at 6:52 AM Hanna Czenczek <hreitz@redhat.com> wrote:
>>> +    struct vhost_user *u = dev->opaque;
>>> +    int svq_idx = file->index - dev->vq_index;
>>> +    if (u->user->memory_isolation) {
>>> +        VhostShadowVirtqueue *svq = g_ptr_array_index(u->shadow_vqs,
>>> +                                                      svq_idx);
>> Bounds checking via `vhost_user_get_vq_index()` would be nice. (Same below.)
>>
> Adding the bounds checking outside of the memory-isolation only code path, since
> file->index wasn't being checked previously.
>
>>> +        if (svq->hdev_kick.initialized == false) {
>>> +            int r = event_notifier_init(&svq->hdev_kick, 0);
>>> +            if (r) {
>>> +                error_report("Failed to create kick event notifier");
>>> +                return r;
>>> +            }
>>> +        }
>>> +
>>> +        file->fd = event_notifier_get_fd(&svq->hdev_kick);
>> What if `file->fd` was -1, i.e. `VHOST_FILE_UNBIND`? Should we put a
>> real FD here then or just continue with -1?
>>
> Good point!  I think we need to propagate the -1 to the backend device, and
> run event_notifier_cleanup on the existing svq->hdev_kick.
>
> ...
>
>>> +
>>> +        if (svq->hdev_call.initialized == false) {
>>> +            int r = event_notifier_init(&svq->hdev_call, 0);
>> Do we need to check if the call event FD is already active?
>>
> Do you mean, do we need to call event_notifier_test_and_clear to see if it's
> been set and potentially run the call handler at this stage?

Yes, that is what I’m wondering. `vhost_svq_set_svq_kick_fd()` has this 
comment:

 > event_notifier_set_handler already checks for guest's notifications if
 > they arrive at the new file descriptor in the switch, so there is no
 > need to explicitly check for them.

So it sounds like there is some mechanism that will ensure that when the 
kick FD is set on a shadow virtqueue, it is checked for activity 
(notification), and if it is active, this is… handled? Which I suppose 
means the installed handler will be called.

But it’s entirely possible I misunderstand something because the same 
code in `vhost_svq_set_svq_kick_fd()` also just calls 
`event_notifier_set(svq_kick)`, so… kinda always emulates a kick, I assume?

In any case, there seems to be worry about pending checking kicks from 
the guest in this switch, and so I wonder if the same needs to be done 
for the call FD.

Hanna

> If so, this might be where some of the intricacies of when things happen in
> vhost-user vs in vhost would require some extra care, unless isolation mode
> implementation moved up into vhost.
>
> ...
>
>>> +    /*Modified from vhost-vdpa*/
>>> +    u->shadow_vqs = g_ptr_array_new_full(dev->nvqs, vhost_svq_free);
>>> +    for (int i = 0; i < dev->nvqs; i++) {
>>> +        VhostShadowVirtqueue *svq;
>>> +        svq = vhost_svq_new(NULL, NULL);
>> Patch 15 adds clean-up for this; I would squash it into here, as far as
>> possible. Ideally, a patch series can be stopped at any step and not
>> break anything, so if an allocation is added, the accompanying freeing
>> should come with it.
>>
>> Hanna
>>
> In hindsight I shouldn't have considered cleanup a separate stage in the
> development.  Too easy to introduce bugs that way.  I'll move cleanup up
> and comment in the Patch 15 thread.
>