[PATCH 0/3] hw/display: add vhost-user-media device

[PATCH 0/3] hw/display: add vhost-user-media device

[Albert Esteve]

This series introduces the QEMU frontend for the virtio-media device,
a VirtIO device that exposes V4L2 video device functionality to the guest
over the virtio-media protocol, as specified in the VirtIO specification
v1.4, section 5.22 [1].

Note on patch 1
---------------
Patch 1 adds a temporary virtio-media device ID to QEMU's copy of the
Linux standard headers. It is included to unblock testing but is marked
"DO NOT MERGE": it will be superseded once the corresponding kernel-side
patch [2] is merged upstream and QEMU's headers are updated to match.
Patches 2 and 3 depend on patch 1 for the VIRTIO_ID_MEDIA definition.

Overview
--------
Patch 2 introduces the vhost-user-media device frontend as defined by
the specification.

Patch 3 adds a 4 GiB shared memory PCI BAR (BAR 2) used by the guest
driver as a DMA cache for video buffer exchange with the host backend.
The region is exposed to the guest through a VirtIO shared memory
capability.

Testing
-------
The series was tested with:

  Guest driver: the virtio-media out-of-tree Linux kernel driver [3].
  An upstream series for this driver has also been posted [4].

  Host backend: the rust-vmm vhost-device-media backend [5]:
      cargo run -- -s /path/to/media.sock -d /dev/video0 --backend v4l2-proxy

  Full QEMU invocation:

      qemu-system-x86_64 \
          -chardev socket,path=/tmp/media.sock,id=media \
          -device vhost-user-media-pci,chardev=media,id=media \
          -object memory-backend-memfd,id=mem,size=4G,share=on \
          -numa node,memdev=mem

[1] https://docs.oasis-open.org/virtio/virtio/v1.4/cs01/virtio-v1.4-cs01.html
[2] https://lore.kernel.org/all/20260310-virtio-media-id-v1-1-be211bcf682b@redhat.com/
[3] https://github.com/chromeos/virtio-media/tree/main/driver
[4] https://lore.kernel.org/all/20260622204343.1994418-1-briandaniels@google.com/
[5] https://github.com/rust-vmm/vhost-device/pull/944

Albert Esteve (3):
  virtio_ids: Add ID for virtio media
  hw/display: add vhost-user-media device
  hw/display/vhost-user-media: add shared memory cache BAR

 hw/display/Kconfig                          |   5 +
 hw/display/meson.build                      |   3 +
 hw/display/vhost-user-media-pci.c           | 105 ++++++
 hw/display/vhost-user-media.c               | 397 ++++++++++++++++++++
 hw/virtio/virtio.c                          |   3 +-
 include/hw/virtio/vhost-user-media.h        |  47 +++
 include/standard-headers/linux/virtio_ids.h |   1 +
 7 files changed, 560 insertions(+), 1 deletion(-)
 create mode 100644 hw/display/vhost-user-media-pci.c
 create mode 100644 hw/display/vhost-user-media.c
 create mode 100644 include/hw/virtio/vhost-user-media.h

-- 
2.54.0

[PATCH 1/3] virtio_ids: Add ID for virtio media

[Albert Esteve]

Temporary commit. DO NOT MERGE.

While we wait for the patch [1] to get merged upstream,
manually update and add the virtio-media ID to the kernel
header for testing purposes.

[1] https://lore.kernel.org/all/20260310-virtio-media-id-v1-1-be211bcf682b@redhat.com/

Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
 include/standard-headers/linux/virtio_ids.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/standard-headers/linux/virtio_ids.h b/include/standard-headers/linux/virtio_ids.h
index 6c12db16fa..fd73ae9d13 100644
--- a/include/standard-headers/linux/virtio_ids.h
+++ b/include/standard-headers/linux/virtio_ids.h
@@ -69,6 +69,7 @@
 #define VIRTIO_ID_BT			40 /* virtio bluetooth */
 #define VIRTIO_ID_GPIO			41 /* virtio gpio */
 #define VIRTIO_ID_SPI			45 /* virtio spi */
+#define VIRTIO_ID_MEDIA         48 /* virtio media */
 
 /*
  * Virtio Transitional IDs
-- 
2.54.0

[PATCH 2/3] hw/display: add vhost-user-media device

[Albert Esteve]

Add the QEMU side of the vhost-user-media device, which connects to a
virtio-media vhost-user backend daemon implementing the V4L2 API over
the virtio-media protocol.

The virtio-media device is specified in the VirtIO specification v1.4,
section 5.22:
    https://docs.oasis-open.org/virtio/virtio/v1.4/cs01/virtio-v1.4-cs01.html

Tested with the rust-vmm vhost-device-media backend [1]:

    cargo run -- -s /path/to/media.sock -d /dev/video0 --backend v4l2-proxy

Example invocation:

    qemu-system-x86_64 \
        -chardev socket,path=/tmp/media.sock,id=media \
        -device vhost-user-media-pci,chardev=media,id=media

[1] https://github.com/rust-vmm/vhost-device/pull/944

Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
 hw/display/Kconfig                   |   5 +
 hw/display/meson.build               |   3 +
 hw/display/vhost-user-media-pci.c    |  77 ++++++
 hw/display/vhost-user-media.c        | 393 +++++++++++++++++++++++++++
 hw/virtio/virtio.c                   |   3 +-
 include/hw/virtio/vhost-user-media.h |  47 ++++
 6 files changed, 527 insertions(+), 1 deletion(-)
 create mode 100644 hw/display/vhost-user-media-pci.c
 create mode 100644 hw/display/vhost-user-media.c
 create mode 100644 include/hw/virtio/vhost-user-media.h

diff --git a/hw/display/Kconfig b/hw/display/Kconfig
index b3593fe981..060274b1e6 100644
--- a/hw/display/Kconfig
+++ b/hw/display/Kconfig
@@ -120,6 +120,11 @@ config VHOST_USER_VGA
     default y
     depends on VIRTIO_VGA && VHOST_USER_GPU
 
+config VHOST_USER_MEDIA
+    bool
+    default y
+    depends on VIRTIO && VHOST_USER
+
 config DPCD
     bool
     select AUX
diff --git a/hw/display/meson.build b/hw/display/meson.build
index ffecedbf70..214c047df6 100644
--- a/hw/display/meson.build
+++ b/hw/display/meson.build
@@ -37,6 +37,9 @@ system_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
 system_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
 system_ss.add(when: 'CONFIG_VIRTIO', if_true: files('virtio-dmabuf.c'))
 system_ss.add(when: 'CONFIG_DM163', if_true: files('dm163.c'))
+system_ss.add(when: 'CONFIG_VHOST_USER_MEDIA', if_true: files('vhost-user-media.c'))
+system_ss.add(when: ['CONFIG_VHOST_USER_MEDIA', 'CONFIG_VIRTIO_PCI' ],
+  if_true: files('vhost-user-media-pci.c'))
 
 stub_ss.add([files('acpi-vga-stub.c'), pixman])
 if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
diff --git a/hw/display/vhost-user-media-pci.c b/hw/display/vhost-user-media-pci.c
new file mode 100644
index 0000000000..ad9e3900a4
--- /dev/null
+++ b/hw/display/vhost-user-media-pci.c
@@ -0,0 +1,77 @@
+/*
+ * Vhost-user MEDIA virtio device PCI glue
+ *
+ * Copyright Red Hat, Inc. 2024
+ * Authors: Albert Esteve <aesteve@redhat.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/core/qdev-properties.h"
+#include "hw/virtio/vhost-user-media.h"
+#include "hw/virtio/virtio-pci.h"
+
+#define TYPE_VHOST_USER_MEDIA_PCI "vhost-user-media-pci-base"
+OBJECT_DECLARE_SIMPLE_TYPE(VHostUserMEDIAPCI, VHOST_USER_MEDIA_PCI)
+
+struct VHostUserMEDIAPCI {
+    VirtIOPCIProxy parent_obj;
+    VHostUserMEDIA vdev;
+};
+
+static const Property vumedia_pci_properties[] = {
+    DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags,
+                    VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true),
+    DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors,
+                       DEV_NVECTORS_UNSPECIFIED),
+};
+
+static void vumedia_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
+{
+    VHostUserMEDIAPCI *dev = VHOST_USER_MEDIA_PCI(vpci_dev);
+    DeviceState *dev_state = DEVICE(&dev->vdev);
+
+    if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
+        vpci_dev->nvectors = 1;
+    }
+
+    qdev_realize(dev_state, BUS(&vpci_dev->bus), errp);
+}
+
+static void vumedia_pci_class_init(ObjectClass *klass, const void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass);
+    PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass);
+    k->realize = vumedia_pci_realize;
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+    device_class_set_props(dc, vumedia_pci_properties);
+    pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET;
+    pcidev_k->device_id = 0; /* Set by virtio-pci based on virtio id */
+    pcidev_k->revision = 0x00;
+    pcidev_k->class_id = PCI_CLASS_STORAGE_OTHER;
+}
+
+static void vumedia_pci_instance_init(Object *obj)
+{
+    VHostUserMEDIAPCI *dev = VHOST_USER_MEDIA_PCI(obj);
+
+    virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev),
+                                TYPE_VHOST_USER_MEDIA);
+}
+
+static const VirtioPCIDeviceTypeInfo vumedia_pci_info = {
+    .base_name             = TYPE_VHOST_USER_MEDIA_PCI,
+    .non_transitional_name = "vhost-user-media-pci",
+    .instance_size = sizeof(VHostUserMEDIAPCI),
+    .instance_init = vumedia_pci_instance_init,
+    .class_init    = vumedia_pci_class_init,
+};
+
+static void vumedia_pci_register(void)
+{
+    virtio_pci_types_register(&vumedia_pci_info);
+}
+
+type_init(vumedia_pci_register);
diff --git a/hw/display/vhost-user-media.c b/hw/display/vhost-user-media.c
new file mode 100644
index 0000000000..5c627437e7
--- /dev/null
+++ b/hw/display/vhost-user-media.c
@@ -0,0 +1,393 @@
+/*
+ * Vhost-user Media device
+ *
+ * Copyright Red Hat, Inc. 2024
+ *
+ * This is the boilerplate for instantiating a vhost-user device
+ * implementing a virtio-media device.
+ *
+ * Authors:
+ *     Albert Esteve <aesteve@redhat.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/core/qdev-properties.h"
+#include "standard-headers/linux/virtio_ids.h"
+#include "hw/virtio/virtio-bus.h"
+#include "hw/virtio/vhost-user-media.h"
+
+static const int feature_bits[] = {
+    VIRTIO_F_RING_RESET,
+    VHOST_INVALID_FEATURE_BIT
+};
+
+static void
+vhost_user_media_get_config(VirtIODevice *vdev, uint8_t *config_data)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+    Error *local_err = NULL;
+    int ret;
+
+    memset(config_data, 0, sizeof(struct virtio_media_config));
+
+    ret = vhost_dev_get_config(&media->vhost_dev,
+                               config_data, sizeof(struct virtio_media_config),
+                               &local_err);
+    if (ret) {
+        error_report_err(local_err);
+        return;
+    }
+}
+
+static void vhost_user_media_start(VirtIODevice *vdev)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+    BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
+    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
+    int ret;
+    int i;
+
+    if (!k->set_guest_notifiers) {
+        error_report("binding does not support guest notifiers");
+        return;
+    }
+
+    ret = vhost_dev_enable_notifiers(&media->vhost_dev, vdev);
+    if (ret < 0) {
+        error_report("Error enabling host notifiers: %d", -ret);
+        return;
+    }
+
+    ret = k->set_guest_notifiers(qbus->parent, media->vhost_dev.nvqs, true);
+    if (ret < 0) {
+        error_report("Error binding guest notifier: %d", -ret);
+        goto err_host_notifiers;
+    }
+
+    media->vhost_dev.acked_features = vdev->guest_features;
+
+    media->vhost_dev.vq_index_end = media->vhost_dev.nvqs;
+    ret = vhost_dev_start(&media->vhost_dev, vdev, true);
+    if (ret < 0) {
+        error_report("Error starting vhost-user-media: %d", -ret);
+        goto err_guest_notifiers;
+    }
+
+    /*
+     * guest_notifier_mask/pending not used yet, so just unmask
+     * everything here.  virtio-pci will do the right thing by
+     * enabling/disabling irqfd.
+     */
+    for (i = 0; i < media->vhost_dev.nvqs; i++) {
+        vhost_virtqueue_mask(&media->vhost_dev, vdev, i, false);
+    }
+
+    return;
+
+err_guest_notifiers:
+    k->set_guest_notifiers(qbus->parent, media->vhost_dev.nvqs, false);
+err_host_notifiers:
+    vhost_dev_disable_notifiers(&media->vhost_dev, vdev);
+}
+
+static void vhost_user_media_stop(VirtIODevice *vdev)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+    BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
+    VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
+    int ret;
+
+    if (!k->set_guest_notifiers) {
+        return;
+    }
+
+    vhost_dev_stop(&media->vhost_dev, vdev, true);
+
+    ret = k->set_guest_notifiers(qbus->parent, media->vhost_dev.nvqs, false);
+    if (ret < 0) {
+        error_report("vhost guest notifier cleanup failed: %d", ret);
+        return;
+    }
+
+    vhost_dev_disable_notifiers(&media->vhost_dev, vdev);
+}
+
+static int vhost_user_media_set_status(VirtIODevice *vdev, uint8_t status)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+    bool should_start = virtio_device_started(vdev, status);
+
+    if (!vdev->vm_running) {
+        should_start = false;
+    }
+
+    if (media->vhost_dev.started == should_start) {
+        return 0;
+    }
+
+    if (should_start) {
+        vhost_user_media_start(vdev);
+    } else {
+        vhost_user_media_stop(vdev);
+    }
+    return 0;
+}
+
+static uint64_t vhost_user_media_get_features(VirtIODevice *vdev,
+                                      uint64_t requested_features,
+                                      Error **errp)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    return vhost_get_features(&media->vhost_dev, feature_bits,
+                              requested_features);
+}
+
+static void vhost_user_media_handle_output(VirtIODevice *vdev, VirtQueue *vq)
+{
+    /*
+     * Not normally called; it's the daemon that handles the queue;
+     * however virtio's cleanup path can call this.
+     */
+}
+
+static void vhost_user_media_guest_notifier_mask(VirtIODevice *vdev, int idx,
+                                            bool mask)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    if (idx == VIRTIO_CONFIG_IRQ_IDX) {
+        return;
+    }
+
+    vhost_virtqueue_mask(&media->vhost_dev, vdev, idx, mask);
+}
+
+static bool vhost_user_media_guest_notifier_pending(VirtIODevice *vdev, int idx)
+{
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    if (idx == VIRTIO_CONFIG_IRQ_IDX) {
+        return false;
+    }
+
+    return vhost_virtqueue_pending(&media->vhost_dev, idx);
+}
+
+/*
+ * Chardev connect/disconnect events
+ */
+
+static int vhost_user_media_handle_config_change(struct vhost_dev *dev)
+{
+    int ret;
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(dev->vdev);
+    Error *local_err = NULL;
+
+    ret = vhost_dev_get_config(dev, (uint8_t *)&media->conf.config,
+                               sizeof(struct virtio_media_config), &local_err);
+    if (ret < 0) {
+        error_report("get config space failed");
+        return -1;
+    }
+
+    return 0;
+}
+
+static const VhostDevConfigOps media_ops = {
+    .vhost_dev_config_notifier = vhost_user_media_handle_config_change,
+};
+
+static int vhost_user_media_connect(DeviceState *dev)
+{
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    if (media->connected) {
+        return 0;
+    }
+    media->connected = true;
+
+    /* restore vhost state */
+    if (virtio_device_started(vdev, vdev->status)) {
+        vhost_user_media_start(vdev);
+    }
+
+    return 0;
+}
+
+static void vhost_user_media_disconnect(DeviceState *dev)
+{
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    if (!media->connected) {
+        return;
+    }
+    media->connected = false;
+
+    if (media->vhost_dev.started) {
+        vhost_user_media_stop(vdev);
+    }
+
+    vhost_dev_cleanup(&media->vhost_dev);
+}
+
+static void vhost_user_media_event(void *opaque, QEMUChrEvent event)
+{
+    DeviceState *dev = opaque;
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(vdev);
+
+    switch (event) {
+    case CHR_EVENT_OPENED:
+        if (vhost_user_media_connect(dev) < 0) {
+            qemu_chr_fe_disconnect(&media->conf.chardev);
+            return;
+        }
+        break;
+    case CHR_EVENT_CLOSED:
+        vhost_user_media_disconnect(dev);
+        break;
+    case CHR_EVENT_BREAK:
+    case CHR_EVENT_MUX_IN:
+    case CHR_EVENT_MUX_OUT:
+        /* Ignore */
+        break;
+    }
+}
+
+static void do_vhost_user_cleanup(VirtIODevice *vdev, VHostUserMEDIA *media)
+{
+    vhost_user_cleanup(&media->vhost_user);
+    virtio_delete_queue(media->command_vq);
+    virtio_delete_queue(media->event_vq);
+    virtio_cleanup(vdev);
+    g_free(media->vhost_dev.vqs);
+    media->vhost_dev.vqs = NULL;
+}
+
+static void vhost_user_media_device_realize(DeviceState *dev, Error **errp)
+{
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(dev);
+    int ret;
+
+    if (!media->conf.chardev.chr) {
+        error_setg(errp, "vhost-user-media: chardev is mandatory");
+        return;
+    }
+
+    if (!vhost_user_init(&media->vhost_user, &media->conf.chardev, errp)) {
+        return;
+    }
+
+    virtio_init(vdev, VIRTIO_ID_MEDIA, sizeof(struct virtio_media_config));
+
+    /* one command queue and one event queue */
+    media->vhost_dev.nvqs = 2;
+    media->vhost_dev.vqs = g_new0(struct vhost_virtqueue,
+                                  media->vhost_dev.nvqs);
+    media->vhost_dev.vq_index = 0;
+
+    vhost_dev_set_config_notifier(&media->vhost_dev, &media_ops);
+    media->vhost_user.supports_config = true;
+
+    ret = vhost_dev_init(&media->vhost_dev, &media->vhost_user,
+                         VHOST_BACKEND_TYPE_USER, 0, errp);
+    if (ret < 0) {
+        error_report("vhost_dev_init failed: %s", strerror(-ret));
+        goto vhost_dev_init_failed;
+    }
+
+    /* One command queue, for sending commands */
+    media->command_vq = virtio_add_queue(vdev, 128,
+                                         vhost_user_media_handle_output);
+
+    if (!media->command_vq) {
+        error_setg_errno(errp, -1, "virtio_add_queue() failed");
+        goto cmd_q_fail;
+    }
+
+    /* One event queue, for sending events */
+    media->event_vq = virtio_add_queue(vdev, 128,
+                                       vhost_user_media_handle_output);
+
+    if (!media->event_vq) {
+        error_setg_errno(errp, -1, "virtio_add_queue() failed");
+        goto event_q_fail;
+    }
+
+    /*
+     * At this point the next event we will get is a connection from
+     * the daemon on the control socket.
+     */
+
+    qemu_chr_fe_set_handlers(&media->conf.chardev, NULL,
+                             NULL, vhost_user_media_event,
+                             NULL, (void *)dev, NULL, true);
+
+    return;
+
+event_q_fail:
+    virtio_delete_queue(media->event_vq);
+cmd_q_fail:
+    vhost_user_cleanup(&media->vhost_user);
+vhost_dev_init_failed:
+    virtio_cleanup(vdev);
+}
+
+static void vhost_user_media_device_unrealize(DeviceState *dev)
+{
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+    VHostUserMEDIA *media = VHOST_USER_MEDIA(dev);
+
+    /* This will stop vhost backend if appropriate. */
+    vhost_user_media_set_status(vdev, 0);
+
+    do_vhost_user_cleanup(vdev, media);
+}
+
+static const VMStateDescription vhost_user_media_vmstate = {
+    .name = "vhost-user-media",
+    .unmigratable = 1,
+};
+
+static const Property vhost_user_media_properties[] = {
+    DEFINE_PROP_CHR("chardev", VHostUserMEDIA, conf.chardev),
+};
+
+static void vhost_user_media_class_init(ObjectClass *klass, const void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass);
+
+    device_class_set_props(dc, vhost_user_media_properties);
+    dc->vmsd = &vhost_user_media_vmstate;
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+    vdc->realize = vhost_user_media_device_realize;
+    vdc->unrealize = vhost_user_media_device_unrealize;
+    vdc->get_features = vhost_user_media_get_features;
+    vdc->get_config = vhost_user_media_get_config;
+    vdc->set_status = vhost_user_media_set_status;
+    vdc->guest_notifier_mask = vhost_user_media_guest_notifier_mask;
+    vdc->guest_notifier_pending = vhost_user_media_guest_notifier_pending;
+}
+
+static const TypeInfo vhost_user_media_info = {
+    .name = TYPE_VHOST_USER_MEDIA,
+    .parent = TYPE_VIRTIO_DEVICE,
+    .instance_size = sizeof(VHostUserMEDIA),
+    .class_init = vhost_user_media_class_init,
+};
+
+static void vhost_user_media_register_types(void)
+{
+    type_register_static(&vhost_user_media_info);
+}
+
+type_init(vhost_user_media_register_types)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index f4d86a3655..a07ef66c58 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -198,7 +198,8 @@ const char *virtio_device_names[] = {
     [VIRTIO_ID_AUDIO_POLICY] = "virtio-audio-pol",
     [VIRTIO_ID_BT] = "virtio-bluetooth",
     [VIRTIO_ID_GPIO] = "virtio-gpio",
-    [VIRTIO_ID_SPI] = "virtio-spi"
+    [VIRTIO_ID_SPI] = "virtio-spi",
+    [VIRTIO_ID_MEDIA] = "virtio-media",
 };
 
 static const char *virtio_id_to_name(uint16_t device_id)
diff --git a/include/hw/virtio/vhost-user-media.h b/include/hw/virtio/vhost-user-media.h
new file mode 100644
index 0000000000..30bdef8ce1
--- /dev/null
+++ b/include/hw/virtio/vhost-user-media.h
@@ -0,0 +1,47 @@
+/*
+ * vhost-user-media virtio device
+ *
+ * Copyright Red Hat, Inc. 2024
+ *
+ * Authors:
+ *     Albert Esteve <aesteve@redhat.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef QEMU_VHOST_USER_MEDIA_H
+#define QEMU_VHOST_USER_MEDIA_H
+
+#include "hw/virtio/virtio.h"
+#include "hw/virtio/vhost.h"
+#include "hw/virtio/vhost-user.h"
+#include "chardev/char-fe.h"
+#include "qom/object.h"
+
+#define TYPE_VHOST_USER_MEDIA "vhost-user-media-device"
+OBJECT_DECLARE_SIMPLE_TYPE(VHostUserMEDIA, VHOST_USER_MEDIA)
+
+struct virtio_media_config {
+    uint32_t device_caps;
+    uint32_t device_type;
+    uint8_t card[32];
+};
+
+typedef struct {
+    CharFrontend chardev;
+    struct virtio_media_config config;
+} VHostUserMEDIAConf;
+
+struct VHostUserMEDIA {
+    /*< private >*/
+    VirtIODevice parent;
+    VHostUserMEDIAConf conf;
+    struct vhost_dev vhost_dev;
+    VhostUserState vhost_user;
+    VirtQueue *command_vq;
+    VirtQueue *event_vq;
+    bool connected;
+    /*< public >*/
+};
+
+#endif /* QEMU_VHOST_USER_MEDIA_H */
-- 
2.54.0

[PATCH 3/3] hw/display/vhost-user-media: add shared memory cache BAR

[Albert Esteve]

Register a 4 GiB shared memory PCI BAR (BAR 2) so the guest driver
can use it as a DMA cache for video buffers. The memory region is
allocated on device realize and exposed to the guest through a VirtIO
shared memory capability (shmem ID 0).

The host must be started with a memfd-backed memory object covering
at least the shared region size, e.g.:

    -object memory-backend-memfd,id=mem,size=4G,share=on \
    -numa node,memdev=mem

Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
 hw/display/vhost-user-media-pci.c | 28 ++++++++++++++++++++++++++++
 hw/display/vhost-user-media.c     |  4 ++++
 2 files changed, 32 insertions(+)

diff --git a/hw/display/vhost-user-media-pci.c b/hw/display/vhost-user-media-pci.c
index ad9e3900a4..838a9da71d 100644
--- a/hw/display/vhost-user-media-pci.c
+++ b/hw/display/vhost-user-media-pci.c
@@ -12,12 +12,18 @@
 #include "hw/virtio/vhost-user-media.h"
 #include "hw/virtio/virtio-pci.h"
 
+/* BAR 2 is used for the shared memory cache region exposed to the guest */
+#define VIRTIO_MEDIA_PCI_CACHE_BAR 2
+
+#define VIRTIO_MEDIA_PCI_SHMCAP_ID_CACHE 0
+
 #define TYPE_VHOST_USER_MEDIA_PCI "vhost-user-media-pci-base"
 OBJECT_DECLARE_SIMPLE_TYPE(VHostUserMEDIAPCI, VHOST_USER_MEDIA_PCI)
 
 struct VHostUserMEDIAPCI {
     VirtIOPCIProxy parent_obj;
     VHostUserMEDIA vdev;
+    MemoryRegion cachebar;
 };
 
 static const Property vumedia_pci_properties[] = {
@@ -31,12 +37,34 @@ static void vumedia_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
 {
     VHostUserMEDIAPCI *dev = VHOST_USER_MEDIA_PCI(vpci_dev);
     DeviceState *dev_state = DEVICE(&dev->vdev);
+    VirtIODevice *vdev = VIRTIO_DEVICE(dev_state);
+    VirtioSharedMemory *shmem;
+    uint64_t cache_size;
 
     if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) {
         vpci_dev->nvectors = 1;
     }
 
     qdev_realize(dev_state, BUS(&vpci_dev->bus), errp);
+    if (*errp) {
+        return;
+    }
+
+    shmem = QSIMPLEQ_LAST(&vdev->shmem_list, VirtioSharedMemory, entry);
+    cache_size = memory_region_size(&shmem->mr);
+
+    memory_region_init(&dev->cachebar, OBJECT(vpci_dev),
+                       "vhost-media-pci-cachebar", cache_size);
+    memory_region_add_subregion(&dev->cachebar, 0, &shmem->mr);
+    virtio_pci_add_shm_cap(vpci_dev, VIRTIO_MEDIA_PCI_CACHE_BAR, 0,
+                            cache_size, VIRTIO_MEDIA_PCI_SHMCAP_ID_CACHE);
+
+    /* After 'realized' so the memory region exists */
+    pci_register_bar(&vpci_dev->pci_dev, VIRTIO_MEDIA_PCI_CACHE_BAR,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY |
+                     PCI_BASE_ADDRESS_MEM_PREFETCH |
+                     PCI_BASE_ADDRESS_MEM_TYPE_64,
+                     &dev->cachebar);
 }
 
 static void vumedia_pci_class_init(ObjectClass *klass, const void *data)
diff --git a/hw/display/vhost-user-media.c b/hw/display/vhost-user-media.c
index 5c627437e7..bbb64ea4d3 100644
--- a/hw/display/vhost-user-media.c
+++ b/hw/display/vhost-user-media.c
@@ -20,6 +20,8 @@
 #include "hw/virtio/virtio-bus.h"
 #include "hw/virtio/vhost-user-media.h"
 
+#define CACHE_SIZE (1ull << 32)
+
 static const int feature_bits[] = {
     VIRTIO_F_RING_RESET,
     VHOST_INVALID_FEATURE_BIT
@@ -288,6 +290,8 @@ static void vhost_user_media_device_realize(DeviceState *dev, Error **errp)
 
     virtio_init(vdev, VIRTIO_ID_MEDIA, sizeof(struct virtio_media_config));
 
+    virtio_new_shmem_region(vdev, 0, CACHE_SIZE);
+
     /* one command queue and one event queue */
     media->vhost_dev.nvqs = 2;
     media->vhost_dev.vqs = g_new0(struct vhost_virtqueue,
-- 
2.54.0

Re: [PATCH 2/3] hw/display: add vhost-user-media device

[Daniel P. Berrangé]

On Tue, Jun 30, 2026 at 01:23:09PM +0200, Albert Esteve wrote:
> Add the QEMU side of the vhost-user-media device, which connects to a
> virtio-media vhost-user backend daemon implementing the V4L2 API over
> the virtio-media protocol.
> 
> The virtio-media device is specified in the VirtIO specification v1.4,
> section 5.22:
>     https://docs.oasis-open.org/virtio/virtio/v1.4/cs01/virtio-v1.4-cs01.html
> 
> Tested with the rust-vmm vhost-device-media backend [1]:
> 
>     cargo run -- -s /path/to/media.sock -d /dev/video0 --backend v4l2-proxy
> 
> Example invocation:
> 
>     qemu-system-x86_64 \
>         -chardev socket,path=/tmp/media.sock,id=media \
>         -device vhost-user-media-pci,chardev=media,id=media
> 
> [1] https://github.com/rust-vmm/vhost-device/pull/944
> 
> Signed-off-by: Albert Esteve <aesteve@redhat.com>
> ---
>  hw/display/Kconfig                   |   5 +
>  hw/display/meson.build               |   3 +
>  hw/display/vhost-user-media-pci.c    |  77 ++++++
>  hw/display/vhost-user-media.c        | 393 +++++++++++++++++++++++++++
>  hw/virtio/virtio.c                   |   3 +-
>  include/hw/virtio/vhost-user-media.h |  47 ++++
>  6 files changed, 527 insertions(+), 1 deletion(-)
>  create mode 100644 hw/display/vhost-user-media-pci.c
>  create mode 100644 hw/display/vhost-user-media.c
>  create mode 100644 include/hw/virtio/vhost-user-media.h
> 
> diff --git a/hw/display/Kconfig b/hw/display/Kconfig
> index b3593fe981..060274b1e6 100644
> --- a/hw/display/Kconfig
> +++ b/hw/display/Kconfig
> @@ -120,6 +120,11 @@ config VHOST_USER_VGA
>      default y
>      depends on VIRTIO_VGA && VHOST_USER_GPU
>  
> +config VHOST_USER_MEDIA
> +    bool
> +    default y
> +    depends on VIRTIO && VHOST_USER
> +
>  config DPCD
>      bool
>      select AUX
> diff --git a/hw/display/meson.build b/hw/display/meson.build
> index ffecedbf70..214c047df6 100644
> --- a/hw/display/meson.build
> +++ b/hw/display/meson.build
> @@ -37,6 +37,9 @@ system_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
>  system_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
>  system_ss.add(when: 'CONFIG_VIRTIO', if_true: files('virtio-dmabuf.c'))
>  system_ss.add(when: 'CONFIG_DM163', if_true: files('dm163.c'))
> +system_ss.add(when: 'CONFIG_VHOST_USER_MEDIA', if_true: files('vhost-user-media.c'))
> +system_ss.add(when: ['CONFIG_VHOST_USER_MEDIA', 'CONFIG_VIRTIO_PCI' ],
> +  if_true: files('vhost-user-media-pci.c'))
>  
>  stub_ss.add([files('acpi-vga-stub.c'), pixman])
>  if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> diff --git a/hw/display/vhost-user-media-pci.c b/hw/display/vhost-user-media-pci.c
> new file mode 100644
> index 0000000000..ad9e3900a4
> --- /dev/null
> +++ b/hw/display/vhost-user-media-pci.c
> @@ -0,0 +1,77 @@
> +/*
> + * Vhost-user MEDIA virtio device PCI glue
> + *
> + * Copyright Red Hat, Inc. 2024

Have you really been hanging on to this new code for 2 years
before posting ? Should this be 2026 ? 

> + * Authors: Albert Esteve <aesteve@redhat.com>
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */

> diff --git a/hw/display/vhost-user-media.c b/hw/display/vhost-user-media.c
> new file mode 100644
> index 0000000000..5c627437e7
> --- /dev/null
> +++ b/hw/display/vhost-user-media.c
> @@ -0,0 +1,393 @@
> +/*
> + * Vhost-user Media device
> + *
> + * Copyright Red Hat, Inc. 2024

Same Q

> + *
> + * This is the boilerplate for instantiating a vhost-user device
> + * implementing a virtio-media device.
> + *
> + * Authors:
> + *     Albert Esteve <aesteve@redhat.com>
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +

> diff --git a/include/hw/virtio/vhost-user-media.h b/include/hw/virtio/vhost-user-media.h
> new file mode 100644
> index 0000000000..30bdef8ce1
> --- /dev/null
> +++ b/include/hw/virtio/vhost-user-media.h
> @@ -0,0 +1,47 @@
> +/*
> + * vhost-user-media virtio device
> + *
> + * Copyright Red Hat, Inc. 2024

Again


With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH 2/3] hw/display: add vhost-user-media device

[Albert Esteve]

On Tue, Jun 30, 2026 at 1:29 PM Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Tue, Jun 30, 2026 at 01:23:09PM +0200, Albert Esteve wrote:
> > Add the QEMU side of the vhost-user-media device, which connects to a
> > virtio-media vhost-user backend daemon implementing the V4L2 API over
> > the virtio-media protocol.
> >
> > The virtio-media device is specified in the VirtIO specification v1.4,
> > section 5.22:
> >     https://docs.oasis-open.org/virtio/virtio/v1.4/cs01/virtio-v1.4-cs01.html
> >
> > Tested with the rust-vmm vhost-device-media backend [1]:
> >
> >     cargo run -- -s /path/to/media.sock -d /dev/video0 --backend v4l2-proxy
> >
> > Example invocation:
> >
> >     qemu-system-x86_64 \
> >         -chardev socket,path=/tmp/media.sock,id=media \
> >         -device vhost-user-media-pci,chardev=media,id=media
> >
> > [1] https://github.com/rust-vmm/vhost-device/pull/944
> >
> > Signed-off-by: Albert Esteve <aesteve@redhat.com>
> > ---
> >  hw/display/Kconfig                   |   5 +
> >  hw/display/meson.build               |   3 +
> >  hw/display/vhost-user-media-pci.c    |  77 ++++++
> >  hw/display/vhost-user-media.c        | 393 +++++++++++++++++++++++++++
> >  hw/virtio/virtio.c                   |   3 +-
> >  include/hw/virtio/vhost-user-media.h |  47 ++++
> >  6 files changed, 527 insertions(+), 1 deletion(-)
> >  create mode 100644 hw/display/vhost-user-media-pci.c
> >  create mode 100644 hw/display/vhost-user-media.c
> >  create mode 100644 include/hw/virtio/vhost-user-media.h
> >
> > diff --git a/hw/display/Kconfig b/hw/display/Kconfig
> > index b3593fe981..060274b1e6 100644
> > --- a/hw/display/Kconfig
> > +++ b/hw/display/Kconfig
> > @@ -120,6 +120,11 @@ config VHOST_USER_VGA
> >      default y
> >      depends on VIRTIO_VGA && VHOST_USER_GPU
> >
> > +config VHOST_USER_MEDIA
> > +    bool
> > +    default y
> > +    depends on VIRTIO && VHOST_USER
> > +
> >  config DPCD
> >      bool
> >      select AUX
> > diff --git a/hw/display/meson.build b/hw/display/meson.build
> > index ffecedbf70..214c047df6 100644
> > --- a/hw/display/meson.build
> > +++ b/hw/display/meson.build
> > @@ -37,6 +37,9 @@ system_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c'))
> >  system_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c'))
> >  system_ss.add(when: 'CONFIG_VIRTIO', if_true: files('virtio-dmabuf.c'))
> >  system_ss.add(when: 'CONFIG_DM163', if_true: files('dm163.c'))
> > +system_ss.add(when: 'CONFIG_VHOST_USER_MEDIA', if_true: files('vhost-user-media.c'))
> > +system_ss.add(when: ['CONFIG_VHOST_USER_MEDIA', 'CONFIG_VIRTIO_PCI' ],
> > +  if_true: files('vhost-user-media-pci.c'))
> >
> >  stub_ss.add([files('acpi-vga-stub.c'), pixman])
> >  if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or
> > diff --git a/hw/display/vhost-user-media-pci.c b/hw/display/vhost-user-media-pci.c
> > new file mode 100644
> > index 0000000000..ad9e3900a4
> > --- /dev/null
> > +++ b/hw/display/vhost-user-media-pci.c
> > @@ -0,0 +1,77 @@
> > +/*
> > + * Vhost-user MEDIA virtio device PCI glue
> > + *
> > + * Copyright Red Hat, Inc. 2024
>
> Have you really been hanging on to this new code for 2 years
> before posting ? Should this be 2026 ?

Yup! Rebasing/updating has been painful a few times. But it was
waiting for the specs to land, and the shared memory mapping patch.
And other developments in parallel (e.g., virtio_ids.h kernel header
is still in the process).

But to your point, yes, I will update the copyright in the next
version. It makes sense to match the copyright year to when it is
upstreamed.

BR,
Albert.

>
> > + * Authors: Albert Esteve <aesteve@redhat.com>
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + */
>
> > diff --git a/hw/display/vhost-user-media.c b/hw/display/vhost-user-media.c
> > new file mode 100644
> > index 0000000000..5c627437e7
> > --- /dev/null
> > +++ b/hw/display/vhost-user-media.c
> > @@ -0,0 +1,393 @@
> > +/*
> > + * Vhost-user Media device
> > + *
> > + * Copyright Red Hat, Inc. 2024
>
> Same Q
>
> > + *
> > + * This is the boilerplate for instantiating a vhost-user device
> > + * implementing a virtio-media device.
> > + *
> > + * Authors:
> > + *     Albert Esteve <aesteve@redhat.com>
> > + *
> > + * SPDX-License-Identifier: GPL-2.0-or-later
> > + */
> > +
>
> > diff --git a/include/hw/virtio/vhost-user-media.h b/include/hw/virtio/vhost-user-media.h
> > new file mode 100644
> > index 0000000000..30bdef8ce1
> > --- /dev/null
> > +++ b/include/hw/virtio/vhost-user-media.h
> > @@ -0,0 +1,47 @@
> > +/*
> > + * vhost-user-media virtio device
> > + *
> > + * Copyright Red Hat, Inc. 2024
>
> Again
>
>
> With regards,
> Daniel
> --
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
>

Re: [PATCH 0/3] hw/display: add vhost-user-media device

[Dorinda Bassey]

Tested with vhost-device-media (v4l2-proxy backend) on Fedora 43 guest
(kernel 6.17.1), proxying a host integrated camera. Device detected and
frame capture works via v4l2-ctl --stream-mmap.

Tested-by: Dorinda Bassey <dbassey@redhat.com>