[RFC] Proposal for a QEMU video subsystem for cameras and webcams

[RFC] Proposal for a QEMU video subsystem for cameras and webcams

[David Milosevic]

[-- Attachment #1: Type: text/plain, Size: 1631 bytes --]

Dear QEMU Developers,

I would like to propose the development of a video subsystem in QEMU, with
the initial
implementation focusing on UVC video device emulation and support for
multiple
backends, including V4L2, GStreamer, and libcamera.

This work is already in progress at 9elements, and we would like to
upstream it.

== Motivation

Currently, USB pass-through is the only way to make video devices available
to guests, which

    - excludes non-USB cameras (e.g., MIPI)
    - performs poorly with high-resolution cameras
    - does not work with USB 3.0 video devices (Issue #1613)

== Proposal

We aim to introduce a video subsystem in QEMU that allows for the
implementation of various
video devices, similar to how QEMU handles audio. The first device
implementation will be
UVC (USB Video Class) device emulation, with support for multiple backends.
Future extensions
could include virtio-video or other PCI-based video devices.

Supported backends:

    - Video4Linux (V4L2)
    - GStreamer
    - libcamera

== Example: V4L2 Backend

Once implemented, a typical QEMU command line for using a V4L2 backend
would look like this

    ./build/qemu-system-x86_64 \
        -device qemu-xhci \
        -videodev v4l2,id=cam0,device=/dev/video0 \
        -device usb-video,videodev=cam0

This sets up a UVC emulated device in the guest, using /dev/video0 from the
host via the V4L2 backend.

== Next Steps

We welcome feedback on design considerations and integration approaches.
Let us know
if there are existing discussions or preferred directions for this work.

Best regards,

David Milosevic
Firmware Developer
9elements

[-- Attachment #2: Type: text/html, Size: 2053 bytes --]

Re: [RFC] Proposal for a QEMU video subsystem for cameras and webcams

[Alex Bennée]

David Milosevic <david.milosevic@9elements.com> writes:

> Dear QEMU Developers,
>
> I would like to propose the development of a video subsystem in QEMU, with the initial
> implementation focusing on UVC video device emulation and support for multiple
> backends, including V4L2, GStreamer, and libcamera.
>
> This work is already in progress at 9elements, and we would like to upstream it.
>
> == Motivation
>
> Currently, USB pass-through is the only way to make video devices available to guests, which
>
>     - excludes non-USB cameras (e.g., MIPI)
>     - performs poorly with high-resolution cameras
>     - does not work with USB 3.0 video devices (Issue #1613)
>
> == Proposal
>
> We aim to introduce a video subsystem in QEMU that allows for the implementation of various
> video devices, similar to how QEMU handles audio. The first device implementation will be
> UVC (USB Video Class) device emulation, with support for multiple backends. Future extensions
> could include virtio-video or other PCI-based video devices.

Are you aware of virtio-media? It was an alternative proposal to
virtio-video which effectively becomes an encapsulation of v4l to the
guest.

Kernel patches: https://lore.kernel.org/lkml/20250201-virtio-media-v2-1-ac840681452d@gmail.com/
VirtIO spec: https://lore.kernel.org/virtio-comment/20250120085015.956057-1-aesteve@redhat.com/T/
 

> Supported backends:
>
>     - Video4Linux (V4L2)
>     - GStreamer
>     - libcamera
>
> == Example: V4L2 Backend
>
> Once implemented, a typical QEMU command line for using a V4L2 backend would look like this
>
>     ./build/qemu-system-x86_64 \
>         -device qemu-xhci \
>         -videodev v4l2,id=cam0,device=/dev/video0 \
>         -device usb-video,videodev=cam0
>
> This sets up a UVC emulated device in the guest, using /dev/video0 from the
> host via the V4L2 backend.
>
> == Next Steps
>
> We welcome feedback on design considerations and integration approaches. Let us know
> if there are existing discussions or preferred directions for this work.
>
> Best regards,
>
> David Milosevic
> Firmware Developer
> 9elements

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC] Proposal for a QEMU video subsystem for cameras and webcams

[Thomas Huth]

On 10/03/2025 17.51, Alex Bennée wrote:
> David Milosevic <david.milosevic@9elements.com> writes:
> 
>> Dear QEMU Developers,
>>
>> I would like to propose the development of a video subsystem in QEMU, with the initial
>> implementation focusing on UVC video device emulation and support for multiple
>> backends, including V4L2, GStreamer, and libcamera.
>>
>> This work is already in progress at 9elements, and we would like to upstream it.
>>
>> == Motivation
>>
>> Currently, USB pass-through is the only way to make video devices available to guests, which
>>
>>      - excludes non-USB cameras (e.g., MIPI)
>>      - performs poorly with high-resolution cameras
>>      - does not work with USB 3.0 video devices (Issue #1613)
>>
>> == Proposal
>>
>> We aim to introduce a video subsystem in QEMU that allows for the implementation of various
>> video devices, similar to how QEMU handles audio. The first device implementation will be
>> UVC (USB Video Class) device emulation, with support for multiple backends. Future extensions
>> could include virtio-video or other PCI-based video devices.
> 
> Are you aware of virtio-media? It was an alternative proposal to
> virtio-video which effectively becomes an encapsulation of v4l to the
> guest.

... but USB video would also be nice, wouldn't it? That could enable guests 
to use webcams without needing additional virtio drivers for it, I think?

>> Supported backends:
>>
>>      - Video4Linux (V4L2)
>>      - GStreamer
>>      - libcamera
>>
>> == Example: V4L2 Backend
>>
>> Once implemented, a typical QEMU command line for using a V4L2 backend would look like this
>>
>>      ./build/qemu-system-x86_64 \
>>          -device qemu-xhci \
>>          -videodev v4l2,id=cam0,device=/dev/video0 \

Just a quick comment here: QEMU tries often to avoid to introduce new 
top-level command line switches nowadays. Would it be possible to use 
"-object" for this instead, like it is e.g. done with the memory backends or 
rng backends?

  Thomas


>>          -device usb-video,videodev=cam0
>>
>> This sets up a UVC emulated device in the guest, using /dev/video0 from the
>> host via the V4L2 backend.
>>
>> == Next Steps
>>
>> We welcome feedback on design considerations and integration approaches. Let us know
>> if there are existing discussions or preferred directions for this work.
>>
>> Best regards,
>>
>> David Milosevic
>> Firmware Developer
>> 9elements
> 

Re: Re: [RFC] Proposal for a QEMU video subsystem for cameras and webcams

[zhenwei pi]

On 3/11/25 00:57, Thomas Huth wrote:
> On 10/03/2025 17.51, Alex Bennée wrote:
>> David Milosevic <david.milosevic@9elements.com> writes:
>>
>>> Dear QEMU Developers,
>>>
>>> I would like to propose the development of a video subsystem in QEMU, 
>>> with the initial
>>> implementation focusing on UVC video device emulation and support for 
>>> multiple
>>> backends, including V4L2, GStreamer, and libcamera.
>>>
>>> This work is already in progress at 9elements, and we would like to 
>>> upstream it.
>>>
>>> == Motivation
>>>
>>> Currently, USB pass-through is the only way to make video devices 
>>> available to guests, which
>>>
>>>      - excludes non-USB cameras (e.g., MIPI)
>>>      - performs poorly with high-resolution cameras
>>>      - does not work with USB 3.0 video devices (Issue #1613)
>>>
>>> == Proposal
>>>
>>> We aim to introduce a video subsystem in QEMU that allows for the 
>>> implementation of various
>>> video devices, similar to how QEMU handles audio. The first device 
>>> implementation will be
>>> UVC (USB Video Class) device emulation, with support for multiple 
>>> backends. Future extensions
>>> could include virtio-video or other PCI-based video devices.
>>
>> Are you aware of virtio-media? It was an alternative proposal to
>> virtio-video which effectively becomes an encapsulation of v4l to the
>> guest.
> 
> ... but USB video would also be nice, wouldn't it? That could enable 
> guests to use webcams without needing additional virtio drivers for it, 
> I think?
> 

Agree. UVC based hardware emulation does not need additional guest 
driver, and it may help guest kernel driver debug. It's also useful for 
USB video related education.

Virtio based hardware will use the common video/camera framework, this 
is also another good shape: hardware devices emulation -> generic 
framework -> multiple backend driver implement.