Re: [PATCH v1 9/9] docs/system: add basic virtio-gpu documentation

[Gurchetan Singh <gurchetansingh@chromium.org>]

On Wed, Jul 12, 2023 at 2:40 PM Akihiko Odaki <akihiko.odaki@gmail.com> wrote:
>
> On 2023/07/11 11:56, Gurchetan Singh wrote:
> > This adds basic documentation for virtio-gpu.
>
> Thank you for adding documentation for other backends too. I have been
> asked how virtio-gpu works so many times and always had to explain by
> myself though Gerd does have a nice article.* This documentation will help.
>
> * https://www.kraxel.org/blog/2021/05/virtio-gpu-qemu-graphics-update/
>
> >
> > Suggested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
> > Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
> > ---
> >   docs/system/device-emulation.rst   |  1 +
> >   docs/system/devices/virtio-gpu.rst | 80 ++++++++++++++++++++++++++++++
> >   2 files changed, 81 insertions(+)
> >   create mode 100644 docs/system/devices/virtio-gpu.rst
> >
> > diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulation.rst
> > index 4491c4cbf7..1167f3a9f2 100644
> > --- a/docs/system/device-emulation.rst
> > +++ b/docs/system/device-emulation.rst
> > @@ -91,6 +91,7 @@ Emulated Devices
> >      devices/nvme.rst
> >      devices/usb.rst
> >      devices/vhost-user.rst
> > +   devices/virtio-gpu.rst
> >      devices/virtio-pmem.rst
> >      devices/vhost-user-rng.rst
> >      devices/canokey.rst
> > diff --git a/docs/system/devices/virtio-gpu.rst b/docs/system/devices/virtio-gpu.rst
> > new file mode 100644
> > index 0000000000..2426039540
> > --- /dev/null
> > +++ b/docs/system/devices/virtio-gpu.rst
> > @@ -0,0 +1,80 @@
> > +..
> > +   SPDX-License-Identifier: GPL-2.0
> > +
> > +virtio-gpu
> > +==========
> > +
> > +This document explains the setup and usage of the virtio-gpu device.
> > +The virtio-gpu device paravirtualizes the GPU and display controller.
> > +
> > +Linux kernel support
> > +--------------------
> > +
> > +virtio-gpu requires a guest Linux kernel built with the
> > +``CONFIG_DRM_VIRTIO_GPU`` option.
> > +
> > +QEMU virtio-gpu variants
> > +------------------------
> > +
> > +There are many virtio-gpu device variants, listed below:
> > +
> > + * ``virtio-vga``
> > + * ``virtio-gpu-pci``
> > + * ``virtio-vga-gl``
> > + * ``virtio-gpu-gl-pci``
> > + * ``virtio-vga-rutabaga``
> > + * ``virtio-gpu-rutabaga-pci``
> > + * ``vhost-user-vga``
> > + * ``vhost-user-gl-pci``
>
> > +
> > +QEMU provides a 2D virtio-gpu backend, and two accelerated backends:
> > +virglrenderer ('gl' device label) and rutabaga_gfx ('rutabaga' device
> > +label).  There is also a vhost-user backend that runs the 2D device > +in a separate process.  Each device type as VGA or PCI variant.  This
> > +document uses the PCI variant in examples.
>
> I suggest to replace "2D device" with "graphics stack"; vhost-user works
> with 3D too. It's also slightly awkward to say a device runs in a
> separate process as some portion of device emulation always stuck in
> QEMU. In my opinion, the point of vhost-user backend is to isolate the
> gigantic graphics stack so let's put this phrase.
>
> I also have a bit different understanding regarding virtio-gpu variants.
> First, the variants can be classified into VGA and non-VGA ones. The VGA
> ones are prefixed with virtio-vga or vhost-user-vga while the non-VGA
> ones are prefixed with virtio-gpu or vhost-user-gpu.
>
> The VGA ones always use PCI interface, but for the non-VGA ones, you can
> further pick simple MMIO or PCI. For MMIO, you can suffix the device
> name with -device though vhost-user-gpu apparently does not support
> MMIO. For PCI, you can suffix it with -pci. Without these suffixes, the
> platform default will be chosen.
>
> Since enumerating all variants will result in a long list, you may
> provide abstract syntaxes like the following for this explanation:
>
> * virtio-vga[-BACKEND]
> * virtio-gpu[-BACKEND][-INTERFACE]
> * vhost-user-vga
> * vhost-user-pci
>
> > +
> > +virtio-gpu 2d
> > +-------------
> > +
> > +The default 2D mode uses a guest software renderer (llvmpipe, lavapipe,
> > +Swiftshader) to provide the OpenGL/Vulkan implementations.
>
> It's certainly possible to use virtio-gpu without software
> OpenGL/Vulkan. A major example is Windows; its software renderer is
> somewhat limited in my understanding.
>
> My suggestion:
> The default 2D backend only performs 2D operations. The guest needs to
> employ a software renderer for 3D graphics.
>
> It's also better to provide links for the renderers. Apparently lavapipe
> does not have a dedicated documentation, so you may add a link for Mesa
> and mention them like:
> LLVMpipe and Lavapipe included in `Mesa`_, or `SwiftShader`_
>
> And I think it will be helpful to say LLVMpipe and Lavapipe work out of
> box on typical modern Linux distributions as that should be what people
> care.
>
> > +
> > +.. parsed-literal::
> > +    -device virtio-gpu-pci
> > +
> > +virtio-gpu virglrenderer
> > +------------------------
> > +
> > +When using virgl accelerated graphics mode, OpenGL API calls are translated
> > +into an intermediate representation (see `Gallium3D`_). The intermediate
> > +representation is communicated to the host and the `virglrenderer`_ library
> > +on the host translates the intermediate representation back to OpenGL API
> > +calls.
> It should be mentioned that the translation occurs in the guest side,
> and the guest side component is included in Linux distributions as like
> LLVMpipe and Lavapipe are.
>
> > +
> > +.. parsed-literal::
> > +    -device virtio-gpu-gl-pci
> > +
> > +.. _Gallium3D: https://www.freedesktop.org/wiki/Software/gallium/
> > +.. _virglrenderer: https://gitlab.freedesktop.org/virgl/virglrenderer/
> > +
> > +virtio-gpu rutabaga
> > +-------------------
> > +
> > +virtio-gpu can also leverage `rutabaga_gfx`_ to provide `gfxstream`_ rendering
> > +and `Wayland display passthrough`_.  With the gfxstream rendering mode, GLES
> > +and Vulkan calls are forwarded directly to the host with minimal modification.
>
> I find the description included in the PDF you posted on GitLab* quite a
> useful so I suggest to incorporate its content.
>
> You may omit the overall design diagram as it mentions guest side and
> Rutabaga details and crosvm and may be confusing for QEMU users.
>
> The detailed commands for building dependencies may also be omitted and
> instead point to the documentation of respective projects as they should
> be subject to future changes.
>
> It's unfortunate that rutabaga_gfx and goldfish-opengl do not come with
> proper documentations (and I wonder rutabaga_gfx still need a hack
> mentioned in the PDF). For now the procedure to build them should be
> included in the documentation since it will take hours to figure out for
> a first-time reader otherwise.
>
> *
> https://gitlab.com/qemu-project/qemu/uploads/f960580bf0f19077e0330960b4a3152e/gfxstream_+_QEMU_setup__public_.pdf

The new doc in https://gitlab.com/qemu-project/qemu/-/issues/1611#note_1464562962
doesn't require the hack patch.  I'll incorporate your other
suggestions in v2.



> > +
> > +Please refer the `crosvm book`_ on how to setup the guest for Wayland
> > +passthrough (QEMU uses the same implementation).
> > +
> > +This device does require host blob support (``hostmem`` field below), but not
> > +all capsets (``capset_names`` below) have to enabled when starting the device.
> > +
> > +.. parsed-literal::
> > +    -device virtio-gpu-rutabaga-pci,capset_names=gfxstream-vulkan:cross-domain,\\
> > +      hostmem=8G,wayland_socket_path="$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY"
> > +
> > +.. _rutabaga_gfx: https://github.com/google/crosvm/blob/main/rutabaga_gfx/ffi/src/include/rutabaga_gfx_ffi.h
> > +.. _gfxstream: https://android.googlesource.com/platform/hardware/google/gfxstream/
> > +.. _Wayland display passthrough: https://www.youtube.com/watch?v=OZJiHMtIQ2M
> > +.. _crosvm book: https://crosvm.dev/book/devices/wayland.html