Re: [PATCH v1 9/9] docs/system: add basic virtio-gpu documentation

[Akihiko Odaki <akihiko.odaki@gmail.com>]

On 2023/07/11 11:56, Gurchetan Singh wrote:
> This adds basic documentation for virtio-gpu.

Thank you for adding documentation for other backends too. I have been 
asked how virtio-gpu works so many times and always had to explain by 
myself though Gerd does have a nice article.* This documentation will help.

* https://www.kraxel.org/blog/2021/05/virtio-gpu-qemu-graphics-update/

> 
> Suggested-by: Akihiko Odaki <akihiko.odaki@daynix.com>
> Signed-off-by: Gurchetan Singh <gurchetansingh@chromium.org>
> ---
>   docs/system/device-emulation.rst   |  1 +
>   docs/system/devices/virtio-gpu.rst | 80 ++++++++++++++++++++++++++++++
>   2 files changed, 81 insertions(+)
>   create mode 100644 docs/system/devices/virtio-gpu.rst
> 
> diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulation.rst
> index 4491c4cbf7..1167f3a9f2 100644
> --- a/docs/system/device-emulation.rst
> +++ b/docs/system/device-emulation.rst
> @@ -91,6 +91,7 @@ Emulated Devices
>      devices/nvme.rst
>      devices/usb.rst
>      devices/vhost-user.rst
> +   devices/virtio-gpu.rst
>      devices/virtio-pmem.rst
>      devices/vhost-user-rng.rst
>      devices/canokey.rst
> diff --git a/docs/system/devices/virtio-gpu.rst b/docs/system/devices/virtio-gpu.rst
> new file mode 100644
> index 0000000000..2426039540
> --- /dev/null
> +++ b/docs/system/devices/virtio-gpu.rst
> @@ -0,0 +1,80 @@
> +..
> +   SPDX-License-Identifier: GPL-2.0
> +
> +virtio-gpu
> +==========
> +
> +This document explains the setup and usage of the virtio-gpu device.
> +The virtio-gpu device paravirtualizes the GPU and display controller.
> +
> +Linux kernel support
> +--------------------
> +
> +virtio-gpu requires a guest Linux kernel built with the
> +``CONFIG_DRM_VIRTIO_GPU`` option.
> +
> +QEMU virtio-gpu variants
> +------------------------
> +
> +There are many virtio-gpu device variants, listed below:
> +
> + * ``virtio-vga``
> + * ``virtio-gpu-pci``
> + * ``virtio-vga-gl``
> + * ``virtio-gpu-gl-pci``
> + * ``virtio-vga-rutabaga``
> + * ``virtio-gpu-rutabaga-pci``
> + * ``vhost-user-vga``
> + * ``vhost-user-gl-pci``

> +
> +QEMU provides a 2D virtio-gpu backend, and two accelerated backends:
> +virglrenderer ('gl' device label) and rutabaga_gfx ('rutabaga' device
> +label).  There is also a vhost-user backend that runs the 2D device > +in a separate process.  Each device type as VGA or PCI variant.  This
> +document uses the PCI variant in examples.

I suggest to replace "2D device" with "graphics stack"; vhost-user works 
with 3D too. It's also slightly awkward to say a device runs in a 
separate process as some portion of device emulation always stuck in 
QEMU. In my opinion, the point of vhost-user backend is to isolate the 
gigantic graphics stack so let's put this phrase.

I also have a bit different understanding regarding virtio-gpu variants.
First, the variants can be classified into VGA and non-VGA ones. The VGA 
ones are prefixed with virtio-vga or vhost-user-vga while the non-VGA 
ones are prefixed with virtio-gpu or vhost-user-gpu.

The VGA ones always use PCI interface, but for the non-VGA ones, you can 
further pick simple MMIO or PCI. For MMIO, you can suffix the device 
name with -device though vhost-user-gpu apparently does not support 
MMIO. For PCI, you can suffix it with -pci. Without these suffixes, the 
platform default will be chosen.

Since enumerating all variants will result in a long list, you may 
provide abstract syntaxes like the following for this explanation:

* virtio-vga[-BACKEND]
* virtio-gpu[-BACKEND][-INTERFACE]
* vhost-user-vga
* vhost-user-pci

> +
> +virtio-gpu 2d
> +-------------
> +
> +The default 2D mode uses a guest software renderer (llvmpipe, lavapipe,
> +Swiftshader) to provide the OpenGL/Vulkan implementations.

It's certainly possible to use virtio-gpu without software 
OpenGL/Vulkan. A major example is Windows; its software renderer is 
somewhat limited in my understanding.

My suggestion:
The default 2D backend only performs 2D operations. The guest needs to 
employ a software renderer for 3D graphics.

It's also better to provide links for the renderers. Apparently lavapipe 
does not have a dedicated documentation, so you may add a link for Mesa 
and mention them like:
LLVMpipe and Lavapipe included in `Mesa`_, or `SwiftShader`_

And I think it will be helpful to say LLVMpipe and Lavapipe work out of 
box on typical modern Linux distributions as that should be what people 
care.

> +
> +.. parsed-literal::
> +    -device virtio-gpu-pci
> +
> +virtio-gpu virglrenderer
> +------------------------
> +
> +When using virgl accelerated graphics mode, OpenGL API calls are translated
> +into an intermediate representation (see `Gallium3D`_). The intermediate
> +representation is communicated to the host and the `virglrenderer`_ library
> +on the host translates the intermediate representation back to OpenGL API
> +calls.
It should be mentioned that the translation occurs in the guest side, 
and the guest side component is included in Linux distributions as like 
LLVMpipe and Lavapipe are.

> +
> +.. parsed-literal::
> +    -device virtio-gpu-gl-pci
> +
> +.. _Gallium3D: https://www.freedesktop.org/wiki/Software/gallium/
> +.. _virglrenderer: https://gitlab.freedesktop.org/virgl/virglrenderer/
> +
> +virtio-gpu rutabaga
> +-------------------
> +
> +virtio-gpu can also leverage `rutabaga_gfx`_ to provide `gfxstream`_ rendering
> +and `Wayland display passthrough`_.  With the gfxstream rendering mode, GLES
> +and Vulkan calls are forwarded directly to the host with minimal modification.

I find the description included in the PDF you posted on GitLab* quite a 
useful so I suggest to incorporate its content.

You may omit the overall design diagram as it mentions guest side and 
Rutabaga details and crosvm and may be confusing for QEMU users.

The detailed commands for building dependencies may also be omitted and 
instead point to the documentation of respective projects as they should 
be subject to future changes.

It's unfortunate that rutabaga_gfx and goldfish-opengl do not come with 
proper documentations (and I wonder rutabaga_gfx still need a hack 
mentioned in the PDF). For now the procedure to build them should be 
included in the documentation since it will take hours to figure out for 
a first-time reader otherwise.

* 
https://gitlab.com/qemu-project/qemu/uploads/f960580bf0f19077e0330960b4a3152e/gfxstream_+_QEMU_setup__public_.pdf

> +
> +Please refer the `crosvm book`_ on how to setup the guest for Wayland
> +passthrough (QEMU uses the same implementation).
> +
> +This device does require host blob support (``hostmem`` field below), but not
> +all capsets (``capset_names`` below) have to enabled when starting the device.
> +
> +.. parsed-literal::
> +    -device virtio-gpu-rutabaga-pci,capset_names=gfxstream-vulkan:cross-domain,\\
> +      hostmem=8G,wayland_socket_path="$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY"
> +
> +.. _rutabaga_gfx: https://github.com/google/crosvm/blob/main/rutabaga_gfx/ffi/src/include/rutabaga_gfx_ffi.h
> +.. _gfxstream: https://android.googlesource.com/platform/hardware/google/gfxstream/
> +.. _Wayland display passthrough: https://www.youtube.com/watch?v=OZJiHMtIQ2M
> +.. _crosvm book: https://crosvm.dev/book/devices/wayland.html