Re: [PATCH 0/3] virtio-gpu: Enable virglrenderer backend for rutabaga

[Weifeng Liu <weifeng.liu.z@gmail.com>]

Hi Alex,

On Thu, 2024-06-06 at 11:43 +0100, Alex Bennée wrote:
> Weifeng Liu <weifeng.liu.z@gmail.com> writes:
> 
> > Greetings,
> > 
> > I'd like to introduce you my attempt to enable virglrenderer backend for
> > rutabaga empowered virtio-gpu device.  I am aware that there have been
> > effort in supporting venus in virtio-gpu-virgl.c [1], but there is no
> > reason to prevent us from leveraging the virglrenderer component in
> > rutabaga_gfx, especially it being not very hard to add this
> > functionality.
> > 
> > Generally, the gap is the polling capability, i.e., virglrenderer
> > requires the main thread (namely the GPU command handling thread) to
> > poll virglrenderer at proper moments, which is not yet supported in
> > virtio-gpu-rutabaga device. This patch set try to add this so that
> > virglrenderer backend (including virgl and venus) can work as expected.
> > 
> > Slight change to rutabaga_gfx_ffi is also a requirement, which is
> > included in [2].
> > 
> > Further effort is required to tune the performance, since copying is
> > present before the rendered images get displayed. But I still think this
> > patch set could be a good starting point for the pending work.
> > 
> > For those interested in setting up environment and playing around with
> > this patch set, here is guideline in brief:
> > 
> > 1. Clone the master/main branch of virglrenderer, compile and install it.
> > 
> > 	git clone https://gitlab.freedesktop.org/virgl/virglrenderer
> > 	cd virglrenderer
> > 	meson setup builddir \
> > 	  --prefix=$INSTALL_DIR/virglrenderer \
> > 	  -Dvenus=true
> > 	ninja -C builddir install
> > 
> > 2. Clone the patched CrosVM, build and install rutabaga_gfx_ffi.
> > 
> > 	git clone -b rutabaga_ffi_virgl https://github.com/phreer/crosvm.git
> > 	cd crosvm/rutabaga_gfx/ffi
> > 	export PKG_CONFIG_PATH=$INSTALL_DIR/virglrenderer/lib64/pkgconfig/
> > 	meson setup builddir/ \
> > 	  --prefix $HOME/install/rutabaga_gfx/rutabaga_gfx_ffi/ \
> > 	  -Dvirglrenderer=true
> > 	ninja -C builddir install
> 
> Is there a PR going in for this? The moving parts for rutabaga are
> complex enough I think we need support upstream before merging this.
> 

It's true that this patch set depends on the change of
rutabaga_gfx_ffi. I am trying get the modifications of
crosvm/rubataga_gfx_ffi merged in upstream, please refer to this link:
https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645

> Is this branch where I should be getting the poll helpers from?
> 
>   cc -m64 @qemu-system-arm.rsp
>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_fence_poll':
>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:909: undefined reference to `rutabaga_poll'
>   /usr/bin/ld: libcommon.fa.p/hw_display_virtio-gpu-rutabaga.c.o: in function `virtio_gpu_rutabaga_init':
>   /home/alex/lsrc/qemu.git/builds/vulkan/../../hw/display/virtio-gpu-rutabaga.c:1122: undefined reference to `rutabaga_poll_descriptor'
>   collect2: error: ld returned 1 exit status
>   ninja: build stopped: subcommand failed.
> 

The required patches are applied to the rutabaga_ffi_virgl branch of my
clone of crosvm already, so please check out to that branch.

> 
> > 3. Applied this patch set to QEMU, build and install it:
> > 
> > 	cd qemu	
> > 	# Apply this patch set atop main branch ...
> > 	mkdir builddir; cd builddir
> > 	../configure --prefix=$INSTALL_DIR/qemu \
> > 	  --target-list=x86_64-softmmu \
> > 	  --disable-virglrenderer \
> > 	  --enable-rutabaga_gfx
> > 	ninja -C builddir install
> > 
> > 4. If you are lucky and everything goes fine, you are prepared to launch
> >    VM with virglrenderer backed virtio-gpu-rutabaga device:
> > 
> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/virglrenderer/lib64/:$LD_LIBRARY_PATH
> > 	export LD_LIBRARY_PATH=$INSTALL_DIR/rutabaga_gfx_ffi/lib64/:$LD_LIBRARY_PATH
> > 	$INSTALL_DIR/qemu/bin/qemu-system-x86_64
> > 	$QEMU -d guest_errors -enable-kvm -M q35 -smp 4 -m $MEM \
> > 	  -object memory-backend-memfd,id=mem1,size=$MEM \
> > 	  -machine memory-backend=mem1 \
> > 	  -device virtio-vga-rutabaga,venus=on,virgl2=on,wsi=surfaceless,hostmem=$MEM \
> > 
> 
> This should go into docs/system/devices/virtio-gpu.rst with some
> explanation. Is there anything we need on the guest side or does this
> skip the encapsulating requirements of wayland?
> 

Yeah, it's a good idea to add doc to explain the usage, thanks!

Best regards,
Weifeng

> > Note:
> > 
> > - You might need this patch set [3] to avoid KVM bad address error when
> >   you are running on a GPU using TTM for memory management.
> > 
> > [1] https://lore.kernel.org/all/dba6eb97-e1d1-4694-bfb6-e72db95715dd@daynix.com/T/
> > [2] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/5599645/1
> > [3] https://lore.kernel.org/kvm/20240229025759.1187910-1-stevensd@google.com/
> > 
> > Weifeng Liu (3):
> >   virtio-gpu: rutabaga: Properly set stride when copying resources
> >   virtio-gpu: rutabaga: Poll rutabaga upon events
> >   virtio-gpu: rutabaga: Add options to enable virgl and venus contexts
> > 
> >  hw/display/virtio-gpu-rutabaga.c | 104 ++++++++++++++++++++++++++++++-
> >  include/hw/virtio/virtio-gpu.h   |   1 +
> >  2 files changed, 104 insertions(+), 1 deletion(-)
>