Re: [PATCH v6 00/10] Support virtio-gpu DRM native context

["Alex Bennée" <alex.bennee@linaro.org>]

Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> On 2/14/25 17:33, Alex Bennée wrote:
>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>> 
>>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>>
>>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>>> DRM native context [1] mediates lower level kernel driver UAPI, which
>>> reflects in a less CPU overhead and less/simpler code needed to support it.
>>> DRM context consists of a host and guest parts that have to be implemented
>>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>>> a real/native host GPU device for GL/VK applications.
>>>
>> <snip>
>> 
>> So first the good news. I can now get this up and running (x86/kvm guest
>> with Intel graphics) and as far as I can tell the native context mode is
>> working. With Dongwon Kim's patch the mirroring/corruption I was seeing
>> is gone.
>> 
>> I can successfully run glmark2-wayland (although see bellow) but vkmark
>> completely fails to start reporting:
>> 
>>   MESA: info: virtgpu backend not enabling VIRTGPU_PARAM_CREATE_FENCE_PASSING
>>   MESA: info: virtgpu backend not enabling VIRTGPU_PARAM_CREATE_GUEST_HANDLE
>>   MESA: error: DRM_IOCTL_VIRTGPU_GET_CAPS failed with Invalid argument
>>   MESA: error: DRM_IOCTL_VIRTGPU_CONTEXT_INIT failed with Invalid argument, continuing without context...
>>   MESA: error: DRM_VIRTGPU_RESOURCE_CREATE_BLOB failed with No space left on device
>>   MESA: error: Failed to create virtgpu AddressSpaceStream
>>   MESA: error: vulkan: Failed to get host connection
>>   MESA: error: DRM_VIRTGPU_RESOURCE_CREATE_BLOB failed with No space left on device
>>   MESA: error: Failed to create virtgpu AddressSpaceStream
>>   MESA: error: vulkan: Failed to get host connection
>>   MESA: error: DRM_VIRTGPU_RESOURCE_CREATE_BLOB failed with No space left on device
>>   MESA: error: Failed to create virtgpu AddressSpaceStream
>>   MESA: error: vulkan: Failed to get host connection
>>   MESA: warning: ../src/gfxstream/guest/vulkan/gfxstream_vk_device.cpp:681: VK_ERROR_DEVICE_LOST
>>   MESA: error: DRM_VIRTGPU_RESOURCE_CREATE_BLOB failed with No space left on device
>>   MESA: error: Failed to create virtgpu AddressSpaceStream
>>   MESA: error: vulkan: Failed to get host connection
>>   MESA: warning: ../src/gfxstream/guest/vulkan/gfxstream_vk_device.cpp:332: VK_ERROR_DEVICE_LOST
>>   === Physical Device 0 ===
>>       Vendor ID:      0x8086
>>       Device ID:      0xA780
>>       Device Name:    Intel(R) Graphics (RPL-S)
>>       Driver Version: 101068899
>>       Device UUID:    b39e1cf39b101489e3c6039406f78d6c
>> 
>> I was booting with 4G of shared memory.
>
> Thanks for the testing.
>
> I assume all these errors are generated by the failing gfxstream. Hence,
> may ignore them since you don't have enabled gfxstream.
>
>> Later versions of vkmark (2025.01) fail due to missing the
>> VK_KHR_display extension required as of
>> https://github.com/vkmark/vkmark/commit/7c3189c6482cb84c3c0e69d6dabb9d80e0c0092a
>
> This VK_KHR_display problem is only reproducible with your rootfs that
> you shared with me. It could be a trouble with your build configs or a
> buggy package version used by your rootfs build, more likely the
> former.

So you have built that latest vkmark? This is a recent addition to
vkmark for the 2025.1 release.

Does vulkaninfo --summary show the extension available for you? It is
certainly available on the host side:

VK_KHR_display                         : extension revision 23

>>> # Note about known performance problem in Qemu:
>>>
>>> DRM contexts are mapping host blobs extensively and these mapping
>>> operations work slowly in Qemu. Exact reason is unknown. Mappings work
>>> fast on Crosvm For DRM contexts this problem is more visible than for
>>> Venus/Virgl.
>> 
>> And how!
>> 
>> With drm_native I get a lot of stutter while running and barely 100FPS
>> (compared to ~8000 on pure venus). IMHO we need to figure out why there
>> is such a discrepancy before merging because currently it makes more
>> sense to use 
> If you'd run with Xorg/Wayland directly without a DE, then it should
> work okay. This should be a problem with unmapping performance that I'm
> thinking about.
>
> That unmapping problem is partially understood. Unmapping code works
> correctly, but we'll need to optimize the flatview code to perform
> unmapping immediately.

Why immediately? Surely if we are unmapping we can defer it. Or is this
a case of having stale mappings making the life of new allocations
harder?

> Meanwhile, you may apply the QEMU hack below, it
> should resolve most of the stutter, please let me know if it helps.
>
> There is also a pending Mesa intel-virtio blob mapping optimization that
> currently isn't available in my gitlab code, I'll refresh that feature
> and then ask you to try it.
>
> Could be that there is more to the unmapping perf issue in QEMU. I'm
> investigating.
>
> AMDGPU nctx is less affected by the bad unmapping performance. I expect
> it will work well for you.
>
>
>
> diff --git a/util/rcu.c b/util/rcu.c
> index fa32c942e4bb..aac3522c323c 100644
> --- a/util/rcu.c
> +++ b/util/rcu.c
> @@ -174,7 +174,7 @@ void synchronize_rcu(void)
>  }
>
>
> -#define RCU_CALL_MIN_SIZE        30
> +#define RCU_CALL_MIN_SIZE        1
>
>  /* Multi-producer, single-consumer queue based on urcu/static/wfqueue.h
>   * from liburcu.  Note that head is only used by the consumer.
> @@ -267,7 +267,7 @@ static void *call_rcu_thread(void *opaque)
>           * added before synchronize_rcu() starts.
>           */
>          while (n == 0 || (n < RCU_CALL_MIN_SIZE && ++tries <= 5)) {
> -            g_usleep(10000);
> +            g_usleep(1000);
>              if (n == 0) {
>                  qemu_event_reset(&rcu_call_ready_event);
>                  n = qatomic_read(&rcu_call_count);

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro