[PATCH v5 0/8] Support virtio-gpu DRM native context

[PATCH v5 0/8] Support virtio-gpu DRM native context

[Dmitry Osipenko]

This patchset adds DRM native context support to VirtIO-GPU on Qemu.

Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
DRM native context [1] mediates lower level kernel driver UAPI, which
reflects in a less CPU overhead and less/simpler code needed to support it.
DRM context consists of a host and guest parts that have to be implemented
for each GPU driver. On a guest side, DRM context presents a virtual GPU as
a real/native host GPU device for GL/VK applications.

[1] https://www.youtube.com/watch?v=9sFP_yddLLQ

Today there are four known DRM native context drivers existing in a wild:

  - Freedreno (Qualcomm SoC GPUs), completely upstreamed
  - AMDGPU, mostly merged into upstreams
  - Intel (i915), merge requests are opened
  - Asahi (Apple SoC GPUs), WIP status


# How to try out DRM context:

1. DRM context uses host blobs and requires latest developer version 
of Linux kernel [2] that has necessary KVM fixes.

[2] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/

2. Use latest libvirglrenderer from upstream git/main for Freedreno
and AMDGPU native contexts. For Intel use patches [3].

[3] https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1384

3. On guest, use latest Mesa version for Freedreno. For AMDGPU use
Mesa patches [4], for Intel [5].

[4] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21658
[5] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29870

4. On guest, use latest Linux kernel v6.6+. Apply patch [6] if you're
   running Xorg in guest.

[6] https://lore.kernel.org/dri-devel/20241020224725.179937-1-dmitry.osipenko@collabora.com/

Example Qemu cmdline that enables DRM context:

  qemu-system-x86_64 -device virtio-vga-gl,hostmem=4G,blob=on,drm_native_context=on \
      -machine q35,accel=kvm,memory-backend=mem1 \
      -object memory-backend-memfd,id=mem1,size=8G -m 8G


# Note about known performance problem in Qemu:

DRM contexts are mapping host blobs extensively and these mapping
operations work slowly in Qemu. Exact reason is unknown. Mappings work
fast on Crosvm For DRM contexts this problem is more visible than for
Venus/Virgl.

Changelog:

v5: - Added r-bs from Akihiko Odaki.

    - Added acks from Michael Tsirkin.

    - Fixed compilation warning using older version of virglrenderer that
      was reported by Alex Bennée. Noticed that I need to keep old
      virgl_write_fence() code around for the older virglrenderer in
      "Support  asynchronous fencing" patch, so added it back and verified
      that old virglrenderer works properly.

    - Added new patch from Alex Bennée that adds more virtio-gpu 
      documentation with a couple corrections and additions to it from me.

    - Rebased patches on top of latest staging tree.

v4: - Improved SDL2/dmabuf patch by reusing existing Meson X11 config 
      option, better handling EGL error and extending comment telling
      that it's safe to enable SDL2 EGL preference hint. As was suggested
      by Akihiko Odaki.

    - Replaced another QSLIST_FOREACH_SAFE with QSLIST_EMPTY+FIRST in
      the async-fencing patch for more consistency of the code. As was
      suggested by Akihiko Odaki.

    - Added missing braces around if-statement that was spotted by
      Alex Bennée.

    - Renamed 'drm=on' option of virtio-gpu-gl device to 
      'drm_native_context=on' for more clarity as was suggested by 
      Alex Bennée. Haven't added added new context-type option that 
      was also proposed by Alex, might do it with a separate patch.
      This context-type option will duplicate and depecate existing
      options, but in a longer run likely will be worthwhile adding
      it.

    - Dropped Linux headers-update patch as headers has been updated
      in the staging tree.

v3: - Improved EGL presence-check code on X11 systems for the SDL2
      hint that prefers EGL over GLX by using better ifdefs and checking
      Xlib presence at a build time to avoid build failure if lib SDL2
      and system are configured with a disabled X11 support. Also added
      clarifying comment telling that X11 hint doesn't affect Wayland
      systems. Suggested by Akihiko Odaki.

    - Corrected strerror(err) that used negative error where it should
      be positive and vice versa that was caught by Akihiko Odaki. Added
      clarifying comment for the case where we get positive error code
      from virglrenderer that differs from other virglrenderer API functions.

    - Improved QSLIST usage by dropping mutex protecting the async fence
      list and using atomic variant of QSLIST helpers instead. Switched away
      from using FOREACH helper to improve readability of the code, showing
      that we don't precess list in unoptimal way. Like was suggested by
      Akihiko Odaki.

    - Updated patchset base to Venus v18.

v2: - Updated SDL2-dmabuf patch by making use of error_report() and
      checking presense of X11+EGL in the system before making SDL2
      to prefer EGL backend over GLX, suggested by Akihiko Odaki.

    - Improved SDL2's dmabuf-presence check that wasn't done properly
      in v1, where EGL was set up only after first console was fully
      inited, and thus, SDL's display .has_dmabuf callback didn't work
      for the first console. Now dmabuf support status is pre-checked
      before console is registered.

    - Updated commit description of the patch that fixes SDL2's context
      switching logic with a more detailed explanation of the problem.
      Suggested by Akihiko Odaki.

    - Corrected rebase typo in the async-fencing patch and switched
      async-fencing to use a sigle-linked list instead of the double,
      as was suggested by Akihiko Odaki.

    - Replaced "=true" with "=on" in the DRM native context documentation
      example and made virtio_gpu_virgl_init() to fail with a error message
      if DRM context can't be initialized instead of giving a warning
      message, as was suggested by Akihiko Odaki.

    - Added patchew's dependecy tag to the cover letter as was suggested by
      Akihiko Odaki.


Alex Bennée (1):
  docs/system: Expand the virtio-gpu documentation

Dmitry Osipenko (6):
  ui/sdl2: Restore original context after new context creation
  virtio-gpu: Handle virgl fence creation errors
  virtio-gpu: Support asynchronous fencing
  virtio-gpu: Support DRM native context
  ui/sdl2: Don't disable scanout when display is refreshed
  ui/gtk: Don't disable scanout when display is refreshed

Pierre-Eric Pelloux-Prayer (1):
  ui/sdl2: Implement dpy dmabuf functions

 docs/system/devices/virtio-gpu.rst | 105 +++++++++++++++++--
 hw/display/virtio-gpu-gl.c         |   5 +
 hw/display/virtio-gpu-virgl.c      | 159 ++++++++++++++++++++++++++++-
 hw/display/virtio-gpu.c            |  15 +++
 include/hw/virtio/virtio-gpu.h     |  16 +++
 include/ui/sdl2.h                  |   7 ++
 meson.build                        |   6 +-
 ui/gtk-egl.c                       |   1 -
 ui/gtk-gl-area.c                   |   1 -
 ui/sdl2-gl.c                       |  68 +++++++++++-
 ui/sdl2.c                          |  42 ++++++++
 11 files changed, 411 insertions(+), 14 deletions(-)

-- 
2.47.1

[PATCH v5 1/8] ui/sdl2: Restore original context after new context creation

[Dmitry Osipenko]

SDL API changes GL context to a newly created GL context, which differs
from other GL providers that don't switch context. Change SDL backend to
restore the original GL context. This allows Qemu's virtio-gpu to support
new virglrenderer async-fencing feature for Virgl contexts, otherwise
virglrenderer's vrend creates a fence-sync context on the Qemu's
main-loop thread that erroneously stays in-use by the main-loop after
creation, not allowing vrend's fence-sync thread switch to this new
context that belongs to it.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 ui/sdl2-gl.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ui/sdl2-gl.c b/ui/sdl2-gl.c
index e01d9ab0c7bf..b1fe96d6af22 100644
--- a/ui/sdl2-gl.c
+++ b/ui/sdl2-gl.c
@@ -168,6 +168,9 @@ QEMUGLContext sdl2_gl_create_context(DisplayGLCtx *dgc,
                             SDL_GL_CONTEXT_PROFILE_ES);
         ctx = SDL_GL_CreateContext(scon->real_window);
     }
+
+    SDL_GL_MakeCurrent(scon->real_window, scon->winctx);
+
     return (QEMUGLContext)ctx;
 }
 
-- 
2.47.1

[PATCH v5 2/8] ui/sdl2: Implement dpy dmabuf functions

[Dmitry Osipenko]

From: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>

If EGL is used, we can rely on dmabuf to import textures without
doing copies.

To get this working on X11, we use the existing SDL hint:
SDL_HINT_VIDEO_X11_FORCE_EGL (because dmabuf can't be used with GLX).

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 include/ui/sdl2.h |  7 ++++++
 meson.build       |  6 ++---
 ui/sdl2-gl.c      | 64 +++++++++++++++++++++++++++++++++++++++++++++++
 ui/sdl2.c         | 42 +++++++++++++++++++++++++++++++
 4 files changed, 115 insertions(+), 4 deletions(-)

diff --git a/include/ui/sdl2.h b/include/ui/sdl2.h
index dbe6e3d9739b..9daf5ecffae7 100644
--- a/include/ui/sdl2.h
+++ b/include/ui/sdl2.h
@@ -45,6 +45,7 @@ struct sdl2_console {
     bool gui_keysym;
     SDL_GLContext winctx;
     QKbdState *kbd;
+    bool has_dmabuf;
 #ifdef CONFIG_OPENGL
     QemuGLShader *gls;
     egl_fb guest_fb;
@@ -96,5 +97,11 @@ void sdl2_gl_scanout_texture(DisplayChangeListener *dcl,
                              void *d3d_tex2d);
 void sdl2_gl_scanout_flush(DisplayChangeListener *dcl,
                            uint32_t x, uint32_t y, uint32_t w, uint32_t h);
+void sdl2_gl_scanout_dmabuf(DisplayChangeListener *dcl,
+                            QemuDmaBuf *dmabuf);
+void sdl2_gl_release_dmabuf(DisplayChangeListener *dcl,
+                            QemuDmaBuf *dmabuf);
+bool sdl2_gl_has_dmabuf(DisplayChangeListener *dcl);
+void sdl2_gl_console_init(struct sdl2_console *scon);
 
 #endif /* SDL2_H */
diff --git a/meson.build b/meson.build
index 15a066043b7b..dfee104582b0 100644
--- a/meson.build
+++ b/meson.build
@@ -1964,10 +1964,8 @@ if get_option('gtk') \
   endif
 endif
 
-x11 = not_found
-if gtkx11.found()
-  x11 = dependency('x11', method: 'pkg-config', required: gtkx11.found())
-endif
+x11 = dependency('x11', method: 'pkg-config', required: gtkx11.found())
+
 png = not_found
 if get_option('png').allowed() and have_system
    png = dependency('libpng', version: '>=1.6.34', required: get_option('png'),
diff --git a/ui/sdl2-gl.c b/ui/sdl2-gl.c
index b1fe96d6af22..8d53e340d40d 100644
--- a/ui/sdl2-gl.c
+++ b/ui/sdl2-gl.c
@@ -26,6 +26,8 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
 #include "ui/console.h"
 #include "ui/input.h"
 #include "ui/sdl2.h"
@@ -249,3 +251,65 @@ void sdl2_gl_scanout_flush(DisplayChangeListener *dcl,
 
     SDL_GL_SwapWindow(scon->real_window);
 }
+
+void sdl2_gl_scanout_dmabuf(DisplayChangeListener *dcl,
+                            QemuDmaBuf *dmabuf)
+{
+    struct sdl2_console *scon = container_of(dcl, struct sdl2_console, dcl);
+
+    assert(scon->opengl);
+    SDL_GL_MakeCurrent(scon->real_window, scon->winctx);
+
+    egl_dmabuf_import_texture(dmabuf);
+    if (!qemu_dmabuf_get_texture(dmabuf)) {
+        error_report("%s: failed fd=%d", __func__, qemu_dmabuf_get_fd(dmabuf));
+        return;
+    }
+
+    sdl2_gl_scanout_texture(dcl, qemu_dmabuf_get_texture(dmabuf), false,
+                            qemu_dmabuf_get_width(dmabuf),
+                            qemu_dmabuf_get_height(dmabuf),
+                            0, 0,
+                            qemu_dmabuf_get_width(dmabuf),
+                            qemu_dmabuf_get_height(dmabuf),
+                            NULL);
+
+    if (qemu_dmabuf_get_allow_fences(dmabuf)) {
+        scon->guest_fb.dmabuf = dmabuf;
+    }
+}
+
+void sdl2_gl_release_dmabuf(DisplayChangeListener *dcl,
+                            QemuDmaBuf *dmabuf)
+{
+    egl_dmabuf_release_texture(dmabuf);
+}
+
+bool sdl2_gl_has_dmabuf(DisplayChangeListener *dcl)
+{
+    struct sdl2_console *scon = container_of(dcl, struct sdl2_console, dcl);
+
+    return scon->has_dmabuf;
+}
+
+void sdl2_gl_console_init(struct sdl2_console *scon)
+{
+    bool hidden = scon->hidden;
+
+    scon->hidden = true;
+    scon->surface = qemu_create_displaysurface(1, 1);
+    sdl2_window_create(scon);
+
+    /*
+     * QEMU checks whether console supports dma-buf before switching
+     * to the console.  To break this chicken-egg problem we pre-check
+     * dma-buf availability beforehand using a dummy SDL window.
+     */
+    scon->has_dmabuf = qemu_egl_has_dmabuf();
+
+    sdl2_window_destroy(scon);
+    qemu_free_displaysurface(scon->surface);
+
+    scon->surface = NULL;
+    scon->hidden = hidden;
+}
diff --git a/ui/sdl2.c b/ui/sdl2.c
index 445eb1dd9f9a..3e262b994231 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -36,6 +36,10 @@
 #include "qemu/log.h"
 #include "qemu-main.h"
 
+#ifdef CONFIG_X11
+#include <X11/Xlib.h>
+#endif
+
 static int sdl2_num_outputs;
 static struct sdl2_console *sdl2_console;
 
@@ -121,6 +125,9 @@ void sdl2_window_create(struct sdl2_console *scon)
         /* The SDL renderer is only used by sdl2-2D, when OpenGL is disabled */
         scon->real_renderer = SDL_CreateRenderer(scon->real_window, -1, 0);
     }
+
+    qemu_egl_display = eglGetCurrentDisplay();
+
     sdl_update_caption(scon);
 }
 
@@ -821,6 +828,10 @@ static const DisplayChangeListenerOps dcl_gl_ops = {
     .dpy_gl_scanout_disable  = sdl2_gl_scanout_disable,
     .dpy_gl_scanout_texture  = sdl2_gl_scanout_texture,
     .dpy_gl_update           = sdl2_gl_scanout_flush,
+
+    .dpy_gl_scanout_dmabuf   = sdl2_gl_scanout_dmabuf,
+    .dpy_gl_release_dmabuf   = sdl2_gl_release_dmabuf,
+    .dpy_has_dmabuf          = sdl2_gl_has_dmabuf,
 };
 
 static bool
@@ -848,6 +859,35 @@ static void sdl2_display_early_init(DisplayOptions *o)
     }
 }
 
+static void sdl2_set_hint_x11_force_egl(void)
+{
+#if defined(SDL_HINT_VIDEO_X11_FORCE_EGL) && defined(CONFIG_OPENGL) && \
+    defined(CONFIG_X11)
+    Display *x_disp = XOpenDisplay(NULL);
+    EGLDisplay egl_display;
+
+    if (!x_disp) {
+        return;
+    }
+
+    /* Prefer EGL over GLX to get dma-buf support. */
+    egl_display = eglGetDisplay((EGLNativeDisplayType)x_disp);
+
+    if (egl_display != EGL_NO_DISPLAY) {
+        /*
+         * Setting X11_FORCE_EGL hint doesn't make SDL to prefer X11 over
+         * Wayland. SDL will use Wayland driver even if XWayland presents.
+         * It's always safe to set the hint even if X11 is not used by SDL.
+         * SDL will work regardless of the hint.
+         */
+        SDL_SetHint(SDL_HINT_VIDEO_X11_FORCE_EGL, "1");
+        eglTerminate(egl_display);
+    }
+
+    XCloseDisplay(x_disp);
+#endif
+}
+
 static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
 {
     uint8_t data = 0;
@@ -878,6 +918,7 @@ static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
     SDL_SetHint(SDL_HINT_ALLOW_ALT_TAB_WHILE_GRABBED, "0");
 #endif
     SDL_SetHint(SDL_HINT_WINDOWS_NO_CLOSE_ON_ALT_F4, "1");
+    sdl2_set_hint_x11_force_egl();
     SDL_EnableScreenSaver();
     memset(&info, 0, sizeof(info));
     SDL_VERSION(&info.version);
@@ -924,6 +965,7 @@ static void sdl2_display_init(DisplayState *ds, DisplayOptions *o)
         sdl2_console[i].kbd = qkbd_state_init(con);
         if (display_opengl) {
             qemu_console_set_display_gl_ctx(con, &sdl2_console[i].dgc);
+            sdl2_gl_console_init(&sdl2_console[i]);
         }
         register_displaychangelistener(&sdl2_console[i].dcl);
 
-- 
2.47.1

[PATCH v5 3/8] virtio-gpu: Handle virgl fence creation errors

[Dmitry Osipenko]

Print out error messages when virgl fence creation fails to aid debugging
of the fence-related bugs.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 hw/display/virtio-gpu-virgl.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
index 145a0b387956..2eb6aaab4e84 100644
--- a/hw/display/virtio-gpu-virgl.c
+++ b/hw/display/virtio-gpu-virgl.c
@@ -872,6 +872,7 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
                                       struct virtio_gpu_ctrl_command *cmd)
 {
     bool cmd_suspended = false;
+    int ret;
 
     VIRTIO_GPU_FILL_CMD(cmd->cmd_hdr);
 
@@ -970,7 +971,17 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
     }
 
     trace_virtio_gpu_fence_ctrl(cmd->cmd_hdr.fence_id, cmd->cmd_hdr.type);
-    virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, cmd->cmd_hdr.type);
+
+    /*
+     * Unlike other virglrenderer functions, this one returns a positive
+     * error code.
+     */
+    ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0);
+    if (ret) {
+        qemu_log_mask(LOG_GUEST_ERROR,
+                      "%s: virgl_renderer_create_fence error: %s",
+                      __func__, strerror(ret));
+    }
 }
 
 static void virgl_write_fence(void *opaque, uint32_t fence)
-- 
2.47.1

[PATCH v5 4/8] virtio-gpu: Support asynchronous fencing

[Dmitry Osipenko]

Support asynchronous fencing feature of virglrenderer. It allows Qemu to
handle fence as soon as it's signalled instead of periodically polling
the fence status. This feature is required for enabling DRM context
support in Qemu because legacy fencing mode isn't supported for DRM
contexts in virglrenderer.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 hw/display/virtio-gpu-gl.c     |   3 +
 hw/display/virtio-gpu-virgl.c  | 142 ++++++++++++++++++++++++++++++---
 include/hw/virtio/virtio-gpu.h |  13 +++
 3 files changed, 149 insertions(+), 9 deletions(-)

diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index 683fad3bf8a8..d9bb50ac1d4a 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -169,6 +169,9 @@ static void virtio_gpu_gl_device_unrealize(DeviceState *qdev)
     if (gl->renderer_state >= RS_INITED) {
 #if VIRGL_VERSION_MAJOR >= 1
         qemu_bh_delete(gl->cmdq_resume_bh);
+
+        virtio_gpu_virgl_reset_async_fences(g);
+        qemu_bh_delete(gl->async_fence_bh);
 #endif
         if (virtio_gpu_stats_enabled(g->parent_obj.conf)) {
             timer_free(gl->print_stats);
diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
index 2eb6aaab4e84..15a465acf43b 100644
--- a/hw/display/virtio-gpu-virgl.c
+++ b/hw/display/virtio-gpu-virgl.c
@@ -871,6 +871,7 @@ static void virgl_cmd_set_scanout_blob(VirtIOGPU *g,
 void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
                                       struct virtio_gpu_ctrl_command *cmd)
 {
+    VirtIOGPUGL *gl = VIRTIO_GPU_GL(g);
     bool cmd_suspended = false;
     int ret;
 
@@ -972,15 +973,29 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
 
     trace_virtio_gpu_fence_ctrl(cmd->cmd_hdr.fence_id, cmd->cmd_hdr.type);
 
-    /*
-     * Unlike other virglrenderer functions, this one returns a positive
-     * error code.
-     */
-    ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0);
-    if (ret) {
-        qemu_log_mask(LOG_GUEST_ERROR,
-                      "%s: virgl_renderer_create_fence error: %s",
-                      __func__, strerror(ret));
+    if (gl->context_fence_enabled &&
+        (cmd->cmd_hdr.flags & VIRTIO_GPU_FLAG_INFO_RING_IDX)) {
+        uint32_t flags = 0;
+
+        ret = virgl_renderer_context_create_fence(cmd->cmd_hdr.ctx_id, flags,
+                                                  cmd->cmd_hdr.ring_idx,
+                                                  cmd->cmd_hdr.fence_id);
+        if (ret) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "%s: virgl_renderer_context_create_fence error: %s",
+                          __func__, strerror(-ret));
+        }
+    } else {
+        /*
+         * Unlike other virglrenderer functions, this one returns a positive
+         * error code.
+         */
+        ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0);
+        if (ret) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "%s: virgl_renderer_create_fence error: %s",
+                          __func__, strerror(ret));
+        }
     }
 }
 
@@ -1008,6 +1023,102 @@ static void virgl_write_fence(void *opaque, uint32_t fence)
     }
 }
 
+void virtio_gpu_virgl_reset_async_fences(VirtIOGPU *g)
+{
+    struct virtio_gpu_virgl_context_fence *f;
+    VirtIOGPUGL *gl = VIRTIO_GPU_GL(g);
+
+    while (!QSLIST_EMPTY(&gl->async_fenceq)) {
+        f = QSLIST_FIRST(&gl->async_fenceq);
+
+        QSLIST_REMOVE_HEAD(&gl->async_fenceq, next);
+
+        g_free(f);
+    }
+}
+
+#if VIRGL_VERSION_MAJOR >= 1
+static void virtio_gpu_virgl_async_fence_bh(void *opaque)
+{
+    QSLIST_HEAD(, virtio_gpu_virgl_context_fence) async_fenceq;
+    struct virtio_gpu_ctrl_command *cmd, *tmp;
+    struct virtio_gpu_virgl_context_fence *f;
+    VirtIOGPU *g = opaque;
+    VirtIOGPUGL *gl = VIRTIO_GPU_GL(g);
+
+    QSLIST_MOVE_ATOMIC(&async_fenceq, &gl->async_fenceq);
+
+    while (!QSLIST_EMPTY(&async_fenceq)) {
+        f = QSLIST_FIRST(&async_fenceq);
+
+        QSLIST_REMOVE_HEAD(&async_fenceq, next);
+
+        QTAILQ_FOREACH_SAFE(cmd, &g->fenceq, next, tmp) {
+            /*
+             * the guest can end up emitting fences out of order
+             * so we should check all fenced cmds not just the first one.
+             */
+            if (cmd->cmd_hdr.fence_id > f->fence_id) {
+                continue;
+            }
+            if (cmd->cmd_hdr.flags & VIRTIO_GPU_FLAG_INFO_RING_IDX) {
+                if (cmd->cmd_hdr.ring_idx != f->ring_idx) {
+                    continue;
+                }
+                if (cmd->cmd_hdr.ctx_id != f->ctx_id) {
+                    continue;
+                }
+            } else if (f->ring_idx >= 0) {
+                /* ctx0 GL-query fences don't have ring info */
+                continue;
+            }
+            virtio_gpu_ctrl_response_nodata(g, cmd, VIRTIO_GPU_RESP_OK_NODATA);
+            QTAILQ_REMOVE(&g->fenceq, cmd, next);
+            g_free(cmd);
+        }
+
+        trace_virtio_gpu_fence_resp(f->fence_id);
+        g_free(f);
+        g->inflight--;
+        if (virtio_gpu_stats_enabled(g->parent_obj.conf)) {
+            trace_virtio_gpu_dec_inflight_fences(g->inflight);
+        }
+    }
+}
+
+static void
+virtio_gpu_virgl_push_async_fence(VirtIOGPU *g, uint32_t ctx_id,
+                                  int64_t ring_idx, uint64_t fence_id)
+{
+    struct virtio_gpu_virgl_context_fence *f;
+    VirtIOGPUGL *gl = VIRTIO_GPU_GL(g);
+
+    f = g_new(struct virtio_gpu_virgl_context_fence, 1);
+    f->ctx_id = ctx_id;
+    f->ring_idx = ring_idx;
+    f->fence_id = fence_id;
+
+    QSLIST_INSERT_HEAD_ATOMIC(&gl->async_fenceq, f, next);
+
+    qemu_bh_schedule(gl->async_fence_bh);
+}
+
+static void virgl_write_async_fence(void *opaque, uint32_t fence)
+{
+    VirtIOGPU *g = opaque;
+
+    virtio_gpu_virgl_push_async_fence(g, 0, -1, fence);
+}
+
+static void virgl_write_async_context_fence(void *opaque, uint32_t ctx_id,
+                                            uint32_t ring_idx, uint64_t fence)
+{
+    VirtIOGPU *g = opaque;
+
+    virtio_gpu_virgl_push_async_fence(g, ctx_id, ring_idx, fence);
+}
+#endif
+
 static virgl_renderer_gl_context
 virgl_create_context(void *opaque, int scanout_idx,
                      struct virgl_renderer_gl_ctx_param *params)
@@ -1095,6 +1206,8 @@ void virtio_gpu_virgl_reset_scanout(VirtIOGPU *g)
         dpy_gfx_replace_surface(g->parent_obj.scanout[i].con, NULL);
         dpy_gl_scanout_disable(g->parent_obj.scanout[i].con);
     }
+
+    virtio_gpu_virgl_reset_async_fences(g);
 }
 
 void virtio_gpu_virgl_reset(VirtIOGPU *g)
@@ -1112,6 +1225,13 @@ int virtio_gpu_virgl_init(VirtIOGPU *g)
     if (qemu_egl_display) {
         virtio_gpu_3d_cbs.version = 4;
         virtio_gpu_3d_cbs.get_egl_display = virgl_get_egl_display;
+#if VIRGL_VERSION_MAJOR >= 1
+        virtio_gpu_3d_cbs.write_fence         = virgl_write_async_fence;
+        virtio_gpu_3d_cbs.write_context_fence = virgl_write_async_context_fence;
+        flags |= VIRGL_RENDERER_ASYNC_FENCE_CB;
+        flags |= VIRGL_RENDERER_THREAD_SYNC;
+        gl->context_fence_enabled = true;
+#endif
     }
 #endif
 #ifdef VIRGL_RENDERER_D3D11_SHARE_TEXTURE
@@ -1145,6 +1265,10 @@ int virtio_gpu_virgl_init(VirtIOGPU *g)
     gl->cmdq_resume_bh = aio_bh_new(qemu_get_aio_context(),
                                     virtio_gpu_virgl_resume_cmdq_bh,
                                     g);
+
+    gl->async_fence_bh = aio_bh_new(qemu_get_aio_context(),
+                                    virtio_gpu_virgl_async_fence_bh,
+                                    g);
 #endif
 
     return 0;
diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
index a42957c4e2cc..bd2cccdc60d7 100644
--- a/include/hw/virtio/virtio-gpu.h
+++ b/include/hw/virtio/virtio-gpu.h
@@ -230,6 +230,13 @@ struct VirtIOGPUClass {
                              Error **errp);
 };
 
+struct virtio_gpu_virgl_context_fence {
+    uint32_t ctx_id;
+    int64_t ring_idx;
+    uint64_t fence_id;
+    QSLIST_ENTRY(virtio_gpu_virgl_context_fence) next;
+};
+
 /* VirtIOGPUGL renderer states */
 typedef enum {
     RS_START,       /* starting state */
@@ -247,6 +254,11 @@ struct VirtIOGPUGL {
     QEMUTimer *print_stats;
 
     QEMUBH *cmdq_resume_bh;
+
+    QEMUBH *async_fence_bh;
+    QSLIST_HEAD(, virtio_gpu_virgl_context_fence) async_fenceq;
+
+    bool context_fence_enabled;
 };
 
 struct VhostUserGPU {
@@ -376,5 +388,6 @@ void virtio_gpu_virgl_reset_scanout(VirtIOGPU *g);
 void virtio_gpu_virgl_reset(VirtIOGPU *g);
 int virtio_gpu_virgl_init(VirtIOGPU *g);
 GArray *virtio_gpu_virgl_get_capsets(VirtIOGPU *g);
+void virtio_gpu_virgl_reset_async_fences(VirtIOGPU *g);
 
 #endif
-- 
2.47.1

Re: [PATCH v5 4/8] virtio-gpu: Support asynchronous fencing

[Dmitry Osipenko]

On 1/20/25 16:56, Alex Bennée wrote:
...
>> @@ -972,15 +973,29 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g,
>>  
>>      trace_virtio_gpu_fence_ctrl(cmd->cmd_hdr.fence_id, cmd->cmd_hdr.type);
>>  
>> -    /*
>> -     * Unlike other virglrenderer functions, this one returns a positive
>> -     * error code.
>> -     */
>> -    ret = virgl_renderer_create_fence(cmd->cmd_hdr.fence_id, 0);
>> -    if (ret) {
>> -        qemu_log_mask(LOG_GUEST_ERROR,
>> -                      "%s: virgl_renderer_create_fence error: %s",
>> -                      __func__, strerror(ret));
>> +    if (gl->context_fence_enabled &&
>> +        (cmd->cmd_hdr.flags & VIRTIO_GPU_FLAG_INFO_RING_IDX)) {
>> +        uint32_t flags = 0;
> 
> Is this is constant or do we expect to change this later?
There are no immediate plans for using this flags variable in QEMU
today. But in general context-specific flags could be specified here.
Crosv makes use of the flags.

>> +        ret = virgl_renderer_context_create_fence(cmd->cmd_hdr.ctx_id, flags,
>> +                                                  cmd->cmd_hdr.ring_idx,
>> +                                                  cmd->cmd_hdr.fence_id);
>> +        if (ret) {
>> +            qemu_log_mask(LOG_GUEST_ERROR,
>> +                          "%s: virgl_renderer_context_create_fence error: %s",
>> +                          __func__, strerror(-ret));
> 
> This still fails with older virglrenderers:
> 
> ../../hw/display/virtio-gpu-virgl.c: In function ‘virtio_gpu_virgl_process_cmd’:
> ../../hw/display/virtio-gpu-virgl.c:980:15: error: implicit declaration of function ‘virgl_renderer_context_create_fence’; did you mean ‘virgl_renderer_context_create’? [-Werror=implicit-function-declaration]
>   980 |         ret = virgl_renderer_context_create_fence(cmd->cmd_hdr.ctx_id, flags,
>       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>       |               virgl_renderer_context_create
> ../../hw/display/virtio-gpu-virgl.c:980:15: error: nested extern declaration of ‘virgl_renderer_context_create_fence’ [-Werror=nested-externs]
> cc1: all warnings being treated as errors
> [1981/2819] Compiling C object libcommon.a.p/ui_sdl2-gl.c.o

Indeed! Good catch again, thanks! Will fix in v6

-- 
Best regards,
Dmitry

[PATCH v5 5/8] virtio-gpu: Support DRM native context

[Dmitry Osipenko]

Add support for DRM native contexts to VirtIO-GPU. DRM context is enabled
using a new virtio-gpu-gl device option "drm_native_context=on".

Unlike Virgl and Venus contexts that operate on application API level,
DRM native contexts work on a kernel UAPI level. This lower level results
in a lightweight context implementations that yield better performance.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 docs/system/devices/virtio-gpu.rst | 11 +++++++++++
 hw/display/virtio-gpu-gl.c         |  2 ++
 hw/display/virtio-gpu-virgl.c      | 22 ++++++++++++++++++++++
 hw/display/virtio-gpu.c            | 15 +++++++++++++++
 include/hw/virtio/virtio-gpu.h     |  3 +++
 5 files changed, 53 insertions(+)

diff --git a/docs/system/devices/virtio-gpu.rst b/docs/system/devices/virtio-gpu.rst
index b7eb0fc0e727..f20c60016376 100644
--- a/docs/system/devices/virtio-gpu.rst
+++ b/docs/system/devices/virtio-gpu.rst
@@ -82,6 +82,17 @@ of virtio-gpu host memory window. This is typically between 256M and 8G.
 
 .. _venus: https://gitlab.freedesktop.org/virgl/venus-protocol/
 
+DRM native context is supported since release of `virglrenderer`_ v1.0.0
+using `drm`_ protocol.  ``DRM`` virtio-gpu capability set ("capset") requires
+host blob support (``hostmem`` and ``blob`` fields) and should be enabled
+using ``drm_native_context`` field.  The ``hostmem`` field specifies the size
+of virtio-gpu host memory window. This is typically between 256M and 8G.
+
+.. parsed-literal::
+    -device virtio-gpu-gl,hostmem=8G,blob=on,drm_native_context=on
+
+.. _drm: https://gitlab.freedesktop.org/virgl/virglrenderer/-/tree/main/src/drm
+
 virtio-gpu rutabaga
 -------------------
 
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index d9bb50ac1d4a..5f374ad56396 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -159,6 +159,8 @@ static const Property virtio_gpu_gl_properties[] = {
                     VIRTIO_GPU_FLAG_STATS_ENABLED, false),
     DEFINE_PROP_BIT("venus", VirtIOGPU, parent_obj.conf.flags,
                     VIRTIO_GPU_FLAG_VENUS_ENABLED, false),
+    DEFINE_PROP_BIT("drm_native_context", VirtIOGPU, parent_obj.conf.flags,
+                    VIRTIO_GPU_FLAG_DRM_ENABLED, false),
 };
 
 static void virtio_gpu_gl_device_unrealize(DeviceState *qdev)
diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c
index 15a465acf43b..34ba8118f31d 100644
--- a/hw/display/virtio-gpu-virgl.c
+++ b/hw/display/virtio-gpu-virgl.c
@@ -1243,6 +1243,19 @@ int virtio_gpu_virgl_init(VirtIOGPU *g)
     if (virtio_gpu_venus_enabled(g->parent_obj.conf)) {
         flags |= VIRGL_RENDERER_VENUS | VIRGL_RENDERER_RENDER_SERVER;
     }
+    if (virtio_gpu_drm_enabled(g->parent_obj.conf)) {
+        flags |= VIRGL_RENDERER_DRM;
+
+        if (!gl->context_fence_enabled) {
+            /*
+             * Virglrenderer skips enabling DRM context support without
+             * enabled async-fence feature. VirtIO-GPU will initialize
+             * successfully, but DRM context won't be available in guest.
+             */
+            error_report("DRM native context requires EGL display");
+            return -EINVAL;
+        }
+    }
 #endif
 
     ret = virgl_renderer_init(g, flags, &virtio_gpu_3d_cbs);
@@ -1305,5 +1318,14 @@ GArray *virtio_gpu_virgl_get_capsets(VirtIOGPU *g)
         }
     }
 
+    if (virtio_gpu_drm_enabled(g->parent_obj.conf)) {
+        virgl_renderer_get_cap_set(VIRTIO_GPU_CAPSET_DRM,
+                                   &capset_max_ver,
+                                   &capset_max_size);
+        if (capset_max_size) {
+            virtio_gpu_virgl_add_capset(capset_ids, VIRTIO_GPU_CAPSET_DRM);
+        }
+    }
+
     return capset_ids;
 }
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 11a7a8575027..165a0976480d 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1505,6 +1505,21 @@ void virtio_gpu_device_realize(DeviceState *qdev, Error **errp)
 #endif
     }
 
+    if (virtio_gpu_drm_enabled(g->parent_obj.conf)) {
+#ifdef VIRGL_VERSION_MAJOR
+    #if VIRGL_VERSION_MAJOR >= 1
+        if (!virtio_gpu_blob_enabled(g->parent_obj.conf) ||
+            !virtio_gpu_hostmem_enabled(g->parent_obj.conf)) {
+            error_setg(errp, "drm requires enabled blob and hostmem options");
+            return;
+        }
+    #else
+        error_setg(errp, "old virglrenderer, drm unsupported");
+        return;
+    #endif
+#endif
+    }
+
     if (!virtio_gpu_base_device_realize(qdev,
                                         virtio_gpu_handle_ctrl_cb,
                                         virtio_gpu_handle_cursor_cb,
diff --git a/include/hw/virtio/virtio-gpu.h b/include/hw/virtio/virtio-gpu.h
index bd2cccdc60d7..dcdf52b192b5 100644
--- a/include/hw/virtio/virtio-gpu.h
+++ b/include/hw/virtio/virtio-gpu.h
@@ -99,6 +99,7 @@ enum virtio_gpu_base_conf_flags {
     VIRTIO_GPU_FLAG_RUTABAGA_ENABLED,
     VIRTIO_GPU_FLAG_VENUS_ENABLED,
     VIRTIO_GPU_FLAG_RESOURCE_UUID_ENABLED,
+    VIRTIO_GPU_FLAG_DRM_ENABLED,
 };
 
 #define virtio_gpu_virgl_enabled(_cfg) \
@@ -121,6 +122,8 @@ enum virtio_gpu_base_conf_flags {
     (_cfg.hostmem > 0)
 #define virtio_gpu_venus_enabled(_cfg) \
     (_cfg.flags & (1 << VIRTIO_GPU_FLAG_VENUS_ENABLED))
+#define virtio_gpu_drm_enabled(_cfg) \
+    (_cfg.flags & (1 << VIRTIO_GPU_FLAG_DRM_ENABLED))
 
 struct virtio_gpu_base_conf {
     uint32_t max_outputs;
-- 
2.47.1

[PATCH v5 6/8] ui/sdl2: Don't disable scanout when display is refreshed

[Dmitry Osipenko]

Display refreshment is invoked by a timer and it erroneously disables
the active scanout if it happens to be invoked after scanout has been
enabled. This offending scanout-disable race condition with a timer
can be easily hit when Qemu runs with a disabled vsync by using SDL or
GTK displays (with vblank_mode=0 for GTK). Refreshment of display's
content shouldn't disable the active display. Fix it by keeping the
scanout's state unchanged when display is redrawn.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 ui/sdl2-gl.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/ui/sdl2-gl.c b/ui/sdl2-gl.c
index 8d53e340d40d..31f8fbe03286 100644
--- a/ui/sdl2-gl.c
+++ b/ui/sdl2-gl.c
@@ -53,7 +53,6 @@ static void sdl2_gl_render_surface(struct sdl2_console *scon)
     int ww, wh;
 
     SDL_GL_MakeCurrent(scon->real_window, scon->winctx);
-    sdl2_set_scanout_mode(scon, false);
 
     SDL_GetWindowSize(scon->real_window, &ww, &wh);
     surface_gl_setup_viewport(scon->gls, scon->surface, ww, wh);
-- 
2.47.1

[PATCH v5 7/8] ui/gtk: Don't disable scanout when display is refreshed

[Dmitry Osipenko]

Display refreshment is invoked by a timer and it erroneously disables
the active scanout if it happens to be invoked after scanout has been
enabled. This offending scanout-disable race condition with a timer
can be easily hit when Qemu runs with a disabled vsync by using SDL or
GTK displays (with vblank_mode=0 for GTK). Refreshment of display's
content shouldn't disable the active display. Fix it by keeping the
scanout's state unchanged when display is redrawn.

Reviewed-by: Akihiko Odaki <akihiko.odaki@daynix.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
---
 ui/gtk-egl.c     | 1 -
 ui/gtk-gl-area.c | 1 -
 2 files changed, 2 deletions(-)

diff --git a/ui/gtk-egl.c b/ui/gtk-egl.c
index f7a428c86a8d..0d1547d63ad0 100644
--- a/ui/gtk-egl.c
+++ b/ui/gtk-egl.c
@@ -179,7 +179,6 @@ void gd_egl_refresh(DisplayChangeListener *dcl)
 
     if (vc->gfx.glupdates) {
         vc->gfx.glupdates = 0;
-        gtk_egl_set_scanout_mode(vc, false);
         gd_egl_draw(vc);
     }
 }
diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
index 2c9a0db42571..53d81124f211 100644
--- a/ui/gtk-gl-area.c
+++ b/ui/gtk-gl-area.c
@@ -148,7 +148,6 @@ void gd_gl_area_refresh(DisplayChangeListener *dcl)
 
     if (vc->gfx.glupdates) {
         vc->gfx.glupdates = 0;
-        gtk_gl_area_set_scanout_mode(vc, false);
         gtk_gl_area_queue_render(GTK_GL_AREA(vc->gfx.drawing_area));
     }
 }
-- 
2.47.1

[PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

From: Alex Bennée <alex.bennee@linaro.org>

This attempts to tidy up the VirtIO GPU documentation to make the list
of requirements clearer. There are still a lot of moving parts and the
distros have some catching up to do before this is all handled
automatically.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Cc: Sergio Lopez Pascual <slp@redhat.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
[dmitry.osipenko@collabora.com: Extended and corrected doc]
---
 docs/system/devices/virtio-gpu.rst | 94 ++++++++++++++++++++++++++++--
 1 file changed, 88 insertions(+), 6 deletions(-)

diff --git a/docs/system/devices/virtio-gpu.rst b/docs/system/devices/virtio-gpu.rst
index f20c60016376..79af291a9316 100644
--- a/docs/system/devices/virtio-gpu.rst
+++ b/docs/system/devices/virtio-gpu.rst
@@ -4,14 +4,96 @@
 virtio-gpu
 ==========
 
-This document explains the setup and usage of the virtio-gpu device.
-The virtio-gpu device paravirtualizes the GPU and display controller.
-
-Linux kernel support
---------------------
+The virtio-gpu device provides a GPU and display controller
+paravirtualized using VirtIO. It supports a number of different modes
+from simple 2D displays to fully accelerated 3D graphics.
+
+Dependencies
+............
+
+.. note::
+  GPU virtualisation is still an evolving field. Depending on the mode
+  you are running you may need to override distribution supplied
+  libraries with more recent versions or enable build options.
+
+Host requirements
+-----------------
+
+Depending on the mode there are a number of requirements the host must
+meet to be able to be able to support guests. For 3D acceleration QEMU
+must be able to access the hosts GPU and for the best performance be
+able to reliably share GPU memory with the guest.
+
+.. list-table:: Host Requirements
+  :header-rows: 1
+
+  * - Mode
+    - Kernel
+    - Userspace
+  * - virtio-gpu
+    - framebuffer enabled
+    - GTK or SDL display
+  * - virtio-gpu-gl (OpenGL pass-through)
+    - GPU enabled
+    - libvirglrenderer (virgl support)
+  * - virtio-gpu-gl (rutabaga/gfxstream)
+    - GPU enabled
+    - aemu/rutabaga_gfx_ffi or vhost-user client with support
+  * - virtio-gpu-gl (Vulkan pass-through)
+    - Linux 6.13+
+    - libvirglrenderer (>= 1.0.0, venus support)
+  * - virtio-gpu-gl (vDRM native context/AMD)
+    - Linux 6.13+
+    - libvirglrenderer (>= 1.1.0, DRM renderer support)
+  * - virtio-gpu-gl (vDRM native context/Freedreno)
+    - Linux 6.13+
+    - libvirglrenderer (>= 1.0.0, DRM renderer support)
+  * - virtio-gpu-gl (vDRM native context/Intel i915)
+    - Linux 6.13+
+    - libvirglrenderer (`mr1384`_, DRM renderer support)
+
+.. _mr1384: https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1384
+
+Guest requirements
+------------------
 
 virtio-gpu requires a guest Linux kernel built with the
-``CONFIG_DRM_VIRTIO_GPU`` option.
+``CONFIG_DRM_VIRTIO_GPU`` option. Otherwise for 3D accelerations you
+will need support from Mesa configured for whichever encapsulation you
+need.
+
+.. list-table:: Guest Requirements
+  :header-rows: 1
+
+  * - Mode
+    - Mesa Version
+    - Mesa build flags
+  * - virtio-gpu
+    - n/a
+    - n/a
+  * - virtio-gpu-gl (OpenGL pass-through)
+    - 20.3.0+
+    - -Dgallium-drivers=virgl
+  * - virtio-gpu-gl (rutabaga/gfxstream)
+    - 24.3.0+
+    - -Dvulkan-drivers=gfxstream
+  * - virtio-gpu-gl (Vulkan pass-through)
+    - 24.2+
+    - -Dvulkan-drivers=virtio
+  * - virtio-gpu-gl (vDRM native context/AMD)
+    - 25.0.0+
+    - -Dgallium-drivers=radeonsi -Dvulkan-drivers=amd -Damdgpu-virtio=true
+  * - virtio-gpu-gl (vDRM native context/Freedreno)
+    - 23.1.0+
+    - -Dgallium-drivers=freedreno -Dvulkan-drivers=freedreno
+  * - virtio-gpu-gl (vDRM native context/Intel i915)
+    - `mr29870`_
+    - -Dgallium-drivers=iris -Dvulkan-drivers=intel -Dintel-virtio-experimental=true
+
+.. _mr29870: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29870
+
+Further information
+...................
 
 QEMU virtio-gpu variants
 ------------------------
-- 
2.47.1

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/01/20 21:23, Alex Bennée wrote:
> Akihiko Odaki <akihiko.odaki@daynix.com> writes:
> 
>> On 2025/01/20 7:00, Dmitry Osipenko wrote:
>>> From: Alex Bennée <alex.bennee@linaro.org>
>>> This attempts to tidy up the VirtIO GPU documentation to make the
>>> list
>>> of requirements clearer. There are still a lot of moving parts and the
>>> distros have some catching up to do before this is all handled
>>> automatically.
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> Cc: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> Cc: Sergio Lopez Pascual <slp@redhat.com>
>>> Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
>>> [dmitry.osipenko@collabora.com: Extended and corrected doc]
>>> ---
>>>    docs/system/devices/virtio-gpu.rst | 94 ++++++++++++++++++++++++++++--
>>>    1 file changed, 88 insertions(+), 6 deletions(-)
>>> diff --git a/docs/system/devices/virtio-gpu.rst
>>> b/docs/system/devices/virtio-gpu.rst
>>> index f20c60016376..79af291a9316 100644
>>> --- a/docs/system/devices/virtio-gpu.rst
>>> +++ b/docs/system/devices/virtio-gpu.rst
>>> @@ -4,14 +4,96 @@
>>>    virtio-gpu
>>>    ==========
>>>    -This document explains the setup and usage of the virtio-gpu
>>> device.
>>> -The virtio-gpu device paravirtualizes the GPU and display controller.
>>> -
>>> -Linux kernel support
>>> ---------------------
>>> +The virtio-gpu device provides a GPU and display controller
>>> +paravirtualized using VirtIO. It supports a number of different modes
>>> +from simple 2D displays to fully accelerated 3D graphics.
>>> +
>>> +Dependencies
>>> +............
>>> +
>>> +.. note::
>>> +  GPU virtualisation is still an evolving field. Depending on the mode
>>> +  you are running you may need to override distribution supplied
>>> +  libraries with more recent versions or enable build options.
>>> +
>>> +Host requirements
>>> +-----------------
>>> +
>>> +Depending on the mode there are a number of requirements the host must
>>> +meet to be able to be able to support guests. For 3D acceleration QEMU
>>> +must be able to access the hosts GPU and for the best performance be
>>> +able to reliably share GPU memory with the guest.
>>> +
>>> +.. list-table:: Host Requirements
>>> +  :header-rows: 1
>>> +
>>> +  * - Mode
>>> +    - Kernel
>>> +    - Userspace
>>> +  * - virtio-gpu
>>> +    - framebuffer enabled
>>> +    - GTK or SDL display
>>> +  * - virtio-gpu-gl (OpenGL pass-through)
>>> +    - GPU enabled
>>> +    - libvirglrenderer (virgl support)
>>> +  * - virtio-gpu-gl (rutabaga/gfxstream)
>>> +    - GPU enabled
>>> +    - aemu/rutabaga_gfx_ffi or vhost-user client with support
>>> +  * - virtio-gpu-gl (Vulkan pass-through)
>>> +    - Linux 6.13+
>>> +    - libvirglrenderer (>= 1.0.0, venus support)
>>> +  * - virtio-gpu-gl (vDRM native context/AMD)
>>> +    - Linux 6.13+
>>> +    - libvirglrenderer (>= 1.1.0, DRM renderer support)
>>> +  * - virtio-gpu-gl (vDRM native context/Freedreno)
>>> +    - Linux 6.13+
>>> +    - libvirglrenderer (>= 1.0.0, DRM renderer support)
>>> +  * - virtio-gpu-gl (vDRM native context/Intel i915)
>>> +    - Linux 6.13+
>>> +    - libvirglrenderer (`mr1384`_, DRM renderer support)
>>> +
>>> +.. _mr1384: https://gitlab.freedesktop.org/virgl/virglrenderer/-/merge_requests/1384
>>> +
>>> +Guest requirements
>>> +------------------
>>>      virtio-gpu requires a guest Linux kernel built with the
>>> -``CONFIG_DRM_VIRTIO_GPU`` option.
>>> +``CONFIG_DRM_VIRTIO_GPU`` option. Otherwise for 3D accelerations you
>>> +will need support from Mesa configured for whichever encapsulation you
>>> +need.
>>> +
>>> +.. list-table:: Guest Requirements
>>> +  :header-rows: 1
>>> +
>>> +  * - Mode
>>> +    - Mesa Version
>>> +    - Mesa build flags
>>> +  * - virtio-gpu
>>> +    - n/a
>>> +    - n/a
>>> +  * - virtio-gpu-gl (OpenGL pass-through)
>>> +    - 20.3.0+
>>> +    - -Dgallium-drivers=virgl
>>> +  * - virtio-gpu-gl (rutabaga/gfxstream)
>>> +    - 24.3.0+
>>> +    - -Dvulkan-drivers=gfxstream
>>> +  * - virtio-gpu-gl (Vulkan pass-through)
>>> +    - 24.2+
>>> +    - -Dvulkan-drivers=virtio
>>> +  * - virtio-gpu-gl (vDRM native context/AMD)
>>> +    - 25.0.0+
>>> +    - -Dgallium-drivers=radeonsi -Dvulkan-drivers=amd -Damdgpu-virtio=true
>>> +  * - virtio-gpu-gl (vDRM native context/Freedreno)
>>> +    - 23.1.0+
>>> +    - -Dgallium-drivers=freedreno -Dvulkan-drivers=freedreno
>>> +  * - virtio-gpu-gl (vDRM native context/Intel i915)
>>> +    - `mr29870`_
>>> +    - -Dgallium-drivers=iris -Dvulkan-drivers=intel -Dintel-virtio-experimental=true
>>> +
>>> +.. _mr29870: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29870
>>
>> I feel the dependency information for virglrenderer and Mesa are more
>> suited for the Mesa documentation as they are not specific to QEMU and
>> potentially useful also for e.g., libkrun and crosvm.
> 
> I think while everything is in so much flux it doesn't hurt to include
> in our docs. I don't know if mesa currently has a dedicated page for GPU
> virtualisation.

Mesa has pages for VirGL and Venus, which can be linked from the 
respective parts of this documentation. gfxstream is not documented but 
I think most people will use it only for Android anyway. A documentation 
for DRM native context will be a nice addition for Mesa. I will not 
object if you put this information to QEMU documentation though.

> 
> 
>>
>>> +
>>> +Further information
>>> +...................
>>
>> I'm not sure about putting the below sections behind "Further
>> information". Anyone who want to use this feature will need to know
>> the command line described below.
>>
>>>      QEMU virtio-gpu variants
>>>    ------------------------
> 

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 1/21/25 07:26, Akihiko Odaki wrote:
...
>>> I feel the dependency information for virglrenderer and Mesa are more
>>> suited for the Mesa documentation as they are not specific to QEMU and
>>> potentially useful also for e.g., libkrun and crosvm.
>>
>> I think while everything is in so much flux it doesn't hurt to include
>> in our docs. I don't know if mesa currently has a dedicated page for GPU
>> virtualisation.
> 
> Mesa has pages for VirGL and Venus, which can be linked from the
> respective parts of this documentation. gfxstream is not documented but
> I think most people will use it only for Android anyway. A documentation
> for DRM native context will be a nice addition for Mesa. I will not
> object if you put this information to QEMU documentation though.

Adding native context doc to Mesa indeed will be a good move, as well as
adding links to the Mesa virgl/venus pages in QEMU.

RE requirements documentation, it's also a valid point that stuff like
build flags should belong to the relevant projects. On the other hand,
it's a common headache for a newcoming people to figure everything out
from scratch and having more centralized documentation helps. The build
requirements aren't cleanly documented anywhere AFAICT, and the
requirements also differ based on VMM. I'll update and keep this patch
in v6, the requirements info should stay actual for a next couple years
IMO. Let's discuss it further in v6 if more objections will arise.

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/01/27 3:06, Dmitry Osipenko wrote:
> On 1/21/25 07:26, Akihiko Odaki wrote:
> ...
>>>> I feel the dependency information for virglrenderer and Mesa are more
>>>> suited for the Mesa documentation as they are not specific to QEMU and
>>>> potentially useful also for e.g., libkrun and crosvm.
>>>
>>> I think while everything is in so much flux it doesn't hurt to include
>>> in our docs. I don't know if mesa currently has a dedicated page for GPU
>>> virtualisation.
>>
>> Mesa has pages for VirGL and Venus, which can be linked from the
>> respective parts of this documentation. gfxstream is not documented but
>> I think most people will use it only for Android anyway. A documentation
>> for DRM native context will be a nice addition for Mesa. I will not
>> object if you put this information to QEMU documentation though.
> 
> Adding native context doc to Mesa indeed will be a good move, as well as
> adding links to the Mesa virgl/venus pages in QEMU.
> 
> RE requirements documentation, it's also a valid point that stuff like
> build flags should belong to the relevant projects. On the other hand,
> it's a common headache for a newcoming people to figure everything out
> from scratch and having more centralized documentation helps. The build
> requirements aren't cleanly documented anywhere AFAICT, and the
> requirements also differ based on VMM. I'll update and keep this patch
> in v6, the requirements info should stay actual for a next couple years
> IMO. Let's discuss it further in v6 if more objections will arise.
> 

I think it's fine to require one click to reach the relevant documentation.

How do the requirements described here vary with VMM?

I'm not entirely sure the documentation will stay as is for that long. 
The requirements of Intel native context refer to merge requests that 
can be merged sooner or later. Asahi may need more updates if you 
document it too because its DRM ABI is still unstable.

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 1/27/25 07:57, Akihiko Odaki wrote:
> On 2025/01/27 3:06, Dmitry Osipenko wrote:
>> On 1/21/25 07:26, Akihiko Odaki wrote:
>> ...
>>>>> I feel the dependency information for virglrenderer and Mesa are more
>>>>> suited for the Mesa documentation as they are not specific to QEMU and
>>>>> potentially useful also for e.g., libkrun and crosvm.
>>>>
>>>> I think while everything is in so much flux it doesn't hurt to include
>>>> in our docs. I don't know if mesa currently has a dedicated page for
>>>> GPU
>>>> virtualisation.
>>>
>>> Mesa has pages for VirGL and Venus, which can be linked from the
>>> respective parts of this documentation. gfxstream is not documented but
>>> I think most people will use it only for Android anyway. A documentation
>>> for DRM native context will be a nice addition for Mesa. I will not
>>> object if you put this information to QEMU documentation though.
>>
>> Adding native context doc to Mesa indeed will be a good move, as well as
>> adding links to the Mesa virgl/venus pages in QEMU.
>>
>> RE requirements documentation, it's also a valid point that stuff like
>> build flags should belong to the relevant projects. On the other hand,
>> it's a common headache for a newcoming people to figure everything out
>> from scratch and having more centralized documentation helps. The build
>> requirements aren't cleanly documented anywhere AFAICT, and the
>> requirements also differ based on VMM. I'll update and keep this patch
>> in v6, the requirements info should stay actual for a next couple years
>> IMO. Let's discuss it further in v6 if more objections will arise.
>>
> 
> I think it's fine to require one click to reach the relevant documentation.
> 
> How do the requirements described here vary with VMM?

Requirements don't vary much. For example virglrenderer minigbm support
is mandatory for crosvm, while for QEMU it's not.

> I'm not entirely sure the documentation will stay as is for that long.
> The requirements of Intel native context refer to merge requests that
> can be merged sooner or later. Asahi may need more updates if you
> document it too because its DRM ABI is still unstable.

The unstable parts of course will need to be updated sooner, but the
stable should be solid for years. I expect that about a year later
requirements will need to be revisited.

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/02/03 7:08, Dmitry Osipenko wrote:
> On 1/27/25 07:57, Akihiko Odaki wrote:
>> On 2025/01/27 3:06, Dmitry Osipenko wrote:
>>> On 1/21/25 07:26, Akihiko Odaki wrote:
>>> ...
>>>>>> I feel the dependency information for virglrenderer and Mesa are more
>>>>>> suited for the Mesa documentation as they are not specific to QEMU and
>>>>>> potentially useful also for e.g., libkrun and crosvm.
>>>>>
>>>>> I think while everything is in so much flux it doesn't hurt to include
>>>>> in our docs. I don't know if mesa currently has a dedicated page for
>>>>> GPU
>>>>> virtualisation.
>>>>
>>>> Mesa has pages for VirGL and Venus, which can be linked from the
>>>> respective parts of this documentation. gfxstream is not documented but
>>>> I think most people will use it only for Android anyway. A documentation
>>>> for DRM native context will be a nice addition for Mesa. I will not
>>>> object if you put this information to QEMU documentation though.
>>>
>>> Adding native context doc to Mesa indeed will be a good move, as well as
>>> adding links to the Mesa virgl/venus pages in QEMU.
>>>
>>> RE requirements documentation, it's also a valid point that stuff like
>>> build flags should belong to the relevant projects. On the other hand,
>>> it's a common headache for a newcoming people to figure everything out
>>> from scratch and having more centralized documentation helps. The build
>>> requirements aren't cleanly documented anywhere AFAICT, and the
>>> requirements also differ based on VMM. I'll update and keep this patch
>>> in v6, the requirements info should stay actual for a next couple years
>>> IMO. Let's discuss it further in v6 if more objections will arise.
>>>
>>
>> I think it's fine to require one click to reach the relevant documentation.
>>
>> How do the requirements described here vary with VMM?
> 
> Requirements don't vary much. For example virglrenderer minigbm support
> is mandatory for crosvm, while for QEMU it's not.

Is that true? It seems that virglrenderer uses builds without minigbm 
support to run tests on GitLab CI.

Anyway, if there is any variance in the build procedure, that may 
justify having a separate build instruction in QEMU tree to avoid 
confusion. Otherwise, it's better to have a documentation shared with 
other VMMs.

> 
>> I'm not entirely sure the documentation will stay as is for that long.
>> The requirements of Intel native context refer to merge requests that
>> can be merged sooner or later. Asahi may need more updates if you
>> document it too because its DRM ABI is still unstable.
> 
> The unstable parts of course will need to be updated sooner, but the
> stable should be solid for years. I expect that about a year later
> requirements will need to be revisited.
> 

It will be some burden in the future. Now you are adding this 
documentation just for QEMU, but crosvm and libkrun may gain similar 
documentation. The DRM native context support for Intel and Asahi is in 
development, and I guess nvk will support it someday.

So, a very rough estimation of future documentation updates will be:
(number of VMMs) * (number of DRM native contexts in development)
= 3 * 3
= 9

That's manageable but suboptimal.

Regards,
Akihiko Odaki

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 2/3/25 08:31, Akihiko Odaki wrote:
...
>> Requirements don't vary much. For example virglrenderer minigbm support
>> is mandatory for crosvm, while for QEMU it's not.
> 
> Is that true? It seems that virglrenderer uses builds without minigbm
> support to run tests on GitLab CI.

CI is running in a headless mode using software renderer. For a
full-featured crosvm support running on a baremetal, minigbm should be
needed, along with other downstream features.

> Anyway, if there is any variance in the build procedure, that may
> justify having a separate build instruction in QEMU tree to avoid
> confusion. Otherwise, it's better to have a documentation shared with
> other VMMs.
> 
>>
>>> I'm not entirely sure the documentation will stay as is for that long.
>>> The requirements of Intel native context refer to merge requests that
>>> can be merged sooner or later. Asahi may need more updates if you
>>> document it too because its DRM ABI is still unstable.
>>
>> The unstable parts of course will need to be updated sooner, but the
>> stable should be solid for years. I expect that about a year later
>> requirements will need to be revisited.
>>
> 
> It will be some burden in the future. Now you are adding this
> documentation just for QEMU, but crosvm and libkrun may gain similar
> documentation. The DRM native context support for Intel and Asahi is in
> development, and I guess nvk will support it someday.
> 
> So, a very rough estimation of future documentation updates will be:
> (number of VMMs) * (number of DRM native contexts in development)
> = 3 * 3
> = 9
> 
> That's manageable but suboptimal.

I don't mind deferring the doc addition if that's preferred. Either way
is fine with me. Yet it's better to have doc than not.

In my view crosvm and libkrun exist separately from QEMU, they serve a
different purpose. Majority of QEMU users likely never heard about those
other VMMs. A unified doc won't be a worthwhile effort, IMO.

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/02/06 2:40, Dmitry Osipenko wrote:
> On 2/3/25 08:31, Akihiko Odaki wrote:
> ...
>>> Requirements don't vary much. For example virglrenderer minigbm support
>>> is mandatory for crosvm, while for QEMU it's not.
>>
>> Is that true? It seems that virglrenderer uses builds without minigbm
>> support to run tests on GitLab CI.
> 
> CI is running in a headless mode using software renderer. For a
> full-featured crosvm support running on a baremetal, minigbm should be
> needed, along with other downstream features.

That makes sense.

Based on your input, for QEMU, I don't think we need a separate 
documentation to describe libvirglrenderer's build flags though crosvm 
should have some documentation saying it requires minigbm.

> 
>> Anyway, if there is any variance in the build procedure, that may
>> justify having a separate build instruction in QEMU tree to avoid
>> confusion. Otherwise, it's better to have a documentation shared with
>> other VMMs.
>>
>>>
>>>> I'm not entirely sure the documentation will stay as is for that long.
>>>> The requirements of Intel native context refer to merge requests that
>>>> can be merged sooner or later. Asahi may need more updates if you
>>>> document it too because its DRM ABI is still unstable.
>>>
>>> The unstable parts of course will need to be updated sooner, but the
>>> stable should be solid for years. I expect that about a year later
>>> requirements will need to be revisited.
>>>
>>
>> It will be some burden in the future. Now you are adding this
>> documentation just for QEMU, but crosvm and libkrun may gain similar
>> documentation. The DRM native context support for Intel and Asahi is in
>> development, and I guess nvk will support it someday.
>>
>> So, a very rough estimation of future documentation updates will be:
>> (number of VMMs) * (number of DRM native contexts in development)
>> = 3 * 3
>> = 9
>>
>> That's manageable but suboptimal.
> 
> I don't mind deferring the doc addition if that's preferred. Either way
> is fine with me. Yet it's better to have doc than not.

My suggestion is not to defer the addition, but to add it to Mesa, which 
does not require deferring.

> 
> In my view crosvm and libkrun exist separately from QEMU, they serve a
> different purpose. Majority of QEMU users likely never heard about those
> other VMMs. A unified doc won't be a worthwhile effort, IMO.
> 

When evaluating the utility of a unified documentation, Whether the 
majority of Mesa/Virgl users care VMMs other than QEMU matters more. And 
I think it is true; libkrun and crosvm are excellent options for 
graphics-accelerated VMs.

If we have a unified documentation, any VM can point to it for the build 
instruction of Mesa and virglrenderer. Once that's done, QEMU users who 
want graphics acceleration can take the following steps:
1. See docs/system/devices/virtio-gpu.rst
2. Figure out that they need Mesa and virglrenderer
3. Click the link to the unified documentation
4. Build Mesa and virglrenderer accordingly

No other VMMs will bother them in this procedure.

Regards,
Akihiko Odaki

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 2/6/25 08:41, Akihiko Odaki wrote:
> On 2025/02/06 2:40, Dmitry Osipenko wrote:
>> On 2/3/25 08:31, Akihiko Odaki wrote:
>> ...
>>>> Requirements don't vary much. For example virglrenderer minigbm support
>>>> is mandatory for crosvm, while for QEMU it's not.
>>>
>>> Is that true? It seems that virglrenderer uses builds without minigbm
>>> support to run tests on GitLab CI.
>>
>> CI is running in a headless mode using software renderer. For a
>> full-featured crosvm support running on a baremetal, minigbm should be
>> needed, along with other downstream features.
> 
> That makes sense.
> 
> Based on your input, for QEMU, I don't think we need a separate
> documentation to describe libvirglrenderer's build flags though crosvm
> should have some documentation saying it requires minigbm.
> 
>>
>>> Anyway, if there is any variance in the build procedure, that may
>>> justify having a separate build instruction in QEMU tree to avoid
>>> confusion. Otherwise, it's better to have a documentation shared with
>>> other VMMs.
>>>
>>>>
>>>>> I'm not entirely sure the documentation will stay as is for that long.
>>>>> The requirements of Intel native context refer to merge requests that
>>>>> can be merged sooner or later. Asahi may need more updates if you
>>>>> document it too because its DRM ABI is still unstable.
>>>>
>>>> The unstable parts of course will need to be updated sooner, but the
>>>> stable should be solid for years. I expect that about a year later
>>>> requirements will need to be revisited.
>>>>
>>>
>>> It will be some burden in the future. Now you are adding this
>>> documentation just for QEMU, but crosvm and libkrun may gain similar
>>> documentation. The DRM native context support for Intel and Asahi is in
>>> development, and I guess nvk will support it someday.
>>>
>>> So, a very rough estimation of future documentation updates will be:
>>> (number of VMMs) * (number of DRM native contexts in development)
>>> = 3 * 3
>>> = 9
>>>
>>> That's manageable but suboptimal.
>>
>> I don't mind deferring the doc addition if that's preferred. Either way
>> is fine with me. Yet it's better to have doc than not.
> 
> My suggestion is not to defer the addition, but to add it to Mesa, which
> does not require deferring.
> 
>>
>> In my view crosvm and libkrun exist separately from QEMU, they serve a
>> different purpose. Majority of QEMU users likely never heard about those
>> other VMMs. A unified doc won't be a worthwhile effort, IMO.
>>
> 
> When evaluating the utility of a unified documentation, Whether the
> majority of Mesa/Virgl users care VMMs other than QEMU matters more. And
> I think it is true; libkrun and crosvm are excellent options for
> graphics-accelerated VMs.
> 
> If we have a unified documentation, any VM can point to it for the build
> instruction of Mesa and virglrenderer. Once that's done, QEMU users who
> want graphics acceleration can take the following steps:
> 1. See docs/system/devices/virtio-gpu.rst
> 2. Figure out that they need Mesa and virglrenderer
> 3. Click the link to the unified documentation
> 4. Build Mesa and virglrenderer accordingly
> 
> No other VMMs will bother them in this procedure.

Will see. For the starter, adding example build flags to QEMU doesn't
hurt, it's a very minimal information. Later on, if and when all
relevant Mesa/virglrenderer doc pages will appear, it won't be a problem
replace QEMU flags with the links. Please let's do it step-by-step, one
step at a time :)

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/02/10 6:03, Dmitry Osipenko wrote:
> On 2/6/25 08:41, Akihiko Odaki wrote:
>> On 2025/02/06 2:40, Dmitry Osipenko wrote:
>>> On 2/3/25 08:31, Akihiko Odaki wrote:
>>> ...
>>>>> Requirements don't vary much. For example virglrenderer minigbm support
>>>>> is mandatory for crosvm, while for QEMU it's not.
>>>>
>>>> Is that true? It seems that virglrenderer uses builds without minigbm
>>>> support to run tests on GitLab CI.
>>>
>>> CI is running in a headless mode using software renderer. For a
>>> full-featured crosvm support running on a baremetal, minigbm should be
>>> needed, along with other downstream features.
>>
>> That makes sense.
>>
>> Based on your input, for QEMU, I don't think we need a separate
>> documentation to describe libvirglrenderer's build flags though crosvm
>> should have some documentation saying it requires minigbm.
>>
>>>
>>>> Anyway, if there is any variance in the build procedure, that may
>>>> justify having a separate build instruction in QEMU tree to avoid
>>>> confusion. Otherwise, it's better to have a documentation shared with
>>>> other VMMs.
>>>>
>>>>>
>>>>>> I'm not entirely sure the documentation will stay as is for that long.
>>>>>> The requirements of Intel native context refer to merge requests that
>>>>>> can be merged sooner or later. Asahi may need more updates if you
>>>>>> document it too because its DRM ABI is still unstable.
>>>>>
>>>>> The unstable parts of course will need to be updated sooner, but the
>>>>> stable should be solid for years. I expect that about a year later
>>>>> requirements will need to be revisited.
>>>>>
>>>>
>>>> It will be some burden in the future. Now you are adding this
>>>> documentation just for QEMU, but crosvm and libkrun may gain similar
>>>> documentation. The DRM native context support for Intel and Asahi is in
>>>> development, and I guess nvk will support it someday.
>>>>
>>>> So, a very rough estimation of future documentation updates will be:
>>>> (number of VMMs) * (number of DRM native contexts in development)
>>>> = 3 * 3
>>>> = 9
>>>>
>>>> That's manageable but suboptimal.
>>>
>>> I don't mind deferring the doc addition if that's preferred. Either way
>>> is fine with me. Yet it's better to have doc than not.
>>
>> My suggestion is not to defer the addition, but to add it to Mesa, which
>> does not require deferring.
>>
>>>
>>> In my view crosvm and libkrun exist separately from QEMU, they serve a
>>> different purpose. Majority of QEMU users likely never heard about those
>>> other VMMs. A unified doc won't be a worthwhile effort, IMO.
>>>
>>
>> When evaluating the utility of a unified documentation, Whether the
>> majority of Mesa/Virgl users care VMMs other than QEMU matters more. And
>> I think it is true; libkrun and crosvm are excellent options for
>> graphics-accelerated VMs.
>>
>> If we have a unified documentation, any VM can point to it for the build
>> instruction of Mesa and virglrenderer. Once that's done, QEMU users who
>> want graphics acceleration can take the following steps:
>> 1. See docs/system/devices/virtio-gpu.rst
>> 2. Figure out that they need Mesa and virglrenderer
>> 3. Click the link to the unified documentation
>> 4. Build Mesa and virglrenderer accordingly
>>
>> No other VMMs will bother them in this procedure.
> 
> Will see. For the starter, adding example build flags to QEMU doesn't
> hurt, it's a very minimal information. Later on, if and when all
> relevant Mesa/virglrenderer doc pages will appear, it won't be a problem
> replace QEMU flags with the links. Please let's do it step-by-step, one
> step at a time :)
> 

To be honest, I'm concerned that you may be using QEMU as a staging tree 
for Mesa/virglrenderer. Submitting a documentation to QEMU as a 
preparation to submit one to Mesa is not OK.

You shouldn't submit a documentation to QEMU if upstream
Mesa developers rejects it because it contains too little information. 
It may not hurt QEMU, but still lacks a valid reasoning.

Mesa should have more people who care virtio-gpu as there are people 
using other VMMs and perhaps it may be difficult to convince them to add 
a documentation like this. It is still not a good idea to workaround 
that by adding one to QEMU. The documentation submitted to QEMU is 
mostly reviewed only by me, who barely used Venus and DRM native 
contexts, which is not a good sign. Getting reviewed by more people is 
one of the advantage of open-source contribution after all so let's keep it.

I can help you add a documentation to Mesa by reviewing and supporting 
one if you want, but I cannot support adding one to QEMU if it's done to 
avoid a potential challenge to add it to Mesa.

Regards,
Akihiko Odaki

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 2/13/25 07:32, Akihiko Odaki wrote:
> On 2025/02/10 6:03, Dmitry Osipenko wrote:
>> On 2/6/25 08:41, Akihiko Odaki wrote:
>>> On 2025/02/06 2:40, Dmitry Osipenko wrote:
>>>> On 2/3/25 08:31, Akihiko Odaki wrote:
>>>> ...
>>>>>> Requirements don't vary much. For example virglrenderer minigbm
>>>>>> support
>>>>>> is mandatory for crosvm, while for QEMU it's not.
>>>>>
>>>>> Is that true? It seems that virglrenderer uses builds without minigbm
>>>>> support to run tests on GitLab CI.
>>>>
>>>> CI is running in a headless mode using software renderer. For a
>>>> full-featured crosvm support running on a baremetal, minigbm should be
>>>> needed, along with other downstream features.
>>>
>>> That makes sense.
>>>
>>> Based on your input, for QEMU, I don't think we need a separate
>>> documentation to describe libvirglrenderer's build flags though crosvm
>>> should have some documentation saying it requires minigbm.
>>>
>>>>
>>>>> Anyway, if there is any variance in the build procedure, that may
>>>>> justify having a separate build instruction in QEMU tree to avoid
>>>>> confusion. Otherwise, it's better to have a documentation shared with
>>>>> other VMMs.
>>>>>
>>>>>>
>>>>>>> I'm not entirely sure the documentation will stay as is for that
>>>>>>> long.
>>>>>>> The requirements of Intel native context refer to merge requests
>>>>>>> that
>>>>>>> can be merged sooner or later. Asahi may need more updates if you
>>>>>>> document it too because its DRM ABI is still unstable.
>>>>>>
>>>>>> The unstable parts of course will need to be updated sooner, but the
>>>>>> stable should be solid for years. I expect that about a year later
>>>>>> requirements will need to be revisited.
>>>>>>
>>>>>
>>>>> It will be some burden in the future. Now you are adding this
>>>>> documentation just for QEMU, but crosvm and libkrun may gain similar
>>>>> documentation. The DRM native context support for Intel and Asahi
>>>>> is in
>>>>> development, and I guess nvk will support it someday.
>>>>>
>>>>> So, a very rough estimation of future documentation updates will be:
>>>>> (number of VMMs) * (number of DRM native contexts in development)
>>>>> = 3 * 3
>>>>> = 9
>>>>>
>>>>> That's manageable but suboptimal.
>>>>
>>>> I don't mind deferring the doc addition if that's preferred. Either way
>>>> is fine with me. Yet it's better to have doc than not.
>>>
>>> My suggestion is not to defer the addition, but to add it to Mesa, which
>>> does not require deferring.
>>>
>>>>
>>>> In my view crosvm and libkrun exist separately from QEMU, they serve a
>>>> different purpose. Majority of QEMU users likely never heard about
>>>> those
>>>> other VMMs. A unified doc won't be a worthwhile effort, IMO.
>>>>
>>>
>>> When evaluating the utility of a unified documentation, Whether the
>>> majority of Mesa/Virgl users care VMMs other than QEMU matters more. And
>>> I think it is true; libkrun and crosvm are excellent options for
>>> graphics-accelerated VMs.
>>>
>>> If we have a unified documentation, any VM can point to it for the build
>>> instruction of Mesa and virglrenderer. Once that's done, QEMU users who
>>> want graphics acceleration can take the following steps:
>>> 1. See docs/system/devices/virtio-gpu.rst
>>> 2. Figure out that they need Mesa and virglrenderer
>>> 3. Click the link to the unified documentation
>>> 4. Build Mesa and virglrenderer accordingly
>>>
>>> No other VMMs will bother them in this procedure.
>>
>> Will see. For the starter, adding example build flags to QEMU doesn't
>> hurt, it's a very minimal information. Later on, if and when all
>> relevant Mesa/virglrenderer doc pages will appear, it won't be a problem
>> replace QEMU flags with the links. Please let's do it step-by-step, one
>> step at a time :)
>>
> 
> To be honest, I'm concerned that you may be using QEMU as a staging tree
> for Mesa/virglrenderer. Submitting a documentation to QEMU as a
> preparation to submit one to Mesa is not OK.
> 
> You shouldn't submit a documentation to QEMU if upstream
> Mesa developers rejects it because it contains too little information.
> It may not hurt QEMU, but still lacks a valid reasoning.

Don't understand what you're talking about here. I may remind that this
is not my QEMU doc patch to begin with, hence it has nothing to do with
Mesa nor with virglrenderer. Alex wants to help QEMU users by adding
more QEMU documentation.

Maybe you're also not very familiar with the Mesa development process.
This is okay, no problems.

> Mesa should have more people who care virtio-gpu as there are people
> using other VMMs and perhaps it may be difficult to convince them to add
> a documentation like this. It is still not a good idea to workaround
> that by adding one to QEMU. The documentation submitted to QEMU is
> mostly reviewed only by me, who barely used Venus and DRM native
> contexts, which is not a good sign. Getting reviewed by more people is
> one of the advantage of open-source contribution after all so let's keep
> it.
> 
> I can help you add a documentation to Mesa by reviewing and supporting
> one if you want, but I cannot support adding one to QEMU if it's done to
> avoid a potential challenge to add it to Mesa.

Thanks a lot! More reviewers always good to have, will definitely ping
you. Right now writing more Mesa documentation isn't a priority, hence
will take time to get to it.

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Dmitry Osipenko]

On 2/18/25 09:27, Dmitry Osipenko wrote:
>> To be honest, I'm concerned that you may be using QEMU as a staging tree
>> for Mesa/virglrenderer. Submitting a documentation to QEMU as a
>> preparation to submit one to Mesa is not OK.
>>
>> You shouldn't submit a documentation to QEMU if upstream
>> Mesa developers rejects it because it contains too little information.
>> It may not hurt QEMU, but still lacks a valid reasoning.
> Don't understand what you're talking about here. I may remind that this
> is not my QEMU doc patch to begin with, hence it has nothing to do with
> Mesa nor with virglrenderer. Alex wants to help QEMU users by adding
> more QEMU documentation.
> 
> Maybe you're also not very familiar with the Mesa development process.
> This is okay, no problems.
> 
>> Mesa should have more people who care virtio-gpu as there are people
>> using other VMMs and perhaps it may be difficult to convince them to add
>> a documentation like this. It is still not a good idea to workaround
>> that by adding one to QEMU. The documentation submitted to QEMU is
>> mostly reviewed only by me, who barely used Venus and DRM native
>> contexts, which is not a good sign. Getting reviewed by more people is
>> one of the advantage of open-source contribution after all so let's keep
>> it.
>>
>> I can help you add a documentation to Mesa by reviewing and supporting
>> one if you want, but I cannot support adding one to QEMU if it's done to
>> avoid a potential challenge to add it to Mesa.
> Thanks a lot! More reviewers always good to have, will definitely ping
> you. Right now writing more Mesa documentation isn't a priority, hence
> will take time to get to it.

To finish this thread, I'll repeat what was said on the v8 that I'll
drop the build flags from the QEMU doc since they are bothering you too
much and your voice has a high value. Thanks again for helping with the
reviewing, very appreciate it! :)

-- 
Best regards,
Dmitry

Re: [PATCH v5 8/8] docs/system: Expand the virtio-gpu documentation

[Akihiko Odaki]

On 2025/02/18 15:27, Dmitry Osipenko wrote:
> On 2/13/25 07:32, Akihiko Odaki wrote:
>> On 2025/02/10 6:03, Dmitry Osipenko wrote:
>>> On 2/6/25 08:41, Akihiko Odaki wrote:
>>>> On 2025/02/06 2:40, Dmitry Osipenko wrote:
>>>>> On 2/3/25 08:31, Akihiko Odaki wrote:
>>>>> ...
>>>>>>> Requirements don't vary much. For example virglrenderer minigbm
>>>>>>> support
>>>>>>> is mandatory for crosvm, while for QEMU it's not.
>>>>>>
>>>>>> Is that true? It seems that virglrenderer uses builds without minigbm
>>>>>> support to run tests on GitLab CI.
>>>>>
>>>>> CI is running in a headless mode using software renderer. For a
>>>>> full-featured crosvm support running on a baremetal, minigbm should be
>>>>> needed, along with other downstream features.
>>>>
>>>> That makes sense.
>>>>
>>>> Based on your input, for QEMU, I don't think we need a separate
>>>> documentation to describe libvirglrenderer's build flags though crosvm
>>>> should have some documentation saying it requires minigbm.
>>>>
>>>>>
>>>>>> Anyway, if there is any variance in the build procedure, that may
>>>>>> justify having a separate build instruction in QEMU tree to avoid
>>>>>> confusion. Otherwise, it's better to have a documentation shared with
>>>>>> other VMMs.
>>>>>>
>>>>>>>
>>>>>>>> I'm not entirely sure the documentation will stay as is for that
>>>>>>>> long.
>>>>>>>> The requirements of Intel native context refer to merge requests
>>>>>>>> that
>>>>>>>> can be merged sooner or later. Asahi may need more updates if you
>>>>>>>> document it too because its DRM ABI is still unstable.
>>>>>>>
>>>>>>> The unstable parts of course will need to be updated sooner, but the
>>>>>>> stable should be solid for years. I expect that about a year later
>>>>>>> requirements will need to be revisited.
>>>>>>>
>>>>>>
>>>>>> It will be some burden in the future. Now you are adding this
>>>>>> documentation just for QEMU, but crosvm and libkrun may gain similar
>>>>>> documentation. The DRM native context support for Intel and Asahi
>>>>>> is in
>>>>>> development, and I guess nvk will support it someday.
>>>>>>
>>>>>> So, a very rough estimation of future documentation updates will be:
>>>>>> (number of VMMs) * (number of DRM native contexts in development)
>>>>>> = 3 * 3
>>>>>> = 9
>>>>>>
>>>>>> That's manageable but suboptimal.
>>>>>
>>>>> I don't mind deferring the doc addition if that's preferred. Either way
>>>>> is fine with me. Yet it's better to have doc than not.
>>>>
>>>> My suggestion is not to defer the addition, but to add it to Mesa, which
>>>> does not require deferring.
>>>>
>>>>>
>>>>> In my view crosvm and libkrun exist separately from QEMU, they serve a
>>>>> different purpose. Majority of QEMU users likely never heard about
>>>>> those
>>>>> other VMMs. A unified doc won't be a worthwhile effort, IMO.
>>>>>
>>>>
>>>> When evaluating the utility of a unified documentation, Whether the
>>>> majority of Mesa/Virgl users care VMMs other than QEMU matters more. And
>>>> I think it is true; libkrun and crosvm are excellent options for
>>>> graphics-accelerated VMs.
>>>>
>>>> If we have a unified documentation, any VM can point to it for the build
>>>> instruction of Mesa and virglrenderer. Once that's done, QEMU users who
>>>> want graphics acceleration can take the following steps:
>>>> 1. See docs/system/devices/virtio-gpu.rst
>>>> 2. Figure out that they need Mesa and virglrenderer
>>>> 3. Click the link to the unified documentation
>>>> 4. Build Mesa and virglrenderer accordingly
>>>>
>>>> No other VMMs will bother them in this procedure.
>>>
>>> Will see. For the starter, adding example build flags to QEMU doesn't
>>> hurt, it's a very minimal information. Later on, if and when all
>>> relevant Mesa/virglrenderer doc pages will appear, it won't be a problem
>>> replace QEMU flags with the links. Please let's do it step-by-step, one
>>> step at a time :)
>>>
>>
>> To be honest, I'm concerned that you may be using QEMU as a staging tree
>> for Mesa/virglrenderer. Submitting a documentation to QEMU as a
>> preparation to submit one to Mesa is not OK.
>>
>> You shouldn't submit a documentation to QEMU if upstream
>> Mesa developers rejects it because it contains too little information.
>> It may not hurt QEMU, but still lacks a valid reasoning.
> 
> Don't understand what you're talking about here. I may remind that this
> is not my QEMU doc patch to begin with, hence it has nothing to do with
> Mesa nor with virglrenderer. Alex wants to help QEMU users by adding
> more QEMU documentation.

I understand Alex and you intend to help QEMU users, but sometimes a 
change to help QEMU users is better to be done in the upstream of its 
dependency.

For example, when I submitted a patch to suppress a sanitizer warning in 
the past, I was asked if it may be better to propose a change of the 
sanitizer behavior instead of changing QEMU code. We eventually 
concluded we should change our code at that time, but it could have 
turned out otherwise. It is no different from the normal review process; 
a submitter proposes a solution, a reviewer suggests alternatives, and 
both figure out the optimal one in the discussion.

> 
> Maybe you're also not very familiar with the Mesa development process.
> This is okay, no problems.
> 
>> Mesa should have more people who care virtio-gpu as there are people
>> using other VMMs and perhaps it may be difficult to convince them to add
>> a documentation like this. It is still not a good idea to workaround
>> that by adding one to QEMU. The documentation submitted to QEMU is
>> mostly reviewed only by me, who barely used Venus and DRM native
>> contexts, which is not a good sign. Getting reviewed by more people is
>> one of the advantage of open-source contribution after all so let's keep
>> it.
>>
>> I can help you add a documentation to Mesa by reviewing and supporting
>> one if you want, but I cannot support adding one to QEMU if it's done to
>> avoid a potential challenge to add it to Mesa.
> 
> Thanks a lot! More reviewers always good to have, will definitely ping
> you. Right now writing more Mesa documentation isn't a priority, hence
> will take time to get to it.
> 

I appreciate your contribution to the documentation too. Once there was 
no documentation, but Gurchetan Singh added one along with their 
Rutabaga patches. With your change, the documentation will be 
comprehensive also for virglrenderer.

Regards,
Akihiko Odaki

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Dmitry Osipenko]

On 1/20/25 18:41, Alex Bennée wrote:
> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
> 
>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>
>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>> DRM native context [1] mediates lower level kernel driver UAPI, which
>> reflects in a less CPU overhead and less/simpler code needed to support it.
>> DRM context consists of a host and guest parts that have to be implemented
>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>> a real/native host GPU device for GL/VK applications.
>>
>> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>>
>> Today there are four known DRM native context drivers existing in a wild:
>>
>>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>>   - AMDGPU, mostly merged into upstreams
>>   - Intel (i915), merge requests are opened
> 
> With the patch for the build failure:
> 
> Tested-by: Alex Bennée <alex.bennee@linaro.org>
> 
> Host:
> 
>   x86
>   Trixie
>   virglrenderer @ digitx/native-context-iris/a0b1872d252430a2b7f007db9fdbb0526385cfc0 
>   -display sdl,gl=on
> 
> KVM Guest
> 
>   x86
>   Trixie
>   mesa @ digitx/native-context-iris/78b1508c3f06
> 
> With gtk,gl=on I'm still seeing a lot of screen corruption which mirrors
> the terminal an leaves a destructive trail under the mouse cursor.
> show-cursor on or off makes no difference.

Thank you for the review and testing! I'm looking into that issue. Only
some people are hitting it and Pierre-Eric said he had that mirroring
issue without using nctx. Still interesting that the bug affects only
certain setups and is triggered by nctx.

-- 
Best regards,
Dmitry

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Alex Bennée]

Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>
> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
> DRM native context [1] mediates lower level kernel driver UAPI, which
> reflects in a less CPU overhead and less/simpler code needed to support it.
> DRM context consists of a host and guest parts that have to be implemented
> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
> a real/native host GPU device for GL/VK applications.
>
> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>
> Today there are four known DRM native context drivers existing in a wild:
>
>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>   - AMDGPU, mostly merged into upstreams

I tried my AMD system today with:

Host:
  Aarch64 AVA system
  Trixie
  virglrenderer @ v1.1.0/99557f5aa130930d11f04ffeb07f3a9aa5963182
  -display sdl,gl=on (gtk,gl=on also came up but handled window resizing
  poorly)
  
KVM Guest

  Aarch64
  Trixie
  mesa @ main/d27748a76f7dd9236bfcf9ef172dc13b8c0e170f
  -Dvulkan-drivers=virtio,amd -Dgallium-drivers=virgl,radeonsi -Damdgpu-virtio=true

However when I ran vulkan-info --summary KVM faulted with:

  debian-trixie login: error: kvm run failed Bad address
   PC=0000ffffb9aa1eb0 X00=0000ffffba0450a4 X01=0000aaaaf7f32400
  X02=000000000000013c X03=0000ffffba045098 X04=0000aaaaf7f3253c
  X05=0000ffffba0451d4 X06=00000000c0016900 X07=000000000000000e
  X08=0000000000000014 X09=00000000000000ff X10=0000aaaaf7f32500
  X11=0000aaaaf7e4d028 X12=0000aaaaf7edbcb0 X13=0000000000000001
  X14=000000000000000c X15=0000000000007718 X16=0000ffffb93601f0
  X17=0000ffffb9aa1dc0 X18=00000000000076f0 X19=0000aaaaf7f31330
  X20=0000aaaaf7f323f0 X21=0000aaaaf7f235e0 X22=000000000000004c
  X23=0000aaaaf7f2b5e0 X24=0000aaaaf7ee0cb0 X25=00000000000000ff
  X26=0000000000000076 X27=0000ffffcd2b18a8 X28=0000aaaaf7ee0cb0
  X29=0000ffffcd2b0bd0 X30=0000ffffb86c8b98  SP=0000ffffcd2b0bd0
  PSTATE=20001000 --C- EL0t
  QEMU 9.2.50 monitor - type 'help' for more information
  (qemu) quit

Which looks very much like the PFN locking failure. However booting up
with venus=on instead works. Could there be any differences in the way
device memory is mapped in the two cases?

>   - Intel (i915), merge requests are opened
>   - Asahi (Apple SoC GPUs), WIP status
>
<snip>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Dmitry Osipenko]

On 1/22/25 20:00, Alex Bennée wrote:
> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
> 
>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>
>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>> DRM native context [1] mediates lower level kernel driver UAPI, which
>> reflects in a less CPU overhead and less/simpler code needed to support it.
>> DRM context consists of a host and guest parts that have to be implemented
>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>> a real/native host GPU device for GL/VK applications.
>>
>> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>>
>> Today there are four known DRM native context drivers existing in a wild:
>>
>>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>>   - AMDGPU, mostly merged into upstreams
> 
> I tried my AMD system today with:
> 
> Host:
>   Aarch64 AVA system
>   Trixie
>   virglrenderer @ v1.1.0/99557f5aa130930d11f04ffeb07f3a9aa5963182
>   -display sdl,gl=on (gtk,gl=on also came up but handled window resizing
>   poorly)
>   
> KVM Guest
> 
>   Aarch64
>   Trixie
>   mesa @ main/d27748a76f7dd9236bfcf9ef172dc13b8c0e170f
>   -Dvulkan-drivers=virtio,amd -Dgallium-drivers=virgl,radeonsi -Damdgpu-virtio=true
> 
> However when I ran vulkan-info --summary KVM faulted with:
> 
>   debian-trixie login: error: kvm run failed Bad address
>    PC=0000ffffb9aa1eb0 X00=0000ffffba0450a4 X01=0000aaaaf7f32400
>   X02=000000000000013c X03=0000ffffba045098 X04=0000aaaaf7f3253c
>   X05=0000ffffba0451d4 X06=00000000c0016900 X07=000000000000000e
>   X08=0000000000000014 X09=00000000000000ff X10=0000aaaaf7f32500
>   X11=0000aaaaf7e4d028 X12=0000aaaaf7edbcb0 X13=0000000000000001
>   X14=000000000000000c X15=0000000000007718 X16=0000ffffb93601f0
>   X17=0000ffffb9aa1dc0 X18=00000000000076f0 X19=0000aaaaf7f31330
>   X20=0000aaaaf7f323f0 X21=0000aaaaf7f235e0 X22=000000000000004c
>   X23=0000aaaaf7f2b5e0 X24=0000aaaaf7ee0cb0 X25=00000000000000ff
>   X26=0000000000000076 X27=0000ffffcd2b18a8 X28=0000aaaaf7ee0cb0
>   X29=0000ffffcd2b0bd0 X30=0000ffffb86c8b98  SP=0000ffffcd2b0bd0
>   PSTATE=20001000 --C- EL0t
>   QEMU 9.2.50 monitor - type 'help' for more information
>   (qemu) quit
> 
> Which looks very much like the PFN locking failure. However booting up
> with venus=on instead works. Could there be any differences in the way
> device memory is mapped in the two cases?

Memory mapping works exactly the same for nctx and venus. Are you on
6.13 host kernel?

-- 
Best regards,
Dmitry

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Alex Bennée]

Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> On 1/22/25 20:00, Alex Bennée wrote:
>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>> 
>>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>>
>>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>>> DRM native context [1] mediates lower level kernel driver UAPI, which
>>> reflects in a less CPU overhead and less/simpler code needed to support it.
>>> DRM context consists of a host and guest parts that have to be implemented
>>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>>> a real/native host GPU device for GL/VK applications.
>>>
>>> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>>>
>>> Today there are four known DRM native context drivers existing in a wild:
>>>
>>>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>>>   - AMDGPU, mostly merged into upstreams
>> 
>> I tried my AMD system today with:
>> 
>> Host:
>>   Aarch64 AVA system
>>   Trixie
>>   virglrenderer @ v1.1.0/99557f5aa130930d11f04ffeb07f3a9aa5963182
>>   -display sdl,gl=on (gtk,gl=on also came up but handled window resizing
>>   poorly)
>>   
>> KVM Guest
>> 
>>   Aarch64
>>   Trixie
>>   mesa @ main/d27748a76f7dd9236bfcf9ef172dc13b8c0e170f
>>   -Dvulkan-drivers=virtio,amd -Dgallium-drivers=virgl,radeonsi -Damdgpu-virtio=true
>> 
>> However when I ran vulkan-info --summary KVM faulted with:
>> 
>>   debian-trixie login: error: kvm run failed Bad address
>>    PC=0000ffffb9aa1eb0 X00=0000ffffba0450a4 X01=0000aaaaf7f32400
>>   X02=000000000000013c X03=0000ffffba045098 X04=0000aaaaf7f3253c
>>   X05=0000ffffba0451d4 X06=00000000c0016900 X07=000000000000000e
>>   X08=0000000000000014 X09=00000000000000ff X10=0000aaaaf7f32500
>>   X11=0000aaaaf7e4d028 X12=0000aaaaf7edbcb0 X13=0000000000000001
>>   X14=000000000000000c X15=0000000000007718 X16=0000ffffb93601f0
>>   X17=0000ffffb9aa1dc0 X18=00000000000076f0 X19=0000aaaaf7f31330
>>   X20=0000aaaaf7f323f0 X21=0000aaaaf7f235e0 X22=000000000000004c
>>   X23=0000aaaaf7f2b5e0 X24=0000aaaaf7ee0cb0 X25=00000000000000ff
>>   X26=0000000000000076 X27=0000ffffcd2b18a8 X28=0000aaaaf7ee0cb0
>>   X29=0000ffffcd2b0bd0 X30=0000ffffb86c8b98  SP=0000ffffcd2b0bd0
>>   PSTATE=20001000 --C- EL0t
>>   QEMU 9.2.50 monitor - type 'help' for more information
>>   (qemu) quit
>> 
>> Which looks very much like the PFN locking failure. However booting up
>> with venus=on instead works. Could there be any differences in the way
>> device memory is mapped in the two cases?
>
> Memory mapping works exactly the same for nctx and venus. Are you on
> 6.13 host kernel?

Yes - with the Altra PCI workaround patches on both host and guest
kernel.

Is there anyway to trace the sharing of device memory on the host so I
can verify its an attempt at device access? The PC looks like its in
user-space but once this fails the guest is suspended so I can't poke
around in its environment.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Dmitry Osipenko]

On 1/23/25 14:58, Alex Bennée wrote:
> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
> 
>> On 1/22/25 20:00, Alex Bennée wrote:
>>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>>>
>>>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>>>
>>>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>>>> DRM native context [1] mediates lower level kernel driver UAPI, which
>>>> reflects in a less CPU overhead and less/simpler code needed to support it.
>>>> DRM context consists of a host and guest parts that have to be implemented
>>>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>>>> a real/native host GPU device for GL/VK applications.
>>>>
>>>> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>>>>
>>>> Today there are four known DRM native context drivers existing in a wild:
>>>>
>>>>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>>>>   - AMDGPU, mostly merged into upstreams
>>>
>>> I tried my AMD system today with:
>>>
>>> Host:
>>>   Aarch64 AVA system
>>>   Trixie
>>>   virglrenderer @ v1.1.0/99557f5aa130930d11f04ffeb07f3a9aa5963182
>>>   -display sdl,gl=on (gtk,gl=on also came up but handled window resizing
>>>   poorly)
>>>   
>>> KVM Guest
>>>
>>>   Aarch64
>>>   Trixie
>>>   mesa @ main/d27748a76f7dd9236bfcf9ef172dc13b8c0e170f
>>>   -Dvulkan-drivers=virtio,amd -Dgallium-drivers=virgl,radeonsi -Damdgpu-virtio=true
>>>
>>> However when I ran vulkan-info --summary KVM faulted with:
>>>
>>>   debian-trixie login: error: kvm run failed Bad address
>>>    PC=0000ffffb9aa1eb0 X00=0000ffffba0450a4 X01=0000aaaaf7f32400
>>>   X02=000000000000013c X03=0000ffffba045098 X04=0000aaaaf7f3253c
>>>   X05=0000ffffba0451d4 X06=00000000c0016900 X07=000000000000000e
>>>   X08=0000000000000014 X09=00000000000000ff X10=0000aaaaf7f32500
>>>   X11=0000aaaaf7e4d028 X12=0000aaaaf7edbcb0 X13=0000000000000001
>>>   X14=000000000000000c X15=0000000000007718 X16=0000ffffb93601f0
>>>   X17=0000ffffb9aa1dc0 X18=00000000000076f0 X19=0000aaaaf7f31330
>>>   X20=0000aaaaf7f323f0 X21=0000aaaaf7f235e0 X22=000000000000004c
>>>   X23=0000aaaaf7f2b5e0 X24=0000aaaaf7ee0cb0 X25=00000000000000ff
>>>   X26=0000000000000076 X27=0000ffffcd2b18a8 X28=0000aaaaf7ee0cb0
>>>   X29=0000ffffcd2b0bd0 X30=0000ffffb86c8b98  SP=0000ffffcd2b0bd0
>>>   PSTATE=20001000 --C- EL0t
>>>   QEMU 9.2.50 monitor - type 'help' for more information
>>>   (qemu) quit
>>>
>>> Which looks very much like the PFN locking failure. However booting up
>>> with venus=on instead works. Could there be any differences in the way
>>> device memory is mapped in the two cases?
>>
>> Memory mapping works exactly the same for nctx and venus. Are you on
>> 6.13 host kernel?
> 
> Yes - with the Altra PCI workaround patches on both host and guest
> kernel.
> 
> Is there anyway to trace the sharing of device memory on the host so I
> can verify its an attempt at device access? The PC looks like its in
> user-space but once this fails the guest is suspended so I can't poke
> around in its environment.

I'm adding printk's to kernel in a such cases. Likely there is no other
better way to find why it fails.

Does your ARM VM and host both use 4k page size?

Well, if it's a page refcounting bug on ARM/KMV, then applying [1] to
the host driver will make it work and we will know where the problem is.
Please try.

[1]
https://patchwork.kernel.org/project/kvm/patch/20220815095423.11131-1-dmitry.osipenko@collabora.com/

-- 
Best regards,
Dmitry

Re: [PATCH v5 0/8] Support virtio-gpu DRM native context

[Alex Bennée]

Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:

> On 1/23/25 14:58, Alex Bennée wrote:
>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>> 
>>> On 1/22/25 20:00, Alex Bennée wrote:
>>>> Dmitry Osipenko <dmitry.osipenko@collabora.com> writes:
>>>>
>>>>> This patchset adds DRM native context support to VirtIO-GPU on Qemu.
>>>>>
>>>>> Contarary to Virgl and Venus contexts that mediates high level GFX APIs,
>>>>> DRM native context [1] mediates lower level kernel driver UAPI, which
>>>>> reflects in a less CPU overhead and less/simpler code needed to support it.
>>>>> DRM context consists of a host and guest parts that have to be implemented
>>>>> for each GPU driver. On a guest side, DRM context presents a virtual GPU as
>>>>> a real/native host GPU device for GL/VK applications.
>>>>>
>>>>> [1] https://www.youtube.com/watch?v=9sFP_yddLLQ
>>>>>
>>>>> Today there are four known DRM native context drivers existing in a wild:
>>>>>
>>>>>   - Freedreno (Qualcomm SoC GPUs), completely upstreamed
>>>>>   - AMDGPU, mostly merged into upstreams
>>>>
>>>> I tried my AMD system today with:
>>>>
>>>> Host:
>>>>   Aarch64 AVA system
>>>>   Trixie
>>>>   virglrenderer @ v1.1.0/99557f5aa130930d11f04ffeb07f3a9aa5963182
>>>>   -display sdl,gl=on (gtk,gl=on also came up but handled window resizing
>>>>   poorly)
>>>>   
>>>> KVM Guest
>>>>
>>>>   Aarch64
>>>>   Trixie
>>>>   mesa @ main/d27748a76f7dd9236bfcf9ef172dc13b8c0e170f
>>>>   -Dvulkan-drivers=virtio,amd -Dgallium-drivers=virgl,radeonsi -Damdgpu-virtio=true
>>>>
>>>> However when I ran vulkan-info --summary KVM faulted with:
>>>>
>>>>   debian-trixie login: error: kvm run failed Bad address
>>>>    PC=0000ffffb9aa1eb0 X00=0000ffffba0450a4 X01=0000aaaaf7f32400
>>>>   X02=000000000000013c X03=0000ffffba045098 X04=0000aaaaf7f3253c
>>>>   X05=0000ffffba0451d4 X06=00000000c0016900 X07=000000000000000e
>>>>   X08=0000000000000014 X09=00000000000000ff X10=0000aaaaf7f32500
>>>>   X11=0000aaaaf7e4d028 X12=0000aaaaf7edbcb0 X13=0000000000000001
>>>>   X14=000000000000000c X15=0000000000007718 X16=0000ffffb93601f0
>>>>   X17=0000ffffb9aa1dc0 X18=00000000000076f0 X19=0000aaaaf7f31330
>>>>   X20=0000aaaaf7f323f0 X21=0000aaaaf7f235e0 X22=000000000000004c
>>>>   X23=0000aaaaf7f2b5e0 X24=0000aaaaf7ee0cb0 X25=00000000000000ff
>>>>   X26=0000000000000076 X27=0000ffffcd2b18a8 X28=0000aaaaf7ee0cb0
>>>>   X29=0000ffffcd2b0bd0 X30=0000ffffb86c8b98  SP=0000ffffcd2b0bd0
>>>>   PSTATE=20001000 --C- EL0t
>>>>   QEMU 9.2.50 monitor - type 'help' for more information
>>>>   (qemu) quit
>>>>
>>>> Which looks very much like the PFN locking failure. However booting up
>>>> with venus=on instead works. Could there be any differences in the way
>>>> device memory is mapped in the two cases?
>>>
>>> Memory mapping works exactly the same for nctx and venus. Are you on
>>> 6.13 host kernel?
>> 
>> Yes - with the Altra PCI workaround patches on both host and guest
>> kernel.
>> 
>> Is there anyway to trace the sharing of device memory on the host so I
>> can verify its an attempt at device access? The PC looks like its in
>> user-space but once this fails the guest is suspended so I can't poke
>> around in its environment.
>
> I'm adding printk's to kernel in a such cases. Likely there is no other
> better way to find why it fails.
>
> Does your ARM VM and host both use 4k page size?
>
> Well, if it's a page refcounting bug on ARM/KMV, then applying [1] to
> the host driver will make it work and we will know where the problem is.
> Please try.
>
> [1]
> https://patchwork.kernel.org/project/kvm/patch/20220815095423.11131-1-dmitry.osipenko@collabora.com/

That makes no difference.

AFAICT the fault is triggered in userspace:

  error: kvm run failed Bad address
   PC=0000ffffb1911eb0 X00=0000ffffb1eb60a4 X01=0000aaaaeb1f5400
  X02=000000000000013c X03=0000ffffb1eb6098 X04=0000aaaaeb1f553c
  X05=0000ffffb1eb61d4 X06=00000000c0016900 X07=000000000000000e
  X08=0000000000000014 X09=00000000000000ff X10=0000aaaaeb1f5500
  X11=0000aaaaeb110028 X12=0000aaaaeb19ecb0 X13=0000000000000001
  X14=000000000000000c X15=0000000000007718 X16=0000ffffb11d01f0
  X17=0000ffffb1911dc0 X18=00000000000076f0 X19=0000aaaaeb1f4330
  X20=0000aaaaeb1f53f0 X21=0000aaaaeb1e65e0 X22=000000000000004c
  X23=0000aaaaeb1ee5e0 X24=0000aaaaeb1a3cb0 X25=00000000000000ff
  X26=0000000000000076 X27=0000ffffc7db4e58 X28=0000aaaaeb1a3cb0
  X29=0000ffffc7db4180 X30=0000ffffb0538b98  SP=0000ffffc7db4180
  PSTATE=20001000 --C- EL0t
  QEMU 9.2.50 monitor - type 'help' for more information
  (qemu) quit

  Thread 4 received signal SIGABRT, Aborted.
  [Switching to Thread 1.4]
  cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  32              arm_cpuidle_restore_irq_context(&context);
  (gdb) alex
  Undefined command: "alex".  Try "help".
  (gdb) bt
  #0  cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  #1  0xffff800081962180 in arch_cpu_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:44
  #2  0xffff8000819622c4 in default_idle_call () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:117
  #3  0xffff80008013af8c in cpuidle_idle_call () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:185
  #4  do_idle () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:325
  #5  0xffff80008013b208 in cpu_startup_entry (state=state@entry=CPUHP_AP_ONLINE_IDLE) at /home/alex/lsrc/linux.git/kernel/sched/idle.c:423
  #6  0xffff800080043668 in secondary_start_kernel () at /home/alex/lsrc/linux.git/arch/arm64/kernel/smp.c:279
  #7  0xffff800080051f78 in __secondary_switched () at /home/alex/lsrc/linux.git/arch/arm64/kernel/head.S:420
  Backtrace stopped: previous frame identical to this frame (corrupt stack?)
  (gdb) info threads
    Id   Target Id                    Frame 
    1    Thread 1.1 (CPU#0 [running]) cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
    2    Thread 1.2 (CPU#1 [halted ]) 0x0000ffffb1911eb0 in ?? ()
    3    Thread 1.3 (CPU#2 [halted ]) cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  * 4    Thread 1.4 (CPU#3 [halted ]) cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  (gdb) thread 2
  [Switching to thread 2 (Thread 1.2)]
  #0  0x0000ffffb1911eb0 in ?? ()
  (gdb) bt
  #0  0x0000ffffb1911eb0 in ?? ()
  #1  0x0000aaaaeb1ea5e0 in ?? ()
  Backtrace stopped: previous frame inner to this frame (corrupt stack?)
  (gdb) frame 0
  #0  0x0000ffffb1911eb0 in ?? ()
  (gdb) x/5i $pc
  => 0xffffb1911eb0:      str     q3, [x0]
     0xffffb1911eb4:      ldp     q2, q3, [x1, #48]
     0xffffb1911eb8:      subs    x2, x2, #0x90
     0xffffb1911ebc:      b.ls    0xffffb1911ee0  // b.plast
     0xffffb1911ec0:      stp     q0, q1, [x3, #16]
  (gdb) p/x $x0
  $1 = 0xffffb1eb60a4

I suspect that is memcpy again but I'll try and track it down. The only
other note is:

[  411.509647] kvm [7713]: Unsupported FSC: EC=0x24 xFSC=0x21 ESR_EL2=0x92000061

Which is:

  EC 0x24 - Data Abort from lower EL
  DFSC 0x21 - Alignment fault
  WnR 1 - Caused by write
  
-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro