[PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[Weifeng Liu]

On startup, scale_x and scale_y were set to 1 that didn't reflect the
real situation of the scan-out, resulting in incorrect cursor
coordinates to be sent when moving the mouse pointer. Simply updating
the scales before rendering the image fixes this issue.

Cc: hikalium <hikalium@hikalium.com>
Cc: Alexander Orzechowski <orzechowski.alexander@gmail.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Weifeng Liu <weifeng.liu.z@gmail.com>
---
 ui/gtk-gl-area.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
index 2c9a0db425..01235f876a 100644
--- a/ui/gtk-gl-area.c
+++ b/ui/gtk-gl-area.c
@@ -42,6 +42,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
 #ifdef CONFIG_GBM
     QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
 #endif
+    int fbw, fbh;
     int ww, wh, ws, y1, y2;
 
     if (!vc->gfx.gls) {
@@ -53,6 +54,11 @@ void gd_gl_area_draw(VirtualConsole *vc)
     ww = gtk_widget_get_allocated_width(vc->gfx.drawing_area) * ws;
     wh = gtk_widget_get_allocated_height(vc->gfx.drawing_area) * ws;
 
+    fbw = surface_width(vc->gfx.ds);
+    fbh = surface_height(vc->gfx.ds);
+    vc->gfx.scale_x = (double)ww / fbw / ws;
+    vc->gfx.scale_y = (double)wh / fbh / ws;
+
     if (vc->gfx.scanout_mode) {
         if (!vc->gfx.guest_fb.framebuffer) {
             return;
-- 
2.49.0

Re: [PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[liuweife]

Sorry, just realized that I forgot to cc Marc-André :D.

Here is a description of the issue this patch is trying to solve:

--------------------------
How to reproduce the issue
--------------------------
Launch the guest with gtk backed virtual display using command like

IMAGE_FOLDER=$HOME/workspace/images/fedora
DISK_PATH=$IMAGE_FOLDER/rootfs.qcow2
UEFI_CODE_PATH=$IMAGE_FOLDER/OVMF_CODE.fd
UEFI_DATA_PATH=$IMAGE_FOLDER/OVMF_VARS.fd
ISO_PATH=$IMAGE_FOLDER/Fedora-Workstation-Live-42-1.1.x86_64.iso
MEM=4G
$QEMU \
    -enable-kvm                                                    \
    -M q35                                                         \
    -object memory-backend-memfd,id=mem1,size=$MEM                 \
    -machine memory-backend=mem1                                   \
    -smp 4                                                         \
    -cpu host                                                      \
    -boot order=c,menu=on,splash-time=3                            \
    -cdrom $ISO_PATH                                               \
    -drive if=pflash,format=raw,readonly=on,file=$UEFI_CODE_PATH   \
    -drive if=pflash,format=raw,file=$UEFI_DATA_PATH               \
    -hda $DISK_PATH                                                \
    -display gtk,gl=on,show-cursor=on                              \
    -device virtio-vga-gl,venus=false,blob=true,hostmem=$MEM       \
    -usb -device usb-tablet

When guest GUI is ready, click the guest display and we are likely to
find that the cursor coordinates are incorrectly reported to the guest
with an offset.

------------------
Cause of the issue
------------------
There is a mapping from the gtk glarea widget coordinates to the guest
display coordinates, and the scale factor plays an essential role in
the calculation. However, the scale factor defaults to 1 and is not
updated to reflect the fact on startup, rendering incorrect cursor
coordinates.

Best regards,
Weifeng

On Sat, 2025-04-19 at 17:48 +0800, Weifeng Liu wrote:
> On startup, scale_x and scale_y were set to 1 that didn't reflect the
> real situation of the scan-out, resulting in incorrect cursor
> coordinates to be sent when moving the mouse pointer. Simply updating
> the scales before rendering the image fixes this issue.
> 
> Cc: hikalium <hikalium@hikalium.com>
> Cc: Alexander Orzechowski <orzechowski.alexander@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Signed-off-by: Weifeng Liu <weifeng.liu.z@gmail.com>
> ---
>  ui/gtk-gl-area.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
> index 2c9a0db425..01235f876a 100644
> --- a/ui/gtk-gl-area.c
> +++ b/ui/gtk-gl-area.c
> @@ -42,6 +42,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
>  #ifdef CONFIG_GBM
>      QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
>  #endif
> +    int fbw, fbh;
>      int ww, wh, ws, y1, y2;
>  
>      if (!vc->gfx.gls) {
> @@ -53,6 +54,11 @@ void gd_gl_area_draw(VirtualConsole *vc)
>      ww = gtk_widget_get_allocated_width(vc->gfx.drawing_area) * ws;
>      wh = gtk_widget_get_allocated_height(vc->gfx.drawing_area) * ws;
>  
> +    fbw = surface_width(vc->gfx.ds);
> +    fbh = surface_height(vc->gfx.ds);
> +    vc->gfx.scale_x = (double)ww / fbw / ws;
> +    vc->gfx.scale_y = (double)wh / fbh / ws;
> +
>      if (vc->gfx.scanout_mode) {
>          if (!vc->gfx.guest_fb.framebuffer) {
>              return;

Re: [PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[Marc-André Lureau]

Hi

On Sat, Apr 19, 2025 at 1:51 PM Weifeng Liu <weifeng.liu.z@gmail.com> wrote:
>
> On startup, scale_x and scale_y were set to 1 that didn't reflect the
> real situation of the scan-out, resulting in incorrect cursor
> coordinates to be sent when moving the mouse pointer. Simply updating
> the scales before rendering the image fixes this issue.
>
> Cc: hikalium <hikalium@hikalium.com>
> Cc: Alexander Orzechowski <orzechowski.alexander@gmail.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Signed-off-by: Weifeng Liu <weifeng.liu.z@gmail.com>


Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>

(Hopefully someone has enough motivation to actually understand this
change better - otherwise I'll simply queue it?)

> ---
>  ui/gtk-gl-area.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
> index 2c9a0db425..01235f876a 100644
> --- a/ui/gtk-gl-area.c
> +++ b/ui/gtk-gl-area.c
> @@ -42,6 +42,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
>  #ifdef CONFIG_GBM
>      QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
>  #endif
> +    int fbw, fbh;
>      int ww, wh, ws, y1, y2;
>
>      if (!vc->gfx.gls) {
> @@ -53,6 +54,11 @@ void gd_gl_area_draw(VirtualConsole *vc)
>      ww = gtk_widget_get_allocated_width(vc->gfx.drawing_area) * ws;
>      wh = gtk_widget_get_allocated_height(vc->gfx.drawing_area) * ws;
>
> +    fbw = surface_width(vc->gfx.ds);
> +    fbh = surface_height(vc->gfx.ds);
> +    vc->gfx.scale_x = (double)ww / fbw / ws;
> +    vc->gfx.scale_y = (double)wh / fbh / ws;
> +
>      if (vc->gfx.scanout_mode) {
>          if (!vc->gfx.guest_fb.framebuffer) {
>              return;
> --
> 2.49.0
>
>


-- 
Marc-André Lureau

Re: [PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[Weifeng Liu]

Hi Marc-André,

I am WIP on a patch set including a successor of this one as well as
other fixes, doc and refactoring regarding scale handling. Hopefully
the patch set would make scale handling into good shape. I will send
the patch set out ASAP.

Best regards,
Weifeng

On Mon, 2025-04-28 at 15:23 +0400, Marc-André Lureau wrote:
> Hi
> 
> On Sat, Apr 19, 2025 at 1:51 PM Weifeng Liu <weifeng.liu.z@gmail.com> wrote:
> > 
> > On startup, scale_x and scale_y were set to 1 that didn't reflect the
> > real situation of the scan-out, resulting in incorrect cursor
> > coordinates to be sent when moving the mouse pointer. Simply updating
> > the scales before rendering the image fixes this issue.
> > 
> > Cc: hikalium <hikalium@hikalium.com>
> > Cc: Alexander Orzechowski <orzechowski.alexander@gmail.com>
> > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > Signed-off-by: Weifeng Liu <weifeng.liu.z@gmail.com>
> 
> 
> Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> (Hopefully someone has enough motivation to actually understand this
> change better - otherwise I'll simply queue it?)
> 
> > ---
> >  ui/gtk-gl-area.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
> > index 2c9a0db425..01235f876a 100644
> > --- a/ui/gtk-gl-area.c
> > +++ b/ui/gtk-gl-area.c
> > @@ -42,6 +42,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
> >  #ifdef CONFIG_GBM
> >      QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
> >  #endif
> > +    int fbw, fbh;
> >      int ww, wh, ws, y1, y2;
> > 
> >      if (!vc->gfx.gls) {
> > @@ -53,6 +54,11 @@ void gd_gl_area_draw(VirtualConsole *vc)
> >      ww = gtk_widget_get_allocated_width(vc->gfx.drawing_area) * ws;
> >      wh = gtk_widget_get_allocated_height(vc->gfx.drawing_area) * ws;
> > 
> > +    fbw = surface_width(vc->gfx.ds);
> > +    fbh = surface_height(vc->gfx.ds);
> > +    vc->gfx.scale_x = (double)ww / fbw / ws;
> > +    vc->gfx.scale_y = (double)wh / fbh / ws;
> > +
> >      if (vc->gfx.scanout_mode) {
> >          if (!vc->gfx.guest_fb.framebuffer) {
> >              return;
> > --
> > 2.49.0
> > 
> > 
> 

Re: [PATCH] ui/gtk: Properly apply x/y scale when rendering GL area

[Weifeng Liu]

Hi Marc-André,

I am WIP on a patch set including a successor of this one as well as
other fixes, doc and refactoring regarding scale handling. Hopefully
the patch set would make scale handling into good shape. I will send
the patch set out ASAP.

Best regards,
Weifeng

On Mon, 2025-04-28 at 15:23 +0400, Marc-André Lureau wrote:
> Hi
> 
> On Sat, Apr 19, 2025 at 1:51 PM Weifeng Liu <weifeng.liu.z@gmail.com> wrote:
> > 
> > On startup, scale_x and scale_y were set to 1 that didn't reflect the
> > real situation of the scan-out, resulting in incorrect cursor
> > coordinates to be sent when moving the mouse pointer. Simply updating
> > the scales before rendering the image fixes this issue.
> > 
> > Cc: hikalium <hikalium@hikalium.com>
> > Cc: Alexander Orzechowski <orzechowski.alexander@gmail.com>
> > Cc: Gerd Hoffmann <kraxel@redhat.com>
> > Signed-off-by: Weifeng Liu <weifeng.liu.z@gmail.com>
> 
> 
> Tested-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> (Hopefully someone has enough motivation to actually understand this
> change better - otherwise I'll simply queue it?)
> 
> > ---
> >  ui/gtk-gl-area.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/ui/gtk-gl-area.c b/ui/gtk-gl-area.c
> > index 2c9a0db425..01235f876a 100644
> > --- a/ui/gtk-gl-area.c
> > +++ b/ui/gtk-gl-area.c
> > @@ -42,6 +42,7 @@ void gd_gl_area_draw(VirtualConsole *vc)
> >  #ifdef CONFIG_GBM
> >      QemuDmaBuf *dmabuf = vc->gfx.guest_fb.dmabuf;
> >  #endif
> > +    int fbw, fbh;
> >      int ww, wh, ws, y1, y2;
> > 
> >      if (!vc->gfx.gls) {
> > @@ -53,6 +54,11 @@ void gd_gl_area_draw(VirtualConsole *vc)
> >      ww = gtk_widget_get_allocated_width(vc->gfx.drawing_area) * ws;
> >      wh = gtk_widget_get_allocated_height(vc->gfx.drawing_area) * ws;
> > 
> > +    fbw = surface_width(vc->gfx.ds);
> > +    fbh = surface_height(vc->gfx.ds);
> > +    vc->gfx.scale_x = (double)ww / fbw / ws;
> > +    vc->gfx.scale_y = (double)wh / fbh / ws;
> > +
> >      if (vc->gfx.scanout_mode) {
> >          if (!vc->gfx.guest_fb.framebuffer) {
> >              return;
> > --
> > 2.49.0
> > 
> > 
>