[PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Shiyuan Gao]

When only use PS/2 mouse without usb-tablet, the mouse pointer of the
guest on the VNC will work badly that the cursor of VNC is inconsistent
with the mouse pointer of guest.

The reason is the PS/2 mouse use relative coordinates and we can't know
the initial position of the guest mouse pointer.

So move the guest mouse pointer to (0, 0) of the screen when connect the
VNC, and then move the mouse pointer to the cursor of VNC(absolute
coordinates are also relative coordinates).

On windows VM, also need disable "Enhance Pointer Precision" Option in
"Pointer Options" (Control Panel -> Mouse).

Signed-off-by: Shiyuan Gao <gaoshiyuan@baidu.com>
---
 hw/input/ps2.c | 2 +-
 ui/vnc.c       | 5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/hw/input/ps2.c b/hw/input/ps2.c
index 45af76a837..e1f44bd298 100644
--- a/hw/input/ps2.c
+++ b/hw/input/ps2.c
@@ -77,7 +77,7 @@
 #define MOUSE_STATUS_ENABLED    0x20
 #define MOUSE_STATUS_SCALE21    0x10
 
-#define PS2_QUEUE_SIZE      16  /* Queue size required by PS/2 protocol */
+#define PS2_QUEUE_SIZE      32  /* Queue size required by PS/2 protocol */
 #define PS2_QUEUE_HEADROOM  8   /* Queue size for keyboard command replies */
 
 /* Bits for 'modifiers' field in PS2KbdState */
diff --git a/ui/vnc.c b/ui/vnc.c
index 92964dcc0c..a1a6048ee4 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -1816,6 +1816,11 @@ static void pointer_event(VncState *vs, int button_mask, int x, int y)
         if (vs->last_x != -1) {
             qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
             qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
+        } else {
+            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
+            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
+            x = 0;
+            y = 0;
         }
         vs->last_x = x;
         vs->last_y = y;
-- 
2.27.0

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 2845 bytes --]

Hi Shiyuan

On Mon, Jul 17, 2023 at 7:16 AM Shiyuan Gao <gaoshiyuan@baidu.com> wrote:

> When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> guest on the VNC will work badly that the cursor of VNC is inconsistent
> with the mouse pointer of guest.
>
>
Afaik, VNC doesn't support client-side drawing of guest mouse (there are no
message to set guest mouse position). So the guest mouse should be drawn by
the server, and currently QEMU doesn't do it.


> The reason is the PS/2 mouse use relative coordinates and we can't know
> the initial position of the guest mouse pointer.
>

It's not just about the initial position.


>
>
So move the guest mouse pointer to (0, 0) of the screen when connect the
> VNC, and then move the mouse pointer to the cursor of VNC(absolute
> coordinates are also relative coordinates).
>
>
It's hardly a solution, you still have no clue what will be the guest mouse
position.


> On windows VM, also need disable "Enhance Pointer Precision" Option in
> "Pointer Options" (Control Panel -> Mouse).
>
>
Apparently, this option doesn't have much to do with relative mouse motion.
Can you explain what it does with this change?


Which guest OS are you using? Hopefully they all support either usb-tablet
or vmmouse extension for absolute positioning. Otherwise, I'd suggest using
Spice, which has those messages for client side guest-mouse drawing.



> Signed-off-by: Shiyuan Gao <gaoshiyuan@baidu.com>
> ---
>  hw/input/ps2.c | 2 +-
>  ui/vnc.c       | 5 +++++
>  2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/hw/input/ps2.c b/hw/input/ps2.c
> index 45af76a837..e1f44bd298 100644
> --- a/hw/input/ps2.c
> +++ b/hw/input/ps2.c
> @@ -77,7 +77,7 @@
>  #define MOUSE_STATUS_ENABLED    0x20
>  #define MOUSE_STATUS_SCALE21    0x10
>
> -#define PS2_QUEUE_SIZE      16  /* Queue size required by PS/2 protocol */
> +#define PS2_QUEUE_SIZE      32  /* Queue size required by PS/2 protocol */
>  #define PS2_QUEUE_HEADROOM  8   /* Queue size for keyboard command
> replies */
>
>  /* Bits for 'modifiers' field in PS2KbdState */
> diff --git a/ui/vnc.c b/ui/vnc.c
> index 92964dcc0c..a1a6048ee4 100644
> --- a/ui/vnc.c
> +++ b/ui/vnc.c
> @@ -1816,6 +1816,11 @@ static void pointer_event(VncState *vs, int
> button_mask, int x, int y)
>          if (vs->last_x != -1) {
>              qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
>              qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
> +        } else {
> +            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> +            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> +            x = 0;
> +            y = 0;
>          }
>          vs->last_x = x;
>          vs->last_y = y;
> --
> 2.27.0
>
>

[-- Attachment #2: Type: text/html, Size: 4314 bytes --]

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Gao,Shiyuan]

Hi,
> Hi Shiyuan
>
> On Mon, Jul 17, 2023 at 7:16 AM Shiyuan Gao <gaoshiyuan@baidu.com> wrote:
>
> > When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> > guest on the VNC will work badly that the cursor of VNC is inconsistent
> > with the mouse pointer of guest.
> >
> >
> Afaik, VNC doesn't support client-side drawing of guest mouse (there are no
> message to set guest mouse position). So the guest mouse should be drawn by
> the server, and currently QEMU doesn't do it.
>
>
> > The reason is the PS/2 mouse use relative coordinates and we can't know
> > the initial position of the guest mouse pointer.
> >
>
> It's not just about the initial position.

Now pointer_event use the cursor of VNC init the vs->last_x, vs->last_y when
connect the VNC and the relative coordinates is based on VNC cursor. If the
initial position of VNC cursor is inconsistent with guest mouse, the
inconsistency will keep. I think the root cause is we cann't know the postion
of the guest mouse pointer.

>
> >
> >
> So move the guest mouse pointer to (0, 0) of the screen when connect the
> > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > coordinates are also relative coordinates).
> >
> >
> It's hardly a solution, you still have no clue what will be the guest mouse
> position.

We have no clue what will be the guest mouse position, we can move the guest
mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
relative coordinates. In a way, this is a quirk to know the guest mouse position.

>
>
> > On windows VM, also need disable "Enhance Pointer Precision" Option in
> > "Pointer Options" (Control Panel -> Mouse).
> >
> >
> Apparently, this option doesn't have much to do with relative mouse motion.
> Can you explain what it does with this change?

Emmm, I don't know why this can solve the problem. It's quite effective and
get this way from
https://forum.proxmox.com/threads/is-it-possible-not-to-use-usbdevice-tablet.1498/.

Only use this way, we need to manually align the VNC cursor with the guest mouse.

>
> Which guest OS are you using? Hopefully they all support either usb-tablet
> or vmmouse extension for absolute positioning. Otherwise, I'd suggest using
> Spice, which has those messages for client side guest-mouse drawing.
>

Yeah, I know the best way is use a absolute positioning. We need remove all usb devices,
I can only use the PS/2 mouse.

Anyway, when connecting to VNC, move the mouse to the upper left corner, at least not
to make things worse.

>
> > Signed-off-by: Shiyuan Gao <gaoshiyuan@baidu.com>
> > ---
> >  hw/input/ps2.c | 2 +-
> >  ui/vnc.c       | 5 +++++
> >  2 files changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/input/ps2.c b/hw/input/ps2.c
> > index 45af76a837..e1f44bd298 100644
> > --- a/hw/input/ps2.c
> > +++ b/hw/input/ps2.c
> > @@ -77,7 +77,7 @@
> >  #define MOUSE_STATUS_ENABLED    0x20
> >  #define MOUSE_STATUS_SCALE21    0x10
> >
> > -#define PS2_QUEUE_SIZE      16  /* Queue size required by PS/2 protocol */
> > +#define PS2_QUEUE_SIZE      32  /* Queue size required by PS/2 protocol */
> >  #define PS2_QUEUE_HEADROOM  8   /* Queue size for keyboard command
> > replies */
> >
> >  /* Bits for 'modifiers' field in PS2KbdState */
> > diff --git a/ui/vnc.c b/ui/vnc.c
> > index 92964dcc0c..a1a6048ee4 100644
> > --- a/ui/vnc.c
> > +++ b/ui/vnc.c
> > @@ -1816,6 +1816,11 @@ static void pointer_event(VncState *vs, int
> > button_mask, int x, int y)
> >          if (vs->last_x != -1) {
> >              qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> >              qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
> > +        } else {
> > +            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> > +            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> > +            x = 0;
> > +            y = 0;
> >          }
> >          vs->last_x = x;
> >          vs->last_y = y;
> > --
> > 2.27.0
> >
> >

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 5423 bytes --]

Hi

On Mon, Jul 17, 2023 at 12:54 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:

> Hi,
> > Hi Shiyuan
> >
> > On Mon, Jul 17, 2023 at 7:16 AM Shiyuan Gao <gaoshiyuan@baidu.com>
> wrote:
> >
> > > When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> > > guest on the VNC will work badly that the cursor of VNC is inconsistent
> > > with the mouse pointer of guest.
> > >
> > >
> > Afaik, VNC doesn't support client-side drawing of guest mouse (there are
> no
> > message to set guest mouse position). So the guest mouse should be drawn
> by
> > the server, and currently QEMU doesn't do it.
> >
> >
> > > The reason is the PS/2 mouse use relative coordinates and we can't know
> > > the initial position of the guest mouse pointer.
> > >
> >
> > It's not just about the initial position.
>
> Now pointer_event use the cursor of VNC init the vs->last_x, vs->last_y
> when
> connect the VNC and the relative coordinates is based on VNC cursor. If the
> initial position of VNC cursor is inconsistent with guest mouse, the
> inconsistency will keep. I think the root cause is we cann't know the
> postion
> of the guest mouse pointer.
>

last_x/last_y is only used when the client doesn't support relative motion
(pointer-motion change pseudo encoding): it doesn't have a direct relation
with the guest pointer position, it's only use to compute the relative
motion of the client pointer as a fallback.


> >
> > >
> > >
> > So move the guest mouse pointer to (0, 0) of the screen when connect the
> > > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > > coordinates are also relative coordinates).
> > >
> > >
> > It's hardly a solution, you still have no clue what will be the guest
> mouse
> > position.
>
> We have no clue what will be the guest mouse position, we can move the
> guest
> mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
> relative coordinates. In a way, this is a quirk to know the guest mouse
> position.
>

There is no guarantee the guest pointer will be at (0,0) though, and that
doesn't explain how that would help. Which client are you using? Are you
drawing the guest cursor? This can't be done currently with the lack of a
message to tell the guest mouse position. (moving / sync-ing the client
cursor position would be even worse in many ways)


> >
> >
> > > On windows VM, also need disable "Enhance Pointer Precision" Option in
> > > "Pointer Options" (Control Panel -> Mouse).
> > >
> > >
> > Apparently, this option doesn't have much to do with relative mouse
> motion.
> > Can you explain what it does with this change?
>
> Emmm, I don't know why this can solve the problem. It's quite effective and
> get this way from
>
> https://forum.proxmox.com/threads/is-it-possible-not-to-use-usbdevice-tablet.1498/
> .
>
> Only use this way, we need to manually align the VNC cursor with the guest
> mouse.
>

It's a bit vague what the "VNC cursor" mean. If you mean the client cursor
position, there is no way to align / sync with the guest without an extra
message to inform the client of its position. There is even a comment about
that in vnc.c  /* can we ask the client(s) to move the pointer ??? */)



> >
> > Which guest OS are you using? Hopefully they all support either
> usb-tablet
> > or vmmouse extension for absolute positioning. Otherwise, I'd suggest
> using
> > Spice, which has those messages for client side guest-mouse drawing.
> >
>
> Yeah, I know the best way is use a absolute positioning. We need remove
> all usb devices,
> I can only use the PS/2 mouse.
>

Can't you use vmmouse extension?


>
> Anyway, when connecting to VNC, move the mouse to the upper left corner,
> at least not
> to make things worse.
>
> >
> > > Signed-off-by: Shiyuan Gao <gaoshiyuan@baidu.com>
> > > ---
> > >  hw/input/ps2.c | 2 +-
> > >  ui/vnc.c       | 5 +++++
> > >  2 files changed, 6 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/hw/input/ps2.c b/hw/input/ps2.c
> > > index 45af76a837..e1f44bd298 100644
> > > --- a/hw/input/ps2.c
> > > +++ b/hw/input/ps2.c
> > > @@ -77,7 +77,7 @@
> > >  #define MOUSE_STATUS_ENABLED    0x20
> > >  #define MOUSE_STATUS_SCALE21    0x10
> > >
> > > -#define PS2_QUEUE_SIZE      16  /* Queue size required by PS/2
> protocol */
> > > +#define PS2_QUEUE_SIZE      32  /* Queue size required by PS/2
> protocol */
> > >  #define PS2_QUEUE_HEADROOM  8   /* Queue size for keyboard command
> > > replies */
> > >
> > >  /* Bits for 'modifiers' field in PS2KbdState */
> > > diff --git a/ui/vnc.c b/ui/vnc.c
> > > index 92964dcc0c..a1a6048ee4 100644
> > > --- a/ui/vnc.c
> > > +++ b/ui/vnc.c
> > > @@ -1816,6 +1816,11 @@ static void pointer_event(VncState *vs, int
> > > button_mask, int x, int y)
> > >          if (vs->last_x != -1) {
> > >              qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> > >              qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
> > > +        } else {
> > > +            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> > > +            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> > > +            x = 0;
> > > +            y = 0;
> > >          }
> > >          vs->last_x = x;
> > >          vs->last_y = y;
> > > --
> > > 2.27.0
> > >
> > >
>
>

[-- Attachment #2: Type: text/html, Size: 7489 bytes --]

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Daniel P. Berrangé]

On Mon, Jul 17, 2023 at 01:16:41PM +0400, Marc-André Lureau wrote:
> Hi
> 
> On Mon, Jul 17, 2023 at 12:54 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:
> 
> > Hi,
> > > Hi Shiyuan
> > >
> > > On Mon, Jul 17, 2023 at 7:16 AM Shiyuan Gao <gaoshiyuan@baidu.com>
> > wrote:
> > >
> > > > When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> > > > guest on the VNC will work badly that the cursor of VNC is inconsistent
> > > > with the mouse pointer of guest.
> > > >
> > > >
> > > Afaik, VNC doesn't support client-side drawing of guest mouse (there are
> > no
> > > message to set guest mouse position). So the guest mouse should be drawn
> > by
> > > the server, and currently QEMU doesn't do it.
> > >
> > >
> > > > The reason is the PS/2 mouse use relative coordinates and we can't know
> > > > the initial position of the guest mouse pointer.
> > > >
> > >
> > > It's not just about the initial position.
> >
> > Now pointer_event use the cursor of VNC init the vs->last_x, vs->last_y
> > when
> > connect the VNC and the relative coordinates is based on VNC cursor. If the
> > initial position of VNC cursor is inconsistent with guest mouse, the
> > inconsistency will keep. I think the root cause is we cann't know the
> > postion
> > of the guest mouse pointer.
> >
> 
> last_x/last_y is only used when the client doesn't support relative motion
> (pointer-motion change pseudo encoding): it doesn't have a direct relation
> with the guest pointer position, it's only use to compute the relative
> motion of the client pointer as a fallback.
> 
> 
> > >
> > > >
> > > >
> > > So move the guest mouse pointer to (0, 0) of the screen when connect the
> > > > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > > > coordinates are also relative coordinates).
> > > >
> > > >
> > > It's hardly a solution, you still have no clue what will be the guest
> > mouse
> > > position.
> >
> > We have no clue what will be the guest mouse position, we can move the
> > guest
> > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
> > relative coordinates. In a way, this is a quirk to know the guest mouse
> > position.
> >
> 
> There is no guarantee the guest pointer will be at (0,0) though, and that
> doesn't explain how that would help. Which client are you using? Are you
> drawing the guest cursor? This can't be done currently with the lack of a
> message to tell the guest mouse position. (moving / sync-ing the client
> cursor position would be even worse in many ways)
> 
> 
> > >
> > >
> > > > On windows VM, also need disable "Enhance Pointer Precision" Option in
> > > > "Pointer Options" (Control Panel -> Mouse).
> > > >
> > > >
> > > Apparently, this option doesn't have much to do with relative mouse
> > motion.
> > > Can you explain what it does with this change?
> >
> > Emmm, I don't know why this can solve the problem. It's quite effective and
> > get this way from
> >
> > https://forum.proxmox.com/threads/is-it-possible-not-to-use-usbdevice-tablet.1498/
> > .
> >
> > Only use this way, we need to manually align the VNC cursor with the guest
> > mouse.
> >
> 
> It's a bit vague what the "VNC cursor" mean. If you mean the client cursor
> position, there is no way to align / sync with the guest without an extra
> message to inform the client of its position. There is even a comment about
> that in vnc.c  /* can we ask the client(s) to move the pointer ??? */)
> 
> 
> 
> > >
> > > Which guest OS are you using? Hopefully they all support either
> > usb-tablet
> > > or vmmouse extension for absolute positioning. Otherwise, I'd suggest
> > using
> > > Spice, which has those messages for client side guest-mouse drawing.
> > >
> >
> > Yeah, I know the best way is use a absolute positioning. We need remove
> > all usb devices,
> > I can only use the PS/2 mouse.
> >
> 
> Can't you use vmmouse extension?

For modern Linux there is virtio-input too IIUC


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Gao,Shiyuan]

> > > >
> > > So move the guest mouse pointer to (0, 0) of the screen when connect the
> > > > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > > > coordinates are also relative coordinates).
> > > >
> > > >
> > > It's hardly a solution, you still have no clue what will be the guest
> > mouse
> > > position.
> >
> > We have no clue what will be the guest mouse position, we can move the
> > guest
> > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
> > relative coordinates. In a way, this is a quirk to know the guest mouse
> > position.
> >
>
> There is no guarantee the guest pointer will be at (0,0) though, and that
> doesn't explain how that would help. Which client are you using? Are you
> drawing the guest cursor? This can't be done currently with the lack of a
> message to tell the guest mouse position. (moving / sync-ing the client
> cursor position would be even worse in many ways)

Sorry, my description isn't accurate.

When connect the vnc server,
    vnc_connect
      -> vs->last_x = -1;
      -> vs->last_y = -1;

move client cursor to vnc screen,
    vnc_client_io
      ->vnc_client_read
        ->protocol_client_msg
          ->pointer_event(x,y)
            -> qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
            -> qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
            -> x=0,y=0
            -> vs->last_x = x;vs->last_y = y;
            -> qemu_input_event_sync   // this will inform the guest move to (0, 0)

the next event,
          ->pointer_event(x,y)
            -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
            -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
            -> qemu_input_event_sync    // this will inform the guest from (0,0) move to (x,y),
                                                                 // the client cursor and guest mouse will sync.

and if the resolution is high, alse need increace the queue size of ps2 PS2_QUEUE_SIZE.
Otherwise, if the guest mouse far from the (0,0) before vnc connect, it can't move to (0,0).

>
> > >
> > >
> > > > On windows VM, also need disable "Enhance Pointer Precision" Option in
> > > > "Pointer Options" (Control Panel -> Mouse).
> > > >
> > > >
> > > Apparently, this option doesn't have much to do with relative mouse
> > motion.
> > > Can you explain what it does with this change?
> >
> > Emmm, I don't know why this can solve the problem. It's quite effective and
> > get this way from
> >
> > https://forum.proxmox.com/threads/is-it-possible-not-to-use-usbdevice-tablet.1498/
> > .
> >
> > Only use this way, we need to manually align the VNC cursor with the guest
> > mouse.
> >
>
> It's a bit vague what the "VNC cursor" mean. If you mean the client cursor
> position, there is no way to align / sync with the guest without an extra
> message to inform the client of its position. There is even a comment about
> that in vnc.c  /* can we ask the client(s) to move the pointer ??? */)

Sorry, I mean the client cursor position. I found disable "Enhance Pointer Precision"
Option, move cursor to the bound of the VNC screen, they maybe align.

>
>
> > >
> > > Which guest OS are you using? Hopefully they all support either
> > usb-tablet
> > > or vmmouse extension for absolute positioning. Otherwise, I'd suggest
> > using
> > > Spice, which has those messages for client side guest-mouse drawing.
> > >
> >
> > Yeah, I know the best way is use a absolute positioning. We need remove
> > all usb devices,
> > I can only use the PS/2 mouse.
> >
>
> Can't you use vmmouse extension?

Thanks, I will try it. I cann't found the official driver of vmmouse, can I
install it for free?

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 4780 bytes --]

Hi

On Mon, Jul 17, 2023 at 4:53 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:

> > > > >
> > > > So move the guest mouse pointer to (0, 0) of the screen when connect
> the
> > > > > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > > > > coordinates are also relative coordinates).
> > > > >
> > > > >
> > > > It's hardly a solution, you still have no clue what will be the guest
> > > mouse
> > > > position.
> > >
> > > We have no clue what will be the guest mouse position, we can move the
> > > guest
> > > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
> > > relative coordinates. In a way, this is a quirk to know the guest mouse
> > > position.
> > >
> >
> > There is no guarantee the guest pointer will be at (0,0) though, and that
> > doesn't explain how that would help. Which client are you using? Are you
> > drawing the guest cursor? This can't be done currently with the lack of a
> > message to tell the guest mouse position. (moving / sync-ing the client
> > cursor position would be even worse in many ways)
>
> Sorry, my description isn't accurate.
>
> When connect the vnc server,
>     vnc_connect
>       -> vs->last_x = -1;
>       -> vs->last_y = -1;
>
> move client cursor to vnc screen,
>     vnc_client_io
>       ->vnc_client_read
>         ->protocol_client_msg
>           ->pointer_event(x,y)
>             -> qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
>             -> qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
>             -> x=0,y=0
>             -> vs->last_x = x;vs->last_y = y;
>             -> qemu_input_event_sync   // this will inform the guest move
> to (0, 0)
>
> the next event,
>           ->pointer_event(x,y)
>             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
>             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
>             -> qemu_input_event_sync    // this will inform the guest from
> (0,0) move to (x,y),
>                                                                  // the
> client cursor and guest mouse will sync.
>

Actually, it will be ( x - vs->last_x, y - vs->last_y), not necessarily (x,
y), unless you also set last_x = 0 / last_y = 0.

But even then, there is no guarantee the guest position will be a x/y...


> and if the resolution is high, alse need increace the queue size of ps2
> PS2_QUEUE_SIZE.
> Otherwise, if the guest mouse far from the (0,0) before vnc connect, it
> can't move to (0,0).
>
>
Oh indeed, ps2 will queue multiple messages.. another reason you can't do
such big relative moves reliably.


> >
> > > >
> > > >
> > > > > On windows VM, also need disable "Enhance Pointer Precision"
> Option in
> > > > > "Pointer Options" (Control Panel -> Mouse).
> > > > >
> > > > >
> > > > Apparently, this option doesn't have much to do with relative mouse
> > > motion.
> > > > Can you explain what it does with this change?
> > >
> > > Emmm, I don't know why this can solve the problem. It's quite
> effective and
> > > get this way from
> > >
> > >
> https://forum.proxmox.com/threads/is-it-possible-not-to-use-usbdevice-tablet.1498/
> > > .
> > >
> > > Only use this way, we need to manually align the VNC cursor with the
> guest
> > > mouse.
> > >
> >
> > It's a bit vague what the "VNC cursor" mean. If you mean the client
> cursor
> > position, there is no way to align / sync with the guest without an extra
> > message to inform the client of its position. There is even a comment
> about
> > that in vnc.c  /* can we ask the client(s) to move the pointer ??? */)
>
> Sorry, I mean the client cursor position. I found disable "Enhance Pointer
> Precision"
> Option, move cursor to the bound of the VNC screen, they maybe align.
>
> >
> >
> > > >
> > > > Which guest OS are you using? Hopefully they all support either
> > > usb-tablet
> > > > or vmmouse extension for absolute positioning. Otherwise, I'd suggest
> > > using
> > > > Spice, which has those messages for client side guest-mouse drawing.
> > > >
> > >
> > > Yeah, I know the best way is use a absolute positioning. We need remove
> > > all usb devices,
> > > I can only use the PS/2 mouse.
> > >
> >
> > Can't you use vmmouse extension?
>
> Thanks, I will try it. I cann't found the official driver of vmmouse, can I
> install it for free?
>
>
It's been implemented on Linux VM for a long while (first in userspace,
then in kernel since ~2015). But I don't know about the Windows support, it
looks like you need a driver such as provided by VMWave, but it seems
closed-source...  At this point it's probably better to use virtio-input,
which has open-source/free windows drivers.

[-- Attachment #2: Type: text/html, Size: 6467 bytes --]

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Gao,Shiyuan]

> Hi
>
> On Mon, Jul 17, 2023 at 4:53 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:
>
> > > > > >
> > > > > So move the guest mouse pointer to (0, 0) of the screen when connect
> > the
> > > > > > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > > > > > coordinates are also relative coordinates).
> > > > > >
> > > > > >
> > > > > It's hardly a solution, you still have no clue what will be the guest
> > > > mouse
> > > > > position.
> > > >
> > > > We have no clue what will be the guest mouse position, we can move the
> > > > guest
> > > > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will be the
> > > > relative coordinates. In a way, this is a quirk to know the guest mouse
> > > > position.
> > > >
> > >
> > > There is no guarantee the guest pointer will be at (0,0) though, and that
> > > doesn't explain how that would help. Which client are you using? Are you
> > > drawing the guest cursor? This can't be done currently with the lack of a
> > > message to tell the guest mouse position. (moving / sync-ing the client
> > > cursor position would be even worse in many ways)
> >
> > Sorry, my description isn't accurate.
> >
> > When connect the vnc server,
> >     vnc_connect
> >       -> vs->last_x = -1;
> >       -> vs->last_y = -1;
> >
> > move client cursor to vnc screen,
> >     vnc_client_io
> >       ->vnc_client_read
> >         ->protocol_client_msg
> >           ->pointer_event(x,y)
> >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> >             -> qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> >             -> x=0,y=0
> >             -> vs->last_x = x;vs->last_y = y;
> >             -> qemu_input_event_sync   // this will inform the guest move
> > to (0, 0)
> >
> > the next event,
> >           ->pointer_event(x,y)
> >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> >             -> qemu_input_event_sync    // this will inform the guest from
> > (0,0) move to (x,y),
> >                                                                  // the
> > client cursor and guest mouse will sync.
> >
>
> Actually, it will be ( x - vs->last_x, y - vs->last_y), not necessarily (x,
> y), unless you also set last_x = 0 / last_y = 0.

yes, set last_x = 0 / last_y = 0 and only the first time enter pointer_event.

         if (vs->last_x != -1) {
             qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
             qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
+        } else {
+            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
+            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
+            x = 0;
+            y = 0;
         }
         vs->last_x = x;
         vs->last_y = y;

>
> But even then, there is no guarantee the guest position will be a x/y...
>

Emmm, compared to the current implementation, it is at least getting closer.
The current state is simply unbearable.

> It's been implemented on Linux VM for a long while (first in userspace,
> then in kernel since ~2015). But I don't know about the Windows support, it
> looks like you need a driver such as provided by VMWave, but it seems
> closed-source...  At this point it's probably better to use virtio-input,
> which has open-source/free windows drivers.

Thanks, I tried install vmmouse driver in windows success and reboot. However 'info mice' as
follows and don't have vmmouse (absolute).

* Mouse #2: QEMU PS/2 Mouse

The guest mouse isn't sync with client cursor...

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 4093 bytes --]

Hi

On Mon, Jul 17, 2023 at 5:43 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:

> > Hi
> >
> > On Mon, Jul 17, 2023 at 4:53 PM Gao,Shiyuan <gaoshiyuan@baidu.com>
> wrote:
> >
> > > > > > >
> > > > > > So move the guest mouse pointer to (0, 0) of the screen when
> connect
> > > the
> > > > > > > VNC, and then move the mouse pointer to the cursor of
> VNC(absolute
> > > > > > > coordinates are also relative coordinates).
> > > > > > >
> > > > > > >
> > > > > > It's hardly a solution, you still have no clue what will be the
> guest
> > > > > mouse
> > > > > > position.
> > > > >
> > > > > We have no clue what will be the guest mouse position, we can move
> the
> > > > > guest
> > > > > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will
> be the
> > > > > relative coordinates. In a way, this is a quirk to know the guest
> mouse
> > > > > position.
> > > > >
> > > >
> > > > There is no guarantee the guest pointer will be at (0,0) though, and
> that
> > > > doesn't explain how that would help. Which client are you using? Are
> you
> > > > drawing the guest cursor? This can't be done currently with the lack
> of a
> > > > message to tell the guest mouse position. (moving / sync-ing the
> client
> > > > cursor position would be even worse in many ways)
> > >
> > > Sorry, my description isn't accurate.
> > >
> > > When connect the vnc server,
> > >     vnc_connect
> > >       -> vs->last_x = -1;
> > >       -> vs->last_y = -1;
> > >
> > > move client cursor to vnc screen,
> > >     vnc_client_io
> > >       ->vnc_client_read
> > >         ->protocol_client_msg
> > >           ->pointer_event(x,y)
> > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> > >             -> qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> > >             -> x=0,y=0
> > >             -> vs->last_x = x;vs->last_y = y;
> > >             -> qemu_input_event_sync   // this will inform the guest
> move
> > > to (0, 0)
> > >
> > > the next event,
> > >           ->pointer_event(x,y)
> > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> > >             -> qemu_input_event_sync    // this will inform the guest
> from
> > > (0,0) move to (x,y),
> > >                                                                  // the
> > > client cursor and guest mouse will sync.
> > >
> >
> > Actually, it will be ( x - vs->last_x, y - vs->last_y), not necessarily
> (x,
> > y), unless you also set last_x = 0 / last_y = 0.
>
> yes, set last_x = 0 / last_y = 0 and only the first time enter
> pointer_event.
>
>          if (vs->last_x != -1) {
>              qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
>              qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
> +        } else {
> +            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> +            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> +            x = 0;
> +            y = 0;
>          }
>          vs->last_x = x;
>          vs->last_y = y;
>
> >
> > But even then, there is no guarantee the guest position will be a x/y...
> >
>
> Emmm, compared to the current implementation, it is at least getting
> closer.
> The current state is simply unbearable.
>
> > It's been implemented on Linux VM for a long while (first in userspace,
> > then in kernel since ~2015). But I don't know about the Windows support,
> it
> > looks like you need a driver such as provided by VMWave, but it seems
> > closed-source...  At this point it's probably better to use virtio-input,
> > which has open-source/free windows drivers.
>
> Thanks, I tried install vmmouse driver in windows success and reboot.
> However 'info mice' as
> follows and don't have vmmouse (absolute).
>
> * Mouse #2: QEMU PS/2 Mouse
>
> The guest mouse isn't sync with client cursor...
>

Look if you have "dev: vmmouse, id " in "info qtree" or set -machine
vmport=on.

[-- Attachment #2: Type: text/html, Size: 5465 bytes --]

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Gao,Shiyuan]

> Hi
>
> On Mon, Jul 17, 2023 at 5:43 PM Gao,Shiyuan <gaoshiyuan@baidu.com> wrote:
>
> > > Hi
> > >
> > > On Mon, Jul 17, 2023 at 4:53 PM Gao,Shiyuan <gaoshiyuan@baidu.com>
> > wrote:
> > >
> > > > > > > >
> > > > > > > So move the guest mouse pointer to (0, 0) of the screen when
> > connect
> > > > the
> > > > > > > > VNC, and then move the mouse pointer to the cursor of
> > VNC(absolute
> > > > > > > > coordinates are also relative coordinates).
> > > > > > > >
> > > > > > > >
> > > > > > > It's hardly a solution, you still have no clue what will be the
> > guest
> > > > > > mouse
> > > > > > > position.
> > > > > >
> > > > > > We have no clue what will be the guest mouse position, we can move
> > the
> > > > > > guest
> > > > > > mouse to (0,0) each connect the VNC. Now, the cursor of VNC will
> > be the
> > > > > > relative coordinates. In a way, this is a quirk to know the guest
> > mouse
> > > > > > position.
> > > > > >
> > > > >
> > > > > There is no guarantee the guest pointer will be at (0,0) though, and
> > that
> > > > > doesn't explain how that would help. Which client are you using? Are
> > you
> > > > > drawing the guest cursor? This can't be done currently with the lack
> > of a
> > > > > message to tell the guest mouse position. (moving / sync-ing the
> > client
> > > > > cursor position would be even worse in many ways)
> > > >
> > > > Sorry, my description isn't accurate.
> > > >
> > > > When connect the vnc server,
> > > >     vnc_connect
> > > >       -> vs->last_x = -1;
> > > >       -> vs->last_y = -1;
> > > >
> > > > move client cursor to vnc screen,
> > > >     vnc_client_io
> > > >       ->vnc_client_read
> > > >         ->protocol_client_msg
> > > >           ->pointer_event(x,y)
> > > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> > > >             -> qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> > > >             -> x=0,y=0
> > > >             -> vs->last_x = x;vs->last_y = y;
> > > >             -> qemu_input_event_sync   // this will inform the guest
> > move
> > > > to (0, 0)
> > > >
> > > > the next event,
> > > >           ->pointer_event(x,y)
> > > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> > > >             -> qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> > > >             -> qemu_input_event_sync    // this will inform the guest
> > from
> > > > (0,0) move to (x,y),
> > > >                                                                  // the
> > > > client cursor and guest mouse will sync.
> > > >
> > >
> > > Actually, it will be ( x - vs->last_x, y - vs->last_y), not necessarily
> > (x,
> > > y), unless you also set last_x = 0 / last_y = 0.
> >
> > yes, set last_x = 0 / last_y = 0 and only the first time enter
> > pointer_event.
> >
> >          if (vs->last_x != -1) {
> >              qemu_input_queue_rel(con, INPUT_AXIS_X, x - vs->last_x);
> >              qemu_input_queue_rel(con, INPUT_AXIS_Y, y - vs->last_y);
> > +        } else {
> > +            qemu_input_queue_rel(con, INPUT_AXIS_X, 0 - width);
> > +            qemu_input_queue_rel(con, INPUT_AXIS_Y, 0 - height);
> > +            x = 0;
> > +            y = 0;
> >          }
> >          vs->last_x = x;
> >          vs->last_y = y;
> >
> > >
> > > But even then, there is no guarantee the guest position will be a x/y...
> > >
> >
> > Emmm, compared to the current implementation, it is at least getting
> > closer.
> > The current state is simply unbearable.
> >
> > > It's been implemented on Linux VM for a long while (first in userspace,
> > > then in kernel since ~2015). But I don't know about the Windows support,
> > it
> > > looks like you need a driver such as provided by VMWave, but it seems
> > > closed-source...  At this point it's probably better to use virtio-input,
> > > which has open-source/free windows drivers.
> >
> > Thanks, I tried install vmmouse driver in windows success and reboot.
> > However 'info mice' as
> > follows and don't have vmmouse (absolute).
> >
> > * Mouse #2: QEMU PS/2 Mouse
> >
> > The guest mouse isn't sync with client cursor...
> >
>
> Look if you have "dev: vmmouse, id " in "info qtree" or set -machine
> vmport=on.

"info qtree" as follows, the client cursor and guest mouse cann't align...

          dev: vmmouse, id ""
          dev: vmport, id ""
            x-read-set-eax = true
            x-signal-unsupported-cmd = true
            x-report-vmx-type = true
            x-cmds-v2 = true
            vmware-vmx-version = 6 (0x6)
            vmware-vmx-type = 2 (0x2)
          dev: i8042, id ""
            gpio-out "a20" 1
            extended-state = true
            kbd-throttle = false
            isa irqs 1,12

I have decided to still use the method in this patch. 

Thanks Marc-André.

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Daniel P. Berrangé]

On Mon, Jul 17, 2023 at 10:59:36AM +0800, Shiyuan Gao wrote:
> When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> guest on the VNC will work badly that the cursor of VNC is inconsistent
> with the mouse pointer of guest.
> 
> The reason is the PS/2 mouse use relative coordinates and we can't know
> the initial position of the guest mouse pointer.
> 
> So move the guest mouse pointer to (0, 0) of the screen when connect the
> VNC, and then move the mouse pointer to the cursor of VNC(absolute
> coordinates are also relative coordinates).

This is not a solution, because even if you know the starting position
of the pointer, the host/guest pointers can have different motion
acceleration behaviour, so they'll loose sync with each other. You'll
have problems such as the host pointer exiting the VNC window before
the guest pointer has got to the far edge.

Trying to make a relative pointer work correctly in combination with
absolute input events from VNC is never going to work.

This is why QEMU created a VNC extension to allows switching the VNC
protocol from absolute to relative pointer motion events.

If you need to run a VM with only the PS/2 pointer, then you need
to be using a client that supports this extension:

  https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#qemu-pointer-motion-change-pseudo-encoding


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] vnc,ps2: fix the PS/2 mouse work badly when connect VNC

[Gao,Shiyuan]

> On Mon, Jul 17, 2023 at 10:59:36AM +0800, Shiyuan Gao wrote:
> > When only use PS/2 mouse without usb-tablet, the mouse pointer of the
> > guest on the VNC will work badly that the cursor of VNC is inconsistent
> > with the mouse pointer of guest.
> >
> > The reason is the PS/2 mouse use relative coordinates and we can't know
> > the initial position of the guest mouse pointer.
> >
> > So move the guest mouse pointer to (0, 0) of the screen when connect the
> > VNC, and then move the mouse pointer to the cursor of VNC(absolute
> > coordinates are also relative coordinates).
>
> This is not a solution, because even if you know the starting position
> of the pointer, the host/guest pointers can have different motion
> acceleration behaviour, so they'll loose sync with each other. You'll
> have problems such as the host pointer exiting the VNC window before
> the guest pointer has got to the far edge.

Maybe they will loose sync with each other again. On windows VM such as win10,
disable "Enhance Pointer Precision" Option in "Pointer Options"
(Control Panel -> Mouse), I can't found this problem.

>
> Trying to make a relative pointer work correctly in combination with
> absolute input events from VNC is never going to work.
>
>
> This is why QEMU created a VNC extension to allows switching the VNC
> protocol from absolute to relative pointer motion events.
>
>
> If you need to run a VM with only the PS/2 pointer, then you need
> to be using a client that supports this extension:
>
>
> https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#qemu-pointer-motion-change-pseudo-encoding <https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#qemu-pointer-motion-change-pseudo-encoding>
>

Thanks a lot, I will read it for more info.