[Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Alexander Graf]

We can currently buffer up to 16 events in our event queue for input
devices. While it sounds like 16 events backlog for everyone should
be enough, our automated testing tools (OpenQA) manage to easily
type faster than our guests can handle.

So we run into queue overflows and start to drop input characters.
By typing quickly, I was even able to reproduce this manually in
a vnc session on SLOF.

Fix this by increasing the queue buffer. With 1k events I'm sure we
can get by for the foreseeable future. To maintain backwards compatibility
on live migration, put the extra queue items into separate subsections.

Reported-by: Dinar Valeev <dvaleev@suse.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
---
 hw/input/hid.c         | 50 ++++++++++++++++++++++++++++++++++++++++++++++----
 include/hw/input/hid.h |  4 ++--
 2 files changed, 48 insertions(+), 6 deletions(-)

diff --git a/hw/input/hid.c b/hw/input/hid.c
index d92c746..56119a3 100644
--- a/hw/input/hid.c
+++ b/hw/input/hid.c
@@ -569,6 +569,16 @@ static int hid_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static bool hid_extra_queue_needed(void *opaque)
+{
+        /*
+         * The queue is a ring, so chances are really good that we need to
+         * submit entries beyond entry 16 anyway. Let's just always send
+         * them, that way we have less chance of missing events by accident.
+         */
+	return true;
+}
+
 static const VMStateDescription vmstate_hid_ptr_queue = {
     .name = "HIDPointerEventQueue",
     .version_id = 1,
@@ -582,19 +592,47 @@ static const VMStateDescription vmstate_hid_ptr_queue = {
     }
 };
 
+static const VMStateDescription vmstate_hid_ptr_extra_queue = {
+    .name = "hid_ptr_extra_queue",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = hid_extra_queue_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_STRUCT_SUB_ARRAY(ptr.queue, HIDState, 16, QUEUE_LENGTH - 16, 0,
+                                 vmstate_hid_ptr_queue, HIDPointerEvent),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 const VMStateDescription vmstate_hid_ptr_device = {
     .name = "HIDPointerDevice",
     .version_id = 1,
     .minimum_version_id = 1,
     .post_load = hid_post_load,
     .fields = (VMStateField[]) {
-        VMSTATE_STRUCT_ARRAY(ptr.queue, HIDState, QUEUE_LENGTH, 0,
-                             vmstate_hid_ptr_queue, HIDPointerEvent),
+        VMSTATE_STRUCT_SUB_ARRAY(ptr.queue, HIDState, 0, 16, 0,
+                                 vmstate_hid_ptr_queue, HIDPointerEvent),
         VMSTATE_UINT32(head, HIDState),
         VMSTATE_UINT32(n, HIDState),
         VMSTATE_INT32(protocol, HIDState),
         VMSTATE_UINT8(idle, HIDState),
         VMSTATE_END_OF_LIST(),
+    },
+    .subsections = (const VMStateDescription*[]) {
+        &vmstate_hid_ptr_extra_queue,
+        NULL
+    }
+};
+
+static const VMStateDescription vmstate_hid_keyboard_extra_keys = {
+    .name = "hid_kbd_extra_keys",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = hid_extra_queue_needed,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT32_SUB_ARRAY(kbd.keycodes, HIDState, 16, QUEUE_LENGTH - 16),
+        VMSTATE_UINT8_SUB_ARRAY(kbd.key, HIDState, 16, QUEUE_LENGTH - 16),
+        VMSTATE_END_OF_LIST()
     }
 };
 
@@ -604,15 +642,19 @@ const VMStateDescription vmstate_hid_keyboard_device = {
     .minimum_version_id = 1,
     .post_load = hid_post_load,
     .fields = (VMStateField[]) {
-        VMSTATE_UINT32_ARRAY(kbd.keycodes, HIDState, QUEUE_LENGTH),
+        VMSTATE_UINT32_SUB_ARRAY(kbd.keycodes, HIDState, 0, 16),
         VMSTATE_UINT32(head, HIDState),
         VMSTATE_UINT32(n, HIDState),
         VMSTATE_UINT16(kbd.modifiers, HIDState),
         VMSTATE_UINT8(kbd.leds, HIDState),
-        VMSTATE_UINT8_ARRAY(kbd.key, HIDState, 16),
+        VMSTATE_UINT8_SUB_ARRAY(kbd.key, HIDState, 0, 16),
         VMSTATE_INT32(kbd.keys, HIDState),
         VMSTATE_INT32(protocol, HIDState),
         VMSTATE_UINT8(idle, HIDState),
         VMSTATE_END_OF_LIST(),
+    },
+    .subsections = (const VMStateDescription*[]) {
+        &vmstate_hid_keyboard_extra_keys,
+        NULL
     }
 };
diff --git a/include/hw/input/hid.h b/include/hw/input/hid.h
index 2127c7c..bc561d4 100644
--- a/include/hw/input/hid.h
+++ b/include/hw/input/hid.h
@@ -13,7 +13,7 @@ typedef struct HIDPointerEvent {
     int32_t dz, buttons_state;
 } HIDPointerEvent;
 
-#define QUEUE_LENGTH    16 /* should be enough for a triple-click */
+#define QUEUE_LENGTH    1024
 #define QUEUE_MASK      (QUEUE_LENGTH-1u)
 #define QUEUE_INCR(v)   ((v)++, (v) &= QUEUE_MASK)
 
@@ -29,7 +29,7 @@ typedef struct HIDKeyboardState {
     uint32_t keycodes[QUEUE_LENGTH];
     uint16_t modifiers;
     uint8_t leds;
-    uint8_t key[16];
+    uint8_t key[QUEUE_LENGTH];
     int32_t keys;
 } HIDKeyboardState;
 
-- 
1.8.5.6

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Gerd Hoffmann]

On Do, 2016-04-14 at 16:25 +0200, Alexander Graf wrote:
> We can currently buffer up to 16 events in our event queue for input
> devices. While it sounds like 16 events backlog for everyone should
> be enough, our automated testing tools (OpenQA) manage to easily
> type faster than our guests can handle.

IMO you should fix OpenQA to type slower then.

Real hardware is made for human typing speeds.  qemu emulates real
hardware.  Typing insane fast is going to cause problems because the
hardware interfaces are not designed for that.

Note the ps/2 keyboard has only a 16 byte queue too.  Used to be 256
bytes, but that caused problems because this diverges from real hardware
(see commit 2858ab09e6f708e381fc1a1cc87e747a690c4884 for all the
details).

Note that the generic input code has support for slow typing already
(see qemu_input_event_send_key_delay in ui/input.c).  That queue will
(a) work for all keyboard devices and (b) isn't visible to the guest, so
using that might be a reasonable option if you can't put the delay logic
into OpenQA.  How to you feed the guest with keystrokes?  monitor?  vnc?

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Alexander Graf]

On 04/14/2016 05:17 PM, Gerd Hoffmann wrote:
> On Do, 2016-04-14 at 16:25 +0200, Alexander Graf wrote:
>> We can currently buffer up to 16 events in our event queue for input
>> devices. While it sounds like 16 events backlog for everyone should
>> be enough, our automated testing tools (OpenQA) manage to easily
>> type faster than our guests can handle.
> IMO you should fix OpenQA to type slower then.
>
> Real hardware is made for human typing speeds.  qemu emulates real
> hardware.  Typing insane fast is going to cause problems because the
> hardware interfaces are not designed for that.
>
> Note the ps/2 keyboard has only a 16 byte queue too.  Used to be 256
> bytes, but that caused problems because this diverges from real hardware
> (see commit 2858ab09e6f708e381fc1a1cc87e747a690c4884 for all the
> details).
>
> Note that the generic input code has support for slow typing already
> (see qemu_input_event_send_key_delay in ui/input.c).  That queue will
> (a) work for all keyboard devices and (b) isn't visible to the guest, so
> using that might be a reasonable option if you can't put the delay logic
> into OpenQA.  How to you feed the guest with keystrokes?  monitor?  vnc?

It used to type via the monitor, but these days everything goes via vnc.

Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
where things fall apart.


Alex

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Gerd Hoffmann]

On Do, 2016-04-14 at 17:29 +0200, Alexander Graf wrote:
> On 04/14/2016 05:17 PM, Gerd Hoffmann wrote:
> > On Do, 2016-04-14 at 16:25 +0200, Alexander Graf wrote:
> >> We can currently buffer up to 16 events in our event queue for input
> >> devices. While it sounds like 16 events backlog for everyone should
> >> be enough, our automated testing tools (OpenQA) manage to easily
> >> type faster than our guests can handle.
> > IMO you should fix OpenQA to type slower then.
> >
> > Real hardware is made for human typing speeds.  qemu emulates real
> > hardware.  Typing insane fast is going to cause problems because the
> > hardware interfaces are not designed for that.
> >
> > Note the ps/2 keyboard has only a 16 byte queue too.  Used to be 256
> > bytes, but that caused problems because this diverges from real hardware
> > (see commit 2858ab09e6f708e381fc1a1cc87e747a690c4884 for all the
> > details).
> >
> > Note that the generic input code has support for slow typing already
> > (see qemu_input_event_send_key_delay in ui/input.c).  That queue will
> > (a) work for all keyboard devices and (b) isn't visible to the guest, so
> > using that might be a reasonable option if you can't put the delay logic
> > into OpenQA.  How to you feed the guest with keystrokes?  monitor?  vnc?
> 
> It used to type via the monitor, but these days everything goes via vnc.
> 
> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
> where things fall apart.

Try a larger delay for starters.

Possibly OpenQA starts typing before the guest detected the device, in
that case the delay wouldn't avoid the queue filling up because the
guest doesn't read it (yet).

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Dinar Valeev]

On 04/14/2016 06:19 PM, Gerd Hoffmann wrote:
> On Do, 2016-04-14 at 17:29 +0200, Alexander Graf wrote:
>> On 04/14/2016 05:17 PM, Gerd Hoffmann wrote:
>>> On Do, 2016-04-14 at 16:25 +0200, Alexander Graf wrote:
>>>> We can currently buffer up to 16 events in our event queue for input
>>>> devices. While it sounds like 16 events backlog for everyone should
>>>> be enough, our automated testing tools (OpenQA) manage to easily
>>>> type faster than our guests can handle.
>>> IMO you should fix OpenQA to type slower then.
>>>
>>> Real hardware is made for human typing speeds.  qemu emulates real
>>> hardware.  Typing insane fast is going to cause problems because the
>>> hardware interfaces are not designed for that.
>>>
>>> Note the ps/2 keyboard has only a 16 byte queue too.  Used to be 256
>>> bytes, but that caused problems because this diverges from real hardware
>>> (see commit 2858ab09e6f708e381fc1a1cc87e747a690c4884 for all the
>>> details).
>>>
>>> Note that the generic input code has support for slow typing already
>>> (see qemu_input_event_send_key_delay in ui/input.c).  That queue will
>>> (a) work for all keyboard devices and (b) isn't visible to the guest, so
>>> using that might be a reasonable option if you can't put the delay logic
>>> into OpenQA.  How to you feed the guest with keystrokes?  monitor?  vnc?
>>
>> It used to type via the monitor, but these days everything goes via vnc.
>>
>> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
>> where things fall apart.
> 
> Try a larger delay for starters.
> 
> Possibly OpenQA starts typing before the guest detected the device, in
> that case the delay wouldn't avoid the queue filling up because the
> guest doesn't read it (yet).
This happens randomly.

Yes we use vnc to type into the screen. If we start 10 guests in parallel,
one or two might have keyboard issues.

like we send "zypper lr" the guest receives "zyppe lr"

or "ps -ef | grep bin/X ; echo __wHP > /dev/hvc0" results into "ps -ef | GREP BIN?X ; echo __wHP > /dev/hvc0"

openQA types with the same speed on all architectures. however we see problems only on aarch64 and POWER due to USB keyboard usage.
> 
> cheers,
>   Gerd
> 

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Gerd Hoffmann]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

  Hi,

> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
> where things fall apart.

Well, not everywhere.  Try the attached patch.

Also worth trying:
 * use xhci instead of ohci (current slof should handle
   kbd-via-xhci fine)
 * use virtio-keyboard (no slof driver yet as far I know, also needs
   a recent linux kernel).

cheers,
  Gerd


[-- Attachment #2: 0001-vnc-add-configurable-keyboard-delay.patch --]
[-- Type: text/x-patch, Size: 3371 bytes --]

From 870daf1865430b5107cfad25142d30d006a0ec2a Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann <kraxel@redhat.com>
Date: Mon, 18 Apr 2016 08:49:00 +0200
Subject: [PATCH] vnc: add configurable keyboard delay

---
 ui/vnc.c | 13 +++++++++++--
 ui/vnc.h |  1 +
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/ui/vnc.c b/ui/vnc.c
index d2ebf1f..ea3b3d4 100644
--- a/ui/vnc.c
+++ b/ui/vnc.c
@@ -1629,6 +1629,7 @@ static void reset_keys(VncState *vs)
     for(i = 0; i < 256; i++) {
         if (vs->modifiers_state[i]) {
             qemu_input_event_send_key_number(vs->vd->dcl.con, i, false);
+            qemu_input_event_send_key_delay(vs->vd->key_delay_ms);
             vs->modifiers_state[i] = 0;
         }
     }
@@ -1638,9 +1639,9 @@ static void press_key(VncState *vs, int keysym)
 {
     int keycode = keysym2scancode(vs->vd->kbd_layout, keysym) & SCANCODE_KEYMASK;
     qemu_input_event_send_key_number(vs->vd->dcl.con, keycode, true);
-    qemu_input_event_send_key_delay(0);
+    qemu_input_event_send_key_delay(vs->vd->key_delay_ms);
     qemu_input_event_send_key_number(vs->vd->dcl.con, keycode, false);
-    qemu_input_event_send_key_delay(0);
+    qemu_input_event_send_key_delay(vs->vd->key_delay_ms);
 }
 
 static int current_led_state(VncState *vs)
@@ -1792,6 +1793,7 @@ static void do_key_event(VncState *vs, int down, int keycode, int sym)
 
     if (qemu_console_is_graphic(NULL)) {
         qemu_input_event_send_key_number(vs->vd->dcl.con, keycode, down);
+        qemu_input_event_send_key_delay(vs->vd->key_delay_ms);
     } else {
         bool numlock = vs->modifiers_state[0x45];
         bool control = (vs->modifiers_state[0x1d] ||
@@ -1913,6 +1915,7 @@ static void vnc_release_modifiers(VncState *vs)
             continue;
         }
         qemu_input_event_send_key_number(vs->vd->dcl.con, keycode, false);
+        qemu_input_event_send_key_delay(vs->vd->key_delay_ms);
     }
 }
 
@@ -3249,6 +3252,9 @@ static QemuOptsList qemu_vnc_opts = {
             .name = "lock-key-sync",
             .type = QEMU_OPT_BOOL,
         },{
+            .name = "key-delay-ms",
+            .type = QEMU_OPT_NUMBER,
+        },{
             .name = "sasl",
             .type = QEMU_OPT_BOOL,
         },{
@@ -3486,6 +3492,7 @@ void vnc_display_open(const char *id, Error **errp)
 #endif
     int acl = 0;
     int lock_key_sync = 1;
+    int key_delay_ms;
 
     if (!vs) {
         error_setg(errp, "VNC display not active");
@@ -3604,6 +3611,7 @@ void vnc_display_open(const char *id, Error **errp)
 
     reverse = qemu_opt_get_bool(opts, "reverse", false);
     lock_key_sync = qemu_opt_get_bool(opts, "lock-key-sync", true);
+    key_delay_ms = qemu_opt_get_number(opts, "key-delay-ms", 1);
     sasl = qemu_opt_get_bool(opts, "sasl", false);
 #ifndef CONFIG_VNC_SASL
     if (sasl) {
@@ -3735,6 +3743,7 @@ void vnc_display_open(const char *id, Error **errp)
     }
 #endif
     vs->lock_key_sync = lock_key_sync;
+    vs->key_delay_ms = key_delay_ms;
 
     device_id = qemu_opt_get(opts, "display");
     if (device_id) {
diff --git a/ui/vnc.h b/ui/vnc.h
index 81a3261..6568bca 100644
--- a/ui/vnc.h
+++ b/ui/vnc.h
@@ -155,6 +155,7 @@ struct VncDisplay
     DisplayChangeListener dcl;
     kbd_layout_t *kbd_layout;
     int lock_key_sync;
+    int key_delay_ms;
     QemuMutex mutex;
 
     QEMUCursor *cursor;
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Alexander Graf]

On 18.04.16 08:53, Gerd Hoffmann wrote:
>   Hi,
> 
>> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
>> where things fall apart.
> 
> Well, not everywhere.  Try the attached patch.
> 
> Also worth trying:
>  * use xhci instead of ohci (current slof should handle
>    kbd-via-xhci fine)
>  * use virtio-keyboard (no slof driver yet as far I know, also needs
>    a recent linux kernel).

Ideally I would really like to switch to virtio-input and virtio-gpu for
AArch64, but there are no drivers at all in OVMF. Do you have any plans
to write them?

IIRC xhci "solved" the problem for PPC, but not on AArch64. I'll have to
double-check with Dirk and Dinar though.

The attached patch also looks sensible. Dinar, Dirk, could you give that
a spin please?


Thanks,

Alex

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Gerd Hoffmann]

On Mo, 2016-04-18 at 09:26 +0200, Alexander Graf wrote:
> 
> On 18.04.16 08:53, Gerd Hoffmann wrote:
> >   Hi,
> > 
> >> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
> >> where things fall apart.
> > 
> > Well, not everywhere.  Try the attached patch.
> > 
> > Also worth trying:
> >  * use xhci instead of ohci (current slof should handle
> >    kbd-via-xhci fine)
> >  * use virtio-keyboard (no slof driver yet as far I know, also needs
> >    a recent linux kernel).
> 
> Ideally I would really like to switch to virtio-input and virtio-gpu for
> AArch64, but there are no drivers at all in OVMF. Do you have any plans
> to write them?

Not investigated yet.  There are virtio 1.0 patches in flight for edk2,
once they are landed it should be alot simpler (virtio 1.0 is mandatory
for virtio-gpu and virtio-input).

Keyboard should be easy, it's a simple device.

GPU is tricky.  Guest is expected to explicitly request display updates.
So simply setting up a dump framebuffer, then depending on dirty page
tracking for screen updates isn't going to fly.  Not sure EFI can handle
that.

The virtio-vga device has a vga compatibility mode additionally to
native virtio.  SLOF uses that IIRC.  But I think on aarch64 that isn't
going to fly either due to the cache coherency issues there.

cheers,
  Gerd

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Alexander Graf]

On 18.04.16 11:21, Gerd Hoffmann wrote:
> On Mo, 2016-04-18 at 09:26 +0200, Alexander Graf wrote:
>>
>> On 18.04.16 08:53, Gerd Hoffmann wrote:
>>>   Hi,
>>>
>>>> Vnc already uses qemu_input_event_send_key_delay today, so I'm not sure 
>>>> where things fall apart.
>>>
>>> Well, not everywhere.  Try the attached patch.
>>>
>>> Also worth trying:
>>>  * use xhci instead of ohci (current slof should handle
>>>    kbd-via-xhci fine)
>>>  * use virtio-keyboard (no slof driver yet as far I know, also needs
>>>    a recent linux kernel).
>>
>> Ideally I would really like to switch to virtio-input and virtio-gpu for
>> AArch64, but there are no drivers at all in OVMF. Do you have any plans
>> to write them?
> 
> Not investigated yet.  There are virtio 1.0 patches in flight for edk2,
> once they are landed it should be alot simpler (virtio 1.0 is mandatory
> for virtio-gpu and virtio-input).
> 
> Keyboard should be easy, it's a simple device.
> 
> GPU is tricky.  Guest is expected to explicitly request display updates.
> So simply setting up a dump framebuffer, then depending on dirty page
> tracking for screen updates isn't going to fly.  Not sure EFI can handle
> that.

It should. The GOP interface usually gets invoked with special BLT
callbacks that copy gfx data from the payload to the frame buffer. Of
course we would need to make sure we don't set the magical "this is
where the frame buffer is, please use it" bit, as otherwise Linux would
try to access it as efifb.

> The virtio-vga device has a vga compatibility mode additionally to
> native virtio.  SLOF uses that IIRC.  But I think on aarch64 that isn't
> going to fly either due to the cache coherency issues there.

Exactly, so a native virtio-gpu driver would be the best option for us
here. Once Linux takes over, it can just switch to its own virtio-gpu
driver.


Alex

Re: [Qemu-devel] [PATCH] hid: Extend the event queue size to 1024

[Juan Quintela]

Alexander Graf <agraf@suse.de> wrote:
> We can currently buffer up to 16 events in our event queue for input
> devices. While it sounds like 16 events backlog for everyone should
> be enough, our automated testing tools (OpenQA) manage to easily
> type faster than our guests can handle.
>
> So we run into queue overflows and start to drop input characters.
> By typing quickly, I was even able to reproduce this manually in
> a vnc session on SLOF.
>
> Fix this by increasing the queue buffer. With 1k events I'm sure we
> can get by for the foreseeable future. To maintain backwards compatibility
> on live migration, put the extra queue items into separate subsections.
>
> Reported-by: Dinar Valeev <dvaleev@suse.de>
> Signed-off-by: Alexander Graf <agraf@suse.de>

Hi

I see that it appears you have agreed in a different solution.

Just to say that this patch is ok from the migration point of view O:-)

Later, Juan.

PD. No, REAL HARDWARE don't have migration either O:-)