From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([2001:4830:134:3::10]:48536)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kraxel@redhat.com>) id 1WYbbu-0004Eq-7e
	for qemu-devel@nongnu.org; Fri, 11 Apr 2014 09:35:19 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <kraxel@redhat.com>) id 1WYbbn-0003rm-0Y
	for qemu-devel@nongnu.org; Fri, 11 Apr 2014 09:35:14 -0400
Received: from mx1.redhat.com ([209.132.183.28]:7304)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <kraxel@redhat.com>) id 1WYbbm-0003qJ-PK
	for qemu-devel@nongnu.org; Fri, 11 Apr 2014 09:35:06 -0400
Message-ID: <1397223298.22291.121.camel@nilsson.home.kraxel.org>
From: Gerd Hoffmann <kraxel@redhat.com>
Date: Fri, 11 Apr 2014 15:34:58 +0200
In-Reply-To: <20110202044825.GB27032@barra.bne.redhat.com>
References: <4D416F07.6060604@redhat.com>
	<20110202044825.GB27032@barra.bne.redhat.com>
Content-Type: text/plain; charset="UTF-8"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: Re: [Qemu-devel] paravirtual mouse/tablet, v5
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Peter Hutterer <peter.hutterer@who-t.net>
Cc: spice-devel <spice-devel@lists.freedesktop.org>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "Michael S. Tsirkin" <mst@redhat.com>

On Mi, 2011-02-02 at 14:48 +1000, Peter Hutterer wrote:
       ^^^^^^^^^^
FYI: It's been a while ...

> sorry, late again. conference last week.
> 
> On Thu, Jan 27, 2011 at 02:11:35PM +0100, Gerd Hoffmann wrote:
> > Next revision the pvmouse protocol.  It is quite different now, I've
> > decided to move to a model with one message per updated value,
> > simliar to the linux input layer.  There isn't a "mouse move"
> > message any more.  A mouse move event will be three messages now:
> > one to update X, one to update Y and a third sync message to mark
> > the end of the message block.  That should be *alot* easier to
> > extend in the future.
> 
> I like this approach, it's a lot more flexible and quite close to what the
> linux kernel already does.
> Having said that, I'm thinking why not do exactly what the linux kernel
> does?

... but there is working code now, doing exactly that.  There is a new
virtio input device, which basically sends evdev events over virtio.

specs:
  http://www.kraxel.org/cgit/virtio-spec/log/?h=virtio-input

linux kernel:
  http://www.kraxel.org/cgit/linux/log/?h=virtio-input

qemu (also containing some unrelated qemu input stuff):
  http://www.kraxel.org/cgit/qemu/log/?h=rebase/input-wip

Of course it's not limited to a mouse, there is a virtio keyboard too,
and we can pass through any host input device to the guest.

cheers,
  Gerd

[ leaving full quote below for those which might look at it
  for historical interest ]

> * the kernel has a stable API that has a number of users and is well
> understood, plus 
> * it allows for code re-use on the consumer side
> * you'd need to do protocol conversion anyway. having the same API means at
> least one platform (i.e. linux) can skip the conversion.
> * MT is being worked on by a number of people and two protocols that cater
> for current hardware (with and without HW tracking features). trying to
> duplicate this means we'll likely re-do the same mistakes.
> 
> fwiw, this protocol is already quite similar
>     MSG_SYNC is SYN_REPORT
>     MSG_POINT is SYN_MT_REPORT
>     AXIS x is the same as EV_ABS + ABS_X
> 
> there may be some optimization for networks of course if you want to change
> the message format but at least the principle seems sound enough.
> 
> > Header file is attached.  Comments are welcome.
> > 
> > thanks,
> >   Gerd
> 
> > #ifndef __QEMU_PVMOUSE__
> > #define __QEMU_PVMOUSE__ 1
> > 
> > /*
> >  * qemu patavirtual mouse/tablet interface
> >  *
> >  * quick overview
> >  * ==============
> >  *
> >  * device initialization
> >  * ---------------------
> >  *
> >  *  (1) host sends INIT with supported feature bits
> >  *  (2) guests sends ACK to ack the features it is
> >  *      able to handle (and willing to use).
> 
> what's the effect of a guest not able to handle certain features? The host
> filters them?
> 
> >  *  (3) host sends a INFO message for each button and
> >  *      each axis supported.  More INFO messages might
> >  *      follow for features added to the protocol later
> >  *      on if enabled by the INIT+ACK handshake.
> >  *      A SYNC message marks the end of the device
> >  *      information messages.
> >  *
> >  * input event reporting
> >  * ---------------------
> >  *
> >  *  host sends one or more BTN_DOWN, BTN_UP and AXIS messages,
> >  *  followed by a SYNC.  A button press would be just BTN_DOWN+SYNC.
> >  *  A mouse move would be two AXIS messages (one for x, one for y)
> >  *  followed by SYNC.
> 
> fwiw, kernel doesn't have BTN_DOWN events but rather BTN_FOO 0 or 1 events
> (for released and pressed, respectively). Same for keys, which also allows
> keys to have a value of 2 for autorepeat.
> 
> it provides more symmetry as well because currently you have
>     AXIS <axis code> <axis value>
>     BTN_DOWN <button code>
> 
> so axis and button_down are asymmetrical in format.
> alternatively, you could use
>     AXIS <axis code> <axis value>
>     BTN <button code> <button value>
> 
> thus making the message format identical.
> 
> >  *
> >  * multitouch events
> >  * -----------------
> >  *
> >  *  Each reported touch point starts with a POINT message, followed by
> >  *  multiple AXIS messages (at least x+y, most likely also pressure).
> >  *  The whole group is followed by SYNC.  A report for two fingers
> >  *  would look like this:
> >  *
> >  *   POINT
> >  *     AXIS x=23
> >  *     AXIS y=42
> >  *     AXIS pressure=3
> >  *   POINT
> >  *     AXIS x=78
> >  *     AXIS y=56
> >  *     AXIS pressure=4
> >  *   SYNC
> >  *
> >  *  In case the device supports ID tracking the POINT message will
> >  *  carry the ID.  Updates need only be sent for points which did
> >  *  change.  IDs are added by using them the first time.  IDs are
> >  *  invalidated when the finger is lifted (aka pressure=0).
> 
> how about devices that support hovering, where the pressure is 0 but the
> touchpoint is still clearly in proximity and active?
> 
> >  *  In case the device doesn't support ID tracking each report must
> >  *  include all current touch points (in unspecified order).
> >  *
> >  */
> > 
> > #include <inttypes.h>
> > 
> > /*
> >  * our virtio-serial channel name(s)
> >  */
> > #define QEMU_PVMOUSE_FORMAT "org.qemu.pvmouse.%d"
> > 
> > enum qemu_pvmouse_msg_type {
> >     /* feature flag negotiation */
> >     QEMU_PVMOUSE_MSG_INIT,
> >     QEMU_PVMOUSE_MSG_ACK,
> >     /* device information */
> >     QEMU_PVMOUSE_MSG_AXIS_INFO,
> >     QEMU_PVMOUSE_MSG_BUTTON_INFO,
> >     /* device events */
> >     QEMU_PVMOUSE_MSG_BTN_DOWN,
> >     QEMU_PVMOUSE_MSG_BTN_UP,
> >     QEMU_PVMOUSE_MSG_AXIS,
> >     /* message grouping */
> >     QEMU_PVMOUSE_MSG_POINT,
> >     QEMU_PVMOUSE_MSG_SYNC,
> > };
> > 
> > typedef enum qemu_pvmouse_features {
> >     QEMU_PVMOUSE_FEATURE_MULTITOUCH,
> > };
> 
> with or without tracking? I can't see any flag that tells us that ahead of
> time but for consumers it's quite important.
> 
> how many simultaneous touchpoint does the device support?
> 
> again, quick overview of the kernel here:
> MT protocol A is essentially just positions + MT_SYN (POINT) events.
> MT protocol B for tracked uses TRACKING_ID as well as MT_SLOT. the slots say
> how many simultaneous trackpoints can go on. tracking ID is >= 0 for any
> given touchpoint and -1 to cancel a touchpoint in the current slot.
> 
> So without kernel-side filtering, you'd get
> 
> ABS_MT_SLOT             0
> ABS_MT_TRACKING_ID      23
> ABS_MT_POSITION_X       100 
> ABS_MT_SLOT             1
> ABS_MT_TRACKING_ID      28
> ABS_MT_POSITION_X       23
> ABS_MT_POSITION_Y       156
> SYN_REPORT
> -----------------------------
> ABS_MT_SLOT             0
> ABS_MT_TRACKING_ID      -1
> ABS_MT_SLOT             1
> ABS_MT_TRACKING_ID      28
> ABS_MT_POSITION_X       30
> ABS_MT_POSITION_Y       162
> SYN_REPORT
> -----------------------------
> ABS_MT_SLOT             0
> ABS_MT_TRACKING_ID      29
> ABS_MT_POSITION_X       600
> ABS_MT_POSITION_Y       876
> ABS_MT_SLOT             1
> ABS_MT_TRACKING_ID      28
> ABS_MT_POSITION_X       30
> ABS_MT_POSITION_Y       162
> SYN_REPORT
> 
> so you have three events, first two touchpoints, then first touchpoint is
> lifted, new touchpoint is created.
> 
> > /*
> >  * QEMU_PVMOUSE_MSG_INIT, host -> guest
> >  * First message.  Sent before any other event.
> >  */
> > typedef struct qemu_pvmouse_init {
> >     uint32_t features;        /* qemu_pvmouse_features */
> > } qemu_pvmouse_init;
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_ACK, guest -> host
> >  * Sent after pvmouse_init.  Host will not send
> >  * additional messages until this is received.
> >  */
> > typedef struct qemu_pvmouse_ack {
> >     uint32_t features; /* qemu_pvtable_features */
> > };
> > 
> > 
> > enum qemu_pvmouse_axis_type {
> >     /* absolute */
> >     QEMU_PVMOUSE_AXIS_POS_X = 1,
> >     QEMU_PVMOUSE_AXIS_POS_Y,
> >     QEMU_PVMOUSE_AXIS_PRESSURE,
> > 
> >     /* relative */
> >     QEMU_PVMOUSE_AXIS_REL_X = 257,
> >     QEMU_PVMOUSE_AXIS_REL_Y,
> >     QEMU_PVMOUSE_AXIS_WHEEL_VERT,
> >     QEMU_PVMOUSE_AXIS_WHEEL_HORIZ,
> > };
> > 
> > enum qemu_pvmouse_axis_flags {
> >     QEMU_PVMOUSE_AXIS_PEN,
> >     QEMU_PVMOUSE_AXIS_TOUCH,
> >     QEMU_PVMOUSE_AXIS_MULTITOUCH,
> >     QEMU_PVMOUSE_AXIS_POINT_ID,
> > };
> > #define QEMU_PVMOUSE_AXIS_MASK_PEN         (1 << QEMU_PVMOUSE_AXIS_PEN)
> > #define QEMU_PVMOUSE_AXIS_MASK_TOUCH       (1 << QEMU_PVMOUSE_AXIS_TOUCH)
> > #define QEMU_PVMOUSE_AXIS_MASK_MULTITOUCH  (1 << QEMU_PVMOUSE_AXIS_MULTITOUCH)
> > #define QEMU_PVMOUSE_AXIS_MASK_POINT_ID    (1 << QEMU_PVMOUSE_AXIS_ID)
> 
> fwiw, the kernel already defines 14 tools, not that far off until you run
> out of tools on 32 bits. why not make the initial feature negotiation a
> stream of events instead of a mask?
> 
> Cheers,
>   Peter
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_AXIS_INFO, host -> guest
> >  * Send axis informations.
> >  */
> > typedef struct qemu_pvmouse_axis_info {
> >     uint32_t axis_id;
> >     uint32_t type;
> >     uint32_t flags;
> >     int32_t  min;
> >     int32_t  max;
> >     char     label[];
> > };
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_AXIS, host -> guest
> >  * Send mouse/pen/finger/wheel move events.
> >  */
> > typedef struct qemu_pvmouse_axis_event {
> >     uint32_t axis_id;
> >     int32_t  value;
> > };
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_BUTTON_INFO, host -> guest
> >  * Send button informations.
> >  */
> > typedef struct qemu_pvmouse_button_info {
> >     uint32_t button_id;
> >     char     label[];
> > };
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_BTN_{DOWN,UP}, host -> guest
> >  * Send button press+release events.
> >  */
> > typedef struct qemu_pvmouse_btn_event {
> >     uint32_t button_id;
> > };
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_POINT, host -> guest
> >  * marks the start of a point group for multitouch devices.
> >  */
> > typedef struct qemu_pvmouse_point {
> >     uint32_t id;
> > };
> > 
> > /*
> >  * QEMU_PVMOUSE_MSG_SYNC, host -> guest
> >  * Marks the end of a message group which belongs together
> >  * and carries the time stamp for all those events.
> >  *
> >  * The timestamp is specified in nanoseconds.  Timebase is undefined.
> >  * This is supposed to be used to figure how much time passed between
> >  * two events, to decide whenever two mouse clicks should be
> >  * interpreted as double click or not and simliar stuff.
> >  */
> > typedef struct qemu_pvmouse_sync {
> >     uint64_t timestamp;
> > };
> > 
> > 
> > typedef struct qemu_pvmouse_header {
> >     uint32_t size;            /* whole message size */
> >     uint32_t type;            /* qemu_pvmouse_type */
> > } qemu_pvmouse_header;
> > 
> > typedef union qemu_pvmouse_payload {
> >     qemu_pvmouse_init     init;
> >     qemu_pvmouse_ack      ack;
> >     /* ... */
> > };
> > 
> > typedef struct qemu_pvmouse_message {
> >     qemu_pvmouse_header  hdr;
> >     qemu_pvmouse_payload data;
> > } qemu_pvmouse_message;
> > 
> > #endif /* __QEMU_PVMOUSE__ */
>