input: mt: Software finger tracking in the kernel?

input: mt: Software finger tracking in the kernel?

[Henrik Rydberg]

Hi Dmitry,

there is an ongoing discussion about adding multitouch to X
(http://lists.x.org/archives/xorg-devel/2010-March/006206.html), which is
beginning to take on more solid form.

One of the suggestions emerging from that discussion is to add the software
finger tracking to the kernel. Back in summer 2009 when I thought about this, I
disregarded it as being too experimental. I have since then reconsidered,
starting to think it really is the right place.

The MT protocol allows applications to take advantage of multi-contact hardware,
but leaves the problems of finger tracking and filtering to the user. Arguably,
no application can make good use of MT without these, so the problem is pushed
forward, in this case to evdev or equivalent.

The knowledge of signal-to-noise ratios and prior input states resides in the
kernel. Because of this, the finger matching and filtering would naturally
reside within the kernel.

So, if there were to appear patches to include matching in the input core, would
you consider them? :-)

Cheers,
Henrik

Re: input: mt: Software finger tracking in the kernel?

[Trilok Soni]

Hi Henrik,

On Fri, Mar 19, 2010 at 4:28 PM, Henrik Rydberg <rydberg@enmesh.se> wrote:
> Hi Dmitry,
>
> there is an ongoing discussion about adding multitouch to X
> (http://lists.x.org/archives/xorg-devel/2010-March/006206.html), which is
> beginning to take on more solid form.
>
> One of the suggestions emerging from that discussion is to add the software
> finger tracking to the kernel. Back in summer 2009 when I thought about this, I
> disregarded it as being too experimental. I have since then reconsidered,
> starting to think it really is the right place.
>
> The MT protocol allows applications to take advantage of multi-contact hardware,
> but leaves the problems of finger tracking and filtering to the user. Arguably,
> no application can make good use of MT without these, so the problem is pushed
> forward, in this case to evdev or equivalent.
>
> The knowledge of signal-to-noise ratios and prior input states resides in the
> kernel. Because of this, the finger matching and filtering would naturally
> reside within the kernel.
>
> So, if there were to appear patches to include matching in the input core, would
> you consider them? :-)

Please post your patches for the further discussion. Last time I
referred the Android userspace code it was also not using TRACKING_ID
as defined in MT protocol even if the kernel driver emits them.


-- 
---Trilok Soni
http://triloksoni.wordpress.com
http://www.linkedin.com/in/triloksoni

Re: input: mt: Software finger tracking in the kernel?

[Ping Cheng]

On Fri, Mar 19, 2010 at 3:58 AM, Henrik Rydberg <rydberg@enmesh.se> wrote:
> Hi Dmitry,
>
> there is an ongoing discussion about adding multitouch to X
> (http://lists.x.org/archives/xorg-devel/2010-March/006206.html), which is
> beginning to take on more solid form.
>
> One of the suggestions emerging from that discussion is to add the software
> finger tracking to the kernel. Back in summer 2009 when I thought about this, I
> disregarded it as being too experimental. I have since then reconsidered,
> starting to think it really is the right place.
>
> The MT protocol allows applications to take advantage of multi-contact hardware,
> but leaves the problems of finger tracking and filtering to the user. Arguably,
> no application can make good use of MT without these, so the problem is pushed
> forward, in this case to evdev or equivalent.
>
> The knowledge of signal-to-noise ratios and prior input states resides in the
> kernel. Because of this, the finger matching and filtering would naturally
> reside within the kernel.
>
> So, if there were to appear patches to include matching in the input core, would
> you consider them? :-)

I support the idea to add a tracking ID and a filter to _MT_.  I'd
also like to make the filtering method more "device driver developer"
friendly, i.e., give the developer the option to turn the filter on or
off.  If you plan to use a constant for the filtering method, please
allow the developer to choose that constant too.

Another attribute associated with _MT_ is the pressure/capacity.
Report this value would offer more room for MT-aware application to
add new features.

Ping

Re: input: mt: Software finger tracking in the kernel?

[Trilok Soni]

Hi Ping,

On Sat, Mar 20, 2010 at 10:21 AM, Ping Cheng <pinglinux@gmail.com> wrote:
> On Fri, Mar 19, 2010 at 3:58 AM, Henrik Rydberg <rydberg@enmesh.se> wrote:
>> Hi Dmitry,
>>
>> there is an ongoing discussion about adding multitouch to X
>> (http://lists.x.org/archives/xorg-devel/2010-March/006206.html), which is
>> beginning to take on more solid form.
>>
>> One of the suggestions emerging from that discussion is to add the software
>> finger tracking to the kernel. Back in summer 2009 when I thought about this, I
>> disregarded it as being too experimental. I have since then reconsidered,
>> starting to think it really is the right place.
>>
>> The MT protocol allows applications to take advantage of multi-contact hardware,
>> but leaves the problems of finger tracking and filtering to the user. Arguably,
>> no application can make good use of MT without these, so the problem is pushed
>> forward, in this case to evdev or equivalent.
>>
>> The knowledge of signal-to-noise ratios and prior input states resides in the
>> kernel. Because of this, the finger matching and filtering would naturally
>> reside within the kernel.
>>
>> So, if there were to appear patches to include matching in the input core, would
>> you consider them? :-)
>
> I support the idea to add a tracking ID and a filter to _MT_.  I'd
> also like to make the filtering method more "device driver developer"
> friendly, i.e., give the developer the option to turn the filter on or
> off.  If you plan to use a constant for the filtering method, please
> allow the developer to choose that constant too.
>
> Another attribute associated with _MT_ is the pressure/capacity.
> Report this value would offer more room for MT-aware application to
> add new features.

We have ABS_MT_PRESSURE added recently. I hope you are asking the same.


-- 
---Trilok Soni
http://triloksoni.wordpress.com
http://www.linkedin.com/in/triloksoni
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: input: mt: Software finger tracking in the kernel?

[Ping Cheng]

On Sat, Mar 20, 2010 at 12:27 PM, Trilok Soni <soni.trilok@gmail.com> wrote:
>> Another attribute associated with _MT_ is the pressure/capacity.
>> Report this value would offer more room for MT-aware application to
>> add new features.
>
> We have ABS_MT_PRESSURE added recently. I hope you are asking the same.

Great. ABS_MT_PRESSURE is what I meant. I must have looked at an old doc/code.

Ping

Re: input: mt: Software finger tracking in the kernel?

[Dmitry Torokhov]

Hi Henrik,

On Fri, Mar 19, 2010 at 11:58:35AM +0100, Henrik Rydberg wrote:
> Hi Dmitry,
> 
> there is an ongoing discussion about adding multitouch to X
> (http://lists.x.org/archives/xorg-devel/2010-March/006206.html), which is
> beginning to take on more solid form.
> 
> One of the suggestions emerging from that discussion is to add the software
> finger tracking to the kernel. Back in summer 2009 when I thought about this, I
> disregarded it as being too experimental. I have since then reconsidered,
> starting to think it really is the right place.
> 
> The MT protocol allows applications to take advantage of multi-contact hardware,
> but leaves the problems of finger tracking and filtering to the user. Arguably,
> no application can make good use of MT without these, so the problem is pushed
> forward, in this case to evdev or equivalent.
> 
> The knowledge of signal-to-noise ratios and prior input states resides in the
> kernel. Because of this, the finger matching and filtering would naturally
> reside within the kernel.
> 
> So, if there were to appear patches to include matching in the input core, would
> you consider them? :-)
> 

I am not sure if input core itself is the proper place to do such
thing, I'd envisioned something more like a library providing common
code that drivers could opt in to use, like we hane ff-memless for
memory-less force-feedback devices.

Does it make any sense? I guess post the skeleton of the code and we can
discuss further.

-- 
Dmitry

Re: input: mt: Software finger tracking in the kernel?

[Henrik Rydberg]

Dmitry Torokhov wrote:
> Hi Henrik
[...]
> I am not sure if input core itself is the proper place to do such
> thing, I'd envisioned something more like a library providing common
> code that drivers could opt in to use, like we hane ff-memless for
> memory-less force-feedback devices.
> 
> Does it make any sense? I guess post the skeleton of the code and we can
> discuss further.

Yes, input core as in input*.c might be wrong. The big thing is that the events
most likely have to be deferred to the bottom half. Otherwise, the scheme would
have fit rather nicely on top of the current filtering in input_device. The
ideas I am currently looking at are:

1. Expand on the new event filtering mechanism, providing a sort of rewrite
functionality which can schedule events for later injection into the event stream.

2. Add additional logic to evdev so that it buffers MT events and flushes the
reworked events directly onto the clients.

3. Add a new multitouch handler, mtdev, which by default does event deferral,
only emitting events by scheduling them upon SYN_REPORT requests.

To my untrained eye, all three options could be made to work with acceptable
latency. Number three probably means applications (read X) have to keep open
both evdev and mtdev, which might be confusing. Number one is quite general, but
probably contains hidden (to me) difficulties. Number two sounds pretty
straight-forward, although tapping into the code somewhat through the back door.

Henrik

Re: input: mt: Software finger tracking in the kernel?

[Henrik Rydberg]

Dmitry Torokhov wrote:
> [...] I'd envisioned something more like a library providing common
> code that drivers could opt in to use, like we hane ff-memless for
> memory-less force-feedback devices.

As long as one can queue work with such a method, it looks like a good idea.

Henrik

Re: input: mt: Software finger tracking in the kernel?

[Rafi Rubin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> I am not sure if input core itself is the proper place to do such
> thing, I'd envisioned something more like a library providing common
> code that drivers could opt in to use, like we hane ff-memless for
> memory-less force-feedback devices.
> 
> Does it make any sense? I guess post the skeleton of the code and we can
> discuss further.

Regardless of where the code goes, I suggest we enable tracking by default for
MT devices and perhaps have a quirt MT_NOTRACK to opt out.

If tracking is the common case and expected down stream, it would be nice to let
HID compliant devices work without needing to explicitly identify them in code.

Rafi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkuljTwACgkQwuRiAT9o60/iBwCgudrOvmam/A0VQ2uA0UtK5R3H
l3oAnAgEV7D+FFTWjofm85Djp25QhWq1
=2+sX
-----END PGP SIGNATURE-----