Passing multi-screen layout to KMS driver

Passing multi-screen layout to KMS driver

[Michael Thayer]

Hello,

I'm afraid I'm not subscribed to the list, so I would greatly appreciate 
a CC on replies (if any).  I have been fiddling with our 
(not-upstreamed) VirtualBox guest drm driver to make it work better with 
the Wayland version of GNOME Shell in Fedora 25.

One problem that I have run into (from my testing it seems to affect 
Qemu too) is that we use an emulated graphics tablet device to make the 
guest pointer position match the host one.  GNOME Shell nicely maps the 
range of the device to the full virtual desktop, as does X.Org.  Unlike 
X.Org though, GNOME Shell/Wayland does not create a single framebuffer 
in video RAM for the whole desktop, where we could deduce the relative 
layout of the screens by looking at the offset of their crtcs in that 
framebuffer.

So I would be interested to know whether anyone else has thought about 
this problem, and possibly even about an interface to let the compositor 
pass the information.  If not, would people be open to the idea?  I 
would much rather have something generally agreed on than hack something up.

Thanks.

Regards
Michael
-- 
Michael Thayer | VirtualBox engineer
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | D-71384 Weinstadt

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister 
der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Passing multi-screen layout to KMS driver

[Gerd Hoffmann]

  Hi,

> So I would be interested to know whether anyone else has thought about 
> this problem, and possibly even about an interface to let the compositor 
> pass the information.  If not, would people be open to the idea?  I 
> would much rather have something generally agreed on than hack something up.

I think the best way to tackle this is to have multiple tablets, one per
display device (touchscreen-style setup).

qemu can do that, with input routing (must configure on the host which
display belongs to which tablet).  Here is some info on that:

http://git.qemu-project.org/?p=qemu.git;a=blob;f=docs/multiseat.txt;hb=HEAD

The setup on the guest side is completely manual.  You have to use
"xinput --map-to-output" to tell Xorg which tablet belongs to which
display.  Maybe it is also possible to stick that into xorg.conf.
Should be improved, and it surely makes sense that qemu and virtualbox
use the same approach here.

Not sure if and how this works automatically with physical touchscreens.
Any clues are welcome.

Another option is to use a guest agent, spice does that since years to
handle multihead.  The guest agent queries the display layout using
xrandr, gets x + y + displayid from the spice client and generates
pointer events from that.  But I expect that scheme breaks with wayland
because wayland is by design alot more restrictive, so spice-agent
probably isn't allowed to send pointer events.  So not really an option
these days ...

cheers,
  Gerd

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Passing multi-screen layout to KMS driver

[Michael Thayer]

Hello Gerd,

14.12.2016 11:11, Gerd Hoffmann wrote:
>> So I would be interested to know whether anyone else has thought about
>> this problem, and possibly even about an interface to let the compositor
>> pass the information.  If not, would people be open to the idea?  I
>> would much rather have something generally agreed on than hack something up.
>
> I think the best way to tackle this is to have multiple tablets, one per
> display device (touchscreen-style setup).
>
> qemu can do that, with input routing (must configure on the host which
> display belongs to which tablet).  Here is some info on that:
>
> http://git.qemu-project.org/?p=qemu.git;a=blob;f=docs/multiseat.txt;hb=HEAD
>
> The setup on the guest side is completely manual.  You have to use
> "xinput --map-to-output" to tell Xorg which tablet belongs to which
> display.  Maybe it is also possible to stick that into xorg.conf.
> Should be improved, and it surely makes sense that qemu and virtualbox
> use the same approach here.
>
> Not sure if and how this works automatically with physical touchscreens.
> Any clues are welcome.

Thanks for the answer.  That was the direction I was initially expecting 
to go too.  In theory libinput lets you map input devices to heads[1] 
using a udev property[2], though I have yet to test whether anyone 
supports that yet (couldn't find it in the Mutter/GNOME Shell source, 
but I'm not familiar with it).  As the API description says, the default 
without that property is still to map the input device to all screens 
like X.Org on Linux does.

[1] 
https://cgit.freedesktop.org/wayland/libinput/tree/src/udev-seat.c?id=1.5.3#n97
[2] 
https://wayland.freedesktop.org/libinput/doc/0.99.1/group__device.html#gaf48626f6190e9c9bc14abb704e66cc22


> Another option is to use a guest agent, spice does that since years to
> handle multihead.  The guest agent queries the display layout using
> xrandr, gets x + y + displayid from the spice client and generates
> pointer events from that.  But I expect that scheme breaks with wayland
> because wayland is by design alot more restrictive, so spice-agent
> probably isn't allowed to send pointer events.  So not really an option
> these days ...

We actually do something similar in Windows guests: older versions 
provided the layout information to the driver directly, but at least 
Windows 10 does not, so we query it with a user-space agent which passes 
it to the driver.  (We send our pointer events from a driver, in all 
supported guest types, which works fine with Wayland too.)  It would be 
nice though as I said if the compositor (or whoever is controlling the 
display) could just provide the layout information to the driver itself. 
  We already have "suggested X" and "suggested Y" for the other 
direction, and for now I have solved it by always providing "suggested 
X" and "suggested Y" hints in the driver.  That which works well enough 
in a first approximation - if the user changes the layout inside the 
virtual machine the mapping breaks, and as soon as they change it 
outside it mends again.  So my idea was to try to have people agree on 
on interface for that.

Regards
Michael

> cheers,
>   Gerd
-- 
Michael Thayer | VirtualBox engineer
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | D-71384 Weinstadt

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister 
der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: Passing multi-screen layout to KMS driver

[Michael Thayer]

Adding Dave on CC here as the original creator of the suggested X and Y 
properties.

15.12.2016 15:03, Michael Thayer wrote:
> 14.12.2016 11:11, Gerd Hoffmann wrote:

[Summary of cut text: VirtualBox and Qemu use emulated graphics tablets 
to pass the host pointer through to the guest.  X.Org and GNOME 
Shell/Wayland map the tablet range to cover the whole virtual desktop, 
but GNOME Shell/Wayland does not provide any way for the host to 
discover the layout of that desktop in order to map positions in host 
windows to positions on guest screens.]

>>> So I would be interested to know whether anyone else has thought about
>>> this problem, and possibly even about an interface to let the compositor
>>> pass the information.  If not, would people be open to the idea?  I
>>> would much rather have something generally agreed on than hack
>>> something up.
[...]
> [...]  It would be
> nice though as I said if the compositor (or whoever is controlling the
> display) could just provide the layout information to the driver itself.
>  We already have "suggested X" and "suggested Y" for the other
> direction, and for now I have solved it by always providing "suggested
> X" and "suggested Y" hints in the driver.

My idea is to submit a patch which removes the DRM_MODE_PROP_IMMUTABLE 
flag from the suggested X and Y connector properties (with appropriate 
documentation), so that the compositor can use them to tell the driver 
about the layout.  It would be greatly appreciated if someone could tell 
me if that is likely to be acceptable.

Regards,

Michael
-- 
Michael Thayer | VirtualBox engineer
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | D-71384 Weinstadt

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister 
der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel