Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori <anthony@codemonkey.ws>]

Ramesh Dharan wrote:
>> qemu-devel-bounces+nolan=nuovasystems.com@nongnu.org wrote:
>>     
>>> This extension is documented at
>>> http://tocm.wikidot.com/pointertypechange
>>>       
>> VMware has a very similar extension for their remote console.  I
>> believe
>> that Ramesh Dharan (whom I've CCed) at some point implemented it in one
>> or more open source clients.  Perhaps some coordination here is
>> possible?
>>     

Hi Ramesh,

Sorry for the delayed response.  Believe it or not, your note just made 
it to qemu-devel (along with a month or so of backlogged mail).

> Yeah I played implemented support for VMware's relative pointer extension, as
> well as our "grab" metaphor, and some other stuff, on top of the RealVNC
> 4.1.1 codebase. I unfortunately never got the patch cleaned up enough for
> submitting back to RealVNC however...
>
> I have a work-in-progress document that defines our extensions but it's not
> really ready for public consumption yet. I'm not sure of the context here,
> but I assume QEMU is using VNC as the wire protocol for displaying
> framebuffer contents remotely?

Yup.

>  Are you writing a new VNC client from scratch
> or building on one of the existing GPL'ed clients?
>   

At the moment, we work out of the box with existing clients (albeit with 
reduced functionality).  I have my own VNC client (gtk-vnc).  The VNC 
extensions are also supported by virt-manager.

As they mature, I'd like to get them into the main VNC clients too.

> The main limitations we've run into with using the VNC protocol for VM
> interaction are:
>
> (1) no support for keyboard scancodes (VNC uses higher level symbolic keys
> which aren't necessarily 1:1 mappable back to their scancode equivalents in
> all the scenarios we care about)
>   

Yes, right now, we do not address this.

> (2) no relative mouse support
>   

We have an extension that addresses this.

> (3) no alpha cursor support
>   

There is a pseudo-encoding that provides alpha cursors with a 1-bit 
alpha channel.  An 8 bit alpha channel would be nice of course.

> (4) no dynamic pixel depth/bpp change support
>   

This is something I do want to fix.  This isn't just a server-side 
issue, there is also a race in the protocol when a client uses 
SetPixelFormat.  A proper notification of when the server switches pixel 
format after a SetPixelFormat would solve this problem.

> (5) No notion of multiplexing displays on a single port. 
>   

Right.  ATM, I don't really plan on addressing this.

> (6) No client->server message negotiation.
>   

We actually can do this.  I've reserved the 255 client message ID which 
I'm using to multiplex various other messages.  I currently use this to 
implement a shared memory pseudo-encoding for VNC.  The idea is that on 
localhost, the server can render directly to a shared memory segment 
that the client also maps.  If the one is smart about how this shared 
memory segment is chosen, it can be used as an XShmImage.

> I implemented new client->server messages for the (1) and (2), and
> server->client pseudorectangle extensions for (3) and (4). We deal with (5)
> in a different way; our display multiplexing is handled at a higher level. We
> ignored (6) since our clients currently only talk to our servers.
>
> Looking at the linked site, it looks like you folks have already identified
> and are planning to deal with (2), possibly (5) (via some games with the
> display name?) and (6). I'd be particularly interested to hear more about
> plans for how to address (5), it would be great if off-the-shelf/3rd-party
> clients could talk to multiple VMs running on a host without needing to know
> and use a separate the port for each VM. 
>
> Our remote consoles/clients are heavily tied to our products and there aren't
> any real plans to divest them or make them more generally useful in the near
> future. However, that's likely to change someday, and anyway I'd be happy to
> provide feedback on, and implement support for these extensions so that e.g.
> a QEMU client could talk to a VMware instance, and in general get
> standardization to the point where it's possible to build a single remote
> client that could talk to the various VM implementations (QEMU, Xen, VMware,
> etc.). 
>   

I believe http://wiki.multimedia.cx/index.php?title=VMNC covers some of 
your extensions right?  Have you gotten the appropriate encodings 
reserved in the RFB spec?  I don't want to support any encodings that 
aren't properly registered so this would be an important first step.

I'm certainly happy to work toward a common set of VNC extensions to 
cover virtualization.  It's in everyones benefit to ensure that the 
largest number of clients Just Work out of the box.

Regards,

Anthony Liguori

> --
> Ramesh
>
>
>
>