RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Nolan Leake]

qemu-devel-bounces+nolan=nuovasystems.com@nongnu.org wrote:
> This extension is documented at
> http://tocm.wikidot.com/pointertypechange

VMware has a very similar extension for their remote console.  I believe
that Ramesh Dharan (whom I've CCed) at some point implemented it in one
or more open source clients.  Perhaps some coordination here is
possible?

- Nolan

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Ramesh Dharan]

> qemu-devel-bounces+nolan=nuovasystems.com@nongnu.org wrote:
> > This extension is documented at
> > http://tocm.wikidot.com/pointertypechange
> 
> VMware has a very similar extension for their remote console.  I
> believe
> that Ramesh Dharan (whom I've CCed) at some point implemented it in one
> or more open source clients.  Perhaps some coordination here is
> possible?

Yeah I played implemented support for VMware's relative pointer extension, as
well as our "grab" metaphor, and some other stuff, on top of the RealVNC
4.1.1 codebase. I unfortunately never got the patch cleaned up enough for
submitting back to RealVNC however...

I have a work-in-progress document that defines our extensions but it's not
really ready for public consumption yet. I'm not sure of the context here,
but I assume QEMU is using VNC as the wire protocol for displaying
framebuffer contents remotely? Are you writing a new VNC client from scratch
or building on one of the existing GPL'ed clients?

The main limitations we've run into with using the VNC protocol for VM
interaction are:

(1) no support for keyboard scancodes (VNC uses higher level symbolic keys
which aren't necessarily 1:1 mappable back to their scancode equivalents in
all the scenarios we care about)
(2) no relative mouse support
(3) no alpha cursor support
(4) no dynamic pixel depth/bpp change support
(5) No notion of multiplexing displays on a single port. 
(6) No client->server message negotiation.

I implemented new client->server messages for the (1) and (2), and
server->client pseudorectangle extensions for (3) and (4). We deal with (5)
in a different way; our display multiplexing is handled at a higher level. We
ignored (6) since our clients currently only talk to our servers.

Looking at the linked site, it looks like you folks have already identified
and are planning to deal with (2), possibly (5) (via some games with the
display name?) and (6). I'd be particularly interested to hear more about
plans for how to address (5), it would be great if off-the-shelf/3rd-party
clients could talk to multiple VMs running on a host without needing to know
and use a separate the port for each VM. 

Our remote consoles/clients are heavily tied to our products and there aren't
any real plans to divest them or make them more generally useful in the near
future. However, that's likely to change someday, and anyway I'd be happy to
provide feedback on, and implement support for these extensions so that e.g.
a QEMU client could talk to a VMware instance, and in general get
standardization to the point where it's possible to build a single remote
client that could talk to the various VM implementations (QEMU, Xen, VMware,
etc.). 

--
Ramesh

Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

Ramesh Dharan wrote:
>> qemu-devel-bounces+nolan=nuovasystems.com@nongnu.org wrote:
>>     
>>> This extension is documented at
>>> http://tocm.wikidot.com/pointertypechange
>>>       
>> VMware has a very similar extension for their remote console.  I
>> believe
>> that Ramesh Dharan (whom I've CCed) at some point implemented it in one
>> or more open source clients.  Perhaps some coordination here is
>> possible?
>>     

Hi Ramesh,

Sorry for the delayed response.  Believe it or not, your note just made 
it to qemu-devel (along with a month or so of backlogged mail).

> Yeah I played implemented support for VMware's relative pointer extension, as
> well as our "grab" metaphor, and some other stuff, on top of the RealVNC
> 4.1.1 codebase. I unfortunately never got the patch cleaned up enough for
> submitting back to RealVNC however...
>
> I have a work-in-progress document that defines our extensions but it's not
> really ready for public consumption yet. I'm not sure of the context here,
> but I assume QEMU is using VNC as the wire protocol for displaying
> framebuffer contents remotely?

Yup.

>  Are you writing a new VNC client from scratch
> or building on one of the existing GPL'ed clients?
>   

At the moment, we work out of the box with existing clients (albeit with 
reduced functionality).  I have my own VNC client (gtk-vnc).  The VNC 
extensions are also supported by virt-manager.

As they mature, I'd like to get them into the main VNC clients too.

> The main limitations we've run into with using the VNC protocol for VM
> interaction are:
>
> (1) no support for keyboard scancodes (VNC uses higher level symbolic keys
> which aren't necessarily 1:1 mappable back to their scancode equivalents in
> all the scenarios we care about)
>   

Yes, right now, we do not address this.

> (2) no relative mouse support
>   

We have an extension that addresses this.

> (3) no alpha cursor support
>   

There is a pseudo-encoding that provides alpha cursors with a 1-bit 
alpha channel.  An 8 bit alpha channel would be nice of course.

> (4) no dynamic pixel depth/bpp change support
>   

This is something I do want to fix.  This isn't just a server-side 
issue, there is also a race in the protocol when a client uses 
SetPixelFormat.  A proper notification of when the server switches pixel 
format after a SetPixelFormat would solve this problem.

> (5) No notion of multiplexing displays on a single port. 
>   

Right.  ATM, I don't really plan on addressing this.

> (6) No client->server message negotiation.
>   

We actually can do this.  I've reserved the 255 client message ID which 
I'm using to multiplex various other messages.  I currently use this to 
implement a shared memory pseudo-encoding for VNC.  The idea is that on 
localhost, the server can render directly to a shared memory segment 
that the client also maps.  If the one is smart about how this shared 
memory segment is chosen, it can be used as an XShmImage.

> I implemented new client->server messages for the (1) and (2), and
> server->client pseudorectangle extensions for (3) and (4). We deal with (5)
> in a different way; our display multiplexing is handled at a higher level. We
> ignored (6) since our clients currently only talk to our servers.
>
> Looking at the linked site, it looks like you folks have already identified
> and are planning to deal with (2), possibly (5) (via some games with the
> display name?) and (6). I'd be particularly interested to hear more about
> plans for how to address (5), it would be great if off-the-shelf/3rd-party
> clients could talk to multiple VMs running on a host without needing to know
> and use a separate the port for each VM. 
>
> Our remote consoles/clients are heavily tied to our products and there aren't
> any real plans to divest them or make them more generally useful in the near
> future. However, that's likely to change someday, and anyway I'd be happy to
> provide feedback on, and implement support for these extensions so that e.g.
> a QEMU client could talk to a VMware instance, and in general get
> standardization to the point where it's possible to build a single remote
> client that could talk to the various VM implementations (QEMU, Xen, VMware,
> etc.). 
>   

I believe http://wiki.multimedia.cx/index.php?title=VMNC covers some of 
your extensions right?  Have you gotten the appropriate encodings 
reserved in the RFB spec?  I don't want to support any encodings that 
aren't properly registered so this would be an important first step.

I'm certainly happy to work toward a common set of VNC extensions to 
cover virtualization.  It's in everyones benefit to ensure that the 
largest number of clients Just Work out of the box.

Regards,

Anthony Liguori

> --
> Ramesh
>
>
>
>   

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Ramesh Dharan]

Hi Anthony, sorry for the delayed response to your delayed response.

I actually haven't been working much on the remote display stuff in a while,
though it's something I'm hoping to get involved with again time permitting.
I'm cc'ing Dustin Byford and Johnson Liu, VMware engineers who handle a lot
of our remote display work these days. 

Also, I'm wondering if it would make sense to set up a separate list so we
don't spam qemu-devel with this discussion, and also so that we can all
subscribe to it rather than having to be cc'ed and so forth. If you think
that's a good idea let me know. We can host the list @vmware.com, or put it
somewhere else (nongnu.org? or elsewhere?) if that would be preferred. 

> > (1) no support for keyboard scancodes (VNC uses higher 
> level symbolic keys
> > which aren't necessarily 1:1 mappable back to their 
> scancode equivalents in
> > all the scenarios we care about)
> 
> Yes, right now, we do not address this.

So how do you handle the keyboard today? Do you just hardcode the reverse
mapping back to a pc105 keyboard? 

It would be nice to get something upstream so that all clients have the
ability to send scancode values. We have an extension in place which we can
just document. We use 16-bit "virtual scan code values" (they're 16-bit more
for convenience of implementation, i.e. directly mapped to a lookup table). I
believe the values we actually are equivalent/identical to PC scancodes. 

Meanwhile, in our newest version of Workstation (6.0) we added server-side
support for emulating different keyboard layouts when standard VNC clients
connect. Basically you can set a server config option to tell the server
"pretend that a de105 keyboard is attached" and it will do the reverse
translation of KeySym to Scancode accordingly. This is good enough for some
people, but the ideal situation is obviously to do the translation on the
client-side where you can actually detect the local keyboard and do the right
thing (which is what the VMware remote client does). 

> > (2) no relative mouse support
> >   
> 
> We have an extension that addresses this.

As do we ;-). 

> > (3) no alpha cursor support
> >   
> 
> There is a pseudo-encoding that provides alpha cursors with a 1-bit 
> alpha channel.  An 8 bit alpha channel would be nice of course.

More than nice, it's pretty much required if you want to draw some modern
Linux guest cursors correctly. The cheap hack of treating < 255 alpha as
fully transparent doesn't really work that well, especially for things an
I-Bar cursor with shadow. 

We do support the standard VNC pseudo-cursor encoding as of Workstation 5.5
and ESX 3.0, and if the guest sets an alpha cursor and the client doesn't
support our alpha cursor extension, we strip out the alpha channel and send
it using the standard VNC encoding. But it doesn't look that great. 

> > (4) no dynamic pixel depth/bpp change support
> >   
> 
> This is something I do want to fix.  This isn't just a server-side 
> issue, there is also a race in the protocol when a client uses 
> SetPixelFormat.  A proper notification of when the server 
> switches pixel 
> format after a SetPixelFormat would solve this problem.

So yeah in the original protocol, depending on how you read it, it's just
illegal to ever send a SetPixelFormat after the original request. But at any
rate, client->server format requests aren't as interesting as server->client
requests. You only do the former if the client decides for some reason that
it wants a different pixel format, because e.g. it decides it doesn't have
enough bandwidth or something. 

Server->client changes OTOH, can be guest driven, e.g. there are several mode
changes when booting a Windows guest. When a standard VNC client is
connected, we hide these from the client by doing pixel format conversion on
the server, which is expensive, and potentially wastes bandwidth on the wire
as well (e.g. when the guest is in 640x480x256 but the client wants 32bpp).
When a client which supports our "VMModeChange" extension is connected, we
just send down the mode change at the start of the next UpdateRequest. 

We do implement the standard VNC desktop size encoding (since otherwise we'd
have to just drop the connection every time the guest changed display size). 

But getting back to your original point, since our client never sends mode
change requests upstream, so we haven't addressed the race that you describe.
I agree that if you ever want to drive these on the client you need to do
something about that. 

> > (5) No notion of multiplexing displays on a single port. 
> >   
> 
> Right.  ATM, I don't really plan on addressing this.

So, the old Connectix Virtual Server product (which was in beta when
Microsoft snapped them up) had a super slick way of dealing with this, that
we actually joked about doing but never seriously thought anyone would do.
They rendered an entire "picker" UI using VNC primitives. The UI would show
live thumbnails of each VM, and then when you click on one of the tiles,
you'd get redirected to that console. You couldn't go back out to the picker,
but still, it was pretty neat. 

I've since looked into doing the same thing, but the closest thing to any VNC
UI/widget rendering library that I've found is TclRFB (which, as the name
would imply, is written in Tcl). 

Anyway, what you really want is a way to say "you've connected to a
multi-home server, here's a list of VMs available (with arbitrary or perhaps
structured metadata attached to each VM tag), which one do you want?". And
then have the ability for the client ot pick. But you can't really do this
sanely with just extensions, I think this is a good candidate for actually
extending the base protocol.

> We actually can do this.  I've reserved the 255 client 
> message ID which 
> I'm using to multiplex various other messages.  

Yeah, we did this too. We use '127' as the client message ID, but it's
otherwise exactly as you describe. It's a nice performance boost, though it
makes implementing CopyRect encoding much trickier <g>.

> I currently use this to implement a shared memory pseudo-encoding for VNC.

> The idea is that on localhost, the server can render directly to a shared 
> memory segment that the client also maps.  If the one is smart about how 
> this shared memory segment is chosen, it can be used as an XShmImage.

Yeah we did the exact same thing, with a slight twist. We get the server to
allocate the SHM segment, and the client just attaches to it. If you do it
this way, you don't need a new client->server message, the server just sends
the client the shm id if the client supports the "SharedFB" extension (that's
what we named it). 

There's some complexity involved with how the server can actually figure out
whether the client is on the same machine (e.g. if the client is tunneling
over SSH, this is not easy to do). So in fact what we do is just if the
client supports SharedFBInfo and SharedFBRect, the server just sends along
the messages. The client then tries to map the segment described in
SharedFBInfo. If it fails, it just sends a new encoding set with those
encodings removed, and the server starts sending real pixel data instead. 

Yes, this means that we always try and fail sharedfb so there's a couple
extra roundtrips before remote clients actually get real pixel data. This was
deemed acceptable (by me ;-) for our use cases since most networks are fast
enough. But if this is too hackish for your taste we could bake in something
better. 

> I believe http://wiki.multimedia.cx/index.php?title=VMNC 
> covers some of 
> your extensions right?  Have you gotten the appropriate encodings 
> reserved in the RFB spec?  I don't want to support any encodings that 
> aren't properly registered so this would be an important first step.

So once upon a time, we tried to register encodings, but we didn't get very
far. It sounds like perhaps the process has improved since then. I will try
to get the ball rolling on this again. 

> I'm certainly happy to work toward a common set of VNC extensions to 
> cover virtualization.  It's in everyones benefit to ensure that the 
> largest number of clients Just Work out of the box.

Agreed. I look forward to hearing more of your thoughts on this subject.

--
Ramesh

Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

Ramesh Dharan wrote:
> Hi Anthony, sorry for the delayed response to your delayed response.
>   

No problem.

> I actually haven't been working much on the remote display stuff in a while,
> though it's something I'm hoping to get involved with again time permitting.
> I'm cc'ing Dustin Byford and Johnson Liu, VMware engineers who handle a lot
> of our remote display work these days. 
>
> Also, I'm wondering if it would make sense to set up a separate list so we
> don't spam qemu-devel with this discussion, and also so that we can all
> subscribe to it rather than having to be cc'ed and so forth. If you think
> that's a good idea let me know. We can host the list @vmware.com, or put it
> somewhere else (nongnu.org? or elsewhere?) if that would be preferred. 
>   

I'll send you a note and we can figure out the best place to host it but 
I do think it's a good idea.

>>> all the scenarios we care about)
>>>       
>> Yes, right now, we do not address this.
>>     
>
> So how do you handle the keyboard today? Do you just hardcode the reverse
> mapping back to a pc105 keyboard? 
>   

We can take a keymap argument which can be used to map to various types 
of keyboards (defaulting to a standard pc105 keyboard).

> It would be nice to get something upstream so that all clients have the
> ability to send scancode values. We have an extension in place which we can
> just document. We use 16-bit "virtual scan code values" (they're 16-bit more
> for convenience of implementation, i.e. directly mapped to a lookup table). I
> believe the values we actually are equivalent/identical to PC scancodes. 
>   

Yeah, that would work out pretty well.

> Meanwhile, in our newest version of Workstation (6.0) we added server-side
> support for emulating different keyboard layouts when standard VNC clients
> connect. Basically you can set a server config option to tell the server
> "pretend that a de105 keyboard is attached" and it will do the reverse
> translation of KeySym to Scancode accordingly. This is good enough for some
> people, but the ideal situation is obviously to do the translation on the
> client-side where you can actually detect the local keyboard and do the right
> thing (which is what the VMware remote client does). 
>   

Yes, this would be much nicer.

  

>> There is a pseudo-encoding that provides alpha cursors with a 1-bit 
>> alpha channel.  An 8 bit alpha channel would be nice of course.
>>     
>
> More than nice, it's pretty much required if you want to draw some modern
> Linux guest cursors correctly. The cheap hack of treating < 255 alpha as
> fully transparent doesn't really work that well, especially for things an
> I-Bar cursor with shadow. 
>   

Right, I've seen this in action and it's not very pretty.

> We do support the standard VNC pseudo-cursor encoding as of Workstation 5.5
> and ESX 3.0, and if the guest sets an alpha cursor and the client doesn't
> support our alpha cursor extension, we strip out the alpha channel and send
> it using the standard VNC encoding. But it doesn't look that great. 
>
>   
>>> (4) no dynamic pixel depth/bpp change support
>>>   
>>>       
>> This is something I do want to fix.  This isn't just a server-side 
>> issue, there is also a race in the protocol when a client uses 
>> SetPixelFormat.  A proper notification of when the server 
>> switches pixel 
>> format after a SetPixelFormat would solve this problem.
>>     
>
> So yeah in the original protocol, depending on how you read it, it's just
> illegal to ever send a SetPixelFormat after the original request. But at any
> rate, client->server format requests aren't as interesting as server->client
> requests. You only do the former if the client decides for some reason that
> it wants a different pixel format, because e.g. it decides it doesn't have
> enough bandwidth or something. 
>
> Server->client changes OTOH, can be guest driven, e.g. there are several mode
> changes when booting a Windows guest. When a standard VNC client is
> connected, we hide these from the client by doing pixel format conversion on
> the server, which is expensive, and potentially wastes bandwidth on the wire
> as well (e.g. when the guest is in 640x480x256 but the client wants 32bpp).
> When a client which supports our "VMModeChange" extension is connected, we
> just send down the mode change at the start of the next UpdateRequest. 
>   

Right now, we always do pixel conversion which is pretty annoying.  It 
would be very nice to be able to tell the client to deal with the 
conversion.

> We do implement the standard VNC desktop size encoding (since otherwise we'd
> have to just drop the connection every time the guest changed display size). 
>
> But getting back to your original point, since our client never sends mode
> change requests upstream, so we haven't addressed the race that you describe.
> I agree that if you ever want to drive these on the client you need to do
> something about that. 
>   

Right, we're particularly prone to this race since our VNC server is 
asynchronous.  A SetServerPixelFormat message would solve this problem 
since it would let a client know when it should start decoding pixels 
differently.

In practice, most VNC servers do not support arbitrary pixel formats too 
so it makes a lot of sense for server's to have this message available 
to them.

>>> (5) No notion of multiplexing displays on a single port. 
>>>   
>>>       
>> Right.  ATM, I don't really plan on addressing this.
>>     
>
> So, the old Connectix Virtual Server product (which was in beta when
> Microsoft snapped them up) had a super slick way of dealing with this, that
> we actually joked about doing but never seriously thought anyone would do.
> They rendered an entire "picker" UI using VNC primitives. The UI would show
> live thumbnails of each VM, and then when you click on one of the tiles,
> you'd get redirected to that console. You couldn't go back out to the picker,
> but still, it was pretty neat. 
>
> I've since looked into doing the same thing, but the closest thing to any VNC
> UI/widget rendering library that I've found is TclRFB (which, as the name
> would imply, is written in Tcl). 
>
> Anyway, what you really want is a way to say "you've connected to a
> multi-home server, here's a list of VMs available (with arbitrary or perhaps
> structured metadata attached to each VM tag), which one do you want?". And
> then have the ability for the client ot pick. But you can't really do this
> sanely with just extensions, I think this is a good candidate for actually
> extending the base protocol.
>   

I have seen KVM switches that use VNC implement an extension that is 
basically, SetName.  They then use a client encoding to switch the KVM.  
This is definitely an interesting area to pursue.

>> The idea is that on localhost, the server can render directly to a shared 
>> memory segment that the client also maps.  If the one is smart about how 
>> this shared memory segment is chosen, it can be used as an XShmImage.
>>     
>
> Yeah we did the exact same thing, with a slight twist. We get the server to
> allocate the SHM segment, and the client just attaches to it. If you do it
> this way, you don't need a new client->server message, the server just sends
> the client the shm id if the client supports the "SharedFB" extension (that's
> what we named it).
>   

I chose to have the client allocate the memory since it knew the X 
server's line size which is required to allocate an XShmImage.  My first 
implementation had the server allocate the buffer but required the 
client to tell the server what the line size should be.

> There's some complexity involved with how the server can actually figure out
> whether the client is on the same machine (e.g. if the client is tunneling
> over SSH, this is not easy to do). So in fact what we do is just if the
> client supports SharedFBInfo and SharedFBRect, the server just sends along
> the messages. The client then tries to map the segment described in
> SharedFBInfo. If it fails, it just sends a new encoding set with those
> encodings removed, and the server starts sending real pixel data instead. 
>   

I've thought about this problem too.  My fear is that one could 
accidentally map a shared memory area that's actually valid.

> Yes, this means that we always try and fail sharedfb so there's a couple
> extra roundtrips before remote clients actually get real pixel data. This was
> deemed acceptable (by me ;-) for our use cases since most networks are fast
> enough. But if this is too hackish for your taste we could bake in something
> better. 
>   

I really haven't thought of a better solution myself.  That doesn't mean 
one exists though.

>> I believe http://wiki.multimedia.cx/index.php?title=VMNC 
>> covers some of 
>> your extensions right?  Have you gotten the appropriate encodings 
>> reserved in the RFB spec?  I don't want to support any encodings that 
>> aren't properly registered so this would be an important first step.
>>     
>
> So once upon a time, we tried to register encodings, but we didn't get very
> far. It sounds like perhaps the process has improved since then. I will try
> to get the ball rolling on this again. 
>   

I can help you with that if you want.  It wasn't a painful process at 
all for me.

>> I'm certainly happy to work toward a common set of VNC extensions to 
>> cover virtualization.  It's in everyones benefit to ensure that the 
>> largest number of clients Just Work out of the box.
>>     
>
> Agreed. I look forward to hearing more of your thoughts on this subject.
>   

Same here!  Thanks for the response.

Regards,

Anthony Liguori

> --
> Ramesh
>
>   

Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

Ramesh Dharan wrote:
> I implemented new client->server messages for the (1) and (2), and
>   

I should have read more carefully.  This means that you're not a 
compliant RFB server ATM.

I guess that means there isn't a point registering the pseudo-encodings 
you are currently using since you have to change the server/client 
anyway.  It's easier anyway since pseudo-encodings are supposed to be 
negative numbers and you're using positive numbers.

I'm happy to start looking at reimplementing your encodings using my 
pseudo-encoding range.  It should be real easy for ya'll to support 
these too.  I'll spend a little time in the next week enumerating these 
extensions and assigning proper values to them.

Do you have any documentation on your currently client messages?  Those 
aren't documented as part of the FFMPEG project.

Regards,

Anthony Liguori

> server->client pseudorectangle extensions for (3) and (4). We deal with (5)
> in a different way; our display multiplexing is handled at a higher level. We
> ignored (6) since our clients currently only talk to our servers.
>
> Looking at the linked site, it looks like you folks have already identified
> and are planning to deal with (2), possibly (5) (via some games with the
> display name?) and (6). I'd be particularly interested to hear more about
> plans for how to address (5), it would be great if off-the-shelf/3rd-party
> clients could talk to multiple VMs running on a host without needing to know
> and use a separate the port for each VM. 
>
> Our remote consoles/clients are heavily tied to our products and there aren't
> any real plans to divest them or make them more generally useful in the near
> future. However, that's likely to change someday, and anyway I'd be happy to
> provide feedback on, and implement support for these extensions so that e.g.
> a QEMU client could talk to a VMware instance, and in general get
> standardization to the point where it's possible to build a single remote
> client that could talk to the various VM implementations (QEMU, Xen, VMware,
> etc.). 
>
> --
> Ramesh
>
>
>
>   

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Ramesh Dharan]

Anthony, I have a detailed response to your earlier e-mail but I wanted to
handle this discussion separately. 

> > I implemented new client->server messages for the (1) and (2), and
> 
> I should have read more carefully.  This means that you're not a 
> compliant RFB server ATM.

I'm not sure I follow you. We support standard VNC clients, and we don't
violate the protocol expectations of any standard VNC clients who talk to us.


However, if a client *happens* to send us these new client->server messages
(which don't exist in the standard spec), then we support them. 

We never send a message to a client that the client doesn't know how to
handle. 

> I guess that means there isn't a point registering the 
> pseudo-encodings 
> you are currently using since you have to change the server/client 
> anyway.  It's easier anyway since pseudo-encodings are supposed to be 
> negative numbers and you're using positive numbers.

No, there's still a point. The display path and input path are essentially
independent. Better input handling requires extending the client->server path
(so the client can send new kinds of data other than the standard VNC
keyboard and pointer events). Unfortunately, extension of the protocol in
this direction was not baked into the original design. 

I think that basically we should actually extend the RFB protocol itself to
just have a server->client message, SetSupportedClientMessages, which works
exactly the way SetEncodings works today, i.e. the server can send down to
the client a list of the messages which it can handle, and clients should not
send message types to the server if the server doesn't support them. 

--
Ramesh

Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

Ramesh Dharan wrote:
> Anthony, I have a detailed response to your earlier e-mail but I wanted to
> handle this discussion separately. 
>
>   
>>> I implemented new client->server messages for the (1) and (2), and
>>>       
>> I should have read more carefully.  This means that you're not a 
>> compliant RFB server ATM.
>>     
>
> I'm not sure I follow you. We support standard VNC clients, and we don't
> violate the protocol expectations of any standard VNC clients who talk to us.
>
>
> However, if a client *happens* to send us these new client->server messages
> (which don't exist in the standard spec), then we support them. 
>
> We never send a message to a client that the client doesn't know how to
> handle. 
>   

Right, but how does your client determine that the server supports these 
new messages?

The proper way to use new client message types (which is now described 
in the RFB spec) is to advertise a new pseudo-encoding for that client 
message type and wait for the server to send the pseudo-encoding back to 
the client.  That lets the client know that it is safe to use the new 
client message type.  This is what I'm using for my shared memory encoding.

Otherwise, there's no way to write a client that works with the 
"enhanced" server and a normal VNC server.

>> I guess that means there isn't a point registering the 
>> pseudo-encodings 
>> you are currently using since you have to change the server/client 
>> anyway.  It's easier anyway since pseudo-encodings are supposed to be 
>> negative numbers and you're using positive numbers.
>>     
>
> No, there's still a point. The display path and input path are essentially
> independent. Better input handling requires extending the client->server path
> (so the client can send new kinds of data other than the standard VNC
> keyboard and pointer events). Unfortunately, extension of the protocol in
> this direction was not baked into the original design. 
>
> I think that basically we should actually extend the RFB protocol itself to
> just have a server->client message, SetSupportedClientMessages, which works
> exactly the way SetEncodings works today, i.e. the server can send down to
> the client a list of the messages which it can handle, and clients should not
> send message types to the server if the server doesn't support them. 
>   

The mechanism I described above is what the current preferred method 
is.  If you want, we can bring the topic up with the VNC authors as 
AFAIK I'm the only person with a reserved client message type.  Of 
course, I think using a pseudo-encoding is a perfectly suitable way to 
address this problem.

Regards,

Anthony Liguori

> --
> Ramesh
>
>   

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Ramesh Dharan]

> The proper way to use new client message types (which is now 
> described 
> in the RFB spec) is to advertise a new pseudo-encoding for 
> that client 
> message type and wait for the server to send the 
> pseudo-encoding back to 
> the client.  That lets the client know that it is safe to use the new 
> client message type.  This is what I'm using for my shared 
> memory encoding.
> 
> Otherwise, there's no way to write a client that works with the 
> "enhanced" server and a normal VNC server.

Ok, yeah that makes sense. So yeah basically we would need to add new server
encodings for our client->server messages, and then you get the server to
send dummy "ack" messages for each one to say "yes, I understand this message
type"?

> The mechanism I described above is what the current preferred method 
> is.  If you want, we can bring the topic up with the VNC authors as 
> AFAIK I'm the only person with a reserved client message type.  Of 
> course, I think using a pseudo-encoding is a perfectly 
> suitable way to 
> address this problem.

Yeah the only problem I see with it is that I don't see how the server can
dynamically *change* the set of client messages that it accepts?

--
Ramesh

Re: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

Ramesh Dharan wrote:
>> The proper way to use new client message types (which is now 
>> described 
>> in the RFB spec) is to advertise a new pseudo-encoding for 
>> that client 
>> message type and wait for the server to send the 
>> pseudo-encoding back to 
>> the client.  That lets the client know that it is safe to use the new 
>> client message type.  This is what I'm using for my shared 
>> memory encoding.
>>
>> Otherwise, there's no way to write a client that works with the 
>> "enhanced" server and a normal VNC server.
>>     
>
> Ok, yeah that makes sense. So yeah basically we would need to add new server
> encodings for our client->server messages, and then you get the server to
> send dummy "ack" messages for each one to say "yes, I understand this message
> type"?
>   

Yup.

>> The mechanism I described above is what the current preferred method 
>> is.  If you want, we can bring the topic up with the VNC authors as 
>> AFAIK I'm the only person with a reserved client message type.  Of 
>> course, I think using a pseudo-encoding is a perfectly 
>> suitable way to 
>> address this problem.
>>     
>
> Yeah the only problem I see with it is that I don't see how the server can
> dynamically *change* the set of client messages that it accepts?
>   

To do it in general?  Yeah, I don't think there's a solution.  Of 
course, a SetServerEncodings would introduce a race.  What does the 
server do if it receives one of the new special client messages after 
sending the SetServerEncodings message (but before the client receives 
the SetServerEncodings message)?

I think it's easier to just add something to the psuedo-encoding to 
allow client messages that make sense to disable to be disabled on a 
case-by-case basis.

Regards,

Anthony Liguori

> --
> Ramesh
>
>   

RE: [Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Ramesh Dharan]

> To do it in general?  Yeah, I don't think there's a solution.  Of 
> course, a SetServerEncodings would introduce a race.  What does the 
> server do if it receives one of the new special client messages after 
> sending the SetServerEncodings message (but before the client 
> receives 
> the SetServerEncodings message)?

It just drops it ;=0. Yeah, you're right, there's no good way. 

> I think it's easier to just add something to the psuedo-encoding to 
> allow client messages that make sense to disable to be disabled on a 
> case-by-case basis.

Fair enough. In any case, I haven't encountered a situation before where the
server wanted to remove support for a client message. The only one I've
encountered is where the client wants to indicate changed conditions on its
end (e.g. failure to map the shared framebuffer).

--
Ramesh

[Qemu-devel] [PATCH] Support VNC PointerTypeChange psuedo-encoding

[Anthony Liguori]

[-- Attachment #1: Type: text/plain, Size: 913 bytes --]

The following patch adds support to the VNC server for the 
PointerTypeChange.  This is a new encoding I've defined specifically for 
virtualization.  It allows a VNC client to support a server that has a 
relative mouse (such as a PS/2 mouse in QEMU).  If you're familiar with 
the mouse being offset in a VNC session, this patch is the proper way to 
address that.

This extension is documented at http://tocm.wikidot.com/pointertypechange

It uses a type that I've reserved in the latest RFB spec.

Currently, the only client that supports this encoding is my PoC client 
available at http://hg.codemonkey.ws/vnc-gui.  I've also talked to Dan 
Berrange, the virt-manager maintainer, and I believe he plans on 
supporting this extension in virt-manager too.  Once QEMU supports the 
encoding, I suspect some of the other more popular VNC clients will 
considering also supporting it.

Regards,

Anthony Liguori

[-- Attachment #2: vnc-relative-mouse.diff --]
[-- Type: text/x-patch, Size: 2569 bytes --]

diff -r a137f714c033 vnc.c
--- a/vnc.c	Sun Dec 17 23:58:38 2006 -0600
+++ b/vnc.c	Sun Dec 17 23:58:51 2006 -0600
@@ -68,6 +68,11 @@ struct VncState
     int depth; /* internal VNC frame buffer byte per pixel */
     int has_resize;
     int has_hextile;
+    int has_pointer_type_change;
+    int absolute;
+    int last_x;
+    int last_y;
+
     Buffer output;
     Buffer input;
     kbd_layout_t *kbd_layout;
@@ -671,6 +676,19 @@ static void client_cut_text(VncState *vs
 {
 }
 
+static void check_pointer_type_change(VncState *vs, int absolute)
+{
+    if (vs->has_pointer_type_change && vs->absolute != absolute) {
+	vnc_write_u8(vs, 0);
+	vnc_write_u8(vs, 0);
+	vnc_write_u16(vs, 1);
+	vnc_framebuffer_update(vs, absolute, 0,
+			       vs->ds->width, vs->ds->height, -257);
+	vnc_flush(vs);
+    }
+    vs->absolute = absolute;
+}
+
 static void pointer_event(VncState *vs, int button_mask, int x, int y)
 {
     int buttons = 0;
@@ -686,21 +704,26 @@ static void pointer_event(VncState *vs, 
 	dz = -1;
     if (button_mask & 0x10)
 	dz = 1;
-	    
-    if (kbd_mouse_is_absolute()) {
+
+    if (vs->absolute) {
 	kbd_mouse_event(x * 0x7FFF / vs->ds->width,
 			y * 0x7FFF / vs->ds->height,
 			dz, buttons);
+    } else if (vs->has_pointer_type_change) {
+	x -= 0x7FFF;
+	y -= 0x7FFF;
+
+	kbd_mouse_event(x, y, dz, buttons);
     } else {
-	static int last_x = -1;
-	static int last_y = -1;
-
-	if (last_x != -1)
-	    kbd_mouse_event(x - last_x, y - last_y, dz, buttons);
-
-	last_x = x;
-	last_y = y;
-    }
+	if (vs->last_x != -1)
+	    kbd_mouse_event(x - vs->last_x,
+			    y - vs->last_y,
+			    dz, buttons);
+	vs->last_x = x;
+	vs->last_y = y;
+    }
+
+    check_pointer_type_change(vs, kbd_mouse_is_absolute());
 }
 
 static void reset_keys(VncState *vs)
@@ -829,6 +852,8 @@ static void set_encodings(VncState *vs, 
 
     vs->has_hextile = 0;
     vs->has_resize = 0;
+    vs->has_pointer_type_change = 0;
+    vs->absolute = -1;
     vs->ds->dpy_copy = NULL;
 
     for (i = n_encodings - 1; i >= 0; i--) {
@@ -845,10 +870,15 @@ static void set_encodings(VncState *vs, 
 	case -223: /* DesktopResize */
 	    vs->has_resize = 1;
 	    break;
+	case -257:
+	    vs->has_pointer_type_change = 1;
+	    break;
 	default:
 	    break;
 	}
     }
+
+    check_pointer_type_change(vs, kbd_mouse_is_absolute());
 }
 
 static int compute_nbits(unsigned int val)
@@ -1124,6 +1154,8 @@ void vnc_display_init(DisplayState *ds, 
     vs->lsock = -1;
     vs->csock = -1;
     vs->depth = 4;
+    vs->last_x = -1;
+    vs->last_y = -1;
 
     vs->ds = ds;