[Qemu-devel] Question on QEMU's VNC Server hextile implementation

[Qemu-devel] Question on QEMU's VNC Server hextile implementation

[Yaniv Kaul]

According to the RFB protocol, section 6.6.4 (hextile encoding), regarding the 'ForegroundSpecified' bit, it says: 'If this bit is set then the SubrectsColoured bit must be zero.'.
It doesn't seem QEMU's VNC server does that. In fact, it looks like both bits are set.
I've verified against a different VNC server, and I didn't see this happening.
(it may be a Wireshark dissector bug, of course).

TIA,
Y.

Re: [Qemu-devel] Question on QEMU's VNC Server hextile implementation

[Anthony Liguori]

Yaniv Kaul wrote:
> According to the RFB protocol, section 6.6.4 (hextile encoding), regarding the 'ForegroundSpecified' bit, it says: 'If this bit is set then the SubrectsColoured bit must be zero.'.
> It doesn't seem QEMU's VNC server does that. In fact, it looks like both bits are set.
> I've verified against a different VNC server, and I didn't see this happening.
> (it may be a Wireshark dissector bug, of course).
>   

Yes, this definitely goes against the wording in the specification.  
It's also wasting bits on the wire.  I'll work up a patch on Monday.

I checked the real vnc client to make sure it handles this combination 
and it does.  I think the reason the spec forbids this is that a 
SubrectsColoured invalidates the foreground color.  Having the two is 
therefore not a useful combination because the foreground color won't 
actually be used for anything.

We do invalidate the foreground color correctly but we're just sending 
out useless data so removing it's a good idea.  Thanks for catching this!

Regards,

Anthony Liguori

> TIA,
> Y.
>
>
>