Re: [PULL 02/23] ui/vnc: take account of client byte order in pixman format

["Daniel P. Berrangé" <berrange@redhat.com>]

On Tue, Jun 03, 2025 at 01:18:55PM +0200, Thomas Huth wrote:
> On 22/05/2025 12.29, Daniel P. Berrangé wrote:
> > The set_pixel_conversion() method is responsible for determining whether
> > the VNC client pixel format matches the server format, and thus whether
> > we can use the fast path "copy" impl for sending pixels, or must use
> > the generic impl with bit swizzling.
> > 
> > The VNC server format is set at build time to VNC_SERVER_FB_FORMAT,
> > which corresponds to PIXMAN_x8r8g8b8.
> > 
> > The qemu_pixman_get_format() method is then responsible for converting
> > the VNC pixel format into a pixman format.
> > 
> > The VNC client pixel shifts are relative to the associated endianness.
> > 
> > The pixman formats are always relative to the host native endianness.
> > 
> > The qemu_pixman_get_format() method does not take into account the
> > VNC client endianness, and is thus returning a pixman format that is
> > only valid with the host endianness matches that of the VNC client.
> ...
> 
>  Hi Daniel,
> 
> this patch breaks the output in the TigerVNC viewer for me.
> If I run "./qemu-system-x86_64 -vnc :1" on my laptop, and then connect to it
> via "vncviewer :1", the output of the BIOS now appears in yellow letters
> (instead of grey ones).
> 
> FWIW, the output of TigerVNC viewer is:
> 
>  TigerVNC viewer v1.15.0
>  Built on: 2025-04-08 00:00
>  Copyright (C) 1999-2025 TigerVNC team and many others (see README.rst)
>  See https://www.tigervnc.org for information on TigerVNC.
> 
>  Tue Jun  3 13:17:50 2025
>   DecodeManager: Detected 16 CPU core(s)
>   DecodeManager: Creating 4 decoder thread(s)
>   CConn:       Connected to host localhost port 5901
>   CConnection: Server supports RFB protocol version 3.8
>   CConnection: Using RFB protocol version 3.8
>   CConnection: Choosing security type None(1)
>   CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
> 
> Could you please have a look what's going wrong here?

Yes, I can reproduce too, will check this out.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|