Re: [Qemu-devel] [PATCH 1/3] vnc: fix server surface pixel format.

[Daniel Gollub <gollub@b1-systems.de>]

On Monday 27 April 2009 04:39:51 pm Gerd Hoffmann wrote:
> Format must be identical to the guest surface, we can't work with
> the 32 bpp used by the default surface allocator.
>
> Without this patch vnc doesn't get the conversions right when sending
> pixel data to the client.  The bug triggers if
>   (a) the client doesn't support WMVi, and
>   (b) the guest screen depth is != 32 bpp.

Tried your patches - qemu crashs now sometimes with:
*** glibc detected *** ./x86_64-softmmu/qemu-system-x86_64: malloc(): memory corruption: 0x0000000000e28e50 ***


(When it doesn't crash it just works perfectly fine again with vncviewer.)

Backtrace looks like this:

Core was generated by `./x86_64-softmmu/qemu-system-x86_64 -hda /home/dgollub/VMs/sles11_x86_64.qcow2'.
Program terminated with signal 6, Aborted.
#0  0x00007f7189867645 in raise () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f7189867645 in raise () from /lib64/libc.so.6
#1  0x00007f7189868c33 in abort () from /lib64/libc.so.6
#2  0x00007f71898a38e8 in ?? () from /lib64/libc.so.6
#3  0x00007f71898a9118 in ?? () from /lib64/libc.so.6
#4  0x00007f71898aba5c in ?? () from /lib64/libc.so.6
#5  0x00007f71898ace64 in ?? () from /lib64/libc.so.6
#6  0x00007f71898ade38 in realloc () from /lib64/libc.so.6
#7  0x000000000046fd3e in qemu_realloc (ptr=0x4195, size=16789) at qemu-malloc.c:52
#8  0x000000000049a10c in buffer_reserve (buffer=0xd3d090, len=<value optimized out>) at vnc.c:313
#9  0x000000000049a16b in vnc_write (vs=0xd2d010, data=0x7fff93b52ca0, len=6) at vnc.c:1135
#10 0x000000000049a29a in vnc_write_pixels_generic (vs=0xd2d010, pixels1=0xe09a40, size=<value optimized out>)
    at vnc.c:457
#11 0x000000000049e0f4 in send_framebuffer_update (vs=0xd2d010, x=0, y=0, w=800, h=600) at vnc.c:479
#12 0x000000000049e6f2 in vnc_update_client (opaque=0xd2d010) at vnc.c:788
#13 0x0000000000409dbc in main_loop_wait (timeout=<value optimized out>) at /home/dgollub/projects/qemu/vl.c:1227
#14 0x000000000040d80d in main (argc=5, argv=0x7fff93b538d8, envp=<value optimized out>)
    at /home/dgollub/projects/qemu/vl.c:4448
(gdb) 

Hope that helps.

Thanks for fixing the vncviewer regression!

Best Regards,
Daniel

-- 
Daniel Gollub                        Geschaeftsfuehrer: Ralph Dehner
FOSS Developer                       Unternehmenssitz:  Vohburg
B1 Systems GmbH                      Amtsgericht:       Ingolstadt
Mobil: +49-(0)-160 47 73 970         Handelsregister:   HRB 3537
EMail: gollub@b1-systems.de          http://www.b1-systems.de

Adresse: B1 Systems GmbH, Osterfeldstraße 7, 85088 Vohburg
http://pgpkeys.pca.dfn.de/pks/lookup?op=get&search=0xED14B95C2F8CA78D