Another VNC crash, qemu-kvm-0.12.3

Chris Webb
2010-03-01T18:14:19Z

We've just seen another VNC related qemu-kvm crash, this time an arithmetic
exception at vnc.c:1424 in the newly release qemu-kvm 0.12.3.

[...]
1423 if (vs->absolute) {
1424 kbd_mouse_event(x * 0x7FFF / (ds_get_width(vs->ds) - 1),
1425 y * 0x7FFF / (ds_get_height(vs->ds) - 1),
1426 dz, buttons);
1427 } else if (vnc_has_feature(vs, VNC_FEATURE_POINTER_TYPE_CHANGE)) {
1428 x -= 0x7FFF;
[...]

and sure enough:

(gdb) p vs->ds->surface->width
$1 = 9
(gdb) p vs->ds->surface->height
$2 = 1

What a 9x1 display surface is doing on this guest is a mystery to me, but you
definitely can't divide by one less than its height!

(gdb) p *vs
$3 = {csock = 19, ds = 0x1c60fa0, dirty = {{4294967295, 4294967295, 4294967295, 4294967295,
4294967295} <repeats 2048 times>}, vd = 0x26a0110, need_update = 1, force_update = 0, features = 67,
absolute = 1, last_x = -1, last_y = -1, vnc_encoding = 5, tight_quality = 9 '\t', tight_compression = 9 '\t',
major = 3, minor = 8, challenge = "¹{\177\226\200kÕjéPñÄA¤o)", output = {capacity = 925115, offset = 0,
buffer = 0x28ba4b0 ""}, input = {capacity = 5120, offset = 6, buffer = 0x28b90a0 "\005"},
write_pixels = 0x4bb9e0 <vnc_write_pixels_generic>, send_hextile_tile = 0x4bcdf0 <send_hextile_tile_generic_32>,
clientds = {flags = 0 '\0', width = 800, height = 600, linesize = 3200, data = 0x7fcd00ab6010 "", pf = {
bits_per_pixel = 32 ' ', bytes_per_pixel = 4 '\004', depth = 24 '\030', rmask = 0, gmask = 0, bmask = 0,
amask = 0, rshift = 16 '\020', gshift = 8 '\b', bshift = 0 '\0', ashift = 24 '\030', rmax = 255 'ÿ',
gmax = 255 'ÿ', bmax = 255 'ÿ', amax = 255 'ÿ', rbits = 8 '\b', gbits = 8 '\b', bbits = 8 '\b',
abits = 8 '\b'}}, audio_cap = 0x0, as = {freq = 44100, nchannels = 2, fmt = AUD_FMT_S16, endianness = 0},
read_handler = 0x4beac0 <protocol_client_msg>, read_handler_expect = 6, modifiers_state = '\0' <repeats 255 times>,
zlib = {capacity = 0, offset = 0, buffer = 0x0}, zlib_tmp = {capacity = 0, offset = 0, buffer = 0x0},
zlib_stream = {{next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0,
state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0,
avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0,
zfree = 0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0,
next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0,
data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0,
avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0,
adler = 0, reserved = 0}}, next = 0x0}

(gdb) p *vs->ds
$4 = {surface = 0x1c81f40, opaque = 0x26a0110, gui_timer = 0x0, allocator = 0x8199d0, listeners = 0x1c95fa0,
mouse_set = 0, cursor_define = 0, next = 0x0}

(gdb) p *vs->ds->surface
$5 = {flags = 2 '\002', width = 9, height = 1, linesize = 36, data = 0x7fcd00ab6010 "", pf = {
bits_per_pixel = 32 ' ', bytes_per_pixel = 4 '\004', depth = 24 '\030', rmask = 16711680, gmask = 65280,
bmask = 255, amask = 0, rshift = 16 '\020', gshift = 8 '\b', bshift = 0 '\0', ashift = 24 '\030', rmax = 255 'ÿ',
gmax = 255 'ÿ', bmax = 255 'ÿ', amax = 255 'ÿ', rbits = 8 '\b', gbits = 8 '\b', bbits = 8 '\b', abits = 8 '\b'}}

Cheers,

Chris.