Re: qemu-kvm segfaults in qemu_del_timer (0.10.5 and 0.10.6)

[Chris Webb <chris@arachsys.com>]

Chris Webb <chris@arachsys.com> writes:

> Avi Kivity <avi@redhat.com> writes:
> 
> > I understand it's hard, but it's nearly impossible to work out the  
> > problem from so little data, so please do make the effort to obtain 
> > dumps.
> 
> We're trying for this at the moment, but since we can't change the rlimit
> for the running qemu-kvm processes (?), we'll have to wait until one of the
> new ones dies, which may take some time. I'll follow up when I do have
> something.

We've been lucky and relatively quickly got a core dump from one of the new
qemu-kvms with the non-zero core file rlimit. A backtrace looks like this:

  (gdb) bt    
  #0  0x00000000004068f7 in qemu_mod_timer (ts=0x30d1f30, expire_time=430489)
      at /packages/qemu-kvm/src-f39tF1/vl.c:1161
  #1  0x0000000000495dd5 in vnc_update_client (opaque=<value optimized out>) at vnc.c:765
  #2  0x00000000004081da in main_loop_wait (timeout=<value optimized out>) at /packages/qemu-kvm/src-f39tF1/vl.c:1240
  #3  0x000000000051613a in kvm_main_loop () at /packages/qemu-kvm/src-f39tF1/qemu-kvm.c:596
  #4  0x000000000040c7b7 in main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>)
      at /packages/qemu-kvm/src-f39tF1/vl.c:3850

The segfault appears to be a null pointer dereference. ts->clock is NULL
and line 1161 uses ts->clock->type:

  (gdb) p ts   
  $4 = (QEMUTimer *) 0x30d1f30
  (gdb) p ts->clock
  $5 = (QEMUClock *) 0x0

The VncState in vnc_update_client is as follows:

  (gdb) f 1        
  #1  0x0000000000495dd5 in vnc_update_client (opaque=<value optimized out>) at vnc.c:765
  765             qemu_mod_timer(vs->timer, qemu_get_clock(rt_clock) + VNC_REFRESH_INTERVAL);
  (gdb) p *vs                        
  $12 = {timer = 0x30d1f30, csock = -986235208, ds = 0x0, vd = 0x0, need_update = 1, dirty_row = {{0, 0, 4294967295, 
        4294967295} <repeats 768 times>, {4294967295, 4294967295, 4294967295, 4294967295} <repeats 1280 times>}, 
    old_data = 0x7f9b8276f010 <Address 0x7f9b8276f010 out of bounds>, features = 98, absolute = 1, last_x = -1, 
    last_y = -1, vnc_encoding = 5, tight_quality = 6 '\006', tight_compression = 1 '\001', major = 3, minor = 3, 
    challenge = "\032\314i\257<\302t1(\320\312\263\024pH\226", output = {capacity = 1545078, offset = 684, 
      buffer = 0x3107860 ""}, input = {capacity = 5120, offset = 0, buffer = 0x3106450 "\020\220(\003"}, 
    write_pixels = 0x490b50 <vnc_write_pixels_generic>, send_hextile_tile = 0x492030 <send_hextile_tile_generic_32>, 
    clientds = {flags = 0 '\0', width = 800, height = 600, linesize = 3200, 
      data = 0x7f9b82944010 <Address 0x7f9b82944010 out of bounds>, pf = {bits_per_pixel = 32 ' ', 
        bytes_per_pixel = 4 '\004', depth = 24 '\030', rmask = 0, gmask = 0, bmask = 0, amask = 0, rshift = 16 '\020', 
        gshift = 8 '\b', bshift = 0 '\0', ashift = 24 '\030', rmax = 255 '\377', gmax = 255 '\377', bmax = 255 '\377', 
        amax = 255 '\377', rbits = 8 '\b', gbits = 8 '\b', bbits = 8 '\b', abits = 8 '\b'}}, serverds = {
      flags = 2 '\002', width = 1024, height = 768, linesize = 4096, data = 0x7f9b8246e010 "", pf = {
        bits_per_pixel = 32 ' ', bytes_per_pixel = 4 '\004', depth = 24 '\030', rmask = 16711680, gmask = 65280, 
        bmask = 255, amask = 0, rshift = 16 '\020', gshift = 8 '\b', bshift = 0 '\0', ashift = 24 '\030', 
        rmax = 255 '\377', gmax = 255 '\377', bmax = 255 '\377', amax = 255 '\377', rbits = 8 '\b', gbits = 8 '\b', 
        bbits = 8 '\b', abits = 8 '\b'}}, audio_cap = 0x0, as = {freq = 44100, nchannels = 2, fmt = AUD_FMT_S16, 
      endianness = 0}, read_handler = 0x494b40 <protocol_client_msg>, read_handler_expect = 1, 
    modifiers_state = '\0' <repeats 255 times>, zlib = {capacity = 0, offset = 0, buffer = 0x0}, zlib_tmp = {
      capacity = 0, offset = 0, buffer = 0x0}, zlib_stream = {{next_in = 0x0, avail_in = 0, total_in = 0, 
        next_out = 0x0, avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, 
        data_type = 0, adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, 
        avail_out = 0, total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0, 
        adler = 0, reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, 
        total_out = 0, msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0, adler = 0, 
        reserved = 0}, {next_in = 0x0, avail_in = 0, total_in = 0, next_out = 0x0, avail_out = 0, total_out = 0, 
        msg = 0x0, state = 0x0, zalloc = 0, zfree = 0, opaque = 0x0, data_type = 0, adler = 0, reserved = 0}}, 
    next = 0x0}

I'm afraid I only have one of these, so I can't say whether the other
segfaults were exactly the same or different (other than knowing the source
line matched), but I'll keep my eye out for more core dumps.

qemu-kvm command line for this guest would have been

  qemu-kvm -m 1024 -smp 1 -usbdevice tablet -boot cd \
    -drive if=ide,bus=0,unit=0,cache=none,file=/dev/mapper/guest:8a2576b2-a523-4126-867a-5f411cb66f18:ide:0:0 \
    -drive if=ide,bus=0,unit=1,media=cdrom,cache=none,file=/dev/mapper/guest:8a2576b2-a523-4126-867a-5f411cb66f18:ide:0:1 \
    -net tap,vlan=0,ifname=tap1,script=no,downscript=no \
    -net nic,model=e1000,macaddr=02:00:53:de:e2:b0,vlan=0 \
    -vnc :2,password -uuid 8a2576b2-a523-4126-867a-5f411cb66f18 \
    -pidfile /var/run/guests/8a2576b2-a523-4126-867a-5f411cb66f18/kvm.pid \
    -monitor unix:/var/lib/guests/8a2576b2-a523-4126-867a-5f411cb66f18/monitor,server,nowait

Best wishes,

Chris.