SMP migration problems with FreeBSD and WinXP guests

SMP migration problems with FreeBSD and WinXP guests

[Chris Webb]

I'm experiencing a number of problems with kvm on a 32-bit x86 host when
migrating some SMP guests. The symptoms are identical on both kvm-68 and
kvm-69.

Two out of my three test guests are currently failing to migrate: a stock
FreeBSD 7.0 install and an out-of-the-box WinXP install. My other test image
is Debian with a 2.6.18 Linux kernel: this appears to migrate absolutely
fine.

I'm running kvm with the arguments

  -m 512 -smp 4 -vnc :1 -hda IMAGEFILE -monitor stdio

saving using migrate file://migrate.dump at the monitor, then restoring using

  -m 512 -smp 4 -vnc :1 -hda IMAGEFILE -monitor stdio -incoming file://migrate.dump

In each case, the problem is fixed by running with -smp 1 but still occurs
with -smp 2. In the WinXP case, the problem also goes away with -no-kvm.
FreeBSD 7.0 appears to grind to a halt without ever booting on SMP with
-no-kvm so I can't test migration there.

Unfortunately, because the migration format changes when -no-kvm is turned
on (I think the phys_ram_page_exist_bitmap before the dumped pages is only
included without -no-kvm?), I can't independently turn on -no-kvm on the
source and destination to detect whether the problem is with dumping the
state or restoring it.

My kvm is built using gcc 3.4.3 with no weird configure options or
optimisation, just a pointer to the kernel build directory plus --disable-sdl
and --disable-gfx-check. I'm using the module source supplied with kvm,
building and running against kernel 2.6.24.4.

I'm happy to put some work in to help debug these, but I don't have any
knowledge of the internals of qemu or the kernel side of kvm so there's
quite a bit of unfamiliar code to dig through: any pointers to likely
culprits or advice of things to test would be gratefully received!


With the FreeBSD 7.0 guest, the VM appears to migrate to a file://migrate.dump
fine, and then appears to restore okay but all disk IO fails after this point
with kernel messages like this:

  ad0: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - completing request directly
  ad0: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - completing request directly
  ad0: WARNING - SET_MULTI taskqueue timeout - completing request directly
  ad0: TIMEOUT - READ_DMA retrying (1 retry left) LBA=817047
  ad0: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - completing request directly
  ad0: WARNING - SETFEATURES SET TRANSFER MODE taskqueue timeout - completing request directly
  ad0: WARNING - SET_MULTI taskqueue timeout - completing request directly
  ad0: TIMEOUT - WRITE_DMA retrying (1 retry left) LBA=7581619


With a WinXP guest, again the VM appears to migrate to a file://migrate.dump
fine, but when I try to start a kvm with -incoming file://migrate.dump, it
crashes after about half a second with the following error:

  exception 13 (33)
  rax 0000000000000000 rbx 0000000000000000 rcx 0000000000000010 rdx 0000000000000000
  rsi 0000000000000001 rdi 0000000000000000 rsp 000000000000fffc rbp 0000000000000000
  r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
  r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
  rip 0000000000002019 rflags 00033046
  cs 1000 (00010000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
  tr 0000 (fffbd000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
  ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
  gdt 0/ffff
  idt 0/ffff
  cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
  code: 20 16 00 00 10 00 00 00 00 00 00 01 00 00 00 fa 00 27 00 f8 --> 0f 00 00 00 00 00 00 c0 ff ff ff 6c 68 06 00 48 1c 16 00 53 a0 16 66 10 1d 16 00 90 6f 61
  Aborted


Cheers,

Chris.