stability issue with KVM using SMP

stability issue with KVM using SMP

[James Stevens]

Summary - I'm getting stability issues running SMP. The guest will run 
for, typically, 6 to 12 hours before causing a problem.

The first time it happened the guest process that was running SMP went 
Zombie, but in a spin loop clocking huge amount of CPU time, and I had 
to reboot the host to clear it. Needless to say I had no access to the 
guest's qmeu console (Ctrl-Alt-2) and a "kill -9 <pid>" from the host 
had no effect.

However, the kernel had been compiled with a 64Gb memory model and I've 
had problems with that in the past - in fact that's the first time I've 
seen a kernel compiled with the 64Gb memory model actually boot under KVM.


So I replaced the kernel and the next time the guest simply locked up - 
I couldn't access the guest o/s at all (no "ping", no linux console 
response etc - in some cases there was a picture on the console [login 
prompt] but it didn't respond to any key presses), but the qemu console 
still worked. I tried a "system_powerdown" and that failed so I did a 
"system_reset" and that rebooted the client.


I have now switched off SMP and the guest is working fine (just SLOW).


I've had quite a few non-SMP linux guests running on this host for some 
time, using various kernels, and not seen a problem. Its only since I 
tried to introduce SMP that its all gone pear shaped.


> # what cpu model (examples: Intel Core Duo, Intel Core 2 Duo, AMD
> Opteron 2210). See /proc/cpuinfo if you're not sure.

dual "Quad-Core AMD Opteron(tm) Processor 2352" with 32Gb RAM

> # what kvm version you are using. If you're using git directly,
> provide the output of 'git describe'.

"kvm-72"

> # the host kernel version

"2.6.26.2" running on Slackware 12

> # what host kernel arch you are using (i386 or x86_64)

# CONFIG_64BIT is not set
CONFIG_X86_32=y
# CONFIG_X86_64 is not set
CONFIG_X86=y

Should I be using 64 bit ?

> # what guest you are using, including OS type (Linux, Windows,
> Solaris, etc.), bitness (32 or 64), kernel version

Slackware 11, but with a new standard (no patches) kernel - 2.6.26.2

I also have an older Slackware 7 based guest with a 2.4.29 SMP kernel 
that sees the "lock-up, with qemu console working" problem, but not the 
Zombie issue. The 2.4.29 kernel uses a 1Gb memory model.

> # the qemu command line you are using to start the guest

/usr/local/bin/qemu-system-x86_64 -hda /opt/kvm/machine_14/vdisk1.img \
	-m 1024 -vnc :14 -k en-gb -smp 4 \
         -net nic,model=e1000,macaddr=52:54:00:14:00:00 -net tap \
         -net nic,vlan=2,model=e1000,macaddr=52:54:00:14:00:01 \
	-net tap,vlan=2,ifname=tap55 \


> # whether the problem goes away if using the -no-kvm-irqchip or
> -no-kvm-pit switch.

Not tried

> # whether the problem also appears with the -no-kvm switch.

I don't use this switch.



James

Re: stability issue with KVM using SMP

[James Stevens]

Further to my previous report..

I have just noticed that, where the host is running on the dual 
"Quad-Core AMD Opteron(tm) Processor 2352" (family 16, model 2) the 
kernels in the *guest* machines (usually 2.6.26) are reporting "WARNING: 
This combination of AMDprocessors is not suitable for SMP." - presumably 
this is why my SMP isn't stable!

The kernel in that host machine (2.6.26.2) does *not* report the messages.



However, where the host is a single "AMD Athlon(tm) 64 X2 Dual Core 
Processor 4000+" (family 15, model 107) the guest kernels do not report 
the WARNING above - but I've not actually run any SMP guest on that 
host, so don't know if it would work or not.



The warning seems to be due to a processor capability test in 
"arch/x86/kernel/smpboot.c" - "add_taint(TAINT_UNSAFE_SMP);"

Its not clear to me why one guess is failing this test and the other 
passing, because both guests are seeing "QEMU Virtual CPU version 
0.9.1", family 6, model 2 (according to "/proc/cpuinfo").

Although most kernels on the Athlon host are running 2.6.21 not 2.6.26




James