Problems with KVM-79

Problems with KVM-79

[Chris Kuhles]

Hi,

I was running KVM-76 with 2 CentOS 5.2 64bit VMs until a few days ago.
Everything was working fine. However, I needed to install a third VM running
Windows 2008 Server 64bit, which randomly kept crashing the host system with a
kernel panic, so I thought, why not upgrade to KVM-79, wanted to do that for a
while anyways. Now, Windows 2008 Server is running absolutely fine, no more host
crashes. However, the two CentOS VMs crash on me every once in a while for no
good reason. This apparently seems to happen more often when one of the CentOS
VMs is under load, like compiling something. I had one of the VMs crash on me
like 10 times to get a PHP 5.2.7 compile thru. I tried starting up the crashing
VM with -no-kvm, and the problem at least seemed to go away. I didn't leave it
running like this too long as the VM is in production and with -no-kvm,
everything ran terribly slow.

Here's some infos about the system I'm using:

Hardware:
- Intel Core2Quad Q6600
- 8 GB RAM (checked with memtest86+ to make sure it's good)
- 2x 320 GB Seagate drives in a software RAID-1

Software:
- Host system: CentOS 5.2 64bit
- VMs: 2x CentOS 5.2 64bit (same kernel as host, see below), 1x Windows Server
2008 64bit, all available updates installed
- manually compiled KVM-79
- Host kernel version: 2.6.18-92.1.18.el5

qemu commands:

VM1 (CentOS 5.2 64bit):
export DISPLAY=:1
qemu-kvm -daemonize -localtime -no-acpi -hda /kvm/vm1.img -m 1024 -boot c -net
nic,macaddr=52:54:00:00:01:00,model=e1000 -net tap,ifname=tap0,script=no

VM2 (Windows Server 2008 64bit):
export DISPLAY=:2
qemu-kvm -daemonize -localtime -hda /kvm/vm2.img -m 2048 -boot c -net
nic,macaddr=52:54:00:00:01:01,model=e1000 -net tap,ifname=tap1,script=no

VM3 (CentOS 5.2 64bit):
export DISPLAY=:3
qemu-kvm -daemonize -localtime -hda /kvm/vm3.img -m 1024 -boot c -net
nic,macaddr=52:54:00:00:01:02,model=e1000 -net tap,ifname=tap2,script=no

(qemu-kvm is a symlink to qemu-system-x86_64, inspired by the CentOS packages
provided by lfarkas.org. Also tried those instead of compiling manually, btw -
same problem.)

When the crashes happen, the qemu processes are still running on the host, but
consume a lot of CPU. Checking the vnc console, there is no output whatsoever
from the VMs, just a blank, black screen that doesn't react to anything (even
tried SysRq, nothing).

Also, what I noticed is that CentOS 5.2 spits loads of errors about kernel
modules (don't have the exact message ready now, but I could get it later on if
needed) when booting up as soon as -smp >=2 is used. This also did not happen
with KVM-76. When I run with -smp 1 or leave that parameter out, everything
works well - except for the random crashes.

As this did in no way happen with KVM-76, I figure there must me a bug in the
most recent version causing this issue.

I'd appreciate any help you could provide. If there's anything else I can do,
like provide debugging output etc., please just let me know.

-- 
Mit freundlichen Gruessen / Best regards

Chris Kuhles
Geschaeftsleitung / Managing Director

Aquatix IT-Services e.K.          Telefon: +49 69 24750456-0
Münzgasse 6                       Telefax: +49 69 24750456-1
60311 Frankfurt am Main           E-Mail:  ck@aquatix.de
AG Frankfurt am Main, HRA 44038   Web:     http://www.aquatix.de/

Re: Problems with KVM-79

[Chris Kuhles]

Hi,

in addition to this, I just tried updating to the most recent development code
(got it via git). The CentOS 5.2 VMs at least booted up fine, however, the
Windows 2008 Server immediately threw a bluescreen right after asking whether to
boot regularly or in safe mode (just killed of the qemu process before).
Switching back to KVM-79 again, everything ran "fine" again, except for the
problems explained below. So this is just to let you know that Win2008 support
seems broken in the current development tree.

Cheers,
Chris


> Hi,
> 
> I was running KVM-76 with 2 CentOS 5.2 64bit VMs until a few days ago.
> Everything was working fine. However, I needed to install a third VM running
> Windows 2008 Server 64bit, which randomly kept crashing the host system with a
> kernel panic, so I thought, why not upgrade to KVM-79, wanted to do that for a
> while anyways. Now, Windows 2008 Server is running absolutely fine, no more host
> crashes. However, the two CentOS VMs crash on me every once in a while for no
> good reason. This apparently seems to happen more often when one of the CentOS
> VMs is under load, like compiling something. I had one of the VMs crash on me
> like 10 times to get a PHP 5.2.7 compile thru. I tried starting up the crashing
> VM with -no-kvm, and the problem at least seemed to go away. I didn't leave it
> running like this too long as the VM is in production and with -no-kvm,
> everything ran terribly slow.
> 
> Here's some infos about the system I'm using:
> 
> Hardware:
> - Intel Core2Quad Q6600
> - 8 GB RAM (checked with memtest86+ to make sure it's good)
> - 2x 320 GB Seagate drives in a software RAID-1
> 
> Software:
> - Host system: CentOS 5.2 64bit
> - VMs: 2x CentOS 5.2 64bit (same kernel as host, see below), 1x Windows Server
> 2008 64bit, all available updates installed
> - manually compiled KVM-79
> - Host kernel version: 2.6.18-92.1.18.el5
> 
> qemu commands:
> 
> VM1 (CentOS 5.2 64bit):
> export DISPLAY=:1
> qemu-kvm -daemonize -localtime -no-acpi -hda /kvm/vm1.img -m 1024 -boot c -net
> nic,macaddr=52:54:00:00:01:00,model=e1000 -net tap,ifname=tap0,script=no
> 
> VM2 (Windows Server 2008 64bit):
> export DISPLAY=:2
> qemu-kvm -daemonize -localtime -hda /kvm/vm2.img -m 2048 -boot c -net
> nic,macaddr=52:54:00:00:01:01,model=e1000 -net tap,ifname=tap1,script=no
> 
> VM3 (CentOS 5.2 64bit):
> export DISPLAY=:3
> qemu-kvm -daemonize -localtime -hda /kvm/vm3.img -m 1024 -boot c -net
> nic,macaddr=52:54:00:00:01:02,model=e1000 -net tap,ifname=tap2,script=no
> 
> (qemu-kvm is a symlink to qemu-system-x86_64, inspired by the CentOS packages
> provided by lfarkas.org. Also tried those instead of compiling manually, btw -
> same problem.)
> 
> When the crashes happen, the qemu processes are still running on the host, but
> consume a lot of CPU. Checking the vnc console, there is no output whatsoever
> from the VMs, just a blank, black screen that doesn't react to anything (even
> tried SysRq, nothing).
> 
> Also, what I noticed is that CentOS 5.2 spits loads of errors about kernel
> modules (don't have the exact message ready now, but I could get it later on if
> needed) when booting up as soon as -smp >=2 is used. This also did not happen
> with KVM-76. When I run with -smp 1 or leave that parameter out, everything
> works well - except for the random crashes.
> 
> As this did in no way happen with KVM-76, I figure there must me a bug in the
> most recent version causing this issue.
> 
> I'd appreciate any help you could provide. If there's anything else I can do,
> like provide debugging output etc., please just let me know.
> 


-- 
Mit freundlichen Gruessen / Best regards

Chris Kuhles
Geschaeftsleitung / Managing Director

Aquatix IT-Services e.K.          Telefon: +49 69 24750456-0
Münzgasse 6                       Telefax: +49 69 24750456-1
60311 Frankfurt am Main           E-Mail:  ck@aquatix.de
AG Frankfurt am Main, HRA 44038   Web:     http://www.aquatix.de/