Re: XP guests sluggish after around 1 week of uptime

Avi Kivity
2009-01-19T12:46:15Z

(re-add kvm@vger)

jmandawg wrote:
>> -----Original Message-----
>> From: Avi Kivity [mailto:avi@redhat.com]
>> Sent: Monday, January 19, 2009 7:06 AM
>> To: jmandawg
>> Cc: kvm@vger.kernel.org
>> Subject: Re: XP guests sluggish after around 1 week of uptime
>>
>> There is a lot of wait time (last column, wrapped around to the first in
>> your output). Not a lot of real I/O though. Can you describe your I/O
>> subsystem? Are you using NFS?
>>
>>
>
>
>> What does 'top' show for the guests? I'm interested in qemu's VIRT and RES
>>
> columns.
>
>> Also, 'vmstat 1' on host and guest while slow copying is happening.
>>
>
>
> I'm using EXT3 on LVM on top of RAID 5. I can try moving one of the guest
> images to a different disk without the RAID 5 if you think that would help.
>

Software RAID or hardware RAID?

I think the root cause is the RAID here, not kvm.

Some suggestions:
- use the raw format directly on top of LVM. That is, use -drive
file=/dev/volgroup/logvol.
- if you use the raw format, disable disk caching: -drive
file=/dev/volgroup/logvol,cache=off
- try non-RAID5 storage

To migrate your volumes, use

qemu-img convert /path/to/image.qcow -O raw /dev/volgroup/logvol

Be sure to size the volume appropriately (same size as the virtual image).
> I just restarted the host machine yesterday, so the file copies are not
> running as slow as before, but still slow.
> Here is the current output of top during a file copy:
>
> top - 07:31:53 up 18:09, 4 users, load average: 0.74, 0.57, 0.46
> Tasks: 157 total, 1 running, 156 sleeping, 0 stopped, 0 zombie
> Cpu(s): 6.7%us, 12.7%sy, 0.0%ni, 59.6%id, 19.9%wa, 0.0%hi, 1.0%si,
> 0.0%st
> Mem: 8190236k total, 8141100k used, 49136k free, 1246732k buffers
> Swap: 8388600k total, 228k used, 8388372k free, 2418596k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 5945 root 20 0 2131m 2.0g 1924 D 13 25.8 44:15.71 kvm
>

S=D: waiting for disk. Low cpu usage as expected.

>
> Here is the current output of vmstat 1 during a file copy:
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id
> wa
> 0 1 228 46008 1210608 2466132 0 0 2308 8024 2695 13290 7 17
> 63 13
>

This seems fine if a bit low. You'll get better numbers sitting
directly atop LVM.