XP guests sluggish after around 1 week of uptime

XP guests sluggish after around 1 week of uptime

[Jay Mann]

If this is the wrong place to ask this question, please let me know.

I’m running kvm-80 on ubuntu 8.04 x64 kernel 2.6.24-22-server, md raid 5 with
LVM.  When my host is first booted my windows XP guest runs fine, but after
around a week or so of uptime, they start to run very sluggish.  It takes around
a minute just to open the start menu.  

I'm starting to wonder if it is a problem with kvm (qemu) disk performance.  The
Read/Write speeds seems to be half as fast as my VMWare XP guest.  But not sure
why this would happen after running fine for around a week.

I have the XP guest set to use 2GB of ram and disabled the paging file but that
still doesn’t help.



Any advice on troubleshooting this would be appreciated.

Thanks,

-J

Re: XP guests sluggish after around 1 week of uptime

[Florent]

Hi Jay,

My host (Intel Core 2 Duo 2.4GHz, 2GB) has successfully run windows 
guests for more than one month. I had to shutdown them to have kvm updated.
What kind of video output do you use? I have no problem with VNC so far. 
Have you kept the video viewer opened for this week?
If you have file servers (such as FTP) on this guest, do they respond 
quickly while the display is as slow as you're described?

Regards,

Florent

Re: XP guests sluggish after around 1 week of uptime

[Jay]

Hi Florent, 
Thanks for the response.

I also use VNC, and i usually leave it open the whole time on the host and then
just remote desktop in from different boxes (work, laptop)

I do have a windows file share i can test next time it slows down.  I'll let you
know what happends.

-J

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

Jay Mann wrote:
> If this is the wrong place to ask this question, please let me know.
>
> I’m running kvm-80 on ubuntu 8.04 x64 kernel 2.6.24-22-server, md raid 5 with
> LVM.  When my host is first booted my windows XP guest runs fine, but after
> around a week or so of uptime, they start to run very sluggish.  It takes around
> a minute just to open the start menu.  
>
> I'm starting to wonder if it is a problem with kvm (qemu) disk performance.  The
> Read/Write speeds seems to be half as fast as my VMWare XP guest.  But not sure
> why this would happen after running fine for around a week.
>
> I have the XP guest set to use 2GB of ram and disabled the paging file but that
> still doesn’t help.
>   

How much memory do you have on the host?  Is the host swapping?  Can you 
post 'vmstat 1' output while Windows is opening the start menu?

-- 
error compiling committee.c: too many arguments to function

Re: XP guests sluggish after around 1 week of uptime

[Jay]

Avi Kivity <avi <at> redhat.com> writes:

> 
> How much memory do you have on the host?  Is the host swapping?  Can you 
> post 'vmstat 1' output while Windows is opening the start menu?
> 

I have 8GB RAM on the host, and i've run htop and the swap space is not being
used at all.  I have a total of 4 guest (2 windows + 2 linux)

I will have to try the 'vmstat 1' command tonight when i get home from work.  My
work has blocked my ssh access to my home PC a few months ago so i started using
proxyTunnel over https which worked fine, but somehow they figured out how to
block that today :(

Maybe it's time to find a new job.

-J

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

Please keep the cc list when replying (use reply-all).


Jay wrote:
> Avi Kivity <avi <at> redhat.com> writes:
>
>   
>> How much memory do you have on the host?  Is the host swapping?  Can you 
>> post 'vmstat 1' output while Windows is opening the start menu?
>>
>>     
>
> I have 8GB RAM on the host, and i've run htop and the swap space is not being
> used at all.  I have a total of 4 guest (2 windows + 2 linux)
>
> I will have to try the 'vmstat 1' command tonight when i get home from work.

Provide kvm_stat output as well.

-- 
error compiling committee.c: too many arguments to function

RE: XP guests sluggish after around 1 week of uptime

[jmandawg]

The guest is not acting up right now since I just rebooted the host on
Sunday.  
Here is the current outputs of the command, and I will post them again when
they start acting up:

[root@e6400:~/ 500]$vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system--
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
wa
 1  0    212  95816 992968 1775292    0    0   464   280   61   28 45  6 46
2
 0  0    212  95608 992968 1775312    0    0     0   120   98 8179  8  8 84
0
 0  0    212  95592 992968 1775312    0    0     0     0   18 8099  7  8 86
0
 1  0    212  95716 992968 1775312    0    0     0     0   42 8626  4  7 89
0
 1  0    212  95716 992968 1775312    0    0     0    56   56 8034  7 11 82
0
 0  0    212  95716 992968 1775312    0    0     8    36   59 8317  7  6 86
1
 1  0    212  95660 992968 1775324    0    0     0    84   61 8203 11 11 78
0
 0  0    212  95304 992968 1775324    0    0     0    48   65 8003  8  7 85
0
 1  0    212  95264 992968 1775324    0    0     0     0   13 7704 15  7 78
0
 0  0    212  95264 992968 1775324    0    0     0    80   97 8018 15  5 80
0
 2  0    212  95468 992968 1775324    0    0     0    16   35 7891  9  8 83
0
 3  0    212  95380 992968 1775324    0    0     0   124  103 8179  9  8 81
1
 0  0    212  95380 992968 1775324    0    0     0    40   48 8006 11  8 80
1
 0  0    212  95336 992972 1775320    0    0     0   116  104 8285  9  5 85
1
 0  0    212  95356 992976 1775320    0    0     0   312  173 8421  7 11 79
3
 1  0    212  95380 992976 1775324    0    0     0     0   42 8233  5  9 87
0
 0  0    212  95460 992976 1775324    0    0     0    72   48 8188  7 11 82
0
 1  0    212  95380 992976 1775324    0    0     0     0   39 8221  8  8 84
0
 0  0    212  95468 992976 1775324    0    0     0     0   17 7982 10  6 83
0
 0  0    212  95460 992976 1775324    0    0     0   100  100 8421  7  6 85
2
 1  0    212  95380 992976 1775324    0    0     0    32   35 8000  8  7 85
0


kvm statistics

 efer_reload                 15       0
 exits               3315693540   22034
 fpu_reload           174984854     724
 halt_exits           216355599    1503
 halt_wakeup           55495383     341
 host_state_reload    867498430    4631
 hypercalls                   0       0
 insn_emulation        56483585     491
 insn_emulation_fail       1025       0
 invlpg                 7980288       5
 io_exits            1796250843   11377
 irq_exits             85442292     224
 irq_injections       250928733    1661
 irq_window             9135944      52
 kvm_request_irq              0       0
 largepages                   0       0
 mmio_exits            28133579     310
 mmu_cache_miss         1041626       0
 mmu_flooded             426300       0
 mmu_pde_zapped          704522       0
 mmu_pte_updated        2702458       0
 mmu_pte_write          3546140       0
 mmu_recycled             16553       0
 mmu_shadow_zapped      1245678       0
 mmu_unsync               11192      -1
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed              65801856      54
 pf_guest              22111366       5
 remote_tlb_flush           235       0
 request_nmi                  0       0
 signal_exits                 5       0
 tlb_flush            850332604    6382

-----Original Message-----
From: Avi Kivity [mailto:avi@redhat.com] 
Sent: Tuesday, January 13, 2009 4:29 AM
To: Jay
Cc: kvm@vger.kernel.org
Subject: Re: XP guests sluggish after around 1 week of uptime

Please keep the cc list when replying (use reply-all).

Jay wrote:
> Avi Kivity <avi <at> redhat.com> writes:
>
>   
>> How much memory do you have on the host?  Is the host swapping?  Can you 
>> post 'vmstat 1' output while Windows is opening the start menu?
>>
>>     
>
> I have 8GB RAM on the host, and i've run htop and the swap space is not
being
> used at all.  I have a total of 4 guest (2 windows + 2 linux)
>
> I will have to try the 'vmstat 1' command tonight when i get home from
work.

Provide kvm_stat output as well.

-- 
error compiling committee.c: too many arguments to function

RE: XP guests sluggish after around 1 week of uptime

[jmandawg]

Ok, it started doing it again, and I actually just rebooted the host on
Friday Morning.  Here are the outputs:

procs -----------memory---------- ---swap-- -----io---- -system--
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
wa
 0  1    204  49520 1008796 2068728    0    0    21    43   33   67  8  7 81
4
 1  1    204  49196 1008824 2069256    0    0   112   712  401 8968  5 12 33
49
 0  1    204  47824 1008836 2070616    0    0   100  1716  386 10048  9 11
40 41
 0  1    204  47760 1008836 2070640    0    0     4    48   73 8797  9 10 46
34
 0  1    204  47588 1008836 2070640    0    0     0    24   66 9007  5  9 42
43
 0  1    204  47488 1008836 2070660    0    0     4   104   80 8641  8 10 41
41
 0  1    204  47464 1008836 2070668    0    0     0    32   40 8812  8  9 45
39
 1  2    204  47240 1008844 2070712    0    0     4   104   71 8843  7  9 21
62
 3  1    204  47388 1008844 2070720    0    0     0    16   37 8786  8  7 43
43
 0  0    204  47016 1008844 2070796    0    0     4    36   69 8986 10  6 67
17
 0  0    204  46940 1008844 2070796    0    0     4    84   43 8720 10  7 48
35
 0  1    204  46940 1008844 2070860    0    0     0    88   70 8732 10  8 56
26
 0  2    204  46892 1008844 2070860    0    0     0    16   34 8855  5  9 42
45
 0  1    204  47164 1008856 2070920    0    0     4    88   64 7857  9  8 39
44
 0  1    204  47164 1008860 2070924    0    0     0    12   34 6123  4  3 44
48
 0  0    204  47016 1008860 2070924    0    0     4    44   75 8745 11 10 49
30
 0  1    204  47064 1008860 2070976    0    0     0    88   34 8840 10  9 51
30
 0  1    204  47288 1008860 2070976    0    0     4    76   59 8820  5 11 39
45
 0  1    204  46968 1008868 2071044    0    0     4    24   38 6535  2  3 51
44
 0  1    204  46668 1008868 2071048    0    0     4    84   57 7942  6  7 41
46
 1  1    204  46592 1008868 2071176    0    0     4    84   41 8969 10  8 41
41
procs -----------memory---------- ---swap-- -----io---- -system--
----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
wa
 1  1    204  46792 1008868 2071176    0    0     0     8   56 9162 14  6 43
37
 2  1    204  46148 1008876 2071992    0    0    92  1004  229 9344  9  7 44
39
 1  1    204  45948 1008876 2072004    0    0    12    80   70 7934 25  9 26
40
 2  1    204  46072 1008876 2072068    0    0     4    64   38 8642  8  7 42
42
 3  1    204  45776 1008876 2072116    0    0     4    52   57 8655  8 11 42
39
 0  0    204  45924 1008876 2072212    0    0     4    36   48 8632  7  7 64
22
 0  1    204  45776 1008880 2072208    0    0     0   120  133 8054 10  6 40
44
 0  2    204  45700 1008884 2072276    0    0     4    72  114 7845  6  8 35
52
 1  1    204  52744 1008892 2065788    0    0    60   924  261 9307  8  8 30
54
 3  1    204  50872 1008904 2066632    0    0   128  1996  571 10162 13 17
32 38
 0  1    204  50548 1008928 2067232    0    0   332   896  343 7798 13 14 38
34
 0  1    204  50688 1008928 2067340    0    0     8    92   49 7719 16 17 48
18
 0  2    204  50564 1008928 2067404    0    0     4   132  185 9020  9  8 48
36
 1  1    204  50456 1008928 2067480    0    0     4    36   42 9006 10  8 24
58
 1  1    204  50464 1008928 2067480    0    0     0    52   64 7598 22 17 18
42
 1  2    204  50284 1008932 2067476    0    0     4    76   50 8743 11  9 22
58
 0  1    204  50192 1008936 2067508    0    0     8    12   71 8419  9 12 23
56
 0  1    204  50324 1008936 2067512    0    0     0   124   45 7672 26 16 37
21
 2  1    204  50440 1008936 2067520    0    0     8    48   83 9076  9  7 51
33
 1  1    204  50564 1008936 2067524    0    0    12    24   34 8741 12 11 38
40
 0  2    204  50564 1008944 2067532    0    0     0    48   62 8829 10  8 36
46


kvm statistics

 efer_reload                 17       0
 exits               3821966006   18394
 fpu_reload           142068215     725
 halt_exits           278799576    1487
 halt_wakeup           65717572     329
 host_state_reload    842507576    4219
 hypercalls                   0       0
 insn_emulation        62556294     206
 insn_emulation_fail       1015       0
 invlpg                14737923       2
 io_exits            1322828696    6784
 irq_exits             45319236     249
 irq_injections       311333395    1610
 irq_window             9206502      35
 kvm_request_irq              0       0
 largepages                   0       0
 mmio_exits            24868562      64
 mmu_cache_miss         1099292       1
 mmu_flooded             481532       0
 mmu_pde_zapped          723088       0
 mmu_pte_updated        5740122       0
 mmu_pte_write          6589738       0
 mmu_recycled             23245       0
 mmu_shadow_zapped      1347124       1
 mmu_unsync               11803       0
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed              95632964     282
 pf_guest              37388330     129
 remote_tlb_flush             7       0
 request_nmi                  0       0
 signal_exits                 5       0
 tlb_flush           1533760876    7017

kvm statistics

 efer_reload                 17       0
 exits               3822814952   23106
 fpu_reload           142094651     811
 halt_exits           278856080    1446
 halt_wakeup           65731489     341
 host_state_reload    842686149    4763
 hypercalls                   0       0
 insn_emulation        62568224     553
 insn_emulation_fail       1015       0
 invlpg                14741639     123
 io_exits            1323153639    7657
 irq_exits             45329372     252
 irq_injections       311402755    1701
 irq_window             9214599      61
 kvm_request_irq              0       0
 largepages                   0       0
 mmio_exits            24871965     400
 mmu_cache_miss         1100097      10
 mmu_flooded             482025       6
 mmu_pde_zapped          723420       4
 mmu_pte_updated        5740727       8
 mmu_pte_write          6590675      12
 mmu_recycled             23287       0
 mmu_shadow_zapped      1348261      13
 mmu_unsync               11803       1
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed              95663549     732
 pf_guest              37398263     274
 remote_tlb_flush             7       0
 request_nmi                  0       0
 signal_exits                 5       0
 tlb_flush           1534079607   10076


kvm statistics

 efer_reload                 17       0
 exits               3819143501   18578
 fpu_reload           141974265     694
 halt_exits           278594554    1499
 halt_wakeup           65669505     337
 host_state_reload    841893357    4180
 hypercalls                   0       0
 insn_emulation        62517121     218
 insn_emulation_fail       1015       0
 invlpg                14729583      80
 io_exits            1321824794    6220
 irq_exits             45282278     220
 irq_injections       311098803    1608
 irq_window             9193364      36
 kvm_request_irq              0       0
 largepages                   0       0
 mmio_exits            24856333      54
 mmu_cache_miss         1098433       0
 mmu_flooded             481127       0
 mmu_pde_zapped          722788       0
 mmu_pte_updated        5739239       2
 mmu_pte_write          6588555       2
 mmu_recycled             23224       0
 mmu_shadow_zapped      1346024       1
 mmu_unsync               11790      -2
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed              95573807     218
 pf_guest              37370248      35
 remote_tlb_flush             7       0
 request_nmi                  0       0
 signal_exits                 5       0
 tlb_flush           1532611984    7928

Any suggestions would be appreciated, I REALLY don't want to have to go back
to vmware.

Thanks in advance,

-J


-----Original Message-----
From: Avi Kivity [mailto:avi@redhat.com] 
Sent: Tuesday, January 13, 2009 4:29 AM
To: Jay
Cc: kvm@vger.kernel.org
Subject: Re: XP guests sluggish after around 1 week of uptime

Please keep the cc list when replying (use reply-all).


Jay wrote:
> Avi Kivity <avi <at> redhat.com> writes:
>
>   
>> How much memory do you have on the host?  Is the host swapping?  Can you 
>> post 'vmstat 1' output while Windows is opening the start menu?
>>
>>     
>
> I have 8GB RAM on the host, and i've run htop and the swap space is not
being
> used at all.  I have a total of 4 guest (2 windows + 2 linux)
>
> I will have to try the 'vmstat 1' command tonight when i get home from
work.

Provide kvm_stat output as well.

-- 
error compiling committee.c: too many arguments to function

RE: XP guests sluggish after around 1 week of uptime

[jmandawg]

It's definitely some kind of Disk IO problem, I can't even copy a 700MB file
on that guest without waiting a loooong time.  Running bonnie on my linux
guest shows considerable slow down in disk performance.

Results after a fresh Reboot of the Host:

root@wserver:~# cat bonnieresults.txt
Version 1.03b       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
wserver        1G 10617  23 12241   1 10328   2 47915  93 301061  93  5095
98
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
+++
wserver,1G,10617,23,12241,1,10328,2,47915,93,301061,93,5094.8,98,16,+++++,++
+,                    +++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

Results after noticing considerable slow down on other guest machine:

root@wserver:~# cat bonnieresults_SLOW.txt
Version 1.03b       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
wserver        1G  2252   3  3771   0  6844   1 46057  70 355858  50 752.3
7
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
+++
wserver,1G,2252,3,3771,0,6844,1,46057,70,355858,50,752.3,7,16,+++++,+++,++++
+,                    +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

After a reboot of the host, they guest are fine again, but for how long...

Any advice or suggestions?

Thanks,

-J

-----Original Message-----
From: Avi Kivity [mailto:avi@redhat.com] 
Sent: Tuesday, January 13, 2009 4:29 AM
To: Jay
Cc: kvm@vger.kernel.org
Subject: Re: XP guests sluggish after around 1 week of uptime

Please keep the cc list when replying (use reply-all).


Jay wrote:
> Avi Kivity <avi <at> redhat.com> writes:
>
>   
>> How much memory do you have on the host?  Is the host swapping?  Can you 
>> post 'vmstat 1' output while Windows is opening the start menu?
>>
>>     
>
> I have 8GB RAM on the host, and i've run htop and the swap space is not
being
> used at all.  I have a total of 4 guest (2 windows + 2 linux)
>
> I will have to try the 'vmstat 1' command tonight when i get home from
work.

Provide kvm_stat output as well.

-- 
error compiling committee.c: too many arguments to function

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

jmandawg wrote:
> It's definitely some kind of Disk IO problem, I can't even copy a 700MB file
> on that guest without waiting a loooong time.  Running bonnie on my linux
> guest shows considerable slow down in disk performance.
>
> Results after a fresh Reboot of the Host:
>
> root@wserver:~# cat bonnieresults.txt
> Version 1.03b       ------Sequential Output------ --Sequential Input-
> --Random-
>                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
> %CP
> wserver        1G 10617  23 12241   1 10328   2 47915  93 301061  93  5095
> 98
>                     ------Sequential Create------ --------Random
> Create--------
>                     -Create-- --Read--- -Delete-- -Create-- --Read---
> -Delete--
>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
> %CP
>                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
> +++
> wserver,1G,10617,23,12241,1,10328,2,47915,93,301061,93,5094.8,98,16,+++++,++
> +,                    +++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
>
> Results after noticing considerable slow down on other guest machine:
>
> root@wserver:~# cat bonnieresults_SLOW.txt
> Version 1.03b       ------Sequential Output------ --Sequential Input-
> --Random-
>                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
> %CP
> wserver        1G  2252   3  3771   0  6844   1 46057  70 355858  50 752.3
> 7
>                     ------Sequential Create------ --------Random
> Create--------
>                     -Create-- --Read--- -Delete-- -Create-- --Read---
> -Delete--
>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
> %CP
>                  16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++
> +++
> wserver,1G,2252,3,3771,0,6844,1,46057,70,355858,50,752.3,7,16,+++++,+++,++++
> +,                    +++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
>
> After a reboot of the host, they guest are fine again, but for how long...
>
> Any advice or suggestions?
>   

What does 'top' show for the guests?  I'm interested in qemu's VIRT and 
RES columns.

Also, 'vmstat 1' on host and guest while slow copying is happening.


-- 
error compiling committee.c: too many arguments to function

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

jmandawg wrote:
> Ok, it started doing it again, and I actually just rebooted the host on
> Friday Morning.  Here are the outputs:
>
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
> wa
>  0  1    204  49520 1008796 2068728    0    0    21    43   33   67  8  7 81
> 4
>  1  1    204  49196 1008824 2069256    0    0   112   712  401 8968  5 12 33
> 49
>  0  1    204  47824 1008836 2070616    0    0   100  1716  386 10048  9 11
> 40 41
>  0  1    204  47760 1008836 2070640    0    0     4    48   73 8797  9 10 46
> 34
>  0  1    204  47588 1008836 2070640    0    0     0    24   66 9007  5  9 42
> 43
>  0  1    204  47488 1008836 2070660    0    0     4   104   80 8641  8 10 41
> 41
>  0  1    204  47464 1008836 2070668    0    0     0    32   40 8812  8  9 45
> 39
>  1  2    204  47240 1008844 2070712    0    0     4   104   71 8843  7  9 2

There is a lot of wait time (last column, wrapped around to the first in 
your output).  Not a lot of real I/O though.  Can you describe your I/O 
subsystem?  Are you using NFS?


-- 
error compiling committee.c: too many arguments to function

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

(re-add kvm@vger)

jmandawg wrote:
>> -----Original Message-----
>> From: Avi Kivity [mailto:avi@redhat.com]
>> Sent: Monday, January 19, 2009 7:06 AM
>> To: jmandawg
>> Cc: kvm@vger.kernel.org
>> Subject: Re: XP guests sluggish after around 1 week of uptime
>>
>> There is a lot of wait time (last column, wrapped around to the first in
>> your output).  Not a lot of real I/O though.  Can you describe your I/O
>> subsystem?  Are you using NFS?
>>
>>     
>
>   
>> What does 'top' show for the guests?  I'm interested in qemu's VIRT and RES
>>     
> columns.
>   
>> Also, 'vmstat 1' on host and guest while slow copying is happening.
>>     
>
>
> I'm using EXT3 on LVM on top of RAID 5.  I can try moving one of the guest
> images to a different disk without the RAID 5 if you think that would help.
>   

Software RAID or hardware RAID?

I think the root cause is the RAID here, not kvm.

Some suggestions:
- use the raw format directly on top of LVM.  That is, use -drive 
file=/dev/volgroup/logvol.
- if you use the raw format, disable disk caching: -drive 
file=/dev/volgroup/logvol,cache=off
- try non-RAID5 storage

To migrate your volumes, use

  qemu-img convert /path/to/image.qcow -O raw /dev/volgroup/logvol

Be sure to size the volume appropriately (same size as the virtual image).
> I just restarted the host machine yesterday, so the file copies are not
> running as slow as before, but still slow.
> Here is the current output of top during a file copy:
>
> top - 07:31:53 up 18:09,  4 users,  load average: 0.74, 0.57, 0.46
> Tasks: 157 total,   1 running, 156 sleeping,   0 stopped,   0 zombie
> Cpu(s):  6.7%us, 12.7%sy,  0.0%ni, 59.6%id, 19.9%wa,  0.0%hi,  1.0%si,
> 0.0%st
> Mem:   8190236k total,  8141100k used,    49136k free,  1246732k buffers
> Swap:  8388600k total,      228k used,  8388372k free,  2418596k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  5945 root      20   0 2131m 2.0g 1924 D   13 25.8  44:15.71 kvm
>   

S=D: waiting for disk.  Low cpu usage as expected.

>
> Here is the current output of vmstat 1 during a file copy:
> procs -----------memory---------- ---swap-- -----io---- -system--
> ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id
> wa
>  0  1    228  46008 1210608 2466132    0    0  2308  8024 2695 13290  7 17
> 63 13
>   

This seems fine if a bit low.  You'll get better numbers sitting 
directly atop LVM.

-- 
error compiling committee.c: too many arguments to function

RE: XP guests sluggish after around 1 week of uptime

[jmandawg]

> -----Original Message-----
> From: Avi Kivity [mailto:avi@redhat.com]
> Sent: Monday, January 19, 2009 7:46 AM
> To: jmandawg
> Cc: KVM list
> Subject: Re: XP guests sluggish after around 1 week of uptime
> 
> (re-add kvm@vger)
> 
> 
> Software RAID or hardware RAID?
> 
> I think the root cause is the RAID here, not kvm.
> Some suggestions:
> - use the raw format directly on top of LVM.  That is, use -drive
> file=/dev/volgroup/logvol.
> - if you use the raw format, disable disk caching: -drive
> file=/dev/volgroup/logvol,cache=off
> - try non-RAID5 storage
> 
> To migrate your volumes, use
> 
>   qemu-img convert /path/to/image.qcow -O raw /dev/volgroup/logvol
> 
> Be sure to size the volume appropriately (same size as the virtual image).
> 
> This seems fine if a bit low.  You'll get better numbers sitting
> directly atop LVM.
> 
> --
> error compiling committee.c: too many arguments to function

Yes, it is software raid, I will try moving the guest images to one of my
other disks that is not RAIDed on their own LVM volume.

Does the LVM logical volume size need to be equal the guest disk size
exactly?

I will let you know the results.

Thanks again for the help,

-J  

Re: XP guests sluggish after around 1 week of uptime

[Avi Kivity]

jmandawg wrote:
> Does the LVM logical volume size need to be equal the guest disk size
> exactly?
>   

It can be a larger.  In fact, I recommend making it larger, because if 
it is smaller even by a small amount, data loss is likely.

-- 
error compiling committee.c: too many arguments to function