Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3)

[Avi Kivity <avi@qumranet.com>]

David S. Ahern wrote:
> The short answer is that I am still see large system time hiccups in the
> guests due to kscand in the guest scanning its active lists. I do see
> better response for a KVM_MAX_PTE_HISTORY of 3 than with 4. (For
> completeness I also tried a history of 2, but it performed worse than 3
> which is no surprise given the meaning of it.)
>
>
> I have been able to scratch out a simplistic program that stimulates
> kscand activity similar to what is going on in my real guest (see
> attached). The program requests a memory allocation, initializes it (to
> get it backed) and then in a loop sweeps through the memory in chunks
> similar to a program using parts of its memory here and there but
> eventually accessing all of it.
>
> Start the RHEL3/CentOS 3 guest with *2GB* of RAM (or more). The key is
> using a fair amount of highmem. Start a couple of instances of the
> attached. For example, I've been using these 2:
>
> 	memuser 768M 120 5 300
> 	memuser 384M 300 10 600
>
> Together these instances take up a 1GB of RAM and once initialized
> consume very little CPU. On kvm they make kscand and kswapd go nuts
> every 5-15 minutes. For comparison, I do not see the same behavior for
> an identical setup running on esx 3.5.
>   

I haven't been able to reproduce this:

> [root@localhost root]# ps -elf | grep -E 'memuser|kscand'
> 1 S root         7     1  1  75   0    -     0 schedu 10:07 ?        
> 00:00:26 [kscand]
> 0 S root      1464     1  1  75   0    - 196986 schedu 10:20 pts/0   
> 00:00:21 ./memuser 768M 120 5 300
> 0 S root      1465     1  0  75   0    - 98683 schedu 10:20 pts/0    
> 00:00:10 ./memuser 384M 300 10 600
> 0 S root      2148  1293  0  75   0    -   922 pipe_w 10:48 pts/0    
> 00:00:00 grep -E memuser|kscand

The workload has been running for about half an hour, and kswapd cpu 
usage doesn't seem significant.  This is a 2GB guest running with my 
patch ported to kvm.git HEAD.  Guest is has 2G of memory.


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.