Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3)

["David S. Ahern" <daahern@cisco.com>]

Does the fact that the hugemem kernel works just fine have any bearing
on your options? Or rather, is there something unique about the way
kscand works in the hugemem kernel that its performance is ok?

I mentioned last month (so without your first patch) that running the
hugemem kernel showed a remarkable improvement in performance compared
to the standard smp kernel. Over the weekend I ran a test with your
first patch and with the flood detector at 3 (I have not run a case with
the detector at 5) and performance with the hugemem was even better in
the sense that 1-minute averages of guest system time show no noticeable
spikes.

In an earlier post I showed a diff in the config files for the standard
SMP and hugemem kernels. See:
http://article.gmane.org/gmane.comp.emulators.kvm.devel/16944/

david



Avi Kivity wrote:
> David S. Ahern wrote:
>>> [dsa] No. I saw the same problem with the flood count at 5. The
>>> attachment in the last email shows kvm_stat data during a kscand event.
>>> The data was collected with the patch you posted. With the flood count
>>> at 3 the mmu cache/flood counters are in the 18,000/sec and pte updates
>>> at ~50,000/sec and writes at 70,000/sec. With the flood count at 5
>>> mmu_cache/flood drops to 0 and pte updates and writes both hit
>>> 180,000+/second. In both cases these last for 30 seconds or more. I only
>>> included data for the onset as it's pretty flat during the kscand
>>> activity.
>>>     
> 
> It makes sense.  We removed a flooding false positive, and introduced a
> false negative.
> 
> The guest access sequence is:
> - point kmap pte at page table
> - use the new pte to access the page table
> 
> Prior to the patch, the mmu didn't see the 'use' part, so it concluded
> the kmap pte would be better off unshadowed.  This shows up as a high
> flood count.
> 
> After the patch, this no longer happens, so the sequence can repreat for
> long periods.  However the pte that is the result of the 'use' part is
> never accessed, so it should be detected as flooded!  But our flood
> detection mechanism looks at one page at a time (per vcpu), while there
> are two pages involved here.
> 
> There are (at least) three options available:
> - detect and special-case this scenario
> - change the flood detector to be per page table instead of per vcpu
> - change the flood detector to look at a list of recently used page
> tables instead of the last page table
> 
> I'm having a hard time trying to pick between the second and third options.
>