Re: performance with guests running 2.4 kernels (specifically RHEL3)

David S. Ahern
2008-04-21T04:31:11Z

I added the traces and captured data over another apparent lockup of the guest.
This seems to be representative of the sequence (pid/vcpu removed).

(+4776) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016127c ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ]
(+3632) VMENTRY
(+4552) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016104a ]
(+ 0) PAGE_FAULT [ errorcode = 0x0000000b, virt = 0x00000000 fffb61c8 ]
(+ 54928) VMENTRY
(+4568) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610e7 ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ]
(+ 0) PTE_WRITE [ gpa = 0x00000000 00009db4 gpte = 0x00000000 41c5d363 ]
(+8432) VMENTRY
(+3936) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610ee ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db0 ]
(+ 0) PTE_WRITE [ gpa = 0x00000000 00009db0 gpte = 0x00000000 00000000 ]
(+ 13832) VMENTRY

(+5768) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016127c ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ]
(+3712) VMENTRY
(+4576) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016104a ]
(+ 0) PAGE_FAULT [ errorcode = 0x0000000b, virt = 0x00000000 fffb61d0 ]
(+ 0) PTE_WRITE [ gpa = 0x00000000 3d5981d0 gpte = 0x00000000 3d55d047 ]
(+ 65216) VMENTRY
(+4232) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610e7 ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ]
(+ 0) PTE_WRITE [ gpa = 0x00000000 00009db4 gpte = 0x00000000 3d598363 ]
(+8640) VMENTRY
(+3936) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610ee ]
(+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db0 ]
(+ 0) PTE_WRITE [ gpa = 0x00000000 00009db0 gpte = 0x00000000 00000000 ]
(+ 14160) VMENTRY

I can forward a more complete time snippet if you'd like. vcpu0 + corresponding
vcpu1 files have 85000 total lines and compressed the files total ~500k.

I did not see the FLOODED trace come out during this sample though I did bump
the count from 3 to 4 as you suggested.

Correlating rip addresses to the 2.4 kernel:

c0160d00-c0161290 = page_referenced

It looks like the event is kscand running through the pages. I suspected this
some time ago, and tried tweaking the kscand_work_percent sysctl variable. It
appeared to lower the peak of the spikes, but maybe I imagined it. I believe
lowering that value makes kscand wake up more often but do less work (page
scanning) each time it is awakened.

david

Avi Kivity wrote:
> Can you add a trace at mmu_guess_page_from_pte_write(), right before "if
> (is_present_pte(gpte))"? I'm interested in gpa and gpte. Also a trace
> at kvm_mmu_pte_write(), where it sets flooded = 1 (hmm, try to increase
> the 3 to 4 in the line right above that, maybe the fork detector is
> misfiring).

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.