From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: performance with guests running 2.4 kernels (specifically RHEL3) Date: Wed, 23 Apr 2008 18:53:34 +0300 Message-ID: <480F5B7E.5020407@qumranet.com> References: <48054518.3000104@cisco.com> <4805BCF1.6040605@qumranet.com> <4807BD53.6020304@cisco.com> <48085485.3090205@qumranet.com> <480C188F.3020101@cisco.com> <480C5C39.4040300@qumranet.com> <480E492B.3060500@cisco.com> <480EEDA0.3080209@qumranet.com> <480F546C.2030608@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel To: "David S. Ahern" Return-path: In-Reply-To: <480F546C.2030608@cisco.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces@lists.sourceforge.net Errors-To: kvm-devel-bounces@lists.sourceforge.net List-Id: kvm.vger.kernel.org David S. Ahern wrote: > I've continued drilling down with the tracers to answer that question. I have > done runs with tracers in paging64_page_fault and it showed the overhead is with > the fetch() function. On my last run the tracers are in paging64_fetch() as follows: > > 1. after is_present_pte() check > 2. before kvm_mmu_get_page() > 3. after kvm_mmu_get_page() > 4. after if (!metaphysical) {} > > The delta between 2 and 3 shows the cycles to run kvm_mmu_get_page(). The delta > between 3 and 4 shows the cycles to run kvm_read_guest_atomic(), if it is run. > Tracer1 dumps vcpu->arch.last_pt_write_count (a carryover from when the new > tracers were in paging64_page_fault); tracer2 dumps the level, metaphysical and > access variables; tracer5 dumps value in shadow_ent. > > A representative trace sample is: > > (+ 4576) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016104a ] > (+ 0) PAGE_FAULT [ errorcode = 0x0000000b, virt = 0x00000000 fffb6950 ] > (+ 2664) PAGE_FAULT1 [ write_count = 0 ] > (+ 472) PAGE_FAULT2 [ level = 2 metaphysical = 0 access 0x00000007 ] > (+ 50416) PAGE_FAULT3 > (+ 472) PAGE_FAULT4 > (+ 856) PAGE_FAULT5 [ shadow_ent_val = 0x80000000 9276d043 ] > (+ 1528) VMENTRY > (+ 4992) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610e7 ] > (+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ] > (+ 2296) PAGE_FAULT1 [ write_count = 0 ] > (+ 816) PAGE_FAULT5 [ shadow_ent_val = 0x00000000 8a809041 ] > (+ 0) PTE_WRITE [ gpa = 0x00000000 00009db4 gpte = 0x00000000 4176d363 ] > (+ 6424) VMENTRY > (+ 3864) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c01610ee ] > (+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db0 ] > (+ 2496) PAGE_FAULT1 [ write_count = 1 ] > (+ 856) PAGE_FAULT5 [ shadow_ent_val = 0x00000000 8a809041 ] > (+ 0) PTE_WRITE [ gpa = 0x00000000 00009db0 gpte = 0x00000000 00000000 ] > (+ 10248) VMENTRY > (+ 4744) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016127c ] > (+ 0) PAGE_FAULT [ errorcode = 0x00000003, virt = 0x00000000 c0009db4 ] > (+ 2408) PAGE_FAULT1 [ write_count = 2 ] > (+ 760) PAGE_FAULT5 [ shadow_ent_val = 0x00000000 8a809043 ] > (+ 1240) VMENTRY > (+ 4624) VMEXIT [ exitcode = 0x00000000, rip = 0x00000000 c016104a ] > (+ 0) PAGE_FAULT [ errorcode = 0x0000000b, virt = 0x00000000 fffb6950 ] > (+ 2512) PAGE_FAULT1 [ write_count = 0 ] > (+ 496) PAGE_FAULT2 [ level = 2 metaphysical = 0 access 0x00000007 ] > (+ 48664) PAGE_FAULT3 > (+ 472) PAGE_FAULT4 > (+ 856) PAGE_FAULT5 [ shadow_ent_val = 0x80000000 9272d043 ] > (+ 1576) VMENTRY > > So basically every 4th trip through the fetch function it runs > kvm_mmu_get_page(). A summary of the entire trace file shows this function > rarely executes in less than 50,000 cycles. Also, vcpu->arch.last_pt_write_count > is always 0 when the high cycles are hit. > > Ah! The flood detector is not seeing the access through the kmap_atomic() pte, because that access has gone through the emulator. last_updated_pte_accessed(vcpu) will never return true. Can you verify that last_updated_pte_accessed(vcpu) indeed always returns false? -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone