From mboxrd@z Thu Jan  1 00:00:00 1970
From: "David S. Ahern" <daahern@cisco.com>
Subject: Re: performance with guests running 2.4
 kernels	(specifically RHEL3)
Date: Fri, 25 Apr 2008 11:33:18 -0600
Message-ID: <481215DE.3000302@cisco.com>
References: <48054518.3000104@cisco.com>
	<4805BCF1.6040605@qumranet.com>	<4807BD53.6020304@cisco.com>
	<48085485.3090205@qumranet.com> <480C188F.3020101@cisco.com>
	<480C5C39.4040300@qumranet.com> <480E492B.3060500@cisco.com>
	<480EEDA0.3080209@qumranet.com> <480F546C.2030608@cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Cc: kvm-devel <kvm-devel@lists.sourceforge.net>
To: Avi Kivity <avi@qumranet.com>
Return-path: <kvm-devel-bounces@lists.sourceforge.net>
In-Reply-To: <480F546C.2030608@cisco.com>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/kvm-devel>,
	<mailto:kvm-devel-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=kvm-devel>
List-Post: <mailto:kvm-devel@lists.sourceforge.net>
List-Help: <mailto:kvm-devel-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/kvm-devel>,
	<mailto:kvm-devel-request@lists.sourceforge.net?subject=subscribe>
Sender: kvm-devel-bounces@lists.sourceforge.net
Errors-To: kvm-devel-bounces@lists.sourceforge.net
List-Id: kvm.vger.kernel.org

David S. Ahern wrote:
> Avi Kivity wrote:
>
>> David S. Ahern wrote:
>>
>>> I added the traces and captured data over another apparent lockup of
>>> the guest.
>>> This seems to be representative of the sequence (pid/vcpu removed).
>>>
>>> (+4776)  VMEXIT         [ exitcode = 0x00000000, rip = 0x00000000
>>> c016127c ]
>>> (+   0)  PAGE_FAULT     [ errorcode = 0x00000003, virt = 0x00000000
>>> c0009db4 ]
>>> (+3632)  VMENTRY
>>> (+4552)  VMEXIT         [ exitcode = 0x00000000, rip = 0x00000000
>>> c016104a ]
>>> (+   0)  PAGE_FAULT     [ errorcode = 0x0000000b, virt = 0x00000000
>>> fffb61c8 ]
>>> (+   54928)  VMENTRY
>>>
>> Can you oprofile the host to see where the 54K cycles are spent?
>>

Most of the cycles (~80% of that 54k+) are spent in paging64_prefetch_page():

        for (i = 0; i < PT64_ENT_PER_PAGE; ++i) {
                gpa_t pte_gpa = gfn_to_gpa(sp->gfn);
                pte_gpa += (i+offset) * sizeof(pt_element_t);

                r = kvm_read_guest_atomic(vcpu->kvm, pte_gpa, &pt,
                                          sizeof(pt_element_t));
                if (r || is_present_pte(pt))
                        sp->spt[i] = shadow_trap_nonpresent_pte;
                else
                        sp->spt[i] = shadow_notrap_nonpresent_pte;
        }

This loop is run 512 times and takes a total of ~45k cycles, or ~88 cycles per
loop.

This function gets run >20,000/sec during some of the kscand loops.

david

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone