From mboxrd@z Thu Jan 1 00:00:00 1970 From: "David S. Ahern" Subject: Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3) Date: Thu, 29 May 2008 09:11:10 -0600 Message-ID: <483EC78E.3000302@cisco.com> References: <48318E64.8090706@qumranet.com> <4832DDEB.4000100@qumranet.com> <4835EEF5.9010600@cisco.com> <483D391F.7050007@qumranet.com> <483D6898.2050605@cisco.com> <20080528144850.GX27375@duo.random> <483D7C45.5020300@qumranet.com> <483D7D8D.3030309@cisco.com> <20080528170410.GC8086@duo.random> <483E7EE2.8010508@qumranet.com> <20080529142703.GJ8086@duo.random> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Avi Kivity , kvm@vger.kernel.org To: Andrea Arcangeli Return-path: Received: from sj-iport-3.cisco.com ([171.71.176.72]:18100 "EHLO sj-iport-3.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752636AbYE2PLc (ORCPT ); Thu, 29 May 2008 11:11:32 -0400 In-Reply-To: <20080529142703.GJ8086@duo.random> Sender: kvm-owner@vger.kernel.org List-ID: Andrea Arcangeli wrote: > On Thu, May 29, 2008 at 01:01:06PM +0300, Avi Kivity wrote: >> No, two: >> >> static inline void set_pte(pte_t *ptep, pte_t pte) >> { >> ptep->pte_high = pte.pte_high; >> smp_wmb(); >> ptep->pte_low = pte.pte_low; >> } > > Right, that can be 2 or 1 depending on PAE non-PAE, other 2.4 > enterprise distro with pte-highmem ships non-PAE kernels by default. RHEL3U8 has CONFIG_X86_PAE set. >>>>> - an access to tear down the kmap >>>>> >>> Yep, pte_clear on the fixmap pte_t followed by an invlpg (if that >>> matters). >>> >> Looking at the code, that only happens if CONFIG_HIGHMEM_DEBUG is set. > > 2.4 yes. 2.6 is will do similar to CONFIG_HIGHMEM_DEBUG. > > 2.4 without HIGHMEM_DEBUG sets the pte and invlpg in kmap_atomic and > does nothing in kunmap_atomic. > > 2.6 sets the pte in kmap_atomic, and clears it+invlpg in kunmap_atomic. CONFIG_DEBUG_HIGHMEM is set. >> One possible optimization is that if we see the first part of the kmap >> instantiation, we emulate a few more instructions before returning to the >> guest. Xen does this IIRC. > > Surely this would avoid 1 wrprotect fault per kmap_atomic, but I'm not > sure if 32bit PAE is that important to do this. Most 32bit enterprise > kernels I worked aren't compiled with PAE, only one called bigsmp is. RHEL3 has a hugemem kernel which basically just enables the 4G/4G split. My guest with the hugemem kernel runs much better than the standard smp kernel. If you care to download it the RHEL3U8 kernel source is posted here: ftp://ftp.redhat.com/pub/redhat/linux/updates/enterprise/3AS/en/os/SRPMS/kernel-2.4.21-47.EL.src.rpm Red Hat does heavily patch kernels, so they will be dramatically different than the kernel.org kernel with the same number. david