Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3)

["David S. Ahern" <daahern@cisco.com>]

Andrea Arcangeli wrote:
> On Thu, May 29, 2008 at 01:01:06PM +0300, Avi Kivity wrote:
>> No, two:
>>
>> static inline void set_pte(pte_t *ptep, pte_t pte)
>> {
>>        ptep->pte_high = pte.pte_high;
>>        smp_wmb();
>>        ptep->pte_low = pte.pte_low;
>> }
> 
> Right, that can be 2 or 1 depending on PAE non-PAE, other 2.4
> enterprise distro with pte-highmem ships non-PAE kernels by default.

RHEL3U8 has CONFIG_X86_PAE set.

<snipped>

>>>>> - an access to tear down the kmap
>>>>>       
>>> Yep, pte_clear on the fixmap pte_t followed by an invlpg (if that
>>> matters).
>>>   
>> Looking at the code, that only happens if CONFIG_HIGHMEM_DEBUG is set.
> 
> 2.4 yes. 2.6 is will do similar to CONFIG_HIGHMEM_DEBUG.
> 
> 2.4 without HIGHMEM_DEBUG sets the pte and invlpg in kmap_atomic and
> does nothing in kunmap_atomic.
> 
> 2.6 sets the pte in kmap_atomic, and clears it+invlpg in kunmap_atomic.

CONFIG_DEBUG_HIGHMEM is set.

<snipped>

>> One possible optimization is that if we see the first part of the kmap 
>> instantiation, we emulate a few more instructions before returning to the 
>> guest.  Xen does this IIRC.
> 
> Surely this would avoid 1 wrprotect fault per kmap_atomic, but I'm not
> sure if 32bit PAE is that important to do this. Most 32bit enterprise
> kernels I worked aren't compiled with PAE, only one called bigsmp is.

RHEL3 has a hugemem kernel which basically just enables the 4G/4G split.
My guest with the hugemem kernel runs much better than the standard smp
kernel.


If you care to download it the RHEL3U8 kernel source is posted here:
ftp://ftp.redhat.com/pub/redhat/linux/updates/enterprise/3AS/en/os/SRPMS/kernel-2.4.21-47.EL.src.rpm

Red Hat does heavily patch kernels, so they will be dramatically
different than the kernel.org kernel with the same number.

david