From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [kvm-devel] performance with guests running 2.4 kernels (specifically RHEL3) Date: Sun, 22 Jun 2008 09:34:06 +0300 Message-ID: <485DF25E.6060400@qumranet.com> References: <48054518.3000104@cisco.com> <480E492B.3060500@cisco.com> <480EEDA0.3080209@qumranet.com> <480F546C.2030608@cisco.com> <481215DE.3000302@cisco.com> <20080428181550.GA3965@dmt> <4816617F.3080403@cisco.com> <4817F30C.6050308@cisco.com> <48184228.2020701@qumranet.com> <481876A9.1010806@cisco.com> <48187903.2070409@qumranet.com> <4826E744.1080107@qumranet.com> <4826F668.6030305@qumranet.com> <48290FC2.4070505@cisco.com> <48294272.5020801@qumranet.com> <482B4D29.7010202@cisco.com> <482C1633.5070302@qumranet.com> <482E5F9C.6000207@cisco.com> <482FCEE1.5040306@qumranet.com> <4830F90A.1020809@cisco.com> <4830FE8D.6010006@cisco.com> <48318E64.8090706@qumranet.com> <4832DDEB.4000100@qumranet.com> <4835EEF5.9010600@cisco.com> <483D391F.7050007@qumranet.com> <483EDCEE.6070307@cisco.com> <4 841094A.8090507@qumranet.com> <484422EE.5090501@cisco.com> <4847A5B8.6020503@qumranet.com> <48481250.6060005@cisco.com> <4849686E.8000402@qumranet.com> <4859DEA6.5010503@cisco.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------030701030102070007050600" Cc: kvm@vger.kernel.org To: "David S. Ahern" Return-path: Received: from il.qumranet.com ([212.179.150.194]:30836 "EHLO il.qumranet.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751145AbYFVGd5 (ORCPT ); Sun, 22 Jun 2008 02:33:57 -0400 In-Reply-To: <4859DEA6.5010503@cisco.com> Sender: kvm-owner@vger.kernel.org List-ID: This is a multi-part message in MIME format. --------------030701030102070007050600 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit David S. Ahern wrote: > Avi: > > We did not get a chance to do this at the Forum. I'd be interested in > whatever options you have for reducing the scan time further (e.g., try > to get scan time down to 1-2 seconds). > > I'm unlikely to get time to do this properly for at least a week, as this will be quite difficult and I'm already horribly backlogged. However there's an alternative option, modifying the source and getting it upstreamed, as I think RHEL 3 is still maintained. The attached patch (untested) should give a 3X boost for kmap_atomics, by folding the two accesses to set the pte into one, and by dropping the access that clears the pte. Unfortunately it breaks the ABI, since external modules will inline the original kmap_atomic() which expects the pte to be cleared. This can be worked around by allocating new fixmap slots for kmap_atomic with the new behavior, and keeping the old slots with the old behavior, but we should first see if the maintainers are open to performance optimizations targeting kvm. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. --------------030701030102070007050600 Content-Type: text/x-patch; name="faster-2.4-kmap_atomic.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="faster-2.4-kmap_atomic.patch" --- include/asm-i386/atomic_kmap.h.orig 2007-06-12 00:24:29.000000000 +0300 +++ include/asm-i386/atomic_kmap.h 2008-06-22 09:23:26.000000000 +0300 @@ -51,18 +51,13 @@ static inline void *__kmap_atomic(struct idx = type + KM_TYPE_NR*smp_processor_id(); vaddr = __fix_to_virt(FIX_KMAP_BEGIN + idx); -#if HIGHMEM_DEBUG - if (!pte_none(*(kmap_pte-idx))) - out_of_line_bug(); -#else /* * Performance optimization - do not flush if the new * pte is the same as the old one: */ if (pte_val(*(kmap_pte-idx)) == pte_val(mk_pte(page, kmap_prot))) return (void *) vaddr; -#endif - set_pte(kmap_pte-idx, mk_pte(page, kmap_prot)); + set_pte_atomic(kmap_pte-idx, mk_pte(page, kmap_prot)); __flush_tlb_one(vaddr); return (void*) vaddr; @@ -77,12 +72,6 @@ static inline void __kunmap_atomic(void if (vaddr != __fix_to_virt(FIX_KMAP_BEGIN+idx)) out_of_line_bug(); - /* - * force other mappings to Oops if they'll try to access - * this pte without first remap it - */ - pte_clear(kmap_pte-idx); - __flush_tlb_one(vaddr); #endif } --------------030701030102070007050600--