From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: [PATCH][SVM] Lazy fast syscall MSR save/restore Date: Fri, 27 Apr 2007 09:23:14 +0300 Message-ID: <463196D2.6060300@qumranet.com> References: <46318FF0.7000207@us.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel To: Anthony Liguori Return-path: In-reply-to: <46318FF0.7000207-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Anthony Liguori wrote: > Howdy, > > The attached patch only reloads the host fast syscall MSRs when we > leave KVM (when we put_cpu()). Since we'll never execute a > sys{enter,call,ret,exit} until we put_cpu() in the host, this should > be safe. > > This has a huge impact on in-kernel vmexit performance. I see a > savings of about a 1,000 cycles going from 4,700 to about 3,700 or ~21%. > Which means on a 3GHz Opteron we're getting close to a million vmexits per second when spinning... > > static void svm_vcpu_load(struct kvm_vcpu *vcpu) > { > - int cpu; > + int cpu, i; > > cpu = get_cpu(); > if (unlikely(cpu != vcpu->cpu)) { > @@ -626,10 +626,18 @@ > vcpu->svm->vmcb->control.tsc_offset += delta; > vcpu->cpu = cpu; > } > + > + for (i = 0; i < NR_HOST_SAVE_USER_MSRS; i++) > + rdmsrl(host_save_user_msrs[i], vcpu->svm->host_user_msrs[i]); > } > All the msrs you mentioned are constant throughout the lifetime of the host and don't vary by processor, so the save loop can be moved to module init (it's in vcpu init for vmx). Not sure how expensive rdmsr is, though, so it may not be worthwhile to move it, see below. > > Index: kernel/drivers/kvm/kvm_svm.h > =================================================================== > --- kernel.orig/drivers/kvm/kvm_svm.h 2007-04-26 18:07:09.949899400 -0500 > +++ kernel/drivers/kvm/kvm_svm.h 2007-04-27 00:38:30.310342040 -0500 > @@ -11,15 +11,19 @@ > > static const u32 host_save_msrs[] = { > #ifdef CONFIG_X86_64 > - MSR_STAR, MSR_LSTAR, MSR_CSTAR, MSR_SYSCALL_MASK, MSR_KERNEL_GS_BASE, > - MSR_FS_BASE, MSR_GS_BASE, > + MSR_KERNEL_GS_BASE, MSR_FS_BASE, MSR_GS_BASE, > MSR_KERNEL_GS_BASE, as its name so clearly indicates, is a user msr. True, while executing in userspace it holds a kernel address, but once we hit the kernel we execute swapgs, which writes the value of MSR_GS_BASE into MSR_KERNEL_GS_BASE. So, we can save/restore it on vcpu_load()/vcpu_put(). MSR_FS_BASE is a also a user msr (MSR_GS_BASE is not, since it holds the value of MSR_KERNEL_GS_BASE while in the kernel and is used to access the pda). An additional consideration is that non-longmode guests will not touch longmode msrs, so the saving and loading thereof can be avoided completely. The vmx code does that. It's worthy of an independenet patch, though. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/