Re: [PATCH][SVM] Lazy fast syscall MSR save/restore

[Anthony Liguori <aliguori-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org>]

Avi Kivity wrote:
> Anthony Liguori wrote:
>   
>> Howdy,
>>
>> The attached patch only reloads the host fast syscall MSRs when we
>> leave KVM (when we put_cpu()).  Since we'll never execute a
>> sys{enter,call,ret,exit} until we put_cpu() in the host, this should
>> be safe.
>>
>> This has a huge impact on in-kernel vmexit performance.  I see a
>> savings of about a 1,000 cycles going from 4,700 to about 3,700 or ~21%.
>>
>>     
>
> Which means on a 3GHz Opteron we're getting close to a million vmexits
> per second when spinning...
>
>   
>>  
>>  static void svm_vcpu_load(struct kvm_vcpu *vcpu)
>>  {
>> -	int cpu;
>> +	int cpu, i;
>>  
>>  	cpu = get_cpu();
>>  	if (unlikely(cpu != vcpu->cpu)) {
>> @@ -626,10 +626,18 @@
>>  		vcpu->svm->vmcb->control.tsc_offset += delta;
>>  		vcpu->cpu = cpu;
>>  	}
>> +
>> +	for (i = 0; i < NR_HOST_SAVE_USER_MSRS; i++)
>> +		rdmsrl(host_save_user_msrs[i], vcpu->svm->host_user_msrs[i]);
>>  }
>>   
>>     
>
> All the msrs you mentioned are constant throughout the lifetime of the
> host and don't vary by processor, so the save loop can be moved to
> module init (it's in vcpu init for vmx).  Not sure how expensive rdmsr
> is, though, so it may not be worthwhile to move it, see below.
>
>   
>>  
>> Index: kernel/drivers/kvm/kvm_svm.h
>> ===================================================================
>> --- kernel.orig/drivers/kvm/kvm_svm.h	2007-04-26 18:07:09.949899400 -0500
>> +++ kernel/drivers/kvm/kvm_svm.h	2007-04-27 00:38:30.310342040 -0500
>> @@ -11,15 +11,19 @@
>>  
>>  static const u32 host_save_msrs[] = {
>>  #ifdef CONFIG_X86_64
>> -	MSR_STAR, MSR_LSTAR, MSR_CSTAR, MSR_SYSCALL_MASK, MSR_KERNEL_GS_BASE,
>> -	MSR_FS_BASE, MSR_GS_BASE,
>> +	MSR_KERNEL_GS_BASE, MSR_FS_BASE, MSR_GS_BASE,
>>   
>>     
>
> MSR_KERNEL_GS_BASE, as its name so clearly indicates, is a user msr. 
> True, while executing in userspace it holds a kernel address, but once
> we hit the kernel we execute swapgs, which writes the value of
> MSR_GS_BASE into MSR_KERNEL_GS_BASE.  So, we can save/restore it on
> vcpu_load()/vcpu_put().
>   

This trims another 200 cycles.  Another one I wanted to look at today 
was DEBUGCTL but I can't seem to figure out how it gets loaded by the 
guest.  DEBUGCTL costs 300 cycles to save/restore.

> MSR_FS_BASE is a also a user msr (MSR_GS_BASE is not, since it holds the
> value of MSR_KERNEL_GS_BASE while in the kernel and is used to access
> the pda).
>
> An additional consideration is that non-longmode guests will not touch
> longmode msrs, so the saving and loading thereof can be avoided
> completely.  The vmx code does that.  It's worthy of an independenet
> patch, though.
>   

Regards,

Anthony Liguori


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/