Re: [PATCH][RFC] Improve PIO latency

[Anthony Liguori <aliguori-NZpS4cJIG2HvQtjrzfazuQ@public.gmane.org>]

Dor Laor wrote:
>> Anthony Liguori wrote:
>>     
>>> The attached patch modifies libkvmctl to only make SET_REG/GET_REG
>>> ioctls when needed for PIO instructions.  I was only able to do this
>>> for out instructions because I didn't want to break the kernel ABI.
>>>
>>> I think we should change the API though so we can do this with other
>>> types of IO instructions.  Before this patch, the time line for a PIO
>>> instruction looks something like this:
>>>
>>> All times in nanoseconds and are round trips from the guests
>>> perspective for an out instruction on an AMD X2 4200.
>>>
>>> 1015 - immediately after restoring saving guest registers
>>> 1991 - handled within the kernel in io_interception
>>> 2294 - libkvmctl returns immediately
>>> 2437 - w/ patch
>>> 3311 - w/o patch
>>>
>>> The first data point is the best we could possible do.  The only work
>>> being done after the VMRUN is a VMSAVE/VMLOAD, saving the guest
>>> registers, and restoring the host registers.  The VMSAVE/VMLOAD is
>>> needed so that vmcb->save.eip can be updated.[1]  I played around
>>> reducing the register savings but the differences weren't noticable.
>>>
>>> I suspect that more intelligent handling of things like FPU
>>> save/restore should be able to reduce the second data point.  This
>>> will also improve some other exit paths (like shadow paging).  We
>>> save/restore an awful lot of state considering that we probably
>>>       
> return
>   
>>> back to the guest for the vast majority of exits.
>>>
>>>       
>> These are very encouraging numbers.  I'd expected the vmexit to be more
>> expensive, and fpu save/restore to be less expensive.  Since, as you
>>     
>
>
> I tried to see the performance gain using dd if=/file iflag=direct ...
>   

Have you compared against the host?  I used dbench.

> And didn't get any visible gain. So maybe all the vmexit/disk latency
> are shadowing the performance gain?
> Anthony can you please send the virt bench patch?
>   

You don't need a patched version of virtbench.  Just try use:

virtbench local pio

http://ozlabs.org/~rusty/virtbench

Regards,

Anthony Liguori

>> say, we can eliminate the fpu save/restore in many cases, we have a net
>> win :)
>>
>> The planned userspace api changes will eliminate registers and virtual
>> addresses for pio.  This will both improve performance and make the api
>> more architecture agnostic.
>>
>> I'm applying the patch now, even though it will be obsoleted soon, as
>> it's always nice to have a performance improvement.
>>
>>
>> ps. that [1] is a dangling reference?
>>
>> --
>> Do not meddle in the internals of kernels, for they are subtle and
>>     
> quick to
>   
>> panic.
>>
>>
>> -----------------------------------------------------------------------
>>     
> --
>   
>> Using Tomcat but need to do more? Need to support web services,
>>     
> security?
>   
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier.
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>     
> Geronimo
>   
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=12164
>>     
> 2
>   
>> _______________________________________________
>> kvm-devel mailing list
>> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>>     


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642