From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Subject: performance specs on VT-x vmenter/exit Date: Thu, 6 Dec 2007 13:09:46 -0500 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0123673549==" To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org --===============0123673549== Content-Type: multipart/alternative; boundary="----=_Part_11915_14058545.1196964586625" ------=_Part_11915_14058545.1196964586625 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Greetings Honorable KVM Devs! I am doing research on the performance characteristics of the Intel VMX instructions, specifically with respect to KVM. I need to count the cycles for a VMENTER and VMEXIT instruction under various conditions. (But getting it to work at all would be cool :) So the idea is to drop the cycle count in a register from the VMM, force a VMENTER, drop the count in a different register, and force a VMEXIT. Then get a last count and do appropriate adding/subtracting. I'm working with a 2.6.22.6 vanilla kernel on an Core2Duo with KVM-54. Questions: Is there a better way to get these numbers? If not, where in the KVM source should I aim to put these instructions (file/line number/general area)? Any tips on forcing a vmexit out of the guest immediately after the count? Thank you so much for your time. :dan ------=_Part_11915_14058545.1196964586625 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Greetings Honorable KVM Devs!

I am doing research on the performance characteristics of the Intel VMX instructions, specifically with respect to KVM. I need to count the cycles for a VMENTER and VMEXIT instruction under various conditions. (But getting it to work at all would be cool :)

So the idea is to drop the cycle count in a register from the VMM, force a VMENTER, drop the count in a different register, and force a VMEXIT. Then get a last count and do appropriate adding/subtracting.
I'm working with a 2.6.22.6 vanilla kernel on an Core2Duo with KVM-54.

Questions:
 Is there a better way to get these numbers?
 If not, where in the KVM source should I aim to put these instructions (file/line number/general area)?
 Any tips on forcing a vmexit out of the guest immediately after the count?

Thank you so much for your time.
  :dan
------=_Part_11915_14058545.1196964586625-- --===============0123673549== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 --===============0123673549== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ kvm-devel mailing list kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org https://lists.sourceforge.net/lists/listinfo/kvm-devel --===============0123673549==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: performance specs on VT-x vmenter/exit Date: Thu, 06 Dec 2007 20:36:50 +0200 Message-ID: <47584142.10600@qumranet.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org To: Dan Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Dan wrote: > Greetings Honorable KVM Devs! > > I am doing research on the performance characteristics of the Intel > VMX instructions, specifically with respect to KVM. I need to count > the cycles for a VMENTER and VMEXIT instruction under various > conditions. (But getting it to work at all would be cool :) > > So the idea is to drop the cycle count in a register from the VMM, > force a VMENTER, drop the count in a different register, and force a > VMEXIT. Then get a last count and do appropriate adding/subtracting. > I'm working with a 2.6.22.6 vanilla kernel on an > Core2Duo with KVM-54. > > Questions: > Is there a better way to get these numbers? > If not, where in the KVM source should I aim to put these > instructions (file/line number/general area)? > Any tips on forcing a vmexit out of the guest immediately after the > count? > > Thank you so much for your time. > :dan Have a look at user/test/x86/vmexit.c, which does exactly that. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: performance specs on VT-x vmenter/exit Date: Thu, 06 Dec 2007 12:49:15 -0600 Message-ID: <4758442B.8060401@codemonkey.ws> References: <47584142.10600@qumranet.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org, Dan To: Avi Kivity Return-path: In-Reply-To: <47584142.10600-atKUWr5tajBWk0Htik3J/w@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Avi Kivity wrote: > Dan wrote: > >> Greetings Honorable KVM Devs! >> >> I am doing research on the performance characteristics of the Intel >> VMX instructions, specifically with respect to KVM. I need to count >> the cycles for a VMENTER and VMEXIT instruction under various >> conditions. (But getting it to work at all would be cool :) >> >> So the idea is to drop the cycle count in a register from the VMM, >> force a VMENTER, drop the count in a different register, and force a >> VMEXIT. Then get a last count and do appropriate adding/subtracting. >> I'm working with a 2.6.22.6 vanilla kernel on an >> Core2Duo with KVM-54. >> >> Questions: >> Is there a better way to get these numbers? >> If not, where in the KVM source should I aim to put these >> instructions (file/line number/general area)? >> Any tips on forcing a vmexit out of the guest immediately after the >> count? >> >> Thank you so much for your time. >> :dan >> > > > Have a look at user/test/x86/vmexit.c, which does exactly that. > > That only gives you the combined count of a vmenter and vmexit. If you want to measure the individual counts of a vmenter and a vmexit, the easiest thing to do is make sure the tsc_offset is always zero, and has the host and guest cooperate. The host stashes away the tsc before entering the guest, the guest immediately would take the tsc and stash it in a register and then immediately enter generate an exit. The host would then immediately take the tsc and stash it away. You now have three readings and can use them to identify the cost of vmenter and vmexit. Regards, Anthony Liguori ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: performance specs on VT-x vmenter/exit Date: Thu, 06 Dec 2007 21:06:26 +0200 Message-ID: <47584832.9010708@qumranet.com> References: <47584142.10600@qumranet.com> <4758442B.8060401@codemonkey.ws> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org, Dan To: Anthony Liguori Return-path: In-Reply-To: <4758442B.8060401-rdkfGonbjUSkNkDKm+mE6A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Anthony Liguori wrote: >> >> >> Have a look at user/test/x86/vmexit.c, which does exactly that. >> >> > > That only gives you the combined count of a vmenter and vmexit. Right; furthermore it measures kvm overhead and not just instruction latency. -- Any sufficiently difficult bug is indistinguishable from a feature. ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: performance specs on VT-x vmenter/exit Date: Thu, 06 Dec 2007 13:56:16 -0600 Message-ID: <475853E0.7080005@codemonkey.ws> References: <47584142.10600@qumranet.com> <4758442B.8060401@codemonkey.ws> <47584832.9010708@qumranet.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org, Dan To: Avi Kivity Return-path: In-Reply-To: <47584832.9010708-atKUWr5tajBWk0Htik3J/w@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org Avi Kivity wrote: > Anthony Liguori wrote: >>> >>> >>> Have a look at user/test/x86/vmexit.c, which does exactly that. >>> >>> >> >> That only gives you the combined count of a vmenter and vmexit. > > Right; furthermore it measures kvm overhead and not just instruction > latency. True. Also, I don't know what Dan is looking to measure, but the cost of vmexits will varying depending on the reason for exit. vmexit.c uses cpuid() which I suspect is slower than vmexit or pio but faster than pf. It would be interesting to measure the individual exit reasons. Regards, Anthony Liguori ------------------------------------------------------------------------- SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Subject: Re: performance specs on VT-x vmenter/exit Date: Thu, 6 Dec 2007 16:17:14 -0500 Message-ID: References: <47584142.10600@qumranet.com> <4758442B.8060401@codemonkey.ws> <47584832.9010708@qumranet.com> <475853E0.7080005@codemonkey.ws> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0119479256==" Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org, Avi Kivity To: "Anthony Liguori" Return-path: In-Reply-To: <475853E0.7080005-rdkfGonbjUSkNkDKm+mE6A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org Errors-To: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org List-Id: kvm.vger.kernel.org --===============0119479256== Content-Type: multipart/alternative; boundary="----=_Part_7871_11875643.1196975834989" ------=_Part_7871_11875643.1196975834989 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Thanks for the lightning-fast response, guys. I'll play around with vmexit.c and let some of this sink in. I was hoping to get a clean count of ticks/vm[enter/exit] without the kvm latency, and I thought putting in into the code in the kvm code in the kernel source would accomplish something like that. Getting counts per exit reason would be really neat too. Thanks again, I'm sure I'll come back with more precise questions later :D :dan On Dec 6, 2007 2:56 PM, Anthony Liguori wrote: > Avi Kivity wrote: > > Anthony Liguori wrote: > >>> > >>> > >>> Have a look at user/test/x86/vmexit.c, which does exactly that. > >>> > >>> > >> > >> That only gives you the combined count of a vmenter and vmexit. > > > > Right; furthermore it measures kvm overhead and not just instruction > > latency. > > True. Also, I don't know what Dan is looking to measure, but the cost > of vmexits will varying depending on the reason for exit. vmexit.c uses > cpuid() which I suspect is slower than vmexit or pio but faster than > pf. It would be interesting to measure the individual exit reasons. > > Regards, > > Anthony Liguori > > > ------=_Part_7871_11875643.1196975834989 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Thanks for the lightning-fast response, guys.

I'll play around with vmexit.c and let some of this sink in.
I was hoping to get a clean count of ticks/vm[enter/exit] without the kvm latency, and I thought putting in into the code in the kvm code in the kernel source would accomplish something like that.

Getting counts per exit reason would be really neat too.

Thanks again, I'm sure I'll come back with more precise questions later :D
  :dan



On Dec 6, 2007 2:56 PM, Anthony Liguori < anthony-rdkfGonbjUSkNkDKm+mE6A@public.gmane.org> wrote:
Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>>
>>> Have a look at user/test/x86/vmexit.c, which does exactly that.
>>>
>>>
>>
>> That only gives you the combined count of a vmenter and vmexit.
>
> Right; furthermore it measures kvm overhead and not just instruction
> latency.

True.  Also, I don't know what Dan is looking to measure, but the cost
of vmexits will varying depending on the reason for exit.   vmexit.c uses
cpuid() which I suspect is slower than vmexit or pio but faster than
pf.  It would be interesting to measure the individual exit reasons.

Regards,

Anthony Liguori



------=_Part_7871_11875643.1196975834989-- --===============0119479256== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- SF.Net email is sponsored by: Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php --===============0119479256== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ kvm-devel mailing list kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org https://lists.sourceforge.net/lists/listinfo/kvm-devel --===============0119479256==--