From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bhatia, Nikhil" Subject: Performance Counter Virtualization in Xen Date: Fri, 18 May 2007 15:15:18 -0400 Message-ID: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2010493538==" Return-path: Content-class: urn:content-classes:message List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Mime-version: 1.0 Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-users@lists.xensource.com Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============2010493538== Content-type: multipart/alternative; boundary="Boundary_(ID_QR6bE2EE0VAmJgKwDbeOlw)" Content-class: urn:content-classes:message This is a multi-part message in MIME format. --Boundary_(ID_QR6bE2EE0VAmJgKwDbeOlw) Content-type: text/plain; charset="us-ascii" Content-transfer-encoding: quoted-printable Hello Xenthusiasts, =20 I have a few questions regarding the performance monitoring of applications and domains running on the latest version of Xen VMM via the performance monitoring counters available in x86 architectures.=20 =20 - I have used Xenoprof by patching Xen-3.0.2 Xenolinux and the Oprofile layer which enabled me to collect performance counter values per domain (for a passive domain) and per application (for an active domain). I was able to perform the profiling via command line interface using Opcontrol and then reporting the counters using Opreport. However, I was unable to find a way to profile an application (using an API like PAPI) because the perfctr port was not available for Xen. Is there a way through which applications can be monitored using performance counters?=20 =20 - Is there a Xen tool/utility which provides meaningful information about per-domain metrics for dynamic memory usage (e.g. reserved memory by Xen, total heap size in use, swap space reserved per domain etc.), I/O done to/from the disc and the bytes sent/received from the network over fixed intervals of time?=20 =20 =20 Thanks, Best Regards, Nikhil --Boundary_(ID_QR6bE2EE0VAmJgKwDbeOlw) Content-type: text/html; charset="us-ascii" Content-transfer-encoding: quoted-printable

Hello Xenthusiasts,

I have a few questions regarding the performance = monitoring of applications and domains running on the latest version of Xen VMM via = the performance monitoring counters available in x86 architectures. =

- I have used Xenoprof by = patching Xen-3.0.2 Xenolinux and the Oprofile layer which enabled me to collect performance counter values per domain (for a passive domain) and per application (for an active domain). I was able to perform the profiling = via command line interface using Opcontrol and then reporting the counters = using Opreport. However, I was unable to find a way to profile an application (using an = API like PAPI) because the perfctr port was not available for Xen. Is there = a way through which applications can be monitored using performance counters? =

- Is there a Xen tool/utility = which provides meaningful information about per-domain metrics for dynamic = memory usage (e.g. reserved memory by Xen, total heap size in use, swap space = reserved per domain etc.), I/O done to/from the disc and the bytes sent/received = from the network over fixed intervals of time?

Thanks,

Best Regards,

Nikhil

--Boundary_(ID_QR6bE2EE0VAmJgKwDbeOlw)-- --===============2010493538== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============2010493538==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Santos, Jose Renato G" Subject: RE: Performance Counter Virtualization in Xen Date: Fri, 18 May 2007 20:42:58 -0000 Message-ID: <08CA2245AFCF444DB3AC415E47CC40AFB5264C@G3W0072.americas.hpqcorp.net> References: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0240344006==" Return-path: Content-class: urn:content-classes:message In-Reply-To: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Mime-version: 1.0 Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Bhatia, Nikhil" , xen-users@lists.xensource.com Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --===============0240344006== Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C7998D.24B3E0AD" This is a multi-part message in MIME format. ------_=_NextPart_001_01C7998D.24B3E0AD Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable You are right, currently the only available tool for using hardware performance counters in Xen is XenOprofile. To expose a more generic API to guests such as Perfmon, we would need to virtualize performance counters. One of the challenges of virtualizing performance counters is the high cost of saving and restoring perf. counters on domain context switches. It may be possible to do some optimizations such as lazy save/restore that could help. Also it is expected that the cost for accessing performance counters is going to be better in new processors that are coming out from Intel and AMD. afaik, no one is actively working on virtualizing HW performance counters in Xen. =20 For other software metrics such as number of I/O operations you can try Xenmon which report values of Xen performance counters (i.e SW counters).=20 Regards =20 Renato ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Bhatia, Nikhil Sent: Friday, May 18, 2007 12:15 PM To: xen-users@lists.xensource.com Cc: xen-devel@lists.xensource.com Subject: [Xen-devel] Performance Counter Virtualization in Xen =09 =09 Hello Xenthusiasts, =20 I have a few questions regarding the performance monitoring of applications and domains running on the latest version of Xen VMM via the performance monitoring counters available in x86 architectures.=20 =20 - I have used Xenoprof by patching Xen-3.0.2 Xenolinux and the Oprofile layer which enabled me to collect performance counter values per domain (for a passive domain) and per application (for an active domain). I was able to perform the profiling via command line interface using Opcontrol and then reporting the counters using Opreport. However, I was unable to find a way to profile an application (using an API like PAPI) because the perfctr port was not available for Xen. Is there a way through which applications can be monitored using performance counters?=20 =20 - Is there a Xen tool/utility which provides meaningful information about per-domain metrics for dynamic memory usage (e.g. reserved memory by Xen, total heap size in use, swap space reserved per domain etc.), I/O done to/from the disc and the bytes sent/received from the network over fixed intervals of time?=20 =20 =20 Thanks, Best Regards, Nikhil ------_=_NextPart_001_01C7998D.24B3E0AD Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

You are right, currently the only = available=20 tool for using hardware performance counters in Xen = is XenOprofile. To=20 expose a more generic API to guests such as Perfmon, we would need = to=20 virtualize performance counters. One of the challenges of = virtualizing=20 performance counters is the high cost of saving and restoring perf. = counters on=20 domain context switches. It may be possible to do some optimizations = such as=20 lazy save/restore that could help. Also it is expected = that the=20 cost for accessing performance counters is going to be better in new = processors=20 that are coming out from Intel and AMD. afaik, no one is actively = working on=20 virtualizing HW performance counters in = Xen.

For other software metrics such as number of = I/O operations=20 you can try Xenmon which report values of Xen performance counters (i.e = SW=20 counters).

Regards

Renato

From: = xen-devel-bounces@lists.xensource.com=20 [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of = Bhatia,=20 Nikhil
Sent: Friday, May 18, 2007 12:15 PM
To:=20 xen-users@lists.xensource.com
Cc:=20 xen-devel@lists.xensource.com
Subject: [Xen-devel] = Performance=20 Counter Virtualization in Xen

Hello=20 Xenthusiasts,

I have a few questions = regarding=20 the performance monitoring of applications and domains running on the = latest=20 version of Xen VMM via the performance monitoring counters available = in x86=20 architectures.

- =20 I have used Xenoprof by = patching=20 Xen-3.0.2 Xenolinux and the Oprofile layer which enabled me to collect = performance counter values per domain (for a passive domain) and per=20 application (for an active domain). I was able to perform the = profiling via=20 command line interface using Opcontrol and then reporting the counters = using=20 Opreport. However, I was unable to find a way to profile an = application (using=20 an API like PAPI) because the perfctr port was not available for Xen. = Is there=20 a way through which applications can be monitored using performance = counters?=20

- =20 Is there a Xen = tool/utility which=20 provides meaningful information about per-domain metrics for dynamic = memory=20 usage (e.g. reserved memory by Xen, total heap size in use, swap space = reserved per domain etc.), I/O done to/from the disc and the bytes=20 sent/received from the network over fixed intervals of time?=20

Thanks,

Best=20 Regards,

Nikhil

------_=_NextPart_001_01C7998D.24B3E0AD-- --===============0240344006== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============0240344006==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christian Ehrhardt Subject: Re: Performance Counter Virtualization in Xen Date: Mon, 21 May 2007 08:36:32 +0200 Message-ID: <46513DF0.3000105@linux.vnet.ibm.com> References: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> <08CA2245AFCF444DB3AC415E47CC40AFB5264C@G3W0072.americas.hpqcorp.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <08CA2245AFCF444DB3AC415E47CC40AFB5264C@G3W0072.americas.hpqcorp.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Santos, Jose Renato G" Cc: "Bhatia, Nikhil" , xen-devel@lists.xensource.com, xen-users@lists.xensource.com, xen-ppc-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Well it's not "x86 architectures", but the current profiling support in=20 xen on powerpc allows the use of performance counters from the domains=20 as used from non-virtualized environments. It is currently only tested with oprofile and has to be considered=20 experimental because there are some known issues, but depending on your=20 needs this might work for you. As far as I read on=20 http://icl.cs.utk.edu/papi/custom/index.html?lid=3D62&slid=3D96 PAPI has=20 PPC970 support. I do not know PAPI in detail, but it is possible that you can run it=20 with less/no patching xen/linux on the current xenppc head. Santos, Jose Renato G wrote: > You are right, currently the only available tool for using hardware=20 > performance counters in Xen is XenOprofile. To expose a more generic=20 > API to guests such as Perfmon, we would need to virtualize performance=20 > counters. One of the challenges of virtualizing performance counters=20 > is the high cost of saving and restoring perf. counters on domain=20 > context switches. It may be possible to do some optimizations such as=20 > lazy save/restore that could help. Also it is expected that the cost=20 > for accessing performance counters is going to be better in new=20 > processors that are coming out from Intel and AMD. afaik, no one is=20 > actively working on virtualizing HW performance counters in Xen. > =20 > For other software metrics such as number of I/O operations you can=20 > try Xenmon which report values of Xen performance counters (i.e SW=20 > counters). > Regards > =20 > Renato > > -------------------------------------------------------------------= ----- > *From:* xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] *On Behalf Of > *Bhatia, Nikhil > *Sent:* Friday, May 18, 2007 12:15 PM > *To:* xen-users@lists.xensource.com > *Cc:* xen-devel@lists.xensource.com > *Subject:* [Xen-devel] Performance Counter Virtualization in Xen > > Hello Xenthusiasts, > > =20 > > I have a few questions regarding the performance monitoring of > applications and domains running on the latest version of Xen VMM > via the performance monitoring counters available in x86 > architectures. > > =20 > > - I have used Xenoprof by patching Xen-3.0.2 Xenolinux > and the Oprofile layer which enabled me to collect performance > counter values per domain (for a passive domain) and per > application (for an active domain). I was able to perform the > profiling via command line interface using Opcontrol and then > reporting the counters using Opreport. However, I was unable to > find a way to profile an application (using an API like PAPI) > because the perfctr port was not available for Xen. Is there a way > through which applications can be monitored using performance > counters? > > =20 > > - Is there a Xen tool/utility which provides meaningful > information about per-domain metrics for dynamic memory usage > (e.g. reserved memory by Xen, total heap size in use, swap space > reserved per domain etc.), I/O done to/from the disc and the bytes > sent/received from the network over fixed intervals of time? > > =20 > > =20 > > Thanks, > > Best Regards, > > Nikhil > > -----------------------------------------------------------------------= - > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > =20 --=20 Gr=FCsse / regards,=20 Christian Ehrhardt IBM Linux Technology Center, Open Virtualization +49 7031/16-3385 Ehrhardt@linux.vmnet.ibm.com Ehrhardt@de.ibm.com IBM Deutschland Entwicklung GmbH Vorsitzender des Aufsichtsrats: Johann Weihen=20 Gesch=E4ftsf=FChrung: Herbert Kircher=20 Sitz der Gesellschaft: B=F6blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bhatia, Nikhil" Subject: RE: Performance Counter Virtualization in Xen Date: Mon, 21 May 2007 02:52:19 -0400 Message-ID: <537C6C0940C6C143AA46A88946B8541708160CB8@ORNLEXCHANGE.ornl.gov> References: <46513DF0.3000105@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-reply-to: <46513DF0.3000105@linux.vnet.ibm.com> Content-class: urn:content-classes:message List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Christian Ehrhardt , "Santos, Jose Renato G" Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com, xen-ppc-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Hi Christian, Does the power PC still use the PM API for getting performance counters? = I think, PAPI uses PMAPI underneath to get the performance counter data = on a power PC. So, if there is a patched PMAPI library available for the = Xen port on power PC, then a user might be able to easily use PAPI for = per-domain profiling. Currently, I don't have an access to a Power PC = machine with root privileges. However, if I get hold of such a machine, = I would love to try this out and share my findings.=20 Thanks, Nikhil=20 -----Original Message----- From: Christian Ehrhardt [mailto:ehrhardt@linux.vnet.ibm.com]=20 Sent: Monday, May 21, 2007 2:37 AM To: Santos, Jose Renato G Cc: Bhatia, Nikhil; xen-users@lists.xensource.com; = xen-devel@lists.xensource.com; xen-ppc-devel@lists.xensource.com Subject: Re: [Xen-devel] Performance Counter Virtualization in Xen Well it's not "x86 architectures", but the current profiling support in=20 xen on powerpc allows the use of performance counters from the domains=20 as used from non-virtualized environments. It is currently only tested with oprofile and has to be considered=20 experimental because there are some known issues, but depending on your=20 needs this might work for you. As far as I read on=20 http://icl.cs.utk.edu/papi/custom/index.html?lid=3D62&slid=3D96 PAPI has = PPC970 support. I do not know PAPI in detail, but it is possible that you can run it=20 with less/no patching xen/linux on the current xenppc head. Santos, Jose Renato G wrote: > You are right, currently the only available tool for using hardware=20 > performance counters in Xen is XenOprofile. To expose a more generic=20 > API to guests such as Perfmon, we would need to virtualize performance = > counters. One of the challenges of virtualizing performance counters=20 > is the high cost of saving and restoring perf. counters on domain=20 > context switches. It may be possible to do some optimizations such as=20 > lazy save/restore that could help. Also it is expected that the cost=20 > for accessing performance counters is going to be better in new=20 > processors that are coming out from Intel and AMD. afaik, no one is=20 > actively working on virtualizing HW performance counters in Xen. > =20 > For other software metrics such as number of I/O operations you can=20 > try Xenmon which report values of Xen performance counters (i.e SW=20 > counters). > Regards > =20 > Renato > > = ------------------------------------------------------------------------ > *From:* xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] *On Behalf Of > *Bhatia, Nikhil > *Sent:* Friday, May 18, 2007 12:15 PM > *To:* xen-users@lists.xensource.com > *Cc:* xen-devel@lists.xensource.com > *Subject:* [Xen-devel] Performance Counter Virtualization in Xen > > Hello Xenthusiasts, > > =20 > > I have a few questions regarding the performance monitoring of > applications and domains running on the latest version of Xen VMM > via the performance monitoring counters available in x86 > architectures. > > =20 > > - I have used Xenoprof by patching Xen-3.0.2 Xenolinux > and the Oprofile layer which enabled me to collect performance > counter values per domain (for a passive domain) and per > application (for an active domain). I was able to perform the > profiling via command line interface using Opcontrol and then > reporting the counters using Opreport. However, I was unable to > find a way to profile an application (using an API like PAPI) > because the perfctr port was not available for Xen. Is there a way > through which applications can be monitored using performance > counters? > > =20 > > - Is there a Xen tool/utility which provides meaningful > information about per-domain metrics for dynamic memory usage > (e.g. reserved memory by Xen, total heap size in use, swap space > reserved per domain etc.), I/O done to/from the disc and the bytes > sent/received from the network over fixed intervals of time? > > =20 > > =20 > > Thanks, > > Best Regards, > > Nikhil > > = ------------------------------------------------------------------------ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > =20 --=20 Gr=FCsse / regards,=20 Christian Ehrhardt IBM Linux Technology Center, Open Virtualization +49 7031/16-3385 Ehrhardt@linux.vmnet.ibm.com Ehrhardt@de.ibm.com IBM Deutschland Entwicklung GmbH Vorsitzender des Aufsichtsrats: Johann Weihen=20 Gesch=E4ftsf=FChrung: Herbert Kircher=20 Sitz der Gesellschaft: B=F6blingen Registergericht: Amtsgericht Stuttgart, HRB 243294 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Williamson Subject: Re: Performance Counter Virtualization in Xen Date: Wed, 30 May 2007 11:49:53 +0100 Message-ID: <200705301149.54148.mark.williamson@cl.cam.ac.uk> References: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <537C6C0940C6C143AA46A88946B8541708160B01@ORNLEXCHANGE.ornl.gov> Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com Cc: "Bhatia, Nikhil" , xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org > Hello Xenthusiasts, > - Is there a Xen tool/utility which provides meaningful > information about per-domain metrics for dynamic memory usage (e.g. > reserved memory by Xen, total heap size in use, swap space reserved per > domain etc.), I/O done to/from the disc and the bytes sent/received from > the network over fixed intervals of time? Network IO you should be able to monitor by looking at the stats associated with the vif to the domain in dom0 (at least for paravirt vifs). Block IO, I'm not so sure of the mechanism. Regarding swap space - all the swapping is done by the domain itself onto a virtual block device you've provided it. If you give it a separate virtual block device and configure it to use that for swap then monitoring that VBD would give you an idea of swap activity - but not current swap use. If you want to know domain-internal stats, such as swap use, the easiest approach is to get the domain to stick some entries in XenStore to tell you. The nearest to the tool you want would be "xm top" or "xenmon", however I think these still don't monitor all the things you're interested in. HTH, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel!