Re: XENOPROF Problem

Re: XENOPROF Problem

[Andrew Evans]

[-- Attachment #1.1: Type: text/plain, Size: 3292 bytes --]

Hi Ahmad,

I'm certainly not a Xenoprof expert, but I've learned a bit about it by
trial and error, with help from the xen-devel list. I've taken the
liberty of Cc:ing xen-devel so that others more knowledgeable than me
can help you. I haven't tried passive domains yet, and I'll say up front
that I don't know why Xenoprof isn't working for you with passive domains.

On 02/11/10 13:37, Ahmad Hassan wrote:
> Hi Andrew,
>
> I need help in using xenoprof. I installed the xenoprof on Dom0 and
> DomU as follows:
>
>     * Download Oprofile:
>       http://prdownloads.sourceforge.net/oprofile/oprofile-0.9.5.tar.gz
>     * Download XenoProf
>       patch: http://xenoprof.sourceforge.net/oprofile-0.9.5-xen.patch
>     * tar –zxvf oprofile-0.9.5.tar.gz
>     * cd oprofile-0.9.5
>     * patch –p1 < ../oprofile-0.9.5-xen-rc2.patch ./configure
>       –with-kernel-support
>     * make
>     * make install
>

That looks right to me. You also need Xenoprof support in your dom0
kernel. If you're using one of the kernels from the Xen source tree and
you can `modprobe oprofile` successfully, you should be ok.

> I am interested in measuring profiling overhead in XEN. So which kind
> of event should I capture.

You want to measure the overhead of the profiler itself? I guess you get
that "for free" -- time spent in oprofile kernel code should show up in
opreport in dom0, I think.

Which performance counter you use depends on what you're trying to
measure. If you just want to find out where CPU time is spent,
CPU_CLK_UNHALTED should do that for you. That's what I've been doing.

> Is it possible to capture all the events?

You can specify multiple events, though I haven't tried it. The opreport
web page talks about output for multiple events here:

http://oprofile.sourceforge.net/doc/opreport.html#opreport-comparison

I'd guess you'd just specify --event multiple times when you --start-daemon.

>   I followed the following passive profiling approach but I did not
> get any data.
>
> Dom0: opcontrol --reset
> Dom1: opcontrol --reset
> Dom0: opcontrol --start-daemon --event=
> CPU_IO_REQUESTS_TO_MEMORY_IO:1000000 --xen=/boot/xen-syms-3.4.1  
> --vmlinux=/boot/vmlinux  --passive-domains=1
> --passive-images=/boot/vmlinux
> *At the above command, I get the warning as :
> /dev/oprofile/passive-domains Permission Denied* but daemon starts
> eventually

This is a silly question, but you are running this as root, correct? If
you are running as root and getting this error, I think something is
wrong either in Xen or your dom0 kernel. `xm dmesg` may have error
messages that give clues -- check it before and after starting the daemon.

> Dom0: opcontrol --start
> AFTER 30 MINUTE
> Dom0: opcontrol –stop
> Dom0: opcontrol --dump
> Dom0: opcontrol –shutdown
>
> But when I try to "opreport -l" ti says that: "opreport error: No
> sample file found: try running opcontrol --dump or specfiy a session
> containing sample file"
>
>
> Can you please help me in that?
>
> Thanks.
>
> Best Regards, Hassan
> http://cern.ch/ahmadh/portfolio
>
> Erasmus Student
> The University of Reading, UK
> Aristotle University of Thessaloniki, Greece
> The University Of Carlos III, Madrid Spain
>
>


[-- Attachment #1.2: Type: text/html, Size: 7730 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: XENOPROF Problem

[Ahmad Hassan]

[-- Attachment #1.1: Type: text/plain, Size: 1620 bytes --]

Hi Andrew,

Thanks for the message.

That looks right to me. You also need Xenoprof support in your dom0 kernel.
> If you're using one of the kernels from the Xen source tree and you can
> `modprobe oprofile` successfully, you should be ok.
>

Yes I loaded the oprofile as "modprobe oprofile". It loads successfully.

>
>
> Dom0: opcontrol --start-daemon --event= CPU_IO_REQUESTS_TO_MEMORY_IO:1000000
> --xen=/boot/xen-syms-3.4.1   --vmlinux=/boot/vmlinux  --passive-domains=1
> --passive-images=/boot/vmlinux
> *At the above command, I get the warning as :
> /dev/oprofile/passive-domains Permission Denied* but daemon starts
> eventually
>
>
If i start the Dom0 daemon for passive domains using the above command  the
the following exact output I get

*At the above command, I get the warning as : /dev/oprofile/passive-domains
Permission Denied*
*Using* *2*.*6*+ *OProfile* *kernel* interface.
*Using* *log* *file* /*var*/*lib*/*oprofile*/*oprofiled*.*log*
*Daemon* *started*.
*Profiler* *running*

Yes I am running it as a root. If I look at the * /dev/oprofile the I don't
find any file name passive-domains.

Opreport -l shows the following error:

* "opreport error: No sample file found: try running opcontrol --dump or
specfiy a session containing sample file"


When I try active-domains option then inside DomU, I get the following error
while starting the daemon:

*"You cannot specify any performance counter events because OProfile is in
timer mode.*"

Why the above error appears for active domains? Do I need to pass "lapic"
parameter in the grub file of dom0?

Thanks for the help

Best Regards, Hassan

[-- Attachment #1.2: Type: text/html, Size: 2990 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Re: XENOPROF Problem

[Santos, Jose Renato G]

[-- Attachment #1.1: Type: text/plain, Size: 3826 bytes --]

Acive domain profiling has not been well tested with pvops kernel.
Dulloor who ported xenoprof parches to the pvops kernel said in a previous post that he would look into that but not sure if he did.
For now your alternatives are:
  - Use passive domain prifiling instead
  - Use 2.6.18-xen with Xen 4.0 or latest unstable which are known to work with active domain profiling

Renato

________________________________
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Andrew Evans
Sent: Thursday, February 11, 2010 3:39 PM
To: Ahmad Hassan
Cc: xen-devel@lists.xensource.com
Subject: [Xen-devel] Re: XENOPROF Problem

Hi Ahmad,

I'm certainly not a Xenoprof expert, but I've learned a bit about it by trial and error, with help from the xen-devel list. I've taken the liberty of Cc:ing xen-devel so that others more knowledgeable than me can help you. I haven't tried passive domains yet, and I'll say up front that I don't know why Xenoprof isn't working for you with passive domains.

On 02/11/10 13:37, Ahmad Hassan wrote:
Hi Andrew,

I need help in using xenoprof. I installed the xenoprof on Dom0 and DomU as follows:


 *   Download Oprofile: http://prdownloads.sourceforge.net/oprofile/oprofile-0.9.5.tar.gz
 *   Download XenoProf patch: http://xenoprof.sourceforge.net/oprofile-0.9.5-xen.patch
 *   tar -zxvf oprofile-0.9.5.tar.gz
 *   cd oprofile-0.9.5
 *   patch -p1 < ../oprofile-0.9.5-xen-rc2.patch ./configure -with-kernel-support
 *   make
 *   make install

That looks right to me. You also need Xenoprof support in your dom0 kernel. If you're using one of the kernels from the Xen source tree and you can `modprobe oprofile` successfully, you should be ok.

I am interested in measuring profiling overhead in XEN. So which kind of event should I capture.

You want to measure the overhead of the profiler itself? I guess you get that "for free" -- time spent in oprofile kernel code should show up in opreport in dom0, I think.

Which performance counter you use depends on what you're trying to measure. If you just want to find out where CPU time is spent, CPU_CLK_UNHALTED should do that for you. That's what I've been doing.

Is it possible to capture all the events?

You can specify multiple events, though I haven't tried it. The opreport web page talks about output for multiple events here:

http://oprofile.sourceforge.net/doc/opreport.html#opreport-comparison

I'd guess you'd just specify --event multiple times when you --start-daemon.

  I followed the following passive profiling approach but I did not get any data.

Dom0: opcontrol --reset
Dom1: opcontrol --reset
Dom0: opcontrol --start-daemon --event= CPU_IO_REQUESTS_TO_MEMORY_IO:1000000 --xen=/boot/xen-syms-3.4.1   --vmlinux=/boot/vmlinux  --passive-domains=1 --passive-images=/boot/vmlinux
At the above command, I get the warning as : /dev/oprofile/passive-domains Permission Denied but daemon starts eventually

This is a silly question, but you are running this as root, correct? If you are running as root and getting this error, I think something is wrong either in Xen or your dom0 kernel. `xm dmesg` may have error messages that give clues -- check it before and after starting the daemon.

Dom0: opcontrol --start
AFTER 30 MINUTE
Dom0: opcontrol -stop
Dom0: opcontrol --dump
Dom0: opcontrol -shutdown

But when I try to "opreport -l" ti says that: "opreport error: No sample file found: try running opcontrol --dump or specfiy a session containing sample file"


Can you please help me in that?

Thanks.

Best Regards, Hassan
http://cern.ch/ahmadh/portfolio

Erasmus Student
The University of Reading, UK
Aristotle University of Thessaloniki, Greece
The University Of Carlos III, Madrid Spain




[-- Attachment #1.2: Type: text/html, Size: 10160 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

xenoprof problem

[Xin Zhao]

I am using xen-unstable with kernel 2.6.16. I enabled the auditd when 
compiling kernel. After compile and restart, I did "opcontrol --start" 
in DomU. But I got the error message as following:

------------------------------
Using default event: GLOBAL_POWER_EVENTS:100000:1:1:1
Using 2.6+ OProfile kernel interface.
Failed to open profile device: Operation not permitted
Couldn't start oprofiled.
Check the log file "/var/lib/oprofile/oprofiled.log" and kernel syslog
-------------------------------------------------

Does anyone have similar experience? Is this the right way to profiling 
the performance in xen systems?

Thanks in advance for your help!

Xin

RE: xenoprof problem

[Santos, Jose Renato G]

Xin,

XenOprofile does not support single domain profiling yet,
but only system wide profiling. That means that profiling
has to be controled by dom0. You will not be able to run
Oprofile from a domU without running oprofile in dom0.
You can find a more detailed description in:
http://xenoprof.sourceforge.net/xenoprof_2.0.txt

Below is a typical sequence of commands to run Oprofile in
multiple domains:

A) Sequence of commands to start profiling:
     1) On the initiator domain (dom0)
         > opcontrol --reset
           (clear out any previous data of current session)
         > opcontrol --start-daemon
                     [--active-domains=<active_list>]
                     [--passive-domains=<passive_list>] 
                     <events> <dom0 kernel and xen images specs> ...
           (start OProfile daemon and specify the set of active and
           passive domains in the session)

     2) On each active domain (domU to be profiled)
         > opcontrol --reset
         > opcontrol --start <events> <domU kernel and xen images specs>
         (indicates domain is ready to process performance events)
     3) On initiator
         > opcontrol --start
         (Multi-domain profiling session starts)
         (This is only successful if all active domains are ready)

 B) Sequence of commands to stop profiling
     -  On initiator domain:
         > opcontrol --stop

Renato

>> -----Original Message-----
>> From: xen-devel-bounces@lists.xensource.com 
>> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Xin Zhao
>> Sent: Monday, May 22, 2006 10:50 PM
>> To: Xen devel list
>> Subject: [Xen-devel] xenoprof problem
>> 
>> I am using xen-unstable with kernel 2.6.16. I enabled the 
>> auditd when compiling kernel. After compile and restart, I 
>> did "opcontrol --start" 
>> in DomU. But I got the error message as following:
>> 
>> ------------------------------
>> Using default event: GLOBAL_POWER_EVENTS:100000:1:1:1 Using 
>> 2.6+ OProfile kernel interface.
>> Failed to open profile device: Operation not permitted 
>> Couldn't start oprofiled.
>> Check the log file "/var/lib/oprofile/oprofiled.log" and 
>> kernel syslog
>> -------------------------------------------------
>> 
>> Does anyone have similar experience? Is this the right way 
>> to profiling the performance in xen systems?
>> 
>> Thanks in advance for your help!
>> 
>> Xin
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>