* reading PMU counts with perf leader sampling and perf script
@ 2016-09-09 14:48 Omar Awile
2016-09-11 3:21 ` Andi Kleen
2016-09-12 20:04 ` Arnaldo Carvalho de Melo
0 siblings, 2 replies; 6+ messages in thread
From: Omar Awile @ 2016-09-09 14:48 UTC (permalink / raw)
To: linux-perf-users@vger.kernel.org
Hello,
I’ve been trying to get periodic readouts of PMU event counts using perf record (similar to what perf stat -I does but with a higher precision than 10ms).
For this I thought I could use leader sampling with ref_cycles and the events I am targeting.
For example something like this:
$ perf record -P -c 2000000 -e '{cycles,instructions,branch-misses}:S' — ./workload
This seems to read out the sample values for the other events whenever the cycle event is sampled.
The next step would be for me to use perf script and a little python script to get the event counts but I can’t seem to find a way to do this. Can someone point me in the right direction?
perf report seems to be able to read these samples but they are shown only as aggregates. Also, perf script would allow me to process/analyze the data myself. If this is currently not possible, how would I go about adding the missing functionality in perf script?
Thank you,
Omar Awile
--
Omar Awile, Ph.D.
Systems Performance Engineer
CERN - 513/R-046
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: reading PMU counts with perf leader sampling and perf script
2016-09-09 14:48 reading PMU counts with perf leader sampling and perf script Omar Awile
@ 2016-09-11 3:21 ` Andi Kleen
2016-09-20 14:33 ` Omar Awile
2016-09-12 20:04 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 6+ messages in thread
From: Andi Kleen @ 2016-09-11 3:21 UTC (permalink / raw)
To: Omar Awile; +Cc: linux-perf-users@vger.kernel.org
Omar Awile <omar.awile@cern.ch> writes:
>
> The next step would be for me to use perf script and a little python
> script to get the event counts but I can’t seem to find a way to do
> this. Can someone point me in the right direction?
The easiest is to just run perf script without arguments and parse the output.
-Andi
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: reading PMU counts with perf leader sampling and perf script
2016-09-09 14:48 reading PMU counts with perf leader sampling and perf script Omar Awile
2016-09-11 3:21 ` Andi Kleen
@ 2016-09-12 20:04 ` Arnaldo Carvalho de Melo
[not found] ` <etPan.57e15392.6894e226.675c@cern.ch>
1 sibling, 1 reply; 6+ messages in thread
From: Arnaldo Carvalho de Melo @ 2016-09-12 20:04 UTC (permalink / raw)
To: Omar Awile
Cc: linux-perf-users, Jiri Olsa, Kan Liang, David Ahern, Namhyung Kim,
Peter Zijlstra
Em Fri, Sep 09, 2016 at 02:48:53PM +0000, Omar Awile escreveu:
> Hello,
> I’ve been trying to get periodic readouts of PMU event counts using
> perf record (similar to what perf stat -I does but with a higher
> precision than 10ms).
> For this I thought I could use leader sampling with ref_cycles and the
> events I am targeting.
> For example something like this:
> $ perf record -P -c 2000000 -e '{cycles,instructions,branch-misses}:S' — ./workload
> This seems to read out the sample values for the other events whenever
> the cycle event is sampled.
> The next step would be for me to use perf script and a little python
> script to get the event counts but I can’t seem to find a way to do
> this. Can someone point me in the right direction?
> perf report seems to be able to read these samples but they are shown
> only as aggregates. Also, perf script would allow me to
> process/analyze the data myself. If this is currently not possible,
> how would I go about adding the missing functionality in perf script?
Can you please take a look at tools/perf/scripts/python/stat-cpi.py,
this processes hardware counters, in python, the changeset that
introduces it:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/?id=b8a1962d17b4e3cfdd7b7dc9ebd94affbcb4c1c5
<SNIP>
get CPI systemwide:
$ perf stat -e cycles,instructions -a -I 1000 record sleep 3
# time counts unit events
1.000158618 594,274,711 cycles (100.00%)
1.000158618 441,898,250 instructions
2.000350973 567,649,705 cycles (100.00%)
2.000350973 432,669,206 instructions
3.000559210 561,940,430 cycles (100.00%)
3.000559210 420,403,465 instructions
3.000670798 780,105 cycles (100.00%)
3.000670798 326,516 instructions
$ perf script -s ./scripts/python/stat-cpi.py
1.000159: cpu -1, thread -1 -> cpi 1.344823 (594274711/441898250)
2.000351: cpu -1, thread -1 -> cpi 1.311972 (567649705/432669206)
3.000559: cpu -1, thread -1 -> cpi 1.336669 (561940430/420403465)
3.000671: cpu -1, thread -1 -> cpi 2.389178 (780105/326516)
$ perf stat -e cycles,instructions -a -I 1000 record sleep 3 | perf script -s ./scripts/python/stat-cpi.py
1.000202: cpu -1, thread -1 -> cpi 1.035091 (940778881/908885530)
2.000392: cpu -1, thread -1 -> cpi 1.442600 (627493992/434974455)
3.000545: cpu -1, thread -1 -> cpi 1.353612 (741463930/547766890)
3.000622: cpu -1, thread -1 -> cpi 2.642110 (784083/296764)
<SNIP>
And that -I limitation can be lifted, no?
- Arnaldo
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: reading PMU counts with perf leader sampling and perf script
2016-09-11 3:21 ` Andi Kleen
@ 2016-09-20 14:33 ` Omar Awile
0 siblings, 0 replies; 6+ messages in thread
From: Omar Awile @ 2016-09-20 14:33 UTC (permalink / raw)
To: Andi Kleen; +Cc: linux-perf-users@vger.kernel.org
I wrote a python perf script that dumps the counts and this seems to work very nicely.
thanks!
omar
On 11 September 2016 at 05:21:58, Andi Kleen (andi@firstfloor.org(mailto:andi@firstfloor.org)) wrote:
> Omar Awile writes:
> >
> > The next step would be for me to use perf script and a little python
> > script to get the event counts but I can’t seem to find a way to do
> > this. Can someone point me in the right direction?
>
> The easiest is to just run perf script without arguments and parse the output.
>
> -Andi
> --
> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Omar Awile, Ph.D.
Systems Performance Engineer
CERN - 513/R-046
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: reading PMU counts with perf leader sampling and perf script
[not found] ` <etPan.57e15392.6894e226.675c@cern.ch>
@ 2016-09-20 19:19 ` Arnaldo Carvalho de Melo
2016-09-20 20:16 ` Andi Kleen
0 siblings, 1 reply; 6+ messages in thread
From: Arnaldo Carvalho de Melo @ 2016-09-20 19:19 UTC (permalink / raw)
To: Omar Awile
Cc: Namhyung Kim, David Ahern, linux-perf-users@vger.kernel.org,
Jiri Olsa, Kan Liang, Peter Zijlstra
Em Tue, Sep 20, 2016 at 03:19:47PM +0000, Omar Awile escreveu:
> Hi Arnaldo,
> Ah that’s pretty neat. I will try it out. I found perf script really
> useful to generate properly formatted CSV (after I finally understood
> how to correctly use it). This might be something worth adding as a
> feature across all perf tools (stat, report, script). I know that some
> of the tools can print something kind of CSV, but it’s still just
> kind-of and not properly formatted, which makes parsing and processing
So, can you provide specific examples of things you think are wrong so
that we can see if something can be made to improve it?
- Arnaldo
> with other scripts/libraries cumbersome.
> cheers,
>
> omar
>
>
> On 12 September 2016 at 22:04:26, Arnaldo Carvalho de Melo (acme@kernel.org<mailto:acme@kernel.org>) wrote:
>
> Em Fri, Sep 09, 2016 at 02:48:53PM +0000, Omar Awile escreveu:
> > Hello,
>
> > I’ve been trying to get periodic readouts of PMU event counts using
> > perf record (similar to what perf stat -I does but with a higher
> > precision than 10ms).
>
> > For this I thought I could use leader sampling with ref_cycles and the
> > events I am targeting.
>
> > For example something like this:
>
> > $ perf record -P -c 2000000 -e '{cycles,instructions,branch-misses}:S' — ./workload
>
> > This seems to read out the sample values for the other events whenever
> > the cycle event is sampled.
>
> > The next step would be for me to use perf script and a little python
> > script to get the event counts but I can’t seem to find a way to do
> > this. Can someone point me in the right direction?
>
> > perf report seems to be able to read these samples but they are shown
> > only as aggregates. Also, perf script would allow me to
> > process/analyze the data myself. If this is currently not possible,
> > how would I go about adding the missing functionality in perf script?
>
> Can you please take a look at tools/perf/scripts/python/stat-cpi.py,
> this processes hardware counters, in python, the changeset that
> introduces it:
>
> https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/commit/?id=b8a1962d17b4e3cfdd7b7dc9ebd94affbcb4c1c5
>
> <SNIP>
> get CPI systemwide:
>
> $ perf stat -e cycles,instructions -a -I 1000 record sleep 3
> # time counts unit events
> 1.000158618 594,274,711 cycles (100.00%)
> 1.000158618 441,898,250 instructions
> 2.000350973 567,649,705 cycles (100.00%)
> 2.000350973 432,669,206 instructions
> 3.000559210 561,940,430 cycles (100.00%)
> 3.000559210 420,403,465 instructions
> 3.000670798 780,105 cycles (100.00%)
> 3.000670798 326,516 instructions
>
> $ perf script -s ./scripts/python/stat-cpi.py
> 1.000159: cpu -1, thread -1 -> cpi 1.344823 (594274711/441898250)
> 2.000351: cpu -1, thread -1 -> cpi 1.311972 (567649705/432669206)
> 3.000559: cpu -1, thread -1 -> cpi 1.336669 (561940430/420403465)
> 3.000671: cpu -1, thread -1 -> cpi 2.389178 (780105/326516)
>
> $ perf stat -e cycles,instructions -a -I 1000 record sleep 3 | perf script -s ./scripts/python/stat-cpi.py
> 1.000202: cpu -1, thread -1 -> cpi 1.035091 (940778881/908885530)
> 2.000392: cpu -1, thread -1 -> cpi 1.442600 (627493992/434974455)
> 3.000545: cpu -1, thread -1 -> cpi 1.353612 (741463930/547766890)
> 3.000622: cpu -1, thread -1 -> cpi 2.642110 (784083/296764)
>
> <SNIP>
>
> And that -I limitation can be lifted, no?
>
> - Arnaldo
> --
> Omar Awile, Ph.D.
> Systems Performance Engineer
> CERN - 513/R-046
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: reading PMU counts with perf leader sampling and perf script
2016-09-20 19:19 ` Arnaldo Carvalho de Melo
@ 2016-09-20 20:16 ` Andi Kleen
0 siblings, 0 replies; 6+ messages in thread
From: Andi Kleen @ 2016-09-20 20:16 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Omar Awile, Namhyung Kim, David Ahern,
linux-perf-users@vger.kernel.org, Jiri Olsa, Kan Liang,
Peter Zijlstra
Arnaldo Carvalho de Melo <acme@kernel.org> writes:
> Em Tue, Sep 20, 2016 at 03:19:47PM +0000, Omar Awile escreveu:
>> Hi Arnaldo,
>
>> Ah that’s pretty neat. I will try it out. I found perf script really
>> useful to generate properly formatted CSV (after I finally understood
>> how to correctly use it). This might be something worth adding as a
>> feature across all perf tools (stat, report, script). I know that some
>> of the tools can print something kind of CSV, but it’s still just
>> kind-of and not properly formatted, which makes parsing and processing
>
> So, can you provide specific examples of things you think are wrong so
> that we can see if something can be made to improve it?
The main problem is that commas are not quoted (or rather not put into
""), and cpu events have a lot of commas. So -x, tends to output
invalid CSV.
However it generally works well enough with -x; and most CSV tools
can handle that too.
-Andi
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-09-20 20:16 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-09-09 14:48 reading PMU counts with perf leader sampling and perf script Omar Awile
2016-09-11 3:21 ` Andi Kleen
2016-09-20 14:33 ` Omar Awile
2016-09-12 20:04 ` Arnaldo Carvalho de Melo
[not found] ` <etPan.57e15392.6894e226.675c@cern.ch>
2016-09-20 19:19 ` Arnaldo Carvalho de Melo
2016-09-20 20:16 ` Andi Kleen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).