Re: perf, tools: Refactor and support interval and CSV metrics

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Tue, Mar 01, 2016 at 10:57:45AM -0800, Andi Kleen escreveu:
> Fixed even more last feedback.
> 
> [v5: Fix mainly bisect problems. No regressions introduced by one
> patch and fixed again later. Some minor fixes in addition]
> [v6: Fix running/noise printing patch.]
> [v7: Reorder and merge two patches to avoid a bisect hole where unsupported was
> printed as 0]
> [v8: Minor fixes for review feedback. See changelog in patches.]
> [v9: Fix newline bug. Add support for -A for --metric-only]
> [v10: Remove extra "noise" printing (Jiri)
>       Fix fields in documentation (Jiri)]
> [v11: Fix manpage again. Avoid extra metric output in CSV mode.]
> [v12: Move CSV metrics fields to after running/enabled/variance.
>       Minor fixes.]
> [v13: Address review comments. Now probe for stalled events 
>       in advance to avoid empty columns or lines. Fix -A shadowing.
>       Various minor changes. Drop merged patches.]
> [v14: Fix empty lines with CSV metrics. Avoid one more empty column
>       in metric-only.]
> [v15: Add missing fields in manpage. Use extra init function 
>       for frontend event. Various smaller fixes. Add acked-by.]

Please check acme/perf/core, I processed various patches that you are
resubmitting.

https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/core

Doing that you force me to check if there were changes in the patches
already applied :-\

I already collected the Acked-by tags.

I'll continue after the ones I already merged.

- Arnaldo
 
> Currently perf stat does not support printing computed metrics for interval (-I xxx)
> or CSV (-x,) mode. For example IPC or TSX metrics over time are quite useful to know.
> 
> This patch implements them. The main obstacle was that the
> metrics printing was all open coded all over the metrics computation code.
> The second patch refactors the metrics printing to work through call backs that
> can be more easily changed. This also cleans up the metrics printing significantly.
> The indentation is now handled through printf, no more need to manually count spaces.
> 
> Then based on that it implements metrics printing for CSV and interval mode,
> and finally a --metric-only mode.
> 
> Example output:
> 
> % perf stat  -I1000 -a sleep 1
> #          time              counts unit events                    metric                              multiplex
>      1.001301370       12020.049593      task-clock (msec)                                             (100.00%)
>      1.001301370              3,952      context-switches          #    0.329 K/sec                    (100.00%)
>      1.001301370                 69      cpu-migrations            #    0.006 K/sec                    (100.00%)
>      1.001301370                 76      page-faults               #    0.006 K/sec                  
>      1.001301370        386,582,789      cycles                    #    0.032 GHz                      (100.00%)
>      1.001301370        716,441,544      stalled-cycles-frontend   #  185.33% frontend cycles idle     (100.00%)
>      1.001301370    <not supported>      stalled-cycles-backend   
>      1.001301370        101,751,678      instructions              #    0.26  insn per cycle         
>      1.001301370                                                   #    7.04  stalled cycles per insn  (100.00%)
>      1.001301370         20,914,692      branches                  #    1.740 M/sec                    (100.00%)
>      1.001301370          1,943,630      branch-misses             #    9.29% of all branches        
> 
> CSV mode:
> 
> % perf stat  -x, -I1000 -a sleep 1
>      1.000982778,12006.549977,,task-clock,12006547787,100.00,,,,
>      1.000982778,12822,,context-switches,12007100604,100.00,0.001,M/sec
>      1.000982778,175,,cpu-migrations,12007180306,100.00,0.015,K/sec
>      1.000982778,3404,,page-faults,12007185482,100.00,0.284,K/sec
>      1.000982778,1930307489,,cycles,12007018233,100.00,0.161,GHz
>      1.000982778,6971803638,,stalled-cycles-frontend,12006902870,100.00,361.18,frontend cycles idle
>      1.000982778,464493941,,instructions,12006873327,100.00,0.24,insn per cycle
>      1.000982778,,,,,,15.01,stalled cycles per insn
>      1.000982778,86548409,,branches,12006758420,100.00,7.208,M/sec
>      1.000982778,4933638,,branch-misses,12006648104,100.00,5.70,of all branches
> 
> Now includes metrics
> 
> Metric only mode:
> 
> Concicse information if you only care about computed metrics, not raw values
> 
> % perf stat --metric-only  -a -I 1000
>          1.001452803 frontend cycles idle insn per cycle       stalled cycles per insn branch-misses of all branches
>          1.001452803  158.91%               0.66                2.39                    2.92%
>          2.002192321  180.63%               0.76                2.08                    2.96%
>          3.003088282  150.59%               0.62                2.57                    2.84%
>          4.004369835  196.20%               0.98                1.62                    3.79%
>          5.005227314  231.98%               0.84                1.90                    4.71%
> 
> 
> Metric only mode in CSV (flat format, easy to plot and analyze in statistical tools like JMP, R, pandas, gnuplot):
> 
> % perf stat -x, --metric-only  -a -I 1000
>      1.001381652,frontend cycles idle,insn per cycle,stalled cycles per insn,branch-misses of all branches,
>      1.001381652,173.32,0.83,2.09,1.73,
>      2.002073343,199.47,1.07,1.60,2.14,
>      3.002875524,109.52,0.22,7.83,1.63,
>      4.003970059,132.10,0.17,10.85,1.51,
>      5.004818754,181.60,0.22,8.87,2.22,
> 
> 
> Available in
> git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc-2.6 perf/stat-metrics-19