Re: [GIT PULL 00/43] perf/core new feature: 'perf stat record/report'

[Ingo Molnar <mingo@kernel.org>]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> From: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> Hi Ingo,
> 
> 	Please consider pulling, cool new feature! This is on top of the
> perf-core-for-mingo-2.1 tag, with that RHEL6.7 bugfix, I had also to go
> over this one fixing stuff in many spots :-\
> 
> - Arnaldo
> 
> The following changes since commit 1843b4e057b7717db21a3ad96fa16d6b4ee8f6c4:
> 
>   tools subcmd: Rename subcmd header include guards (2015-12-17 14:27:14 -0300)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-3
> 
> for you to fetch changes up to 89af4e05c21d68f22e07fe66940ea675615a49ed:
> 
>   perf stat report: Allow to override aggr_mode (2015-12-17 16:30:30 -0300)
> 
> ----------------------------------------------------------------
> perf/core improvement.
> 
> User visible:
> 
> - Generate perf.data files from 'perf stat', to tap into the scripting
>   capabilities perf has instead of defining a 'perf stat' specific scripting
>   support to calculate event ratios, etc. Simple example:
> 
>   $ perf stat record -e cycles usleep 1
> 
>    Performance counter stats for 'usleep 1':
> 
>          1,134,996      cycles
> 
>        0.000670644 seconds time elapsed
> 
>   $ perf stat report
> 
>    Performance counter stats for '/home/acme/bin/perf stat record -e cycles usleep 1':
> 
>          1,134,996      cycles
> 
>        0.000670644 seconds time elapsed
> 
>   $
> 
>   It generates PERF_RECORD_ userspace records to store the details:
> 
>   $ perf report -D | grep PERF_RECORD
>   0xf0 [0x28]: PERF_RECORD_THREAD_MAP nr: 1 thread: 27637
>   0x118 [0x12]: PERF_RECORD_CPU_MAP nr: 1 cpu: 65535
>   0x12a [0x40]: PERF_RECORD_STAT_CONFIG
>   0x16a [0x30]: PERF_RECORD_STAT
>   -1 -1 0x19a [0x40]: PERF_RECORD_MMAP -1/0: [0xffffffff81000000(0x1f000000) @ 0xffffffff81000000]: x [kernel.kallsyms]_text
>   0x1da [0x18]: PERF_RECORD_STAT_ROUND
>   [acme@ssdandy linux]$
> 
>   An effort was made to make perf.data files generated like this to not
>   generate cryptic messages when processed by older tools.
> 
>   The 'perf script' bits need rebasing, will go up later.

Nice feature!

> 
> Jiri's cover letter for this series:
> 
> The initial attempt defined its own formula lang and allowed triggering user's
> script on the end of the stat command:
> 
>   http://marc.info/?l=linux-kernel&m=136742146322273&w=2
> 
> This patchset abandons the idea of new formula language and rather adds support
> to:
> 
>   - store stat data into perf.data file
>   - add python support to process stat events
> 
> Basically it allows to store stat data into perf.data and post process it with
> python scripts in a similar way we do for sampling data.
> 
> The stat data are stored in new stat, stat-round, stat-config user events.
>   stat        - stored for each read syscall of the counter
>   stat round  - stored for each interval or end of the command invocation
>   stat config - stores all the config information needed to process data
>                 so report tool could restore the same output as record
> 
> The python script can now define 'stat__<eventname>_<modifier>' functions
> to get stat events data and 'stat__interval' to get stat-round data.
> 
> See CPI script example in scripts/python/stat-cpi.py.
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Jiri Olsa (43):
>       perf thread_map: Add thread_map user level event
>       perf thread_map: Add thread_map event sythesize function
>       perf thread_map: Add thread_map__new_event function
>       perf thread_map: Add perf_event__fprintf_thread_map function
>       perf cpu_map: Add cpu_map user level event
>       perf cpu_map: Add cpu_map event synthesize function
>       perf cpu_map: Add cpu_map__new_event function
>       perf cpu_map: Add perf_event__fprintf_cpu_map function
>       perf tools: Add stat config user level event
>       perf tools: Add stat config event synthesize function
>       perf tools: Add stat config event read function
>       perf tools: Add stat user level event
>       perf tools: Add stat event synthesize function
>       perf tools: Add stat event read function
>       perf tools: Add stat round user level event
>       perf tools: Add stat round event synthesize function
>       perf tools: Add stat events fprintf functions
>       perf tools: Add event_update user level event
>       perf tools: Add event_update event unit type
>       perf tools: Add event_update event scale type
>       perf tools: Add event_update event name type
>       perf tools: Add event_update event cpus type
>       perf tools: Add perf_event__fprintf_event_update function
>       perf report: Display newly added events in raw dump
>       perf tools: Introduce stat perf.data header feature
>       perf stat record: Add record command
>       perf stat record: Initialize record features
>       perf stat record: Synthesize stat record data
>       perf evlist: Export id_add_fd()
>       perf stat record: Store events IDs in perf data file
>       perf stat record: Add pipe support for record command
>       perf stat record: Write stat events on record
>       perf stat record: Write stat round events on record
>       perf stat record: Do not allow record with multiple runs mode
>       perf stat record: Synthesize event update events
>       perf stat report: Add report command
>       perf stat report: Process cpu/threads maps
>       perf stat report: Process stat config event
>       perf stat report: Add support to initialize aggr_map from file
>       perf stat report: Move csv_sep initialization before report command
>       perf stat report: Process stat and stat round events
>       perf stat report: Process event update events
>       perf stat report: Allow to override aggr_mode
> 
>  tools/perf/Documentation/perf-stat.txt |  34 ++
>  tools/perf/builtin-record.c            |   2 +
>  tools/perf/builtin-stat.c              | 614 ++++++++++++++++++++++++++++++++-
>  tools/perf/tests/Build                 |   3 +
>  tools/perf/tests/builtin-test.c        |  24 ++
>  tools/perf/tests/cpumap.c              |  88 +++++
>  tools/perf/tests/event_update.c        | 117 +++++++
>  tools/perf/tests/stat.c                | 111 ++++++
>  tools/perf/tests/tests.h               |   6 +
>  tools/perf/tests/thread-map.c          |  43 +++
>  tools/perf/util/cpumap.c               |  42 +++
>  tools/perf/util/cpumap.h               |   1 +
>  tools/perf/util/event.c                | 308 +++++++++++++++++
>  tools/perf/util/event.h                | 150 +++++++-
>  tools/perf/util/evlist.c               |   6 +-
>  tools/perf/util/evlist.h               |   3 +
>  tools/perf/util/header.c               | 205 +++++++++++
>  tools/perf/util/header.h               |  17 +
>  tools/perf/util/session.c              | 189 ++++++++++
>  tools/perf/util/stat.c                 |  62 ++++
>  tools/perf/util/stat.h                 |  10 +
>  tools/perf/util/thread_map.c           |  27 ++
>  tools/perf/util/thread_map.h           |   3 +
>  tools/perf/util/tool.h                 |   8 +-
>  24 files changed, 2054 insertions(+), 19 deletions(-)
>  create mode 100644 tools/perf/tests/cpumap.c
>  create mode 100644 tools/perf/tests/event_update.c
>  create mode 100644 tools/perf/tests/stat.c

Pulled, thanks a lot Arnaldo!

	Ingo