[PATCH V6 00/25] perf tools: Introduce an abstraction for AUX Area and Instruction Tracing

[Adrian Hunter <adrian.hunter@intel.com>]

Hi

Here is V6 of some more preparatory patches for Intel PT
that introduce an abstraction for using the AUX area and
Instruction tracing.

The master branch of the tree:

	git://git.infradead.org/users/ahunter/linux-perf.git

contains these patches and working Intel PT and Intel BTS.
Intel BTS can be used on most recent Intel CPUs. Intel PT
is available on Broadwell.

Examples:

	Trace 'ls' with Intel BTS userspace only

	perf record --per-thread -e intel_bts//u ls
	perf report
	perf script

	Trace 'ls' with Intel BTS kernel and userspace

	~/libexec/perf-core/perf-with-kcore record bts-ls --per-thread -e intel_bts// -- ls
	~/libexec/perf-core/perf-with-kcore report bts-ls
	~/libexec/perf-core/perf-with-kcore script bts-ls

	Trace 'ls' with Intel PT userspace only

	perf record -e intel_pt//u ls
	perf report
	perf script

	Trace 'ls' with Intel PT kernel and userspace

	~/libexec/perf-core/perf-with-kcore record pt-ls -e intel_pt// -- ls
	~/libexec/perf-core/perf-with-kcore report pt-ls
	~/libexec/perf-core/perf-with-kcore script pt-ls

Changes in V6:

	Renamed itrace to auxtrace to better reflect the use of the
	AUX area. In some cases itrace was retained where it seemed
	to relate exclusively to Instruction Tracing.
	
	Print warnings for separate auxtrace error types

	Re-based on Arnaldo's perf/core branch:

		107eb964d8c04417e8bce9e9ec2ed61d9261aec6
		Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core

Changes in V5:

	perf tools: Add a user event for Instruction Tracing errors
		Print error count from stats->nr_events

	perf session: Add hooks to allow transparent decoding of Instruction Tracing data
		Don't count Instruction Tracing errors (using stats->nr_events instead)

	perf itrace: Add helpers for Instruction Tracing errors
		Don't count Instruction Tracing errors (using stats->nr_events instead)

	perf itrace: Add option to synthesize events for transactions
		New patch

	perf script: Add field option 'flags' to print sample flags
		New patch

	Re-based on tip perf/core branch:

		94ac003b665fc04f13a7ab3b2be896b9b9503451
		Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core

Changes in V4:

	perf tools: Add build option NO_ITRACE to exclude Instruction Tracing
		New patch

	Re-based on Arnaldo's perf/core branch:

		3dd417d4010c8e141b0f32121cdc8d82aa4a9c6a
		perf tools: Remove some unused functions from color.c

Changes in V3:

	perf tools: Add support for Instruction Trace recording
		Added evsel as a parameter to itrace_record__init

	perf record: Add basic Instruction Tracing support
		Moved the call to itrace_record__init after parse
		options so that evsel could be passed and the
		selected events used to determine what kind of
		Instruction Tracing to use e.g. Intel PT vs BTS

	Re-based on Arnaldo's perf/core branch:

		41e950c033b7df997d4b38653efe6554be9b96a7
		Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent

Changes in V2:

	Dropped patches already applied.

	Re-based on Arnaldo's perf/core branch:

		a84808083688d82d7f1e5786ccf5df0ff7d448cb
		perf tools: Only override the default :tid comm entry


The abstraction has two separate aspects:
	1. recording AUX area data
	2. processing AUX area data

Recording consists of mmapping a separate buffer and copying
the data into the perf.data file.  The buffer is an AUX area
buffer although the details of the AUX area are not implemented
because the kernel support is pending.  The data is written
preceded by a new user event PERF_RECORD_AUXTRACE.  The data is
too big to fit in the event but follows immediately afterward.
Session processing has to skip to get to the next event header
in a similar fashion to the existing PERF_RECORD_HEADER_TRACING_DATA
event.  The main recording patches are:

      perf evlist: Add initial support for mmapping an AUX area buffer
      perf tools: Add user events for AUX area tracing
      perf tools: Add support for AUX area recording
      perf record: Add basic AUX area tracing support

Processing consists of providing hooks in session processing
to enable a decoder to see all the events and deliver synthesized
events transparently into the event stream.  The main processing
patch is:

      perf session: Add hooks to allow transparent decoding of AUX area tracing data


Adrian Hunter (25):
      perf header: Add AUX area tracing feature
      perf evlist: Add initial support for mmapping an AUX area buffer
      perf tools: Add user events for AUX area tracing
      perf tools: Add support for AUX area recording
      perf record: Add basic AUX area tracing support
      perf record: Extend -m option for AUX area tracing mmap pages
      perf tools: Add a user event for AUX area tracing errors
      perf session: Add hooks to allow transparent decoding of AUX area tracing data
      perf session: Add instruction tracing options
      perf auxtrace: Add helpers for AUX area tracing errors
      perf auxtrace: Add helpers for queuing AUX area tracing data
      perf auxtrace: Add a heap for sorting AUX area tracing queues
      perf auxtrace: Add processing for AUX area tracing events
      perf auxtrace: Add a hashtable for caching
      perf tools: Add member to struct dso for an instruction cache
      perf script: Add Instruction Tracing support
      perf script: Always allow fields 'addr' and 'cpu' for auxtrace
      perf report: Add Instruction Tracing support
      perf inject: Re-pipe AUX area tracing events
      perf inject: Add Instruction Tracing support
      perf tools: Add AUX area tracing index
      perf tools: Hit all build ids when AUX area tracing
      perf tools: Add build option NO_AUXTRACE to exclude AUX area tracing
      perf auxtrace: Add option to synthesize events for transactions
      perf script: Add field option 'flags' to print sample flags

 tools/perf/Documentation/perf-inject.txt |   27 +
 tools/perf/Documentation/perf-record.txt |    2 +
 tools/perf/Documentation/perf-report.txt |   29 +
 tools/perf/Documentation/perf-script.txt |   38 +-
 tools/perf/Makefile.perf                 |    2 +
 tools/perf/builtin-buildid-list.c        |    9 +
 tools/perf/builtin-inject.c              |  172 +++-
 tools/perf/builtin-record.c              |  167 +++-
 tools/perf/builtin-report.c              |   11 +
 tools/perf/builtin-script.c              |   74 +-
 tools/perf/config/Makefile               |    5 +
 tools/perf/perf.h                        |    2 +
 tools/perf/tests/make                    |    2 +
 tools/perf/util/Build                    |    1 +
 tools/perf/util/auxtrace.c               | 1293 ++++++++++++++++++++++++++++++
 tools/perf/util/auxtrace.h               |  575 +++++++++++++
 tools/perf/util/dso.c                    |    2 +
 tools/perf/util/dso.h                    |    3 +
 tools/perf/util/event.c                  |    3 +
 tools/perf/util/event.h                  |   46 ++
 tools/perf/util/evlist.c                 |   71 +-
 tools/perf/util/evlist.h                 |    6 +
 tools/perf/util/header.c                 |   37 +
 tools/perf/util/header.h                 |    1 +
 tools/perf/util/record.c                 |   11 +-
 tools/perf/util/session.c                |  169 +++-
 tools/perf/util/session.h                |    6 +
 tools/perf/util/tool.h                   |   10 +-
 28 files changed, 2720 insertions(+), 54 deletions(-)
 create mode 100644 tools/perf/util/auxtrace.c
 create mode 100644 tools/perf/util/auxtrace.h


Regards
Adrian