[GIT PULL 00/59] perf/core improvements and fixes

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Hi Ingo,

        Please consider pulling, more to come, bpf, make build-test and
hopefully Stephane's JIT stuff, etc.

Best Regards,

- Arnaldo

The following changes since commit 9cc2617de5b9222abb39cd02e90d57dfea99c6d7:

  perf/x86/amd: Remove l1-dcache-stores event for AMD (2016-01-06 11:15:39 +0100)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo

for you to fetch changes up to 775d8a1b0d75211cc6123915c6b5b688f2002478:

  perf evlist: Add --trace-fields option to show trace fields (2016-01-08 14:23:02 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

- Allow using trace events fields as sort order keys, making 'perf evlist --trace_fields'
  show those, and then the user can select a subset and use like:

    perf top -e sched:sched_switch -s prev_comm,next_comm

  That works as well in 'perf report' when handling files containing
  tracepoints.

  The default when just tracepoint events are found in a perf.data file is to
  format it like ftrace, using the libtraceevent formatters, plugins, etc (Namhyung Kim)

- Add support in 'perf script' to process 'perf stat record' generated files,
  culminating in a python perf script that calculates CPI (Cycles per
  Instruction) (Jiri Olsa)

- Show random perf tool tips in the 'perf report' bottom line (Namhyung Kim)

- perf report now defaults to --group if the perf.data file has grouped events, try it with:

  # perf record -e '{cycles,instructions}' -a sleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 1.093 MB perf.data (1247 samples) ]
  # perf report
  # Samples: 1K of event 'anon group { cycles, instructions }'
  # Event count (approx.): 1955219195
  #
  #       Overhead  Command     Shared Object      Symbol

     2.86%   0.22%  swapper     [kernel.kallsyms]  [k] intel_idle
     1.05%   0.33%  firefox     libxul.so          [.] js::SetObjectElement
     1.05%   0.00%  kworker/0:3 [kernel.kallsyms]  [k] gen6_ring_get_seqno
     0.88%   0.17%  chrome      chrome             [.] 0x0000000000ee27ab
     0.65%   0.86%  firefox     libxul.so          [.] js::ValueToId<(js::AllowGC)1>
     0.64%   0.23%  JS Helper   libxul.so          [.] js::SplayTree<js::jit::LiveRange*, js::jit::LiveRange>::splay
     0.62%   1.27%  firefox     libxul.so          [.] js::GetIterator
     0.61%   1.74%  firefox     libxul.so          [.] js::NativeSetProperty
     0.61%   0.31%  firefox     libxul.so          [.] js::SetPropertyByDefining

User visible fixes:

- Coect data mmaps so that the DWARF unwinder can handle usecases needing them,
  like softice (Jiri Olsa)

- Decay callchains in fractal mode, fixing up cases where 'perf top -g' would
  show entries with more than 100% (Namhyung Kim)

Infrastructure:

- Sync tools/lib with the lib/ in the kernel sources for find_bit.c and
  move bitmap.[ch] from tools/perf/util/ to tools/lib/ (Arnaldo Carvalho de Melo)

- No need to set attr.sample_freq in some 'perf test' entries that only
  want to deal with PERF_RECORD_ meta-events, improve a bit error output
  for CQM test (Arnaldo Carvalho de Melo)

- Fix python binding build, adding some missing object files now required
  due to cpumap using find_bit stuff (Arnaldo Carvalho de Melo)

- tools/build improvemnts (Jiri Olsa)

- Add more files to cscope/ctags databases (Jiri Olsa)

- Do not show 'trace' in 'perf help' if it is not compiled in (Jiri Olsa)

- Make perf_evlist__open() open evsels with their cpus and threads,
  like perf record does, making them consistent (Adrian Hunter)

- Fix pmu snapshot initialization bug (Stephane Eranian)

- Add missing headers in perf's MANIFEST (Wang Nan)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Adrian Hunter (1):
      perf evlist: Make perf_evlist__open() open evsels with their cpus and threads (like perf record does)

Arnaldo Carvalho de Melo (10):
      perf tests: No need to set attr.sample_freq in the perf time to TSC test
      perf evlist: Introduce perf_evlist__new_dummy constructor
      perf test: Use "dummy" events in the PERF_RECORD_ test
      perf test: No need for setting attr.sample_freq on the RECORD test
      perf python: Add missing files to binding link list
      perf tests: No need to set attr.sample_freq for tracking !PERF_RECORD_SAMPLE
      perf tests: Give a bit more information on the CQM test failure path
      tools lib: Move find_next_bit.c to tools/lib/
      tools lib: Sync tools/lib/find_bit.c with the kernel
      tools lib: Move bitmap.[ch] from tools/perf/ to tools/{lib,include}/

Jiri Olsa (25):
      tools build feature: Fix feature_check_display_code typo
      tools build feature: Move dwarf post unwind choice output into perf
      tools build feature: Introduce feature_assign macro
      tools build feature: Use value assignment form for FEATURE-DUMP file
      perf build: Use FEATURE-DUMP in bpf subproject
      perf stat record: Keep sample_type 0 for pipe session
      perf script: Process cpu/threads maps
      perf script: Process stat config event
      perf script: Add process_stat/process_stat_interval scripting interface
      perf script: Add stat default handlers
      perf script: Add python support for stat events
      perf cpumap: Fix cpu conversion in cpu_map__from_entries
      perf script: Display stat events by default
      perf script: Add stat-cpi.py script
      perf tools: Do not show trace command if it's not compiled in
      perf script: Align event name properly
      perf tools: Include all tools/lib directory for tags/cscope/TAGS targets
      perf tools: Remove list entry from struct sort_entry
      perf tools: Add overhead/overhead_children keys defaults via string
      perf diff: Use perf_hpp__register_sort_field interface
      perf evlist: Remove perf_evlist__(enable|disable)_event functions
      perf unwind: Use find_map function in access_dso_mem
      perf unwind: Check for mmaps also in MAP__VARIABLE tree
      perf libdw: Check for mmaps also in MAP__VARIABLE tree
      perf record: Store data mmaps for dwarf unwind

Namhyung Kim (21):
      perf hist: Pass struct sample to __hists__add_entry()
      perf hist: Save raw_data/size for tracepoint events
      tools lib traceevent: Factor out and export print_event_field[s]()
      perf top: Create the evlist sooner
      perf tools: Pass evlist to setup_sorting()
      perf tools: Add dynamic sort key for tracepoint events
      perf tools: Try to show pretty printed output for dynamic sort keys
      perf tools: Add 'trace' sort key
      perf report/top: Add --raw-trace option
      perf tools: Support shortcuts for events in dynamic sort keys
      perf tools: Support '<event>.*' dynamic sort key
      perf tools: Skip dynamic fields not defined for current event
      perf tools: Add 'trace_fields' dynamic sort key
      perf tools: Make 'trace' or 'trace_fields' sort key default for tracepoint events
      perf tools: Add all matching dynamic sort keys for field name
      perf report: Add documentation for dynamic sort keys
      perf top: Decay periods in callchains
      perf report: Change default to use event group view
      perf hists: Export a couple of hist functions
      perf report: Show random usage tip on the help line
      perf evlist: Add --trace-fields option to show trace fields

Stephane Eranian (1):
      perf pmu: fix alias->snapshot missing initialization bug

Wang Nan (1):
      perf tools: Add missing headers in perf's MANIFEST

 tools/build/Makefile.feature                       |  41 +-
 tools/{perf/util => }/include/linux/bitmap.h       |   2 +
 tools/{perf/util => lib}/bitmap.c                  |   0
 tools/lib/bpf/Makefile                             |   4 +
 tools/lib/find_bit.c                               |  84 +++
 tools/lib/traceevent/event-parse.c                 | 129 ++---
 tools/lib/traceevent/event-parse.h                 |   4 +
 tools/lib/util/find_next_bit.c                     |  89 ---
 tools/perf/Build                                   |   1 +
 tools/perf/Documentation/perf-evlist.txt           |   3 +
 tools/perf/Documentation/perf-report.txt           |  27 +
 tools/perf/Documentation/perf-top.txt              |   3 +
 tools/perf/Documentation/tips.txt                  |  14 +
 tools/perf/MANIFEST                                |   4 +-
 tools/perf/Makefile.perf                           |   7 +-
 tools/perf/arch/x86/tests/intel-cqm.c              |   2 +-
 tools/perf/arch/x86/tests/perf-time-to-tsc.c       |   1 -
 tools/perf/arch/x86/util/intel-bts.c               |   4 +-
 tools/perf/arch/x86/util/intel-pt.c                |   4 +-
 tools/perf/builtin-annotate.c                      |   9 +-
 tools/perf/builtin-diff.c                          |  15 +-
 tools/perf/builtin-evlist.c                        |  11 +-
 tools/perf/builtin-record.c                        |   6 +-
 tools/perf/builtin-report.c                        |   6 +-
 tools/perf/builtin-script.c                        | 170 +++++-
 tools/perf/builtin-stat.c                          |   9 +-
 tools/perf/builtin-top.c                           |  16 +-
 tools/perf/command-list.txt                        |   2 +-
 tools/perf/config/Makefile                         |  10 +
 tools/perf/perf.c                                  |   4 +
 tools/perf/scripts/python/stat-cpi.py              |  77 +++
 tools/perf/tests/hists_cumulate.c                  |   8 +-
 tools/perf/tests/hists_filter.c                    |   2 +-
 tools/perf/tests/hists_link.c                      |   8 +-
 tools/perf/tests/hists_output.c                    |  10 +-
 tools/perf/tests/keep-tracking.c                   |   3 +-
 tools/perf/tests/perf-record.c                     |   6 +-
 tools/perf/tests/switch-tracking.c                 |   6 +-
 tools/perf/ui/browsers/hists.c                     |   7 +-
 tools/perf/ui/gtk/hists.c                          |   4 +-
 tools/perf/ui/hist.c                               |  14 +-
 tools/perf/ui/stdio/hist.c                         |   6 +-
 tools/perf/util/Build                              |  11 +-
 tools/perf/util/callchain.c                        |  28 +
 tools/perf/util/callchain.h                        |   1 +
 tools/perf/util/cpumap.c                           |  13 +-
 tools/perf/util/evlist.c                           |  72 +--
 tools/perf/util/evlist.h                           |   7 +-
 tools/perf/util/evsel.c                            |  23 +
 tools/perf/util/evsel.h                            |   1 +
 tools/perf/util/generate-cmdlist.sh                |  15 +
 tools/perf/util/hist.c                             |  78 ++-
 tools/perf/util/hist.h                             |  24 +-
 tools/perf/util/pmu.c                              |   1 +
 tools/perf/util/python-ext-sources                 |   2 +
 .../util/scripting-engines/trace-event-python.c    | 115 +++-
 tools/perf/util/sort.c                             | 601 ++++++++++++++++++++-
 tools/perf/util/sort.h                             |  12 +-
 tools/perf/util/symbol.c                           |   1 +
 tools/perf/util/symbol.h                           |   3 +-
 tools/perf/util/trace-event.h                      |   4 +
 tools/perf/util/unwind-libdw.c                     |  10 +
 tools/perf/util/unwind-libunwind.c                 |  20 +-
 tools/perf/util/util.c                             |  27 +
 tools/perf/util/util.h                             |   2 +
 65 files changed, 1556 insertions(+), 347 deletions(-)
 rename tools/{perf/util => }/include/linux/bitmap.h (95%)
 rename tools/{perf/util => lib}/bitmap.c (100%)
 create mode 100644 tools/lib/find_bit.c
 delete mode 100644 tools/lib/util/find_next_bit.c
 create mode 100644 tools/perf/Documentation/tips.txt
 create mode 100644 tools/perf/scripts/python/stat-cpi.py