[GIT PULL 00/22] perf/core improvements and fixes

[Arnaldo Carvalho de Melo <acme@kernel.org>]

From: Arnaldo Carvalho de Melo <acme@redhat.com>

Hi Ingo,

	Please consider pulling, I had most of this queued before your first
pull req to Linus for 4.10, most are fixes, with 'perf sched timehist --idle'
as a followup new feature to the 'perf sched timehist' command introduced in
this window.

Thanks,

- Arnaldo

Test results at the end of this message, as usual.

The following changes since commit b0c1ef52959582144bbea9a2b37db7f4c9e399f7:

  perf/x86: Fix exclusion of BTS and LBR for Goldmont (2016-12-11 13:06:09 +0100)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20161213

for you to fetch changes up to a03f73547fb6e0f7f2942c46cce9b48df50238ba:

  samples/bpf: Drop unnecessary build targets. (2016-12-13 10:38:10 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

- Introduce 'perf sched timehist --idle', to analyse processes
  going to/from idle state (Namhyung Kim)

- Add scanning of SDT (Software Defined Tracing) probles arguments (Alexis Berlemont)

Fixes:

- Allow 'perf record -u user' to continue when facing races with threads
  going away after having scanned them via /proc (Jiri Olsa)

- Fix 'perf mem' --all-user/--all-kernel options (Jiri Olsa)

Infrastructure:

- Switch over samples/bpf/ to tools/lib/bpf, removing libbpf duplication (Joe Stringer)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Alexis Berlemont (1):
      perf sdt: Add scanning of sdt probles arguments

Arnaldo Carvalho de Melo (1):
      perf tools: Remove some needless __maybe_unused

Jiri Olsa (6):
      perf tools: Move headers check into bash script
      perf mem: Fix --all-user/--all-kernel options
      perf evsel: Use variable instead of repeating lengthy FD macro
      perf thread_map: Add thread_map__remove function
      perf evsel: Allow to ignore missing pid
      perf record: Force ignore_missing_thread for uid option

Joe Stringer (8):
      tools lib bpf: Sync {tools,}/include/uapi/linux/bpf.h
      tools lib bpf: use __u32 from linux/types.h
      tools lib bpf: Add flags to bpf_create_map()
      samples/bpf: Make samples more libbpf-centric
      samples/bpf: Switch over to libbpf
      samples/bpf: Remove perf_event_open() declaration
      samples/bpf: Move open_raw_sock to separate header
      samples/bpf: Drop unnecessary build targets.

Namhyung Kim (6):
      perf sched timehist: Split is_idle_sample()
      perf sched timehist: Introduce struct idle_time_data
      perf sched timehist: Save callchain when entering idle
      perf sched timehist: Skip non-idle events when necessary
      perf sched timehist: Add -I/--idle-hist option
      perf sched timehist: Show callchains for idle stat

 samples/bpf/Makefile                              |  60 ++---
 samples/bpf/README.rst                            |   4 +-
 samples/bpf/bpf_load.c                            |  20 +-
 samples/bpf/fds_example.c                         |  10 +-
 samples/bpf/lathist_user.c                        |   3 +-
 samples/bpf/libbpf.c                              | 155 -------------
 samples/bpf/libbpf.h                              |  25 +--
 samples/bpf/map_perf_test_user.c                  |   1 +
 samples/bpf/offwaketime_user.c                    |  10 +-
 samples/bpf/sampleip_user.c                       |   8 +-
 samples/bpf/sock_example.c                        |  11 +-
 samples/bpf/sock_example.h                        |  35 +++
 samples/bpf/sockex1_user.c                        |   9 +-
 samples/bpf/sockex2_user.c                        |   7 +-
 samples/bpf/sockex3_user.c                        |   7 +-
 samples/bpf/spintest_user.c                       |  10 +-
 samples/bpf/tc_l2_redirect_user.c                 |   4 +-
 samples/bpf/test_cgrp2_array_pin.c                |   4 +-
 samples/bpf/test_current_task_under_cgroup_user.c |  10 +-
 samples/bpf/test_maps.c                           | 142 ++++++------
 samples/bpf/test_overhead_user.c                  |   2 +
 samples/bpf/test_probe_write_user_user.c          |   4 +-
 samples/bpf/test_verifier.c                       |   8 +-
 samples/bpf/trace_event_user.c                    |  24 +-
 samples/bpf/trace_output_user.c                   |   6 +-
 samples/bpf/tracex1_user.c                        |   2 +
 samples/bpf/tracex2_user.c                        |  12 +-
 samples/bpf/tracex3_user.c                        |   6 +-
 samples/bpf/tracex4_user.c                        |   6 +-
 samples/bpf/tracex5_user.c                        |   2 +
 samples/bpf/tracex6_user.c                        |   7 +-
 samples/bpf/xdp1_user.c                           |   4 +-
 tools/include/uapi/linux/bpf.h                    |  51 +++++
 tools/lib/bpf/bpf.c                               |   7 +-
 tools/lib/bpf/bpf.h                               |   6 +-
 tools/lib/bpf/libbpf.c                            |   3 +-
 tools/perf/Documentation/perf-sched.txt           |   4 +
 tools/perf/Makefile.perf                          |  94 +-------
 tools/perf/builtin-c2c.c                          |  13 +-
 tools/perf/builtin-mem.c                          |   4 +-
 tools/perf/builtin-record.c                       |   3 +
 tools/perf/builtin-report.c                       |   2 +-
 tools/perf/builtin-sched.c                        | 261 +++++++++++++++++++---
 tools/perf/builtin-stat.c                         |   6 +-
 tools/perf/check-headers.sh                       |  59 +++++
 tools/perf/perf.h                                 |   1 +
 tools/perf/tests/builtin-test.c                   |   4 +
 tools/perf/tests/tests.h                          |   1 +
 tools/perf/tests/thread-map.c                     |  44 ++++
 tools/perf/util/evsel.c                           |  61 ++++-
 tools/perf/util/evsel.h                           |   1 +
 tools/perf/util/symbol-elf.c                      |  25 ++-
 tools/perf/util/symbol.h                          |   1 +
 tools/perf/util/thread_map.c                      |  22 ++
 tools/perf/util/thread_map.h                      |   1 +
 55 files changed, 786 insertions(+), 506 deletions(-)
 delete mode 100644 samples/bpf/libbpf.c
 create mode 100644 samples/bpf/sock_example.h
 create mode 100755 tools/perf/check-headers.sh

  # uname -a
  Linux jouet 4.9.0-rc8+ #1 SMP Mon Dec 12 11:20:49 BRT 2016 x86_64 x86_64 x86_64 GNU/Linux
  # perf test
   1: vmlinux symtab matches kallsyms            : Ok
   2: Detect openat syscall event                : Ok
   3: Detect openat syscall event on all cpus    : Ok
   4: Read samples using the mmap interface      : Ok
   5: Parse event definition strings             : Ok
   6: PERF_RECORD_* events & perf_sample fields  : Ok
   7: Parse perf pmu format                      : Ok
   8: DSO data read                              : Ok
   9: DSO data cache                             : Ok
  10: DSO data reopen                            : Ok
  11: Roundtrip evsel->name                      : Ok
  12: Parse sched tracepoints fields             : Ok
  13: syscalls:sys_enter_openat event fields     : Ok
  14: Setup struct perf_event_attr               : Ok
  15: Match and link multiple hists              : Ok
  16: 'import perf' in python                    : Ok
  17: Breakpoint overflow signal handler         : Ok
  18: Breakpoint overflow sampling               : Ok
  19: Number of exit events of a simple workload : Ok
  20: Software clock events period values        : Ok
  21: Object code reading                        : Ok
  22: Sample parsing                             : Ok
  23: Use a dummy software event to keep tracking: Ok
  24: Parse with no sample_id_all bit set        : Ok
  25: Filter hist entries                        : Ok
  26: Lookup mmap thread                         : Ok
  27: Share thread mg                            : Ok
  28: Sort output of hist entries                : Ok
  29: Cumulate child hist entries                : Ok
  30: Track with sched_switch                    : Ok
  31: Filter fds with revents mask in a fdarray  : Ok
  32: Add fd to a fdarray, making it autogrow    : Ok
  33: kmod_path__parse                           : Ok
  34: Thread map                                 : Ok
  35: LLVM search and compile                    :
  35.1: Basic BPF llvm compile                    : Ok
  35.2: kbuild searching                          : Ok
  35.3: Compile source for BPF prologue generation: Ok
  35.4: Compile source for BPF relocation         : Ok
  36: Session topology                           : Ok
  37: BPF filter                                 :
  37.1: Basic BPF filtering                      : Ok
  37.2: BPF prologue generation                  : Ok
  37.3: BPF relocation checker                   : Ok
  38: Synthesize thread map                      : Ok
  39: Remove thread map                          : Ok
  40: Synthesize cpu map                         : Ok
  41: Synthesize stat config                     : Ok
  42: Synthesize stat                            : Ok
  43: Synthesize stat round                      : Ok
  44: Synthesize attr update                     : Ok
  45: Event times                                : Ok
  46: Read backward ring buffer                  : Ok
  47: Print cpu map                              : Ok
  48: Probe SDT events                           : Ok
  49: is_printable_array                         : Ok
  50: Print bitmap                               : Ok
  51: perf hooks                                 : Ok
  52: builtin clang support                      : Skip (not compiled in)
  53: x86 rdpmc                                  : Ok
  54: Convert perf time to TSC                   : Ok
  55: DWARF unwind                               : Ok
  56: x86 instruction decoder - new instructions : Ok
  57: Intel cqm nmi context read                 : Skip
  #
  # time dm
     1 alpine:3.4: Ok
     2 android-ndk:r12b-arm: Ok
     3 archlinux:latest: Ok
     4 centos:5: Ok
     5 centos:6: Ok
     6 centos:7: Ok
     7 debian:7: Ok
     8 debian:8: Ok
     9 debian:experimental: Ok
    10 debian:experimental-x-mips64: Ok
    11 fedora:20: Ok
    12 fedora:21: Ok
    13 fedora:22: Ok
    14 fedora:23: Ok
    15 fedora:24: Ok
    16 fedora:24-x-ARC-uClibc: Ok
    17 fedora:25: Ok
    18 fedora:rawhide: Ok
    19 mageia:5: Ok
    20 opensuse:13.2: Ok
    21 opensuse:42.1: Ok
    22 opensuse:tumbleweed: Ok
    23 ubuntu:12.04.5: Ok
    24 ubuntu:14.04.4-x-linaro-arm64: Ok
    25 ubuntu:16.04: Ok
    26 ubuntu:16.04-x-arm: Ok
    27 ubuntu:16.04-x-arm64: Ok
    28 ubuntu:16.04-x-powerpc: Ok
    29 ubuntu:16.04-x-powerpc64: Ok
    30 ubuntu:16.04-x-powerpc64el: Ok
    31 ubuntu:16.04-x-s390: Ok
    32 ubuntu:16.10: Ok
  #
  [acme@felicio linux]$ make -C tools/perf build-test
  make: Entering directory `/home/acme/git/linux/tools/perf'
  - tarpkg: ./tests/perf-targz-src-pkg .
        make_with_babeltrace_O: make LIBBABELTRACE=1
         make_with_clangllvm_O: make LIBCLANGLLVM=1
                    make_doc_O: make doc
                 make_cscope_O: make cscope
                  make_debug_O: make DEBUG=1
             make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
                 make_perf_o_O: make perf.o
            make_install_bin_O: make install-bin
                make_no_newt_O: make NO_NEWT=1
               make_no_slang_O: make NO_SLANG=1
              make_clean_all_O: make clean all
                   make_help_O: make help
           make_no_libbionic_O: make NO_LIBBIONIC=1
  make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
            make_no_demangle_O: make NO_DEMANGLE=1
              make_no_libelf_O: make NO_LIBELF=1
                   make_pure_O: make
             make_no_libnuma_O: make NO_LIBNUMA=1
              make_no_libbpf_O: make NO_LIBBPF=1
                make_no_gtk2_O: make NO_GTK2=1
           make_no_libpython_O: make NO_LIBPYTHON=1
             make_no_libperl_O: make NO_LIBPERL=1
            make_no_libaudit_O: make NO_LIBAUDIT=1
                 make_static_O: make LDFLAGS=-static
             make_util_map_o_O: make util/map.o
   make_install_prefix_slash_O: make install prefix=/tmp/krava/
         make_install_prefix_O: make install prefix=/tmp/krava
       make_util_pmu_bison_o_O: make util/pmu-bison.o
                   make_tags_O: make tags
                  make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
                make_install_O: make install
            make_no_auxtrace_O: make NO_AUXTRACE=1
           make_no_backtrace_O: make NO_BACKTRACE=1
                make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1 NO_JVMTI=1
           make_no_libunwind_O: make NO_LIBUNWIND=1
  OK
  make: Leaving directory `/home/acme/git/linux/tools/perf'
  [acme@felicio linux]$