[GIT PULL 00/10] perf/core improvements and fixes

[GIT PULL 00/10] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Hi Ingo,

	Please consider pulling,

- Arnaldo

The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:

  Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620

for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:

  perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

. Add --dry-run option to 'perf record' to check if command line options can be
  parsed, but not doing any recording (Wang Nan)

. Allow dumping the object files generated by llvm when processing eBPF
  scriptlet events (Wang Nan)

- Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)

Documentation:

. Fix 'perf script'  documentation of '-f' when it should be '-F' (Adrian Hunter)

Infrastructure:

- Fix write_backwards fallback when using a new tool on older kernels
  without support for this feature (Arnaldo Carvalho de Melo)

- Find vdso file in cross-platform perf.data file processing scenarios (He Kuang)

- Remove some leftovers from the initial codebase copying from git
  (Arnaldo Carvalho de Melo)

- List libelf-devel as an alternative, as this is how the libelf
  development package is called on OpenSuSE (Jean Delvare)

- Rename __hists__add_entry to hists__add_entry (Jiri Olsa)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Adrian Hunter (1):
      perf script: Fix documentation of '-f' when it should be '-F'

Arnaldo Carvalho de Melo (3):
      perf tools: Remove some unused functions
      perf tools: Remove --perf-dir and --work-dir
      perf evsel: Fix write_backwards fallback

He Kuang (1):
      perf tools: Find vdso with the consider of cross-platform

Jean Delvare (1):
      kbuild: List libelf-devel as an alternative

Jiri Olsa (1):
      perf hists: Rename __hists__add_entry to hists__add_entry

Paolo Bonzini (1):
      perf script: Add stackcollapse.py script

Wang Nan (2):
      perf llvm: Allow dump llvm output object file using llvm.dump-obj
      perf record: Add --dry-run option to check cmdline options

 Makefile                                           |   2 +-
 tools/perf/Documentation/perf-record.txt           |   7 ++
 tools/perf/Documentation/perf-script.txt           |  20 ++--
 tools/perf/builtin-annotate.c                      |   2 +-
 tools/perf/builtin-diff.c                          |  12 +-
 tools/perf/builtin-record.c                        |   7 ++
 tools/perf/config/Makefile                         |   2 +-
 tools/perf/perf.c                                  |  36 ------
 tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
 tools/perf/scripts/python/bin/stackcollapse-report |   3 +
 tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
 tools/perf/tests/hists_link.c                      |   4 +-
 tools/perf/util/cache.h                            |   9 --
 tools/perf/util/config.c                           |   2 +-
 tools/perf/util/evsel.c                            |  23 ++--
 tools/perf/util/hist.c                             |  34 +++---
 tools/perf/util/hist.h                             |  14 +--
 tools/perf/util/llvm-utils.c                       |  42 +++++++
 tools/perf/util/llvm-utils.h                       |   5 +
 tools/perf/util/path.c                             |  65 +----------
 tools/perf/util/vdso.c                             |  40 ++++++-
 21 files changed, 288 insertions(+), 176 deletions(-)
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
 create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
 create mode 100755 tools/perf/scripts/python/stackcollapse.py

[PATCH 02/10] kbuild: List libelf-devel as an alternative

[Arnaldo Carvalho de Melo]

From: Jean Delvare <jdelvare@suse.de>

On openSUSE, the libelf development files are in package libelf-devel.

Signed-off-by: Jean Delvare <jdelvare@suse.de>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Michal Marek <mmarek@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kbuild@vger.kernel.org
Link: http://lkml.kernel.org/n/tip-s8nyk3pyy2927sd7qp7u42oi@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 Makefile                   | 2 +-
 tools/perf/config/Makefile | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index b409076c7c18..4a1fe88d6839 100644
--- a/Makefile
+++ b/Makefile
@@ -1038,7 +1038,7 @@ ifdef CONFIG_STACK_VALIDATION
   ifeq ($(has_libelf),1)
     objtool_target := tools/objtool FORCE
   else
-    $(warning "Cannot use CONFIG_STACK_VALIDATION, please install libelf-dev or elfutils-libelf-devel")
+    $(warning "Cannot use CONFIG_STACK_VALIDATION, please install libelf-dev, libelf-devel or elfutils-libelf-devel")
     SKIP_STACK_VALIDATION := 1
     export SKIP_STACK_VALIDATION
   endif
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 80018feb99c0..534c81176f6c 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -257,7 +257,7 @@ else
       LIBC_SUPPORT := 1
     endif
     ifeq ($(LIBC_SUPPORT),1)
-      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install elfutils-libelf-devel/libelf-dev);
+      msg := $(warning No libelf found, disables 'probe' tool and BPF support in 'perf record', please install libelf-dev, libelf-devel or elfutils-libelf-devel);
 
       NO_LIBELF := 1
       NO_DWARF := 1
-- 
2.5.5

Re: [GIT PULL 00/10] perf/core improvements and fixes

[Namhyung Kim]

Hi Arnaldo,

On Mon, Jun 20, 2016 at 07:23:20PM -0300, Arnaldo Carvalho de Melo wrote:
> Hi Ingo,
> 
> 	Please consider pulling,
> 
> - Arnaldo
> 
> The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:
> 
>   Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620
> 
> for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:
> 
>   perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)
> 
> ----------------------------------------------------------------
> perf/core improvements and fixes:
> 
> New features:
> 
> . Add --dry-run option to 'perf record' to check if command line options can be
>   parsed, but not doing any recording (Wang Nan)
> 
> . Allow dumping the object files generated by llvm when processing eBPF
>   scriptlet events (Wang Nan)
> 
> - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)

I think this is already done by '-g folded'.  Please see:

  http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html

Thanks,
Namhyung


> 
> Documentation:
> 
> . Fix 'perf script'  documentation of '-f' when it should be '-F' (Adrian Hunter)
> 
> Infrastructure:
> 
> - Fix write_backwards fallback when using a new tool on older kernels
>   without support for this feature (Arnaldo Carvalho de Melo)
> 
> - Find vdso file in cross-platform perf.data file processing scenarios (He Kuang)
> 
> - Remove some leftovers from the initial codebase copying from git
>   (Arnaldo Carvalho de Melo)
> 
> - List libelf-devel as an alternative, as this is how the libelf
>   development package is called on OpenSuSE (Jean Delvare)
> 
> - Rename __hists__add_entry to hists__add_entry (Jiri Olsa)
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Adrian Hunter (1):
>       perf script: Fix documentation of '-f' when it should be '-F'
> 
> Arnaldo Carvalho de Melo (3):
>       perf tools: Remove some unused functions
>       perf tools: Remove --perf-dir and --work-dir
>       perf evsel: Fix write_backwards fallback
> 
> He Kuang (1):
>       perf tools: Find vdso with the consider of cross-platform
> 
> Jean Delvare (1):
>       kbuild: List libelf-devel as an alternative
> 
> Jiri Olsa (1):
>       perf hists: Rename __hists__add_entry to hists__add_entry
> 
> Paolo Bonzini (1):
>       perf script: Add stackcollapse.py script
> 
> Wang Nan (2):
>       perf llvm: Allow dump llvm output object file using llvm.dump-obj
>       perf record: Add --dry-run option to check cmdline options
> 
>  Makefile                                           |   2 +-
>  tools/perf/Documentation/perf-record.txt           |   7 ++
>  tools/perf/Documentation/perf-script.txt           |  20 ++--
>  tools/perf/builtin-annotate.c                      |   2 +-
>  tools/perf/builtin-diff.c                          |  12 +-
>  tools/perf/builtin-record.c                        |   7 ++
>  tools/perf/config/Makefile                         |   2 +-
>  tools/perf/perf.c                                  |  36 ------
>  tools/perf/scripts/python/bin/stackcollapse-record |   8 ++
>  tools/perf/scripts/python/bin/stackcollapse-report |   3 +
>  tools/perf/scripts/python/stackcollapse.py         | 127 +++++++++++++++++++++
>  tools/perf/tests/hists_link.c                      |   4 +-
>  tools/perf/util/cache.h                            |   9 --
>  tools/perf/util/config.c                           |   2 +-
>  tools/perf/util/evsel.c                            |  23 ++--
>  tools/perf/util/hist.c                             |  34 +++---
>  tools/perf/util/hist.h                             |  14 +--
>  tools/perf/util/llvm-utils.c                       |  42 +++++++
>  tools/perf/util/llvm-utils.h                       |   5 +
>  tools/perf/util/path.c                             |  65 +----------
>  tools/perf/util/vdso.c                             |  40 ++++++-
>  21 files changed, 288 insertions(+), 176 deletions(-)
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
>  create mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
>  create mode 100755 tools/perf/scripts/python/stackcollapse.py

Re: [GIT PULL 00/10] perf/core improvements and fixes

[Brendan Gregg]

On Mon, Jun 20, 2016 at 7:18 PM, Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hi Arnaldo,
>
> On Mon, Jun 20, 2016 at 07:23:20PM -0300, Arnaldo Carvalho de Melo wrote:
> > Hi Ingo,
> >
> >       Please consider pulling,
> >
> > - Arnaldo
> >
> > The following changes since commit 02469a95096a549508c5adf61d84a1d72851c85b:
> >
> >   Merge tag 'perf-core-for-mingo-20160615' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-06-16 10:27:35 +0200)
> >
> > are available in the git repository at:
> >
> >   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160620
> >
> > for you to fetch changes up to 2a0a7c72702bac1b87cd4d49196a334483386fab:
> >
> >   perf script: Add stackcollapse.py script (2016-06-20 17:50:39 -0300)
> >
> > ----------------------------------------------------------------
> > perf/core improvements and fixes:
> >
> > New features:
> >
> > . Add --dry-run option to 'perf record' to check if command line options can be
> >   parsed, but not doing any recording (Wang Nan)
> >
> > . Allow dumping the object files generated by llvm when processing eBPF
> >   scriptlet events (Wang Nan)
> >
> > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
>
> I think this is already done by '-g folded'.  Please see:
>
>   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
>

Pretty much. Two similar solutions were developed around the same
time. Although I have to use some awk to get "perf -g folded" in the
exact right format, and stackcollapse-perf.py does that directly.

Brendan

Brendan Gregg, Senior Performance Architect, Netflix

Re: [GIT PULL 00/10] perf/core improvements and fixes

[Paolo Bonzini]

On 21/06/2016 05:11, Brendan Gregg wrote:
>>> > > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
>> >
>> > I think this is already done by '-g folded'.  Please see:
>> >
>> >   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
>> >
> Pretty much. Two similar solutions were developed around the same
> time. Although I have to use some awk to get "perf -g folded" in the
> exact right format, and stackcollapse-perf.py does that directly.

Yes, the idea for stackcollapse-perf.py was:

- to do everything directly and emit "the right" format for the
flamegraph tools.  This however is a very minimal output, and it's not
necessarily the best for perf in general.

- to provide an example of visiting callgraphs from the scripting
interface, since it's not really documented.  From the commit message:
"Add stackcollapse.py script as an example of parsing call chains, and
also of using optparse to access command line options".

Paolo

Re: [GIT PULL 00/10] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Em Tue, Jun 21, 2016 at 12:08:47PM +0200, Paolo Bonzini escreveu:
> On 21/06/2016 05:11, Brendan Gregg wrote:
> >>> > > - Add stackcollapse.py script to help generating flame graphs (Paolo Bonzini)
> >> >
> >> > I think this is already done by '-g folded'.  Please see:
> >> >
> >> >   http://www.brendangregg.com/blog/2016-04-30/linux-perf-folded.html
> >> >
> > Pretty much. Two similar solutions were developed around the same
> > time. Although I have to use some awk to get "perf -g folded" in the
> > exact right format, and stackcollapse-perf.py does that directly.
> 
> Yes, the idea for stackcollapse-perf.py was:
> 
> - to do everything directly and emit "the right" format for the
> flamegraph tools.  This however is a very minimal output, and it's not
> necessarily the best for perf in general.
> 
> - to provide an example of visiting callgraphs from the scripting
> interface, since it's not really documented.  From the commit message:
> "Add stackcollapse.py script as an example of parsing call chains, and
> also of using optparse to access command line options".

Yeah, agreed, its short, and it provides the output ready to be used, no
need for extra processing, up to the user to decide if one or the other
is more suitable to their needs.

As a bonus, it documents callgraphs in scripts, good deal :-)

- Arnaldo