linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC 0/23] perf tool: Add support for multiple data file storage
@ 2013-07-17 17:49 Jiri Olsa
  2013-07-17 17:49 ` [PATCH 01/23] perf tools: Use session->fd instead of passing fd as argument Jiri Olsa
                   ` (22 more replies)
  0 siblings, 23 replies; 49+ messages in thread
From: Jiri Olsa @ 2013-07-17 17:49 UTC (permalink / raw)
  To: linux-kernel
  Cc: Jiri Olsa, Corey Ashford, Frederic Weisbecker, Ingo Molnar,
	Namhyung Kim, Paul Mackerras, Peter Zijlstra,
	Arnaldo Carvalho de Melo, Andi Kleen, David Ahern

hi,
this *RFC* adds support for record command to split perf
data file output into multiple files. Each such data file
is fully-fledged and readable upon by any perf command.

This patchset is actually doing 3 essential things:
  1) adds new perf data format version (v3)
  2) adds class/object to handle perf data file
  3) adds record '-M size' option

ad 1) Along the way I realized it'd be much easier to have the
      format simplified. I ended up with just 'DATA' and 'FEATURES'
      sections making the file format more generic.

      The easier/generic format will be for example useful for the
      cpu hotplug notification (and maybe even for ftrace data files
      features?)

      The version 2 should be still readable by perf, while it
      produces data in version 3 from this patchset onwards.

ad 2) I needed to unify perf data file handling to make the -M
      implementation clear.

Loose ends/TODO/ideas.. like:
  - test/fix bugs
  - more automated tests for version 2 data reading so we're
    sure we don't break v2 compatibility.
  - record '-M time'
  - separate work thread to finish the data file (write header),
    while record command continues to poll/read/store to new file
  - perf daemon running on background storing continuously data 
  - perf top storing data on key press

It's reachable here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/file22

thanks for comments,
jirka


Example:
  $ ./perf record -M 100000 -e cycles:u yes > /dev/null
  ^C[ perf record: Woken up 25 times to write data ]
  [ perf record: Captured and wrote 0.630 MB perf-[0-7].data(~27531 samples) ]
  yes: Interrupt
  $ ls -l perf-0000*
  -rw------- 1 jolsa jolsa 111828 Jul 17 18:23 perf-00000.data
  -rw------- 1 jolsa jolsa 115240 Jul 17 18:23 perf-00001.data
  -rw------- 1 jolsa jolsa 112020 Jul 17 18:23 perf-00002.data
  -rw------- 1 jolsa jolsa 112020 Jul 17 18:23 perf-00003.data
  -rw------- 1 jolsa jolsa 112120 Jul 17 18:23 perf-00004.data
  -rw------- 1 jolsa jolsa 112120 Jul 17 18:23 perf-00005.data
  -rw------- 1 jolsa jolsa  18900 Jul 17 18:23 perf-00006.data
  $ ./perf diff perf-0000*
  # Event 'cycles:u'
  #
  # Data files:
  #  [0] perf-00000.data (Baseline)
  #  [1] perf-00001.data
  #  [2] perf-00002.data
  #  [3] perf-00003.data
  #  [4] perf-00004.data
  #  [5] perf-00005.data
  #  [6] perf-00006.data
  #
  # Baseline/0  Delta/1  Delta/2  Delta/3  Delta/4  Delta/5  Delta/6      Shared Object                              Symbol
  # ..........  .......  .......  .......  .......  .......  .......  .................  ..................................
  #
        37.03%   +1.23%   +0.12%   +0.61%   +1.12%   +0.93%   -7.53%  libc-2.15.so       [.] _IO_file_xsputn@@GLIBC_2.2.5
        31.40%   -0.55%   -0.79%   -1.00%   -0.66%   -0.59%   +4.09%  yes                [.] main
        16.36%   -0.18%   +1.39%   +1.07%   +0.66%   +0.23%   +1.72%  libc-2.15.so       [.] __strlen_sse2
        14.66%   -0.61%   -0.81%   -0.58%   -1.18%   -0.26%   +1.66%  libc-2.15.so       [.] fputs_unlocked
         0.24%   +0.34%   +0.27%   +0.08%   +0.24%   -0.12%   +0.35%  yes                [.] fputs_unlocked@plt
         0.12%                     -0.08%   -0.08%   -0.08%           libc-2.15.so       [.] _IO_file_write@@GLIBC_2.2.5
         0.08%                                                        libc-2.15.so       [.] new_do_write
         0.04%                                                        libc-2.15.so       [.] _IO_default_xsputn
  ...



Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Ahern <dsahern@gmail.com>
---
Jiri Olsa (23):
      perf tools: Use session->fd instead of passing fd as argument
      perf tools: Remove data_offset seek as it's not needed
      perf tools: Remove attr_offset from perf_header
      perf tools: Introduce feat_offset into perf_header
      perf tests: Add simple session read/write test
      perf doc: Add perf data file documentation
      perf tools: Recognize version number for perf data file
      perf tools: Introduce perf data file version CHECK macro
      perf tools: Introduce swap_features function
      perf tools: Introduce swap_header function
      perf tools: Separate version 2 specific perf data header bits
      perf tools: Using evlist as a holder for event_desc feature
      perf tools: Introduce perf.data version 3 format
      perf tools: Add perf data version 3 header swap
      perf tools: Add perf data version 3 header read
      perf tools: Add perf.data version 3 header write
      perf tools: Get rid of post_processing_offset in record command
      perf tools: Move synthetizing into single function
      perf tools: Add data object to handle perf data file
      perf tools: Add perf_data_file__open interface to data object
      perf tools: Separating data file properties from session
      perf tools: Add multi file '-M' option for record command
      perf tools: Have the process properly sythesized in subsequent data files

 tools/perf/Documentation/perf-data-file-v2.txt | 256 ++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/Documentation/perf-data-file-v3.txt |  60 ++++++++++++
 tools/perf/Makefile                            |   2 +
 tools/perf/builtin-annotate.c                  |  11 ++-
 tools/perf/builtin-buildid-cache.c             |   8 +-
 tools/perf/builtin-buildid-list.c              |  11 ++-
 tools/perf/builtin-diff.c                      |  19 ++--
 tools/perf/builtin-evlist.c                    |   7 +-
 tools/perf/builtin-inject.c                    |   9 +-
 tools/perf/builtin-kmem.c                      |   7 +-
 tools/perf/builtin-kvm.c                       |   8 +-
 tools/perf/builtin-lock.c                      |   8 +-
 tools/perf/builtin-mem.c                       |   9 +-
 tools/perf/builtin-record.c                    | 400 +++++++++++++++++++++++++++++++++++++++++++++++++-------------------------
 tools/perf/builtin-report.c                    |  18 +++-
 tools/perf/builtin-sched.c                     |   6 +-
 tools/perf/builtin-script.c                    |  16 ++-
 tools/perf/builtin-timechart.c                 |  10 +-
 tools/perf/builtin-top.c                       |   7 +-
 tools/perf/perf.h                              |   2 +-
 tools/perf/tests/builtin-test.c                |   4 +
 tools/perf/tests/session-simple.c              | 275 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h                       |   1 +
 tools/perf/util/data.c                         | 120 +++++++++++++++++++++++
 tools/perf/util/data.h                         |  48 +++++++++
 tools/perf/util/evsel.c                        |   5 +
 tools/perf/util/header.c                       | 450 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------------
 tools/perf/util/header.h                       |  49 +++++++---
 tools/perf/util/session.c                      | 116 ++++++++--------------
 tools/perf/util/session.h                      |  11 +--
 30 files changed, 1501 insertions(+), 452 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-data-file-v2.txt
 create mode 100644 tools/perf/Documentation/perf-data-file-v3.txt
 create mode 100644 tools/perf/tests/session-simple.c
 create mode 100644 tools/perf/util/data.c
 create mode 100644 tools/perf/util/data.h

^ permalink raw reply	[flat|nested] 49+ messages in thread

end of thread, other threads:[~2013-07-22 13:45 UTC | newest]

Thread overview: 49+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-07-17 17:49 [RFC 0/23] perf tool: Add support for multiple data file storage Jiri Olsa
2013-07-17 17:49 ` [PATCH 01/23] perf tools: Use session->fd instead of passing fd as argument Jiri Olsa
2013-07-19  7:54   ` [tip:perf/core] perf session: Use session-> fd " tip-bot for Jiri Olsa
2013-07-17 17:49 ` [PATCH 02/23] perf tools: Remove data_offset seek as it's not needed Jiri Olsa
2013-07-18  7:27   ` Namhyung Kim
2013-07-18 12:46     ` Jiri Olsa
2013-07-19  7:54   ` [tip:perf/core] perf header: Remove data_offset seek as it' s " tip-bot for Jiri Olsa
2013-07-17 17:49 ` [PATCH 03/23] perf tools: Remove attr_offset from perf_header Jiri Olsa
2013-07-19  7:54   ` [tip:perf/core] perf header: " tip-bot for Jiri Olsa
2013-07-17 17:49 ` [PATCH 04/23] perf tools: Introduce feat_offset into perf_header Jiri Olsa
2013-07-19  7:54   ` [tip:perf/core] perf header: " tip-bot for Jiri Olsa
2013-07-17 17:49 ` [PATCH 05/23] perf tests: Add simple session read/write test Jiri Olsa
2013-07-18  7:46   ` Namhyung Kim
2013-07-18 12:52     ` Jiri Olsa
     [not found]   ` <20130717193313.GA5127@infradead.org>
2013-07-18 12:52     ` Jiri Olsa
2013-07-18 14:26     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 06/23] perf doc: Add perf data file documentation Jiri Olsa
2013-07-17 19:59   ` Arnaldo Carvalho de Melo
2013-07-18 12:40     ` Jiri Olsa
2013-07-19 15:46   ` Andi Kleen
2013-07-17 17:49 ` [PATCH 07/23] perf tools: Recognize version number for perf data file Jiri Olsa
2013-07-19  7:54   ` [tip:perf/core] perf header: " tip-bot for Jiri Olsa
2013-07-17 17:49 ` [PATCH 08/23] perf tools: Introduce perf data file version CHECK macro Jiri Olsa
     [not found]   ` <20130717194250.GB5127@infradead.org>
2013-07-18 12:39     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 09/23] perf tools: Introduce swap_features function Jiri Olsa
2013-07-17 17:49 ` [PATCH 10/23] perf tools: Introduce swap_header function Jiri Olsa
2013-07-19 11:33   ` Namhyung Kim
2013-07-22 13:44     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 11/23] perf tools: Separate version 2 specific perf data header bits Jiri Olsa
2013-07-17 17:49 ` [PATCH 12/23] perf tools: Using evlist as a holder for event_desc feature Jiri Olsa
2013-07-17 17:49 ` [PATCH 13/23] perf tools: Introduce perf.data version 3 format Jiri Olsa
2013-07-19 12:06   ` Namhyung Kim
2013-07-22 10:00     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 14/23] perf tools: Add perf data version 3 header swap Jiri Olsa
2013-07-17 17:49 ` [PATCH 15/23] perf tools: Add perf data version 3 header read Jiri Olsa
2013-07-19 12:11   ` Namhyung Kim
2013-07-22 10:02     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 16/23] perf tools: Add perf.data version 3 header write Jiri Olsa
2013-07-17 17:49 ` [PATCH 17/23] perf tools: Get rid of post_processing_offset in record command Jiri Olsa
2013-07-17 17:49 ` [PATCH 18/23] perf tools: Move synthetizing into single function Jiri Olsa
2013-07-19 12:30   ` Namhyung Kim
2013-07-22 10:15     ` Jiri Olsa
2013-07-17 17:49 ` [PATCH 19/23] perf tools: Add data object to handle perf data file Jiri Olsa
2013-07-17 17:50 ` [PATCH 20/23] perf tools: Add perf_data_file__open interface to data object Jiri Olsa
2013-07-17 17:50 ` [PATCH 21/23] perf tools: Separating data file properties from session Jiri Olsa
2013-07-17 17:50 ` [PATCH 22/23] perf tools: Add multi file '-M' option for record command Jiri Olsa
2013-07-19 13:02   ` Namhyung Kim
2013-07-22 10:17     ` Jiri Olsa
2013-07-17 17:50 ` [PATCH 23/23] perf tools: Have the process properly sythesized in subsequent data files Jiri Olsa

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).