From: Wang Nan <wangnan0@huawei.com>
To: <acme@kernel.org>
Cc: <linux-kernel@vger.kernel.org>, <pi3orama@163.com>,
<lizefan@huawei.com>, <netdev@vger.kernel.org>,
<davem@davemloft.net>, Wang Nan <wangnan0@huawei.com>
Subject: [PATCH 00/53] perf tools: Bugfix, BPF improvement and perf record flight record mode
Date: Mon, 11 Jan 2016 13:47:51 +0000 [thread overview]
Message-ID: <1452520124-2073-1-git-send-email-wangnan0@huawei.com> (raw)
Hi Arnaldo,
This patch set is based on today's perf/core. It contains 3 parts:
1. Bugfix in my local tree. Most of them are resent (patch 1 - 17).
2. BPF related improvement. Also, you should have read them last
year. Nearly no change (18 - 26).
3. The most exciting feature I'd like to introduce to you and others:
perf record overwrite mode support:
This feature is based on a patch which is discussed but not merged
yet [1]. I also send it in this series as patch 27. In this patch,
kernel appends the size of an event at the end of the event data
in the ring buffer, which enables us reading as much data as
possible from a overwrite ring buffer, so it works like a flight
recorder. Patch 28 - 53 add support of it. This is an example:
# perf record -a -e cycles/overwrite/ \
-e raw_syscalls:sys_enter/overwrite/ \
-e raw_syscalls:sys_exit/overwrite/ \
-e sched:sched_switch/overwrite/ \
--switch-output --tail-tracking
Then send 3 SIGUSR2 to 'perf' in another console:
# kill -s SIGUSR2 `ps -e | grep 'pts.*perf' | awk '{print $1}'`
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2016011205392208 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2016011205392597 ]
[ perf record: dump data: Woken up 1 times ]
[ perf record: Dump perf.data.2016011205392906 ]
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Dump perf.data.2016011205393040
Here's the result:
# ls -l ./perf.data.*
-rw------- 1 root root 4284861 Jan 12 05:39
./perf.data.2016011205392208
-rw------- 1 root root 4578477 Jan 12 05:39
./perf.data.2016011205392597
-rw------- 1 root root 4602757 Jan 12 05:39
./perf.data.2016011205392906
-rw------- 1 root root 5655429 Jan 12 05:39
./perf.data.2016011205393040
In each perf.data output, we get about 4M events before it receives
signal.
This should be useful if we have a extra monitor checks performance
metrics. When it found something unusual, it can send a SIGUSR2 to
perf to collect data near the bad things happen.
My next step is trying to trigger event dumping using eBPF. Then we can
triggeer a perf.data output immediately after a system call takes too
long or when we detect a losting of a display update.
Patch 28 - 40 add a switch-output mode, make perf dump a new perf.data
when it receive a SIGUSR2.
Patch 41 - 45 introduce a concept called 'channel', which allows perf to
collect data through more than one group of mmaped ring buffer with different
configurations.
Patch 46 - 53 are the core of flight record mode. Patch 51 does real
reading from flight recorder ring buffer.
[1] http://lkml.kernel.org/g/1452518653-1794-1-git-send-email-wangnan0@huawei.com
He Kuang (1):
perf tools: Support perf event alias name
Jiri Olsa (1):
perf tools: Add missing sources in perf's MANIFEST
Naveen N. Rao (1):
perf: bpf: Fix build breakage due to libbpf
Wang Nan (50):
perf tools: Add -lutil in python lib list for broken python-config
perf tools: Fix phony build target for build-test
perf tools: Set parallel making options build-test
perf tools: Pass O option to Makefile.perf in build-test
perf tools: Test correct path of perf in build-test
perf tools: Fix PowerPC native building
tools: Move Makefile.arch from perf/config to tools/scripts
tools build: Add BPF feature check to test-all
perf test: Fix false TEST_OK result for 'perf test hist'
perf test: Reset err after using it hold errcode in hist testcases
perf tools: Prevent calling machine__delete() on non-allocated machine
perf test: Check environment before start real BPF test
perf tools: Fix symbols searching for offline module in buildid-cache
perf tools: Fix mmap2 event allocation in synthesize code
perf test: Improve bp_signal
perf tools: Add API to config maps in bpf object
perf tools: Enable BPF object configure syntax
perf record: Apply config to BPF objects before recording
perf tools: Enable passing event to BPF object
perf tools: Support setting different slots in a BPF map separately
perf tools: Enable indices setting syntax for BPF maps
perf tools: Introduce bpf-output event
perf data: Support converting data from bpf_perf_event_output()
perf/core: Put size of a sample at the end of it by
PERF_SAMPLE_TAILSIZE
perf tools: Move timestamp creation to util
perf tools: Make ordered_events reusable
perf record: Extract synthesize code to record__synthesize()
perf tools: Add perf_data_file__switch() helper
perf record: Turns auxtrace_snapshot_enable into 3 states
perf record: Introduce record__finish_output() to finish a perf.data
perf record: Use OPT_BOOLEAN_SET for buildid cache related options
perf record: Add '--timestamp-filename' option to append timestamp to
output filename
perf record: Split output into multiple files via '--switch-output'
perf record: Force enable --timestamp-filename when --switch-output is
provided
perf record: Disable buildid cache options by default in switch output
mode
perf record: Re-synthesize tracking events after output switching
perf record: Generate tracking events for process forked by perf
perf record: Ensure return non-zero rc when mmap fail
perf record: Prevent reading invalid data in record__mmap_read
perf tools: Add evlist channel helpers
perf tools: Automatically add new channel according to evlist
perf tools: Operate multiple channels
perf tools: Squash overwrite setting into channel
perf record: Don't read from and poll overwrite channel
perf tools: Enable overwrite settings
perf tools: Consider TAILSIZE bit when caclulate is_pos
perf tools: Set tailsize attribut bit for overwrite events
perf record: Read from tailsize ring buffer
perf record: Toggle tailsize ring buffer for reading
perf record: Allow generate tracking events at the end of output
include/linux/perf_event.h | 17 +-
include/uapi/linux/perf_event.h | 3 +-
kernel/events/core.c | 82 +++-
kernel/events/ring_buffer.c | 7 +-
tools/build/feature/test-all.c | 5 +
tools/build/feature/test-bpf.c | 20 +-
tools/lib/bpf/Makefile | 16 +-
tools/lib/bpf/bpf.c | 4 +-
tools/perf/MANIFEST | 2 +
tools/perf/builtin-buildid-cache.c | 14 +-
tools/perf/builtin-record.c | 549 +++++++++++++++++----
tools/perf/config/Makefile | 4 +-
tools/perf/perf.h | 1 +
tools/perf/tests/bp_signal.c | 140 +++++-
tools/perf/tests/bpf.c | 37 ++
tools/perf/tests/hists_common.c | 5 -
tools/perf/tests/hists_cumulate.c | 1 +
tools/perf/tests/hists_filter.c | 1 +
tools/perf/tests/hists_link.c | 1 +
tools/perf/tests/hists_output.c | 1 +
tools/perf/tests/make | 72 ++-
tools/perf/tests/vmlinux-kallsyms.c | 4 +-
tools/perf/util/bpf-loader.c | 699 +++++++++++++++++++++++++++
tools/perf/util/bpf-loader.h | 59 +++
tools/perf/util/build-id.c | 44 ++
tools/perf/util/build-id.h | 1 +
tools/perf/util/data-convert-bt.c | 112 ++++-
tools/perf/util/data.c | 36 ++
tools/perf/util/data.h | 11 +-
tools/perf/util/event.c | 28 +-
tools/perf/util/evlist.c | 307 ++++++++++--
tools/perf/util/evlist.h | 67 ++-
tools/perf/util/evsel.c | 42 +-
tools/perf/util/evsel.h | 13 +
tools/perf/util/machine.c | 13 +-
tools/perf/util/machine.h | 3 +-
tools/perf/util/ordered-events.c | 9 +
tools/perf/util/ordered-events.h | 1 +
tools/perf/util/parse-events.c | 139 +++++-
tools/perf/util/parse-events.h | 24 +-
tools/perf/util/parse-events.l | 18 +-
tools/perf/util/parse-events.y | 123 ++++-
tools/perf/util/session.c | 4 +-
tools/perf/util/symbol.c | 4 +
tools/perf/util/util.c | 17 +
tools/perf/util/util.h | 1 +
tools/{perf/config => scripts}/Makefile.arch | 0
47 files changed, 2482 insertions(+), 279 deletions(-)
rename tools/{perf/config => scripts}/Makefile.arch (100%)
--
1.8.3.4
next reply other threads:[~2016-01-11 13:47 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-11 13:47 Wang Nan [this message]
2016-01-11 13:47 ` [PATCH 01/53] perf tools: Add -lutil in python lib list for broken python-config Wang Nan
2016-01-12 9:43 ` Jiri Olsa
2016-01-11 13:47 ` [PATCH 02/53] perf tools: Fix phony build target for build-test Wang Nan
2016-01-11 13:47 ` [PATCH 03/53] perf tools: Set parallel making options build-test Wang Nan
2016-01-11 13:47 ` [PATCH 04/53] perf tools: Pass O option to Makefile.perf in build-test Wang Nan
2016-01-11 13:47 ` [PATCH 05/53] perf tools: Test correct path of perf " Wang Nan
2016-01-11 15:24 ` Arnaldo Carvalho de Melo
2016-01-11 22:06 ` Arnaldo Carvalho de Melo
2016-01-11 22:39 ` Arnaldo Carvalho de Melo
2016-01-11 22:39 ` Arnaldo Carvalho de Melo
2016-01-12 7:16 ` Wangnan (F)
2016-01-12 14:08 ` Arnaldo Carvalho de Melo
2016-01-11 13:47 ` [PATCH 06/53] perf tools: Fix PowerPC native building Wang Nan
2016-01-11 13:47 ` [PATCH 07/53] tools: Move Makefile.arch from perf/config to tools/scripts Wang Nan
2016-01-11 13:52 ` Wangnan (F)
2016-01-11 14:10 ` Arnaldo Carvalho de Melo
2016-01-11 13:47 ` [PATCH 08/53] perf tools: Add missing sources in perf's MANIFEST Wang Nan
2016-01-11 13:48 ` [PATCH 09/53] perf: bpf: Fix build breakage due to libbpf Wang Nan
2016-01-11 13:48 ` [PATCH 10/53] tools build: Add BPF feature check to test-all Wang Nan
2016-01-11 13:48 ` [PATCH 11/53] perf test: Fix false TEST_OK result for 'perf test hist' Wang Nan
2016-01-11 14:25 ` Sergei Shtylyov
2016-01-11 14:58 ` Arnaldo Carvalho de Melo
2016-01-11 15:32 ` Arnaldo Carvalho de Melo
2016-01-11 13:48 ` [PATCH 12/53] perf test: Reset err after using it hold errcode in hist testcases Wang Nan
2016-01-11 13:48 ` [PATCH 13/53] perf tools: Prevent calling machine__delete() on non-allocated machine Wang Nan
2016-01-11 15:42 ` Arnaldo Carvalho de Melo
2016-01-12 7:03 ` Wangnan (F)
2016-01-12 14:07 ` Arnaldo Carvalho de Melo
2016-01-11 13:48 ` [PATCH 14/53] perf test: Check environment before start real BPF test Wang Nan
2016-01-11 21:55 ` Arnaldo Carvalho de Melo
2016-01-12 7:40 ` Wangnan (F)
2016-01-12 14:10 ` Arnaldo Carvalho de Melo
2016-01-11 13:48 ` [PATCH 15/53] perf tools: Fix symbols searching for offline module in buildid-cache Wang Nan
2016-01-11 13:48 ` [PATCH 16/53] perf tools: Fix mmap2 event allocation in synthesize code Wang Nan
2016-01-11 13:48 ` [PATCH 17/53] perf test: Improve bp_signal Wang Nan
2016-01-11 21:37 ` Arnaldo Carvalho de Melo
2016-01-12 4:13 ` Wangnan (F)
2016-01-12 9:21 ` Jiri Olsa
2016-01-12 14:11 ` Arnaldo Carvalho de Melo
2016-01-12 14:17 ` Will Deacon
2016-01-11 13:48 ` [PATCH 18/53] perf tools: Add API to config maps in bpf object Wang Nan
2016-01-11 13:48 ` [PATCH 19/53] perf tools: Enable BPF object configure syntax Wang Nan
2016-01-11 13:48 ` [PATCH 20/53] perf record: Apply config to BPF objects before recording Wang Nan
2016-01-11 13:48 ` [PATCH 21/53] perf tools: Enable passing event to BPF object Wang Nan
2016-01-11 13:48 ` [PATCH 22/53] perf tools: Support perf event alias name Wang Nan
2016-01-11 13:48 ` [PATCH 23/53] perf tools: Support setting different slots in a BPF map separately Wang Nan
2016-01-11 13:48 ` [PATCH 24/53] perf tools: Enable indices setting syntax for BPF maps Wang Nan
2016-01-11 13:48 ` [PATCH 25/53] perf tools: Introduce bpf-output event Wang Nan
2016-01-11 13:48 ` [PATCH 26/53] perf data: Support converting data from bpf_perf_event_output() Wang Nan
2016-01-11 13:48 ` [PATCH 27/53] perf/core: Put size of a sample at the end of it by PERF_SAMPLE_TAILSIZE Wang Nan
2016-01-11 18:09 ` Alexei Starovoitov
2016-01-12 5:33 ` Wangnan (F)
2016-01-12 6:11 ` Alexei Starovoitov
2016-01-12 12:36 ` Wangnan (F)
2016-01-12 19:56 ` Alexei Starovoitov
2016-01-13 4:34 ` Wangnan (F)
2016-01-13 5:14 ` Alexei Starovoitov
2016-01-12 14:05 ` Peter Zijlstra
2016-01-12 14:14 ` Peter Zijlstra
2016-01-11 13:48 ` [PATCH 28/53] perf tools: Move timestamp creation to util Wang Nan
2016-01-11 13:48 ` [PATCH 29/53] perf tools: Make ordered_events reusable Wang Nan
2016-01-11 21:33 ` Arnaldo Carvalho de Melo
2016-01-11 13:48 ` [PATCH 30/53] perf record: Extract synthesize code to record__synthesize() Wang Nan
2016-01-11 13:48 ` [PATCH 31/53] perf tools: Add perf_data_file__switch() helper Wang Nan
2016-01-11 13:48 ` [PATCH 32/53] perf record: Turns auxtrace_snapshot_enable into 3 states Wang Nan
2016-01-11 13:48 ` [PATCH 33/53] perf record: Introduce record__finish_output() to finish a perf.data Wang Nan
2016-01-11 13:48 ` [PATCH 34/53] perf record: Use OPT_BOOLEAN_SET for buildid cache related options Wang Nan
2016-01-11 13:48 ` [PATCH 35/53] perf record: Add '--timestamp-filename' option to append timestamp to output filename Wang Nan
2016-01-11 13:48 ` [PATCH 36/53] perf record: Split output into multiple files via '--switch-output' Wang Nan
2016-01-11 13:48 ` [PATCH 37/53] perf record: Force enable --timestamp-filename when --switch-output is provided Wang Nan
2016-01-11 13:48 ` [PATCH 38/53] perf record: Disable buildid cache options by default in switch output mode Wang Nan
2016-01-11 13:48 ` [PATCH 39/53] perf record: Re-synthesize tracking events after output switching Wang Nan
2016-01-11 13:48 ` [PATCH 40/53] perf record: Generate tracking events for process forked by perf Wang Nan
2016-01-11 13:48 ` [PATCH 41/53] perf record: Ensure return non-zero rc when mmap fail Wang Nan
2016-01-11 13:48 ` [PATCH 42/53] perf record: Prevent reading invalid data in record__mmap_read Wang Nan
2016-01-11 14:21 ` Sergei Shtylyov
2016-01-11 15:00 ` Arnaldo Carvalho de Melo
2016-01-11 15:01 ` Arnaldo Carvalho de Melo
2016-01-11 13:48 ` [PATCH 43/53] perf tools: Add evlist channel helpers Wang Nan
2016-01-11 13:48 ` [PATCH 44/53] perf tools: Automatically add new channel according to evlist Wang Nan
2016-01-11 13:48 ` [PATCH 45/53] perf tools: Operate multiple channels Wang Nan
2016-01-11 13:48 ` [PATCH 46/53] perf tools: Squash overwrite setting into channel Wang Nan
2016-01-11 13:48 ` [PATCH 47/53] perf record: Don't read from and poll overwrite channel Wang Nan
2016-01-11 13:48 ` [PATCH 48/53] perf tools: Enable overwrite settings Wang Nan
2016-01-11 13:48 ` [PATCH 49/53] perf tools: Consider TAILSIZE bit when caclulate is_pos Wang Nan
2016-01-11 13:48 ` [PATCH 50/53] perf tools: Set tailsize attribut bit for overwrite events Wang Nan
2016-01-11 13:48 ` [PATCH 51/53] perf record: Read from tailsize ring buffer Wang Nan
2016-01-11 13:48 ` [PATCH 52/53] perf record: Toggle tailsize ring buffer for reading Wang Nan
2016-01-11 13:48 ` [PATCH 53/53] perf record: Allow generate tracking events at the end of output Wang Nan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1452520124-2073-1-git-send-email-wangnan0@huawei.com \
--to=wangnan0@huawei.com \
--cc=acme@kernel.org \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=netdev@vger.kernel.org \
--cc=pi3orama@163.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).