From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Ingo Molnar <mingo@kernel.org>
Cc: linux-kernel@vger.kernel.org,
Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>,
Jiri Olsa <jolsa@redhat.com>,
Li Zhang <zhlcindy@linux.vnet.ibm.com>,
Arnaldo Carvalho de Melo <acme@redhat.com>
Subject: [PATCH 6/8] perf trace: Fix race condition at the end of started workloads
Date: Wed, 17 Jun 2015 18:22:33 -0300 [thread overview]
Message-ID: <1434576155-30038-7-git-send-email-acme@kernel.org> (raw)
In-Reply-To: <1434576155-30038-1-git-send-email-acme@kernel.org>
From: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
I get following crash on multiple systems and across several releases
(at least since v3.18).
Core was generated by `/tmp/perf trace sleep 0.2 '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
195 u64 head = ACCESS_ONCE(pc->data_head);
(gdb) bt
#0 perf_mmap__read_head (mm=0x3fff9bf30070) at util/evlist.h:195
#1 perf_evlist__mmap_read (evlist=0x10027f11910, idx=<optimized out>)
at util/evlist.c:637
#2 0x000000001003ce4c in trace__run (argv=<optimized out>,
argc=<optimized out>, trace=0x3fffd7b28288) at builtin-trace.c:2259
#3 cmd_trace (argc=<optimized out>, argv=<optimized out>,
prefix=<optimized out>) at builtin-trace.c:2799
#4 0x00000000100657b8 in run_builtin (p=0x10176798 <commands+480>, argc=3,
argv=0x3fffd7b2b550) at perf.c:370
#5 0x00000000100063e8 in handle_internal_command (argv=0x3fffd7b2b550, argc=3)
at perf.c:429
#6 run_argv (argv=0x3fffd7b2af70, argcp=0x3fffd7b2af7c) at perf.c:473
#7 main (argc=3, argv=0x3fffd7b2b550) at perf.c:588
The problem seems to be a race condition, when the application has just
exited. Some/all fds associated with the perf-events (tracepoints) go
into a POLLHUP/ POLLERR state and the mmap region associated with those
events are unmapped (in perf_evlist__filter_pollfd()).
But we go back and do a perf_evlist__mmap_read() which assumes that the
mmaps are still valid and we hit the crash.
If the mapping for an event is released, its refcnt is 0 (and ->base
is NULL), so ensure we have non-zero refcount before accessing the map.
Note that perf-record has a similar logic but unlike perf-trace, the
record__mmap_read_all() checks the evlist->mmap[i].base before accessing
the map.
Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Li Zhang <zhlcindy@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20150612060003.GA19913@us.ibm.com
[ Fixed it up to use atomic_read() ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/evlist.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index dc1dc2c181ef..6b58a47a79ec 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -634,11 +634,18 @@ static struct perf_evsel *perf_evlist__event2evsel(struct perf_evlist *evlist,
union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx)
{
struct perf_mmap *md = &evlist->mmap[idx];
- u64 head = perf_mmap__read_head(md);
+ u64 head;
u64 old = md->prev;
unsigned char *data = md->base + page_size;
union perf_event *event = NULL;
+ /*
+ * Check if event was unmapped due to a POLLHUP/POLLERR.
+ */
+ if (!atomic_read(&md->refcnt))
+ return NULL;
+
+ head = perf_mmap__read_head(md);
if (evlist->overwrite) {
/*
* If we're further behind than half the buffer, there's a chance
--
2.1.0
next prev parent reply other threads:[~2015-06-17 21:24 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-17 21:22 [GIT PULL 0/8] perf/core improvements and fixes Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 1/8] perf tools: Ignore .config-detected in .gitignore Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 2/8] perf tools: Fix a problem when opening old perf.data with different byte order Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 3/8] perf tools: Move libtraceevent dynamic list to separated LDFLAGS variable Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 4/8] perf probe: Show usage even if the last event is skipped Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 5/8] perf probe: Speed up perf probe --list by caching debuginfo Arnaldo Carvalho de Melo
2015-06-17 21:22 ` Arnaldo Carvalho de Melo [this message]
2015-06-17 21:22 ` [PATCH 7/8] perf evlist: Add toggle_enable() method Arnaldo Carvalho de Melo
2015-06-17 21:22 ` [PATCH 8/8] perf top: Allow disabling/enabling events dynamicly Arnaldo Carvalho de Melo
2015-06-18 7:40 ` [GIT PULL 0/8] perf/core improvements and fixes Ingo Molnar
2015-06-18 20:18 ` [RFC] hotkey for disabling/enabling events in 'perf top' TUI was " Arnaldo Carvalho de Melo
2015-06-18 20:58 ` Ingo Molnar
2015-06-18 21:39 ` Arnaldo Carvalho de Melo
2015-06-19 6:27 ` Ingo Molnar
2015-06-19 20:07 ` Arnaldo Carvalho de Melo
2015-06-19 23:05 ` Ingo Molnar
2015-06-22 14:52 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1434576155-30038-7-git-send-email-acme@kernel.org \
--to=acme@kernel.org \
--cc=acme@redhat.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=sukadev@linux.vnet.ibm.com \
--cc=zhlcindy@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.