From: Dave Jones <davej@redhat.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Linux Kernel <linux-kernel@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
Ingo Molnar <mingo@kernel.org>
Subject: Re: trinity finds ftrace/perf bug. Film at 11.
Date: Thu, 12 Sep 2013 14:38:49 -0400 [thread overview]
Message-ID: <20130912183849.GA16674@redhat.com> (raw)
In-Reply-To: <20130912141913.2ffc7077@gandalf.local.home>
On Thu, Sep 12, 2013 at 02:19:13PM -0400, Steven Rostedt wrote:
>
> This is something that either Peter or Frederic need to look at.
I spent yesterday trying to bisect this, but kept running into other perf/ftrace traces,
which makes it pretty much impossible.
The current one I'm staring at is this from LIST_DEBUG..
list_del corruption. next->prev should be prev (ffff88000fb812b0), but was ffff88000fb812b0. (next=ffff88009df8b7b0).
The sharp eyed will notice that those first two addresses are actually the same.
So we matched the next->prev != prev test, but at the time we got to the printk that follows
in the WARN, the inverse was true. Smells like a race of some kind.
[ 2481.228418] Call Trace:
[ 2481.232705] [<ffffffff8171f5dd>] dump_stack+0x54/0x74
[ 2481.237025] [<ffffffff810540bd>] warn_slowpath_common+0x7d/0xa0
[ 2481.241357] [<ffffffff8105412c>] warn_slowpath_fmt+0x4c/0x50
[ 2481.245637] [<ffffffff810540e5>] ? warn_slowpath_fmt+0x5/0x50
[ 2481.249890] [<ffffffff8132c222>] __list_del_entry+0x62/0xf0
[ 2481.254109] [<ffffffff81143a88>] list_del_event+0xc8/0xe0
[ 2481.258310] [<ffffffff81143b2b>] perf_remove_from_context+0x8b/0xf0
[ 2481.262497] [<ffffffff81145444>] perf_event_release_kernel+0x54/0x90
[ 2481.266688] [<ffffffff811464d3>] put_event+0x133/0x1c0
[ 2481.270866] [<ffffffff811463d0>] ? put_event+0x30/0x1c0
[ 2481.275066] [<ffffffff81146570>] perf_release+0x10/0x20
[ 2481.279277] [<ffffffff811bf2ca>] __fput+0xfa/0x2e0
[ 2481.283488] [<ffffffff811bf4fe>] ____fput+0xe/0x10
[ 2481.287713] [<ffffffff8107c11c>] task_work_run+0xac/0xe0
[ 2481.291958] [<ffffffff810559bc>] do_exit+0x2cc/0xcc0
[ 2481.296194] [<ffffffff81732880>] ? ftrace_call+0x5/0x2f
[ 2481.300435] [<ffffffff810577cc>] do_group_exit+0x4c/0xc0
[ 2481.304654] [<ffffffff81057854>] SyS_exit_group+0x14/0x20
[ 2481.308857] [<ffffffff81732e54>] tracesys+0xdd/0xe2
[ 2481.313051] ---[ end trace fa3e551eb20e0c0f ]---
Dave
next prev parent reply other threads:[~2013-09-12 18:39 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-11 13:54 trinity finds ftrace/perf bug. Film at 11 Dave Jones
2013-09-12 18:19 ` Steven Rostedt
2013-09-12 18:38 ` Dave Jones [this message]
2013-10-03 15:44 ` Peter Zijlstra
2013-10-03 16:57 ` Ingo Molnar
2013-09-13 10:56 ` Peter Zijlstra
2013-09-13 14:16 ` Dave Jones
2013-09-16 9:37 ` Peter Zijlstra
2013-09-16 10:25 ` Ingo Molnar
2013-09-16 10:43 ` Peter Zijlstra
2013-09-16 16:45 ` Dave Jones
2013-09-16 16:44 ` Dave Jones
2013-09-18 11:34 ` Peter Zijlstra
2013-09-18 13:22 ` Dave Jones
2013-09-13 14:32 ` Steven Rostedt
2013-09-13 14:58 ` Peter Zijlstra
2013-09-13 11:14 ` Peter Zijlstra
2013-09-13 14:28 ` Dave Jones
2013-09-13 14:58 ` Peter Zijlstra
2013-11-19 19:18 ` [tip:perf/urgent] perf: Remove fragile swevent hlist optimization tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130912183849.GA16674@redhat.com \
--to=davej@redhat.com \
--cc=fweisbec@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.