linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ingo Molnar <mingo@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Dave Jones <davej@redhat.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Linux Kernel <linux-kernel@vger.kernel.org>,
	Frederic Weisbecker <fweisbec@gmail.com>,
	jolsa@redhat.com
Subject: Re: trinity finds ftrace/perf bug. Film at 11.
Date: Thu, 3 Oct 2013 18:57:53 +0200	[thread overview]
Message-ID: <20131003165753.GC32755@gmail.com> (raw)
In-Reply-To: <20131003154401.GO31370@twins.programming.kicks-ass.net>


* Peter Zijlstra <peterz@infradead.org> wrote:

> On Thu, Sep 12, 2013 at 02:38:49PM -0400, Dave Jones wrote:
> 
> > The current one I'm staring at is this from LIST_DEBUG..
> > 
> > list_del corruption. next->prev should be prev (ffff88000fb812b0), but was ffff88000fb812b0. (next=ffff88009df8b7b0).
> > 
> > The sharp eyed will notice that those first two addresses are actually the same.
> > So we matched the next->prev != prev test, but at the time we got to the printk that follows
> > in the WARN, the inverse was true. Smells like a race of some kind.
> 
> FWIW, I've not given up on this. I've found one bug in the list_entry
> handling already -- although not this one.
> 
> Today I hit a more revealing error:
> 
> [ 8348.150303] list_del corruption. prev->next should be ffff88042839c108, but was 6b6b6b6b6b6b6b6b
> 
> And as we all know that 6b is POISON_FREE, so we're staring at a
> use-after-free here.
> 
> Brain did give out for today though.. more staring at reference counts
> tomorrow.
> 
> The sad thing is that reproduction takes forever on my machine; I need 
> to let trinity run for a good hour and then ^C abort the run. Only then 
> will I hit it fairly reliably.

Maybe run trinity in an infinite loop and also run this script in the 
background:

	while :; do killall -SIGINT trinity; sleep $[RANDOM/500]; done &

this will Ctrl-C trinity every 0...60 seconds, randomized, average 30 
seconds runtime.

Thanks,

	Ingo

  reply	other threads:[~2013-10-03 16:57 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-11 13:54 trinity finds ftrace/perf bug. Film at 11 Dave Jones
2013-09-12 18:19 ` Steven Rostedt
2013-09-12 18:38   ` Dave Jones
2013-10-03 15:44     ` Peter Zijlstra
2013-10-03 16:57       ` Ingo Molnar [this message]
2013-09-13 10:56   ` Peter Zijlstra
2013-09-13 14:16     ` Dave Jones
2013-09-16  9:37       ` Peter Zijlstra
2013-09-16 10:25         ` Ingo Molnar
2013-09-16 10:43           ` Peter Zijlstra
2013-09-16 16:45             ` Dave Jones
2013-09-16 16:44         ` Dave Jones
2013-09-18 11:34           ` Peter Zijlstra
2013-09-18 13:22             ` Dave Jones
2013-09-13 14:32     ` Steven Rostedt
2013-09-13 14:58       ` Peter Zijlstra
2013-09-13 11:14   ` Peter Zijlstra
2013-09-13 14:28     ` Dave Jones
2013-09-13 14:58       ` Peter Zijlstra
2013-11-19 19:18     ` [tip:perf/urgent] perf: Remove fragile swevent hlist optimization tip-bot for Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20131003165753.GC32755@gmail.com \
    --to=mingo@kernel.org \
    --cc=davej@redhat.com \
    --cc=fweisbec@gmail.com \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).