The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Steven Rostedt <srostedt@redhat.com>
To: Jiaying Zhang <jiayingz@google.com>
Cc: Ingo Molnar <mingo@elte.hu>, Michael Rubin <mrubin@google.com>,
	David Sharp <dhsharp@google.com>,
	linux-kernel@vger.kernel.org,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: lockup in rb_get_reader_page
Date: Tue, 30 Mar 2010 20:35:24 -0400	[thread overview]
Message-ID: <1269995724.22564.58.camel@localhost.localdomain> (raw)
In-Reply-To: <m2u5df78e1d1003301627ya1f621f7qaf0ad099f62d787e@mail.gmail.com>

Hi Jiaying,


On Tue, 2010-03-30 at 16:27 -0700, Jiaying Zhang wrote:
> Hi Steven,
> 
> We recently saw some cpu lockups while running kernel tracing.
> The problem started to happen after we synced up our ring buffer
> code with the upstream lockless ring buffer change. It usually
> took many hours and heavy trace load to hit this problem. When
> the lockup happens, the problematic cpu seemed to be in an infinite
> loop of trying to grab the head_page in rb_get_reader_page().
> 
> We would like to check with you on whether this is a known issue.
> If so, do we have a bug fix? If not, do you have any suggestions on
> where we should check?

I'm unaware of any problems with the ring buffer. I've been running the
lockless version for over a year now, and hammering it with very
intensive tracing.

Now, you see this on the reader side. There has been a few recent fixes
that could cause problems when we have multiple readers. Ftrace usage
does not usually encounter multiple readers so I have not had issues.
But Li Zefan had a stress test that did find and trigger the problems.

Are you using the latest ring buffer that is in Linus's tree? Are you
resetting the ring buffer while reading it?

I guess I need to know more exactly what you are doing to understand the
problem.

Thanks,

-- Steve



  reply	other threads:[~2010-03-31  0:35 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-03-30 23:27 lockup in rb_get_reader_page Jiaying Zhang
2010-03-31  0:35 ` Steven Rostedt [this message]
2010-03-31  4:01   ` Jiaying Zhang
2010-04-01  0:28     ` Steven Rostedt
2010-04-01  1:14       ` Jiaying Zhang
2010-04-01  1:23         ` Steven Rostedt
2010-04-02 23:10           ` Jiaying Zhang
2010-04-03 12:45             ` Steven Rostedt
2010-04-06 21:12               ` Jiaying Zhang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1269995724.22564.58.camel@localhost.localdomain \
    --to=srostedt@redhat.com \
    --cc=dhsharp@google.com \
    --cc=jiayingz@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=mrubin@google.com \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox