From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Subject: Re: 3.3.0: "INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)"
Date: Fri, 6 Apr 2012 10:52:10 -0700 [thread overview]
Message-ID: <20120406175210.GA2421@linux.vnet.ibm.com> (raw)
In-Reply-To: <20120328112822.GH16671@charite.de>
On Wed, Mar 28, 2012 at 01:28:22PM +0200, Ralf Hildebrandt wrote:
> * Ralf Hildebrandt <Ralf.Hildebrandt@charite.de>:
> >
> > Mar 23 11:54:00 mail kernel:
> > Mar 23 11:54:00 mail kernel: [347213.025005] INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)
> > Mar 23 11:54:00 mail kernel: [347213.025005] Pid: 12654, comm: cleanup Not tainted 3.3.0 #1
> > Mar 23 11:54:00 mail kernel: [347213.025005] Call Trace:
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c1062ccb>] ? __rcu_pending+0x10a/0x311
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c106388d>] ? rcu_check_callbacks+0x9b/0xa1
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c102c434>] ? update_process_times+0x2a/0x53
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c104f206>] ? tick_sched_timer+0x4f/0x90
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c103a101>] ? __remove_hrtimer+0x25/0x79
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c103a2ff>] ? __run_hrtimer.isra.32+0x38/0xbe
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c103ad95>] ? hrtimer_interrupt+0xda/0x23d
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c1016493>] ? smp_apic_timer_interrupt+0x4c/0x81
> > Mar 23 11:54:00 mail kernel: [347213.025005] [<c127b5d9>] ? apic_timer_interrupt+0x31/0x38
>
> I read http://www.kernel.org/doc/Documentation/RCU/stallwarn.txt
> but couldn't find "rcu_bh detected stall" being mentioned there.
The documentation mentions "rcu_bh_state detected stalls". I will check
this and update if needed.
> I reported the same issue on 3.2.9 (same machine!) on the 13th of march:
>
> Mar 12 12:58:38 mail kernel: [440746.244002] INFO: rcu_bh detected stall on CPU 2 (t=0 jiffies)
> Mar 12 12:58:38 mail kernel: [440746.244002] Pid: 27980, comm: /usr/sbin/amavi Tainted: G W 3.2.9 #1
> Mar 12 12:58:38 mail kernel: [440746.244002] Call Trace:
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1061d26>] ? __rcu_pending+0x10a/0x30e
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c10626ca>] ? rcu_check_callbacks+0xd4/0xde
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1035cf2>] ? update_process_times+0x2a/0x53
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c104e1d2>] ? tick_sched_timer+0x4d/0x8e
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1043b53>] ? __remove_hrtimer+0x25/0x79
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1043d4f>] ? __run_hrtimer.isra.32+0x37/0xbd
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c10445d5>] ? hrtimer_interrupt+0xdb/0x23d
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c101b781>] ? vmalloc_sync_all+0x1/0x1
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1015e4c>] ? smp_apic_timer_interrupt+0x4c/0x81
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1275491>] ? apic_timer_interrupt+0x31/0x38
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c101b781>] ? vmalloc_sync_all+0x1/0x1
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c101b8c2>] ? do_page_fault+0x141/0x38f
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1030d8d>] ? irq_exit+0x34/0x87
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c1015e51>] ? smp_apic_timer_interrupt+0x51/0x81
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c101b781>] ? vmalloc_sync_all+0x1/0x1
> Mar 12 12:58:38 mail kernel: [440746.244002] [<c12756b7>] ? error_code+0x67/0x6c
This stack looks interesting -- might be a problem in vmalloc_sync(),
at least if the stack can be trusted. Could you please reproduce with
frame pointers enabled?
Thanx, Paul
> and so did Tilman Schmidt in February:
> https://lkml.org/lkml/2012/2/18/34
>
> --
> Ralf Hildebrandt Charite Universitätsmedizin Berlin
> ralf.hildebrandt@charite.de Campus Benjamin Franklin
> http://www.charite.de Hindenburgdamm 30, 12203 Berlin
> Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
next prev parent reply other threads:[~2012-04-07 22:14 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-03-27 14:15 3.3.0: "INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)" Ralf Hildebrandt
2012-03-28 11:28 ` Ralf Hildebrandt
2012-04-06 17:52 ` Paul E. McKenney [this message]
2012-04-08 16:21 ` Ralf Hildebrandt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120406175210.GA2421@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.