From: Sasha Levin <levinsasha928@gmail.com>
To: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: linux-kernel <linux-kernel@vger.kernel.org>
Subject: INFO: task rcuc/0:7 blocked for more than 120 seconds.
Date: Mon, 26 Dec 2011 14:16:43 +0200 [thread overview]
Message-ID: <1324901803.31721.4.camel@lappy> (raw)
Hi Paul,
I've recently got the following panic which was caused by khungtask:
[ 1921.589512] INFO: task rcuc/0:7 blocked for more than 120 seconds.
[ 1921.590370] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1921.597103] rcuc/0 D ffff880012f61630 4400 7 2 0x00000000
[ 1921.598646] ffff880012f6b980 0000000000000086 ffff880012f6bfd8 00000000001d4740
[ 1921.600289] ffff880012f6bfd8 ffff880012f61630 ffff880012f6bfd8 ffff880012f6a000
[ 1921.601707] 00000000001d4800 ffff880012f6a000 ffff880012f6bfd8 00000000001d4800
[ 1921.603258] Call Trace:
[ 1921.603703] [<ffffffff8255eefa>] schedule+0x3a/0x50
[ 1921.605462] [<ffffffff8255cd65>] schedule_timeout+0x255/0x4d0
[ 1921.606540] [<ffffffff8112a25e>] ? mark_held_locks+0x6e/0x130
[ 1921.607633] [<ffffffff811277b2>] ? lock_release_holdtime+0xb2/0x160
[ 1921.608798] [<ffffffff825602bb>] ? _raw_spin_unlock_irq+0x2b/0x70
[ 1921.610154] [<ffffffff8255f630>] wait_for_common+0x120/0x170
[ 1921.617878] [<ffffffff81104f30>] ? try_to_wake_up+0x2f0/0x2f0
[ 1921.618949] [<ffffffff811754d0>] ? __call_rcu+0x3c0/0x3c0
[ 1921.621405] [<ffffffff8255f728>] wait_for_completion+0x18/0x20
[ 1921.623622] [<ffffffff810ee0b9>] wait_rcu_gp+0x59/0x80
[ 1921.626789] [<ffffffff810ec0c0>] ? perf_trace_rcu_batch_end+0x120/0x120
[ 1921.629440] [<ffffffff8255f554>] ? wait_for_common+0x44/0x170
[ 1921.632445] [<ffffffff81179d3c>] synchronize_rcu+0x1c/0x20
[ 1921.635455] [<ffffffff810f8980>] atomic_notifier_chain_unregister+0x60/0x80
[ 1921.638550] [<ffffffff8111bab3>] task_handoff_unregister+0x13/0x20
[ 1921.641271] [<ffffffff8211342f>] task_notify_func+0x2f/0x40
[ 1921.643894] [<ffffffff810f8817>] notifier_call_chain+0x67/0x110
[ 1921.646580] [<ffffffff810f8a14>] __atomic_notifier_call_chain+0x74/0x110
[ 1921.654064] [<ffffffff810f89d9>] ? __atomic_notifier_call_chain+0x39/0x110
[ 1921.656034] [<ffffffff810c1a30>] ? __put_task_struct+0xc0/0x120
[ 1921.658625] [<ffffffff810c71a0>] ? will_become_orphaned_pgrp+0x100/0x100
[ 1921.661527] [<ffffffff810f8ac1>] atomic_notifier_call_chain+0x11/0x20
[ 1921.664295] [<ffffffff8111ba55>] profile_handoff_task+0x15/0x20
[ 1921.666885] [<ffffffff810c19eb>] __put_task_struct+0x7b/0x120
[ 1921.669492] [<ffffffff810c71d5>] delayed_put_task_struct+0x35/0x140
[ 1921.672129] [<ffffffff81175994>] rcu_do_batch.clone.12+0x224/0xab0
[ 1921.674697] [<ffffffff8255ea62>] ? __schedule+0x432/0x890
[ 1921.677028] [<ffffffff811783fa>] rcu_cpu_kthread+0x48a/0xa40
[ 1921.682719] [<ffffffff82560245>] ? _raw_spin_unlock_irqrestore+0x55/0xa0
[ 1921.684913] [<ffffffff81177f70>] ? rcu_idle_enter+0xb0/0xb0
[ 1921.686641] [<ffffffff810f11f6>] kthread+0xb6/0xc0
[ 1921.689956] [<ffffffff82562cf4>] kernel_thread_helper+0x4/0x10
[ 1921.692815] [<ffffffff810fe170>] ? finish_task_switch+0x80/0x110
[ 1921.695613] [<ffffffff82561038>] ? retint_restore_args+0x13/0x13
[ 1921.698409] [<ffffffff810f1140>] ? kthread_flush_work_fn+0x10/0x10
[ 1921.701199] [<ffffffff82562cf0>] ? gs_change+0x13/0x13
[ 1921.703680] 1 lock held by rcuc/0/7:
[ 1921.705325] #0: (rcu_read_lock){.+.+..}, at: [<ffffffff810f89d9>] __atomic_notifier_call_chain+0x39/0x110
[ 1921.710297] Kernel panic - not syncing: hung_task: blocked tasks
[ 1921.715383] Rebooting in 1 seconds..
This is the RCU related .config:
# cat .config | grep RCU
# RCU Subsystem
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
CONFIG_RCU_TRACE=y
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
CONFIG_RCU_FAST_NO_HZ=y
CONFIG_TREE_RCU_TRACE=y
CONFIG_RCU_BOOST=y
CONFIG_RCU_BOOST_PRIO=1
CONFIG_RCU_BOOST_DELAY=500
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
CONFIG_PROVE_RCU=y
CONFIG_PROVE_RCU_REPEATEDLY=y
CONFIG_SPARSE_RCU_POINTER=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_TIMEOUT=60
CONFIG_RCU_CPU_STALL_VERBOSE=y
Please let me know if I can help with anything else.
--
Sasha.
next reply other threads:[~2011-12-26 12:16 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-12-26 12:16 Sasha Levin [this message]
2011-12-26 16:31 ` INFO: task rcuc/0:7 blocked for more than 120 seconds Paul E. McKenney
2011-12-26 16:37 ` Frederic Weisbecker
2011-12-26 19:56 ` Paul E. McKenney
2012-01-04 19:03 ` [PATCH] rcu: Improve detection of illegal synchronize_rcu() call from RCU read side Frederic Weisbecker
2012-01-04 21:30 ` Paul E. McKenney
2012-01-05 1:45 ` Frederic Weisbecker
2012-01-05 2:01 ` Paul E. McKenney
2012-01-05 2:06 ` Frederic Weisbecker
2012-01-05 2:17 ` Paul E. McKenney
2011-12-27 9:13 ` INFO: task rcuc/0:7 blocked for more than 120 seconds Sasha Levin
2011-12-28 4:29 ` Paul E. McKenney
2012-01-03 20:27 ` Paul E. McKenney
2012-01-03 20:37 ` Greg KH
2012-01-03 21:38 ` Paul E. McKenney
2012-01-03 21:50 ` Greg KH
2012-01-03 22:26 ` Paul E. McKenney
2012-01-03 22:33 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1324901803.31721.4.camel@lappy \
--to=levinsasha928@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.