From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755222Ab3KKSgR (ORCPT ); Mon, 11 Nov 2013 13:36:17 -0500 Received: from e38.co.us.ibm.com ([32.97.110.159]:55307 "EHLO e38.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753821Ab3KKSgI (ORCPT ); Mon, 11 Nov 2013 13:36:08 -0500 Date: Mon, 11 Nov 2013 10:36:02 -0800 From: "Paul E. McKenney" To: laijs@cn.fujitsu.com Cc: rostedt@goodmis.org, linux-kernel@vger.kernel.org Subject: WARN_ON_ONCE(in_irq() || in_serving_softirq() Message-ID: <20131111183602.GA16592@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13111118-1344-0000-0000-0000032B56DC Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, Lai, I am hitting the new warning in rcu_read_unlock_special() that checks for (in_irq() || in_serving_softirq()). Please see below for the splat. I actually managed to get two CPUs hitting this simultaneously, so got two splats. My first thought is to revert the WARN_ON_ONCE(), going back to: if (in_irq() || in_serving_softirq()) { local_irq_restore(flags); return; } >>From what I can see, the scheduling-clock tick is setting RCU_READ_UNLOCK_NEED_QS, which is causing the softirq handler's RCU read-side critical section to enter rcu_read_unlock_special(). Another fix would be to check for t->rcu_read_unlock_special == 0 in the previous "if (special & RCU_READ_UNLOCK_NEED_QS) {" check. Other thoughts? Thanx, Paul ------------------------------------------------------------------------ [ 192.542052] ------------[ cut here ]------------ [ 192.542054] ------------[ cut here ]------------ [ 192.542072] WARNING: CPU: 1 PID: 674 at /home/paulmck/public_git/linux-rcu/kernel/rcu/tree_plugin.h:367 rcu_read_unlock_special+0x260/0x270() [ 192.542074] Modules linked in: [ 192.542080] CPU: 1 PID: 674 Comm: rcu_torture_rea Not tainted 3.12.0-rc1+ #1 [ 192.542081] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [ 192.542085] 000000000000016f ffff88001fc43cc8 ffffffff817eaf38 0000000000000102 [ 192.542087] 0000000000000000 ffff88001fc43d08 ffffffff81045907 0000000000000001 [ 192.542089] 0000000000000002 0000000000004bcf ffffffff83ccbd40 ffff88001e1cc0c0 [ 192.542090] Call Trace: [ 192.542100] [] dump_stack+0x4f/0x84 [ 192.542106] [] warn_slowpath_common+0x87/0xb0 [ 192.542109] [] warn_slowpath_null+0x15/0x20 [ 192.542111] [] rcu_read_unlock_special+0x260/0x270 [ 192.542114] [] __rcu_read_unlock+0x5e/0x60 [ 192.542117] [] rcu_torture_read_unlock+0x21/0x30 [ 192.542133] [] rcu_torture_timer+0x135/0x150 [ 192.542137] [] ? rcu_torture_reader+0x310/0x310 [ 192.542144] [] call_timer_fn+0x7a/0x200 [ 192.542146] [] ? del_timer+0x70/0x70 [ 192.542148] [] run_timer_softirq+0x215/0x2f0 [ 192.542151] [] ? ktime_get+0x4f/0xe0 [ 192.542153] [] ? rcu_torture_reader+0x310/0x310 [ 192.542156] [] __do_softirq+0xd9/0x2d0 [ 192.542158] [] irq_exit+0x7e/0xa0 [ 192.542163] [] smp_apic_timer_interrupt+0x45/0x60 [ 192.542170] [] apic_timer_interrupt+0x6a/0x70 [ 192.542177] [] ? finish_task_switch+0x46/0xf0 [ 192.542179] [] ? finish_task_switch+0x46/0xf0 [ 192.542186] [] ? _raw_spin_unlock_irq+0x2c/0x60 [ 192.542188] [] ? _raw_spin_unlock_irq+0x26/0x60 [ 192.542190] [] finish_task_switch+0x83/0xf0 [ 192.542192] [] ? finish_task_switch+0x46/0xf0 [ 192.542194] [] __schedule+0x3ba/0x860 [ 192.542197] [] schedule+0x24/0x70 [ 192.542199] [] rcu_torture_reader+0xe0/0x310 [ 192.542201] [] ? rcu_torture_reader+0x310/0x310 [ 192.542204] [] ? rcutorture_trace_dump+0x30/0x30 [ 192.542209] [] kthread+0xd6/0xe0 [ 192.542211] [] ? finish_task_switch+0x46/0xf0 [ 192.542215] [] ? flush_kthread_work+0x190/0x190 [ 192.542217] [] ret_from_fork+0x7c/0xb0 [ 192.542220] [] ? flush_kthread_work+0x190/0x190 [ 192.542222] ---[ end trace 8519fcb7dea5ceee ]--- [ 192.543027] WARNING: CPU: 2 PID: 670 at /home/paulmck/public_git/linux-rcu/kernel/rcu/tree_plugin.h:367 rcu_read_unlock_special+0x260/0x270() [ 192.543027] Modules linked in: [ 192.543027] CPU: 2 PID: 670 Comm: rcu_torture_rea Tainted: G W 3.12.0-rc1+ #1 [ 192.543027] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007 [ 192.543027] 000000000000016f ffff88001fc83cc8 ffffffff817eaf38 0000000000000102 [ 192.543027] 0000000000000000 ffff88001fc83d08 ffffffff81045907 ffff88001e36ffd8 [ 192.543027] 0000000000000002 0000000000004bcf ffffffff83ccbd40 ffff88001e1c8000 [ 192.543027] Call Trace: [ 192.543027] [] dump_stack+0x4f/0x84 [ 192.543027] [] warn_slowpath_common+0x87/0xb0 [ 192.543027] [] warn_slowpath_null+0x15/0x20 [ 192.543027] [] rcu_read_unlock_special+0x260/0x270 [ 192.543027] [] __rcu_read_unlock+0x5e/0x60 [ 192.543027] [] rcu_torture_read_unlock+0x21/0x30 [ 192.543027] [] rcu_torture_timer+0x135/0x150 [ 192.543027] [] ? rcu_torture_reader+0x310/0x310 [ 192.543027] [] call_timer_fn+0x7a/0x200 [ 192.543027] [] ? del_timer+0x70/0x70 [ 192.543027] [] run_timer_softirq+0x215/0x2f0 [ 192.543027] [] ? ktime_get+0x4f/0xe0 [ 192.543027] [] ? rcu_torture_reader+0x310/0x310 [ 192.543027] [] __do_softirq+0xd9/0x2d0 [ 192.543027] [] irq_exit+0x7e/0xa0 [ 192.543027] [] smp_apic_timer_interrupt+0x45/0x60 [ 192.543027] [] apic_timer_interrupt+0x6a/0x70 [ 192.543027] [] ? _raw_spin_unlock_irq+0x2c/0x60 [ 192.543027] [] ? _raw_spin_unlock_irq+0x26/0x60 [ 192.543027] [] __schedule+0x70d/0x860 [ 192.543027] [] schedule+0x24/0x70 [ 192.543027] [] rcu_torture_reader+0xe0/0x310 [ 192.543027] [] ? finish_task_switch+0x46/0xf0 [ 192.543027] [] ? rcu_torture_reader+0x310/0x310 [ 192.543027] [] ? rcutorture_trace_dump+0x30/0x30 [ 192.543027] [] kthread+0xd6/0xe0 [ 192.543027] [] ? finish_task_switch+0x46/0xf0 [ 192.543027] [] ? flush_kthread_work+0x190/0x190 [ 192.543027] [] ret_from_fork+0x7c/0xb0 [ 192.543027] [] ? flush_kthread_work+0x190/0x190 [ 192.543027] ---[ end trace 8519fcb7dea5ceef ]---