From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756535Ab1GGTJC (ORCPT ); Thu, 7 Jul 2011 15:09:02 -0400 Received: from mail.candelatech.com ([208.74.158.172]:57678 "EHLO ns3.lanforge.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756499Ab1GGTI7 (ORCPT ); Thu, 7 Jul 2011 15:08:59 -0400 Message-ID: <4E16044B.9060301@candelatech.com> Date: Thu, 07 Jul 2011 12:08:59 -0700 From: Ben Greear Organization: Candela Technologies User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100430 Fedora/3.0.4-2.fc11 Thunderbird/3.0.4 MIME-Version: 1.0 To: Linux Kernel Mailing List Subject: Lockdep splat in 3.0-rc6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I've been seeing this for a while in the 3.0 series..and it's still in -rc6. This kernel is larded up with debug options, and is running some nfs/rpc patches not in the standard tree. It is heavily loaded doing nfs mounts, traffic, unmounts, etc. As far as I can tell, this has nothing in particular to do with NFS. ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 3.0.0-rc6+ #13 ------------------------------------------------------ ps/25695 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (&lock->wait_lock){+.+...}, at: [] rt_mutex_slowunlock+0x1d/0xdd and this task is already holding: (&(&sighand->siglock)->rlock){-.....}, at: [] __lock_task_sighand+0x6e/0x9c which would create a new lock dependency: (&(&sighand->siglock)->rlock){-.....} -> (&lock->wait_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&(&sighand->siglock)->rlock){-.....} ... which became HARDIRQ-irq-safe at: [] __lock_acquire+0x2b4/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock_irqsave+0x4e/0x60 [] __lock_task_sighand+0x6e/0x9c [] do_send_sig_info+0x27/0x70 [] group_send_sig_info+0x4c/0x57 [] kill_pid_info+0x3f/0x5a [] it_real_fn+0x85/0xb4 [] __run_hrtimer+0xbe/0x1be [] hrtimer_interrupt+0xe5/0x1c0 [] smp_apic_timer_interrupt+0x80/0x93 [] apic_timer_interrupt+0x13/0x20 [] cpu_idle+0x5e/0xb5 [] start_secondary+0x1a3/0x1a8 to a HARDIRQ-irq-unsafe lock: (&lock->wait_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [] __lock_acquire+0x328/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowlock+0x2b/0x132 [] rt_mutex_lock+0x46/0x4a [] rcu_boost_kthread+0x125/0x169 [] kthread+0x7d/0x85 [] kernel_thread_helper+0x4/0x10 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&lock->wait_lock); local_irq_disable(); lock(&(&sighand->siglock)->rlock); lock(&lock->wait_lock); lock(&(&sighand->siglock)->rlock); *** DEADLOCK *** 2 locks held by ps/25695: #0: (&p->lock){+.+.+.}, at: [] seq_read+0x38/0x368 #1: (&(&sighand->siglock)->rlock){-.....}, at: [] __lock_task_sighand+0x6e/0x9c the dependencies between HARDIRQ-irq-safe lock and the holding lock: -> (&(&sighand->siglock)->rlock){-.....} ops: 30710408 { IN-HARDIRQ-W at: [] __lock_acquire+0x2b4/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock_irqsave+0x4e/0x60 [] __lock_task_sighand+0x6e/0x9c [] do_send_sig_info+0x27/0x70 [] group_send_sig_info+0x4c/0x57 [] kill_pid_info+0x3f/0x5a [] it_real_fn+0x85/0xb4 [] __run_hrtimer+0xbe/0x1be [] hrtimer_interrupt+0xe5/0x1c0 [] smp_apic_timer_interrupt+0x80/0x93 [] apic_timer_interrupt+0x13/0x20 [] cpu_idle+0x5e/0xb5 [] start_secondary+0x1a3/0x1a8 INITIAL USE at: [] __lock_acquire+0x39f/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock_irqsave+0x4e/0x60 [] flush_signals+0x1d/0x43 [] ignore_signals+0x28/0x2a [] kthreadd+0x3e/0x13d [] kernel_thread_helper+0x4/0x10 } ... key at: [] __key.56506+0x0/0x8 ... acquired at: [] check_irq_usage+0x5d/0xbe [] __lock_acquire+0xb09/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowunlock+0x1d/0xdd [] rt_mutex_unlock+0x22/0x24 [] __rcu_read_unlock+0x1c0/0x24e [] rcu_read_unlock+0x21/0x23 [] __lock_task_sighand+0x8d/0x9c [] do_task_stat+0x11d/0x84a [] proc_tgid_stat+0xf/0x11 [] proc_single_show+0x54/0x71 [] seq_read+0x181/0x368 [] vfs_read+0xa6/0x102 [] sys_read+0x45/0x6c [] system_call_fastpath+0x16/0x1b the dependencies between the lock to be acquired and HARDIRQ-irq-unsafe lock: -> (&lock->wait_lock){+.+...} ops: 725 { HARDIRQ-ON-W at: [] __lock_acquire+0x328/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowlock+0x2b/0x132 [] rt_mutex_lock+0x46/0x4a [] rcu_boost_kthread+0x125/0x169 [] kthread+0x7d/0x85 [] kernel_thread_helper+0x4/0x10 SOFTIRQ-ON-W at: [] __lock_acquire+0x349/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowlock+0x2b/0x132 [] rt_mutex_lock+0x46/0x4a [] rcu_boost_kthread+0x125/0x169 [] kthread+0x7d/0x85 [] kernel_thread_helper+0x4/0x10 INITIAL USE at: [] __lock_acquire+0x39f/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowlock+0x2b/0x132 [] rt_mutex_lock+0x46/0x4a [] rcu_boost_kthread+0x125/0x169 [] kthread+0x7d/0x85 [] kernel_thread_helper+0x4/0x10 } ... key at: [] __key.22188+0x0/0x8 ... acquired at: [] check_irq_usage+0x5d/0xbe [] __lock_acquire+0xb09/0xdd5 [] lock_acquire+0xf4/0x14b [] _raw_spin_lock+0x36/0x45 [] rt_mutex_slowunlock+0x1d/0xdd [] rt_mutex_unlock+0x22/0x24 [] __rcu_read_unlock+0x1c0/0x24e [] rcu_read_unlock+0x21/0x23 [] __lock_task_sighand+0x8d/0x9c [] do_task_stat+0x11d/0x84a [] proc_tgid_stat+0xf/0x11 [] proc_single_show+0x54/0x71 [] seq_read+0x181/0x368 [] vfs_read+0xa6/0x102 [] sys_read+0x45/0x6c [] system_call_fastpath+0x16/0x1b stack backtrace: Pid: 25695, comm: ps Not tainted 3.0.0-rc6+ #13 Call Trace: [] check_usage+0x364/0x378 [] check_irq_usage+0x5d/0xbe [] __lock_acquire+0xb09/0xdd5 [] ? rcu_start_gp+0x2e7/0x310 [] ? rt_mutex_slowunlock+0x1d/0xdd [] lock_acquire+0xf4/0x14b [] ? rt_mutex_slowunlock+0x1d/0xdd [] _raw_spin_lock+0x36/0x45 [] ? rt_mutex_slowunlock+0x1d/0xdd [] rt_mutex_slowunlock+0x1d/0xdd [] rt_mutex_unlock+0x22/0x24 [] __rcu_read_unlock+0x1c0/0x24e [] rcu_read_unlock+0x21/0x23 [] __lock_task_sighand+0x8d/0x9c [] do_task_stat+0x11d/0x84a [] ? register_lock_class+0x1e/0x336 [] ? mark_lock+0x2d/0x22d [] ? mark_lock+0x2d/0x22d [] ? __lock_acquire+0x39f/0xdd5 [] ? add_partial+0x1b/0x53 [] ? cpumask_weight+0xe/0xe [] proc_tgid_stat+0xf/0x11 [] proc_single_show+0x54/0x71 [] seq_read+0x181/0x368 [] vfs_read+0xa6/0x102 [] ? fget_light+0x35/0xac [] sys_read+0x45/0x6c [] system_call_fastpath+0x16/0x1b -- Ben Greear Candela Technologies Inc http://www.candelatech.com