From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754913Ab1H2Uy3 (ORCPT ); Mon, 29 Aug 2011 16:54:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:61510 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754684Ab1H2Uy2 (ORCPT ); Mon, 29 Aug 2011 16:54:28 -0400 Date: Mon, 29 Aug 2011 16:54:24 -0400 From: Dave Jones To: Linux Kernel , tytso@mit.edu, adilger.kernel@dilger.ca Subject: Re: ext4 lockdep trace (3.1.0rc3) Message-ID: <20110829205424.GA18208@redhat.com> Mail-Followup-To: Dave Jones , Linux Kernel , tytso@mit.edu, adilger.kernel@dilger.ca References: <20110826214930.GA21818@redhat.com> <20110829204830.GA18543@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110829204830.GA18543@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 29, 2011 at 04:48:30PM -0400, Dave Jones wrote: > On Fri, Aug 26, 2011 at 05:49:30PM -0400, Dave Jones wrote: > > just hit this while building a kernel. Laptop wedged for a few seconds > > during the final link, and this was in the log when it unwedged. > > I still see this in rc4, and can reproduce it reliably every time I build. > It only started happening in the last week. I don't see any ext4 or vfs commits > within a few days of that, so I'm not sure why it only just begun > (I do daily builds, and the 26th was the first time I saw it appear) > > Given the lack of obvious commits in that timeframe, I'm not sure a bisect is > going to be particularly fruitful. It might just be that my IO patterns changed ? also a second variant with a different trace. Dave ======================================================= [ INFO: possible circular locking dependency detected ] 3.1.0-rc3+ #151 ------------------------------------------------------- gnome-settings-/2037 is trying to acquire lock: (&sb->s_type->i_mutex_key#14){+.+.+.}, at: [] ext4_evict_inode+0x76/0x33c but task is already holding lock: (&mm->mmap_sem){++++++}, at: [] sys_munmap+0x3b/0x60 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&mm->mmap_sem){++++++}: [] lock_acquire+0xf3/0x13e [] might_fault+0x80/0xa3 [] filldir+0x6f/0xc7 [] call_filldir+0x96/0xc0 [] ext4_readdir+0x1bd/0x548 [] vfs_readdir+0x7b/0xb4 [] sys_getdents+0x7e/0xd1 [] system_call_fastpath+0x16/0x1b -> #0 (&sb->s_type->i_mutex_key#14){+.+.+.}: [] __lock_acquire+0xa2f/0xd0c [] lock_acquire+0xf3/0x13e [] __mutex_lock_common+0x65/0x44a [] mutex_lock_nested+0x3b/0x40 [] ext4_evict_inode+0x76/0x33c [] evict+0x98/0x152 [] iput+0x191/0x199 [] dentry_kill+0x123/0x145 [] dput+0xf2/0x102 [] fput+0x1d8/0x1f0 [] remove_vma+0x51/0x82 [] do_munmap+0x2f2/0x30b [] sys_munmap+0x49/0x60 [] system_call_fastpath+0x16/0x1b other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&mm->mmap_sem); lock(&sb->s_type->i_mutex_key); lock(&mm->mmap_sem); lock(&sb->s_type->i_mutex_key); *** DEADLOCK *** 1 lock held by gnome-settings-/2037: #0: (&mm->mmap_sem){++++++}, at: [] sys_munmap+0x3b/0x60 stack backtrace: Pid: 2037, comm: gnome-settings- Not tainted 3.1.0-rc3+ #151 Call Trace: [] ? up+0x39/0x3e [] print_circular_bug+0x1f8/0x209 [] __lock_acquire+0xa2f/0xd0c [] ? local_clock+0x35/0x4c [] ? mark_lock+0x2d/0x220 [] ? ext4_evict_inode+0x76/0x33c [] lock_acquire+0xf3/0x13e [] ? ext4_evict_inode+0x76/0x33c [] ? __mutex_lock_common+0x3d/0x44a [] ? mutex_lock_nested+0x3b/0x40 [] ? ext4_evict_inode+0x76/0x33c [] __mutex_lock_common+0x65/0x44a [] ? ext4_evict_inode+0x76/0x33c [] ? local_clock+0x35/0x4c [] ? evict+0x8a/0x152 [] ? put_lock_stats+0xe/0x29 [] ? lock_release_holdtime.part.10+0x59/0x62 [] ? evict+0x8a/0x152 [] mutex_lock_nested+0x3b/0x40 [] ext4_evict_inode+0x76/0x33c [] evict+0x98/0x152 [] iput+0x191/0x199 [] dentry_kill+0x123/0x145 [] dput+0xf2/0x102 [] fput+0x1d8/0x1f0 [] remove_vma+0x51/0x82 [] do_munmap+0x2f2/0x30b [] sys_munmap+0x49/0x60 [] system_call_fastpath+0x16/0x1b