From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Gleixner Subject: Re: 3.4.4-rt13: btrfs + xfstests 006 = BOOM.. and a bonus rt_mutex deadlock report for absolutely free! Date: Thu, 12 Jul 2012 15:31:06 +0200 (CEST) Message-ID: References: <1342072060.7338.102.camel@marge.simpson.net> <1342082648.7338.171.camel@marge.simpson.net> <1342086792.7707.9.camel@marge.simpson.net> <1342094233.7707.12.camel@marge.simpson.net> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: "linux-rt-users@vger.kernel.org" , LKML , linux-fsdevel , Steven Rostedt , Peter Zijlstra To: Mike Galbraith Return-path: In-Reply-To: <1342094233.7707.12.camel@marge.simpson.net> Sender: linux-rt-users-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Thu, 12 Jul 2012, Mike Galbraith wrote: > On Thu, 2012-07-12 at 13:43 +0200, Thomas Gleixner wrote: > > rawlock points to ...968 and the node_list to ...970. > > > > struct rt_mutex { > > raw_spinlock_t wait_lock; > > struct plist_head wait_list; > > > > The raw_lock pointer of the plist_head is initialized in > > __rt_mutex_init() so it points to wait_lock. > > > > Can you check the offset of wait_list vs. the rt_mutex itself? > > > > I wouldn't be surprised if it's exactly 8 bytes. And then this thing > > looks like a copied lock with stale pointers to hell. Eew. > > crash> struct rt_mutex -o > struct rt_mutex { > [0] raw_spinlock_t wait_lock; > [8] struct plist_head wait_list; Bingo, that makes it more likely that this is caused by copying w/o initializing the lock and then freeing the original structure. A quick check for memcpy finds that __btrfs_close_devices() does a memcpy of btrfs_device structs w/o initializing the lock in the new copy, but I have no idea whether that's the place we are looking for. Thanks, tglx diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 43baaf0..06c8ced 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -512,6 +512,7 @@ static int __btrfs_close_devices(struct btrfs_fs_devices *fs_devices) new_device->writeable = 0; new_device->in_fs_metadata = 0; new_device->can_discard = 0; + spin_lock_init(&new_device->io_lock); list_replace_rcu(&device->dev_list, &new_device->dev_list); call_rcu(&device->rcu, free_device);