* btrfs: lock inversion between delayed_node->mutex and found->groups_sem
@ 2014-03-15 0:12 Sasha Levin
2014-03-17 13:05 ` David Sterba
0 siblings, 1 reply; 9+ messages in thread
From: Sasha Levin @ 2014-03-15 0:12 UTC (permalink / raw)
To: clm, jbacik; +Cc: linux-btrfs, LKML
Hi all,
While fuzzing with trinity inside a KVM tools guest running the latest -next
kernel I've stumbled on the following:
[ 788.451695] =========================================================
[ 788.452455] [ INFO: possible irq lock inversion dependency detected ]
[ 788.453020] 3.14.0-rc6-next-20140313-sasha-00010-gb8c1db1-dirty #217 Tainted: G W
[ 788.453827] ---------------------------------------------------------
[ 788.454371] kswapd3/4199 just changed the state of lock:
[ 788.454902] (&delayed_node->mutex){+.+.-.}, at: __btrfs_release_delayed_node+0x4f/0x140 (fs/btrfs/delayed-inode.c:263)
[ 788.455890] but this lock took another, RECLAIM_FS-unsafe lock in the past:
[ 788.456543] (&found->groups_sem){+++++.}
and interrupts could create inverse lock ordering between them.
[ 788.457491]
[ 788.457491] other info that might help us debug this:
[ 788.458115] Possible interrupt unsafe locking scenario:
[ 788.458115]
[ 788.458756] CPU0 CPU1
[ 788.459188] ---- ----
[ 788.459625] lock(&found->groups_sem);
[ 788.460041] local_irq_disable();
[ 788.460041] lock(&delayed_node->mutex);
[ 788.460041] lock(&found->groups_sem);
[ 788.460041] <Interrupt>
[ 788.460041] lock(&delayed_node->mutex);
[ 788.460041]
[ 788.460041] *** DEADLOCK ***
[ 788.460041]
[ 788.460041] 2 locks held by kswapd3/4199:
[ 788.460041] #0: (shrinker_rwsem){++++..}, at: shrink_slab+0x3f/0x160 (mm/vmscan.c:360)
[ 788.460041] #1: (&type->s_umount_key#108){.+.+..}, at: grab_super_passive+0x56/0x90 (fs/super.c:361)
[ 788.460041]
[ 788.460041] the shortest dependencies between 2nd lock and 1st lock:
[ 788.460041] -> (&found->groups_sem){+++++.} ops: 46 {
[ 788.460041] HARDIRQ-ON-W at:
[ 788.460041] mark_irqflags+0xf0/0x170 (kernel/locking/lockdep.c:2800)
[ 788.460041] __lock_acquire+0x2de/0x5a0 (kernel/locking/lockdep.c:3138)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_write+0x5c/0xc0 (arch/x86/include/asm/rwsem.h:130 kernel/locking/rwsem.c:50)
[ 788.460041] __link_block_group+0x45/0x110 (fs/btrfs/extent-tree.c:8348)
[ 788.460041] btrfs_read_block_groups+0x3ae/0x700 (fs/btrfs/extent-tree.c:8533)
[ 788.460041] open_ctree+0x1abf/0x2210 (fs/btrfs/disk-io.c:2749)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] HARDIRQ-ON-R at:
[ 788.460041] mark_irqflags+0xbc/0x170 (kernel/locking/lockdep.c:2792)
[ 788.460041] __lock_acquire+0x2de/0x5a0 (kernel/locking/lockdep.c:3138)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_read+0x4c/0xa0 (arch/x86/include/asm/rwsem.h:83 kernel/locking/rwsem.c:23)
[ 788.460041] btrfs_calc_num_tolerated_disk_barrier_failures+0x2a7/0x3a0 (fs/btrfs/disk-io.c:3309)
[ 788.460041] open_ctree+0x1af7/0x2210 (fs/btrfs/disk-io.c:2755)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)
[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] SOFTIRQ-ON-W at:
[ 788.460041] mark_irqflags+0x110/0x170 (kernel/locking/lockdep.c:2804)
[ 788.460041] __lock_acquire+0x2de/0x5a0 (kernel/locking/lockdep.c:3138)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_write+0x5c/0xc0 (arch/x86/include/asm/rwsem.h:130 kernel/locking/rwsem.c:50)
[ 788.460041] __link_block_group+0x45/0x110 (fs/btrfs/extent-tree.c:8348)
[ 788.460041] btrfs_read_block_groups+0x3ae/0x700 (fs/btrfs/extent-tree.c:8533)
[ 788.460041] open_ctree+0x1abf/0x2210 (fs/btrfs/disk-io.c:2749)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)
[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] SOFTIRQ-ON-R at:
[ 788.460041] mark_irqflags+0x110/0x170 (kernel/locking/lockdep.c:2804)
[ 788.460041] __lock_acquire+0x2de/0x5a0 (kernel/locking/lockdep.c:3138)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_read+0x4c/0xa0 (arch/x86/include/asm/rwsem.h:83 kernel/locking/rwsem.c:23)
[ 788.460041] btrfs_calc_num_tolerated_disk_barrier_failures+0x2a7/0x3a0 (fs/btrfs/disk-io.c:3309)
[ 788.460041] open_ctree+0x1af7/0x2210 (fs/btrfs/disk-io.c:2755)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)
[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] RECLAIM_FS-ON-W at:
[ 788.460041] mark_held_locks+0x6c/0x90 (kernel/locking/lockdep.c:2523)
[ 788.460041] lockdep_trace_alloc+0xfd/0x140 (kernel/locking/lockdep.c:2745 kernel/locking/lockdep.c:2760)
[ 788.460041] __kmalloc_track_caller+0x80/0x350 (mm/slub.c:965 mm/slub.c:2402 mm/slub.c:2475 mm/slub.c:3851)
[ 788.460041] kvasprintf+0x5b/0x90 (lib/kasprintf.c:24)
[ 788.460041] kobject_set_name_vargs+0x23/0x70 (lib/kobject.c:266)
[ 788.460041] kobject_add_varg+0x25/0x60 (lib/kobject.c:348)
[ 788.460041] kobject_add+0x70/0x80 (lib/kobject.c:403)
[ 788.460041] __link_block_group+0xad/0x110 (fs/btrfs/extent-tree.c:8355)
[ 788.460041] btrfs_read_block_groups+0x3ae/0x700 (fs/btrfs/extent-tree.c:8533)
[ 788.460041] open_ctree+0x1abf/0x2210 (fs/btrfs/disk-io.c:2749)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)
[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] INITIAL USE at:
[ 788.460041] __lock_acquire+0x301/0x5a0 (kernel/locking/lockdep.c:3142)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_write+0x5c/0xc0 (arch/x86/include/asm/rwsem.h:130 kernel/locking/rwsem.c:50)
[ 788.460041] __link_block_group+0x45/0x110 (fs/btrfs/extent-tree.c:8348)
[ 788.460041] btrfs_read_block_groups+0x3ae/0x700 (fs/btrfs/extent-tree.c:8533)
[ 788.460041] open_ctree+0x1abf/0x2210 (fs/btrfs/disk-io.c:2749)
[ 788.460041] btrfs_fill_super+0x81/0x140 (fs/btrfs/super.c:958)
[ 788.460041] btrfs_mount+0x26a/0x300 (fs/btrfs/super.c:1295)
[ 788.460041] mount_fs+0x8d/0x1a0 (fs/super.c:1091)
[ 788.460041] vfs_kern_mount+0x79/0x150 (fs/namespace.c:813)
[ 788.460041] do_new_mount+0xcd/0x1c0 (fs/namespace.c:2068)
[ 788.460041] do_mount+0x15d/0x210 (fs/namespace.c:2392)
[ 788.460041] SyS_mount+0x9d/0xe0 (fs/namespace.c:2589 fs/namespace.c:2560)
[ 788.460041] tracesys+0xdd/0xe2 (arch/x86/kernel/entry_64.S:749)
[ 788.460041] }
[ 788.460041] ... key at: __key.59054+0x0/0x8 (??:0)
[ 788.460041] ... acquired at:
[ 788.460041] validate_chain+0x6c5/0x7b0 (kernel/locking/lockdep.c:1945 kernel/locking/lockdep.c:2131)
[ 788.460041] __lock_acquire+0x4cd/0x5a0 (kernel/locking/lockdep.c:3182)
[ 788.460041] lock_acquire+0x182/0x1d0 (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 788.460041] down_read+0x4c/0xa0 (arch/x86/include/asm/rwsem.h:83 kernel/locking/rwsem.c:23)
[ 788.460041] find_free_extent+0x391/0xbf0 (fs/btrfs/extent-tree.c:6235)
[ 788.460041] btrfs_reserve_extent+0x7a/0x130 (fs/btrfs/extent-tree.c:6609)
[ 788.460041] btrfs_alloc_free_block+0x94/0x260 (fs/btrfs/extent-tree.c:7003)
[ 788.460041] __btrfs_cow_block+0x14a/0x4c0 (fs/btrfs/ctree.c:1168)
[ 788.460041] btrfs_cow_block+0x159/0x2b0 (fs/btrfs/ctree.c:1600)
[ 788.460041] btrfs_search_slot+0x33c/0x740 (fs/btrfs/ctree.c:2837)
[ 788.460041] btrfs_lookup_inode+0x2f/0xa0 (fs/btrfs/inode-item.c:423)
[ 788.460041] __btrfs_update_delayed_inode+0x61/0x240 (fs/btrfs/delayed-inode.c:1051)
[ 788.460041] __btrfs_run_delayed_items+0x123/0x1f0 (fs/btrfs/delayed-inode.c:1126 fs/btrfs/delayed-inode.c:1145 fs/btrfs/delayed-inode.c:1180)
[ 788.460041] btrfs_run_delayed_items+0x13/0x20 (fs/btrfs/delayed-inode.c:1206)
[ 788.460041] btrfs_flush_all_pending_stuffs+0x24/0x80 (fs/btrfs/transaction.c:1594)
[ 788.460041] btrfs_commit_transaction+0x25b/0x9f0 (fs/btrfs/transaction.c:1728)
[ 788.460041] transaction_kthread+0x133/0x250 (fs/btrfs/disk-io.c:1768)
Thanks,
Sasha
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-03-15 0:12 btrfs: lock inversion between delayed_node->mutex and found->groups_sem Sasha Levin
@ 2014-03-17 13:05 ` David Sterba
2014-03-26 17:01 ` Jeff Mahoney
0 siblings, 1 reply; 9+ messages in thread
From: David Sterba @ 2014-03-17 13:05 UTC (permalink / raw)
To: Sasha Levin; +Cc: clm, jbacik, linux-btrfs, LKML
On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
> While fuzzing with trinity inside a KVM tools guest running the latest -next
> kernel I've stumbled on the following:
>
> [ 788.458756] CPU0 CPU1
> [ 788.459188] ---- ----
> [ 788.459625] lock(&found->groups_sem);
> [ 788.460041] local_irq_disable();
> [ 788.460041] lock(&delayed_node->mutex);
> [ 788.460041] lock(&found->groups_sem);
> [ 788.460041] <Interrupt>
> [ 788.460041] lock(&delayed_node->mutex);
> [ 788.460041]
> [ 788.460041] *** DEADLOCK ***
> [ 788.460041]
> [ 788.460041] 2 locks held by kswapd3/4199:
I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
2 locks held by 224/31203:
#0: (shrinker_rwsem){++++..}, at: [<ffffffff8113be6d>] shrink_slab+0x3d/0x110
#1: (&type->s_umount_key#32){++++++}, at: [<ffffffff8117cd84>] grab_super_passive+0x44/0x90
the shortest dependencies between 2nd lock and 1st lock:
-> (&found->groups_sem){+++++.} ops: 405561 {
HARDIRQ-ON-W at:
[<ffffffff810af476>] __lock_acquire+0x7f6/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01d3c>] down_write+0x5c/0xc0
[<ffffffffa001f6a6>] __link_block_group+0x46/0x130 [btrfs]
[<ffffffffa0023411>] btrfs_read_block_groups+0x341/0x690 [btrfs]
[<ffffffffa0031c50>] open_ctree+0x1880/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
HARDIRQ-ON-R at:
[<ffffffff810af265>] __lock_acquire+0x5e5/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01c8c>] down_read+0x4c/0xa0
[<ffffffffa002da32>] btrfs_calc_num_tolerated_disk_barrier_failures+0x142/0x240 [btrfs]
[<ffffffffa0031c6e>] open_ctree+0x189e/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
SOFTIRQ-ON-W at:
[<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01d3c>] down_write+0x5c/0xc0
[<ffffffffa001f6a6>] __link_block_group+0x46/0x130 [btrfs]
[<ffffffffa0023411>] btrfs_read_block_groups+0x341/0x690 [btrfs]
[<ffffffffa0031c50>] open_ctree+0x1880/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
SOFTIRQ-ON-R at:
[<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01c8c>] down_read+0x4c/0xa0
[<ffffffffa002da32>] btrfs_calc_num_tolerated_disk_barrier_failures+0x142/0x240 [btrfs]
[<ffffffffa0031c6e>] open_ctree+0x189e/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
RECLAIM_FS-ON-W at:
[<ffffffff810b1c6c>] mark_held_locks+0x8c/0x170
[<ffffffff810b249a>] lockdep_trace_alloc+0x8a/0xd0
[<ffffffff8116f807>] __kmalloc_track_caller+0x47/0x210
[<ffffffff813cdefb>] kvasprintf+0x5b/0x90
[<ffffffff813c166a>] kobject_set_name_vargs+0x2a/0x70
[<ffffffff813c1ffa>] kobject_add+0x5a/0xb0
[<ffffffffa001f75d>] __link_block_group+0xfd/0x130 [btrfs]
[<ffffffffa0023411>] btrfs_read_block_groups+0x341/0x690 [btrfs]
[<ffffffffa0031c50>] open_ctree+0x1880/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
INITIAL USE at:
[<ffffffff810aefd4>] __lock_acquire+0x354/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01d3c>] down_write+0x5c/0xc0
[<ffffffffa001f6a6>] __link_block_group+0x46/0x130 [btrfs]
[<ffffffffa0023411>] btrfs_read_block_groups+0x341/0x690 [btrfs]
[<ffffffffa0031c50>] open_ctree+0x1880/0x2310 [btrfs]
[<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
[<ffffffff8117d6f0>] mount_fs+0x20/0xe0
[<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
[<ffffffff8119c25d>] do_mount+0x31d/0x970
[<ffffffff8119cc30>] SyS_mount+0x90/0xe0
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
}
... key at: [<ffffffffa00d06e0>] __key.40139+0x0/0xfffffffffffe2920 [btrfs]
... acquired at:
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a01c8c>] down_read+0x4c/0xa0
[<ffffffffa001ff78>] find_free_extent+0x7e8/0xbe0 [btrfs]
[<ffffffffa0020434>] btrfs_reserve_extent+0xa4/0x130 [btrfs]
[<ffffffffa0021903>] btrfs_alloc_free_block+0x103/0x4c0 [btrfs]
[<ffffffffa000beb4>] __btrfs_cow_block+0x124/0x550 [btrfs]
[<ffffffffa000c4ab>] btrfs_cow_block+0x12b/0x1e0 [btrfs]
[<ffffffffa0010429>] btrfs_search_slot+0x1d9/0xa00 [btrfs]
[<ffffffffa00123de>] btrfs_insert_empty_items+0x7e/0xe0 [btrfs]
[<ffffffffa008a7a4>] btrfs_insert_delayed_items+0x84/0x460 [btrfs]
[<ffffffffa008b299>] __btrfs_run_delayed_items+0xd9/0x1f0 [btrfs]
[<ffffffffa008b7a3>] btrfs_run_delayed_items+0x13/0x20 [btrfs]
[<ffffffffa0034e6e>] btrfs_commit_transaction+0x3ae/0xa30 [btrfs]
[<ffffffffa006ae72>] btrfs_mksubvol+0x3a2/0x3b0 [btrfs]
[<ffffffffa006b036>] btrfs_ioctl_snap_create_transid+0x1b6/0x1c0 [btrfs]
[<ffffffffa006b1be>] btrfs_ioctl_snap_create_v2+0xfe/0x140 [btrfs]
[<ffffffffa006d1d2>] btrfs_ioctl+0x652/0x1940 [btrfs]
[<ffffffff8118c0a1>] do_vfs_ioctl+0x91/0x560
[<ffffffff8118c5c7>] SyS_ioctl+0x57/0x90
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
-> (&delayed_node->mutex){+.+.-.} ops: 8191147 {
HARDIRQ-ON-W at:
[<ffffffff810af476>] __lock_acquire+0x7f6/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa008c5c9>] btrfs_delayed_update_inode+0x49/0x670 [btrfs]
[<ffffffffa003b76a>] btrfs_update_inode+0x6a/0x100 [btrfs]
[<ffffffffa0043e8e>] btrfs_create+0x16e/0x220 [btrfs]
[<ffffffff81188979>] vfs_create+0x89/0xc0
[<ffffffff81189144>] do_last+0x794/0xd50
[<ffffffff811897c7>] path_openat+0xc7/0x620
[<ffffffff8118a1da>] do_filp_open+0x4a/0xa0
[<ffffffff8117883e>] do_sys_open+0x11e/0x230
[<ffffffff8117896e>] SyS_open+0x1e/0x20
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
SOFTIRQ-ON-W at:
[<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa008c5c9>] btrfs_delayed_update_inode+0x49/0x670 [btrfs]
[<ffffffffa003b76a>] btrfs_update_inode+0x6a/0x100 [btrfs]
[<ffffffffa0043e8e>] btrfs_create+0x16e/0x220 [btrfs]
[<ffffffff81188979>] vfs_create+0x89/0xc0
[<ffffffff81189144>] do_last+0x794/0xd50
[<ffffffff811897c7>] path_openat+0xc7/0x620
[<ffffffff8118a1da>] do_filp_open+0x4a/0xa0
[<ffffffff8117883e>] do_sys_open+0x11e/0x230
[<ffffffff8117896e>] SyS_open+0x1e/0x20
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
IN-RECLAIM_FS-W at:
[<ffffffff810af2ec>] __lock_acquire+0x66c/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa0089dff>] __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30 [btrfs]
[<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
[<ffffffff81195188>] evict+0xb8/0x1c0
[<ffffffff811952df>] dispose_list+0x4f/0x60
[<ffffffff811963dc>] prune_icache_sb+0x4c/0x60
[<ffffffff8117d006>] super_cache_scan+0x126/0x190
[<ffffffff8113a1fe>] shrink_slab_node+0x14e/0x2c0
[<ffffffff8113bea8>] shrink_slab+0x78/0x110
[<ffffffff8113f1bd>] do_try_to_free_pages+0x24d/0x410
[<ffffffff8113f554>] try_to_free_pages+0xe4/0x190
[<ffffffff8113235b>] __alloc_pages_nodemask+0x68b/0xa60
[<ffffffff8116edac>] cache_alloc_refill+0x40c/0x7b0
[<ffffffff8116e98f>] kmem_cache_alloc+0x1ef/0x200
[<ffffffff8105efae>] copy_process+0x13e/0x1910
[<ffffffff810608a5>] do_fork+0x65/0x300
[<ffffffff81060bc6>] SyS_clone+0x16/0x20
[<ffffffff81a0cdb9>] stub_clone+0x69/0x90
INITIAL USE at:
[<ffffffff810aefd4>] __lock_acquire+0x354/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa008c5c9>] btrfs_delayed_update_inode+0x49/0x670 [btrfs]
[<ffffffffa003b76a>] btrfs_update_inode+0x6a/0x100 [btrfs]
[<ffffffffa0043e8e>] btrfs_create+0x16e/0x220 [btrfs]
[<ffffffff81188979>] vfs_create+0x89/0xc0
[<ffffffff81189144>] do_last+0x794/0xd50
[<ffffffff811897c7>] path_openat+0xc7/0x620
[<ffffffff8118a1da>] do_filp_open+0x4a/0xa0
[<ffffffff8117883e>] do_sys_open+0x11e/0x230
[<ffffffff8117896e>] SyS_open+0x1e/0x20
[<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b
}
... key at: [<ffffffffa00d4648>] __key.35416+0x0/0xfffffffffffde9b8 [btrfs]
... acquired at:
[<ffffffff810add8a>] check_usage_forwards+0xaa/0x120
[<ffffffff810ae9f9>] mark_lock+0x1a9/0x430
[<ffffffff810af2ec>] __lock_acquire+0x66c/0x1fb0
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa0089dff>] __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30 [btrfs]
[<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
[<ffffffff81195188>] evict+0xb8/0x1c0
[<ffffffff811952df>] dispose_list+0x4f/0x60
[<ffffffff811963dc>] prune_icache_sb+0x4c/0x60
[<ffffffff8117d006>] super_cache_scan+0x126/0x190
[<ffffffff8113a1fe>] shrink_slab_node+0x14e/0x2c0
[<ffffffff8113bea8>] shrink_slab+0x78/0x110
[<ffffffff8113f1bd>] do_try_to_free_pages+0x24d/0x410
[<ffffffff8113f554>] try_to_free_pages+0xe4/0x190
[<ffffffff8113235b>] __alloc_pages_nodemask+0x68b/0xa60
[<ffffffff8116edac>] cache_alloc_refill+0x40c/0x7b0
[<ffffffff8116e98f>] kmem_cache_alloc+0x1ef/0x200
[<ffffffff8105efae>] copy_process+0x13e/0x1910
[<ffffffff810608a5>] do_fork+0x65/0x300
[<ffffffff81060bc6>] SyS_clone+0x16/0x20
[<ffffffff81a0cdb9>] stub_clone+0x69/0x90
stack backtrace:
CPU: 1 PID: 31203 Comm: 224 Tainted: G W 3.14.0-rc5-default #121
Hardware name: Intel Corporation Santa Rosa platform/Matanzas, BIOS TSRSCRB1.86C.0047.B00.0610170821 10/17/06
ffffffff82c37220 ffff88003faa9460 ffffffff819fd463 0000000000000002
ffffffff82c37220 ffff88003faa94b0 ffffffff810adc97 ffff88003faa9500
ffffffff81f16ba4 ffff88003faa94cc ffff88003faa5730 ffff88003faa94c0
Call Trace:
[<ffffffff819fd463>] dump_stack+0x51/0x6e
[<ffffffff810adc97>] print_irq_inversion_bug+0x1c7/0x210
[<ffffffff810add8a>] check_usage_forwards+0xaa/0x120
[<ffffffff810adce0>] ? print_irq_inversion_bug+0x210/0x210
[<ffffffff810ae9f9>] mark_lock+0x1a9/0x430
[<ffffffff810af2ec>] __lock_acquire+0x66c/0x1fb0
[<ffffffff810adce0>] ? print_irq_inversion_bug+0x210/0x210
[<ffffffff810ae4cf>] ? check_irq_usage+0x9f/0xf0
[<ffffffff810b002b>] ? __lock_acquire+0x13ab/0x1fb0
[<ffffffff810ab11d>] ? trace_hardirqs_off+0xd/0x10
[<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffff810b12e2>] lock_acquire+0x92/0x120
[<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
[<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffff8109a138>] ? sched_clock_cpu+0xa8/0xd0
[<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffff810ab62d>] ? lock_release_holdtime+0x3d/0x1c0
[<ffffffffa0089dff>] __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
[<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30 [btrfs]
[<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
[<ffffffff81a03c6b>] ? _raw_spin_unlock+0x2b/0x40
[<ffffffff81195188>] evict+0xb8/0x1c0
[<ffffffff81195ed0>] ? insert_inode_locked+0x1a0/0x1a0
[<ffffffff811952df>] dispose_list+0x4f/0x60
[<ffffffff811963dc>] prune_icache_sb+0x4c/0x60
[<ffffffff8117d006>] super_cache_scan+0x126/0x190
[<ffffffff8113a1fe>] shrink_slab_node+0x14e/0x2c0
[<ffffffff8109a176>] ? local_clock+0x16/0x30
[<ffffffff8113bea8>] shrink_slab+0x78/0x110
[<ffffffff8113f1bd>] do_try_to_free_pages+0x24d/0x410
[<ffffffff8113f554>] try_to_free_pages+0xe4/0x190
[<ffffffff8113235b>] __alloc_pages_nodemask+0x68b/0xa60
[<ffffffff8116ed6e>] ? cache_alloc_refill+0x3ce/0x7b0
[<ffffffff8116edac>] cache_alloc_refill+0x40c/0x7b0
[<ffffffff8105efae>] ? copy_process+0x13e/0x1910
[<ffffffff8116e98f>] kmem_cache_alloc+0x1ef/0x200
[<ffffffff8105efae>] copy_process+0x13e/0x1910
[<ffffffff8109a138>] ? sched_clock_cpu+0xa8/0xd0
[<ffffffff8109a176>] ? local_clock+0x16/0x30
[<ffffffff810ab62d>] ? lock_release_holdtime+0x3d/0x1c0
[<ffffffff810b0fc5>] ? lock_release_non_nested+0x395/0x3e0
[<ffffffff8114fd86>] ? might_fault+0x66/0xc0
[<ffffffff810608a5>] do_fork+0x65/0x300
[<ffffffff8114fd86>] ? might_fault+0x66/0xc0
[<ffffffff81a0cab7>] ? sysret_check+0x1b/0x56
[<ffffffff81060bc6>] SyS_clone+0x16/0x20
[<ffffffff81a0cdb9>] stub_clone+0x69/0x90
[<ffffffff81a0ca92>] ? system_call_fastpath+0x16/0x1b
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-03-17 13:05 ` David Sterba
@ 2014-03-26 17:01 ` Jeff Mahoney
2014-04-04 21:15 ` Sasha Levin
0 siblings, 1 reply; 9+ messages in thread
From: Jeff Mahoney @ 2014-03-26 17:01 UTC (permalink / raw)
To: dsterba, Sasha Levin, clm, jbacik, linux-btrfs, LKML
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 3/17/14, 9:05 AM, David Sterba wrote:
> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>> While fuzzing with trinity inside a KVM tools guest running the
>> latest -next kernel I've stumbled on the following:
>>
>> [ 788.458756] CPU0 CPU1 [ 788.459188]
>> ---- ---- [ 788.459625]
>> lock(&found->groups_sem); [ 788.460041]
>> local_irq_disable(); [ 788.460041]
>> lock(&delayed_node->mutex); [ 788.460041]
>> lock(&found->groups_sem); [ 788.460041] <Interrupt> [
>> 788.460041] lock(&delayed_node->mutex); [ 788.460041] [
>> 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2
>> locks held by kswapd3/4199:
>
> I've once (3.14-rc5) seen the same warning also caused by
> xfstests/generic/224
I think this is from my sysfs patches. We call kobject_add while
holding the group_sem. kobject_add ultimately allocates with
GFP_KERNEL, so it can enter reclaim. This particular case isn't
dangerous, but it could hit while hot-adding a device. The fix should
be pretty simple.
- -Jeff
> 2 locks held by 224/31203: #0: (shrinker_rwsem){++++..}, at:
> [<ffffffff8113be6d>] shrink_slab+0x3d/0x110 #1:
> (&type->s_umount_key#32){++++++}, at: [<ffffffff8117cd84>]
> grab_super_passive+0x44/0x90
>
> the shortest dependencies between 2nd lock and 1st lock: ->
> (&found->groups_sem){+++++.} ops: 405561 { HARDIRQ-ON-W at:
> [<ffffffff810af476>] __lock_acquire+0x7f6/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a01d3c>]
> down_write+0x5c/0xc0 [<ffffffffa001f6a6>]
> __link_block_group+0x46/0x130 [btrfs] [<ffffffffa0023411>]
> btrfs_read_block_groups+0x341/0x690 [btrfs] [<ffffffffa0031c50>]
> open_ctree+0x1880/0x2310 [btrfs] [<ffffffffa00065db>]
> btrfs_mount+0x55b/0x860 [btrfs] [<ffffffff8117d6f0>]
> mount_fs+0x20/0xe0 [<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
> [<ffffffff8119c25d>] do_mount+0x31d/0x970 [<ffffffff8119cc30>]
> SyS_mount+0x90/0xe0 [<ffffffff81a0ca92>]
> system_call_fastpath+0x16/0x1b HARDIRQ-ON-R at:
> [<ffffffff810af265>] __lock_acquire+0x5e5/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a01c8c>]
> down_read+0x4c/0xa0 [<ffffffffa002da32>]
> btrfs_calc_num_tolerated_disk_barrier_failures+0x142/0x240 [btrfs]
> [<ffffffffa0031c6e>] open_ctree+0x189e/0x2310 [btrfs]
> [<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
> [<ffffffff8117d6f0>] mount_fs+0x20/0xe0 [<ffffffff81199ab6>]
> vfs_kern_mount+0x76/0x160 [<ffffffff8119c25d>]
> do_mount+0x31d/0x970 [<ffffffff8119cc30>] SyS_mount+0x90/0xe0
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b SOFTIRQ-ON-W
> at: [<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a01d3c>]
> down_write+0x5c/0xc0 [<ffffffffa001f6a6>]
> __link_block_group+0x46/0x130 [btrfs] [<ffffffffa0023411>]
> btrfs_read_block_groups+0x341/0x690 [btrfs] [<ffffffffa0031c50>]
> open_ctree+0x1880/0x2310 [btrfs] [<ffffffffa00065db>]
> btrfs_mount+0x55b/0x860 [btrfs] [<ffffffff8117d6f0>]
> mount_fs+0x20/0xe0 [<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
> [<ffffffff8119c25d>] do_mount+0x31d/0x970 [<ffffffff8119cc30>]
> SyS_mount+0x90/0xe0 [<ffffffff81a0ca92>]
> system_call_fastpath+0x16/0x1b SOFTIRQ-ON-R at:
> [<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a01c8c>]
> down_read+0x4c/0xa0 [<ffffffffa002da32>]
> btrfs_calc_num_tolerated_disk_barrier_failures+0x142/0x240 [btrfs]
> [<ffffffffa0031c6e>] open_ctree+0x189e/0x2310 [btrfs]
> [<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
> [<ffffffff8117d6f0>] mount_fs+0x20/0xe0 [<ffffffff81199ab6>]
> vfs_kern_mount+0x76/0x160 [<ffffffff8119c25d>]
> do_mount+0x31d/0x970 [<ffffffff8119cc30>] SyS_mount+0x90/0xe0
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b RECLAIM_FS-ON-W
> at: [<ffffffff810b1c6c>] mark_held_locks+0x8c/0x170
> [<ffffffff810b249a>] lockdep_trace_alloc+0x8a/0xd0
> [<ffffffff8116f807>] __kmalloc_track_caller+0x47/0x210
> [<ffffffff813cdefb>] kvasprintf+0x5b/0x90 [<ffffffff813c166a>]
> kobject_set_name_vargs+0x2a/0x70 [<ffffffff813c1ffa>]
> kobject_add+0x5a/0xb0 [<ffffffffa001f75d>]
> __link_block_group+0xfd/0x130 [btrfs] [<ffffffffa0023411>]
> btrfs_read_block_groups+0x341/0x690 [btrfs] [<ffffffffa0031c50>]
> open_ctree+0x1880/0x2310 [btrfs] [<ffffffffa00065db>]
> btrfs_mount+0x55b/0x860 [btrfs] [<ffffffff8117d6f0>]
> mount_fs+0x20/0xe0 [<ffffffff81199ab6>] vfs_kern_mount+0x76/0x160
> [<ffffffff8119c25d>] do_mount+0x31d/0x970 [<ffffffff8119cc30>]
> SyS_mount+0x90/0xe0 [<ffffffff81a0ca92>]
> system_call_fastpath+0x16/0x1b INITIAL USE at: [<ffffffff810aefd4>]
> __lock_acquire+0x354/0x1fb0 [<ffffffff810b12e2>]
> lock_acquire+0x92/0x120 [<ffffffff81a01d3c>] down_write+0x5c/0xc0
> [<ffffffffa001f6a6>] __link_block_group+0x46/0x130 [btrfs]
> [<ffffffffa0023411>] btrfs_read_block_groups+0x341/0x690 [btrfs]
> [<ffffffffa0031c50>] open_ctree+0x1880/0x2310 [btrfs]
> [<ffffffffa00065db>] btrfs_mount+0x55b/0x860 [btrfs]
> [<ffffffff8117d6f0>] mount_fs+0x20/0xe0 [<ffffffff81199ab6>]
> vfs_kern_mount+0x76/0x160 [<ffffffff8119c25d>]
> do_mount+0x31d/0x970 [<ffffffff8119cc30>] SyS_mount+0x90/0xe0
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b } ... key
> at: [<ffffffffa00d06e0>] __key.40139+0x0/0xfffffffffffe2920
> [btrfs] ... acquired at: [<ffffffff810b12e2>]
> lock_acquire+0x92/0x120 [<ffffffff81a01c8c>] down_read+0x4c/0xa0
> [<ffffffffa001ff78>] find_free_extent+0x7e8/0xbe0 [btrfs]
> [<ffffffffa0020434>] btrfs_reserve_extent+0xa4/0x130 [btrfs]
> [<ffffffffa0021903>] btrfs_alloc_free_block+0x103/0x4c0 [btrfs]
> [<ffffffffa000beb4>] __btrfs_cow_block+0x124/0x550 [btrfs]
> [<ffffffffa000c4ab>] btrfs_cow_block+0x12b/0x1e0 [btrfs]
> [<ffffffffa0010429>] btrfs_search_slot+0x1d9/0xa00 [btrfs]
> [<ffffffffa00123de>] btrfs_insert_empty_items+0x7e/0xe0 [btrfs]
> [<ffffffffa008a7a4>] btrfs_insert_delayed_items+0x84/0x460 [btrfs]
> [<ffffffffa008b299>] __btrfs_run_delayed_items+0xd9/0x1f0 [btrfs]
> [<ffffffffa008b7a3>] btrfs_run_delayed_items+0x13/0x20 [btrfs]
> [<ffffffffa0034e6e>] btrfs_commit_transaction+0x3ae/0xa30 [btrfs]
> [<ffffffffa006ae72>] btrfs_mksubvol+0x3a2/0x3b0 [btrfs]
> [<ffffffffa006b036>] btrfs_ioctl_snap_create_transid+0x1b6/0x1c0
> [btrfs] [<ffffffffa006b1be>] btrfs_ioctl_snap_create_v2+0xfe/0x140
> [btrfs] [<ffffffffa006d1d2>] btrfs_ioctl+0x652/0x1940 [btrfs]
> [<ffffffff8118c0a1>] do_vfs_ioctl+0x91/0x560 [<ffffffff8118c5c7>]
> SyS_ioctl+0x57/0x90 [<ffffffff81a0ca92>]
> system_call_fastpath+0x16/0x1b
>
> -> (&delayed_node->mutex){+.+.-.} ops: 8191147 { HARDIRQ-ON-W at:
> [<ffffffff810af476>] __lock_acquire+0x7f6/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a00c4e>]
> mutex_lock_nested+0x6e/0x3a0 [<ffffffffa008c5c9>]
> btrfs_delayed_update_inode+0x49/0x670 [btrfs] [<ffffffffa003b76a>]
> btrfs_update_inode+0x6a/0x100 [btrfs] [<ffffffffa0043e8e>]
> btrfs_create+0x16e/0x220 [btrfs] [<ffffffff81188979>]
> vfs_create+0x89/0xc0 [<ffffffff81189144>] do_last+0x794/0xd50
> [<ffffffff811897c7>] path_openat+0xc7/0x620 [<ffffffff8118a1da>]
> do_filp_open+0x4a/0xa0 [<ffffffff8117883e>]
> do_sys_open+0x11e/0x230 [<ffffffff8117896e>] SyS_open+0x1e/0x20
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b SOFTIRQ-ON-W
> at: [<ffffffff810af4aa>] __lock_acquire+0x82a/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a00c4e>]
> mutex_lock_nested+0x6e/0x3a0 [<ffffffffa008c5c9>]
> btrfs_delayed_update_inode+0x49/0x670 [btrfs] [<ffffffffa003b76a>]
> btrfs_update_inode+0x6a/0x100 [btrfs] [<ffffffffa0043e8e>]
> btrfs_create+0x16e/0x220 [btrfs] [<ffffffff81188979>]
> vfs_create+0x89/0xc0 [<ffffffff81189144>] do_last+0x794/0xd50
> [<ffffffff811897c7>] path_openat+0xc7/0x620 [<ffffffff8118a1da>]
> do_filp_open+0x4a/0xa0 [<ffffffff8117883e>]
> do_sys_open+0x11e/0x230 [<ffffffff8117896e>] SyS_open+0x1e/0x20
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b IN-RECLAIM_FS-W
> at: [<ffffffff810af2ec>] __lock_acquire+0x66c/0x1fb0
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffff81a00c4e>]
> mutex_lock_nested+0x6e/0x3a0 [<ffffffffa0089dff>]
> __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
> [<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30 [btrfs]
> [<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
> [<ffffffff81195188>] evict+0xb8/0x1c0 [<ffffffff811952df>]
> dispose_list+0x4f/0x60 [<ffffffff811963dc>]
> prune_icache_sb+0x4c/0x60 [<ffffffff8117d006>]
> super_cache_scan+0x126/0x190 [<ffffffff8113a1fe>]
> shrink_slab_node+0x14e/0x2c0 [<ffffffff8113bea8>]
> shrink_slab+0x78/0x110 [<ffffffff8113f1bd>]
> do_try_to_free_pages+0x24d/0x410 [<ffffffff8113f554>]
> try_to_free_pages+0xe4/0x190 [<ffffffff8113235b>]
> __alloc_pages_nodemask+0x68b/0xa60 [<ffffffff8116edac>]
> cache_alloc_refill+0x40c/0x7b0 [<ffffffff8116e98f>]
> kmem_cache_alloc+0x1ef/0x200 [<ffffffff8105efae>]
> copy_process+0x13e/0x1910 [<ffffffff810608a5>] do_fork+0x65/0x300
> [<ffffffff81060bc6>] SyS_clone+0x16/0x20 [<ffffffff81a0cdb9>]
> stub_clone+0x69/0x90 INITIAL USE at: [<ffffffff810aefd4>]
> __lock_acquire+0x354/0x1fb0 [<ffffffff810b12e2>]
> lock_acquire+0x92/0x120 [<ffffffff81a00c4e>]
> mutex_lock_nested+0x6e/0x3a0 [<ffffffffa008c5c9>]
> btrfs_delayed_update_inode+0x49/0x670 [btrfs] [<ffffffffa003b76a>]
> btrfs_update_inode+0x6a/0x100 [btrfs] [<ffffffffa0043e8e>]
> btrfs_create+0x16e/0x220 [btrfs] [<ffffffff81188979>]
> vfs_create+0x89/0xc0 [<ffffffff81189144>] do_last+0x794/0xd50
> [<ffffffff811897c7>] path_openat+0xc7/0x620 [<ffffffff8118a1da>]
> do_filp_open+0x4a/0xa0 [<ffffffff8117883e>]
> do_sys_open+0x11e/0x230 [<ffffffff8117896e>] SyS_open+0x1e/0x20
> [<ffffffff81a0ca92>] system_call_fastpath+0x16/0x1b } ... key
> at: [<ffffffffa00d4648>] __key.35416+0x0/0xfffffffffffde9b8
> [btrfs] ... acquired at: [<ffffffff810add8a>]
> check_usage_forwards+0xaa/0x120 [<ffffffff810ae9f9>]
> mark_lock+0x1a9/0x430 [<ffffffff810af2ec>]
> __lock_acquire+0x66c/0x1fb0 [<ffffffff810b12e2>]
> lock_acquire+0x92/0x120 [<ffffffff81a00c4e>]
> mutex_lock_nested+0x6e/0x3a0 [<ffffffffa0089dff>]
> __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
> [<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30 [btrfs]
> [<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
> [<ffffffff81195188>] evict+0xb8/0x1c0 [<ffffffff811952df>]
> dispose_list+0x4f/0x60 [<ffffffff811963dc>]
> prune_icache_sb+0x4c/0x60 [<ffffffff8117d006>]
> super_cache_scan+0x126/0x190 [<ffffffff8113a1fe>]
> shrink_slab_node+0x14e/0x2c0 [<ffffffff8113bea8>]
> shrink_slab+0x78/0x110 [<ffffffff8113f1bd>]
> do_try_to_free_pages+0x24d/0x410 [<ffffffff8113f554>]
> try_to_free_pages+0xe4/0x190 [<ffffffff8113235b>]
> __alloc_pages_nodemask+0x68b/0xa60 [<ffffffff8116edac>]
> cache_alloc_refill+0x40c/0x7b0 [<ffffffff8116e98f>]
> kmem_cache_alloc+0x1ef/0x200 [<ffffffff8105efae>]
> copy_process+0x13e/0x1910 [<ffffffff810608a5>] do_fork+0x65/0x300
> [<ffffffff81060bc6>] SyS_clone+0x16/0x20 [<ffffffff81a0cdb9>]
> stub_clone+0x69/0x90
>
>
> stack backtrace: CPU: 1 PID: 31203 Comm: 224 Tainted: G W
> 3.14.0-rc5-default #121 Hardware name: Intel Corporation Santa Rosa
> platform/Matanzas, BIOS TSRSCRB1.86C.0047.B00.0610170821 10/17/06
> ffffffff82c37220 ffff88003faa9460 ffffffff819fd463
> 0000000000000002 ffffffff82c37220 ffff88003faa94b0 ffffffff810adc97
> ffff88003faa9500 ffffffff81f16ba4 ffff88003faa94cc ffff88003faa5730
> ffff88003faa94c0 Call Trace: [<ffffffff819fd463>]
> dump_stack+0x51/0x6e [<ffffffff810adc97>]
> print_irq_inversion_bug+0x1c7/0x210 [<ffffffff810add8a>]
> check_usage_forwards+0xaa/0x120 [<ffffffff810adce0>] ?
> print_irq_inversion_bug+0x210/0x210 [<ffffffff810ae9f9>]
> mark_lock+0x1a9/0x430 [<ffffffff810af2ec>]
> __lock_acquire+0x66c/0x1fb0 [<ffffffff810adce0>] ?
> print_irq_inversion_bug+0x210/0x210 [<ffffffff810ae4cf>] ?
> check_irq_usage+0x9f/0xf0 [<ffffffff810b002b>] ?
> __lock_acquire+0x13ab/0x1fb0 [<ffffffff810ab11d>] ?
> trace_hardirqs_off+0xd/0x10 [<ffffffffa0089dff>] ?
> __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
> [<ffffffff810b12e2>] lock_acquire+0x92/0x120 [<ffffffffa0089dff>] ?
> __btrfs_release_delayed_node+0x4f/0x220 [btrfs]
> [<ffffffff81a00c4e>] mutex_lock_nested+0x6e/0x3a0
> [<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220
> [btrfs] [<ffffffff8109a138>] ? sched_clock_cpu+0xa8/0xd0
> [<ffffffffa0089dff>] ? __btrfs_release_delayed_node+0x4f/0x220
> [btrfs] [<ffffffff810ab62d>] ? lock_release_holdtime+0x3d/0x1c0
> [<ffffffffa0089dff>] __btrfs_release_delayed_node+0x4f/0x220
> [btrfs] [<ffffffffa008bac4>] btrfs_remove_delayed_node+0x24/0x30
> [btrfs] [<ffffffffa0040c97>] btrfs_evict_inode+0x2a7/0x550 [btrfs]
> [<ffffffff81a03c6b>] ? _raw_spin_unlock+0x2b/0x40
> [<ffffffff81195188>] evict+0xb8/0x1c0 [<ffffffff81195ed0>] ?
> insert_inode_locked+0x1a0/0x1a0 [<ffffffff811952df>]
> dispose_list+0x4f/0x60 [<ffffffff811963dc>]
> prune_icache_sb+0x4c/0x60 [<ffffffff8117d006>]
> super_cache_scan+0x126/0x190 [<ffffffff8113a1fe>]
> shrink_slab_node+0x14e/0x2c0 [<ffffffff8109a176>] ?
> local_clock+0x16/0x30 [<ffffffff8113bea8>] shrink_slab+0x78/0x110
> [<ffffffff8113f1bd>] do_try_to_free_pages+0x24d/0x410
> [<ffffffff8113f554>] try_to_free_pages+0xe4/0x190
> [<ffffffff8113235b>] __alloc_pages_nodemask+0x68b/0xa60
> [<ffffffff8116ed6e>] ? cache_alloc_refill+0x3ce/0x7b0
> [<ffffffff8116edac>] cache_alloc_refill+0x40c/0x7b0
> [<ffffffff8105efae>] ? copy_process+0x13e/0x1910
> [<ffffffff8116e98f>] kmem_cache_alloc+0x1ef/0x200
> [<ffffffff8105efae>] copy_process+0x13e/0x1910 [<ffffffff8109a138>]
> ? sched_clock_cpu+0xa8/0xd0 [<ffffffff8109a176>] ?
> local_clock+0x16/0x30 [<ffffffff810ab62d>] ?
> lock_release_holdtime+0x3d/0x1c0 [<ffffffff810b0fc5>] ?
> lock_release_non_nested+0x395/0x3e0 [<ffffffff8114fd86>] ?
> might_fault+0x66/0xc0 [<ffffffff810608a5>] do_fork+0x65/0x300
> [<ffffffff8114fd86>] ? might_fault+0x66/0xc0 [<ffffffff81a0cab7>] ?
> sysret_check+0x1b/0x56 [<ffffffff81060bc6>] SyS_clone+0x16/0x20
> [<ffffffff81a0cdb9>] stub_clone+0x69/0x90 [<ffffffff81a0ca92>] ?
> system_call_fastpath+0x16/0x1b -- To unsubscribe from this list:
> send the line "unsubscribe linux-btrfs" in the body of a message to
> majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html
>
- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.19 (Darwin)
iQIcBAEBAgAGBQJTMwftAAoJEB57S2MheeWy5awP/jYQgrhE6vw/Vv/A02btb4Ey
X9EhpYohUaT6MU8WZYamijOA4Kx8OR1wEZ3CFQ3auhD/YceCEeq47Y02PpCCEjzq
O78IcKPPRt4vepGrUYkV3FKDpc0N190LG1/Wu1fOQGo44naFxX6z1eH/Vd/o8D+8
NUUC2nDhAGjKM937BuYfGZrqIZiOIuBfWynMNKNXW9x5HY2EbhHbjkyn6iUhssKm
7HSgQ/5upjK9Af+L5hEZWS0qCKjx4QXrj0D9ir5Vll/EOVn2dsL58z09bXk+veor
RtWhk6cwrLA2Twwo3LCLSAIIVJwuaCGKIQdv3rjt6Qp541P7q24WK+MUqcfn1oTn
YvrOWikWoh4mFxuMn0y6XvCg7LjlY49u+Z69OTWd50PrePjx/UPPrrtwNIf8sKUO
vPvk7r5sI1sxIsNjqEn4UUeoR73SbyHzN060T/3GGdySk8Nz+G4LwOYuqP3q6H/x
TnoELE0voAbwr4JvufRwQrId4jS4j80p67qhRcUHJkxoEn8NhiXvg4NhqXzDAyVg
6xyZ6mKvzxGfs6DhXqCN7F/gfuOlyxZpxjvj3ZLhPiHhNTdQXRl3CVPsuNg0CSEj
DNI4mYpIInlZTEDjiWblW3+bN+hGtmKRshe56UEhEl5yGzZtMvNvcGcSRuyui/6z
O1vbSR/A2u+5d2/fLu7C
=FMQp
-----END PGP SIGNATURE-----
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-03-26 17:01 ` Jeff Mahoney
@ 2014-04-04 21:15 ` Sasha Levin
2014-04-07 16:54 ` David Sterba
0 siblings, 1 reply; 9+ messages in thread
From: Sasha Levin @ 2014-04-04 21:15 UTC (permalink / raw)
To: Jeff Mahoney, dsterba, clm, jbacik, linux-btrfs, LKML
On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
> On 3/17/14, 9:05 AM, David Sterba wrote:
>>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
>>>>>
>>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
>>>
>>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
> I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
Is that fix available anywhere? I'm still seeing the issue in -next.
Thanks,
Sasha
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-04-04 21:15 ` Sasha Levin
@ 2014-04-07 16:54 ` David Sterba
2014-04-07 17:17 ` Chris Mason
0 siblings, 1 reply; 9+ messages in thread
From: David Sterba @ 2014-04-07 16:54 UTC (permalink / raw)
To: Sasha Levin; +Cc: Jeff Mahoney, dsterba, clm, jbacik, linux-btrfs, LKML
On Fri, Apr 04, 2014 at 05:15:23PM -0400, Sasha Levin wrote:
> On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
> > On 3/17/14, 9:05 AM, David Sterba wrote:
> >>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
> >>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
> >>>>>
> >>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
> >>>
> >>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
> > I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
>
> Is that fix available anywhere? I'm still seeing the issue in -next.
It is: https://patchwork.kernel.org/patch/3894781/ , will probably hit -rc2
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-04-07 16:54 ` David Sterba
@ 2014-04-07 17:17 ` Chris Mason
2014-04-07 18:03 ` Sasha Levin
0 siblings, 1 reply; 9+ messages in thread
From: Chris Mason @ 2014-04-07 17:17 UTC (permalink / raw)
To: dsterba, Sasha Levin, Jeff Mahoney, jbacik, linux-btrfs, LKML
On 04/07/2014 12:54 PM, David Sterba wrote:
> On Fri, Apr 04, 2014 at 05:15:23PM -0400, Sasha Levin wrote:
>> On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
>>> On 3/17/14, 9:05 AM, David Sterba wrote:
>>>>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>>>>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
>>>>>>>
>>>>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
>>>>>
>>>>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
>>> I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
>>
>> Is that fix available anywhere? I'm still seeing the issue in -next.
>
> It is: https://urldefense.proofpoint.com/v1/url?u=https://patchwork.kernel.org/patch/3894781/&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0A&m=HQJVSK4wPTft1zWwI1cGvwj5OfdmN5UItVlucU1K31o%3D%0A&s=5113699a2e7345a779333c87dd5b1d88b4410a7c7fcd5fa424baeb838ad7d31b , will probably hit -rc2
>
Its in the integration branch now along with some other important fixes.
We'll get it out shortly
-chris
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-04-07 17:17 ` Chris Mason
@ 2014-04-07 18:03 ` Sasha Levin
2014-04-07 18:29 ` Josef Bacik
2014-04-07 19:27 ` Chris Mason
0 siblings, 2 replies; 9+ messages in thread
From: Sasha Levin @ 2014-04-07 18:03 UTC (permalink / raw)
To: Chris Mason, dsterba, Jeff Mahoney, jbacik, linux-btrfs, LKML
On 04/07/2014 01:17 PM, Chris Mason wrote:
>
>
> On 04/07/2014 12:54 PM, David Sterba wrote:
>> On Fri, Apr 04, 2014 at 05:15:23PM -0400, Sasha Levin wrote:
>>> On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
>>>> On 3/17/14, 9:05 AM, David Sterba wrote:
>>>>>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>>>>>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
>>>>>>>>
>>>>>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
>>>>>>
>>>>>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
>>>> I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
>>>
>>> Is that fix available anywhere? I'm still seeing the issue in -next.
>>
>> It is: https://urldefense.proofpoint.com/v1/url?u=https://patchwork.kernel.org/patch/3894781/&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0A&m=HQJVSK4wPTft1zWwI1cGvwj5OfdmN5UItVlucU1K31o%3D%0A&s=5113699a2e7345a779333c87dd5b1d88b4410a7c7fcd5fa424baeb838ad7d31b , will probably hit -rc2
>>
>
> Its in the integration branch now along with some other important fixes. We'll get it out shortly
Chris,
Can I suggest adding the integration branch to linux-next as well? That way
all the folks who report issues coming out of -next would be able to test
the fixes as well.
Thanks,
Sasha
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-04-07 18:03 ` Sasha Levin
@ 2014-04-07 18:29 ` Josef Bacik
2014-04-07 19:27 ` Chris Mason
1 sibling, 0 replies; 9+ messages in thread
From: Josef Bacik @ 2014-04-07 18:29 UTC (permalink / raw)
To: Sasha Levin
Cc: Chris Mason, David Sterba, Jeff Mahoney,
linux-btrfs@vger.kernel.org, LKML
I was on vacation last week, I'll update btrfs-next today once we are happy with integration. Thanks,
Josef
Sasha Levin <sasha.levin@oracle.com> wrote:
On 04/07/2014 01:17 PM, Chris Mason wrote:
>
>
> On 04/07/2014 12:54 PM, David Sterba wrote:
>> On Fri, Apr 04, 2014 at 05:15:23PM -0400, Sasha Levin wrote:
>>> On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
>>>> On 3/17/14, 9:05 AM, David Sterba wrote:
>>>>>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>>>>>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
>>>>>>>>
>>>>>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
>>>>>>
>>>>>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
>>>> I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
>>>
>>> Is that fix available anywhere? I'm still seeing the issue in -next.
>>
>> It is: https://urldefense.proofpoint.com/v1/url?u=https://patchwork.kernel.org/patch/3894781/&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0A&m=HQJVSK4wPTft1zWwI1cGvwj5OfdmN5UItVlucU1K31o%3D%0A&s=5113699a2e7345a779333c87dd5b1d88b4410a7c7fcd5fa424baeb838ad7d31b , will probably hit -rc2
>>
>
> Its in the integration branch now along with some other important fixes. We'll get it out shortly
Chris,
Can I suggest adding the integration branch to linux-next as well? That way
all the folks who report issues coming out of -next would be able to test
the fixes as well.
Thanks,
Sasha
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: btrfs: lock inversion between delayed_node->mutex and found->groups_sem
2014-04-07 18:03 ` Sasha Levin
2014-04-07 18:29 ` Josef Bacik
@ 2014-04-07 19:27 ` Chris Mason
1 sibling, 0 replies; 9+ messages in thread
From: Chris Mason @ 2014-04-07 19:27 UTC (permalink / raw)
To: Sasha Levin, dsterba, Jeff Mahoney, jbacik, linux-btrfs, LKML
On 04/07/2014 02:03 PM, Sasha Levin wrote:
> On 04/07/2014 01:17 PM, Chris Mason wrote:
>>
>>
>> On 04/07/2014 12:54 PM, David Sterba wrote:
>>> On Fri, Apr 04, 2014 at 05:15:23PM -0400, Sasha Levin wrote:
>>>> On 03/26/2014 01:01 PM, Jeff Mahoney wrote:
>>>>> On 3/17/14, 9:05 AM, David Sterba wrote:
>>>>>>> On Fri, Mar 14, 2014 at 08:12:16PM -0400, Sasha Levin wrote:
>>>>>>>>> While fuzzing with trinity inside a KVM tools guest running the latest -next kernel I've stumbled on the following:
>>>>>>>>>
>>>>>>>>> [ 788.458756] CPU0 CPU1 [ 788.459188] ---- ---- [ 788.459625] lock(&found->groups_sem); [ 788.460041] local_irq_disable(); [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] lock(&found->groups_sem); [ 788.460041] <Interrupt> [ 788.460041] lock(&delayed_node->mutex); [ 788.460041] [ 788.460041] *** DEADLOCK *** [ 788.460041] [ 788.460041] 2 locks held by kswapd3/4199:
>>>>>>>
>>>>>>> I've once (3.14-rc5) seen the same warning also caused by xfstests/generic/224
>>>>> I think this is from my sysfs patches. We call kobject_add while holding the group_sem. kobject_add ultimately allocates with GFP_KERNEL, so it can enter reclaim. This particular case isn't dangerous, but it could hit while hot-adding a device. The fix should be pretty simple.
>>>>
>>>> Is that fix available anywhere? I'm still seeing the issue in -next.
>>>
>>> It is: https://urldefense.proofpoint.com/v1/url?u=https://patchwork.kernel.org/patch/3894781/&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0A&m=HQJVSK4wPTft1zWwI1cGvwj5OfdmN5UItVlucU1K31o%3D%0A&s=5113699a2e7345a779333c87dd5b1d88b4410a7c7fcd5fa424baeb838ad7d31b , will probably hit -rc2
>>>
>>
>> Its in the integration branch now along with some other important fixes. We'll get it out shortly
>
> Chris,
>
> Can I suggest adding the integration branch to linux-next as well? That way
> all the folks who report issues coming out of -next would be able to test
> the fixes as well.
>
Hi Sasha,
The ink is still a little wet on the integration branch. It'll
definitely go to linux-next and to Linus.
-chris
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2014-04-07 19:27 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-15 0:12 btrfs: lock inversion between delayed_node->mutex and found->groups_sem Sasha Levin
2014-03-17 13:05 ` David Sterba
2014-03-26 17:01 ` Jeff Mahoney
2014-04-04 21:15 ` Sasha Levin
2014-04-07 16:54 ` David Sterba
2014-04-07 17:17 ` Chris Mason
2014-04-07 18:03 ` Sasha Levin
2014-04-07 18:29 ` Josef Bacik
2014-04-07 19:27 ` Chris Mason
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).