* [PATCH] ext4: fix a potential deadlock in __ext4_es_shrink()
[not found] <20140707233326.GA6076@bbox>
@ 2014-07-12 19:33 ` Theodore Ts'o
2014-07-13 7:00 ` Zheng Liu
0 siblings, 1 reply; 2+ messages in thread
From: Theodore Ts'o @ 2014-07-12 19:33 UTC (permalink / raw)
To: Ext4 Developers List; +Cc: Minchan Kim, Theodore Ts'o, stable, Zheng Liu
This fixes the following lockdep complaint:
[ INFO: possible circular locking dependency detected ]
3.16.0-rc2-mm1+ #7 Tainted: G O
-------------------------------------------------------
kworker/u24:0/4356 is trying to acquire lock:
(&(&sbi->s_es_lru_lock)->rlock){+.+.-.}, at: [<ffffffff81285fff>] __ext4_es_shrink+0x4f/0x2e0
but task is already holding lock:
(&ei->i_es_lock){++++-.}, at: [<ffffffff81286961>] ext4_es_insert_extent+0x71/0x180
which lock already depends on the new lock.
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&ei->i_es_lock);
lock(&(&sbi->s_es_lru_lock)->rlock);
lock(&ei->i_es_lock);
lock(&(&sbi->s_es_lru_lock)->rlock);
*** DEADLOCK ***
6 locks held by kworker/u24:0/4356:
#0: ("writeback"){.+.+.+}, at: [<ffffffff81071d00>] process_one_work+0x180/0x560
#1: ((&(&wb->dwork)->work)){+.+.+.}, at: [<ffffffff81071d00>] process_one_work+0x180/0x560
#2: (&type->s_umount_key#22){++++++}, at: [<ffffffff811a9c74>] grab_super_passive+0x44/0x90
#3: (jbd2_handle){+.+...}, at: [<ffffffff812979f9>] start_this_handle+0x189/0x5f0
#4: (&ei->i_data_sem){++++..}, at: [<ffffffff81247062>] ext4_map_blocks+0x132/0x550
#5: (&ei->i_es_lock){++++-.}, at: [<ffffffff81286961>] ext4_es_insert_extent+0x71/0x180
stack backtrace:
CPU: 0 PID: 4356 Comm: kworker/u24:0 Tainted: G O 3.16.0-rc2-mm1+ #7
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
ffffffff8213dce0 ffff880014b07538 ffffffff815df0bb 0000000000000007
ffffffff8213e040 ffff880014b07588 ffffffff815db3dd ffff880014b07568
ffff880014b07610 ffff88003b868930 ffff88003b868908 ffff88003b868930
Call Trace:
[<ffffffff815df0bb>] dump_stack+0x4e/0x68
[<ffffffff815db3dd>] print_circular_bug+0x1fb/0x20c
[<ffffffff810a7a3e>] __lock_acquire+0x163e/0x1d00
[<ffffffff815e89dc>] ? retint_restore_args+0xe/0xe
[<ffffffff815ddc7b>] ? __slab_alloc+0x4a8/0x4ce
[<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
[<ffffffff810a8707>] lock_acquire+0x87/0x120
[<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
[<ffffffff8128592d>] ? ext4_es_free_extent+0x5d/0x70
[<ffffffff815e6f09>] _raw_spin_lock+0x39/0x50
[<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
[<ffffffff8119760b>] ? kmem_cache_alloc+0x18b/0x1a0
[<ffffffff81285fff>] __ext4_es_shrink+0x4f/0x2e0
[<ffffffff812869b8>] ext4_es_insert_extent+0xc8/0x180
[<ffffffff812470f4>] ext4_map_blocks+0x1c4/0x550
[<ffffffff8124c4c4>] ext4_writepages+0x6d4/0xd00
...
Reported-by: Minchan Kim <minchan@kernel.org>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Reported-by: Minchan Kim <minchan@kernel.org>
Cc: stable@vger.kernel.org
Cc: Zheng Liu <gnehzuil.liu@gmail.com>
---
fs/ext4/extents_status.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 3f5c188..0b7e28e 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -966,10 +966,10 @@ retry:
continue;
}
- if (ei->i_es_lru_nr == 0 || ei == locked_ei)
+ if (ei->i_es_lru_nr == 0 || ei == locked_ei ||
+ !write_trylock(&ei->i_es_lock))
continue;
- write_lock(&ei->i_es_lock);
shrunk = __es_try_to_reclaim_extents(ei, nr_to_scan);
if (ei->i_es_lru_nr == 0)
list_del_init(&ei->i_es_lru);
--
2.0.0
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH] ext4: fix a potential deadlock in __ext4_es_shrink()
2014-07-12 19:33 ` [PATCH] ext4: fix a potential deadlock in __ext4_es_shrink() Theodore Ts'o
@ 2014-07-13 7:00 ` Zheng Liu
0 siblings, 0 replies; 2+ messages in thread
From: Zheng Liu @ 2014-07-13 7:00 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Ext4 Developers List, Minchan Kim, stable
On Sat, Jul 12, 2014 at 03:33:23PM -0400, Theodore Ts'o wrote:
> This fixes the following lockdep complaint:
>
> [ INFO: possible circular locking dependency detected ]
> 3.16.0-rc2-mm1+ #7 Tainted: G O
> -------------------------------------------------------
> kworker/u24:0/4356 is trying to acquire lock:
> (&(&sbi->s_es_lru_lock)->rlock){+.+.-.}, at: [<ffffffff81285fff>] __ext4_es_shrink+0x4f/0x2e0
>
> but task is already holding lock:
> (&ei->i_es_lock){++++-.}, at: [<ffffffff81286961>] ext4_es_insert_extent+0x71/0x180
>
> which lock already depends on the new lock.
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(&ei->i_es_lock);
> lock(&(&sbi->s_es_lru_lock)->rlock);
> lock(&ei->i_es_lock);
> lock(&(&sbi->s_es_lru_lock)->rlock);
>
> *** DEADLOCK ***
>
> 6 locks held by kworker/u24:0/4356:
> #0: ("writeback"){.+.+.+}, at: [<ffffffff81071d00>] process_one_work+0x180/0x560
> #1: ((&(&wb->dwork)->work)){+.+.+.}, at: [<ffffffff81071d00>] process_one_work+0x180/0x560
> #2: (&type->s_umount_key#22){++++++}, at: [<ffffffff811a9c74>] grab_super_passive+0x44/0x90
> #3: (jbd2_handle){+.+...}, at: [<ffffffff812979f9>] start_this_handle+0x189/0x5f0
> #4: (&ei->i_data_sem){++++..}, at: [<ffffffff81247062>] ext4_map_blocks+0x132/0x550
> #5: (&ei->i_es_lock){++++-.}, at: [<ffffffff81286961>] ext4_es_insert_extent+0x71/0x180
>
> stack backtrace:
> CPU: 0 PID: 4356 Comm: kworker/u24:0 Tainted: G O 3.16.0-rc2-mm1+ #7
> Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
> Workqueue: writeback bdi_writeback_workfn (flush-253:0)
> ffffffff8213dce0 ffff880014b07538 ffffffff815df0bb 0000000000000007
> ffffffff8213e040 ffff880014b07588 ffffffff815db3dd ffff880014b07568
> ffff880014b07610 ffff88003b868930 ffff88003b868908 ffff88003b868930
> Call Trace:
> [<ffffffff815df0bb>] dump_stack+0x4e/0x68
> [<ffffffff815db3dd>] print_circular_bug+0x1fb/0x20c
> [<ffffffff810a7a3e>] __lock_acquire+0x163e/0x1d00
> [<ffffffff815e89dc>] ? retint_restore_args+0xe/0xe
> [<ffffffff815ddc7b>] ? __slab_alloc+0x4a8/0x4ce
> [<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
> [<ffffffff810a8707>] lock_acquire+0x87/0x120
> [<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
> [<ffffffff8128592d>] ? ext4_es_free_extent+0x5d/0x70
> [<ffffffff815e6f09>] _raw_spin_lock+0x39/0x50
> [<ffffffff81285fff>] ? __ext4_es_shrink+0x4f/0x2e0
> [<ffffffff8119760b>] ? kmem_cache_alloc+0x18b/0x1a0
> [<ffffffff81285fff>] __ext4_es_shrink+0x4f/0x2e0
> [<ffffffff812869b8>] ext4_es_insert_extent+0xc8/0x180
> [<ffffffff812470f4>] ext4_map_blocks+0x1c4/0x550
> [<ffffffff8124c4c4>] ext4_writepages+0x6d4/0xd00
> ...
>
> Reported-by: Minchan Kim <minchan@kernel.org>
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> Reported-by: Minchan Kim <minchan@kernel.org>
> Cc: stable@vger.kernel.org
> Cc: Zheng Liu <gnehzuil.liu@gmail.com>
Thanks for fixing this. It looks good to me.
Reviewed-by: Zheng Liu <wenqing.lz@taobao.com>
I will pick it up into my patch set for improving es shrinker and look
at whether or not it can reduce the latency.
Thanks,
- Zheng
> ---
> fs/ext4/extents_status.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
> index 3f5c188..0b7e28e 100644
> --- a/fs/ext4/extents_status.c
> +++ b/fs/ext4/extents_status.c
> @@ -966,10 +966,10 @@ retry:
> continue;
> }
>
> - if (ei->i_es_lru_nr == 0 || ei == locked_ei)
> + if (ei->i_es_lru_nr == 0 || ei == locked_ei ||
> + !write_trylock(&ei->i_es_lock))
> continue;
>
> - write_lock(&ei->i_es_lock);
> shrunk = __es_try_to_reclaim_extents(ei, nr_to_scan);
> if (ei->i_es_lru_nr == 0)
> list_del_init(&ei->i_es_lru);
> --
> 2.0.0
>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2014-07-13 7:00 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20140707233326.GA6076@bbox>
2014-07-12 19:33 ` [PATCH] ext4: fix a potential deadlock in __ext4_es_shrink() Theodore Ts'o
2014-07-13 7:00 ` Zheng Liu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).