From: Nick Piggin <npiggin@kernel.dk>
To: Jan Kara <jack@suse.cz>
Cc: Nick Piggin <npiggin@kernel.dk>,
Andrew Morton <akpm@linux-foundation.org>,
Ted Ts'o <tytso@mit.edu>, Eric Sandeen <sandeen@redhat.com>,
linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-btrfs@vger.kernel.org
Subject: Re: [patch] fix up lock order reversal in writeback
Date: Tue, 23 Nov 2010 19:15:05 +1100 [thread overview]
Message-ID: <20101123081505.GB3364@amd> (raw)
In-Reply-To: <20101122181655.GF5012@quack.suse.cz>
Here is another one. Both i_mutex in writeback and s_umount in write(2)
underneath i_mutex seem like an interesting idea. The former you can
probably get away with (provided you solve the previous AA deadlock),
but the latter seems too problematic. I think my trylock patch solves
it.
[ 409.479214] =======================================================
[ 409.479527] [ INFO: possible circular locking dependency detected ]
[ 409.479689] 2.6.37-rc3+ #26[ 409.479837]
-------------------------------------------------------
[ 409.479998] umount/4020 is trying to acquire lock:[ 409.480155]
(ext4-dio-unwritten){+.+...}, at: [<ffffffff81067a80>]
flush_workqueue+0x0/0x540
[ 409.480178] [ 409.480178] but task is already holding lock:
[ 409.480178] (&type->s_umount_key#23){++++..}, at:
[<ffffffff8111b40d>] deactivate_super+0x3d
/0x60
[ 409.480178] [ 409.480178] which lock already depends on the new
lock.
[ 409.480178]
[ 409.480178]
[ 409.480178] the existing dependency chain (in reverse order) is:
[ 409.480178]
[ 409.480178] -> #3 (&type->s_umount_key#23){++++..}:
[ 409.480178] [<ffffffff810852c5>] lock_acquire+0x95/0x1b0
[ 409.480178] [<ffffffff81601ba2>] down_read+0x42/0x60
[ 409.480178] [<ffffffff8113d6d2>]
writeback_inodes_sb_if_idle+0x32/0x60
[ 409.480178] [<ffffffffa003fef8>]
ext4_da_write_begin+0x208/0x2d0 [ext4]
[ 409.480178] [<ffffffff810cbbf4>]
generic_file_buffered_write+0x114/0x2a0
[ 409.480178] [<ffffffff810cc5e0>]
__generic_file_aio_write+0x240/0x470
[ 409.480178] [<ffffffff810cc876>]
generic_file_aio_write+0x66/0xd0
[ 409.480178] [<ffffffffa0034fad>] ext4_file_write+0x3d/0xd0
[ext4]
[ 409.480178] [<ffffffff81117702>] do_sync_write+0xd2/0x110
[ 409.480178] [<ffffffff811179c8>] vfs_write+0xc8/0x190
[ 409.480178] [<ffffffff8111851a>] sys_pwrite64+0x7a/0x90
[ 409.480178] [<ffffffff8100312b>]
system_call_fastpath+0x16/0x1b
[ 409.480178]
[ 409.480178] -> #2 (&sb->s_type->i_mutex_key#13){+.+.+.}:
[ 409.480178] [<ffffffff810852c5>] lock_acquire+0x95/0x1b0
[ 409.480178] [<ffffffff81601329>]
__mutex_lock_common+0x59/0x480
[ 409.480178] [<ffffffff8160182e>] mutex_lock_nested+0x3e/0x50
[ 409.480178] [<ffffffffa0042107>] ext4_end_io_work+0x37/0xb0
[ext4]
[ 409.480178] [<ffffffff81068378>] process_one_work+0x1b8/0x5a0
[ 409.480178] [<ffffffff81069675>] worker_thread+0x175/0x3a0
[ 409.480178] [<ffffffff8106e246>] kthread+0x96/0xa0
[ 409.480178] [<ffffffff81003ed4>] kernel_thread_helper+0x4/0x10
[ 409.480178]
[ 409.480178] -> #1 ((&io->work)){+.+...}:
[ 409.480178] [<ffffffff810852c5>] lock_acquire+0x95/0x1b0
[ 409.480178] [<ffffffff81068364>] process_one_work+0x1a4/0x5a0
[ 409.480178] [<ffffffff81069675>] worker_thread+0x175/0x3a0
[ 409.480178] [<ffffffff8106e246>] kthread+0x96/0xa0
[ 409.480178] [<ffffffff81003ed4>] kernel_thread_helper+0x4/0x10
[ 409.480178]
[ 409.480178] -> #0 (ext4-dio-unwritten){+.+...}:
[ 409.480178] [<ffffffff81085122>] __lock_acquire+0x1382/0x1490
[ 409.480178] [<ffffffff810852c5>] lock_acquire+0x95/0x1b0
[ 409.480178] [<ffffffff81067bc8>] flush_workqueue+0x148/0x540
[ 409.480178] [<ffffffffa004f5db>] ext4_sync_fs+0x3b/0x100
[ext4]
[ 409.480178] [<ffffffff8114304e>] __sync_filesystem+0x5e/0x90
[ 409.480178] [<ffffffff81143132>] sync_filesystem+0x32/0x60
[ 409.480178] [<ffffffff8111a97f>]
generic_shutdown_super+0x2f/0x100
[ 409.480178] [<ffffffff8111aa7c>] kill_block_super+0x2c/0x50
[ 409.480178] [<ffffffff8111b1e5>]
deactivate_locked_super+0x45/0x60
[ 409.480178] [<ffffffff8111b415>] deactivate_super+0x45/0x60
[ 409.480178] [<ffffffff81136430>] mntput_no_expire+0xf0/0x190
[ 409.480178] [<ffffffff811376a9>] sys_umount+0x79/0x3a0
[ 409.480178] [<ffffffff8100312b>]
system_call_fastpath+0x16/0x1b
[ 409.480178] [ 409.480178] other info that might help us debug this:
[ 409.480178]
[ 409.480178] 1 lock held by umount/4020:
[ 409.480178] #0: (&type->s_umount_key#23){++++..}, at:
[<ffffffff8111b40d>] deactivate_super
+0x3d/0x60
[ 409.480178]
[ 409.480178] stack backtrace:
[ 409.480178] Pid: 4020, comm: umount Not tainted 2.6.37-rc3+ #26
[ 409.480178] Call Trace:
[ 409.480178] [<ffffffff81082c39>] print_circular_bug+0xe9/0xf0
[ 409.480178] [<ffffffff81085122>] __lock_acquire+0x1382/0x1490
[ 409.480178] [<ffffffff810852c5>] lock_acquire+0x95/0x1b0
[ 409.480178] [<ffffffff81067a80>] ? flush_workqueue+0x0/0x540
[ 409.480178] [<ffffffff81067bc8>] flush_workqueue+0x148/0x540
[ 409.480178] [<ffffffff81067a80>] ? flush_workqueue+0x0/0x540
[ 409.480178] [<ffffffffa004f5db>] ext4_sync_fs+0x3b/0x100 [ext4]
[ 409.480178] [<ffffffff8113d68d>] ? writeback_inodes_sb+0x4d/0x60
[ 409.480178] [<ffffffff8114304e>] __sync_filesystem+0x5e/0x90
[ 409.480178] [<ffffffff81143132>] sync_filesystem+0x32/0x60
[ 409.480178] [<ffffffff8111a97f>] generic_shutdown_super+0x2f/0x100
[ 409.480178] [<ffffffff8111aa7c>] kill_block_super+0x2c/0x50
[ 409.480178] [<ffffffff8111b1e5>] deactivate_locked_super+0x45/0x60
[ 409.480178] [<ffffffff8111b415>] deactivate_super+0x45/0x60
[ 409.480178] [<ffffffff81136430>] mntput_no_expire+0xf0/0x190
[ 409.480178] [<ffffffff811376a9>] sys_umount+0x79/0x3a0
[ 409.480178] [<ffffffff8100312b>] system_call_fastpath+0x16/0x1b
next prev parent reply other threads:[~2010-11-23 8:15 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-16 11:00 [patch] fix up lock order reversal in writeback Nick Piggin
2010-11-16 13:01 ` Jan Kara
2010-11-17 4:30 ` Eric Sandeen
2010-11-17 4:38 ` Nick Piggin
2010-11-17 5:05 ` Eric Sandeen
2010-11-17 6:10 ` Nick Piggin
2010-11-18 3:06 ` Ted Ts'o
2010-11-18 3:29 ` Andrew Morton
2010-11-18 6:00 ` Nick Piggin
2010-11-18 6:28 ` Andrew Morton
2010-11-18 8:18 ` Nick Piggin
2010-11-18 10:51 ` Theodore Tso
2010-11-18 17:58 ` Andrew Morton
2010-11-19 5:10 ` Nick Piggin
2010-11-19 12:07 ` Theodore Tso
2010-11-18 14:55 ` Eric Sandeen
2010-11-18 17:10 ` Andrew Morton
2010-11-18 18:04 ` Eric Sandeen
2010-11-18 18:24 ` Eric Sandeen
2010-11-18 18:39 ` Chris Mason
2010-11-18 18:36 ` Andrew Morton
2010-11-18 18:51 ` Chris Mason
2010-11-18 20:22 ` Andrew Morton
2010-11-18 20:36 ` Chris Mason
2010-11-18 19:02 ` Eric Sandeen
2010-11-18 20:17 ` Andrew Morton
2010-11-18 18:33 ` Chris Mason
2010-11-18 23:58 ` Jan Kara
2010-11-19 0:45 ` Jan Kara
2010-11-19 5:16 ` Nick Piggin
2010-11-22 18:16 ` Jan Kara
2010-11-23 8:07 ` Nick Piggin
2010-11-23 13:32 ` Jan Kara
2010-11-23 8:15 ` Nick Piggin [this message]
2010-11-18 18:53 ` Al Viro
2010-11-18 3:18 ` Eric Sandeen
2010-11-22 23:43 ` Andrew Morton
2010-11-16 20:32 ` Andrew Morton
2010-11-17 3:56 ` Nick Piggin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101123081505.GB3364@amd \
--to=npiggin@kernel.dk \
--cc=akpm@linux-foundation.org \
--cc=jack@suse.cz \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=sandeen@redhat.com \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.