From: Qu Wenruo <quwenruo.btrfs@gmx.com>
To: Josef Bacik <josef@toxicpanda.com>,
linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH v3 03/54] btrfs: fix lockdep splat in btrfs_recover_relocation
Date: Thu, 3 Dec 2020 09:49:30 +0800 [thread overview]
Message-ID: <9168fd8d-4f22-0120-0810-b796ec3d0b69@gmx.com> (raw)
In-Reply-To: <6d8add2cb8a480cb2d7ea3763ea56c83e2aa3358.1606938211.git.josef@toxicpanda.com>
[-- Attachment #1.1: Type: text/plain, Size: 5151 bytes --]
On 2020/12/3 上午3:50, Josef Bacik wrote:
> While testing the error paths of relocation I hit the following lockdep
> splat
>
> ======================================================
> WARNING: possible circular locking dependency detected
> 5.10.0-rc6+ #217 Not tainted
> ------------------------------------------------------
> mount/779 is trying to acquire lock:
> ffffa0e676945418 (&fs_info->balance_mutex){+.+.}-{3:3}, at: btrfs_recover_balance+0x2f0/0x340
>
> but task is already holding lock:
> ffffa0e60ee31da8 (btrfs-root-00){++++}-{3:3}, at: __btrfs_tree_read_lock+0x27/0x100
>
> which lock already depends on the new lock.
>
> the existing dependency chain (in reverse order) is:
>
> -> #2 (btrfs-root-00){++++}-{3:3}:
> down_read_nested+0x43/0x130
> __btrfs_tree_read_lock+0x27/0x100
> btrfs_read_lock_root_node+0x31/0x40
> btrfs_search_slot+0x462/0x8f0
> btrfs_update_root+0x55/0x2b0
> btrfs_drop_snapshot+0x398/0x750
> clean_dirty_subvols+0xdf/0x120
> btrfs_recover_relocation+0x534/0x5a0
> btrfs_start_pre_rw_mount+0xcb/0x170
> open_ctree+0x151f/0x1726
> btrfs_mount_root.cold+0x12/0xea
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> vfs_kern_mount.part.0+0x71/0xb0
> btrfs_mount+0x10d/0x380
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> path_mount+0x433/0xc10
> __x64_sys_mount+0xe3/0x120
> do_syscall_64+0x33/0x40
> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> -> #1 (sb_internal#2){.+.+}-{0:0}:
> start_transaction+0x444/0x700
> insert_balance_item.isra.0+0x37/0x320
> btrfs_balance+0x354/0xf40
> btrfs_ioctl_balance+0x2cf/0x380
> __x64_sys_ioctl+0x83/0xb0
> do_syscall_64+0x33/0x40
> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> -> #0 (&fs_info->balance_mutex){+.+.}-{3:3}:
> __lock_acquire+0x1120/0x1e10
> lock_acquire+0x116/0x370
> __mutex_lock+0x7e/0x7b0
> btrfs_recover_balance+0x2f0/0x340
> open_ctree+0x1095/0x1726
> btrfs_mount_root.cold+0x12/0xea
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> vfs_kern_mount.part.0+0x71/0xb0
> btrfs_mount+0x10d/0x380
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> path_mount+0x433/0xc10
> __x64_sys_mount+0xe3/0x120
> do_syscall_64+0x33/0x40
> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> other info that might help us debug this:
>
> Chain exists of:
> &fs_info->balance_mutex --> sb_internal#2 --> btrfs-root-00
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(btrfs-root-00);
> lock(sb_internal#2);
> lock(btrfs-root-00);
> lock(&fs_info->balance_mutex);
>
> *** DEADLOCK ***
>
> 2 locks held by mount/779:
> #0: ffffa0e60dc040e0 (&type->s_umount_key#47/1){+.+.}-{3:3}, at: alloc_super+0xb5/0x380
> #1: ffffa0e60ee31da8 (btrfs-root-00){++++}-{3:3}, at: __btrfs_tree_read_lock+0x27/0x100
>
> stack backtrace:
> CPU: 0 PID: 779 Comm: mount Not tainted 5.10.0-rc6+ #217
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.13.0-2.fc32 04/01/2014
> Call Trace:
> dump_stack+0x8b/0xb0
> check_noncircular+0xcf/0xf0
> ? trace_call_bpf+0x139/0x260
> __lock_acquire+0x1120/0x1e10
> lock_acquire+0x116/0x370
> ? btrfs_recover_balance+0x2f0/0x340
> __mutex_lock+0x7e/0x7b0
> ? btrfs_recover_balance+0x2f0/0x340
> ? btrfs_recover_balance+0x2f0/0x340
> ? rcu_read_lock_sched_held+0x3f/0x80
> ? kmem_cache_alloc_trace+0x2c4/0x2f0
> ? btrfs_get_64+0x5e/0x100
> btrfs_recover_balance+0x2f0/0x340
> open_ctree+0x1095/0x1726
> btrfs_mount_root.cold+0x12/0xea
> ? rcu_read_lock_sched_held+0x3f/0x80
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> vfs_kern_mount.part.0+0x71/0xb0
> btrfs_mount+0x10d/0x380
> ? __kmalloc_track_caller+0x2f2/0x320
> legacy_get_tree+0x30/0x50
> vfs_get_tree+0x28/0xc0
> ? capable+0x3a/0x60
> path_mount+0x433/0xc10
> __x64_sys_mount+0xe3/0x120
> do_syscall_64+0x33/0x40
> entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> This is thankfully straightforward to fix, simply release the path
> before we setup the reloc_ctl.
>
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Thanks,
Qu
> ---
> fs/btrfs/volumes.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index 7930e1c78c45..49ba941f0314 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -4318,6 +4318,8 @@ int btrfs_recover_balance(struct btrfs_fs_info *fs_info)
> btrfs_warn(fs_info,
> "balance: cannot set exclusive op status, resume manually");
>
> + btrfs_release_path(path);
> +
> mutex_lock(&fs_info->balance_mutex);
> BUG_ON(fs_info->balance_ctl);
> spin_lock(&fs_info->balance_lock);
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2020-12-03 1:51 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-02 19:50 [PATCH v3 00/54] Cleanup error handling in relocation Josef Bacik
2020-12-02 19:50 ` [PATCH v3 01/54] btrfs: fix error handling in commit_fs_roots Josef Bacik
2020-12-03 1:45 ` Qu Wenruo
2020-12-03 8:09 ` Johannes Thumshirn
2020-12-02 19:50 ` [PATCH v3 02/54] btrfs: allow error injection for btrfs_search_slot and btrfs_cow_block Josef Bacik
2020-12-03 1:48 ` Qu Wenruo
2020-12-03 8:21 ` Johannes Thumshirn
2020-12-02 19:50 ` [PATCH v3 03/54] btrfs: fix lockdep splat in btrfs_recover_relocation Josef Bacik
2020-12-03 1:49 ` Qu Wenruo [this message]
2020-12-03 8:44 ` Johannes Thumshirn
2020-12-02 19:50 ` [PATCH v3 04/54] btrfs: keep track of the root owner for relocation reads Josef Bacik
2020-12-03 2:04 ` Qu Wenruo
2020-12-03 15:55 ` Josef Bacik
2020-12-02 19:50 ` [PATCH v3 05/54] btrfs: noinline btrfs_should_cancel_balance Josef Bacik
2020-12-03 2:06 ` Qu Wenruo
2020-12-03 8:44 ` Johannes Thumshirn
2020-12-03 9:00 ` Nikolay Borisov
2020-12-03 17:04 ` Josef Bacik
2020-12-02 19:50 ` [PATCH v3 06/54] btrfs: do not cleanup upper nodes in btrfs_backref_cleanup_node Josef Bacik
2020-12-03 2:08 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 07/54] btrfs: pass down the tree block level through ref-verify Josef Bacik
2020-12-02 19:50 ` [PATCH v3 08/54] btrfs: make sure owner is set in ref-verify Josef Bacik
2020-12-02 19:50 ` [PATCH v3 09/54] btrfs: don't clear ret in btrfs_start_dirty_block_groups Josef Bacik
2020-12-03 2:13 ` Qu Wenruo
2020-12-03 8:58 ` Johannes Thumshirn
2020-12-02 19:50 ` [PATCH v3 10/54] btrfs: convert some BUG_ON()'s to ASSERT()'s in do_relocation Josef Bacik
2020-12-03 2:14 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 11/54] btrfs: convert BUG_ON()'s in relocate_tree_block Josef Bacik
2020-12-03 2:15 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 12/54] btrfs: return an error from btrfs_record_root_in_trans Josef Bacik
2020-12-03 2:20 ` Qu Wenruo
2020-12-03 13:50 ` Johannes Thumshirn
2020-12-02 19:50 ` [PATCH v3 13/54] btrfs: handle errors from select_reloc_root() Josef Bacik
2020-12-03 2:23 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 14/54] btrfs: convert BUG_ON()'s in select_reloc_root() to proper errors Josef Bacik
2020-12-03 2:29 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 15/54] btrfs: check record_root_in_trans related failures in select_reloc_root Josef Bacik
2020-12-03 2:33 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 16/54] btrfs: do proper error handling in record_reloc_root_in_trans Josef Bacik
2020-12-03 2:39 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 17/54] btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename_exchange Josef Bacik
2020-12-03 2:40 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 18/54] btrfs: handle btrfs_record_root_in_trans failure in btrfs_rename Josef Bacik
2020-12-02 19:50 ` [PATCH v3 19/54] btrfs: handle btrfs_record_root_in_trans failure in btrfs_delete_subvolume Josef Bacik
2020-12-03 2:41 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 20/54] btrfs: handle btrfs_record_root_in_trans failure in btrfs_recover_log_trees Josef Bacik
2020-12-03 2:42 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 21/54] btrfs: handle btrfs_record_root_in_trans failure in create_subvol Josef Bacik
2020-12-03 2:43 ` Qu Wenruo
2020-12-03 16:06 ` Josef Bacik
2020-12-02 19:50 ` [PATCH v3 22/54] btrfs: btrfs: handle btrfs_record_root_in_trans failure in relocate_tree_block Josef Bacik
2020-12-03 2:44 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 23/54] btrfs: handle btrfs_record_root_in_trans failure in start_transaction Josef Bacik
2020-12-03 2:47 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 24/54] btrfs: handle record_root_in_trans failure in qgroup_account_snapshot Josef Bacik
2020-12-03 2:48 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 25/54] btrfs: handle record_root_in_trans failure in btrfs_record_root_in_trans Josef Bacik
2020-12-02 19:50 ` [PATCH v3 26/54] btrfs: handle record_root_in_trans failure in create_pending_snapshot Josef Bacik
2020-12-03 2:56 ` Qu Wenruo
2020-12-03 16:14 ` Josef Bacik
2020-12-02 19:50 ` [PATCH v3 27/54] btrfs: do not panic in __add_reloc_root Josef Bacik
2020-12-03 3:00 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 28/54] btrfs: have proper error handling in btrfs_init_reloc_root Josef Bacik
2020-12-02 19:50 ` [PATCH v3 29/54] btrfs: do proper error handling in create_reloc_root Josef Bacik
2020-12-03 3:29 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 30/54] btrfs: validate ->reloc_root after recording root in trans Josef Bacik
2020-12-03 4:49 ` Qu Wenruo
2020-12-03 16:18 ` Josef Bacik
2020-12-02 19:50 ` [PATCH v3 31/54] btrfs: handle btrfs_update_reloc_root failure in commit_fs_roots Josef Bacik
2020-12-03 4:51 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 32/54] btrfs: change insert_dirty_subvol to return errors Josef Bacik
2020-12-02 19:50 ` [PATCH v3 33/54] btrfs: handle btrfs_update_reloc_root failure in insert_dirty_subvol Josef Bacik
2020-12-02 19:50 ` [PATCH v3 34/54] btrfs: handle btrfs_update_reloc_root failure in prepare_to_merge Josef Bacik
2020-12-02 19:50 ` [PATCH v3 35/54] btrfs: do proper error handling in btrfs_update_reloc_root Josef Bacik
2020-12-03 4:54 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 36/54] btrfs: convert logic BUG_ON()'s in replace_path to ASSERT()'s Josef Bacik
2020-12-03 4:55 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 37/54] btrfs: handle initial btrfs_cow_block error in replace_path Josef Bacik
2020-12-03 5:05 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 38/54] btrfs: handle the loop " Josef Bacik
2020-12-03 5:11 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 39/54] btrfs: handle btrfs_search_slot failure " Josef Bacik
2020-12-03 5:13 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 40/54] btrfs: handle errors in reference count manipulation " Josef Bacik
2020-12-03 5:14 ` Qu Wenruo
2020-12-02 19:50 ` [PATCH v3 41/54] btrfs: handle extent reference errors in do_relocation Josef Bacik
2020-12-03 5:15 ` Qu Wenruo
2020-12-03 16:26 ` Josef Bacik
2020-12-02 19:51 ` [PATCH v3 42/54] btrfs: check for BTRFS_BLOCK_FLAG_FULL_BACKREF being set improperly Josef Bacik
2020-12-03 5:19 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 43/54] btrfs: remove the extent item sanity checks in relocate_block_group Josef Bacik
2020-12-03 5:20 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 44/54] btrfs: do proper error handling in create_reloc_inode Josef Bacik
2020-12-03 5:25 ` Qu Wenruo
2020-12-03 16:34 ` Josef Bacik
2020-12-02 19:51 ` [PATCH v3 45/54] btrfs: handle __add_reloc_root failure in btrfs_recover_relocation Josef Bacik
2020-12-03 5:32 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 46/54] btrfs: handle __add_reloc_root failure in btrfs_reloc_post_snapshot Josef Bacik
2020-12-03 5:34 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 47/54] btrfs: cleanup error handling in prepare_to_merge Josef Bacik
2020-12-03 5:39 ` Qu Wenruo
2020-12-03 16:53 ` Josef Bacik
2020-12-02 19:51 ` [PATCH v3 48/54] btrfs: handle extent corruption with select_one_root properly Josef Bacik
2020-12-03 5:40 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 49/54] btrfs: do proper error handling in merge_reloc_roots Josef Bacik
2020-12-03 5:42 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 50/54] btrfs: check return value of btrfs_commit_transaction in relocation Josef Bacik
2020-12-03 5:42 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 51/54] btrfs: do not WARN_ON() if we can't find the reloc root Josef Bacik
2020-12-02 19:51 ` [PATCH v3 52/54] btrfs: print the actual offset in btrfs_root_name Josef Bacik
2020-12-03 5:44 ` Qu Wenruo
2020-12-02 19:51 ` [PATCH v3 53/54] btrfs: fix reloc root leak with 0 ref reloc roots on recovery Josef Bacik
2020-12-02 19:51 ` [PATCH v3 54/54] btrfs: splice remaining dirty_bg's onto the transaction dirty bg list Josef Bacik
2020-12-03 5:47 ` Qu Wenruo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9168fd8d-4f22-0120-0810-b796ec3d0b69@gmx.com \
--to=quwenruo.btrfs@gmx.com \
--cc=josef@toxicpanda.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox