From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Qu Wenruo <wqu@suse.com>, Jeff Mahoney <jeffm@suse.com>,
David Sterba <dsterba@suse.com>, Sasha Levin <sashal@kernel.org>,
linux-btrfs@vger.kernel.org
Subject: [PATCH AUTOSEL 4.19 30/32] btrfs: qgroup: ensure qgroup_rescan_running is only set when the worker is at least queued
Date: Thu, 9 Apr 2020 23:50:03 -0400 [thread overview]
Message-ID: <20200410035005.9371-30-sashal@kernel.org> (raw)
In-Reply-To: <20200410035005.9371-1-sashal@kernel.org>
From: Qu Wenruo <wqu@suse.com>
[ Upstream commit d61acbbf54c612ea9bf67eed609494cda0857b3a ]
[BUG]
There are some reports about btrfs wait forever to unmount itself, with
the following call trace:
INFO: task umount:4631 blocked for more than 491 seconds.
Tainted: G X 5.3.8-2-default #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
umount D 0 4631 3337 0x00000000
Call Trace:
([<00000000174adf7a>] __schedule+0x342/0x748)
[<00000000174ae3ca>] schedule+0x4a/0xd8
[<00000000174b1f08>] schedule_timeout+0x218/0x420
[<00000000174af10c>] wait_for_common+0x104/0x1d8
[<000003ff804d6994>] btrfs_qgroup_wait_for_completion+0x84/0xb0 [btrfs]
[<000003ff8044a616>] close_ctree+0x4e/0x380 [btrfs]
[<0000000016fa3136>] generic_shutdown_super+0x8e/0x158
[<0000000016fa34d6>] kill_anon_super+0x26/0x40
[<000003ff8041ba88>] btrfs_kill_super+0x28/0xc8 [btrfs]
[<0000000016fa39f8>] deactivate_locked_super+0x68/0x98
[<0000000016fcb198>] cleanup_mnt+0xc0/0x140
[<0000000016d6a846>] task_work_run+0xc6/0x110
[<0000000016d04f76>] do_notify_resume+0xae/0xb8
[<00000000174b30ae>] system_call+0xe2/0x2c8
[CAUSE]
The problem happens when we have called qgroup_rescan_init(), but
not queued the worker. It can be caused mostly by error handling.
Qgroup ioctl thread | Unmount thread
----------------------------------------+-----------------------------------
|
btrfs_qgroup_rescan() |
|- qgroup_rescan_init() |
| |- qgroup_rescan_running = true; |
| |
|- trans = btrfs_join_transaction() |
| Some error happened |
| |
|- btrfs_qgroup_rescan() returns error |
But qgroup_rescan_running == true; |
| close_ctree()
| |- btrfs_qgroup_wait_for_completion()
| |- running == true;
| |- wait_for_completion();
btrfs_qgroup_rescan_worker is never queued, thus no one is going to wake
up close_ctree() and we get a deadlock.
All involved qgroup_rescan_init() callers are:
- btrfs_qgroup_rescan()
The example above. It's possible to trigger the deadlock when error
happened.
- btrfs_quota_enable()
Not possible. Just after qgroup_rescan_init() we queue the work.
- btrfs_read_qgroup_config()
It's possible to trigger the deadlock. It only init the work, the
work queueing happens in btrfs_qgroup_rescan_resume().
Thus if error happened in between, deadlock is possible.
We shouldn't set fs_info->qgroup_rescan_running just in
qgroup_rescan_init(), as at that stage we haven't yet queued qgroup
rescan worker to run.
[FIX]
Set qgroup_rescan_running before queueing the work, so that we ensure
the rescan work is queued when we wait for it.
Fixes: 8d9eddad1946 ("Btrfs: fix qgroup rescan worker initialization")
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ Change subject and cause analyse, use a smaller fix ]
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
fs/btrfs/qgroup.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 0cd043f03081e..cbd40826f5dcf 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -1032,6 +1032,7 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info)
ret = qgroup_rescan_init(fs_info, 0, 1);
if (!ret) {
qgroup_rescan_zero_tracking(fs_info);
+ fs_info->qgroup_rescan_running = true;
btrfs_queue_work(fs_info->qgroup_rescan_workers,
&fs_info->qgroup_rescan_work);
}
@@ -2906,7 +2907,6 @@ qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 progress_objectid,
sizeof(fs_info->qgroup_rescan_progress));
fs_info->qgroup_rescan_progress.objectid = progress_objectid;
init_completion(&fs_info->qgroup_rescan_completion);
- fs_info->qgroup_rescan_running = true;
spin_unlock(&fs_info->qgroup_lock);
mutex_unlock(&fs_info->qgroup_rescan_lock);
@@ -2972,8 +2972,11 @@ btrfs_qgroup_rescan(struct btrfs_fs_info *fs_info)
qgroup_rescan_zero_tracking(fs_info);
+ mutex_lock(&fs_info->qgroup_rescan_lock);
+ fs_info->qgroup_rescan_running = true;
btrfs_queue_work(fs_info->qgroup_rescan_workers,
&fs_info->qgroup_rescan_work);
+ mutex_unlock(&fs_info->qgroup_rescan_lock);
return 0;
}
@@ -3009,9 +3012,13 @@ int btrfs_qgroup_wait_for_completion(struct btrfs_fs_info *fs_info,
void
btrfs_qgroup_rescan_resume(struct btrfs_fs_info *fs_info)
{
- if (fs_info->qgroup_flags & BTRFS_QGROUP_STATUS_FLAG_RESCAN)
+ if (fs_info->qgroup_flags & BTRFS_QGROUP_STATUS_FLAG_RESCAN) {
+ mutex_lock(&fs_info->qgroup_rescan_lock);
+ fs_info->qgroup_rescan_running = true;
btrfs_queue_work(fs_info->qgroup_rescan_workers,
&fs_info->qgroup_rescan_work);
+ mutex_unlock(&fs_info->qgroup_rescan_lock);
+ }
}
/*
--
2.20.1
next prev parent reply other threads:[~2020-04-10 3:53 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-10 3:49 [PATCH AUTOSEL 4.19 01/32] cpufreq: imx6q: Fixes unwanted cpu overclocking on i.MX6ULL Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 02/32] media: venus: hfi_parser: Ignore HEVC encoding for V1 Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 03/32] firmware: arm_sdei: fix double-lock on hibernate with shared events Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 04/32] null_blk: Fix the null_add_dev() error path Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 05/32] null_blk: Handle null_add_dev() failures properly Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 06/32] null_blk: fix spurious IO errors after failed past-wp access Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 07/32] xhci: bail out early if driver can't accress host in resume Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 08/32] x86: Don't let pgprot_modify() change the page encryption bit Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 09/32] block: keep bdi->io_pages in sync with max_sectors_kb for stacked devices Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 10/32] irqchip/versatile-fpga: Handle chained IRQs properly Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 11/32] sched: Avoid scale real weight down to zero Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 12/32] selftests/x86/ptrace_syscall_32: Fix no-vDSO segfault Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 13/32] PCI/switchtec: Fix init_completion race condition with poll_wait() Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 14/32] media: i2c: video-i2c: fix build errors due to 'imply hwmon' Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 15/32] libata: Remove extra scsi_host_put() in ata_scsi_add_hosts() Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 16/32] pstore/platform: fix potential mem leak if pstore_init_fs failed Sasha Levin
2020-04-10 3:49 ` [Cluster-devel] [PATCH AUTOSEL 4.19 17/32] gfs2: Don't demote a glock until its revokes are written Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 18/32] x86/boot: Use unsigned comparison for addresses Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 19/32] efi/x86: Ignore the memory attributes table on i386 Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 20/32] genirq/irqdomain: Check pointer in irq_domain_alloc_irqs_hierarchy() Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 21/32] block: Fix use-after-free issue accessing struct io_cq Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 22/32] media: i2c: ov5695: Fix power on and off sequences Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 23/32] usb: dwc3: core: add support for disabling SS instances in park mode Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 24/32] irqchip/gic-v4: Provide irq_retrigger to avoid circular locking dependency Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 25/32] md: check arrays is suspended in mddev_detach before call quiesce operations Sasha Levin
2020-04-10 3:49 ` [PATCH AUTOSEL 4.19 26/32] firmware: fix a double abort case with fw_load_sysfs_fallback Sasha Levin
2020-04-10 3:50 ` [PATCH AUTOSEL 4.19 27/32] locking/lockdep: Avoid recursion in lockdep_count_{for,back}ward_deps() Sasha Levin
2020-04-10 3:50 ` [PATCH AUTOSEL 4.19 28/32] block, bfq: fix use-after-free in bfq_idle_slice_timer_body Sasha Levin
2020-04-10 3:50 ` [PATCH AUTOSEL 4.19 29/32] btrfs: hold a ref on the root in btrfs_recover_relocation Sasha Levin
2020-04-10 3:50 ` Sasha Levin [this message]
2020-04-10 3:50 ` [PATCH AUTOSEL 4.19 31/32] btrfs: remove a BUG_ON() from merge_reloc_roots() Sasha Levin
2020-04-10 3:50 ` [PATCH AUTOSEL 4.19 32/32] btrfs: track reloc roots based on their commit root bytenr Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200410035005.9371-30-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=dsterba@suse.com \
--cc=jeffm@suse.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
--cc=wqu@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.