[PATCH 0/2] btrfs: qgroup: mark qgroup inconsistent for more

[PATCH 0/2] btrfs: qgroup: mark qgroup inconsistent for more

[Qu Wenruo]

Commit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid
low stall in btrfs_commit_transaction()") covers the most common case
that can cause a long qgroup stall, where a large subtree is rescanned
when dropping a snapshot.

But there are more corner cases for the same long qgroup stall:

- A qgroup rescan is queued immediately after marking inconsistent
  The new rescan will immediately revert the NO_ACCOUNTING flag, thus
  still results long qgroup stalls.

- A subvolume with no shared subtree
  This can still queue a lot of workload into the current transaction.

This series covers both less-common cases.

Qu Wenruo (2):
  btrfs: reject new qgroup rescan during subvolume dropping
  btrfs: avoid long stall when dropping a non-shared large subvolume

 fs/btrfs/disk-io.c     |  4 ++++
 fs/btrfs/extent-tree.c | 11 +++++++++++
 fs/btrfs/qgroup.c      | 29 ++++++++++++++++++++++++++++-
 fs/btrfs/qgroup.h      | 15 +++++++++++++++
 4 files changed, 58 insertions(+), 1 deletion(-)

-- 
2.55.0

[PATCH 1/2] btrfs: reject new qgroup rescan during subvolume dropping

[Qu Wenruo]

Commit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid
low stall in btrfs_commit_transaction()") introduced a threshold to skip
huge subtree scan during subvolume dropping.

But that's not covering all cases, e.g. rescan can still be started
immediately after that huge subtree skipping.
This will cause rescan to do the same accounting for that subtree
anyway, still causing a long stall during transaction commit.

Introduce a new runtime qgroup flag,
BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN, so that during cleanup of a
subvolume, no new qgroup rescan can be initiated.

The rejection uses the same -EINPROGRESS, as if there is already a
running qgroup rescan.

And since we have the extra bit, we can no longer allow plain assignment
in btrfs_quota_enable(), as the plain assignment will override the
REJECT_RESCAN bit.
To co-operate this new flag:

- Make btrfs_quota_enable() to only set BTRFS_QGROUP_STATUS_BIT_ON
  So it won't override the existing
  BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN bit.

- Make btrfs_quota_disable() to clear every non-rescan bit
  This includes:
  * BTRFS_QGROUP_STATUS_BIT_ON
  * BTRFS_QGROUP_STATUS_BIT_INCONSISTENT
  * BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING

  For rescan related bits, they are either cleared by the rescan thread,
  or by the caller who rejects rescan.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/disk-io.c |  4 ++++
 fs/btrfs/qgroup.c  | 10 +++++++++-
 fs/btrfs/qgroup.h  | 13 +++++++++++++
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index e6bbb0b1b38c..dd00e8ced883 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1496,7 +1496,11 @@ static int cleaner_kthread(void *arg)
 
 		btrfs_run_delayed_iputs(fs_info);
 
+		set_bit(BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN,
+			&fs_info->qgroup_flags);
 		again = btrfs_clean_one_deleted_snapshot(fs_info);
+		clear_bit(BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN,
+			  &fs_info->qgroup_flags);
 		mutex_unlock(&fs_info->cleaner_mutex);
 
 		/*
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index 2f2f4e0a3b2e..cc08e7b2f68f 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -1105,7 +1105,7 @@ int btrfs_quota_enable(struct btrfs_fs_info *fs_info,
 				 struct btrfs_qgroup_status_item);
 	btrfs_set_qgroup_status_generation(leaf, ptr, trans->transid);
 	btrfs_set_qgroup_status_version(leaf, ptr, BTRFS_QGROUP_STATUS_VERSION);
-	fs_info->qgroup_flags = (1UL << BTRFS_QGROUP_STATUS_BIT_ON);
+	set_bit(BTRFS_QGROUP_STATUS_BIT_ON, &fs_info->qgroup_flags);
 	if (simple) {
 		set_bit(BTRFS_QGROUP_STATUS_BIT_SIMPLE_MODE, &fs_info->qgroup_flags);
 		btrfs_set_fs_incompat(fs_info, SIMPLE_QUOTA);
@@ -1407,8 +1407,14 @@ int btrfs_quota_disable(struct btrfs_fs_info *fs_info)
 	spin_lock(&fs_info->qgroup_lock);
 	quota_root = fs_info->quota_root;
 	fs_info->quota_root = NULL;
+	/*
+	 * Clear all on-disk and runtime bits, except RESCAN related ones, that
+	 * are either handled by rescan thread, or the caller who rejects rescan.
+	 */
 	clear_bit(BTRFS_QGROUP_STATUS_BIT_ON, &fs_info->qgroup_flags);
 	clear_bit(BTRFS_QGROUP_STATUS_BIT_SIMPLE_MODE, &fs_info->qgroup_flags);
+	clear_bit(BTRFS_QGROUP_STATUS_BIT_INCONSISTENT, &fs_info->qgroup_flags);
+	clear_bit(BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING, &fs_info->qgroup_flags);
 	fs_info->qgroup_drop_subtree_thres = BTRFS_QGROUP_DROP_SUBTREE_THRES_DEFAULT;
 	spin_unlock(&fs_info->qgroup_lock);
 
@@ -3997,6 +4003,8 @@ qgroup_rescan_init(struct btrfs_fs_info *fs_info, u64 progress_objectid,
 
 	if (init_flags) {
 		if (test_bit(BTRFS_QGROUP_STATUS_BIT_RESCAN,
+			     &fs_info->qgroup_flags) ||
+		    test_bit(BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN,
 			     &fs_info->qgroup_flags)) {
 			ret = -EINPROGRESS;
 		} else if (!test_bit(BTRFS_QGROUP_STATUS_BIT_ON,
diff --git a/fs/btrfs/qgroup.h b/fs/btrfs/qgroup.h
index b3aaad5e617d..7cc1a079a0b4 100644
--- a/fs/btrfs/qgroup.h
+++ b/fs/btrfs/qgroup.h
@@ -124,6 +124,19 @@ struct btrfs_qgroup_swapped_blocks;
 #define BTRFS_QGROUP_RUNTIME_BIT_CANCEL_RESCAN		(BITS_PER_LONG - 1)
 #define BTRFS_QGROUP_RUNTIME_BIT_NO_ACCOUNTING		(BITS_PER_LONG - 2)
 
+/*
+ * No new rescan allowed when set.
+ *
+ * During huge subtree dropping, qgroup will be marked inconsistent,
+ * and skip all future accounting to avoid long stall.
+ * But an immediate rescan will re-enable qgroup and still stall
+ * the system.
+ *
+ * This bit is to avoid such rescan during the duration of a subvolume
+ * dropping.
+ */
+#define BTRFS_QGROUP_RUNTIME_BIT_REJECT_RESCAN		(BITS_PER_LONG - 3)
+
 #define BTRFS_QGROUP_DROP_SUBTREE_THRES_DEFAULT		(3)
 
 /*
-- 
2.55.0

[PATCH 2/2] btrfs: avoid long stall when dropping a non-shared large subvolume

[Qu Wenruo]

Commit 011b46c30476 ("btrfs: skip subtree scan if it's too high to avoid
low stall in btrfs_commit_transaction()") introduced a mechanism to skip
large subtree during snapshot dropping.

But even for a subvolume without any shared tree blocks, we can still
queue quite a lot of qgroup records into one transaction, and cause a
long qgroup related stall.

So also add a check against the subvolume root level, to determine if we
need to mark qgroup inconsistent.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/extent-tree.c | 11 +++++++++++
 fs/btrfs/qgroup.c      | 19 +++++++++++++++++++
 fs/btrfs/qgroup.h      |  2 ++
 3 files changed, 32 insertions(+)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index d6a4390ee34a..8977e9ad0d44 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -6315,6 +6315,17 @@ int btrfs_drop_snapshot(struct btrfs_root *root, bool update_ref, bool for_reloc
 	set_bit(BTRFS_ROOT_DELETING, &root->state);
 	unfinished_drop = test_bit(BTRFS_ROOT_UNFINISHED_DROP, &root->state);
 
+	/*
+	 * For subvolume dropping, check if the subvolume is large enough
+	 * so that we need to mark qgroup inconsistent to avoid long qgroup
+	 * stall.
+	 *
+	 * Even for a subvolume without any snapshot, there can still be
+	 * a lot of qgroup records queued into one transaction.
+	 */
+	if (!for_reloc)
+		btrfs_qgroup_check_tree_drop(fs_info, rootid,
+					     btrfs_header_level(root->node));
 	if (btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
 		level = btrfs_header_level(root->node);
 		path->nodes[level] = btrfs_lock_root_node(root);
diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c
index cc08e7b2f68f..cefeedcd6826 100644
--- a/fs/btrfs/qgroup.c
+++ b/fs/btrfs/qgroup.c
@@ -2751,6 +2751,25 @@ int btrfs_qgroup_trace_subtree(struct btrfs_trans_handle *trans,
 	return 0;
 }
 
+void btrfs_qgroup_check_tree_drop(struct btrfs_fs_info *fs_info,
+				  u64 rootid, u8 level)
+{
+	u8 drop_subtree_thres;
+
+	if (btrfs_qgroup_mode(fs_info) != BTRFS_QGROUP_MODE_FULL)
+		return;
+
+	if (!btrfs_is_fstree(rootid))
+		return;
+
+	spin_lock(&fs_info->qgroup_lock);
+	drop_subtree_thres = fs_info->qgroup_drop_subtree_thres;
+	spin_unlock(&fs_info->qgroup_lock);
+
+	if (level >= drop_subtree_thres)
+		qgroup_mark_inconsistent(fs_info, "subtree level reached threshold");
+}
+
 static void qgroup_iterator_nested_add(struct list_head *head, struct btrfs_qgroup *qgroup)
 {
 	if (!list_empty(&qgroup->nested_iterator))
diff --git a/fs/btrfs/qgroup.h b/fs/btrfs/qgroup.h
index 7cc1a079a0b4..c8db380f1576 100644
--- a/fs/btrfs/qgroup.h
+++ b/fs/btrfs/qgroup.h
@@ -378,6 +378,8 @@ int btrfs_qgroup_trace_leaf_items(struct btrfs_trans_handle *trans,
 int btrfs_qgroup_trace_subtree(struct btrfs_trans_handle *trans,
 			       struct extent_buffer *root_eb,
 			       u64 root_gen, int root_level);
+void btrfs_qgroup_check_tree_drop(struct btrfs_fs_info *fs_info,
+				  u64 rootid, u8 level);
 int btrfs_qgroup_account_extent(struct btrfs_trans_handle *trans, u64 bytenr,
 				u64 num_bytes, struct ulist *old_roots,
 				struct ulist *new_roots);
-- 
2.55.0