[PATCH 0/4] Lowmem fsck false alert fixes

[PATCH 0/4] Lowmem fsck false alert fixes

[Qu Wenruo]

Btrfs-progs test case 023 will cause assert and a lot of false alerts
for lowmem mode.

The problems are caused by several reasons, from bad handler for tree
reloc root(calling btrfs_read_fs_root on tree reloc tree) to too
restrict check.

Fix the lowmem mode bugs.

There is another bug which affects both original mode and lowmem mode,
it seems to be caused by this commit:
commit 00e769d04c2c83029d6c71fbded133597d93ad55
Author: Goldwyn Rodrigues <rgoldwyn@suse.com>
Date:   Tue Nov 29 10:24:52 2016 -0600

    btrfs-progs: Correct value printed by assertions/BUG_ON/WARN_ON

Informed Goldwyn to fix it.
So the fix for the common assert is not included in this patchset.

Qu Wenruo (4):
  btrfs-progs: check: Fix assert when using lowmem on fs with tree reloc
    tree
  btrfs-progs: check: Fix lowmem mode stack overflow caused by fsck/023
  btrfs-progs: check: Fix lowmem false alert on tree reloc tree
  btrfs-progs: check: Fix false alert on generation mismatch for tree
    reloc tree

 cmds-check.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 56 insertions(+), 15 deletions(-)

-- 
2.10.2

[PATCH 1/4] btrfs-progs: check: Fix assert when using lowmem on fs with tree reloc tree

[Qu Wenruo]

When using lowmem mode, btrfs check will report ASSERT for calling
btrfs_read_fs_root() on tree reloc tree.

Fix it by checking objectid and call btrfs_read_fs_root_no_cache() for
tree reloc tree.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 cmds-check.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/cmds-check.c b/cmds-check.c
index 30eabb2..30847a0 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -11411,7 +11411,11 @@ static int check_chunks_and_extents_v2(struct btrfs_root *root)
 			goto next;
 		key.offset = (u64)-1;
 
-		cur_root = btrfs_read_fs_root(root->fs_info, &key);
+		if (key.objectid == BTRFS_TREE_RELOC_OBJECTID)
+			cur_root = btrfs_read_fs_root_no_cache(root->fs_info,
+					&key);
+		else
+			cur_root = btrfs_read_fs_root(root->fs_info, &key);
 		if (IS_ERR(cur_root) || !cur_root) {
 			error("failed to read tree: %lld", key.objectid);
 			goto next;
@@ -11420,6 +11424,8 @@ static int check_chunks_and_extents_v2(struct btrfs_root *root)
 		ret = traverse_tree_block(cur_root, cur_root->node);
 		err |= ret;
 
+		if (key.objectid == BTRFS_TREE_RELOC_OBJECTID)
+			btrfs_free_fs_root(cur_root);
 next:
 		ret = btrfs_next_item(root1, &path);
 		if (ret)
-- 
2.10.2

[PATCH 2/4] btrfs-progs: check: Fix lowmem mode stack overflow caused by fsck/023

[Qu Wenruo]

Lowmem mode fsck will overflow its stack since it will do infinite
backref check for tree reloc root.
We should not check backref if it's pointing to itself for tree reloc
root.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 cmds-check.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index 30847a0..ef90d87 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -9999,10 +9999,15 @@ static int check_tree_block_ref(struct btrfs_root *root,
 	u32 nodesize = root->nodesize;
 	u32 item_size;
 	u64 offset;
+	int tree_reloc_root = 0;
 	int found_ref = 0;
 	int err = 0;
 	int ret;
 
+	if (root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID &&
+	    btrfs_header_bytenr(root->node) == bytenr)
+		tree_reloc_root = 1;
+
 	btrfs_init_path(&path);
 	key.objectid = bytenr;
 	if (btrfs_fs_incompat(root->fs_info, SKINNY_METADATA))
@@ -10090,9 +10095,16 @@ static int check_tree_block_ref(struct btrfs_root *root,
 			(offset == root->objectid || offset == owner)) {
 			found_ref = 1;
 		} else if (type == BTRFS_SHARED_BLOCK_REF_KEY) {
+			/*
+			 * Backref of tree reloc root points to itself, no need
+			 * to check backref any more.
+			 */
+			if (tree_reloc_root)
+				found_ref = 1;
+			else
 			/* Check if the backref points to valid referencer */
-			found_ref = !check_tree_block_ref(root, NULL, offset,
-							  level + 1, owner);
+				found_ref = !check_tree_block_ref(root, NULL,
+						offset, level + 1, owner);
 		}
 
 		if (found_ref)
-- 
2.10.2

[PATCH 3/4] btrfs-progs: check: Fix lowmem false alert on tree reloc tree

[Qu Wenruo]

Lowmem mode will report false alert if the fs has tree reloc tree like:
ERROR: shared extent[30011392 4096] lost its parent (parent: 30011392,
level: 1)

The problem is check_shared_block_backref() can't handle tree reloc
tree's self-pointing backref.

And still try to read out the tree block then seeking for the
referencer.

The correct method for it is to check if it's tree reloc root.
In that case, we should check found the ROOT_ITEM of tree reloc tree in
root tree.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 cmds-check.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index ef90d87..d0e1977 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -10465,6 +10465,34 @@ out:
 }
 
 /*
+ * Check if tree block @eb is tree reloc root.
+ * Return 0 if it's not or any problem happens
+ * Return 1 if it's a tree reloc root
+ */
+static int is_tree_reloc_root(struct btrfs_fs_info *fs_info,
+				 struct extent_buffer *eb)
+{
+	struct btrfs_root *tree_reloc_root;
+	struct btrfs_key key;
+	u64 bytenr = btrfs_header_bytenr(eb);
+	u64 owner = btrfs_header_owner(eb);
+	int ret = 0;
+
+	key.objectid = BTRFS_TREE_RELOC_OBJECTID;
+	key.offset = owner;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+
+	tree_reloc_root = btrfs_read_fs_root_no_cache(fs_info, &key);
+	if (IS_ERR(tree_reloc_root))
+		return 0;
+
+	if (bytenr == btrfs_header_bytenr(tree_reloc_root->node))
+		ret = 1;
+	btrfs_free_fs_root(tree_reloc_root);
+	return ret;
+}
+
+/*
  * Check referencer for shared block backref
  * If level == -1, this function will resolve the level.
  */
@@ -10486,6 +10514,13 @@ static int check_shared_block_backref(struct btrfs_fs_info *fs_info,
 	if (level < 0)
 		goto out;
 
+	/* It's possible it's a tree reloc root */
+	if (parent == bytenr) {
+		if (is_tree_reloc_root(fs_info, eb))
+			found_parent = 1;
+		goto out;
+	}
+
 	if (level + 1 != btrfs_header_level(eb))
 		goto out;
 
-- 
2.10.2

[PATCH 4/4] btrfs-progs: check: Fix false alert on generation mismatch for tree reloc tree

[Qu Wenruo]

For fs with tree reloc tree(under balancing), lowmem mode will report
false alert like:
ERROR: extent[62914560 4096] backref generation mismatch, wanted: <=9,
have: 13

This is because lowmem mode adds a more restrict check, to ensure
generation in fs tree won't be smaller than extent tree.

In fact such assumption is not right for tree reloc tree, so remove such
check.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 cmds-check.c | 12 ------------
 1 file changed, 12 deletions(-)

diff --git a/cmds-check.c b/cmds-check.c
index d0e1977..c5f6f70 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -10155,12 +10155,10 @@ static int check_extent_data_item(struct btrfs_root *root,
 	struct btrfs_extent_inline_ref *iref;
 	struct btrfs_extent_data_ref *dref;
 	u64 owner;
-	u64 file_extent_gen;
 	u64 disk_bytenr;
 	u64 disk_num_bytes;
 	u64 extent_num_bytes;
 	u64 extent_flags;
-	u64 extent_gen;
 	u32 item_size;
 	unsigned long end;
 	unsigned long ptr;
@@ -10172,7 +10170,6 @@ static int check_extent_data_item(struct btrfs_root *root,
 
 	btrfs_item_key_to_cpu(eb, &fi_key, slot);
 	fi = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
-	file_extent_gen = btrfs_file_extent_generation(eb, fi);
 
 	/* Nothing to check for hole and inline data extents */
 	if (btrfs_file_extent_type(eb, fi) == BTRFS_FILE_EXTENT_INLINE ||
@@ -10221,7 +10218,6 @@ static int check_extent_data_item(struct btrfs_root *root,
 	ei = btrfs_item_ptr(leaf, slot, struct btrfs_extent_item);
 
 	extent_flags = btrfs_extent_flags(leaf, ei);
-	extent_gen = btrfs_extent_generation(leaf, ei);
 
 	if (!(extent_flags & BTRFS_EXTENT_FLAG_DATA)) {
 		error(
@@ -10231,14 +10227,6 @@ static int check_extent_data_item(struct btrfs_root *root,
 		err |= BACKREF_MISMATCH;
 	}
 
-	if (file_extent_gen < extent_gen) {
-		error(
-"extent[%llu %llu] backref generation mismatch, wanted: <=%llu, have: %llu",
-			disk_bytenr, disk_num_bytes, file_extent_gen,
-			extent_gen);
-		err |= BACKREF_MISMATCH;
-	}
-
 	/* Check data backref inside that extent item */
 	item_size = btrfs_item_size_nr(leaf, path.slots[0]);
 	iref = (struct btrfs_extent_inline_ref *)(ei + 1);
-- 
2.10.2

Re: [PATCH 0/4] Lowmem fsck false alert fixes

[David Sterba]

On Mon, Dec 05, 2016 at 05:07:52PM +0800, Qu Wenruo wrote:
> Btrfs-progs test case 023 will cause assert and a lot of false alerts
> for lowmem mode.
> 
> The problems are caused by several reasons, from bad handler for tree
> reloc root(calling btrfs_read_fs_root on tree reloc tree) to too
> restrict check.
> 
> Fix the lowmem mode bugs.
> 
> There is another bug which affects both original mode and lowmem mode,
> it seems to be caused by this commit:
> commit 00e769d04c2c83029d6c71fbded133597d93ad55
> Author: Goldwyn Rodrigues <rgoldwyn@suse.com>
> Date:   Tue Nov 29 10:24:52 2016 -0600
> 
>     btrfs-progs: Correct value printed by assertions/BUG_ON/WARN_ON
> 
> Informed Goldwyn to fix it.
> So the fix for the common assert is not included in this patchset.
> 
> Qu Wenruo (4):
>   btrfs-progs: check: Fix assert when using lowmem on fs with tree reloc
>     tree
>   btrfs-progs: check: Fix lowmem mode stack overflow caused by fsck/023
>   btrfs-progs: check: Fix lowmem false alert on tree reloc tree
>   btrfs-progs: check: Fix false alert on generation mismatch for tree
>     reloc tree

1-4 applied, thanks.