[PATCH 0/5] btrfs-find-root enhancement for chunk tree corrupted fs

[PATCH 0/5] btrfs-find-root enhancement for chunk tree corrupted fs

[Qu Wenruo]

Before this patchset, btrfs-find-root needs valid chunk tree from the
fs.
However for chunk root corrupted case, btrfs-find-root is of no use due
to above limitation.

This patchset will allow open_ctree_fs_info() to return a fs_info
without any valid tree root, but system chunk map from superblock only.
And modify btrfs-find-root along with some infrastructure to do chunk
root search.

Also fix an old bug where btrfs-find-root will always skip the first
chunk, with its corresponding regression test.

This also provides the basis for later "btrfsck --chunk-root" and faster
chunk-recovery enhancement.

Qu Wenruo (5):
  btrfs: volume: Fix a bug causing btrfs-find-root to skip first chunk
  btrfs: Allow open_ctree to return fs_info even chunk tree is corrupted
  btrfs: Add support for tree block operations on fs_info without roots.
  btrfs: find-root: Allow btrfs-find-root to search chunk root even
    chunk     root is corrupted
  btrfs: misc-test: Add regression test for find-root gives empty result

 btrfs-corrupt-block.c                              |   2 +-
 btrfs-find-root.c                                  |  17 ++--
 ctree.h                                            |   1 +
 disk-io.c                                          |  99 +++++++++++++--------
 disk-io.h                                          |  35 ++++++--
 extent-tree.c                                      |   3 +-
 find-root.c                                        |  10 +--
 find-root.h                                        |   2 +-
 .../first_meta_chunk.btrfs-image                   | Bin 0 -> 4096 bytes
 tests/misc-tests/012-find-root-no-result/test.sh   |  20 +++++
 volumes.c                                          |  20 +++--
 11 files changed, 147 insertions(+), 62 deletions(-)
 create mode 100644 tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image
 create mode 100644 tests/misc-tests/012-find-root-no-result/test.sh

-- 
2.7.1

[PATCH 1/5] btrfs-progs: volume: Fix a bug causing btrfs-find-root to skip first chunk

[Qu Wenruo]

There is a small bug from 2011, where btrfs_next_bg (formally
btrfs_next_metadata) function will always skip the first chunk.

That's OK for that time, as there is always 3 empty temporary chunks.
But now, we may ended up with only one metadata or system chunk, with
empty chunk auto-remove from kernel or new mkfs.btrfs.

So fix it by checking the initial value so btrfs_next_bg() will return
the first chunk if its *logical parameter is 0.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 volumes.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/volumes.c b/volumes.c
index a94be0e..059c4f4 100644
--- a/volumes.c
+++ b/volumes.c
@@ -1170,14 +1170,23 @@ int btrfs_next_bg(struct btrfs_mapping_tree *map_tree, u64 *logical,
 {
 	struct cache_extent *ce;
 	struct map_lookup *map;
+	u64 cur = *logical;
 
-	ce = search_cache_extent(&map_tree->cache_tree, *logical);
+	ce = search_cache_extent(&map_tree->cache_tree, cur);
 
 	while (ce) {
-		ce = next_cache_extent(ce);
-		if (!ce)
-			return -ENOENT;
+		/*
+		 * only jump to next bg if our cur is not 0
+		 * As the initial logical for btrfs_next_bg() is 0, and
+		 * if we jump to next bg, we skipped a valid bg.
+		 */
+		if (cur) {
+			ce = next_cache_extent(ce);
+			if (!ce)
+				return -ENOENT;
+		}
 
+		cur = ce->start;
 		map = container_of(ce, struct map_lookup, ce);
 		if (map->type & type) {
 			*logical = ce->start;
-- 
2.7.1

[PATCH 2/5] btrfs-progs: Allow open_ctree to return fs_info even chunk tree is corrupted

[Qu Wenruo]

Current open_ctree_fs_info() won't return anything if chunk tree root is
corrupted.
This makes some function, like btrfs-find-root unable to find any older
chunk tree root, even it is possible to use system_chunk_array in super
block.

And at least two users in mail list has reported such heavily chunk
corruption.
Although we have 'btrfs rescue chunk-recovery' but it's too time
consuming and sometimes buggy.

This patch adds a new open ctree flag,
OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR, allowing fs_info to be returned from
open_ctree_fs_info() even there is no valid tree root in it.

Also adds a new close_ctree() variant, close_ctree_fs_info() to handle
possible fs_info without any root.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 ctree.h   |  1 +
 disk-io.c | 29 ++++++++++++++++++++++++-----
 disk-io.h | 18 ++++++++++++++++--
 3 files changed, 41 insertions(+), 7 deletions(-)

diff --git a/ctree.h b/ctree.h
index 21b0445..5ab0f4a 100644
--- a/ctree.h
+++ b/ctree.h
@@ -1029,6 +1029,7 @@ struct btrfs_fs_info {
 	unsigned int quota_enabled:1;
 	unsigned int suppress_check_block_errors:1;
 	unsigned int ignore_fsid_mismatch:1;
+	unsigned int ignore_chunk_tree_error:1;
 
 	int (*free_extent_hook)(struct btrfs_trans_handle *trans,
 				struct btrfs_root *root,
diff --git a/disk-io.c b/disk-io.c
index bd0444b..3b5a08e 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -1168,8 +1168,14 @@ int btrfs_setup_chunk_tree_and_device_map(struct btrfs_fs_info *fs_info)
 						    btrfs_super_chunk_root(sb),
 						    blocksize, generation);
 	if (!extent_buffer_uptodate(fs_info->chunk_root->node)) {
-		fprintf(stderr, "Couldn't read chunk root\n");
-		return -EIO;
+		if (fs_info->ignore_chunk_tree_error) {
+			error("Couldn't read chunk root, continue anyway");
+			fs_info->chunk_root = NULL;
+			return 0;
+		} else {
+			error("Couldn't read chunk rootn");
+			return -EIO;
+		}
 	}
 
 	if (!(btrfs_super_flags(sb) & BTRFS_SUPER_FLAG_METADUMP)) {
@@ -1212,6 +1218,8 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 		fs_info->suppress_check_block_errors = 1;
 	if (flags & OPEN_CTREE_IGNORE_FSID_MISMATCH)
 		fs_info->ignore_fsid_mismatch = 1;
+	if (flags & OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR)
+		fs_info->ignore_chunk_tree_error = 1;
 
 	ret = btrfs_scan_fs_devices(fp, path, &fs_devices, sb_bytenr,
 				    (flags & OPEN_CTREE_RECOVER_SUPER),
@@ -1260,13 +1268,18 @@ static struct btrfs_fs_info *__open_ctree_fd(int fp, const char *path,
 	if (ret)
 		goto out_chunk;
 
+	/* Chunk tree root is unable to read, return directly */
+	if (!fs_info->chunk_root)
+		return fs_info;
+
 	eb = fs_info->chunk_root->node;
 	read_extent_buffer(eb, fs_info->chunk_tree_uuid,
 			   btrfs_header_chunk_tree_uuid(eb),
 			   BTRFS_UUID_SIZE);
 
 	ret = btrfs_setup_all_roots(fs_info, root_tree_bytenr, flags);
-	if (ret && !(flags & __OPEN_CTREE_RETURN_CHUNK_ROOT))
+	if (ret && !(flags & __OPEN_CTREE_RETURN_CHUNK_ROOT) &&
+	    !fs_info->ignore_chunk_tree_error)
 		goto out_chunk;
 
 	return fs_info;
@@ -1308,6 +1321,8 @@ struct btrfs_root *open_ctree(const char *filename, u64 sb_bytenr,
 {
 	struct btrfs_fs_info *info;
 
+	/* This flags may not return fs_info with any valid root */
+	BUG_ON(flags & OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR);
 	info = open_ctree_fs_info(filename, sb_bytenr, 0, flags);
 	if (!info)
 		return NULL;
@@ -1320,6 +1335,9 @@ struct btrfs_root *open_ctree_fd(int fp, const char *path, u64 sb_bytenr,
 				 enum btrfs_open_ctree_flags flags)
 {
 	struct btrfs_fs_info *info;
+
+	/* This flags may not return fs_info with any valid root */
+	BUG_ON(flags & OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR);
 	info = __open_ctree_fd(fp, path, sb_bytenr, 0, flags);
 	if (!info)
 		return NULL;
@@ -1658,14 +1676,15 @@ int write_ctree_super(struct btrfs_trans_handle *trans,
 	return ret;
 }
 
-int close_ctree(struct btrfs_root *root)
+int close_ctree_fs_info(struct btrfs_fs_info *fs_info)
 {
 	int ret;
 	struct btrfs_trans_handle *trans;
-	struct btrfs_fs_info *fs_info = root->fs_info;
+	struct btrfs_root *root = fs_info->tree_root;
 
 	if (fs_info->last_trans_committed !=
 	    fs_info->generation) {
+		BUG_ON(!root);
 		trans = btrfs_start_transaction(root, 1);
 		btrfs_commit_transaction(trans, root);
 		trans = btrfs_start_transaction(root, 1);
diff --git a/disk-io.h b/disk-io.h
index d3e3aaa..c2eb1d6 100644
--- a/disk-io.h
+++ b/disk-io.h
@@ -53,7 +53,15 @@ enum btrfs_open_ctree_flags {
 	 * Like split PARTIAL into SKIP_CSUM/SKIP_EXTENT
 	 */
 
-	OPEN_CTREE_IGNORE_FSID_MISMATCH	= (1 << 10)
+	OPEN_CTREE_IGNORE_FSID_MISMATCH	= (1 << 10),
+
+	/*
+	 * Allow open_ctree_fs_info() to return a incomplete fs_info with
+	 * system chunks from super block only.
+	 * It's useful for chunk corruption case.
+	 * Makes no sense for open_ctree variants returning btrfs_root.
+	 */
+	OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR = (1 << 11)
 };
 
 static inline u64 btrfs_sb_offset(int mirror)
@@ -101,7 +109,13 @@ struct btrfs_root *open_ctree_fd(int fp, const char *path, u64 sb_bytenr,
 struct btrfs_fs_info *open_ctree_fs_info(const char *filename,
 					 u64 sb_bytenr, u64 root_tree_bytenr,
 					 enum btrfs_open_ctree_flags flags);
-int close_ctree(struct btrfs_root *root);
+int close_ctree_fs_info(struct btrfs_fs_info *fs_info);
+static inline int close_ctree(struct btrfs_root *root)
+{
+	BUG_ON(!root);
+	return close_ctree_fs_info(root->fs_info);
+}
+
 int write_all_supers(struct btrfs_root *root);
 int write_ctree_super(struct btrfs_trans_handle *trans,
 		      struct btrfs_root *root);
-- 
2.7.1

[PATCH 3/5] btrfs-progs: Add support for tree block operations on fs_info without roots.

[Qu Wenruo]

Since open_ctree_fs_info() now may return a fs_info even without any
roots, modify functions like read_tree_block() to operate with such
fs_info.

This provides the basis for btrfs-find-root to operate on chunk tree
corrupted fs.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 btrfs-corrupt-block.c |  2 +-
 disk-io.c             | 70 +++++++++++++++++++++++++++++----------------------
 disk-io.h             | 17 ++++++++++---
 extent-tree.c         |  3 ++-
 volumes.c             |  3 ++-
 5 files changed, 58 insertions(+), 37 deletions(-)

diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index c908b7e..be5cd7e 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -45,7 +45,7 @@ static struct extent_buffer *debug_corrupt_block(struct btrfs_root *root,
 	int num_copies;
 	int mirror_num = 1;
 
-	eb = btrfs_find_create_tree_block(root, bytenr, blocksize);
+	eb = btrfs_find_create_tree_block(root->fs_info, bytenr, blocksize);
 	if (!eb)
 		return NULL;
 
diff --git a/disk-io.c b/disk-io.c
index 3b5a08e..51a4930 100644
--- a/disk-io.c
+++ b/disk-io.c
@@ -51,10 +51,12 @@ static u32 max_nritems(u8 level, u32 nodesize)
 		sizeof(struct btrfs_key_ptr));
 }
 
-static int check_tree_block(struct btrfs_root *root, struct extent_buffer *buf)
+static int check_tree_block(struct btrfs_fs_info *fs_info,
+			    struct extent_buffer *buf)
 {
 
 	struct btrfs_fs_devices *fs_devices;
+	u32 leafsize = btrfs_super_leafsize(fs_info->super_copy);
 	int ret = BTRFS_BAD_FSID;
 
 	if (buf->start != btrfs_header_bytenr(buf))
@@ -62,12 +64,12 @@ static int check_tree_block(struct btrfs_root *root, struct extent_buffer *buf)
 	if (btrfs_header_level(buf) >= BTRFS_MAX_LEVEL)
 		return BTRFS_BAD_LEVEL;
 	if (btrfs_header_nritems(buf) > max_nritems(btrfs_header_level(buf),
-						    root->nodesize))
+						    leafsize))
 		return BTRFS_BAD_NRITEMS;
 
-	fs_devices = root->fs_info->fs_devices;
+	fs_devices = fs_info->fs_devices;
 	while (fs_devices) {
-		if (root->fs_info->ignore_fsid_mismatch ||
+		if (fs_info->ignore_fsid_mismatch ||
 		    !memcmp_extent_buffer(buf, fs_devices->fsid,
 					  btrfs_header_fsid(),
 					  BTRFS_FSID_SIZE)) {
@@ -79,7 +81,7 @@ static int check_tree_block(struct btrfs_root *root, struct extent_buffer *buf)
 	return ret;
 }
 
-static void print_tree_block_error(struct btrfs_root *root,
+static void print_tree_block_error(struct btrfs_fs_info *fs_info,
 				struct extent_buffer *eb,
 				int err)
 {
@@ -92,7 +94,7 @@ static void print_tree_block_error(struct btrfs_root *root,
 		read_extent_buffer(eb, buf, btrfs_header_fsid(),
 				   BTRFS_UUID_SIZE);
 		uuid_unparse(buf, found_uuid);
-		uuid_unparse(root->fs_info->fsid, fs_uuid);
+		uuid_unparse(fs_info->fsid, fs_uuid);
 		fprintf(stderr, "fsid mismatch, want=%s, have=%s\n",
 			fs_uuid, found_uuid);
 		break;
@@ -157,16 +159,22 @@ int verify_tree_block_csum_silent(struct extent_buffer *buf, u16 csum_size)
 	return __csum_tree_block_size(buf, csum_size, 1, 1);
 }
 
-int csum_tree_block(struct btrfs_root *root, struct extent_buffer *buf,
-			   int verify)
+static int csum_tree_block_fs_info(struct btrfs_fs_info *fs_info,
+				   struct extent_buffer *buf, int verify)
 {
 	u16 csum_size =
-		btrfs_super_csum_size(root->fs_info->super_copy);
-	if (verify && root->fs_info->suppress_check_block_errors)
+		btrfs_super_csum_size(fs_info->super_copy);
+	if (verify && fs_info->suppress_check_block_errors)
 		return verify_tree_block_csum_silent(buf, csum_size);
 	return csum_tree_block_size(buf, csum_size, verify);
 }
 
+int csum_tree_block(struct btrfs_root *root, struct extent_buffer *buf,
+			   int verify)
+{
+	return csum_tree_block_fs_info(root->fs_info, buf, verify);
+}
+
 struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
 					    u64 bytenr, u32 blocksize)
 {
@@ -174,11 +182,11 @@ struct extent_buffer *btrfs_find_tree_block(struct btrfs_root *root,
 				  bytenr, blocksize);
 }
 
-struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
-						 u64 bytenr, u32 blocksize)
+struct extent_buffer *
+btrfs_find_create_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr,
+			     u32 blocksize)
 {
-	return alloc_extent_buffer(&root->fs_info->extent_cache, bytenr,
-				   blocksize);
+	return alloc_extent_buffer(&fs_info->extent_cache, bytenr, blocksize);
 }
 
 void readahead_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
@@ -294,8 +302,9 @@ int read_whole_eb(struct btrfs_fs_info *info, struct extent_buffer *eb, int mirr
 	return 0;
 }
 
-struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
-				     u32 blocksize, u64 parent_transid)
+struct extent_buffer *
+read_tree_block_fs_info(struct btrfs_fs_info *fs_info, u64 bytenr, u32 blocksize,
+			u64 parent_transid)
 {
 	int ret;
 	struct extent_buffer *eb;
@@ -305,7 +314,7 @@ struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
 	int num_copies;
 	int ignore = 0;
 
-	eb = btrfs_find_create_tree_block(root, bytenr, blocksize);
+	eb = btrfs_find_create_tree_block(fs_info, bytenr, blocksize);
 	if (!eb)
 		return ERR_PTR(-ENOMEM);
 
@@ -313,33 +322,33 @@ struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
 		return eb;
 
 	while (1) {
-		ret = read_whole_eb(root->fs_info, eb, mirror_num);
-		if (ret == 0 && csum_tree_block(root, eb, 1) == 0 &&
-		    check_tree_block(root, eb) == 0 &&
+		ret = read_whole_eb(fs_info, eb, mirror_num);
+		if (ret == 0 && csum_tree_block_fs_info(fs_info, eb, 1) == 0 &&
+		    check_tree_block(fs_info, eb) == 0 &&
 		    verify_parent_transid(eb->tree, eb, parent_transid, ignore)
 		    == 0) {
 			if (eb->flags & EXTENT_BAD_TRANSID &&
 			    list_empty(&eb->recow)) {
 				list_add_tail(&eb->recow,
-					      &root->fs_info->recow_ebs);
+					      &fs_info->recow_ebs);
 				eb->refs++;
 			}
 			btrfs_set_buffer_uptodate(eb);
 			return eb;
 		}
 		if (ignore) {
-			if (check_tree_block(root, eb)) {
-				if (!root->fs_info->suppress_check_block_errors)
-					print_tree_block_error(root, eb,
-						check_tree_block(root, eb));
+			if (check_tree_block(fs_info, eb)) {
+				if (!fs_info->suppress_check_block_errors)
+					print_tree_block_error(fs_info, eb,
+						check_tree_block(fs_info, eb));
 			} else {
-				if (!root->fs_info->suppress_check_block_errors)
+				if (!fs_info->suppress_check_block_errors)
 					fprintf(stderr, "Csum didn't match\n");
 			}
 			ret = -EIO;
 			break;
 		}
-		num_copies = btrfs_num_copies(&root->fs_info->mapping_tree,
+		num_copies = btrfs_num_copies(&fs_info->mapping_tree,
 					      eb->start, eb->len);
 		if (num_copies == 1) {
 			ignore = 1;
@@ -431,8 +440,9 @@ int write_tree_block(struct btrfs_trans_handle *trans,
 		     struct btrfs_root *root,
 		     struct extent_buffer *eb)
 {
-	if (check_tree_block(root, eb)) {
-		print_tree_block_error(root, eb, check_tree_block(root, eb));
+	if (check_tree_block(root->fs_info, eb)) {
+		print_tree_block_error(root->fs_info, eb,
+				check_tree_block(root->fs_info, eb));
 		BUG();
 	}
 
@@ -938,7 +948,7 @@ static int setup_root_or_create_block(struct btrfs_fs_info *fs_info,
 		 * million of places that assume a root has a valid ->node
 		 */
 		info_root->node =
-			btrfs_find_create_tree_block(info_root, 0, leafsize);
+			btrfs_find_create_tree_block(fs_info, 0, leafsize);
 		if (!info_root->node)
 			return -ENOMEM;
 		clear_extent_buffer_uptodate(NULL, info_root->node);
diff --git a/disk-io.h b/disk-io.h
index c2eb1d6..d12e222 100644
--- a/disk-io.h
+++ b/disk-io.h
@@ -75,14 +75,23 @@ static inline u64 btrfs_sb_offset(int mirror)
 struct btrfs_device;
 
 int read_whole_eb(struct btrfs_fs_info *info, struct extent_buffer *eb, int mirror);
-struct extent_buffer *read_tree_block(struct btrfs_root *root, u64 bytenr,
-				      u32 blocksize, u64 parent_transid);
+struct extent_buffer *
+read_tree_block_fs_info(struct btrfs_fs_info *fs_info, u64 bytenr, u32 blocksize,
+			u64 parent_transid);
+static inline struct extent_buffer *
+read_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
+		u64 parent_transid)
+{
+	return read_tree_block_fs_info(root->fs_info, bytenr, blocksize,
+			parent_transid);
+}
 int read_extent_data(struct btrfs_root *root, char *data, u64 logical,
 		     u64 *len, int mirror);
 void readahead_tree_block(struct btrfs_root *root, u64 bytenr, u32 blocksize,
 			  u64 parent_transid);
-struct extent_buffer *btrfs_find_create_tree_block(struct btrfs_root *root,
-						   u64 bytenr, u32 blocksize);
+struct extent_buffer *
+btrfs_find_create_tree_block(struct btrfs_fs_info *fs_info, u64 bytenr,
+			     u32 blocksize);
 
 int __setup_root(u32 nodesize, u32 leafsize, u32 sectorsize,
                         u32 stripesize, struct btrfs_root *root,
diff --git a/extent-tree.c b/extent-tree.c
index 1650bdb..b9b00f0 100644
--- a/extent-tree.c
+++ b/extent-tree.c
@@ -2822,7 +2822,8 @@ struct extent_buffer *btrfs_alloc_free_block(struct btrfs_trans_handle *trans,
 		return ERR_PTR(ret);
 	}
 
-	buf = btrfs_find_create_tree_block(root, ins.objectid, blocksize);
+	buf = btrfs_find_create_tree_block(root->fs_info, ins.objectid,
+					   blocksize);
 	if (!buf) {
 		btrfs_free_extent(trans, root, ins.objectid, ins.offset,
 				  0, root->root_key.objectid, level, 0);
diff --git a/volumes.c b/volumes.c
index 059c4f4..cc3e6b7 100644
--- a/volumes.c
+++ b/volumes.c
@@ -1840,7 +1840,8 @@ int btrfs_read_sys_array(struct btrfs_root *root)
 	u32 cur_offset;
 	struct btrfs_key key;
 
-	sb = btrfs_find_create_tree_block(root, BTRFS_SUPER_INFO_OFFSET,
+	sb = btrfs_find_create_tree_block(root->fs_info,
+					  BTRFS_SUPER_INFO_OFFSET,
 					  BTRFS_SUPER_INFO_SIZE);
 	if (!sb)
 		return -ENOMEM;
-- 
2.7.1

[PATCH 4/5] btrfs-progs: find-root: Allow btrfs-find-root to search chunk root even chunk root is corrupted

[Qu Wenruo]

Since now open_ctree_fs_info() can even return a valid fs_info with only
system chunk mapping from super block, use this ability to do chunk root
search for heavily damanged fs.

As an fast alternative for time consuming and buggy chunk-recovery.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 btrfs-find-root.c | 17 +++++++++--------
 find-root.c       | 10 +++++-----
 find-root.h       |  2 +-
 3 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/btrfs-find-root.c b/btrfs-find-root.c
index fc3812c..3d268cb 100644
--- a/btrfs-find-root.c
+++ b/btrfs-find-root.c
@@ -69,7 +69,6 @@ static void get_root_gen_and_level(u64 objectid, struct btrfs_fs_info *fs_info,
 	case BTRFS_CHUNK_TREE_OBJECTID:
 		level = btrfs_super_chunk_root_level(super);
 		gen = btrfs_super_chunk_root_generation(super);
-		printf("Search for chunk root is not supported yet\n");
 		break;
 	case BTRFS_TREE_LOG_OBJECTID:
 		level = btrfs_super_log_root_level(super);
@@ -145,7 +144,7 @@ static void print_find_root_result(struct cache_tree *result,
 
 int main(int argc, char **argv)
 {
-	struct btrfs_root *root;
+	struct btrfs_fs_info *fs_info;
 	struct btrfs_find_root_filter filter = {0};
 	struct cache_tree result;
 	struct cache_extent *found;
@@ -192,16 +191,18 @@ int main(int argc, char **argv)
 		exit(1);
 	}
 
-	root = open_ctree(argv[optind], 0, OPEN_CTREE_CHUNK_ROOT_ONLY);
-	if (!root) {
-		fprintf(stderr, "Open ctree failed\n");
+	fs_info = open_ctree_fs_info(argv[optind], 0, 0,
+			OPEN_CTREE_CHUNK_ROOT_ONLY |
+			OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR);
+	if (!fs_info) {
+		error("Open ctree failed\n");
 		exit(1);
 	}
 	cache_tree_init(&result);
 
-	get_root_gen_and_level(filter.objectid, root->fs_info,
+	get_root_gen_and_level(filter.objectid, fs_info,
 			       &filter.match_gen, &filter.match_level);
-	ret = btrfs_find_root_search(root, &filter, &result, &found);
+	ret = btrfs_find_root_search(fs_info, &filter, &result, &found);
 	if (ret < 0) {
 		fprintf(stderr, "Fail to search the tree root: %s\n",
 			strerror(-ret));
@@ -215,7 +216,7 @@ int main(int argc, char **argv)
 	print_find_root_result(&result, &filter);
 out:
 	btrfs_find_root_free(&result);
-	close_ctree(root);
+	close_ctree_fs_info(fs_info);
 	btrfs_close_all_devices();
 	return ret;
 }
diff --git a/find-root.c b/find-root.c
index f0204c8..823db6a 100644
--- a/find-root.c
+++ b/find-root.c
@@ -101,17 +101,16 @@ static int add_eb_to_result(struct extent_buffer *eb,
  * Return 1 if found root with given gen/level and set *match to it.
  * Return <0 if error happens
  */
-int btrfs_find_root_search(struct btrfs_root *chunk_root,
+int btrfs_find_root_search(struct btrfs_fs_info *fs_info,
 			   struct btrfs_find_root_filter *filter,
 			   struct cache_tree *result,
 			   struct cache_extent **match)
 {
-	struct btrfs_fs_info *fs_info = chunk_root->fs_info;
 	struct extent_buffer *eb;
 	u64 chunk_offset = 0;
 	u64 chunk_size = 0;
 	u64 offset = 0;
-	u32 leafsize = chunk_root->leafsize;
+	u32 leafsize = btrfs_super_leafsize(fs_info->super_copy);
 	int suppress_errors = 0;
 	int ret = 0;
 
@@ -133,8 +132,9 @@ int btrfs_find_root_search(struct btrfs_root *chunk_root,
 		}
 		for (offset = chunk_offset;
 		     offset < chunk_offset + chunk_size;
-		     offset += chunk_root->leafsize) {
-			eb = read_tree_block(chunk_root, offset, leafsize, 0);
+		     offset += leafsize) {
+			eb = read_tree_block_fs_info(fs_info, offset, leafsize,
+						     0);
 			if (!eb || IS_ERR(eb))
 				continue;
 			ret = add_eb_to_result(eb, result, leafsize, filter,
diff --git a/find-root.h b/find-root.h
index 1c67ebc..60d1111 100644
--- a/find-root.h
+++ b/find-root.h
@@ -65,7 +65,7 @@ struct btrfs_find_root_filter {
 	 * This *WILL* take *TONS* of extra time.
 	 */
 };
-int btrfs_find_root_search(struct btrfs_root *chunk_root,
+int btrfs_find_root_search(struct btrfs_fs_info *fs_info,
 			   struct btrfs_find_root_filter *filter,
 			   struct cache_tree *result,
 			   struct cache_extent **match);
-- 
2.7.1

[PATCH 5/5] btrfs-progs: misc-test: Add regression test for find-root gives empty result

[Qu Wenruo]

Add regression test for btrfs-find-root gives empty result even the fs
is OK.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
---
 .../first_meta_chunk.btrfs-image                    | Bin 0 -> 4096 bytes
 tests/misc-tests/012-find-root-no-result/test.sh    |  20 ++++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image
 create mode 100644 tests/misc-tests/012-find-root-no-result/test.sh

diff --git a/tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image b/tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image
new file mode 100644
index 0000000000000000000000000000000000000000..7bf6c50916f68ed00dae7627e848a91abf5455bb
GIT binary patch
literal 4096
zcmeH}doa}NAIGsQmda)AQE~}Ity*QRZLQcu=&WiZ6l<v*_p^n~ZBwar&8AK3-f}Wl
zn<%$(S-VP-TUeKJZKVw%k<BIh!<;|P^!xp0&Oc}7H^1pK&wQTue4b}MpU?A}@AsJ}
z!7Jqgf4ggq*q<i7I*YH5RiU)1#5i2VwYBgyce!p|m#yoTb-izWPwTqY0jw=>Hm`Zm
zT3n2yw&wS*>oxV$eiHae;D3lf=vz4tvI6I_G|}vcg0qnKG;F%tzx=vT`LU|1Dwm&l
zY#~&S$|xNUb(c-000fdb^0r&8Q5JKLui7G*TFq#n_)U;L@UhyJ+sKX4N}H3BiK-BI
zDrdtrk7@#LFe&y|#|m>6UC%HQ<cFx6TYL#r@}7$Yyw;tU+!<iRmDN?T(~q<!cASyu
zh}9h@T-(jv)|f${X@*KCUOjFIvgYcmO6hFciUb6{jpssbZFU<-I1wR^>~{*OZBlge
z-LbiY;#jj&ovF$GF$dF!W4(-?XA#+ebabWZrc5i^VH2v|)HRZdF7YYt9r`|c+eH+k
zApcb8LB1Q<soIp$s2nc@!oNBM30CXqN{B52&U%Kvoove4P<Kqss@>fZ2Yv#ASQuJK
zbCewXJOGlm&%~N98~6}aWNux>7>R@KaBc#FJH=a*g`+JRwkjz3p*J7r>@L#7&5-C4
zNAHIt<%etcS5E8Q9V4-Kwl#Pr58as&{jzq+{?!;oun808q}9qT+w1`tFmU)yK)km?
zVF6zvwO$7S-+Gw`mn?Heipl2YjEO&yusO-wreN%sKBweZueh<8;@^vfh+}O=g=q^z
z5wl+QFP?RNjv0*TcK2@+H1+gF3$P?*jd)BKmEF)jKEp(3<NIr?M@yO}`1hTrLne$i
zPWH91&TXkX!3yfV)>gXfuf45m0+oMLH*1LUn|Tv6#oSsXn%(1mlr&lngYz^LSDWtJ
z3$r|~3LNgcW&ND77hb2eLtb0^h4y!GgyM;SI`5k{GGm*Hw`>j}C`4m$khsrk@&^q(
z!DGh)`w>o#aSBbd@<O&CQjnbyG!F}^EQY|_FVUy_Yg<(8lkVWEdKT&lBfHdF^#)(X
z%@z)R8D5+lA3T3Ph_vFrN0}^i2&2jKS*)r%C75F6j4>vq!5{fU|Cilw%Tr=bAAs7k
zHFtO}c#I1~t~>F~ktrkPP-$Vy{L*WY`yyTx<u_+vZxZn^`$Cwu4A3*59a%$NF?^4g
z(+RnrV;F3#pGbMPW9gA0gz^k{eZ~-+EAc}e5u-cC-LDk(^-vBhk|%s*;yk2&{k>-4
z(Ze~DCf!j&JYLKkM>@SqYyfKm_So7mljhRXQqQU3Q8Wzp6~DCf@w`{aD>!mUvE3G`
z^Xdc)49QUqiI=Gy#S~a>IV9zw)bZr_m0}>K;TES-$}QJYI^%q{=)vr#)+veslWZ<c
zCPlT0(t5Jeno5>}LP`0@EtVskT4=v70rlK7SzFU4QGUHfvO;oaLW}U?_zE0v-fM!|
ziy%HQC8G<*9Rhhzq71(BDT%0_K-Cd@I>@Y--5{4c9uku0%iDjYYM6;{FX>}3c@Y2V
z0HZ_?9kp+kSS03a?(oZoxrx5Ok!)o+Qgmi<-+_woCT2(U?y71IyzKgr{|sagrbsd-
zlTkmgJ-oietEjF6`p`H+rS^lFeJ36%8I#30F6TQyV1X{NIPL>-rqx|iKd==Oa|X`2
zY*h5QSGV;2qx_-ozBntF2R%*<bDB)BO-i}jA`$A^TB@3M^z7Kuwg+b%CQ_S|YC`qw
zyFO_4fWV`W21}~7$fDd$?eV_6-uci)=8~v$E~C5nKqK0Z@M`XEKkd2v8~ri*xBMG+
zw|)M&Q?={+h6vxCnl76k=1+D-Hy1*e4!54gBN8`di8d9Lo=f2k*^r%Ic^r9o3tDKN
zeBpp8`B>cM?t<*j0eU#htLPFIn_QgyDmW8H4WlO>oJ<LBpbGkDI>XwY8Ruoi%s9?-
zAV$3X75zne`>Cf8M+2cV2m(=!*iT0GIDNjK*CukRD(qd33d7@B2Xy4D%3TM6!uw`S
zjU4N9=k+oPBe@BGXa+B0cBt}>5d652F_K2y;RH$F6UzkL)!PvH1(kV<1<_H?|IHVg
z`i1z=w)fPtRO67)l7;O===i`3RIR7ou4?kPGPfUp_dm%qkZHO_qQ#{kA7?N2RU1Zy
z%Q(S{&BbvCGfF)d(xuYqD{!U9@LJ0U&eAMr>5^WQ19(EKc6g6EKRkwsIG8fX6?JHv
zKB1eiDyD0XXzB-_4f-I;x62S|j#&Pe2cbpdio_?_1Z(Y5dE{Gn4E$n-rk`t%Qfbez
zQ6JaGeZ)y|%j~VPa24fRt?D2?0K5pyvQw#48P|%{;;RKu;K*V<BtsMId&IruDUoJ>
z1}*cwsZi?HUV)Envg_#rx?_N0SWTSSCFdUP&cyJ*<<cAV<f(~oQ_J&V=ualF-zEXV
zs@RAdzObe$r_7UwP^Sc)Ng+v((8^;ATUx76TIjIiuMK$1{M*;L9vH6O*yS}9xFJBe
zD<Y2<=-tNB^Nsq*HZB5voo=UX&q%;DCVUZ$0=_g4h2=5Oh3+e7@M!PB_LZ;&v)TA}
z<bukwpgk$Ir*oF$RzwRGPc<ESS4p(i$kq`QfWt6n84^icE}|k<N5-Bn;ZuSkrZWfy
ztXP5~G>V-7knx>G6EA+gAYI|=Uxe+dxO*}UH~^>ejWx*hXq44!tJhd<=FvyzH5wZC
zA}GrL`3QeIdg)w^i(C5r-$jn2OSESOZGFpc5^m2cCgLRjc8Y&wBj@6PxG<e;@Q)<_
Mne-=t|1Sdn0?V+~RR910

literal 0
HcmV?d00001

diff --git a/tests/misc-tests/012-find-root-no-result/test.sh b/tests/misc-tests/012-find-root-no-result/test.sh
new file mode 100644
index 0000000..4951633
--- /dev/null
+++ b/tests/misc-tests/012-find-root-no-result/test.sh
@@ -0,0 +1,20 @@
+#!/bin/bash
+# Regression test for case btrfs-find-root may print no result on a
+# recent fs or balanced fs, whose metadata chunk is the first chunk
+# and the only metadata chunk
+
+source $TOP/tests/common
+
+check_prereq btrfs-find-root
+check_prereq btrfs-image
+
+$TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
+	_fail "failed to extract first_meta_chunk.btrfs-image"
+
+result=$($TOP/btrfs-find-root test.img | sed '/^Superblock/d')
+
+if [ -z "$result" ]; then
+	_fail "btrfs-find-root failed to find tree root"
+fi
+
+rm test.img
-- 
2.7.1

Re: [PATCH 0/5] btrfs-find-root enhancement for chunk tree corrupted fs

[Qu Wenruo]

This patchset can also be fetched from github:
https://github.com/adam900710/btrfs-progs.git syschunk_find_root_20160222

Thanks,
Qu

Qu Wenruo wrote on 2016/02/22 14:59 +0800:
> Before this patchset, btrfs-find-root needs valid chunk tree from the
> fs.
> However for chunk root corrupted case, btrfs-find-root is of no use due
> to above limitation.
>
> This patchset will allow open_ctree_fs_info() to return a fs_info
> without any valid tree root, but system chunk map from superblock only.
> And modify btrfs-find-root along with some infrastructure to do chunk
> root search.
>
> Also fix an old bug where btrfs-find-root will always skip the first
> chunk, with its corresponding regression test.
>
> This also provides the basis for later "btrfsck --chunk-root" and faster
> chunk-recovery enhancement.
>
> Qu Wenruo (5):
>    btrfs: volume: Fix a bug causing btrfs-find-root to skip first chunk
>    btrfs: Allow open_ctree to return fs_info even chunk tree is corrupted
>    btrfs: Add support for tree block operations on fs_info without roots.
>    btrfs: find-root: Allow btrfs-find-root to search chunk root even
>      chunk     root is corrupted
>    btrfs: misc-test: Add regression test for find-root gives empty result
>
>   btrfs-corrupt-block.c                              |   2 +-
>   btrfs-find-root.c                                  |  17 ++--
>   ctree.h                                            |   1 +
>   disk-io.c                                          |  99 +++++++++++++--------
>   disk-io.h                                          |  35 ++++++--
>   extent-tree.c                                      |   3 +-
>   find-root.c                                        |  10 +--
>   find-root.h                                        |   2 +-
>   .../first_meta_chunk.btrfs-image                   | Bin 0 -> 4096 bytes
>   tests/misc-tests/012-find-root-no-result/test.sh   |  20 +++++
>   volumes.c                                          |  20 +++--
>   11 files changed, 147 insertions(+), 62 deletions(-)
>   create mode 100644 tests/misc-tests/012-find-root-no-result/first_meta_chunk.btrfs-image
>   create mode 100644 tests/misc-tests/012-find-root-no-result/test.sh
>

Re: [PATCH 5/5] btrfs-progs: misc-test: Add regression test for find-root gives empty result

[David Sterba]

On Mon, Feb 22, 2016 at 02:59:57PM +0800, Qu Wenruo wrote:
> zA}GrL`3QeIdg)w^i(C5r-$jn2OSESOZGFpc5^m2cCgLRjc8Y&wBj@6PxG<e;@Q)<_
> Mne-=t|1Sdn0?V+~RR910
> 
> literal 0
> HcmV?d00001
> 
> diff --git a/tests/misc-tests/012-find-root-no-result/test.sh b/tests/misc-tests/012-find-root-no-result/test.sh
> new file mode 100644
> index 0000000..4951633
> --- /dev/null
> +++ b/tests/misc-tests/012-find-root-no-result/test.sh
> @@ -0,0 +1,20 @@
> +#!/bin/bash
> +# Regression test for case btrfs-find-root may print no result on a
> +# recent fs or balanced fs, whose metadata chunk is the first chunk
> +# and the only metadata chunk
> +
> +source $TOP/tests/common
> +
> +check_prereq btrfs-find-root
> +check_prereq btrfs-image
> +
> +$TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
> +	_fail "failed to extract first_meta_chunk.btrfs-image"
> +
> +result=$($TOP/btrfs-find-root test.img | sed '/^Superblock/d')
> +
> +if [ -z "$result" ]; then
> +	_fail "btrfs-find-root failed to find tree root"
> +fi
> +
> +rm test.img

Applied with following fixups:

--- a/tests/misc-tests/012-find-root-no-result/test.sh
+++ b/tests/misc-tests/012-find-root-no-result/test.sh
@@ -8,13 +8,17 @@ source $TOP/tests/common
 check_prereq btrfs-find-root
 check_prereq btrfs-image

-$TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
+run_check $TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
        _fail "failed to extract first_meta_chunk.btrfs-image"

-result=$($TOP/btrfs-find-root test.img | sed '/^Superblock/d')
+result=$(run_check_stdout $TOP/btrfs-find-root test.img | sed '/^Superblock/d')

 if [ -z "$result" ]; then
        _fail "btrfs-find-root failed to find tree root"
 fi

+if ! echo "$result" | grep -q 'Found tree root at'; then
+       _fail "btrfs-find-root failed to find tree root, unexpected output"
+fi
+
 rm test.img

Re: [PATCH 0/5] btrfs-find-root enhancement for chunk tree corrupted fs

[David Sterba]

On Mon, Feb 22, 2016 at 02:59:52PM +0800, Qu Wenruo wrote:
> Before this patchset, btrfs-find-root needs valid chunk tree from the
> fs.
> However for chunk root corrupted case, btrfs-find-root is of no use due
> to above limitation.
> 
> This patchset will allow open_ctree_fs_info() to return a fs_info
> without any valid tree root, but system chunk map from superblock only.
> And modify btrfs-find-root along with some infrastructure to do chunk
> root search.
> 
> Also fix an old bug where btrfs-find-root will always skip the first
> chunk, with its corresponding regression test.
> 
> This also provides the basis for later "btrfsck --chunk-root" and faster
> chunk-recovery enhancement.
> 
> Qu Wenruo (5):
>   btrfs: volume: Fix a bug causing btrfs-find-root to skip first chunk
>   btrfs: Allow open_ctree to return fs_info even chunk tree is corrupted
>   btrfs: Add support for tree block operations on fs_info without roots.
>   btrfs: find-root: Allow btrfs-find-root to search chunk root even
>     chunk     root is corrupted
>   btrfs: misc-test: Add regression test for find-root gives empty result

All applied, with minor adjustments. I'm not going to point them out
in the mails, please make a diff against your branch to see.

Re: [PATCH 5/5] btrfs-progs: misc-test: Add regression test for find-root gives empty result

[Qu Wenruo]

David Sterba wrote on 2016/02/24 13:10 +0100:
> On Mon, Feb 22, 2016 at 02:59:57PM +0800, Qu Wenruo wrote:
>> zA}GrL`3QeIdg)w^i(C5r-$jn2OSESOZGFpc5^m2cCgLRjc8Y&wBj@6PxG<e;@Q)<_
>> Mne-=t|1Sdn0?V+~RR910
>>
>> literal 0
>> HcmV?d00001
>>
>> diff --git a/tests/misc-tests/012-find-root-no-result/test.sh b/tests/misc-tests/012-find-root-no-result/test.sh
>> new file mode 100644
>> index 0000000..4951633
>> --- /dev/null
>> +++ b/tests/misc-tests/012-find-root-no-result/test.sh
>> @@ -0,0 +1,20 @@
>> +#!/bin/bash
>> +# Regression test for case btrfs-find-root may print no result on a
>> +# recent fs or balanced fs, whose metadata chunk is the first chunk
>> +# and the only metadata chunk
>> +
>> +source $TOP/tests/common
>> +
>> +check_prereq btrfs-find-root
>> +check_prereq btrfs-image
>> +
>> +$TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
>> +	_fail "failed to extract first_meta_chunk.btrfs-image"
>> +
>> +result=$($TOP/btrfs-find-root test.img | sed '/^Superblock/d')
>> +
>> +if [ -z "$result" ]; then
>> +	_fail "btrfs-find-root failed to find tree root"
>> +fi
>> +
>> +rm test.img
>
> Applied with following fixups:
>
> --- a/tests/misc-tests/012-find-root-no-result/test.sh
> +++ b/tests/misc-tests/012-find-root-no-result/test.sh
> @@ -8,13 +8,17 @@ source $TOP/tests/common
>   check_prereq btrfs-find-root
>   check_prereq btrfs-image
>
> -$TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
> +run_check $TOP/btrfs-image -r first_meta_chunk.btrfs-image test.img || \
>          _fail "failed to extract first_meta_chunk.btrfs-image"
>

Thanks, I just forget that we have run_check suffix to handle errors.

> -result=$($TOP/btrfs-find-root test.img | sed '/^Superblock/d')
> +result=$(run_check_stdout $TOP/btrfs-find-root test.img | sed '/^Superblock/d')
>
>   if [ -z "$result" ]; then
>          _fail "btrfs-find-root failed to find tree root"
>   fi
>
> +if ! echo "$result" | grep -q 'Found tree root at'; then
> +       _fail "btrfs-find-root failed to find tree root, unexpected output"
> +fi
> +
Right, if btrfs-find-root failed to find the tree root matches with 
superblock, it's also a bug.

Thanks,
Qu
>   rm test.img
>
>
>

Re: [PATCH 0/5] btrfs-find-root enhancement for chunk tree corrupted fs

[Qu Wenruo]

David Sterba wrote on 2016/02/24 13:38 +0100:
> On Mon, Feb 22, 2016 at 02:59:52PM +0800, Qu Wenruo wrote:
>> Before this patchset, btrfs-find-root needs valid chunk tree from the
>> fs.
>> However for chunk root corrupted case, btrfs-find-root is of no use due
>> to above limitation.
>>
>> This patchset will allow open_ctree_fs_info() to return a fs_info
>> without any valid tree root, but system chunk map from superblock only.
>> And modify btrfs-find-root along with some infrastructure to do chunk
>> root search.
>>
>> Also fix an old bug where btrfs-find-root will always skip the first
>> chunk, with its corresponding regression test.
>>
>> This also provides the basis for later "btrfsck --chunk-root" and faster
>> chunk-recovery enhancement.
>>
>> Qu Wenruo (5):
>>    btrfs: volume: Fix a bug causing btrfs-find-root to skip first chunk
>>    btrfs: Allow open_ctree to return fs_info even chunk tree is corrupted
>>    btrfs: Add support for tree block operations on fs_info without roots.
>>    btrfs: find-root: Allow btrfs-find-root to search chunk root even
>>      chunk     root is corrupted
>>    btrfs: misc-test: Add regression test for find-root gives empty result
>
> All applied, with minor adjustments. I'm not going to point them out
> in the mails, please make a diff against your branch to see.
>
>
Thanks for the modification.
It's completely OK.

Thanks,
Qu