[PATCH v2 0/2] btrfs: allow more fine control to rescue=usebackuproot

[PATCH v2 0/2] btrfs: allow more fine control to rescue=usebackuproot

[Qu Wenruo]

[CHANGELOG]
v2:
- Fix a missing use_backup_slot update for rescue=all mount option
  Which can trigger the ASSERT() on use_backup_slot.

- Add proper output for btrfs_show_options()

- Slightly update the commit message of the 2nd patch
  To address a false alert from Sashiko, where it thinks it's a bug not
  to load any backup root for the newest slot.
  For the newest slot, it matches the current generation in the super
  block, thus every root should be the same as the super block, and no
  need to load the bytenr from backup.

There is a bug report that for a specific corrupted btrfs, the
"rescue=usebackuproot" still chose the newest slot (aka, the same tree
root as the one in the super block) to mount the fs, and resulted
transid mismatch.

Meanwhile the reporter used btrfs-mod-sb to modify the fs to use a
specific backup slot, then the fs can pass btrfs-check.

This shows the limit of the current automatic backup root detection,
that as long as all tree root nodes can be loaded, btrfs will consider
it as a valid backup slot, without trying any other slot.

And end user has no way to tell btrfs to use a specific slot.

This patchest address the problem by:

- Make "rescue=usebackuproot" to always use the second newest slot
  Which has the highest chance to still get every tree block right
  without transid error.

- Introduce new "rescue=usebackuproot_*" mount option
  Where "*" can be 0/1/2/3.
  0 means the newest slot (aka, the one matching the super block
  generation), 1/2/3 means the second/third/fourth(oldest) newest slot.

  Now "rescue=usebackuproot" is just the same as
  "rescue=usebackuproot_1".

Although those "rescue=usebackuproot*" mount options still requires full
RO.
For proper recovery, we still need to use "btrfs check", and a new
option for btrfs-check will be introduced soon to make the backuproot
usage simpler for progs.

Qu Wenruo (2):
  btrfs: always use the second newest slot for rescue=usebackuproot
  btrfs: introduce more accurate usebackuproot options

 fs/btrfs/disk-io.c | 106 ++++++++++++++++++++-------------------------
 fs/btrfs/fs.h      |   1 +
 fs/btrfs/super.c   |  28 +++++++++++-
 3 files changed, 74 insertions(+), 61 deletions(-)

-- 
2.55.0

[PATCH v2 1/2] btrfs: always use the second newest slot for rescue=usebackuproot

[Qu Wenruo]

[RESCUE FAILURE]
There is a bug report that "rescue=usebackuproot" failed to properly
mount the fs, but if using btrfs-mod-sb to use the second newest slot,
then the fs can even pass the full "btrfs check" and be mounted RW.

[CAUSE]
During init_tree_roots(), "rescue=usebackuproot" only makes a difference
if the tree and chunk roots failed to be load.

The loading only requires the root node/leaf to be properly loaded:

- tree root
- extent tree
- block group tree
- device tree
- data reloc tree
- quota tree (optional)
- uuid tree
- raid stripe tree (for RST feature)

And it doesn't really check any child tree blocks, thus for a lot of
cases, "rescue=usebackuproot" won't make a difference.

[ENHANCEMENT]
Instead of trying every slot and settling down on the first passing
backup slot, always use the second newest slot.

The newest slot should always match the root info in the super block,
and if it worked the end user should not even need to bother to use
backup root.

Older backup slots normally have a very low chance to work.

Link: https://github.com/kdave/btrfs-progs/issues/1162#issuecomment-5475477108
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/disk-io.c | 102 +++++++++++++++++++--------------------------
 1 file changed, 42 insertions(+), 60 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index a1d83ad9a4c0..422fda467440 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2709,73 +2709,55 @@ static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
 	int backup_index = find_newest_super_backup(fs_info);
 	struct btrfs_super_block *sb = fs_info->super_copy;
 	struct btrfs_root *tree_root = fs_info->tree_root;
-	bool handle_error = false;
+	bool use_backup = btrfs_test_opt(fs_info, USEBACKUPROOT);
 	int ret = 0;
-	int i;
 
-	for (i = 0; i < BTRFS_NUM_BACKUP_ROOTS; i++) {
-		if (handle_error) {
-			if (!IS_ERR(tree_root->node))
-				free_extent_buffer(tree_root->node);
-			tree_root->node = NULL;
-
-			if (!btrfs_test_opt(fs_info, USEBACKUPROOT))
-				break;
-
-			free_root_pointers(fs_info, 0);
-
-			/*
-			 * Don't use the log in recovery mode, it won't be
-			 * valid
-			 */
-			btrfs_set_super_log_root(sb, 0);
-
-			btrfs_warn(fs_info, "try to load backup roots slot %d", i);
-			ret = read_backup_root(fs_info, i);
-			backup_index = ret;
-			if (ret < 0)
-				return ret;
+	if (use_backup) {
+		ret = read_backup_root(fs_info, 1);
+		if (ret < 0) {
+			btrfs_err(fs_info,
+				  "failed to load backup roots at slot %d", 1);
+			return ret;
 		}
-
-		ret = load_important_roots(fs_info);
-		if (ret) {
-			handle_error = true;
-			continue;
-		}
-
 		/*
-		 * No need to hold btrfs_root::objectid_mutex since the fs
-		 * hasn't been fully initialised and we are the only user
+		 * Don't use the log in recovery mode, it won't be
+		 * valid
 		 */
-		ret = btrfs_init_root_free_objectid(tree_root);
-		if (ret < 0) {
-			handle_error = true;
-			continue;
-		}
-
-		ASSERT(tree_root->free_objectid <= BTRFS_LAST_FREE_OBJECTID);
-
-		ret = btrfs_read_roots(fs_info);
-		if (ret < 0) {
-			handle_error = true;
-			continue;
-		}
-
-		/* All successful */
-		fs_info->generation = btrfs_header_generation(tree_root->node);
-		btrfs_set_last_trans_committed(fs_info, fs_info->generation);
-		fs_info->last_reloc_trans = 0;
-
-		/* Always begin writing backup roots after the one being used */
-		if (backup_index < 0) {
-			fs_info->backup_root_index = 0;
-		} else {
-			fs_info->backup_root_index = backup_index + 1;
-			fs_info->backup_root_index %= BTRFS_NUM_BACKUP_ROOTS;
-		}
-		break;
+		btrfs_set_super_log_root(sb, 0);
+		backup_index = ret;
+		btrfs_warn(fs_info, "loaded backup roots at slot %d", 1);
 	}
 
+	ret = load_important_roots(fs_info);
+	if (ret)
+		return ret;
+
+	/*
+	 * No need to hold btrfs_root::objectid_mutex since the fs
+	 * hasn't been fully initialised and we are the only user
+	 */
+	ret = btrfs_init_root_free_objectid(tree_root);
+	if (ret < 0)
+		return ret;
+
+	ASSERT(tree_root->free_objectid <= BTRFS_LAST_FREE_OBJECTID);
+
+	ret = btrfs_read_roots(fs_info);
+	if (ret < 0)
+		return ret;
+
+	/* All successful */
+	fs_info->generation = btrfs_header_generation(tree_root->node);
+	btrfs_set_last_trans_committed(fs_info, fs_info->generation);
+	fs_info->last_reloc_trans = 0;
+
+	/* Always begin writing backup roots after the one being used */
+	if (backup_index < 0) {
+		fs_info->backup_root_index = 0;
+	} else {
+		fs_info->backup_root_index = backup_index + 1;
+		fs_info->backup_root_index %= BTRFS_NUM_BACKUP_ROOTS;
+	}
 	return ret;
 }
 
-- 
2.55.0

[PATCH v2 2/2] btrfs: introduce more accurate usebackuproot options

[Qu Wenruo]

Previously we had no way to tell btrfs to use which backup slot, but
always use the newest backup, and retry until all backup roots are
exhausted.

This doesn't give any control to the end user, even if the end user
wants to use a specific backup slot.

To address this problem, allow more fine-control on usebackuproot rescue
option by introducing the following 4 rescue mount options:

- rescue=usebackuproot_0
- rescue=usebackuproot_1
- rescue=usebackuproot_2
- rescue=usebackuproot_3

The number 0 means the newest slot, which should be the same as the one
recorded in the super block and no need to load any backup root.

1/2/3 means the second/third/fourth(oldest) backup slot to use.

The existing "rescue=usebackuproot" is just an alias of
"rescue=usebackuproot_1".

This will allow users to choose which backup root to use, thus can have
a better control than blindly choose any slot that can pass the very
basic root node checks.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/disk-io.c | 12 +++++++++---
 fs/btrfs/fs.h      |  1 +
 fs/btrfs/super.c   | 28 ++++++++++++++++++++++++++--
 3 files changed, 36 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 422fda467440..406c46d1fba1 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2713,10 +2713,14 @@ static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
 	int ret = 0;
 
 	if (use_backup) {
-		ret = read_backup_root(fs_info, 1);
+		ASSERT(fs_info->use_backup_slot >= 0 &&
+		       fs_info->use_backup_slot < BTRFS_NUM_BACKUP_ROOTS);
+
+		ret = read_backup_root(fs_info, fs_info->use_backup_slot);
 		if (ret < 0) {
 			btrfs_err(fs_info,
-				  "failed to load backup roots at slot %d", 1);
+				  "failed to load backup roots at slot %d",
+				  fs_info->use_backup_slot);
 			return ret;
 		}
 		/*
@@ -2725,7 +2729,8 @@ static int __cold init_tree_roots(struct btrfs_fs_info *fs_info)
 		 */
 		btrfs_set_super_log_root(sb, 0);
 		backup_index = ret;
-		btrfs_warn(fs_info, "loaded backup roots at slot %d", 1);
+		btrfs_warn(fs_info, "loaded backup roots at slot %d",
+			   fs_info->use_backup_slot);
 	}
 
 	ret = load_important_roots(fs_info);
@@ -2896,6 +2901,7 @@ void btrfs_init_fs_info(struct btrfs_fs_info *fs_info)
 	init_waitqueue_head(&fs_info->async_submit_wait);
 	init_waitqueue_head(&fs_info->delayed_iputs_wait);
 
+	fs_info->use_backup_slot = -1;
 	/* Usable values until the real ones are cached from the superblock */
 	fs_info->nodesize = 4096;
 	fs_info->sectorsize = 4096;
diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h
index 3eba8438593c..649623c603e8 100644
--- a/fs/btrfs/fs.h
+++ b/fs/btrfs/fs.h
@@ -857,6 +857,7 @@ struct btrfs_fs_info {
 
 	/* Next backup root to be overwritten */
 	int backup_root_index;
+	int use_backup_slot;
 
 	/* Device replace state */
 	struct btrfs_dev_replace dev_replace;
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 464129b1b0d4..b5c1ff357aaf 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -87,6 +87,7 @@ struct btrfs_fs_context {
 	unsigned long long mount_opt;
 	unsigned long compress_type:4;
 	int compress_level;
+	int use_backup_slot;
 	refcount_t refs;
 };
 
@@ -175,6 +176,10 @@ static const struct constant_table btrfs_parameter_space_cache[] = {
 
 enum {
 	Opt_rescue_usebackuproot,
+	Opt_rescue_usebackup_slot_0,
+	Opt_rescue_usebackup_slot_1,
+	Opt_rescue_usebackup_slot_2,
+	Opt_rescue_usebackup_slot_3,
 	Opt_rescue_nologreplay,
 	Opt_rescue_ignorebadroots,
 	Opt_rescue_ignoredatacsums,
@@ -185,6 +190,10 @@ enum {
 
 static const struct constant_table btrfs_parameter_rescue[] = {
 	{ "usebackuproot", Opt_rescue_usebackuproot },
+	{ "usebackuproot_0", Opt_rescue_usebackup_slot_0},
+	{ "usebackuproot_1", Opt_rescue_usebackup_slot_1},
+	{ "usebackuproot_2", Opt_rescue_usebackup_slot_2},
+	{ "usebackuproot_3", Opt_rescue_usebackup_slot_3},
 	{ "nologreplay", Opt_rescue_nologreplay },
 	{ "ignorebadroots", Opt_rescue_ignorebadroots },
 	{ "ibadroots", Opt_rescue_ignorebadroots },
@@ -588,6 +597,14 @@ static int btrfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
 		switch (result.uint_32) {
 		case Opt_rescue_usebackuproot:
 			btrfs_set_opt(ctx->mount_opt, USEBACKUPROOT);
+			ctx->use_backup_slot = 1;
+			break;
+		case Opt_rescue_usebackup_slot_0:
+		case Opt_rescue_usebackup_slot_1:
+		case Opt_rescue_usebackup_slot_2:
+		case Opt_rescue_usebackup_slot_3:
+			btrfs_set_opt(ctx->mount_opt, USEBACKUPROOT);
+			ctx->use_backup_slot = result.uint_32 - Opt_rescue_usebackup_slot_0;
 			break;
 		case Opt_rescue_nologreplay:
 			btrfs_set_opt(ctx->mount_opt, NOLOGREPLAY);
@@ -611,6 +628,7 @@ static int btrfs_parse_param(struct fs_context *fc, struct fs_parameter *param)
 			btrfs_set_opt(ctx->mount_opt, IGNOREBADROOTS);
 			btrfs_set_opt(ctx->mount_opt, NOLOGREPLAY);
 			btrfs_set_opt(ctx->mount_opt, USEBACKUPROOT);
+			ctx->use_backup_slot = 1;
 			break;
 		default:
 			btrfs_info(NULL, "unrecognized rescue option '%s'",
@@ -1089,8 +1107,11 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 		seq_puts(seq, ",notreelog");
 	if (btrfs_test_opt(info, NOLOGREPLAY))
 		print_rescue_option(seq, "nologreplay", &printed);
-	if (btrfs_test_opt(info, USEBACKUPROOT))
-		print_rescue_option(seq, "usebackuproot", &printed);
+	if (btrfs_test_opt(info, USEBACKUPROOT)) {
+		seq_printf(seq, "%susebackuproot_%d", printed ? ":" : ",rescue=",
+			   info->use_backup_slot);
+		printed = true;
+	}
 	if (btrfs_test_opt(info, IGNOREBADROOTS))
 		print_rescue_option(seq, "ignorebadroots", &printed);
 	if (btrfs_test_opt(info, IGNOREDATACSUMS))
@@ -1406,6 +1427,7 @@ static void btrfs_ctx_to_info(struct btrfs_fs_info *fs_info, struct btrfs_fs_con
 	fs_info->mount_opt = ctx->mount_opt;
 	fs_info->compress_type = ctx->compress_type;
 	fs_info->compress_level = ctx->compress_level;
+	fs_info->use_backup_slot = ctx->use_backup_slot;
 }
 
 static void btrfs_info_to_ctx(struct btrfs_fs_info *fs_info, struct btrfs_fs_context *ctx)
@@ -1417,6 +1439,7 @@ static void btrfs_info_to_ctx(struct btrfs_fs_info *fs_info, struct btrfs_fs_con
 	ctx->mount_opt = fs_info->mount_opt;
 	ctx->compress_type = fs_info->compress_type;
 	ctx->compress_level = fs_info->compress_level;
+	ctx->use_backup_slot = fs_info->use_backup_slot;
 }
 
 #define btrfs_info_if_set(fs_info, old_ctx, opt, fmt, args...)			\
@@ -2205,6 +2228,7 @@ static int btrfs_init_fs_context(struct fs_context *fc)
 	if (fc->purpose == FS_CONTEXT_FOR_RECONFIGURE) {
 		btrfs_info_to_ctx(btrfs_sb(fc->root->d_sb), ctx);
 	} else {
+		ctx->use_backup_slot = -1;
 		ctx->thread_pool_size =
 			min_t(unsigned long, num_online_cpus() + 2, 8);
 		ctx->max_inline = BTRFS_DEFAULT_MAX_INLINE;
-- 
2.55.0