[PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

[PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

[Qu Wenruo]

The patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev

There are several reports in mail list for btrfs device size related
problems.

1) kernel refuse to mount some fs, due to mismatched super total_bytes
   Unmountable if super total_bytes is smaller than total rw bytes of
   all devices.
   Root cause seems to be that, at least for kernel v4.13, some
   device resize/add operation on unaligned device can lead to super
   size out of sync with device size.
   And normally this will cause super size to be 1 page (4096) smaller
   than all devices' size.

   This patchset provides the tool to fix it offline.
   (At least better than unmountable forever)

2) Harmless kernel warning for btrfs_update_device()
   v4.14 introduced restrict device size checker.
   This is causing harmless but annoying kernel warnning.

   It can be fixed online with "btrfs filesystem resize".

   This patchset also provide a fallback method to fix it.

Finally, fix a typo which makes "btrfs rescue zero-log" to continue
execution even the fs is mounted and error message outputted.

Changelog:
v2:
  Move the command line tool from "btrfs check" to "btrfs rescue",
  suggested by David. Although fsck can still repair/detect it.
  Function naming change for cmds-check.c part, suggested by Nikolay.
  Remove inaccurate kernel version, suggested by Nikolay.
  Fix a bug when coding in cmds-rescue.c.
  Split the first patch to make review easier.

Qu Wenruo (6):
  btrfs-progs: Introduce function to fix unaligned device size
  btrfs-progs: Introduce function to fix super block total bytes
  btrfs-progs: rescue: Introduce fix-device-size
  btrfs-progs: check: Also check and repair unalignment/mismatch device
    and super size
  btrfs-progs: test/fsck: Add test case image for --fix-dev-size
  btrfs-progs: rescue: Fix zero-log mounted branch

 Documentation/btrfs-rescue.asciidoc                |  29 ++++
 cmds-check.c                                       |  76 +++++++++
 cmds-rescue.c                                      |  49 ++++++
 .../dev_and_super_mismatch_unaligned.raw.xz        | Bin 0 -> 21536 bytes
 volumes.c                                          | 178 +++++++++++++++++++++
 volumes.h                                          |   4 +
 6 files changed, 336 insertions(+)
 create mode 100644 tests/fsck-tests/027-unaligned-super-dev-sizes/dev_and_super_mismatch_unaligned.raw.xz

-- 
2.14.2

[PATCH v2 1/6] btrfs-progs: Introduce function to fix unaligned device size

[Qu Wenruo]

Recent kernel introduced alignment check for dev item, however older
kernel doesn't align device size when adding new device or shrinking
existing device.

This makes noisy kernel warning every time when any DEV_ITEM get updated.

Introduce function to fix device size in btrfs-progs to fix the problem
offline.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 volumes.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 volumes.h |  2 ++
 2 files changed, 73 insertions(+)

diff --git a/volumes.c b/volumes.c
index 2209e5a9100b..5462bf71c611 100644
--- a/volumes.c
+++ b/volumes.c
@@ -2332,3 +2332,74 @@ u64 btrfs_stripe_length(struct btrfs_fs_info *fs_info,
 	}
 	return stripe_len;
 }
+
+/*
+ * Return 0 if size of @device is already good
+ * Return >0 if size of @device is not aligned but fixed without problem
+ * Return <0 if something wrong happened when aligning the size of @device
+ */
+int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
+			  struct btrfs_device *device)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key key;
+	struct btrfs_path path;
+	struct btrfs_root *chunk_root = fs_info->chunk_root;
+	struct btrfs_dev_item *di;
+	u64 old_bytes = device->total_bytes;
+	int ret;
+
+	if (IS_ALIGNED(old_bytes, fs_info->sectorsize))
+		return 0;
+
+	/* Align the in-memory total_bytes first, and use it as correct size */
+	device->total_bytes = round_down(device->total_bytes,
+					 fs_info->sectorsize);
+
+	key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
+	key.type = BTRFS_DEV_ITEM_KEY;
+	key.offset = device->devid;
+
+	trans = btrfs_start_transaction(chunk_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		error("error starting transaction: %d (%s)",
+		      ret, strerror(-ret));
+		return ret;
+	}
+
+	btrfs_init_path(&path);
+	ret = btrfs_search_slot(trans, chunk_root, &key, &path, 0, 1);
+	if (ret > 0) {
+		error("failed to find DEV_ITEM for devid %llu",
+			device->devid);
+		ret = -ENOENT;
+		goto err;
+	}
+	if (ret < 0) {
+		error("failed to search chunk root: %d (%s)",
+			ret, strerror(-ret));
+		goto err;
+	}
+	di = btrfs_item_ptr(path.nodes[0], path.slots[0],
+			    struct btrfs_dev_item);
+	btrfs_set_device_total_bytes(path.nodes[0], di, device->total_bytes);
+	btrfs_mark_buffer_dirty(path.nodes[0]);
+	ret = btrfs_commit_transaction(trans, chunk_root);
+	if (ret < 0) {
+		error("failed to commit current transaction: %d (%s)",
+			ret, strerror(-ret));
+		btrfs_release_path(&path);
+		return ret;
+	}
+	btrfs_release_path(&path);
+	printf("Fixed device size for devid %llu, old size: %llu new size: %llu\n",
+		device->devid, old_bytes, device->total_bytes);
+	return 1;
+
+err:
+	/* We haven't modified anything, it's OK to commit current trans */
+	btrfs_commit_transaction(trans, chunk_root);
+	btrfs_release_path(&path);
+	return ret;
+}
diff --git a/volumes.h b/volumes.h
index d35a4e652b0d..89bd4aa81ae0 100644
--- a/volumes.h
+++ b/volumes.h
@@ -245,4 +245,6 @@ int btrfs_check_chunk_valid(struct btrfs_fs_info *fs_info,
 u64 btrfs_stripe_length(struct btrfs_fs_info *fs_info,
 			struct extent_buffer *leaf,
 			struct btrfs_chunk *chunk);
+int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
+			  struct btrfs_device *device);
 #endif
-- 
2.14.2

[PATCH v2 2/6] btrfs-progs: Introduce function to fix super block total bytes

[Qu Wenruo]

Recent kernel (starting from v4.6) will refuse to mount if super block
total bytes is smaller than all devices' size.

This makes end user unable to do anything to their otherwise quite
healthy fs.

To fix such problem, introduce repair function in btrfs-progs to fix it
offline.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 volumes.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 volumes.h |  1 +
 2 files changed, 53 insertions(+)

diff --git a/volumes.c b/volumes.c
index 5462bf71c611..c86f8a931742 100644
--- a/volumes.c
+++ b/volumes.c
@@ -2403,3 +2403,55 @@ err:
 	btrfs_release_path(&path);
 	return ret;
 }
+
+/*
+ * Return 0 if super block total bytes matches with all devices' total_bytes
+ * Return >0 if super block total bytes mismatch but fixed without problem
+ * Return <0 if we failed to fix super block total_bytes
+ */
+int btrfs_fix_super_size(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_device *device;
+	struct list_head *dev_list = &fs_info->fs_devices->devices;
+	u64 total_bytes = 0;
+	u64 old_bytes = btrfs_super_total_bytes(fs_info->super_copy);
+	int ret;
+
+	list_for_each_entry(device, dev_list, dev_list) {
+		/*
+		 * Caller should ensure this function is called after aligning
+		 * all devices' total_bytes.
+		 */
+		if (!IS_ALIGNED(device->total_bytes, fs_info->sectorsize)) {
+			error("device %llu total_bytes %llu not aligned to %u",
+				device->devid, device->total_bytes,
+				fs_info->sectorsize);
+			return -EUCLEAN;
+		}
+		total_bytes += device->total_bytes;
+	}
+
+	if (total_bytes == old_bytes)
+		return 0;
+
+	btrfs_set_super_total_bytes(fs_info->super_copy, total_bytes);
+
+	/* Commit transaction to update all super blocks */
+	trans = btrfs_start_transaction(fs_info->tree_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		error("error starting transaction:  %d (%s)",
+		      ret, strerror(-ret));
+		return ret;
+	}
+	ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+	if (ret < 0) {
+		error("failed to commit current transaction: %d (%s)",
+			ret, strerror(-ret));
+		return ret;
+	}
+	printf("Fixed super total bytes, old size: %llu new size: %llu\n",
+		old_bytes, total_bytes);
+	return 1;
+}
diff --git a/volumes.h b/volumes.h
index 89bd4aa81ae0..d5bb5f8663a3 100644
--- a/volumes.h
+++ b/volumes.h
@@ -247,4 +247,5 @@ u64 btrfs_stripe_length(struct btrfs_fs_info *fs_info,
 			struct btrfs_chunk *chunk);
 int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
 			  struct btrfs_device *device);
+int btrfs_fix_super_size(struct btrfs_fs_info *fs_info);
 #endif
-- 
2.14.2

[PATCH v2 3/6] btrfs-progs: rescue: Introduce fix-device-size

[Qu Wenruo]

Introduce fix-device-size rescue subcommand to fix device size
alignment related problems.

Especially for people unable to mount their fs with super total bytes
mismatch, this tool should make their fs live again.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 Documentation/btrfs-rescue.asciidoc | 29 +++++++++++++++++++
 cmds-rescue.c                       | 48 ++++++++++++++++++++++++++++++++
 volumes.c                           | 55 +++++++++++++++++++++++++++++++++++++
 volumes.h                           |  1 +
 4 files changed, 133 insertions(+)

diff --git a/Documentation/btrfs-rescue.asciidoc b/Documentation/btrfs-rescue.asciidoc
index 24b619c67c30..815abe3c2926 100644
--- a/Documentation/btrfs-rescue.asciidoc
+++ b/Documentation/btrfs-rescue.asciidoc
@@ -73,6 +73,35 @@ the log and the filesystem may be mounted normally again. The keywords to look
 for are 'open_ctree' which says that it's during mount and function names
 that contain 'replay', 'recover' or 'log_tree'.
 
+*fix-device-size* <device>::
+fix device size and super block total bytes
++
+This command will fix the following problems, by re-aligning all devices' total
+bytes and re-calculating super block total bytes.
++
+1. Newer kernel refuse to mount btrfs caused by mismatch super block total bytes
++
+----
+BTRFS error (device sdb): super_total_bytes 92017859088384 mismatch with fs_devices total_rw_bytes 92017859094528
+----
++
+2. Noisy kernel warning for newer kernels
++
+----
+WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]
+----
++
+And the corresponding line is the `WARN_ON()` line below:
++
+----
+{
+	BUILD_BUG_ON(sizeof(u64) !=
+		     sizeof(((struct btrfs_dev_item *)0))->total_bytes);
+	WARN_ON(!IS_ALIGNED(val, eb->fs_info->sectorsize));
+	btrfs_set_64(eb, s, offsetof(struct btrfs_dev_item, total_bytes), val);
+}
+----
+
 EXIT STATUS
 -----------
 *btrfs rescue* returns a zero exit status if it succeeds. Non zero is
diff --git a/cmds-rescue.c b/cmds-rescue.c
index d1bec0218e68..5b51952cdc10 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -20,6 +20,7 @@
 
 #include <getopt.h>
 #include "ctree.h"
+#include "volumes.h"
 #include "transaction.h"
 #include "disk-io.h"
 #include "commands.h"
@@ -201,6 +202,51 @@ out:
 	return !!ret;
 }
 
+static const char * const cmd_rescue_fix_device_size_usage[] = {
+	"btrfs rescue fix-device-size <device>",
+	"Re-align device and super block sizes. Usable if newer kernel refuse to mount it due to mismatch super size",
+	"",
+	NULL
+};
+
+static int cmd_rescue_fix_device_size(int argc, char **argv)
+{
+	struct btrfs_fs_info *fs_info;
+	char *devname;
+	int ret;
+
+	clean_args_no_options(argc, argv, cmd_rescue_fix_device_size_usage);
+
+	if (check_argc_exact(argc, 2))
+		usage(cmd_rescue_fix_device_size_usage);
+
+	devname = argv[optind];
+	ret = check_mounted(devname);
+	if (ret < 0) {
+		error("could not check mount status: %s", strerror(-ret));
+		goto out;
+	} else if (ret) {
+		error("%s is currently mounted", devname);
+		ret = -EBUSY;
+		goto out;
+	}
+
+	fs_info = open_ctree_fs_info(devname, 0, 0, 0, OPEN_CTREE_WRITES |
+				     OPEN_CTREE_PARTIAL);
+	if (!fs_info) {
+		error("could not open btrfs");
+		ret = -EIO;
+		goto out;
+	}
+
+	ret = btrfs_fix_device_and_super_size(fs_info);
+	if (ret > 0)
+		ret = 0;
+	close_ctree(fs_info->tree_root);
+out:
+	return !!ret;
+}
+
 static const char rescue_cmd_group_info[] =
 "toolbox for specific rescue operations";
 
@@ -211,6 +257,8 @@ const struct cmd_group rescue_cmd_group = {
 		{ "super-recover", cmd_rescue_super_recover,
 			cmd_rescue_super_recover_usage, NULL, 0},
 		{ "zero-log", cmd_rescue_zero_log, cmd_rescue_zero_log_usage, NULL, 0},
+		{ "fix-device-size", cmd_rescue_fix_device_size,
+			cmd_rescue_fix_device_size_usage, NULL, 0},
 		NULL_CMD_STRUCT
 	}
 };
diff --git a/volumes.c b/volumes.c
index c86f8a931742..2235d84784b0 100644
--- a/volumes.c
+++ b/volumes.c
@@ -2455,3 +2455,58 @@ int btrfs_fix_super_size(struct btrfs_fs_info *fs_info)
 		old_bytes, total_bytes);
 	return 1;
 }
+
+/*
+ * Return 0 if all devices and super block size is good
+ * Return >0 if any device/super size problem found, but fixed
+ * Return <0 if something wrong happened during fixing
+ */
+int btrfs_fix_device_and_super_size(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_device *device;
+	struct list_head *dev_list = &fs_info->fs_devices->devices;
+	bool have_bad_value = false;
+	int ret;
+
+	/* Seed device is not support yet */
+	if (fs_info->fs_devices->seed) {
+		error("fixing device size with seed device is not supported yet");
+		return -ENOTTY;
+	}
+
+	/* All devices must be on-line before repairing */
+	if (list_empty(dev_list)) {
+		error("no device found");
+		return -ENODEV;
+	}
+	list_for_each_entry(device, dev_list, dev_list) {
+		if (device->fd == -1 || !device->writeable) {
+			error("devid %llu is missing or not writeable",
+			      device->devid);
+			error("fixing device size needs all device(s) present and writeable");
+			return -ENODEV;
+		}
+	}
+
+	/* Repair total_bytes of each device */
+	list_for_each_entry(device, dev_list, dev_list) {
+		ret = btrfs_fix_device_size(fs_info, device);
+		if (ret < 0)
+			return ret;
+		if (ret > 0)
+			have_bad_value = true;
+	}
+
+	/* Repair super total_byte */
+	ret = btrfs_fix_super_size(fs_info);
+	if (ret > 0)
+		have_bad_value = true;
+	if (have_bad_value) {
+		printf("Fixed unaligned/mismatch total_bytes for super block and device item\n");
+		ret = 1;
+	} else {
+		printf("No device size related problem found\n");
+		ret = 0;
+	}
+	return ret;
+}
diff --git a/volumes.h b/volumes.h
index d5bb5f8663a3..11572e78c04f 100644
--- a/volumes.h
+++ b/volumes.h
@@ -248,4 +248,5 @@ u64 btrfs_stripe_length(struct btrfs_fs_info *fs_info,
 int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
 			  struct btrfs_device *device);
 int btrfs_fix_super_size(struct btrfs_fs_info *fs_info);
+int btrfs_fix_device_and_super_size(struct btrfs_fs_info *fs_info);
 #endif
-- 
2.14.2

[PATCH v2 4/6] btrfs-progs: check: Also check and repair unalignment/mismatch device and super size

[Qu Wenruo]

Along with the rescue introduced, also introduce check and repair for them.

Unlike normal check functions, some of the check is optional, and even if
the image failed to pass optional check, kernel can still run fine.
(But may cause noisy kernel warning)

So some check, mainly for alignment, will not cause btrfs check to fail,
but only to output warning and how to fix it.

For repair, it just calls the same repair in rescue, and is included in
'btrfs check --repair'.
But 'btrfs rescue' is still the preferred method, since it can bypass
all the possible dangerous repair.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
---
 cmds-check.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 76 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 5c822b848608..52250b4d2420 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -9879,6 +9879,67 @@ static int check_device_used(struct device_record *dev_rec,
 	}
 }
 
+/*
+ * Extra (optional) check for dev_item size
+ *
+ * To avoid possible kernel warning for newer kernel.
+ * It's not a deadly problem, just to info newer kernel user.
+ * So it won't return any value, just info user directly.
+ */
+static void check_dev_size_alignment(u64 devid, u64 total_bytes,
+				     u32 sectorsize)
+{
+	if (!IS_ALIGNED(total_bytes, sectorsize)) {
+		warning("unaligned total_bytes detected for devid %llu, have %llu should be aligned to %u",
+			devid, total_bytes, sectorsize);
+		warning("this is OK for older kernel, but may cause kernel warning for newer kernel");
+		warning("this can be fixed by 'btrfs rescue fix-device-size'");
+	}
+}
+
+/*
+ * Unlike device size alignment check above, some super total_bytes check
+ * failure can lead to mount failure for newer kernel.
+ *
+ * So this function will return the error for fatal super total_bytes problem.
+ */
+static bool is_super_size_valid(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_device *dev;
+	struct list_head *dev_list = &fs_info->fs_devices->devices;
+	u64 total_bytes = 0;
+	u64 super_bytes = btrfs_super_total_bytes(fs_info->super_copy);
+
+	list_for_each_entry(dev, dev_list, dev_list)
+		total_bytes += dev->total_bytes;
+
+	/* Important check, which can cause unmountable fs */
+	if (super_bytes < total_bytes) {
+		error("super total bytes %llu smaller than real device(s) size %llu",
+			super_bytes, total_bytes);
+		error("mounting this fs may fail for newer kernels");
+		error("this can be fixed by 'btrfs rescue fix-device-size'");
+		return false;
+	}
+
+	/*
+	 * Optional check, just to make everything aligned and match with
+	 * each other.
+	 *
+	 * For btrfs-image restored fs, we don't need to check it anyway.
+	 */
+	if (btrfs_super_flags(fs_info->super_copy) &
+	    (BTRFS_SUPER_FLAG_METADUMP | BTRFS_SUPER_FLAG_METADUMP_V2))
+		return true;
+	if (!IS_ALIGNED(super_bytes, fs_info->sectorsize) ||
+	    !IS_ALIGNED(total_bytes, fs_info->sectorsize) ||
+	    super_bytes != total_bytes) {
+		warning("minor unaligned/mismatch device size detected");
+		warning("recommended to use 'btrfs rescue fix-device-size' to fix it");
+	}
+	return true;
+}
+
 /* check btrfs_dev_item -> btrfs_dev_extent */
 static int check_devices(struct rb_root *dev_cache,
 			 struct device_extent_tree *dev_extent_cache)
@@ -9896,6 +9957,8 @@ static int check_devices(struct rb_root *dev_cache,
 		if (err)
 			ret = err;
 
+		check_dev_size_alignment(dev_rec->devid, dev_rec->total_byte,
+					 global_info->sectorsize);
 		dev_node = rb_next(dev_node);
 	}
 	list_for_each_entry(dext_rec, &dev_extent_cache->no_device_orphans,
@@ -11141,6 +11204,7 @@ static int check_dev_item(struct btrfs_fs_info *fs_info,
 	struct btrfs_path path;
 	struct btrfs_key key;
 	struct btrfs_dev_extent *ptr;
+	u64 total_bytes;
 	u64 dev_id;
 	u64 used;
 	u64 total = 0;
@@ -11149,6 +11213,7 @@ static int check_dev_item(struct btrfs_fs_info *fs_info,
 	dev_item = btrfs_item_ptr(eb, slot, struct btrfs_dev_item);
 	dev_id = btrfs_device_id(eb, dev_item);
 	used = btrfs_device_bytes_used(eb, dev_item);
+	total_bytes = btrfs_device_total_bytes(eb, dev_item);
 
 	key.objectid = dev_id;
 	key.type = BTRFS_DEV_EXTENT_KEY;
@@ -11193,6 +11258,8 @@ next:
 			BTRFS_DEV_EXTENT_KEY, dev_id);
 		return ACCOUNTING_MISMATCH;
 	}
+	check_dev_size_alignment(dev_id, total_bytes, fs_info->sectorsize);
+
 	return 0;
 }
 
@@ -11757,6 +11824,12 @@ static int do_check_chunks_and_extents(struct btrfs_fs_info *fs_info)
 	else
 		ret = check_chunks_and_extents(fs_info);
 
+	/* Also repair device size related problems */
+	if (repair && !ret) {
+		ret = btrfs_fix_device_and_super_size(fs_info);
+		if (ret > 0)
+			ret = 0;
+	}
 	return ret;
 }
 
@@ -13256,6 +13329,9 @@ int cmd_check(int argc, char **argv)
 		error(
 		"errors found in extent allocation tree or chunk allocation");
 
+	/* Only re-check super size after we checked and repaired the fs */
+	err |= !is_super_size_valid(info);
+
 	ret = repair_root_items(info);
 	err |= !!ret;
 	if (ret < 0) {
-- 
2.14.2

[PATCH v2 5/6] btrfs-progs: test/fsck: Add test case image for --fix-dev-size

[Qu Wenruo]

From: Qu Wenruo <quwenruo.btrfs@gmx.com>

The image has 2 problems mixed:

1) Too small super total_bytes
   This super total_bytes is manually modified to create such problem.

2) Unaligned dev item total_bytes
   This is created by v4.12 kernel, with 128M + 2K device added, and
   original device removed.
   Then we can create such image with unaligned dev item total_bytes.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
---
 .../dev_and_super_mismatch_unaligned.raw.xz             | Bin 0 -> 21536 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/027-unaligned-super-dev-sizes/dev_and_super_mismatch_unaligned.raw.xz

diff --git a/tests/fsck-tests/027-unaligned-super-dev-sizes/dev_and_super_mismatch_unaligned.raw.xz b/tests/fsck-tests/027-unaligned-super-dev-sizes/dev_and_super_mismatch_unaligned.raw.xz
new file mode 100644
index 0000000000000000000000000000000000000000..153e514a89d5f50a7e6b4f9c3d3214896a4070cd
GIT binary patch
literal 21536
zcmeI4XIN8d7RM6^O{57*lP*o^y(5T75s)4_2uSZmrAd*16zK>K7>W=;sUj#fA{`P0
zl_Jt45IV|8lalq>-FbFqlyzoTefV&{<w-u=oA;jczyI?)?|Jtm1E8iL5M*|+S``n(
z20jS_fhhZ+y*M14sK){X^4P=S@@sJy6x5D;OWWu^N#>c&hY@YBV3@;S3bx6}fi){b
zx?*Cq1oO#8ZWK<(aZu?`n$WfR#k{+^Rg-$-s55fhSKZDWzB+29Leoco59gckqI$ne
zXqe$v;@mTR-7@QAoGk*8Tt%{;zm{>4F@VBFB4sQkt;EMp8>1II5(Beim`-kRHEJZ3
z&}d@XGR=Z&bb>aJx2$GPQY|D^c672tTuWv4sb?%nb{KMg+MhsOW%=slLT%FND*IUr
z{~+EH<f1rqb$8n7Zl;fZk^Q`&*CMfv%FSLVyRo=f_}0fkr#17XP=C7$z0grUS${OD
zsEk|l@y7<C?rOR^l9*$eIQp}Xf}3!$rg*fRGK1IklFD+R9c&rDyf&$t-eY{iScoks
z4%gT#k!VS@$8Hi4K#qf4E^t~Sc+eCu<JmUjM_hxb@<35ImdbI;SO99YPQy+n?=Zv>
znvEpP(8|~wE`Oa!(aNd6wVos)Oi)m<;_=K=>%!6gB&w29Yo<DMB{!Y932gp~9<*uR
z|Dg$H#*Bb?yW>si?vCCWUlg+$tMc}kzWf%i9f@pc&uQ0*$FlER8wIRNC=@iWC?`^r
z<0w<MysnvSkqjasiGtM<WHP4Bi7%DI3PVOYq|+p$%V_k7R+T3WX?yfqDKbgx>sj!f
z?&T19k{IRAxJICn(y!h|@iXDA<T<*W$emIG$3}E#FS06*(u$r891-KxxmFaJ!!W9u
z<;Ayl(Y8#vpDnnHfIbC{TFcF9%u&oQ$BW0}W=I?+g$)~ZCMG=8>6eWKl)SVVC8gG>
z1yXz~10D8Lq{#c<86@UKMifp+m<%8*NSL%A1ZcMo%u%<f_~G~={gQ9{KkaNwndF#+
zsVl4v1zpP<<yDgDlSKAPYx8`td7P@I`;ON~?A&wxUTCX8oKq6>ZNXz+jlAMYV3DDH
z%G}=KRGX={C}t_nQD+j?PJ3%mLR?rh$!NL6^*fGuyRZReGu!@+;&Q_IsfhJh^Yur%
z>N5;FNn%KZM-vOWqC?G)JO?KoKzx)e?2RFYwrPMn@PvrMM~=EcL)FxVi5g`#LE)Fo
z1T_39tm?rPRPV*O4cOzJXs3*WlWSv&3g(M%G7K;CI>Khu8Iz?~%ow#`BX4k`iFR3!
ze3*hUYx5-@*2@KW@eG@!T~TokQrOtbSTiX632EaHDk`^6cR;zbZ3%5UJbbBW!}$?~
zSf;=BsEj4LHmrvD)MJ?0ZTbdg4)~>Z$t`F3a@v}j*!DJJRSf<oA~~$IRSpXaCv6IY
z*2cp?w&lrg5&vZM$P=<wC1$seP6V$tM8W0k^I5qCBbsaoAXN>SSoQ_O56AYVd<)~N
zuka^KGp0*d3hmCUnNylwJ&k+NbRTOV?vyCpk<mhNR<pL=apJ^$FFe`|A+6@ped3jI
zhDa@IBTY+2hDKnMg=WA+Ir-9HV!YAuUlrOWgpD9){2M>kHLK5CNWHThd0O(U)zwH>
zZT=G}bAnFuTAcZq=9LEv<9=K%8b<Q?XW4EQb&GFl^XBRqBz5L-K4-3u6x|}wM`Pp)
z-nP7Fp1Wh77g|fKWsdG@FuOI479;UAdy--Kl({Va>_tkCIqzH-6zD<(T$Hb3u{D4i
zTsmW)+2IxtHw;rcC3t-sE~ZV`ymIW-Q}KnAo%cQb0u;N)N#>5}52N~nw`WMtb8+`a
zZfRul<2c0GC&iYBjOd2th>~+50?R><Q1M6YoK8K-whPVeMOxQ|70e(BuAnDGxPu(n
zR=eF}3*GLYrfxoF+0Y^EuHo}$Ad2$_(FZ5bLiPLw_RUGWs$j?X$+f}0{VKn4f!VAh
z!Xwq^h{CNjpBLtL^<oin0>kUsq|?N7$jLF5t6(xZetT{Vah`VgJxoM}EbXAAjMr%E
z#AeCfM<&g6Rg}+SLv@rMR_*HjHT0-TYHtH9plNLx`Qp5<w>yhcGoqE9XWSSCEftKy
zODP_XS-G{ix5?*<2*{pG7Ty_YrnjA>wT>|qL^XAs+f*Nxxwam{=bZFJXKT17HB2u}
z_e3oFI}TX-PT_e(joP&>K~<*MChOawi3YjL;i{9F+$|D2Qg%AesFhz<?nc2D;&yOg
zmob^?@vL?~>a}J0o{)7DWG^4p?LpLem#C++8aK%nVCpM#WR)(3<^we@a(pS>+@s0l
zk?N(T6Ow}C&ImV^WhFB&9r?8Rwi@1;Yr^j5<fHn9;pRHOsIAO_^yTQNyoUHSOJ`YR
zV3w}K;-0B^Re|*{EiEcf@eta#SD3U%&GyRGI0}zHklJ&GLF26Pcq|acl;GzOav|bY
zMW-Z-z9VU&NQl(i`$cT6&qk8*M?WN8|8N$Z<MT<ECa?ifzd8yjI)WEb7=ANkTcg(R
zDtlikmWyKEiU-7U@$S^qnTe%d3DJQ1{_-QvQ&xQJviG%=u$PF*Iam8AI|C$j@~{^Y
zSdZ#7P4LCMZ?7E7BP<y3sU_MMbJW(fQhCULHk9NzBE@d<E5bbZxk-9MXO|F?4K^%C
z_M+p39M#xW@4`>J+2?~^TP(tqspTDVRJJ7ft;ZI<t1zk?2JfT&_=HAVE*T`T-yy`e
z>hhGOL5fIb6dVT^%;l4uUMj(vay|4udLkLoB%N?-%JUwA!)l4AJ5Ih3j-v#h0WJCH
zBleogf`l31weiC4|NgxvvOg^&CggO|8n@pj{m*?bk)m*EJgOAMyFueScZCFukO`5$
zMuZ{lf5I#A;7jXLC{_Ur_JxN47VK}Du5T>Z{x(?TP^KiW7fA^&-xo})*e++eph^zD
z8_Rq|;dp39$$#~j2mjsgW^eBW!1sX61TypgEr$*cRY3c{T0ee!!T1iNl%a{_>*~DA
z?*aDWFLHu|pYwOq@PiP=!IuVd@_)=p!14b9f&mfseGM#7Q~nvV0ucs8*pCJI--@t&
z@YZ28VyFd`wk%`n1fy-WrCCIJ>5@8EeY<N5gfN=ip@Kb%t#0cahE9Fuas}GqT)y=D
zsX5mOIP<FZBMv*qG)!SsF-D!uE!}kHrduQ}A;X$$z?&JR3@HAyo;<fVEM)%eXKa?k
zPKin74jcSpwyBkBW7~($?T%emvgLtT@snaDu~&E$ZB;NU_#xc7MvTxK6-qlx4Z$)H
z6Y!4Kz7k#<@yZ7JoKqs>>+mbW^0u;2NN9UoWDISv3KBei`EmvC+Hx&Tt3Xvfl4ia|
zoRTDSbzs2(^Q^*MRCR!yFZVIMsiE{#W1fPMeTdq}7rsUjSrGT$#*w+0(mVV(fyKMu
zv5|Ao^Uod%7XT__pfUz3<3om#6TzZ~QjkBInBt>e1Vy@>zXa`+s0w2Zl6aGHXXDlJ
zo|Yc5dS{W!5HD3PVh*qywXv8EGyErZ4K^FC?1}*`1GEg#vO}h2l;Fcj`9Kuzm`$VW
zMWP9pZw8y{UQYH^H*a;TO2+1NqKP0Y@bLoWo@hmLP{IrCaxR`!FBbX#bVq+w_w$mj
zI|U$OfQSJicIZS5z@h*a{eL!79#n(3asr(Spi}WZw(EW?!Y059pv8Nm)M4oknIVjt
z5fYQgj0JLA<VflbvL<hA+H}cR_JRE<r}NeQ$6aUpwHft}`1nWHrW-%3(VB8%^4QUg
zrgZ)JPV9^OJHune4_Gt6ngP}fux5W@cH&?$2KGe!&DTA!V8DX?SbqdW7!YAVgaMNr
z{~99stsXy=*cye)6mmj#K+bczk|kIVL^zq;zsPa{w_i$NgHnVF<;?b&Ycaku_Hiqm
z>fFE1_V}mZZF5{RFf8=t{OrLV@V5{HpdtVj0jPMW=JyVl>fcRbZhO!<6C)`2<C~9P
zAg`|+3GjPKOaG(=2Z$Iz!~h}&5HWy=egETEey_DagaHu-MA$!1rueBH$v>Tl1f=A5
z8`S}rE5KX<<_a*^KQdPUhyg$h0Ahy*h=nTIgNQ#*B8inHbk2M}=ab$%HA@7Za|;CC
cM1VsRL(hKxZo!YMo`zOd2><hE5X8jvA3Bau9smFU

literal 0
HcmV?d00001

-- 
2.14.2

[PATCH v2 6/6] btrfs-progs: rescue: Fix zero-log mounted branch

[Qu Wenruo]

Seems to be a typo that, in (ret > 0) branch of check_mounted(),
zero-log set the return value but doesn't return.

Fix it by adding back the missing return.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 cmds-rescue.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/cmds-rescue.c b/cmds-rescue.c
index 5b51952cdc10..c40088ad374e 100644
--- a/cmds-rescue.c
+++ b/cmds-rescue.c
@@ -178,6 +178,7 @@ static int cmd_rescue_zero_log(int argc, char **argv)
 	} else if (ret) {
 		error("%s is currently mounted", devname);
 		ret = -EBUSY;
+		goto out;
 	}
 
 	root = open_ctree(devname, 0, OPEN_CTREE_WRITES | OPEN_CTREE_PARTIAL);
-- 
2.14.2

Re: [PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

[Nikolay Borisov]

On 17.10.2017 12:13, Qu Wenruo wrote:
> The patchset can be fetched from github:
> https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev
> 
> There are several reports in mail list for btrfs device size related
> problems.
> 
> 1) kernel refuse to mount some fs, due to mismatched super total_bytes
>    Unmountable if super total_bytes is smaller than total rw bytes of
>    all devices.
>    Root cause seems to be that, at least for kernel v4.13, some
>    device resize/add operation on unaligned device can lead to super
>    size out of sync with device size.
>    And normally this will cause super size to be 1 page (4096) smaller
>    than all devices' size.
> 
>    This patchset provides the tool to fix it offline.
>    (At least better than unmountable forever)
> 
> 2) Harmless kernel warning for btrfs_update_device()
>    v4.14 introduced restrict device size checker.
>    This is causing harmless but annoying kernel warnning.
> 
>    It can be fixed online with "btrfs filesystem resize".
> 
>    This patchset also provide a fallback method to fix it.
> 
> Finally, fix a typo which makes "btrfs rescue zero-log" to continue
> execution even the fs is mounted and error message outputted.
> 
> Changelog:
> v2:
>   Move the command line tool from "btrfs check" to "btrfs rescue",
>   suggested by David. Although fsck can still repair/detect it.
>   Function naming change for cmds-check.c part, suggested by Nikolay.
>   Remove inaccurate kernel version, suggested by Nikolay.
>   Fix a bug when coding in cmds-rescue.c.
>   Split the first patch to make review easier.
> 

For the whole series:

Reviewed-by: Nikolay Borisov <nborisov@suse.com>

> Qu Wenruo (6):
>   btrfs-progs: Introduce function to fix unaligned device size
>   btrfs-progs: Introduce function to fix super block total bytes
>   btrfs-progs: rescue: Introduce fix-device-size
>   btrfs-progs: check: Also check and repair unalignment/mismatch device
>     and super size
>   btrfs-progs: test/fsck: Add test case image for --fix-dev-size
>   btrfs-progs: rescue: Fix zero-log mounted branch
> 
>  Documentation/btrfs-rescue.asciidoc                |  29 ++++
>  cmds-check.c                                       |  76 +++++++++
>  cmds-rescue.c                                      |  49 ++++++
>  .../dev_and_super_mismatch_unaligned.raw.xz        | Bin 0 -> 21536 bytes
>  volumes.c                                          | 178 +++++++++++++++++++++
>  volumes.h                                          |   4 +
>  6 files changed, 336 insertions(+)
>  create mode 100644 tests/fsck-tests/027-unaligned-super-dev-sizes/dev_and_super_mismatch_unaligned.raw.xz
> 

Re: [PATCH v2 6/6] btrfs-progs: rescue: Fix zero-log mounted branch

[David Sterba]

On Tue, Oct 17, 2017 at 05:13:12PM +0800, Qu Wenruo wrote:
> Seems to be a typo that, in (ret > 0) branch of check_mounted(),
> zero-log set the return value but doesn't return.
> 
> Fix it by adding back the missing return.
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>

Applied, thanks.

Re: [PATCH v2 0/6] Btrfs-progs: rescue: To fix device related

[David Sterba]

On Tue, Oct 17, 2017 at 05:13:06PM +0800, Qu Wenruo wrote:
> The patchset can be fetched from github:
> https://github.com/adam900710/btrfs-progs/tree/check_unaligned_dev
> 
> There are several reports in mail list for btrfs device size related
> problems.
> 
> 1) kernel refuse to mount some fs, due to mismatched super total_bytes
>    Unmountable if super total_bytes is smaller than total rw bytes of
>    all devices.
>    Root cause seems to be that, at least for kernel v4.13, some
>    device resize/add operation on unaligned device can lead to super
>    size out of sync with device size.
>    And normally this will cause super size to be 1 page (4096) smaller
>    than all devices' size.
> 
>    This patchset provides the tool to fix it offline.
>    (At least better than unmountable forever)
> 
> 2) Harmless kernel warning for btrfs_update_device()
>    v4.14 introduced restrict device size checker.
>    This is causing harmless but annoying kernel warnning.
> 
>    It can be fixed online with "btrfs filesystem resize".
> 
>    This patchset also provide a fallback method to fix it.
> 
> Finally, fix a typo which makes "btrfs rescue zero-log" to continue
> execution even the fs is mounted and error message outputted.
> 
> Changelog:
> v2:
>   Move the command line tool from "btrfs check" to "btrfs rescue",
>   suggested by David. Although fsck can still repair/detect it.
>   Function naming change for cmds-check.c part, suggested by Nikolay.
>   Remove inaccurate kernel version, suggested by Nikolay.
>   Fix a bug when coding in cmds-rescue.c.
>   Split the first patch to make review easier.
> 
> Qu Wenruo (6):
>   btrfs-progs: Introduce function to fix unaligned device size
>   btrfs-progs: Introduce function to fix super block total bytes
>   btrfs-progs: rescue: Introduce fix-device-size
>   btrfs-progs: check: Also check and repair unalignment/mismatch device
>     and super size
>   btrfs-progs: test/fsck: Add test case image for --fix-dev-size
>   btrfs-progs: rescue: Fix zero-log mounted branch

Applied, thanks.

>  Documentation/btrfs-rescue.asciidoc                |  29 ++++
>  cmds-check.c                                       |  76 +++++++++
>  cmds-rescue.c                                      |  49 ++++++
>  .../dev_and_super_mismatch_unaligned.raw.xz        | Bin 0 -> 21536 bytes
>  volumes.c                                          | 178 +++++++++++++++++++++
>  volumes.h                                          |   4 +

The repair functions do not belong to volumes.c, but we need to move and
split cmds-check.c first so I'm going to apply it as-is.

I've also extended the test to exercise fix-device-size.