[PATCH 0/3] btrfs-progs: add check and repair ability for shrunk device item

[PATCH 0/3] btrfs-progs: add check and repair ability for shrunk device item

[Qu Wenruo]

There is a bug report that, a btrfs with shrunk underlying device will
be rejected by kernel, but btrfs-check will report nothing wrong.

Furthermore for such case, even if there is no dev extent in the shrunk
range, we have no way to fix it.
Kernel will refuse to mount, thus no way to shrink the device using
"btrfs dev resize".

This patch will:

- Add check ability to report such mismatch

- Add rescue ability to reset the total_bytes in dev items
  This can only be done if there is no dev extent in the shrunk range.

- Add a test case for above behavior

Qu Wenruo (3):
  btrfs-progs: check: verify the underlying block device size is valid
  btrfs-progs: fsck-tests: add test case for shrunk device
  btrfs-progs: rescue: allow fix-device-size to shrink device item

 check/main.c                                 |  28 ++++
 check/mode-lowmem.c                          |  22 +++
 kernel-shared/volumes.c                      | 135 +++++++++++++++++--
 tests/fsck-tests/059-shrinked-device/test.sh |  32 +++++
 4 files changed, 208 insertions(+), 9 deletions(-)
 create mode 100755 tests/fsck-tests/059-shrinked-device/test.sh

-- 
2.37.2

[PATCH 1/3] btrfs-progs: check: verify the underlying block device size is valid

[Qu Wenruo]

[BUG]
There is a bug report that, one btrfs got its underlying device shrinked
accidently.

Fortunately the user has no data at the truncated range.

However kernel will reject such fs, while btrfs-check reports nothing
wrong with it.

This can be easily reproduced by:

  # truncate -s 1G test.img
  # mkfs.btrfs test.img
  # truncate -s 996M test.img
  # btrfs check test.img
  Opening filesystem to check...
  Checking filesystem on test.img
  UUID: dbf0a16d-f158-4383-9025-29d7f4c43f17
  [1/7] checking root items
  [2/7] checking extents
  [3/7] checking free space tree
  [4/7] checking fs roots
  [5/7] checking only csums items (without verifying data)
  [6/7] checking root refs
  [7/7] checking quota groups skipped (not enabled on this FS)
  found 16527360 bytes used, no error found
                             ^^^^^^^^^^^^^^
  total csum bytes: 13836
  total tree bytes: 2359296
  total fs tree bytes: 2162688
  total extent tree bytes: 65536
  btree space waste bytes: 503569
  file data blocks allocated: 14168064
   referenced 14168064

[CAUSE]
Btrfs check really only check the metadata cross references, not really
bothering if the underly device has correct size.

Thus we completely ignored such size mismatch.

[FIX]
For both regular and lowmem mode, add extra check against the underlying
block device size.
If the block device size is smaller than its total_bytes, gives a error
message and error out.

Now the check looks like this for both modes:

  ...
  [2/7] checking extents
  ERROR: block device size is smaller than total_bytes in device item, has 1046478848 expect >= 1073741824
  ERROR: errors found in extent allocation tree or chunk allocation
  [3/7] checking free space tree
  ...
  found 16527360 bytes used, error(s) found

Issue: #504
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c        | 28 ++++++++++++++++++++++++++++
 check/mode-lowmem.c | 22 ++++++++++++++++++++++
 2 files changed, 50 insertions(+)

diff --git a/check/main.c b/check/main.c
index 0c5716a51ad1..dc5b4e57140c 100644
--- a/check/main.c
+++ b/check/main.c
@@ -33,6 +33,7 @@
 #include "kernel-shared/disk-io.h"
 #include "kernel-shared/print-tree.h"
 #include "common/task-utils.h"
+#include "common/device-utils.h"
 #include "kernel-shared/transaction.h"
 #include "common/utils.h"
 #include "cmds/commands.h"
@@ -91,6 +92,8 @@ struct device_record {
 	u64 byte_used;
 
 	u64 real_used;
+
+	bool bad_block_dev_size;
 };
 
 static int compare_data_backref(struct rb_node *node1, struct rb_node *node2)
@@ -5285,6 +5288,7 @@ static int process_chunk_item(struct cache_tree *chunk_cache,
 static int process_device_item(struct rb_root *dev_cache,
 		struct btrfs_key *key, struct extent_buffer *eb, int slot)
 {
+	struct btrfs_device *device;
 	struct btrfs_dev_item *ptr;
 	struct device_record *rec;
 	int ret = 0;
@@ -5308,7 +5312,29 @@ static int process_device_item(struct rb_root *dev_cache,
 	rec->devid = btrfs_device_id(eb, ptr);
 	rec->total_byte = btrfs_device_total_bytes(eb, ptr);
 	rec->byte_used = btrfs_device_bytes_used(eb, ptr);
+	rec->bad_block_dev_size = false;
+
+	device = btrfs_find_device_by_devid(gfs_info->fs_devices, rec->devid, 0);
+	if (device && device->fd >= 0) {
+		struct stat st;
+		u64 block_dev_size;
 
+		ret = fstat(device->fd, &st);
+		if (ret < 0) {
+			warning(
+		"unable to open devid %llu, skipping its block device size check",
+				device->devid);
+			goto skip;
+		}
+		block_dev_size = btrfs_device_size(device->fd, &st);
+		if (block_dev_size < rec->total_byte) {
+			error(
+"block device size is smaller than total_bytes in device item, has %llu expect >= %llu",
+			      block_dev_size, rec->total_byte);
+			rec->bad_block_dev_size = true;
+		}
+	}
+skip:
 	ret = rb_insert(dev_cache, &rec->node, device_record_compare);
 	if (ret) {
 		fprintf(stderr, "Device[%llu] existed.\n", rec->devid);
@@ -8737,6 +8763,8 @@ static int check_devices(struct rb_root *dev_cache,
 
 		check_dev_size_alignment(dev_rec->devid, dev_rec->total_byte,
 					 gfs_info->sectorsize);
+		if (dev_rec->bad_block_dev_size && !ret)
+			ret = 1;
 		dev_node = rb_next(dev_node);
 	}
 	list_for_each_entry(dext_rec, &dev_extent_cache->no_device_orphans,
diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
index fa324506dd5d..051c52626434 100644
--- a/check/mode-lowmem.c
+++ b/check/mode-lowmem.c
@@ -23,6 +23,7 @@
 #include "kernel-shared/backref.h"
 #include "common/internal.h"
 #include "common/utils.h"
+#include "common/device-utils.h"
 #include "kernel-shared/volumes.h"
 #include "check/mode-common.h"
 #include "check/mode-lowmem.h"
@@ -4495,6 +4496,9 @@ static int check_dev_item(struct extent_buffer *eb, int slot,
 	struct btrfs_path path;
 	struct btrfs_key key;
 	struct btrfs_dev_extent *ptr;
+	struct btrfs_device *dev;
+	struct stat st;
+	u64 block_dev_size;
 	u64 total_bytes;
 	u64 dev_id;
 	u64 used;
@@ -4590,6 +4594,24 @@ next:
 	}
 	check_dev_size_alignment(dev_id, total_bytes, gfs_info->sectorsize);
 
+	dev = btrfs_find_device_by_devid(gfs_info->fs_devices, dev_id, 0);
+	if (!dev || dev->fd < 0)
+		return 0;
+
+	ret = fstat(dev->fd, &st);
+	if (ret < 0) {
+		warning(
+	"unable to open devid %llu, skipping its block device size check",
+			dev->devid);
+		return 0;
+	}
+	block_dev_size = btrfs_device_size(dev->fd, &st);
+	if (block_dev_size < total_bytes) {
+		error(
+"block device size is smaller than total_bytes in device item, has %llu expect >= %llu",
+		      block_dev_size, total_bytes);
+		return ACCOUNTING_MISMATCH;
+	}
 	return 0;
 }
 
-- 
2.37.2

[PATCH 2/3] btrfs-progs: fsck-tests: add test case for shrunk device

[Qu Wenruo]

The test case just create a btrfs on a file backed loop block device,
then shrink the file (and its loop block device), then make sure btrfs
check can detect such shrunk device.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/fsck-tests/059-shrinked-device/test.sh | 32 ++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100755 tests/fsck-tests/059-shrinked-device/test.sh

diff --git a/tests/fsck-tests/059-shrinked-device/test.sh b/tests/fsck-tests/059-shrinked-device/test.sh
new file mode 100755
index 000000000000..672219f9c3f3
--- /dev/null
+++ b/tests/fsck-tests/059-shrinked-device/test.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+#
+# Make sure "btrfs check" can detect device smaller than its total_bytes
+# in device item
+#
+
+source "$TEST_TOP/common"
+
+setup_root_helper
+
+file="img"
+# Allocate an initial 1G file for testing.
+truncate -s0 "$file"
+truncate -s1g "$file"
+
+dev=$(run_check_stdout $SUDO_HELPER losetup --find --show "$file")
+
+run_check "$TOP/mkfs.btrfs" -f "$dev"
+
+
+# The original device size from prepare_loopdevs is 2G.
+# Since the fs is empty, shrinking it to 996m will not cause any
+# lose of metadata.
+run_check $SUDO_HELPER losetup -d "$dev"
+truncate -s 996m "$file"
+dev=$(run_check_stdout $SUDO_HELPER losetup --find --show "$file")
+
+run_mustfail "btrfs check should detect errors in device size" \
+	"$TOP/btrfs" check "$dev"
+
+losetup -d "$dev"
+rm -- "$file"
-- 
2.37.2

[PATCH 3/3] btrfs-progs: rescue: allow fix-device-size to shrink device item

[Qu Wenruo]

If we found that the underlying block device size is smaller than
total_bytes in dev item, kernel will reject the mount, and there is no
progs tool to fix it.

Under most case it's just a small mismatch, and there is no dev extent
in the shrunk range.

In that case, we can let "btrfs rescue fix-device-size" to reset the
total_bytes in dev items to fix.

We add some extra checks, like to make sure there is no dev extent in
the shrunk device range, to make sure we won't lose data during the
device item shrink.

And also update the test case to verify the repaired fs can pass the
check.

Issue: #504
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 kernel-shared/volumes.c | 135 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 126 insertions(+), 9 deletions(-)

diff --git a/kernel-shared/volumes.c b/kernel-shared/volumes.c
index 40032a4b4059..af764b7ae915 100644
--- a/kernel-shared/volumes.c
+++ b/kernel-shared/volumes.c
@@ -2786,12 +2786,63 @@ u64 btrfs_stripe_length(struct btrfs_fs_info *fs_info,
 }
 
 /*
- * Return 0 if size of @device is already good
- * Return >0 if size of @device is not aligned but fixed without problems
- * Return <0 if something wrong happened when aligning the size of @device
+ * Return <0 for error.
+ * Return >0 if we can not find any dev extent beyond @physical
+ * REturn 0 if we can find any dev extent beyond @physical or covers @physical.
  */
-int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
-			  struct btrfs_device *device)
+static int check_dev_extent_beyond_bytenr(struct btrfs_fs_info *fs_info,
+					  struct btrfs_device *device,
+					  u64 physical)
+{
+	struct btrfs_root *root = fs_info->dev_root;
+	struct btrfs_path path = { 0 };
+	struct btrfs_dev_extent *dext;
+	struct btrfs_key key;
+	u64 dext_len;
+	u64 last_dev_extent_end = 0;
+	int ret;
+
+	key.objectid = device->devid;
+	key.type = BTRFS_DEV_EXTENT_KEY;
+	key.offset = (u64)-1;
+
+	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
+	if (ret < 0)
+		return ret;
+	if (ret == 0) {
+		ret = -EUCLEAN;
+		error("invalid dev extent found for devid %llu", device->devid);
+		goto out;
+	}
+
+	ret = btrfs_previous_item(root, &path, device->devid,
+				  BTRFS_DEV_EXTENT_KEY);
+	/*
+	 * Either <0 we error out, or ret > 0 we can not find any
+	 * dev extent for this device, then last_dev_extent_end will be 0
+	 * and we will return 1.
+	 */
+	if (ret)
+		goto out;
+
+	btrfs_item_key_to_cpu(path.nodes[0], &key, path.slots[0]);
+	dext = btrfs_item_ptr(path.nodes[0], path.slots[0],
+			      struct btrfs_dev_extent);
+	dext_len = btrfs_dev_extent_length(path.nodes[0], dext);
+	last_dev_extent_end = dext_len + key.offset;
+
+out:
+	btrfs_release_path(&path);
+	if (ret < 0)
+		return ret;
+	if (last_dev_extent_end <= physical)
+		return 1;
+	return 0;
+}
+
+static int reset_device_item_total_bytes(struct btrfs_fs_info *fs_info,
+					 struct btrfs_device *device,
+					 u64 new_size)
 {
 	struct btrfs_trans_handle *trans;
 	struct btrfs_key key;
@@ -2801,12 +2852,10 @@ int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
 	u64 old_bytes = device->total_bytes;
 	int ret;
 
-	if (IS_ALIGNED(old_bytes, fs_info->sectorsize))
-		return 0;
+	ASSERT(IS_ALIGNED(new_size, fs_info->sectorsize));
 
 	/* Align the in-memory total_bytes first, and use it as correct size */
-	device->total_bytes = round_down(device->total_bytes,
-					 fs_info->sectorsize);
+	device->total_bytes = new_size;
 
 	key.objectid = BTRFS_DEV_ITEMS_OBJECTID;
 	key.type = BTRFS_DEV_ITEM_KEY;
@@ -2854,6 +2903,74 @@ err:
 	return ret;
 }
 
+static int btrfs_fix_block_device_size(struct btrfs_fs_info *fs_info,
+				       struct btrfs_device *device)
+{
+	struct stat st;
+	u64 block_dev_size;
+	int ret;
+
+	if (device->fd < 0 || !device->writeable) {
+		error("devid %llu is missing or not writable", device->devid);
+		return -EINVAL;
+	}
+
+	ret = fstat(device->fd, &st);
+	if (ret < 0) {
+		error("failed to get block device size for devid %llu: %m",
+		      device->devid);
+		return -errno;
+	}
+
+	block_dev_size = round_down(btrfs_device_size(device->fd, &st),
+				    fs_info->sectorsize);
+
+	/*
+	 * Total_bytes in device item is no larger than the device block size,
+	 * already the correct case.
+	 */
+	if (device->total_bytes <= block_dev_size)
+		return 0;
+
+	/*
+	 * Now we need to check if there is any device extent beyond
+	 * @block_dev_size.
+	 */
+	ret = check_dev_extent_beyond_bytenr(fs_info, device, block_dev_size);
+	if (ret < 0)
+		return ret;
+
+	if (ret == 0) {
+		error("there are dev extents covering or beyond bytenr %llu, can not shrink the device without losing data",
+			device->devid);
+		return -EINVAL;
+	}
+
+	/* Now we can shrink the device item total_bytes to @block_dev_size. */
+	return reset_device_item_total_bytes(fs_info, device, block_dev_size);
+}
+
+/*
+ * Return 0 if size of @device is already good
+ * Return >0 if size of @device is not aligned but fixed without problems
+ * Return <0 if something wrong happened when aligning the size of @device
+ */
+int btrfs_fix_device_size(struct btrfs_fs_info *fs_info,
+			  struct btrfs_device *device)
+{
+	u64 old_bytes = device->total_bytes;
+
+	/*
+	 * Our value is already good, then check if it's device item mismatch against
+	 * block device size.
+	 */
+	if (IS_ALIGNED(old_bytes, fs_info->sectorsize))
+		return btrfs_fix_block_device_size(fs_info, device);
+
+	return reset_device_item_total_bytes(fs_info, device,
+			round_down(old_bytes, fs_info->sectorsize));
+}
+
 /*
  * Return 0 if super block total_bytes matches all devices' total_bytes
  * Return >0 if super block total_bytes mismatch but fixed without problem
-- 
2.37.2

Re: [PATCH 0/3] btrfs-progs: add check and repair ability for shrunk device item

[David Sterba]

On Tue, Aug 30, 2022 at 02:49:41PM +0800, Qu Wenruo wrote:
> There is a bug report that, a btrfs with shrunk underlying device will
> be rejected by kernel, but btrfs-check will report nothing wrong.
> 
> Furthermore for such case, even if there is no dev extent in the shrunk
> range, we have no way to fix it.
> Kernel will refuse to mount, thus no way to shrink the device using
> "btrfs dev resize".
> 
> This patch will:
> 
> - Add check ability to report such mismatch
> 
> - Add rescue ability to reset the total_bytes in dev items
>   This can only be done if there is no dev extent in the shrunk range.
> 
> - Add a test case for above behavior
> 
> Qu Wenruo (3):
>   btrfs-progs: check: verify the underlying block device size is valid
>   btrfs-progs: fsck-tests: add test case for shrunk device
>   btrfs-progs: rescue: allow fix-device-size to shrink device item

Added to devel, thanks.