[PATCH 0/3] btrfs-progs: restore: handler newer inlined

[PATCH 0/3] btrfs-progs: restore: handler newer inlined

[Qu Wenruo]

The detailed commit message is in the first patch.

And also fix a minor problem related to very rare inlined extent that is
larger than 4K (possible only with page sizes larger than 4K).

Finally a test case for the initial problem.

Qu Wenruo (3):
  btrfs-progs: restore: handle full block compressed inline extent
  btrfs-progs: restore: handle inlined extent larger than 4K
  btrfs-progs: misc-tests: add a test case for btrfs-restore

 cmds/restore.c                                | 23 +++++++++++----
 .../073-restore-inline-compressed/test.sh     | 28 +++++++++++++++++++
 2 files changed, 46 insertions(+), 5 deletions(-)
 create mode 100755 tests/misc-tests/073-restore-inline-compressed/test.sh

--
2.55.0

[PATCH 1/3] btrfs-progs: restore: handle full block compressed inline extent

[Qu Wenruo]

Since kernel commit 3eaf5f082c4c ("btrfs: extract inlined creation into a
dedicated delalloc helper"), the creation of compressed inlined extent
has changed to compress the whole block, then truncate the ram_bytes to
isize.

This means the compressed data needs to have a full block as the output
buffer.
This is fine for kernel, as the inlined extent is alwasy decompressed to
a folio which is at least one block sized.

But for progs, if the output buffer is smaller than a block, the
decompression will fail early (not enough output buffer), causing
btrfs-restore to completely skip the content of the inlined extent.

Fix it by always allocating a full block for copy_one_inline().

Issue: #1171
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 cmds/restore.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/cmds/restore.c b/cmds/restore.c
index 7d16cc99fa6a..00471b486ac6 100644
--- a/cmds/restore.c
+++ b/cmds/restore.c
@@ -298,6 +298,7 @@ static int copy_one_inline(struct btrfs_root *root, int fd,
 	char buf[4096];
 	char *outbuf;
 	u64 ram_size;
+	u64 out_buf_len;
 	ssize_t done;
 	unsigned long ptr;
 	int ret;
@@ -324,13 +325,14 @@ static int copy_one_inline(struct btrfs_root *root, int fd,
 	}
 
 	ram_size = btrfs_file_extent_ram_bytes(leaf, fi);
-	outbuf = calloc(1, ram_size);
+	out_buf_len = root->fs_info->sectorsize;
+	outbuf = malloc(out_buf_len);
 	if (!outbuf) {
 		error_mem(NULL);
 		return -ENOMEM;
 	}
 
-	ret = decompress(root, buf, outbuf, inline_item_len, &ram_size,
+	ret = decompress(root, buf, outbuf, inline_item_len, &out_buf_len,
 			 compress);
 	if (ret) {
 		free(outbuf);
-- 
2.55.0

[PATCH 2/3] btrfs-progs: restore: handle inlined extent larger than 4K

[Qu Wenruo]

Most btrfs are already using 4K sectorsize, but for systems with 64K
page size, it's possible to create large inlined extent that is larger
than 4K:

	item 6 key (257 EXTENT_DATA 0) itemoff 56802 itemsize 8213
		generation 9 type 0 (inline)
		inline extent data size 8192 ram_bytes 8192 compression 0 (none)

In that case, btrfs-restore will crash as its input buffer is only 4K
sized.

Enhance btrfs-restore to handle larger block size correctly, by allocate
a block-sized buffer at input buffer.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 cmds/restore.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/cmds/restore.c b/cmds/restore.c
index 00471b486ac6..b982f2de4699 100644
--- a/cmds/restore.c
+++ b/cmds/restore.c
@@ -295,7 +295,8 @@ static int copy_one_inline(struct btrfs_root *root, int fd,
 {
 	struct extent_buffer *leaf = path->nodes[0];
 	struct btrfs_file_extent_item *fi;
-	char buf[4096];
+	const u32 blocksize = root->fs_info->sectorsize;
+	char *inbuf;
 	char *outbuf;
 	u64 ram_size;
 	u64 out_buf_len;
@@ -306,16 +307,23 @@ static int copy_one_inline(struct btrfs_root *root, int fd,
 	int inline_item_len;
 	int compress;
 
+	inbuf = malloc(blocksize);
+	if (!inbuf) {
+		error_mem(NULL);
+		return -ENOMEM;
+	}
+
 	fi = btrfs_item_ptr(leaf, path->slots[0],
 			    struct btrfs_file_extent_item);
 	ptr = btrfs_file_extent_inline_start(fi);
 	len = btrfs_file_extent_ram_bytes(leaf, fi);
 	inline_item_len = btrfs_file_extent_inline_item_len(leaf, path->slots[0]);
-	read_extent_buffer(leaf, buf, ptr, inline_item_len);
+	read_extent_buffer(leaf, inbuf, ptr, inline_item_len);
 
 	compress = btrfs_file_extent_compression(leaf, fi);
 	if (compress == BTRFS_COMPRESS_NONE) {
-		done = pwrite(fd, buf, len, pos);
+		done = pwrite(fd, inbuf, len, pos);
+		free(inbuf);
 		if (done < len) {
 			error("short inline write, wanted %d, did %zd: %m",
 					len, done);
@@ -329,17 +337,20 @@ static int copy_one_inline(struct btrfs_root *root, int fd,
 	outbuf = malloc(out_buf_len);
 	if (!outbuf) {
 		error_mem(NULL);
+		free(inbuf);
 		return -ENOMEM;
 	}
 
-	ret = decompress(root, buf, outbuf, inline_item_len, &out_buf_len,
+	ret = decompress(root, inbuf, outbuf, inline_item_len, &out_buf_len,
 			 compress);
 	if (ret) {
+		free(inbuf);
 		free(outbuf);
 		return ret;
 	}
 
 	done = pwrite(fd, outbuf, ram_size, pos);
+	free(inbuf);
 	free(outbuf);
 	if (done < ram_size) {
 		error("short compressed inline write, wanted %llu, did %zd: %m",
-- 
2.55.0

[PATCH 3/3] btrfs-progs: misc-tests: add a test case for btrfs-restore

[Qu Wenruo]

The new test case will:

- Create an inlined compressed file extent
- Generate checksum for that file
- Use btrfs-restore to extract that file
- Compare the extracted file with the generated checksum

The checksum should match.

This is a regression test for a behavior change in kernel, which results
older btrfs-restore unable to handle newerly created inlined compressed
file extents.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../073-restore-inline-compressed/test.sh     | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100755 tests/misc-tests/073-restore-inline-compressed/test.sh

diff --git a/tests/misc-tests/073-restore-inline-compressed/test.sh b/tests/misc-tests/073-restore-inline-compressed/test.sh
new file mode 100755
index 000000000000..e558cd372184
--- /dev/null
+++ b/tests/misc-tests/073-restore-inline-compressed/test.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+# Test "btrfs restore" on compressed inlined extents.
+
+source "$TEST_TOP/common" || exit
+source "$TEST_TOP/common.convert" || exit
+
+check_prereq mkfs.btrfs
+check_prereq btrfs
+check_global_prereq md5sum
+
+setup_root_helper
+prepare_test_dev
+
+tmp=$(_mktemp_dir "restore-destination")
+run_check_mkfs_test_dev
+run_check_mount_test_dev -o max_inline=4095
+run_check $SUDO_HELPER dd if=/dev/zero of="$TEST_MNT/foobar" bs=2k count=1 status=noxfer
+md5_before=$(run_check_stdout md5sum "$TEST_MNT/foobar" | cut -d ' ' -f 1)
+run_check_umount_test_dev
+
+run_check $SUDO_HELPER "$TOP/btrfs" restore "$TEST_DEV" "$tmp"
+md5_after=$(run_check_stdout md5sum "$tmp/foobar" | cut -d ' ' -f 1)
+
+rm -rf "$tmp"
+
+if [ "$md5_before" != "$md5_after" ]; then
+	_fail "md5 checksum mismatch"
+fi
-- 
2.55.0