[PATCH v3 0/4] btrfs-progs: support for fs-verity fstests

[PATCH v3 0/4] btrfs-progs: support for fs-verity fstests

[Boris Burkov]

Adding fstests for fs-verity on btrfs needs some light support from
btrfs-progs. Specifically, it needs additional device corruption
features to test corruption detection, and it needs the RO COMPAT flag.

The first patch defines (u64)-1 as "UNSET_U64"
The second patch adds corrupting arbitrary regions of item data with -I.
The third patch adds corrupting holes and prealloc in extent data.
The fourth patch includes BTRFS_FEATURE_RO_COMPAT_VERITY to ctree.h

--
v3: add patch #defining (u64)-1 in btrfs-corrupt-block
    check item bounds in corruption function
    improve usage message for new corruption use case
    add patch with verity ro compat flag
v2: minor cleanups from rebasing after a year  

Boris Burkov (4):
  btrfs-corrupt-block: define (u64)-1 as UNSET_U64
  btrfs-progs: corrupt generic item data with btrfs-corrupt-block
  btrfs-progs: expand corrupt_file_extent in btrfs-corrupt-block
  btrfs-progs: add VERITY ro compat flag

 btrfs-corrupt-block.c | 128 ++++++++++++++++++++++++++++++++++--------
 kernel-shared/ctree.h |   4 +-
 2 files changed, 109 insertions(+), 23 deletions(-)

-- 
2.37.1

[PATCH v3 1/4] btrfs-corrupt-block: define (u64)-1 as UNSET_U64

[Boris Burkov]

we use this placeholder for many inputs in this script, so give it a
name for clarity.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 btrfs-corrupt-block.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index e961255d5..b826c9c2e 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -35,6 +35,7 @@
 #include "common/help.h"
 
 #define FIELD_BUF_LEN 80
+#define UNSET_U64 ((u64)-1)
 
 static int debug_corrupt_block(struct extent_buffer *eb,
 		struct btrfs_root *root, u64 bytenr, u32 blocksize, u64 copy)
@@ -180,7 +181,7 @@ static int corrupt_extent(struct btrfs_trans_handle *trans,
 
 	key.objectid = bytenr;
 	key.type = (u8)-1;
-	key.offset = (u64)-1;
+	key.offset = UNSET_U64;
 
 	extent_root = btrfs_extent_root(trans->fs_info, bytenr);
 	while(1) {
@@ -664,7 +665,7 @@ static int corrupt_inode(struct btrfs_trans_handle *trans,
 
 	key.objectid = inode;
 	key.type = BTRFS_INODE_ITEM_KEY;
-	key.offset = (u64)-1;
+	key.offset = UNSET_U64;
 
 	path = btrfs_alloc_path();
 	if (!path)
@@ -880,7 +881,7 @@ static int corrupt_metadata_block(struct btrfs_fs_info *fs_info, u64 block,
 
 		root_key.objectid = root_objectid;
 		root_key.type = BTRFS_ROOT_ITEM_KEY;
-		root_key.offset = (u64)-1;
+		root_key.offset = UNSET_U64;
 
 		root = btrfs_read_fs_root(fs_info, &root_key);
 		if (IS_ERR(root)) {
@@ -1084,8 +1085,8 @@ static int corrupt_chunk_tree(struct btrfs_trans_handle *trans,
 	if (!path)
 		return -ENOMEM;
 
-	key.objectid = (u64)-1;
-	key.offset = (u64)-1;
+	key.objectid = UNSET_U64;
+	key.offset = UNSET_U64;
 	key.type = (u8)-1;
 
 	/* Here, cow and ins_len must equals 0 for the following reasons:
@@ -1193,7 +1194,7 @@ static struct btrfs_root *open_root(struct btrfs_fs_info *fs_info,
 
 	root_key.objectid = root_objectid;
 	root_key.type = BTRFS_ROOT_ITEM_KEY;
-	root_key.offset = (u64)-1;
+	root_key.offset = UNSET_U64;
 
 	root = btrfs_read_fs_root(fs_info, &root_key);
 	if (IS_ERR(root)) {
@@ -1209,8 +1210,8 @@ int main(int argc, char **argv)
 	struct btrfs_key key;
 	struct btrfs_root *root, *target_root;
 	char *dev;
-	/* chunk offset can be 0,so change to (u64)-1 */
-	u64 logical = (u64)-1;
+	/* chunk offset can be 0,so change to UNSET_U64 */
+	u64 logical = UNSET_U64;
 	int ret = 0;
 	u64 copy = 0;
 	u64 bytes = 4096;
@@ -1225,7 +1226,7 @@ int main(int argc, char **argv)
 	int should_corrupt_key = 0;
 	u64 metadata_block = 0;
 	u64 inode = 0;
-	u64 file_extent = (u64)-1;
+	u64 file_extent = UNSET_U64;
 	u64 root_objectid = 0;
 	u64 csum_bytenr = 0;
 	u64 block_group = 0;
@@ -1353,7 +1354,7 @@ int main(int argc, char **argv)
 	if (extent_rec) {
 		struct btrfs_trans_handle *trans;
 
-		if (logical == (u64)-1)
+		if (logical == UNSET_U64)
 			print_usage(1);
 		trans = btrfs_start_transaction(root, 1);
 		BUG_ON(IS_ERR(trans));
@@ -1378,7 +1379,7 @@ int main(int argc, char **argv)
 		struct btrfs_path *path;
 		int del;
 
-		if (logical == (u64)-1)
+		if (logical == UNSET_U64)
 			print_usage(1);
 		del = rand_range(3);
 		path = btrfs_alloc_path();
@@ -1420,7 +1421,7 @@ int main(int argc, char **argv)
 
 		trans = btrfs_start_transaction(root, 1);
 		BUG_ON(IS_ERR(trans));
-		if (file_extent == (u64)-1) {
+		if (file_extent == UNSET_U64) {
 			printf("corrupting inode\n");
 			ret = corrupt_inode(trans, root, inode, field);
 		} else {
@@ -1481,10 +1482,10 @@ int main(int argc, char **argv)
 	 * If we made it here and we have extent set then we didn't specify
 	 * inode and we're screwed.
 	 */
-	if (file_extent != (u64)-1)
+	if (file_extent != UNSET_U64)
 		print_usage(1);
 
-	if (logical == (u64)-1)
+	if (logical == UNSET_U64)
 		print_usage(1);
 
 	if (bytes == 0)
-- 
2.37.1

[PATCH v3 2/4] btrfs-progs: corrupt generic item data with btrfs-corrupt-block

[Boris Burkov]

btrfs-corrupt-block already has a mix of generic and specific corruption
options, but currently lacks the capacity for totally arbitrary
corruption in item data.

There is already a flag for corruption size (bytes/-b), so add a flag
for an offset and a value to memset the item with. Exercise the new
flags with a new variant for -I (item) corruption. Look up the item as
before, but instead of corrupting a field in the item struct, corrupt an
offset/size in the item data.

The motivating example for this is that in testing fsverity with btrfs,
we need to corrupt the generated Merkle tree--metadata item data which
is an opaque blob to btrfs.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 btrfs-corrupt-block.c | 77 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 74 insertions(+), 3 deletions(-)

diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index b826c9c2e..225818817 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -99,12 +99,14 @@ static void print_usage(int ret)
 	printf("\t-m   The metadata block to corrupt (must also specify -f for the field to corrupt)\n");
 	printf("\t-K <u64,u8,u64> Corrupt the given key (must also specify -f for the field and optionally -r for the root)\n");
 	printf("\t-f   The field in the item to corrupt\n");
-	printf("\t-I <u64,u8,u64> Corrupt an item corresponding to the passed key triplet (must also specify the field to corrupt and root for the item)\n");
+	printf("\t-I <u64,u8,u64> Corrupt an item corresponding to the passed key triplet (must also specify the field, or a (bytes, offset, value) tuple to corrupt and root for the item)\n");
 	printf("\t-D <u64,u8,u64> Corrupt a dir item corresponding to the passed key triplet, must also specify a field\n");
 	printf("\t-d <u64,u8,u64> Delete item corresponding to passed key triplet\n");
 	printf("\t-r   Operate on this root\n");
 	printf("\t-C   Delete a csum for the specified bytenr.  When used with -b it'll delete that many bytes, otherwise it's just sectorsize\n");
 	printf("\t--block-group OFFSET  corrupt the given block group\n");
+	printf("\t-v   Value to use for corrupting item data\n");
+	printf("\t-o   Offset to use for corrupting item data\n");
 	exit(ret);
 }
 
@@ -975,6 +977,56 @@ out:
 	return ret;
 }
 
+static int corrupt_btrfs_item_data(struct btrfs_root *root,
+				   struct btrfs_key *key,
+				   u64 bogus_offset, u64 bogus_size,
+				   char bogus_value)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path *path;
+	int ret;
+	void *data;
+	struct extent_buffer *leaf;
+	int slot;
+	u32 item_size;
+
+	path = btrfs_alloc_path();
+	if (!path)
+		return -ENOMEM;
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		fprintf(stderr, "Couldn't start transaction %ld\n",
+			PTR_ERR(trans));
+		ret = PTR_ERR(trans);
+		goto free_path;
+	}
+
+	ret = btrfs_search_slot(trans, root, key, path, 0, 1);
+	if (ret != 0) {
+		fprintf(stderr, "Error searching to node %d\n", ret);
+		goto commit_txn;
+	}
+	leaf = path->nodes[0];
+	slot = path->slots[0];
+	data = btrfs_item_ptr(leaf, slot, void);
+	item_size = btrfs_item_size(leaf, slot);
+	if (bogus_offset + bogus_size > item_size) {
+		fprintf(stderr, "Item corruption past end of item: %llu > %u\n", bogus_offset + bogus_size, item_size);
+		ret = -EINVAL;
+		goto commit_txn;
+	}
+	data += bogus_offset;
+	memset_extent_buffer(leaf, bogus_value, (unsigned long)data, bogus_size);
+	btrfs_mark_buffer_dirty(leaf);
+
+commit_txn:
+	btrfs_commit_transaction(trans, root);
+free_path:
+	btrfs_free_path(path);
+	return ret;
+}
+
 static int delete_item(struct btrfs_root *root, struct btrfs_key *key)
 {
 	struct btrfs_trans_handle *trans;
@@ -1231,6 +1283,8 @@ int main(int argc, char **argv)
 	u64 csum_bytenr = 0;
 	u64 block_group = 0;
 	char field[FIELD_BUF_LEN];
+	u64 bogus_value = UNSET_U64;
+	u64 bogus_offset = UNSET_U64;
 
 	field[0] = '\0';
 	memset(&key, 0, sizeof(key));
@@ -1259,11 +1313,13 @@ int main(int argc, char **argv)
 			{ "root", no_argument, NULL, 'r'},
 			{ "csum", required_argument, NULL, 'C'},
 			{ "block-group", required_argument, NULL, GETOPT_VAL_BLOCK_GROUP},
+			{ "value", required_argument, NULL, 'v'},
+			{ "offset", required_argument, NULL, 'o'},
 			{ "help", no_argument, NULL, GETOPT_VAL_HELP},
 			{ NULL, 0, NULL, 0 }
 		};
 
-		c = getopt_long(argc, argv, "l:c:b:eEkuUi:f:x:m:K:I:D:d:r:C:",
+		c = getopt_long(argc, argv, "l:c:b:eEkuUi:f:x:m:K:I:D:d:r:C:v:o:",
 				long_options, NULL);
 		if (c < 0)
 			break;
@@ -1329,6 +1385,12 @@ int main(int argc, char **argv)
 			case GETOPT_VAL_BLOCK_GROUP:
 				block_group = arg_strtou64(optarg);
 				break;
+			case 'v':
+				bogus_value = arg_strtou64(optarg);
+				break;
+			case 'o':
+				bogus_offset = arg_strtou64(optarg);
+				break;
 			case GETOPT_VAL_HELP:
 			default:
 				print_usage(c != GETOPT_VAL_HELP);
@@ -1455,7 +1517,16 @@ int main(int argc, char **argv)
 		if (!root_objectid)
 			print_usage(1);
 
-		ret = corrupt_btrfs_item(target_root, &key, field);
+		if (*field != 0)
+			ret = corrupt_btrfs_item(target_root, &key, field);
+		else if (bogus_offset != UNSET_U64 &&
+			 bytes != UNSET_U64 &&
+			 bogus_value != UNSET_U64)
+			ret = corrupt_btrfs_item_data(target_root, &key,
+						      bogus_offset, bytes,
+						      bogus_value);
+		else
+			print_usage(1);
 		goto out_close;
 	}
 	if (delete) {
-- 
2.37.1

[PATCH v3 3/4] btrfs-progs: expand corrupt_file_extent in btrfs-corrupt-block

[Boris Burkov]

To corrupt holes/prealloc/inline extents, we need to mess with
extent data items. This patch makes it possible to modify
disk_bytenr with a specific value (useful for hole corruptions)
and to modify the type field (useful for prealloc corruptions)

Signed-off-by: Boris Burkov <boris@bur.io>
---
 btrfs-corrupt-block.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/btrfs-corrupt-block.c b/btrfs-corrupt-block.c
index 225818817..50e2ebcba 100644
--- a/btrfs-corrupt-block.c
+++ b/btrfs-corrupt-block.c
@@ -308,6 +308,7 @@ enum btrfs_inode_field {
 
 enum btrfs_file_extent_field {
 	BTRFS_FILE_EXTENT_DISK_BYTENR,
+	BTRFS_FILE_EXTENT_TYPE,
 	BTRFS_FILE_EXTENT_BAD,
 };
 
@@ -380,6 +381,8 @@ static enum btrfs_file_extent_field convert_file_extent_field(char *field)
 {
 	if (!strncmp(field, "disk_bytenr", FIELD_BUF_LEN))
 		return BTRFS_FILE_EXTENT_DISK_BYTENR;
+	if (!strncmp(field, "type", FIELD_BUF_LEN))
+		return BTRFS_FILE_EXTENT_TYPE;
 	return BTRFS_FILE_EXTENT_BAD;
 }
 
@@ -753,14 +756,14 @@ out:
 
 static int corrupt_file_extent(struct btrfs_trans_handle *trans,
 			       struct btrfs_root *root, u64 inode, u64 extent,
-			       char *field)
+			       char *field, u64 bogus)
 {
 	struct btrfs_file_extent_item *fi;
 	struct btrfs_path *path;
 	struct btrfs_key key;
 	enum btrfs_file_extent_field corrupt_field;
-	u64 bogus;
 	u64 orig;
+	u8 bogus_type = bogus;
 	int ret = 0;
 
 	corrupt_field = convert_file_extent_field(field);
@@ -792,9 +795,18 @@ static int corrupt_file_extent(struct btrfs_trans_handle *trans,
 	switch (corrupt_field) {
 	case BTRFS_FILE_EXTENT_DISK_BYTENR:
 		orig = btrfs_file_extent_disk_bytenr(path->nodes[0], fi);
-		bogus = generate_u64(orig);
+		if (bogus == UNSET_U64)
+			bogus = generate_u64(orig);
 		btrfs_set_file_extent_disk_bytenr(path->nodes[0], fi, bogus);
 		break;
+	case BTRFS_FILE_EXTENT_TYPE:
+		if (bogus == UNSET_U64) {
+			fprintf(stderr, "Specify a new extent type value (-v)\n");
+			ret = -EINVAL;
+			goto out;
+		}
+		btrfs_set_file_extent_type(path->nodes[0], fi, bogus_type);
+		break;
 	default:
 		ret = -EINVAL;
 		break;
@@ -1487,9 +1499,9 @@ int main(int argc, char **argv)
 			printf("corrupting inode\n");
 			ret = corrupt_inode(trans, root, inode, field);
 		} else {
-			printf("corrupting file extent\n");
 			ret = corrupt_file_extent(trans, root, inode,
-						  file_extent, field);
+						  file_extent, field,
+						  bogus_value);
 		}
 		btrfs_commit_transaction(trans, root);
 		goto out_close;
-- 
2.37.1

[PATCH v3 4/4] btrfs-progs: add VERITY ro compat flag

[Boris Burkov]

This compat flag is missing, but is being checked by mount, and could
well be present legitimately.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 kernel-shared/ctree.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/kernel-shared/ctree.h b/kernel-shared/ctree.h
index fc8b61eda..2070a6e51 100644
--- a/kernel-shared/ctree.h
+++ b/kernel-shared/ctree.h
@@ -484,6 +484,7 @@ BUILD_ASSERT(sizeof(struct btrfs_super_block) == BTRFS_SUPER_INFO_SIZE);
  * tree.
  */
 #define BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID	(1ULL << 1)
+#define BTRFS_FEATURE_COMPAT_RO_VERITY			(1ULL << 2)
 
 #define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF	(1ULL << 0)
 #define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL	(1ULL << 1)
@@ -514,7 +515,8 @@ BUILD_ASSERT(sizeof(struct btrfs_super_block) == BTRFS_SUPER_INFO_SIZE);
  */
 #define BTRFS_FEATURE_COMPAT_RO_SUPP			\
 	(BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE |	\
-	 BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID)
+	 BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE_VALID |\
+	 BTRFS_FEATURE_COMPAT_RO_VERITY)
 
 #if EXPERIMENTAL
 #define BTRFS_FEATURE_INCOMPAT_SUPP			\
-- 
2.37.1

Re: [PATCH v3 0/4] btrfs-progs: support for fs-verity fstests

[Sweet Tea Dorminy]

On 7/18/22 18:13, Boris Burkov wrote:
> Adding fstests for fs-verity on btrfs needs some light support from
> btrfs-progs. Specifically, it needs additional device corruption
> features to test corruption detection, and it needs the RO COMPAT flag.
> 
> The first patch defines (u64)-1 as "UNSET_U64"
> The second patch adds corrupting arbitrary regions of item data with -I.
> The third patch adds corrupting holes and prealloc in extent data.
> The fourth patch includes BTRFS_FEATURE_RO_COMPAT_VERITY to ctree.h
> 
> --
> v3: add patch #defining (u64)-1 in btrfs-corrupt-block
>      check item bounds in corruption function
>      improve usage message for new corruption use case
>      add patch with verity ro compat flag
> v2: minor cleanups from rebasing after a year
> 
> Boris Burkov (4):
>    btrfs-corrupt-block: define (u64)-1 as UNSET_U64
>    btrfs-progs: corrupt generic item data with btrfs-corrupt-block
>    btrfs-progs: expand corrupt_file_extent in btrfs-corrupt-block
>    btrfs-progs: add VERITY ro compat flag
> 
>   btrfs-corrupt-block.c | 128 ++++++++++++++++++++++++++++++++++--------
>   kernel-shared/ctree.h |   4 +-
>   2 files changed, 109 insertions(+), 23 deletions(-)
> 
For the series:
Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me>

Re: [PATCH v3 0/4] btrfs-progs: support for fs-verity fstests

[Josef Bacik]

On Mon, Jul 18, 2022 at 03:13:07PM -0700, Boris Burkov wrote:
> Adding fstests for fs-verity on btrfs needs some light support from
> btrfs-progs. Specifically, it needs additional device corruption
> features to test corruption detection, and it needs the RO COMPAT flag.
> 
> The first patch defines (u64)-1 as "UNSET_U64"
> The second patch adds corrupting arbitrary regions of item data with -I.
> The third patch adds corrupting holes and prealloc in extent data.
> The fourth patch includes BTRFS_FEATURE_RO_COMPAT_VERITY to ctree.h
> 

Validated this made the verity tests do the right thing, you can add

Reviewed-by: Josef Bacik <josef@toxicpanda.com>

Thanks,

Josef