[PATCH v2 1/2] btrfs-progs: limit the max value of leafsize and nodesize

[PATCH v2 1/2] btrfs-progs: limit the max value of leafsize and nodesize

[Robin Dong]

From: Robin Dong <sanbai@taobao.com>

Using mkfs.btrfs like:

        mkfs.btrfs -l 131072 /dev/sda

will return no error, but after mount it, the dmesg will report:

	BTRFS: couldn't mount because metadata blocksize (131072) was too large

The leafsize and nodesize are equal at present, so we just use one function
"check_leaf_or_node_size" to limit leaf and node size below BTRFS_MAX_METADATA_BLOCKSIZE.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Reviewed-by: David Sterba <dave@jikos.cz>
---
 ctree.h |    6 ++++++
 mkfs.c  |   29 +++++++++++++++++++++++------
 2 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/ctree.h b/ctree.h
index 7f55229..75c1e0a 100644
--- a/ctree.h
+++ b/ctree.h
@@ -111,6 +111,12 @@ struct btrfs_trans_handle;
 #define BTRFS_DEV_ITEMS_OBJECTID 1ULL
 
 /*
+ * the max metadata block size.  This limit is somewhat artificial,
+ * but the memmove costs go through the roof for larger blocks.
+ */
+#define BTRFS_MAX_METADATA_BLOCKSIZE 65536
+
+/*
  * we can actually store much bigger names, but lets not confuse the rest
  * of linux
  */
diff --git a/mkfs.c b/mkfs.c
index dff5eb8..8420482 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -1201,6 +1201,27 @@ static int zero_output_file(int out_fd, u64 size, u32 sectorsize)
 	return ret;
 }
 
+static int check_leaf_or_node_size(u32 size, u32 sectorsize)
+{
+	if (size < sectorsize) {
+		fprintf(stderr,
+			"Illegal leafsize (or nodesize) %u (smaller than %u)\n",
+			size, sectorsize);
+		return -1;
+	} else if (size > BTRFS_MAX_METADATA_BLOCKSIZE) {
+		fprintf(stderr,
+			"Illegal leafsize (or nodesize) %u (larger than %u)\n",
+			size, BTRFS_MAX_METADATA_BLOCKSIZE);
+		return -1;
+	} else if (size & (sectorsize - 1)) {
+		fprintf(stderr,
+			"Illegal leafsize (or nodesize) %u (not align to %u)\n",
+			size, sectorsize);
+		return -1;
+	}
+	return 0;
+}
+
 int main(int ac, char **av)
 {
 	char *file;
@@ -1291,14 +1312,10 @@ int main(int ac, char **av)
 		}
 	}
 	sectorsize = max(sectorsize, (u32)getpagesize());
-	if (leafsize < sectorsize || (leafsize & (sectorsize - 1))) {
-		fprintf(stderr, "Illegal leafsize %u\n", leafsize);
+	if (check_leaf_or_node_size(leafsize, sectorsize))
 		exit(1);
-	}
-	if (nodesize < sectorsize || (nodesize & (sectorsize - 1))) {
-		fprintf(stderr, "Illegal nodesize %u\n", nodesize);
+	if (check_leaf_or_node_size(nodesize, sectorsize))
 		exit(1);
-	}
 	ac = ac - optind;
 	if (ac == 0)
 		print_usage();
-- 
1.7.3.2

[PATCH v2 2/2] btrfs-progs: limit the min value of total_bytes

[Robin Dong]

From: Robin Dong <sanbai@taobao.com>

Using mkfs.btrfs like:

        mkfs.btrfs -b 1048576 /dev/sda

will report error:

	mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)' failed.
	Aborted

because the length of dev_extent is 4MB.

But if we use mkfs.btrfs with 8MB total bytes, the newly mounted btrfs filesystem
would not contain even one empty file. So 12MB will be good min-value for block_count.

Signed-off-by: Robin Dong <sanbai@taobao.com>
---
 mkfs.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/mkfs.c b/mkfs.c
index 8420482..496faa8 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -1345,7 +1345,11 @@ int main(int ac, char **av)
 				&dev_block_count, &mixed, nodiscard);
 		if (block_count == 0)
 			block_count = dev_block_count;
-		else if (block_count > dev_block_count) {
+		else if (block_count < 3 * BTRFS_MKFS_SYSTEM_GROUP_SIZE) {
+			fprintf(stderr, "Illegal total number of bytes %u\n",
+					block_count);
+			exit(1);
+		} else if (block_count > dev_block_count) {
 			fprintf(stderr, "%s is smaller than requested size\n", file);
 			exit(1);
 		}
-- 
1.7.3.2

Re: [PATCH v2 2/2] btrfs-progs: limit the min value of total_bytes

[David Sterba]

On Fri, Sep 28, 2012 at 11:02:40AM +0800, Robin Dong wrote:
> From: Robin Dong <sanbai@taobao.com>
> 
> Using mkfs.btrfs like:
> 
>         mkfs.btrfs -b 1048576 /dev/sda
> 
> will report error:
> 
> 	mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)' failed.
> 	Aborted
> 
> because the length of dev_extent is 4MB.
> 
> But if we use mkfs.btrfs with 8MB total bytes, the newly mounted btrfs filesystem
> would not contain even one empty file. So 12MB will be good min-value for block_count.

I'm not able to create a single file even on a 12MB filesystem
(with -d single -m single --mixed), so any limit that would let the mkfs
finish normally should be fine. For the single/single case it's 5MB but
for the dup/dup it's 156MB. It's due to the known bug in the blockgroup
creation with multiple devices (applies on dup as well here) that leads
to:

# btrfs fi df .
System, DUP: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Data+Metadata, DUP: total=64.00MB, used=24.00KB
Data+Metadata: total=8.00MB, used=0.00

8*2 + 4 + 64*2 + 8 = 156

so, 12M is too small to avoid the mkfs crash.

david

Re: [PATCH v2 2/2] btrfs-progs: limit the min value of total_bytes

[Robin Dong]

2012/10/1 David Sterba <dave@jikos.cz>:
> On Fri, Sep 28, 2012 at 11:02:40AM +0800, Robin Dong wrote:
>> From: Robin Dong <sanbai@taobao.com>
>>
>> Using mkfs.btrfs like:
>>
>>         mkfs.btrfs -b 1048576 /dev/sda
>>
>> will report error:
>>
>>       mkfs.btrfs: volumes.c:796: btrfs_alloc_chunk: Assertion `!(ret)' failed.
>>       Aborted
>>
>> because the length of dev_extent is 4MB.
>>
>> But if we use mkfs.btrfs with 8MB total bytes, the newly mounted btrfs filesystem
>> would not contain even one empty file. So 12MB will be good min-value for block_count.
>
> I'm not able to create a single file even on a 12MB filesystem
> (with -d single -m single --mixed), so any limit that would let the mkfs
> finish normally should be fine. For the single/single case it's 5MB but
> for the dup/dup it's 156MB. It's due to the known bug in the blockgroup
> creation with multiple devices (applies on dup as well here) that leads
> to:
>
> # btrfs fi df .
> System, DUP: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Data+Metadata, DUP: total=64.00MB, used=24.00KB
> Data+Metadata: total=8.00MB, used=0.00
>
> 8*2 + 4 + 64*2 + 8 = 156
>
> so, 12M is too small to avoid the mkfs crash.

Thanks for your notice!
>
> david



-- 
--
Best Regard
Robin Dong

Re: [PATCH v2 1/2] btrfs-progs: limit the max value of leafsize and nodesize

[David Sterba]

On Fri, Sep 28, 2012 at 11:02:39AM +0800, Robin Dong wrote:
> From: Robin Dong <sanbai@taobao.com>
> --- a/mkfs.c
> +++ b/mkfs.c
> @@ -1201,6 +1201,27 @@ static int zero_output_file(int out_fd, u64 size, u32 sectorsize)
>  	return ret;
>  }
>  
> +static int check_leaf_or_node_size(u32 size, u32 sectorsize)
> +{
> +	if (size < sectorsize) {
> +		fprintf(stderr,
> +			"Illegal leafsize (or nodesize) %u (smaller than %u)\n",
> +			size, sectorsize);

First looking at the message, the absolute limit value might be
confusing, the condition is really 'if node/leaf is smaller than
sectorsize', and sectorsize will become a varaible eventually (though
now it's fixed to the system's PAGE_SIZE and 4k in most cases).

So, even if this will make the erro message more verbose, I suggest to
enhance the explanation with

"... (smaller than sectorsize %u)"

(If you se a better way how to format the value and the word, please don't
hasitate to change it.)

> +		return -1;
> +	} else if (size > BTRFS_MAX_METADATA_BLOCKSIZE) {
> +		fprintf(stderr,
> +			"Illegal leafsize (or nodesize) %u (larger than %u)\n",
> +			size, BTRFS_MAX_METADATA_BLOCKSIZE);
> +		return -1;
> +	} else if (size & (sectorsize - 1)) {
> +		fprintf(stderr,
> +			"Illegal leafsize (or nodesize) %u (not align to %u)\n",

align -> aligned

> +			size, sectorsize);
> +		return -1;
> +	}
> +	return 0;
> +}
> +

thanks,
david