max file size for ext3

max file size for ext3

[Aneesh Kumar K.V]

Hi,

I am looking at ext4_max_size and was confused how the
number upper_limit = 0x1ff7fffd000LL is arrived. 
The comment says the value is arrived looking at 4K.
So i tried the below program. 

main()
{
	unsigned long long upper_limit, meta_blocks;
	int bits = 12;

	/* total blocks in 512 bytes */
	upper_limit = (1LL << 32) - 1;
	/* total blocks in file system block size */
	upper_limit >>= (bits - 9);


	meta_blocks = 1;
	/* double indirect blocks */
	meta_blocks += 1 + 1LL << (bits-2);
	/* tripple indirect blocks */
	meta_blocks += 1 + 1LL << (bits-2) + 1LL << (2*(bits-2));

	upper_limit -= meta_blocks;
	upper_limit <<= bits;

	printf("%x\n", upper_limit);
}

Can somebody help me to find out what is missing in the above ?



I also think hardcoding 4k block size is not correct. I have the below
patch pending with large file size.

diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 2083c19..9f39cfb 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -1511,6 +1511,7 @@ static void ext4_orphan_cleanup (struct super_block * sb,
 static loff_t ext4_max_size(int bits, struct super_block *sb)
 {
 	loff_t res = EXT4_NDIR_BLOCKS;
+	loff_t total_meta_blocks;
 	/* This constant is calculated to be the largest file size for a
 	 * dense, 4k-blocksize file such that the total number of
 	 * sectors in the file, including data and all indirect blocks,
@@ -1518,11 +1519,34 @@ static loff_t ext4_max_size(int bits, struct super_block *sb)
 	loff_t upper_limit;
 
 	if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_LARGE_BLOCK)) {
-		/* FIXME!! */
-		upper_limit = (1LL << (bits + 32)) - 1;
+		/*
+		 * With __u32 i_blocks representing the total number of blocks
+		 * of the file in file system block size, the max file size
+		 * would be  2**(32) - 1 - blocks taken by the meta data
+		 * blocks multiplied by block size.
+		 */
+		/* total blocks in file system block size*/
+		upper_limit = (1LL << 32) - 1;
+
 	} else {
-		upper_limit = 0x1ff7fffd000LL;
+		/* total blocks in 512 bytes */
+		upper_limit = (1LL << 32) - 1;
+		/* total blocks in file system block size */
+		upper_limit >>= (bits - 9);
+
+		//upper_limit = 0x1ff7fffd000LL;
 	}
+
+	/* indirect blocks */
+	meta_blocks = 1;
+	/* double indirect blocks */
+	meta_blocks += 1 + 1LL << (bits-2);
+	/* tripple indirect blocks */
+	meta_blocks += 1 + 1LL << (bits-2) + 1LL << (2*(bits-2));
+
+	upper_limit -= meta_blocks;
+	upper_limit <<= bits;
+
 	res += 1LL << (bits-2);
 	res += 1LL << (2*(bits-2));
 	res += 1LL << (3*(bits-2));

Re: max file size for ext3

[Aneesh Kumar K.V]

This is what i have now . But still i am not able to map the magic number

#include <stdio.h>
main()
{
	unsigned long long upper_limit;
	int meta_blocks;
	int bits = 12;

	/* total blocks in 512 bytes */
	upper_limit = (1LL << 32) -1;
	/* total blocks in file system block size */
	upper_limit >>= (bits - 9);


	/* indirect blocks */
	meta_blocks = 1;
	/* double indirect blocks */
	meta_blocks += 1 + (1LL << (bits-2));
	/* tripple indirect blocks */
	meta_blocks += 1 + (1LL << (bits-2)) + (1LL << (2*(bits-2)));

	upper_limit -= meta_blocks;
	upper_limit <<= bits;

	printf("expected %llx %llx\n",0x1ff7fffd000LL, upper_limit);
}

expected 1ff7fffd000 1feff7fc000

Re: max file size for ext3

[Eric Sandeen]

Aneesh Kumar K.V wrote:

> I also think hardcoding 4k block size is not correct. I have the below
> patch pending with large file size.

Sounds much better to me.  I think it also needs to be limited so that
it doesn't exceed what the pagecache can address, i.e. 2^32 * PAGE_SIZE
- this will especially come into play w/ the large blocks, right?

-Eric

Re: max file size for ext3

[Jan Kara]

  Hello,

> I am looking at ext4_max_size and was confused how the
> number upper_limit = 0x1ff7fffd000LL is arrived. 
> The comment says the value is arrived looking at 4K.
> So i tried the below program. 
> 
> main()
> {
> 	unsigned long long upper_limit, meta_blocks;
> 	int bits = 12;
> 
> 	/* total blocks in 512 bytes */
> 	upper_limit = (1LL << 32) - 1;
> 	/* total blocks in file system block size */
> 	upper_limit >>= (bits - 9);
> 
> 
> 	meta_blocks = 1;
> 	/* double indirect blocks */
> 	meta_blocks += 1 + 1LL << (bits-2);
> 	/* tripple indirect blocks */
> 	meta_blocks += 1 + 1LL << (bits-2) + 1LL << (2*(bits-2));
> 
> 	upper_limit -= meta_blocks;
> 	upper_limit <<= bits;
> 
> 	printf("%x\n", upper_limit);
> }
> 
> Can somebody help me to find out what is missing in the above ?
  You actually overestimate the number of triply indirect blocks in your
program above (you count all possible but since the limit is around 2TB
only roughly half of them is really needed).

> I also think hardcoding 4k block size is not correct. I have the below
> patch pending with large file size.
  It is incorrect for blocksize larger than 4096, that's right. Your
patch looks fine - we loose a bit of filesize limit but I don't think it
really matters.

								Honza