Re: [PATCH] Breaking ext2 file size limit of 2TB

[Andreas Dilger <adilger@clusterfs.com>]

[-- Attachment #1: Type: text/plain, Size: 3885 bytes --]

On Jun 25, 2004  18:34 +0530, Goldwyn Rodrigues wrote:
> I have made a patch to enable file creation greater than 2TB. I tested it
> using sparse files and it works good.
> 
> Working:
> The file size limit of the ext3 filesystem is limited to 2TB because of
> i_blocks, a variable which stores the number of 512 blocks in the inode.
> i_blocks is a 32 which limits the number it can hold. The patch makes
> use of l_i_reserved1 field to keep the higher order bits of i_blocks.

Do you have a real demand for doing this?  Given that block devices are
limited to 16TB on 32-bit architectures (page size * long), and ext3
files themselves are limited to 4TB+ (i386 page size again) because of
the triple-indirect block limit this isn't much of a win until we go to
something like extents.  Are you using a non-i386 architecture?

If we started using larger blocksizes for systems that have larger than
4kB pages (i.e. not i386) this would become an issue.  At some point
having giant files w/o extents is pointless (performance is too bad),
so we could also put the high blocks count in as part of the extent data
(e.g. i_blocks[14]) since the format would be gratuitously incompatible
anyways.

> @@ -1003,9 +1003,9 @@
>  	res += 1LL << (bits-2);
>  	res += 1LL << (2*(bits-2));
>  	res += 1LL << (3*(bits-2));
> -	res <<= bits;
> -	if (res > (512LL << 32) - (1 << bits))
> -		res = (512LL << 32) - (1 << bits);
> +	/* Since another block is added, we add the same number again */
> +	res += 1LL << (3*(bits-2));
> +	res <<=bits;

This is incorrect.  All that this change does is remove the extra
"res > (512LL << 32) - (1 << bits)" limit.  Even that could be removed
for sparse files without any of these changes if we wanted to check
at block allocation time whether we would overflow the i_blocks limit.

>  struct buffer_head *ext3_bread(handle_t *handle, struct inode * inode,
> -                              int block, int create, int *err)
> +                              sector_t block, int create, int *err)
> {
>         struct buffer_head * bh;
> -       int prev_blocks;
> +       sector_t prev_blocks;

This is a good fix regardless (at least change it to long from int).

> This has been developed and tested on kernel version 2.6.5 using sparse files.

That isn't really a test of anything, since a sparse file will not use 
more than 2^32 blocks.

> #define i_blocks_high		osd1.linux1.l_i_reserved1

If we really wanted to avoid being incompatible with Hurd (I personally
don't care about that, but someone who knows more should comment on how
badly this will screw things for it) we could use one of the other fields in
the inode like m_i_frag + m_i_fsize, or i_faddr as none of them is actually
used.  We also only really need 24 bits of this word before we hit the
64-bit byte i_size limit so we may as well be prudent and mask off the high
byte for later use.

Does anyone know if Hurd actually use both i_translator (i_reserved1) and
i_mode_high field (i_pad1)?

In any case, we need to wrap this with some sort of COMPAT flag in the
superblock, and probably a per-inode flag as well, so we know to trust
this value.

> --- linux-2.6.5-orig/include/linux/fs.h	2004-04-04 09:06:52.000000000 +0530
> +++ linux-2.6.5-4TB/include/linux/fs.h	2004-06-23 12:18:43.000000000 +0530
> @@ -393,7 +393,11 @@
>  	unsigned int		i_blkbits;
>  	unsigned long		i_blksize;
>  	unsigned long		i_version;
> +#if !defined(CONFIG_EXT3_LARGE_FILE_SUPPORT) || defined(CONFIG_64BIT)
>  	unsigned long		i_blocks;
> +#else
> +	unsigned long long	i_blocks;
> +#endif /* CONFIG_EXT3_LARGE_FILE_SUPPORT */

Why not just declare this as sector_t?

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]