Re: [PATCH] Breaking ext2 file size limit of 2TB

[Andreas Dilger <adilger@clusterfs.com>]

[-- Attachment #1: Type: text/plain, Size: 2319 bytes --]

On Jun 26, 2004  11:39 +0530, Goldwyn Rodrigues wrote:
> I am using i386 architecture.
> 
> > If we started using larger blocksizes for systems that have larger than
> > 4kB pages (i.e. not i386) this would become an issue.  At some point
> > having giant files w/o extents is pointless (performance is too bad),
> > so we could also put the high blocks count in as part of the extent data
> > (e.g. i_blocks[14]) since the format would be gratuitously incompatible
> > anyways.
> 
> I din't quite understand this point. Do you mean to say that we keep such
> data elsewhere if required, and then read such data only for large systems.
> As in, do another block read?

No, my point was that on i386 systems there is also a limit at 4TB so
your patch will only serve to double the maximum file size on such
systems.  On non-i386 systems this patch makes somewhat more sense,
but even so it is very inefficient to have enormous files allocating
a billion blocks or even more at larger block sizes.

On i386 we are limited to 4kB pages and this means after we fill the
triple indirect block we can't grow any larger (4096 / 4) ^ 3 * 4096 =
4TB.  However, on ia64 and other 64-bit platforms that have larger page
sizes we can use larger blocks for the filesystem (up to 64kB) and this
increases the triple indirect limit a lot (65536 / 4) ^ 3 * 65536 = 256PB
= 58 bits, at the sake of no longer being mountable on i386 machines.

I think that to have "useful" growth in file sizes we need to take a
hard look at the extents code already written by Alex.  Even with that
we need to be able to store a blocks count > 2^32 so this work isn't
incompatible with that, but if it takes 30 minutes to unlink a giant
file with a billion blocks I don't think just removing the i_blocks
limit will have helped us very much by itself.

Yes, the extent code is incompatible with the current layout (as is this
change), but if we add at least read support (write support separately
selectable if there is some objection to the incompatible change) for
extents to 2.6 then it can become a full-fledged member of ext3 in the
2.8 kernel.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]