Re: How to handle >16TB devices on 32 bit hosts ??

[Andi Kleen <andi@firstfloor.org>]

On Sat, Jul 18, 2009 at 02:52:13AM -0400, Andreas Dilger wrote:
> If you aren't running a 32-bit system with this config, you shouldn't
> really care.  For those systems that need to run in this mode they
> would rather have it work a few percent slower instead of not at all.

Well, it doesn't work at all anyways due to the fsck problem.

> The last test numbers I saw were 5GB of RAM for a 20TB filesystem,
> but since the bitmaps used are fully-allocated arrays that isn't
> surprising.  We are planning to replace this with a tree, since the
> majority of bitmaps used by e2fsck have large contiguous ranges of
> set or unset bits and can be represented much more efficiently.

You would need to get <~2.5GB for 32bit. In practice that's
the limit you have there.

> Also, for filesystems like btrfs or ZFS the checking can be done
> online and incrementally without storing a full representation of
> the state in memory.

You could, but I suspect it would be cheaper to just use a
64bit system than to rewrite fsck. 64bit is available
for a lot of embedded setups these days too.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.