Re: >16TB issues

[Andreas Dilger <adilger@sun.com>]

On Jul 21, 2009  11:52 -0700, Justin Maggard wrote:
> No, no error messages from the kernel.  But your llverdev utility
> ended up showing problems on the device.  After asking around on the
> MD mailing list, that was apparently because of the page cache index
> limit (at the time I was using a 32-bit kernel).
> 
> Switching to a 64-bit kernel allowed me to pass the llverdev test and
> get much further with a very large filesystem, but I'm running into
> other issues now.  I wrote up a very simple script to write 2TB files
> onto the filesystem until the device fills up.  It was able to write
> ~16TB, but after that it ran into some problems.  My kernel log now
> has lots of messages like these:

There is a matching "llverfs" tool that does essentially this, with
data verification also.

> EXT4-fs error (device md2): ext4_mb_generate_buddy: EXT4-fs: group
> 163548: 32744 blocks in bitmap, 32768 in gd
> - and -
> EXT4-fs error (device md2): ext4_mb_mark_diskspace_used: Allocating
> block 4294967391 in system zone of 131072 group
> 
> I shouldn't need e2fsprogs to be compiled 64-bit as well, right?
> Currently I've got a 64-bit kernel with 32-bit userspace.

Yes, that is a potential problem.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.