* ext4 > 16T woes @ 2012-05-23 18:48 Eric Sandeen 2012-05-23 19:33 ` Andreas Dilger 2012-05-25 21:01 ` Eric Sandeen 0 siblings, 2 replies; 4+ messages in thread From: Eric Sandeen @ 2012-05-23 18:48 UTC (permalink / raw) To: ext4 development I'll try to look into this but just wanted to put it out there. After creating a 32T fs and filling it with 1T preallocated files, I get quite a lot of corruption; this wasn't really the test I wanted to do, I wanted to then remove a file with mostly high blocks, run fsstress, and test recovery :( # uname -r 3.4.0-rc4+ # truncate --size 32t imagefile2 # mkfs.ext4 -F imagefile2 mke2fs 1.42.3 (14-May-2012) ... # mount -o loop imagefile2 /mnt/scratch # for I in `seq 1 32`; do fallocate -l 1t /mnt/scratch/1tfile$I; done # umount /mnt/scratch # time e2fsck -fn imagefile2 e2fsck 1.42.3 (14-May-2012) <many many many lines snipped> Free blocks count wrong for group #131072 (0, counted=32768). Fix? no Free blocks count wrong for group #131073 (0, counted=32768). Fix? no Free blocks count wrong for group #131074 (0, counted=4164). Fix? no Free blocks count wrong (3, counted=4294967299). Fix? no imagefile2: ********** WARNING: Filesystem still has errors ********** imagefile2: 43/536870912 files (0.0% non-contiguous), 8589934589/8589934592 blocks real 27m40.133s user 27m12.886s sys 0m6.238s ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ext4 > 16T woes 2012-05-23 18:48 ext4 > 16T woes Eric Sandeen @ 2012-05-23 19:33 ` Andreas Dilger 2012-05-23 19:48 ` Eric Sandeen 2012-05-25 21:01 ` Eric Sandeen 1 sibling, 1 reply; 4+ messages in thread From: Andreas Dilger @ 2012-05-23 19:33 UTC (permalink / raw) To: Eric Sandeen; +Cc: ext4 development On 2012-05-23, at 12:48 PM, Eric Sandeen wrote: > I'll try to look into this but just wanted to put it out there. > > After creating a 32T fs and filling it with 1T preallocated files, > I get quite a lot of corruption; this wasn't really the test > I wanted to do, I wanted to then remove a file with mostly > high blocks, run fsstress, and test recovery :( Hmm, we've done full-filesystem write + read +verify + e2fsck up to 128TB (took several days to write and read so much data) with RHEL6 without similar problems (up to 32TB on RHEL5) using real disks. We don't use fallocate() or loop devices, so it may be that some problems are therein, or possibly with newer kernels. Presumably this is using XFS for the backing loop file? I don't anticipate problems with that, but there may be strange interactions between fallocate(), sparse files, and TRIM/DISCARD? > # uname -r > 3.4.0-rc4+ > # truncate --size 32t imagefile2 > # mkfs.ext4 -F imagefile2 > mke2fs 1.42.3 (14-May-2012) > ... > # mount -o loop imagefile2 /mnt/scratch > # for I in `seq 1 32`; do fallocate -l 1t /mnt/scratch/1tfile$I; done > # umount /mnt/scratch Did you try sync here, or running e2fsck on the loop device instead of the backing file? It looks like nothing was written to disk... > # time e2fsck -fn imagefile2 > e2fsck 1.42.3 (14-May-2012) > <many many many lines snipped> > Free blocks count wrong for group #131072 (0, counted=32768). > Fix? no > > Free blocks count wrong for group #131073 (0, counted=32768). > Fix? no > > Free blocks count wrong for group #131074 (0, counted=4164). > Fix? no > > Free blocks count wrong (3, counted=4294967299). > Fix? no > > > imagefile2: ********** WARNING: Filesystem still has errors ********** > > imagefile2: 43/536870912 files (0.0% non-contiguous), 8589934589/8589934592 blocks > > real 27m40.133s > user 27m12.886s > sys 0m6.238s > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Andreas ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ext4 > 16T woes 2012-05-23 19:33 ` Andreas Dilger @ 2012-05-23 19:48 ` Eric Sandeen 0 siblings, 0 replies; 4+ messages in thread From: Eric Sandeen @ 2012-05-23 19:48 UTC (permalink / raw) To: Andreas Dilger; +Cc: ext4 development On 5/23/12 2:33 PM, Andreas Dilger wrote: > On 2012-05-23, at 12:48 PM, Eric Sandeen wrote: >> I'll try to look into this but just wanted to put it out there. >> >> After creating a 32T fs and filling it with 1T preallocated files, >> I get quite a lot of corruption; this wasn't really the test >> I wanted to do, I wanted to then remove a file with mostly >> high blocks, run fsstress, and test recovery :( > > Hmm, we've done full-filesystem write + read +verify + e2fsck up to > 128TB (took several days to write and read so much data) with RHEL6 > without similar problems (up to 32TB on RHEL5) using real disks. > > We don't use fallocate() or loop devices, so it may be that some > problems are therein, or possibly with newer kernels. > > Presumably this is using XFS for the backing loop file? I don't > anticipate problems with that, but there may be strange interactions > between fallocate(), sparse files, and TRIM/DISCARD? It is backed by XFS, yes. I don't think I'm freeing any blocks during the test, so I wouldn't expect discard to come into it. Really, all it should be doing is writing fairly small amounts of metadata, right? I'll keep investigating. -Eric >> # uname -r >> 3.4.0-rc4+ >> # truncate --size 32t imagefile2 >> # mkfs.ext4 -F imagefile2 >> mke2fs 1.42.3 (14-May-2012) >> ... >> # mount -o loop imagefile2 /mnt/scratch >> # for I in `seq 1 32`; do fallocate -l 1t /mnt/scratch/1tfile$I; done >> # umount /mnt/scratch > > Did you try sync here, or running e2fsck on the loop device instead > of the backing file? It looks like nothing was written to disk... > >> # time e2fsck -fn imagefile2 >> e2fsck 1.42.3 (14-May-2012) >> <many many many lines snipped> >> Free blocks count wrong for group #131072 (0, counted=32768). >> Fix? no >> >> Free blocks count wrong for group #131073 (0, counted=32768). >> Fix? no >> >> Free blocks count wrong for group #131074 (0, counted=4164). >> Fix? no >> >> Free blocks count wrong (3, counted=4294967299). >> Fix? no >> >> >> imagefile2: ********** WARNING: Filesystem still has errors ********** >> >> imagefile2: 43/536870912 files (0.0% non-contiguous), 8589934589/8589934592 blocks >> >> real 27m40.133s >> user 27m12.886s >> sys 0m6.238s >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > Cheers, Andreas > > > > > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: ext4 > 16T woes 2012-05-23 18:48 ext4 > 16T woes Eric Sandeen 2012-05-23 19:33 ` Andreas Dilger @ 2012-05-25 21:01 ` Eric Sandeen 1 sibling, 0 replies; 4+ messages in thread From: Eric Sandeen @ 2012-05-25 21:01 UTC (permalink / raw) To: ext4 development On 5/23/12 1:48 PM, Eric Sandeen wrote: > I'll try to look into this but just wanted to put it out there. > > After creating a 32T fs and filling it with 1T preallocated files, > I get quite a lot of corruption; this wasn't really the test > I wanted to do, I wanted to then remove a file with mostly > high blocks, run fsstress, and test recovery :( FWIW I tracked this down to the rbtree bitmap backend and sent a patch, [PATCH] libext2fs: Fix rbtree backend for extent lengths greater than 2^32 -Eric > # uname -r > 3.4.0-rc4+ > # truncate --size 32t imagefile2 > # mkfs.ext4 -F imagefile2 > mke2fs 1.42.3 (14-May-2012) > ... > # mount -o loop imagefile2 /mnt/scratch > # for I in `seq 1 32`; do fallocate -l 1t /mnt/scratch/1tfile$I; done > # umount /mnt/scratch > # time e2fsck -fn imagefile2 > e2fsck 1.42.3 (14-May-2012) > <many many many lines snipped> > Free blocks count wrong for group #131072 (0, counted=32768). > Fix? no > > Free blocks count wrong for group #131073 (0, counted=32768). > Fix? no > > Free blocks count wrong for group #131074 (0, counted=4164). > Fix? no > > Free blocks count wrong (3, counted=4294967299). > Fix? no > > > imagefile2: ********** WARNING: Filesystem still has errors ********** > > imagefile2: 43/536870912 files (0.0% non-contiguous), 8589934589/8589934592 blocks > > real 27m40.133s > user 27m12.886s > sys 0m6.238s > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-05-25 21:01 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-05-23 18:48 ext4 > 16T woes Eric Sandeen 2012-05-23 19:33 ` Andreas Dilger 2012-05-23 19:48 ` Eric Sandeen 2012-05-25 21:01 ` Eric Sandeen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).