From: Dave Chinner <david@fromorbit.com>
To: xfs@oss.sgi.com
Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: [PATCH 4/8] xfstests: use preallocation for ag-wiper
Date: Mon, 21 Nov 2011 22:31:24 +1100 [thread overview]
Message-ID: <1321875088-30801-5-git-send-email-david@fromorbit.com> (raw)
In-Reply-To: <1321875088-30801-1-git-send-email-david@fromorbit.com>
From: Dave Chinner <dchinner@redhat.com>
To enable sane testing of large scale filesystems, the --large-fs
test option uses xfs_db magic to mark AGs full without doing any IO.
This leaves only a small amount of free space left in the filesystem
to stress the high AGs of the filesystem rather than the low AGs.
This method requires us to have special filesystem check options to
avoid free space checking in xfs_check, and we cannot current run
xfs_repair on such a filesystem at all. As it is, free space
checking on xfs_check does not scale, so we still need to avoid this
checking regardless of how we fill the filesystem.
We can acheive exactly the same fill behaviour by preallocating a
single large file in the filesystem immediately after creating it.
This is a filesystem independent manner of filling the filesystem,
and allows us to do large filesystem testing on more than just XFS.
Further, this preallocation method effectively adds a new "very
large file" test. It also enables us to run an unmodified xfs_repair
or filesystem specific fsck program to check the filesystem for
sanity, so we can now do full sanity checking of such large
filesystems.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
common.rc | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 53 insertions(+), 5 deletions(-)
diff --git a/common.rc b/common.rc
index 455886d..34467ec 100644
--- a/common.rc
+++ b/common.rc
@@ -276,6 +276,47 @@ _scratch_mkfs_options()
echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV
}
+
+_setup_large_xfs_fs()
+{
+ fs_size=$1
+ local tmp_dir=/tmp/
+
+ [ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+ [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+ [ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+
+ # calculate the size of the file we need to allocate.
+ # Default free space in the FS is 50GB, but you can specify more via
+ # SCRATCH_DEV_EMPTY_SPACE
+ file_size=$(($fs_size - 50*1024*1024*1024))
+ file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
+
+ # mount the filesystem, create the file, unmount it
+ _scratch_mount 2>&1 >$tmp_dir/mnt.err
+ local status=$?
+ if [ $status -ne 0 ]; then
+ echo "mount failed"
+ cat $tmp_dir/mnt.err >&2
+ rm -f $tmp_dir/mnt.err
+ return $status
+ fi
+ rm -f $tmp_dir/mnt.err
+
+ xfs_io -F -f \
+ -c "truncate $file_size" \
+ -c "falloc -k 0 $file_size" \
+ $SCRATCH_MNT/.use_space 2>&1 > /dev/null
+ status=$?
+ umount $SCRATCH_MNT
+ if [ $status -ne 0 ]; then
+ echo "large file prealloc failed"
+ cat $tmp_dir/mnt.err >&2
+ return $status
+ fi
+ return 0
+}
+
_scratch_mkfs_xfs()
{
# extra mkfs options can be added by tests
@@ -305,16 +346,23 @@ _scratch_mkfs_xfs()
mkfs_status=$?
fi
+ if [ $mkfs_status -eq 0 -a "$LARGE_SCRATCH_DEV" = yes ]; then
+ # manually parse the mkfs output to get the fs size in bytes
+ local fs_size
+ fs_size=`cat $tmp_dir.mkfsstd | perl -ne '
+ if (/^data\s+=\s+bsize=(\d+)\s+blocks=(\d+)/) {
+ my $size = $1 * $2;
+ print STDOUT "$size\n";
+ }'`
+ _setup_large_xfs_fs $fs_size
+ mkfs_status=$?
+ fi
+
# output stored mkfs output
cat $tmp_dir.mkfserr >&2
cat $tmp_dir.mkfsstd
rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
- if [ "$LARGE_SCRATCH_DEV" = yes ]; then
- [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
- ./tools/ag-wipe -q -r $SCRATCH_DEV_EMPTY_SPACE $SCRATCH_DEV
- fi
next prev parent reply other threads:[~2011-11-21 11:36 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-21 11:31 [RFC][PATCH 0/8] xfstests: rework large filesystem testing Dave Chinner
2011-11-21 11:31 ` [PATCH 1/8] xfstests: add --largefs check option Dave Chinner
2011-12-04 21:14 ` Christoph Hellwig
2011-11-21 11:31 ` [PATCH 2/8] xfstests: rename USE_BIG_LOOPFS to be more generic Dave Chinner
2012-01-16 16:23 ` Mark Tinguely
2011-11-21 11:31 ` [PATCH 3/8] xfstests: rename RETAIN_AG_BYTES Dave Chinner
2011-11-21 11:31 ` Dave Chinner [this message]
2011-11-21 11:31 ` [PATCH 5/8] xfstests: use command line option for setting extra space Dave Chinner
2011-12-04 21:16 ` Christoph Hellwig
2011-11-21 11:31 ` [PATCH 6/8] xfstest: enable xfs_repair for large filesystem testing Dave Chinner
2012-01-16 17:04 ` Mark Tinguely
2011-11-21 11:31 ` [PATCH 7/8] xfstests: always us test option when checking large scratch device Dave Chinner
2011-11-21 11:31 ` [PATCH 8/8] xfstests: enable large fs testing on ext4 Dave Chinner
2011-11-21 12:10 ` [RFC][PATCH 0/8] xfstests: rework large filesystem testing Theodore Tso
2011-11-22 9:28 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1321875088-30801-5-git-send-email-david@fromorbit.com \
--to=david@fromorbit.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).