From: Dmitry Monakhov <dmonakhov@openvz.org>
To: xfs@oss.sgi.com
Cc: linux-fsdevel@vger.kernel.org,
Dmitry Monakhov <dmonakhov@openvz.org>,
linux-ext4@vger.kernel.org, dchinner@redhat.com
Subject: [PATCH 2/4] large-fs: improve space diversification for ext4
Date: Tue, 9 Jul 2013 15:05:16 +0400 [thread overview]
Message-ID: <1373367918-7516-2-git-send-email-dmonakhov@openvz.org> (raw)
In-Reply-To: <1373367918-7516-1-git-send-email-dmonakhov@openvz.org>
Currently we allocated several giant files one by one until limit,
so empty space is located as one chunk which limit code-path coverage.
This patch consume all space with NUM_SPACE_FILES files (by default 1024)
each has same size, and when truncate each one by required delta.
As result we have $NUM_SPACE_FILES chunks of free blocks distributed
across whole filesystem.
This should help us to avoid regressions similar to e7c9e3e99adf6c49
Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
common/rc | 40 ++++++++++++++++------------------------
1 files changed, 16 insertions(+), 24 deletions(-)
diff --git a/common/rc b/common/rc
index c44acea..902fc19 100644
--- a/common/rc
+++ b/common/rc
@@ -440,12 +440,17 @@ _setup_large_ext4_fs()
fs_empty_space=$((50*1024*1024*1024))
[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+ [ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
[ $fs_empty_space -ge $fs_size ] && return 0
# calculate the size of the file we need to allocate.
+
space_to_consume=$(($fs_size - $fs_empty_space))
+ file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
+ file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))
+
# mount the filesystem and create 16TB - 4KB files until we consume
# all the necessary space.
_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -457,33 +462,20 @@ _setup_large_ext4_fs()
return $status
fi
rm -f $tmp_dir/mnt.err
-
- file_size=$((16*1024*1024*1024*1024 - 4096))
- nfiles=0
- while [ $space_to_consume -gt $file_size ]; do
-
+ mkdir $SCRATCH_MNT/.use_space
+ # Consume all space on filesytem
+ for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do
xfs_io -F -f \
- -c "truncate $file_size" \
- -c "falloc -k 0 $file_size" \
- $SCRATCH_MNT/.use_space.$nfiles 2>&1
- status=$?
- if [ $status -ne 0 ]; then
- break;
- fi
-
- space_to_consume=$(( $space_to_consume - $file_size ))
- nfiles=$(($nfiles + 1))
+ -c "truncate $file_size_falloc" \
+ -c "falloc -k 0 $file_size_falloc" \
+ $SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
done
-
- # consume the remaining space.
- if [ $space_to_consume -gt 0 ]; then
+ # Truncate files to smaller size, will free chunks of space
+ for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do
xfs_io -F -f \
- -c "truncate $space_to_consume" \
- -c "falloc -k 0 $space_to_consume" \
- $SCRATCH_MNT/.use_space.$nfiles 2>&1
- status=$?
- fi
- export NUM_SPACE_FILES=$nfiles
+ -c "truncate $file_size_final" \
+ $SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
+ done
umount $SCRATCH_MNT
if [ $status -ne 0 ]; then
--
1.7.1
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2013-07-09 11:05 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-09 11:05 [PATCH 1/4] large-fs: fix large_fs space detection Dmitry Monakhov
2013-07-09 11:05 ` Dmitry Monakhov [this message]
2013-07-11 1:10 ` [PATCH 2/4] large-fs: improve space diversification for ext4 Dave Chinner
2013-07-09 11:05 ` [PATCH 3/4] ext4: ignore valid errors from defragmentation tests Dmitry Monakhov
2013-07-11 1:11 ` Dave Chinner
2013-07-09 11:05 ` [PATCH 4/4] large-fs: fix ext4 " Dmitry Monakhov
2013-07-11 1:17 ` Dave Chinner
2013-07-11 0:54 ` [PATCH 1/4] large-fs: fix large_fs space detection Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1373367918-7516-2-git-send-email-dmonakhov@openvz.org \
--to=dmonakhov@openvz.org \
--cc=dchinner@redhat.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).