linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dmitry Monakhov <dmonakhov@openvz.org>
To: xfs@oss.sgi.com
Cc: linux-fsdevel@vger.kernel.org,
	Dmitry Monakhov <dmonakhov@openvz.org>,
	linux-ext4@vger.kernel.org, dchinner@redhat.com
Subject: [PATCH 2/4] large-fs: improve space diversification for ext4
Date: Tue,  9 Jul 2013 15:05:16 +0400	[thread overview]
Message-ID: <1373367918-7516-2-git-send-email-dmonakhov@openvz.org> (raw)
In-Reply-To: <1373367918-7516-1-git-send-email-dmonakhov@openvz.org>

Currently we allocated several giant files one by one until limit,
so empty space is located as one chunk which limit code-path coverage.
This patch consume all space with NUM_SPACE_FILES files (by default 1024)
each has same size, and when truncate each one by required delta.
As result we have $NUM_SPACE_FILES chunks of free blocks distributed
across whole filesystem.
This should help us to avoid regressions similar to e7c9e3e99adf6c49

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 common/rc |   40 ++++++++++++++++------------------------
 1 files changed, 16 insertions(+), 24 deletions(-)

diff --git a/common/rc b/common/rc
index c44acea..902fc19 100644
--- a/common/rc
+++ b/common/rc
@@ -440,12 +440,17 @@ _setup_large_ext4_fs()
 	fs_empty_space=$((50*1024*1024*1024))
 
 	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
 	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
 	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
 	[ $fs_empty_space -ge $fs_size ] && return 0
 
 	# calculate the size of the file we need to allocate.
+
 	space_to_consume=$(($fs_size - $fs_empty_space))
+	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
+	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))
+
 	# mount the filesystem and create 16TB - 4KB files until we consume
 	# all the necessary space.
 	_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -457,33 +462,20 @@ _setup_large_ext4_fs()
 		return $status
 	fi
 	rm -f $tmp_dir/mnt.err
-
-	file_size=$((16*1024*1024*1024*1024 - 4096))
-	nfiles=0
-	while [ $space_to_consume -gt $file_size ]; do
-
+	mkdir $SCRATCH_MNT/.use_space
+	# Consume all space on filesytem
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $file_size" \
-			-c "falloc -k 0 $file_size" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-		if [ $status -ne 0 ]; then
-			break;
-		fi
-
-		space_to_consume=$(( $space_to_consume - $file_size ))
-		nfiles=$(($nfiles + 1))
+			-c "truncate $file_size_falloc" \
+			-c "falloc -k 0 $file_size_falloc" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
 	done
-
-	# consume the remaining space.
-	if [ $space_to_consume -gt 0 ]; then
+	# Truncate files to smaller size, will free chunks of space
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $space_to_consume" \
-			-c "falloc -k 0 $space_to_consume" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-	fi
-	export NUM_SPACE_FILES=$nfiles
+			-c "truncate $file_size_final" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
+	done
 
 	umount $SCRATCH_MNT
 	if [ $status -ne 0 ]; then
-- 
1.7.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2013-07-09 11:05 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-07-09 11:05 [PATCH 1/4] large-fs: fix large_fs space detection Dmitry Monakhov
2013-07-09 11:05 ` Dmitry Monakhov [this message]
2013-07-11  1:10   ` [PATCH 2/4] large-fs: improve space diversification for ext4 Dave Chinner
2013-07-09 11:05 ` [PATCH 3/4] ext4: ignore valid errors from defragmentation tests Dmitry Monakhov
2013-07-11  1:11   ` Dave Chinner
2013-07-09 11:05 ` [PATCH 4/4] large-fs: fix ext4 " Dmitry Monakhov
2013-07-11  1:17   ` Dave Chinner
2013-07-11  0:54 ` [PATCH 1/4] large-fs: fix large_fs space detection Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1373367918-7516-2-git-send-email-dmonakhov@openvz.org \
    --to=dmonakhov@openvz.org \
    --cc=dchinner@redhat.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).