[PATCH 1/4] large-fs: fix large_fs space detection

[PATCH 1/4] large-fs: fix large_fs space detection

[Dmitry Monakhov]

Currenly large_fs check compare $SCRATCH_DEV_EMPTY_SPACE and $fs_size
which is not correct because total empty size required is $SCRATCH_DEV_EMPTY_SPACE + 50Gb
This path fix space detection, so check becomes valid for all situations.

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 common/rc |   27 +++++++++++++++------------
 1 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/common/rc b/common/rc
index fe6bbfc..c44acea 100644
--- a/common/rc
+++ b/common/rc
@@ -306,16 +306,17 @@ _setup_large_xfs_fs()
 {
 	fs_size=$1
 	local tmp_dir=/tmp/
+	# Default free space in the FS is 50GB, but you can specify more via
+	# SCRATCH_DEV_EMPTY_SPACE
+	fs_empty_space=$((50*1024*1024*1024))
 
 	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
-	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
-	[ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
+	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
+	[ $fs_empty_space -ge $fs_size ] && return 0
 
 	# calculate the size of the file we need to allocate.
-	# Default free space in the FS is 50GB, but you can specify more via
-	# SCRATCH_DEV_EMPTY_SPACE
-	file_size=$(($fs_size - 50*1024*1024*1024))
-	file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
+	file_size=$(($fs_size - $fs_empty_space))
 
 	# mount the filesystem, create the file, unmount it
 	_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -434,15 +435,17 @@ _setup_large_ext4_fs()
 {
 	fs_size=$1
 	local tmp_dir=/tmp/
-
-	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
-	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
-	[ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0

[PATCH 2/4] large-fs: improve space diversification for ext4

[Dmitry Monakhov]

Currently we allocated several giant files one by one until limit,
so empty space is located as one chunk which limit code-path coverage.
This patch consume all space with NUM_SPACE_FILES files (by default 1024)
each has same size, and when truncate each one by required delta.
As result we have $NUM_SPACE_FILES chunks of free blocks distributed
across whole filesystem.
This should help us to avoid regressions similar to e7c9e3e99adf6c49

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 common/rc |   40 ++++++++++++++++------------------------
 1 files changed, 16 insertions(+), 24 deletions(-)

diff --git a/common/rc b/common/rc
index c44acea..902fc19 100644
--- a/common/rc
+++ b/common/rc
@@ -440,12 +440,17 @@ _setup_large_ext4_fs()
 	fs_empty_space=$((50*1024*1024*1024))
 
 	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
 	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
 	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
 	[ $fs_empty_space -ge $fs_size ] && return 0
 
 	# calculate the size of the file we need to allocate.
+
 	space_to_consume=$(($fs_size - $fs_empty_space))
+	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
+	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))
+
 	# mount the filesystem and create 16TB - 4KB files until we consume
 	# all the necessary space.
 	_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -457,33 +462,20 @@ _setup_large_ext4_fs()
 		return $status
 	fi
 	rm -f $tmp_dir/mnt.err
-
-	file_size=$((16*1024*1024*1024*1024 - 4096))
-	nfiles=0
-	while [ $space_to_consume -gt $file_size ]; do
-
+	mkdir $SCRATCH_MNT/.use_space
+	# Consume all space on filesytem
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $file_size" \
-			-c "falloc -k 0 $file_size" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-		if [ $status -ne 0 ]; then
-			break;
-		fi
-
-		space_to_consume=$(( $space_to_consume - $file_size ))
-		nfiles=$(($nfiles + 1))
+			-c "truncate $file_size_falloc" \
+			-c "falloc -k 0 $file_size_falloc" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
 	done
-
-	# consume the remaining space.
-	if [ $space_to_consume -gt 0 ]; then
+	# Truncate files to smaller size, will free chunks of space
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $space_to_consume" \
-			-c "falloc -k 0 $space_to_consume" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-	fi
-	export NUM_SPACE_FILES=$nfiles
+			-c "truncate $file_size_final" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
+	done
 
 	umount $SCRATCH_MNT
 	if [ $status -ne 0 ]; then
-- 
1.7.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/4] large-fs: improve space diversification for ext4

[Dave Chinner]

On Tue, Jul 09, 2013 at 03:05:16PM +0400, Dmitry Monakhov wrote:
> Currently we allocated several giant files one by one until limit,
> so empty space is located as one chunk which limit code-path coverage.
> This patch consume all space with NUM_SPACE_FILES files (by default 1024)
> each has same size, and when truncate each one by required delta.
> As result we have $NUM_SPACE_FILES chunks of free blocks distributed
> across whole filesystem.
> This should help us to avoid regressions similar to e7c9e3e99adf6c49

Sounds like a good idea - distributing free space around the
filesystem - but why limit this to ext4?  If you turn this into a
generic "largefs fill space" function, it will work just as well
with XFS as it does for ext4, and with any other filesystem that we
want to support --largefs testing on....

I'd also add a CLI option to check to set NUM_SPACE_FILES like we
do for LARGE_SCRATCH_DEV and SCRATCH_DEV_EMPTY_SPACE. I'd probably
also call it SCRATCH_DEV_EMPTY_SPACE_FILES....

> Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
> ---
>  common/rc |   40 ++++++++++++++++------------------------
>  1 files changed, 16 insertions(+), 24 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index c44acea..902fc19 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -440,12 +440,17 @@ _setup_large_ext4_fs()
>  	fs_empty_space=$((50*1024*1024*1024))
>  
>  	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
> +	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
>  	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
>  	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
>  	[ $fs_empty_space -ge $fs_size ] && return 0
>  
>  	# calculate the size of the file we need to allocate.
> +
>  	space_to_consume=$(($fs_size - $fs_empty_space))
> +	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
> +	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))

spaces around "/"

> +
>  	# mount the filesystem and create 16TB - 4KB files until we consume
>  	# all the necessary space.
>  	_scratch_mount 2>&1 >$tmp_dir/mnt.err
> @@ -457,33 +462,20 @@ _setup_large_ext4_fs()
>  		return $status
>  	fi
>  	rm -f $tmp_dir/mnt.err
> -
> -	file_size=$((16*1024*1024*1024*1024 - 4096))
> -	nfiles=0
> -	while [ $space_to_consume -gt $file_size ]; do
> -
> +	mkdir $SCRATCH_MNT/.use_space
> +	# Consume all space on filesytem
> +	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 

Is that bashism supported on older versions of bash? i.e. like the
versions found on RHEL5, SLES10, etc? If not, then a simple:

	for nfiles in `seq 0 1 $nfiles_total`; do

will work just as well....

>  		xfs_io -F -f \

change that to XFS_IO_PROG and we can drop the -F there.

> -			-c "truncate $file_size" \
> -			-c "falloc -k 0 $file_size" \
> -			$SCRATCH_MNT/.use_space.$nfiles 2>&1
> -		status=$?
> -		if [ $status -ne 0 ]; then
> -			break;
> -		fi
> -
> -		space_to_consume=$(( $space_to_consume - $file_size ))
> -		nfiles=$(($nfiles + 1))
> +			-c "truncate $file_size_falloc" \
> +			-c "falloc -k 0 $file_size_falloc" \
> +			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1

Is there any need for the truncate + falloc -k? I can't remember why
I did that in the first place. Just a "falloc 0 $file_size_falloc"
shoul dbe sufficient, right?

>  	done
> -
> -	# consume the remaining space.
> -	if [ $space_to_consume -gt 0 ]; then
> +	# Truncate files to smaller size, will free chunks of space
> +	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
>  		xfs_io -F -f \

Same again for XFS_IO_PROG.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

[PATCH 3/4] ext4: ignore valid errors from defragmentation tests

[Dmitry Monakhov]

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 tests/ext4/301 |    3 +++
 tests/ext4/302 |    3 +++
 tests/ext4/303 |    6 ++++++
 tests/ext4/304 |    3 +++
 4 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/tests/ext4/301 b/tests/ext4/301
index 0ca3622..30e4273 100755
--- a/tests/ext4/301
+++ b/tests/ext4/301
@@ -74,6 +74,9 @@ donorname=test1.def
 filename=test1
 inplace=0
 rw=write
+continue_on_error=write
+ignore_error=,ENOSPC:EBUSY
+error_dump=0
 numjobs=${NUM_JOBS}
 runtime=30*${TIME_FACTOR}
 time_based
diff --git a/tests/ext4/302 b/tests/ext4/302
index 497f4c6..5ce007b 100755
--- a/tests/ext4/302
+++ b/tests/ext4/302
@@ -76,6 +76,9 @@ donorname=test2.def
 filename=test2
 inplace=0
 rw=write
+continue_on_error=write
+ignore_error=,ENOSPC:EBUSY
+error_dump=0
 numjobs=${LOAD_FACTOR}
 runtime=30*${TIME_FACTOR}
 time_based
diff --git a/tests/ext4/303 b/tests/ext4/303
index 93354e5..f96264e 100755
--- a/tests/ext4/303
+++ b/tests/ext4/303
@@ -72,6 +72,9 @@ donorname=test3.def
 filename=test31
 inplace=0
 rw=write
+continue_on_error=write
+ignore_error=,ENOSPC:EBUSY
+error_dump=0
 numjobs=${LOAD_FACTOR}
 runtime=30*${TIME_FACTOR}
 time_based
@@ -84,6 +87,9 @@ donorname=test3.def
 filename=test32
 inplace=0
 rw=write
+continue_on_error=write
+ignore_error=,ENOSPC:EBUSY
+error_dump=0
 numjobs=${LOAD_FACTOR}
 runtime=30*${TIME_FACTOR}
 time_based
diff --git a/tests/ext4/304 b/tests/ext4/304
index 74e601a..5e03d4a 100755
--- a/tests/ext4/304
+++ b/tests/ext4/304
@@ -76,6 +76,9 @@ donorname=test4.def
 filename=test4
 inplace=1
 rw=randwrite
+continue_on_error=write
+ignore_error=,ENOSPC:EBUSY:EINVAL:61
+error_dump=0
 numjobs=4*${LOAD_FACTOR}
 runtime=30*${TIME_FACTOR}
 time_based
-- 
1.7.1

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 3/4] ext4: ignore valid errors from defragmentation tests

[Dave Chinner]

On Tue, Jul 09, 2013 at 03:05:17PM +0400, Dmitry Monakhov wrote:
> 
> Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
> ---
>  tests/ext4/301 |    3 +++
>  tests/ext4/302 |    3 +++
>  tests/ext4/303 |    6 ++++++
>  tests/ext4/304 |    3 +++
>  4 files changed, 15 insertions(+), 0 deletions(-)
> 
> diff --git a/tests/ext4/301 b/tests/ext4/301
> index 0ca3622..30e4273 100755
> --- a/tests/ext4/301
> +++ b/tests/ext4/301
> @@ -74,6 +74,9 @@ donorname=test1.def
>  filename=test1
>  inplace=0
>  rw=write
> +continue_on_error=write
> +ignore_error=,ENOSPC:EBUSY
> +error_dump=0
>  numjobs=${NUM_JOBS}
>  runtime=30*${TIME_FACTOR}
>  time_based

Seems harmless enough to me.

Acked-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 4/4] large-fs: fix ext4 defragmentation tests

[Dmitry Monakhov]

Initially work space size calculation was based on size of blkdev which
obviously wrong for large-fs case. This patch calculate work space based
on `df` cmd.

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 tests/ext4/301 |   20 +++++++++++---------
 tests/ext4/302 |   15 ++++++++-------
 tests/ext4/303 |   15 ++++++++-------
 tests/ext4/304 |   15 ++++++++-------
 4 files changed, 35 insertions(+), 30 deletions(-)

diff --git a/tests/ext4/301 b/tests/ext4/301
index 30e4273..8c698b5 100755
--- a/tests/ext4/301
+++ b/tests/ext4/301
@@ -45,18 +45,13 @@ _require_scratch
 _require_defrag
 
 NUM_JOBS=$((4*LOAD_FACTOR))
-BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
-# We need space for 2 files (test file, and donor one)
-# reserve 30% in order to avoid ENOSPC
-FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1))))
-
 cat >$fio_config <<EOF
 # Common e4defrag regression tests
 [global]
 ioengine=ioe_e4defrag
 iodepth=1
 directory=${SCRATCH_MNT}
-filesize=${FILE_SIZE}
+filesize=\${FIO_FILE_SIZE}
 size=999G
 buffered=0
 fadvise_hint=0
@@ -105,11 +100,18 @@ _workout()
 	echo ""
 	echo " Start defragment activity"
 	echo ""
+	free_space=`df -klP /mnt_scratch | grep -v Filesystem | awk '{print $2 - $3 }'`
+	# We need space for 2 files (test file and donor)
+	# reserve 20% in order to avoid ENOSPC (1024*0.80 == 816)
+	export FIO_FILE_SIZE=$((($free_space * 816) / 2))
+	echo "FIO_FILE_SIZE=$FIO_FILE_SIZE" >> $seqres.full
 	cat $fio_config >>  $seqres.full
-	run_check $FIO_PROG $fio_config
-}
+	run_check $FIO_PROG  $fio_config 
 
-_require_fio $fio_config
+}
+# Temproraly export FIO_FILE_SIZE which is required for fio's job validation
+export FIO_FILE_SIZE=16384
+_require_fio  $fio_config
 
 _scratch_mkfs  >> $seqres.full 2>&1
 _scratch_mount
diff --git a/tests/ext4/302 b/tests/ext4/302
index 5ce007b..e525977 100755
--- a/tests/ext4/302
+++ b/tests/ext4/302
@@ -45,18 +45,13 @@ _need_to_be_root
 _require_scratch
 _require_defrag
 
-BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
-# We need space for 2 files (test file, and donor one)
-# reserve 30% in order to avoid ENOSPC
-FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1))))
-
 cat >$fio_config <<EOF
 # Common e4defrag regression tests
 [global]
 ioengine=ioe_e4defrag
 iodepth=1
 directory=${SCRATCH_MNT}
-filesize=${FILE_SIZE}
+filesize=\${FIO_FILE_SIZE}
 size=999G
 buffered=0
 fadvise_hint=0
@@ -121,10 +116,16 @@ _workout()
 	echo ""
 	echo " Start defragment activity"
 	echo ""
+	free_space=`df -klP /mnt_scratch | grep -v Filesystem | awk '{print $2 - $3 }'`
+	# We need space for 2 files (test file, and donor one)
+	# reserve 20% in order to avoid ENOSPC (1024*0.80 == 816)
+	export FIO_FILE_SIZE=$((($free_space * 816) / 2))
+	echo "FIO_FILE_SIZE=$FIO_FILE_SIZE" >> $seqres.full
 	cat $fio_config >>  $seqres.full
 	run_check $FIO_PROG $fio_config
 }
-
+# Temproraly export FIO_FILE_SIZE which is required for fio's job validation
+export FIO_FILE_SIZE=16384
 _require_fio $fio_config
 
 _scratch_mkfs  >> $seqres.full 2>&1
diff --git a/tests/ext4/303 b/tests/ext4/303
index f96264e..6fb6357 100755
--- a/tests/ext4/303
+++ b/tests/ext4/303
@@ -45,18 +45,13 @@ _need_to_be_root
 _require_scratch
 _require_defrag
 
-BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
-# We need space for 3 files (one donor file and two test files)
-# Reserve space for 4 files in order to avoid ENOSPC
-FILE_SIZE=$((BLK_DEV_SIZE * (512 / (3+1))))
-
 cat >$fio_config <<EOF
 # Common e4defrag regression tests
 [global]
 ioengine=ioe_e4defrag
 iodepth=1
 directory=${SCRATCH_MNT}
-filesize=${FILE_SIZE}
+filesize=\${FIO_FILE_SIZE}
 size=999G
 buffered=0
 fadvise_hint=0
@@ -134,10 +129,16 @@ _workout()
 	echo ""
 	echo " Start defragment activity"
 	echo ""
+	free_space=`df -klP /mnt_scratch | grep -v Filesystem | awk '{print $2 - $3 }'`
+	# We need space for 3 files (test file and donor)
+	# reserve 20% in order to avoid ENOSPC (1024*0.80 == 816)
+	export FIO_FILE_SIZE=$((($free_space * 816) / 3))
+	echo "FIO_FILE_SIZE=$FIO_FILE_SIZE" >> $seqres.full
 	cat $fio_config >>  $seqres.full
 	run_check $FIO_PROG $fio_config
 }
-
+# Temproraly export FIO_FILE_SIZE which is required for fio's job validation
+export FIO_FILE_SIZE=16384
 _require_fio $fio_config
 
 _scratch_mkfs  >> $seqres.full 2>&1
diff --git a/tests/ext4/304 b/tests/ext4/304
index 5e03d4a..7cbf1dd 100755
--- a/tests/ext4/304
+++ b/tests/ext4/304
@@ -46,18 +46,13 @@ _need_to_be_root
 _require_scratch
 _require_defrag
 
-BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
-# We need space for 2 files (test file, and donor one)
-# reserve 30% in order to avoid ENOSPC
-FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1))))
-
 cat >$fio_config <<EOF
 # Common e4defrag regression tests
 [global]
 ioengine=ioe_e4defrag
 iodepth=1
 directory=${SCRATCH_MNT}
-filesize=${FILE_SIZE}
+filesize=\${FIO_FILE_SIZE}
 size=999G
 buffered=0
 fadvise_hint=0
@@ -108,10 +103,16 @@ _workout()
 	echo ""
 	echo " Start defragment activity"
 	echo ""
+	free_space=`df -klP /mnt_scratch | grep -v Filesystem | awk '{print $2 - $3 }'`
+	# We need space for test file only
+	# reserve 20% in order to avoid ENOSPC (1024*0.80 == 816)
+	export FIO_FILE_SIZE=$((($free_space * 816) / 1))
+	echo "FIO_FILE_SIZE=$FIO_FILE_SIZE" >> $seqres.full
 	cat $fio_config >>  $seqres.full
 	run_check $FIO_PROG $fio_config
 }
-
+# Temproraly export FIO_FILE_SIZE which is required for fio's job validation
+export FIO_FILE_SIZE=16384
 _require_fio $fio_config
 
 _scratch_mkfs  >> $seqres.full 2>&1
-- 
1.7.1

Re: [PATCH 4/4] large-fs: fix ext4 defragmentation tests

[Dave Chinner]

On Tue, Jul 09, 2013 at 03:05:18PM +0400, Dmitry Monakhov wrote:
> Initially work space size calculation was based on size of blkdev which
> obviously wrong for large-fs case. This patch calculate work space based
> on `df` cmd.

when have a "_used" command in common/rc for getting the used space
on a device (i.e. _used $SCRATCH_DEV). This loocks like a case for a
"_free" equivalent....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 1/4] large-fs: fix large_fs space detection

[Dave Chinner]

On Tue, Jul 09, 2013 at 03:05:15PM +0400, Dmitry Monakhov wrote:
> Currenly large_fs check compare $SCRATCH_DEV_EMPTY_SPACE and $fs_size
> which is not correct because total empty size required is $SCRATCH_DEV_EMPTY_SPACE + 50Gb
> This path fix space detection, so check becomes valid for all situations.

I'm not sure what problem you're fixing from this description?

It's takenme a few minutes to work out that:

"If SCRATCH_DEV_EMPTY_SPACE + 50GB is larger than the filesystem
size being tested, then the configuration being tested is invalid
and should fail. Currently we only check that
SCRATCH_DEV_EMPTY_SPACE is greater than the the filesystem size. Fix
it to check the combined empty space fits in the filesystem being
tested."

Otherwise the change looks good.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com