[PATCH v2 0/3] xfstests: Add a group of reservation space test

[PATCH v2 0/3] xfstests: Add a group of reservation space test

[WuBo]

Changes v1->v2:
- Make test-set support generic

This patch set add a group of reservation space test. Especailly for btrfs.
It includes three parts: copy workload, prealloc and write posix.

For test 273, I hope it's usefull for Josef's reserve improve work.
For test 274 and 275, the current btrfs is not pass yet.

Wu Bo (3):
  xfstests 273: add a copy and reserve test
  xfstests 274: add a prealloc and reserve test
  xfstests 275: add a write and reserve test

 273     |  145 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 273.out |    4 ++
 274     |   91 +++++++++++++++++++++++++++++++++++++++
 274.out |    5 ++
 275     |   89 ++++++++++++++++++++++++++++++++++++++
 275.out |    5 ++
 group   |    3 +
 7 files changed, 342 insertions(+), 0 deletions(-)
 create mode 100755 273
 create mode 100644 273.out
 create mode 100755 274
 create mode 100644 274.out
 create mode 100755 275
 create mode 100644 275.out

-- 
1.7.3.1

[PATCH v2 1/3] xfstests 273: add a copy and reserve test

[WuBo]

This test is a stress test. It creates a set of threads for coping small files 
into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is 
not full under kenerl 3.0 with intel64.

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 273     |  145 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 273.out |    4 ++
 group   |    1 +
 3 files changed, 150 insertions(+), 0 deletions(-)
 create mode 100755 273
 create mode 100644 273.out

diff --git a/273 b/273
new file mode 100755
index 0000000..2965132
--- /dev/null
+++ b/273
@@ -0,0 +1,145 @@
+#! /bin/bash
+# FS QA Test No. 273
+#
+# reservation test with heavy cp workload
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0	# success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -rf $SCRATCH_MNT/* $tmp.*
+	_scratch_unmount
+}
+
+. ./common.rc
+. ./common.filter
+
+threads=50
+count=1
+
+_threads_set()
+{
+	_cpu_num=`cat /proc/cpuinfo | grep "processor" | wc -l`
+	threads=$(($_cpu_num * 50))
+	if [ $threads -gt 200 ]
+	then
+		threads=200
+	fi
+}
+
+_file_create()
+{
+	_i=0
+
+	if ! mkdir $SCRATCH_MNT/origin
+	then
+		echo "mkdir origin err"
+		status=1
+		exit
+	fi
+
+	cd $SCRATCH_MNT/origin
+
+	_disksize=`df --block-size=1 | grep $SCRATCH_DEV | awk '{print $2}'`
+	_disksize=$(($_disksize / 3))
+	_num=$(($_disksize / $count / $threads / 4096))
+	_count=$count
+	while [ $_i -lt $_num ]
+	do
+		dd if=/dev/zero of=file_$_i bs=4096 count=$_count >/dev/null 2>&1
+		_i=$(($_i + 1))
+	done
+
+	cd $here
+}
+
+_porter()
+{
+	_suffix=$1
+
+	if ! mkdir $SCRATCH_MNT/sub_$_suffix
+	then
+		echo "mkdir sub_xxx err"
+		status=1
+		exit
+	fi
+
+	cp -r $SCRATCH_MNT/origin $SCRATCH_MNT/sub_$_suffix >$seq.full 2>&1
+	if [ $? -ne 0 ]
+	then
+		echo "_porter $_suffix not complete"
+	fi
+	
+	sync
+}
+
+_do_workload()
+{
+	_pids=""
+	_pid=1
+	
+	rm -rf $SCRATCH_MNT/*
+
+	_threads_set
+	_file_create
+
+	_threads=$threads
+
+	while [ $_pid -lt $_threads ]
+	do
+		_porter $_pid &
+		_pids="$_pids $!"
+		_pid=$(($_pid + 1))
+	done
+
+	wait $_pids
+}
+
+# real QA test starts here
+_supported_fs generic
+_supported_os IRIX Linux
+_require_scratch
+
+echo "------------------------------"
+echo "start the workload"
+echo "------------------------------"
+
+rm -f $seq.full
+
+umount $SCRATCH_DEV 2>/dev/null
+_scratch_mkfs_sized $((2 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
+_scratch_mount
+
+_do_workload
+
+_check_scratch_fs
+status=$?
+exit
diff --git a/273.out b/273.out
new file mode 100644
index 0000000..146af43
--- /dev/null
+++ b/273.out
@@ -0,0 +1,4 @@
+QA output created by 273
+------------------------------
+start the workload
+------------------------------
diff --git a/group b/group
index 616cbdd..b8245bf 100644
--- a/group
+++ b/group
@@ -386,3 +386,4 @@ deprecated
 270 auto quota rw prealloc ioctl enospc
 271 auto rw quick
 272 auto enospc rw
+273 auto rw
-- 
1.7.3.1

[PATCH v2 2/3] xfstests 274: add a prealloc and reserve test

[WuBo]

This test is for preallocation test. If the disk is full, just with a prealloc
file has some free space that prealloc early. We need to check whether the write
to the free space is success or not.

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 274     |   91 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 274.out |    5 +++
 group   |    1 +
 3 files changed, 97 insertions(+), 0 deletions(-)
 create mode 100755 274
 create mode 100644 274.out

diff --git a/274 b/274
new file mode 100755
index 0000000..b658004
--- /dev/null
+++ b/274
@@ -0,0 +1,91 @@
+#! /bin/bash
+# FS QA Test No. 274
+#
+# preallocation test
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0    # success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $SCRATCH_MNT/* $tmp.*
+	_scratch_unmount
+}
+
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+_supported_fs generic
+_supported_os IRIX Linux
+_require_scratch
+
+echo "------------------------------"
+echo "preallocation test"
+echo "------------------------------"
+
+rm -f $seq.full
+
+umount $SCRATCH_DEV 2>/dev/null
+_scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
+_scratch_mount
+
+rm -rf $SCRATCH_MNT/*
+cd $SCRATCH_MNT
+dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "create file err"
+	status=1
+	exit
+fi
+
+fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "fallocate file err"
+	status=1
+	exit
+fi
+
+dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
+dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+sync
+
+dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "fill prealloc range err"
+	status=1
+	exit
+fi
+
+echo "done"
+exit
diff --git a/274.out b/274.out
new file mode 100644
index 0000000..edbf3cc
--- /dev/null
+++ b/274.out
@@ -0,0 +1,5 @@
+QA output created by 274
+------------------------------
+preallocation test
+------------------------------
+done
diff --git a/group b/group
index b8245bf..44a9999 100644
--- a/group
+++ b/group
@@ -387,3 +387,4 @@ deprecated
 271 auto rw quick
 272 auto enospc rw
 273 auto rw
+274 auto rw
-- 
1.7.3.1

[PATCH v2 3/3] xfstests 275: add a write and reserve test

[WuBo]

This test is for write-posix test. If writing a file when the disk is almost
full, the posix wants the call to write as much as possible but not none.

quote the POSIX:
If a write() requests that more bytes be written than there is room for
(for example, [XSI] [Option Start] the process' file size limit or
[Option End] the physical end of a medium), only as many bytes as there
is room for shall be written. For example, suppose there is space for 20
bytes more in a file before reaching a limit. A write of 512 bytes will
return 20. The next write of a non-zero number of bytes would give a
failure return (except as noted below).

Signed-off-by: Wu Bo <Wu.Bo@cn.fujitsu.com>
---
 275     |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 275.out |    5 +++
 group   |    1 +
 3 files changed, 95 insertions(+), 0 deletions(-)
 create mode 100755 275
 create mode 100644 275.out

diff --git a/275 b/275
new file mode 100755
index 0000000..214262e
--- /dev/null
+++ b/275
@@ -0,0 +1,89 @@
+#! /bin/bash
+# FS QA Test No. 275
+#
+# The posix write test. when write size is larger than disk free size,
+# should write as more as possible
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+#creator
+owner=wu.bo@cn.fujitsu.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=0    # success is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $SCRATCH_MNT/* $tmp.*
+	_scratch_unmount
+}
+
+. ./common.rc
+. ./common.filter
+
+# real QA test starts here
+_supported_fs generic
+_supported_os IRIX Linux
+_require_scratch
+
+echo "------------------------------"
+echo "write lack test"
+echo "------------------------------"
+
+rm -f $seq.full
+
+umount $SCRATCH_DEV 2>/dev/null
+_scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
+_scratch_mount
+
+rm -rf $SCRATCH_MNT/*
+cd $SCRATCH_MNT
+
+dd if=/dev/zero of=tmp1 bs=4K count=1 >/dev/null 2>&1
+if [ $? -ne 0 ]
+then
+	echo "create file err"
+	status=1
+	exit
+fi
+
+dd if=/dev/zero of=tmp2 bs=1M >/dev/null 2>&1
+dd if=/dev/zero of=tmp3 bs=4K >/dev/null 2>&1
+sync
+
+rm -f tmp1
+sync
+
+dd if=/dev/zero of=tmp1 bs=8K count=1 >/dev/null 2>&1
+_filesize=`du tmp1 | awk '{print $1}'`
+if [ $_filesize -ne 4 ]
+then
+	echo "write file err"
+	status=1
+	exit
+fi
+
+echo "done"
+exit
diff --git a/275.out b/275.out
new file mode 100644
index 0000000..30af43c
--- /dev/null
+++ b/275.out
@@ -0,0 +1,5 @@
+QA output created by 275
+------------------------------
+write lack test
+------------------------------
+done
diff --git a/group b/group
index 44a9999..8cdddd6 100644
--- a/group
+++ b/group
@@ -388,3 +388,4 @@ deprecated
 272 auto enospc rw
 273 auto rw
 274 auto rw
+275 auto rw
-- 
1.7.3.1

Re: [PATCH v2 0/3] xfstests: Add a group of reservation space test

[Christoph Hellwig]

On Tue, Nov 08, 2011 at 11:38:41AM +0800, WuBo wrote:
> Changes v1->v2:
> - Make test-set support generic
> 
> This patch set add a group of reservation space test. Especailly for btrfs.
> It includes three parts: copy workload, prealloc and write posix.
> 
> For test 273, I hope it's usefull for Josef's reserve improve work.
> For test 274 and 275, the current btrfs is not pass yet.
> 
> Wu Bo (3):
>   xfstests 273: add a copy and reserve test
>   xfstests 274: add a prealloc and reserve test
>   xfstests 275: add a write and reserve test

The patches look good to me.  Unless someone else complains until
tomorrow I'll put them in.