[PATCH 0/10] xfstests: rework large filesystem testing

[PATCH 0/10] xfstests: rework large filesystem testing

[Dave Chinner]

This is a resend of a series that has been sitting inmy local
workarea for months. It reworks the large filesystem testing
infrastructure for XFS to allow it to use xfs_repair by switching
the filesystem filling to using preallocation rather than xfs_db
hacks.

It also introduces support for ext4 testing, also via preallocation
to fill the filesystem. btrfs isn't yet supported because I haven't
worked out how to reliably get the size of the filesystem and
calculate how much space is actually available in it.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 01/10] xfstests: add --largefs check option

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Make it easier to check large filesystems quickly by adding a
--large-fs option to check to turn on shortcuts for large scratch
device filesystem testing.

Also, reject invalid command line options with a usage message.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common |   63 +++++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 39 insertions(+), 24 deletions(-)

diff --git a/common b/common
index 0723224..7d13078 100644
--- a/common
+++ b/common
@@ -27,6 +27,35 @@ _setenvironment()
     export MSGVERB
 }
 
+usage()
+{
+    echo "Usage: $0 [options] [testlist]"'
+
+common options
+    -v			verbose
+
+check options
+    -xfs                test XFS (default)
+    -udf                test UDF
+    -nfs                test NFS
+    -l			line mode diff
+    -xdiff		graphical mode diff
+    -udiff		show unified diff (default)
+    -n			show me, do not run tests
+    -q			quick [deprecated]
+    -T			output timestamps
+    -r			randomize test order
+    --large-fs		optimise scratch device for large filesystems
+
+testlist options
+    -g group[,group...]	include tests from these groups
+    -x group[,group...]	exclude tests from these groups
+    NNN			include test NNN
+    NNN-NNN		include test range (eg. 012-021)
+'
+	    exit 0
+}
+
 here=`pwd`
 rm -f $here/$iam.out
 _setenvironment
@@ -117,30 +146,7 @@ s/ .*//p
     in
 
 	-\? | -h | --help)	# usage
-	    echo "Usage: $0 [options] [testlist]"'
-
-common options
-    -v			verbose
-
-check options
-    -xfs                test XFS (default)
-    -udf                test UDF
-    -nfs                test NFS
-    -l			line mode diff
-    -xdiff		graphical mode diff
-    -udiff		show unified diff (default)
-    -n			show me, do not run tests
-    -q			quick [deprecated]
-    -T			output timestamps
-    -r 			randomize test order
-    
-testlist options
-    -g group[,group...]	include tests from these groups
-    -x group[,group...]	exclude tests from these groups
-    NNN			include test NNN
-    NNN-NNN		include test range (eg. 012-021)
-'
-	    exit 0
+	    usage
 	    ;;
 
 	-udf)	# -udf ... set FSTYP to udf
@@ -232,6 +238,15 @@ testlist options
 	    fi
 	    ;;
 
+	--large-fs)
+	    export USE_BIG_LOOPFS=yes
+	    xpand=false
+	    ;;
+
+	-*)
+	    usage
+	    ;;
+
 	*)
 	    start=$r
 	    end=$r
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 01/10] xfstests: add --largefs check option

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Make it easier to check large filesystems quickly by adding a
> --large-fs option to check to turn on shortcuts for large scratch
> device filesystem testing.
>
> Also, reject invalid command line options with a usage message.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common |   63 +++++++++++++++++++++++++++++++++++++++------------------------
>   1 file changed, 39 insertions(+), 24 deletions(-)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 01/10] xfstests: add --largefs check option

[Christoph Hellwig]

On Thu, Jul 26, 2012 at 06:39:40PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Make it easier to check large filesystems quickly by adding a
> --large-fs option to check to turn on shortcuts for large scratch
> device filesystem testing.
> 
> Also, reject invalid command line options with a usage message.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
easily as loop devices. It really means we are testing a large scratch device
and that we should enable the special filesystem filling and checking options
that enable xfstests to be run sanely on large XFS filesystems.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 004       |    2 +-
 015       |    2 +-
 030       |    2 +-
 031       |    2 +-
 032       |    2 +-
 033       |    4 ++--
 041       |    2 +-
 049       |    2 +-
 083       |    2 +-
 092       |    2 +-
 148       |    2 +-
 149       |    2 +-
 common    |    2 +-
 common.rc |   12 ++++++------
 setup     |    5 +++--
 15 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/004 b/004
index 9f28e17..23729da 100755
--- a/004
+++ b/004
@@ -64,7 +64,7 @@ _supported_os IRIX Linux
 
 _need_to_be_root
 _require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
 
 rm -f $seq.full
 
diff --git a/015 b/015
index 4206b93..a99f1ed 100755
--- a/015
+++ b/015
@@ -52,7 +52,7 @@ _supported_fs generic
 _supported_os IRIX Linux
 
 _require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
 
 _scratch_mkfs_sized `expr 50 \* 1024 \* 1024` >/dev/null 2>&1 \
     || _fail "mkfs failed"
diff --git a/030 b/030
index fe57372..a8fb92c 100755
--- a/030
+++ b/030
@@ -64,8 +64,8 @@ _check_ag()
 _supported_fs xfs
 _supported_os IRIX Linux
 
-_require_nobigloopfs
 _require_scratch
+_require_no_large_scratch_dev
 
 DSIZE="-dsize=100m,agcount=6"
 
diff --git a/031 b/031
index b062277..fb6f15b 100755
--- a/031
+++ b/031
@@ -97,8 +97,8 @@ EOF
 _supported_fs xfs
 _supported_os IRIX Linux
 
-_require_nobigloopfs
 _require_scratch
+_require_no_large_scratch_dev
 
 # sanity test - default + one root directory entry
 # Note: must do this proto/mkfs now for later inode size calcs
diff --git a/032 b/032
index 4261ca2..d093b45 100755
--- a/032
+++ b/032
@@ -41,8 +41,8 @@ rm -f $seq.full
 _supported_fs xfs
 _supported_os Linux
 
-_require_nobigloopfs
 _require_scratch
+_require_no_large_scratch_dev
 
 echo "Silence is golden."
 for fs in `echo /sbin/mkfs.* | sed -e 's/.sbin.mkfs.//g'`
diff --git a/033 b/033
index 9651f26..68a688e 100755
--- a/033
+++ b/033
@@ -76,9 +76,9 @@ _filter_bad_ids()
 # real QA test starts here
 _supported_fs xfs
 _supported_os IRIX Linux
- 
-_require_nobigloopfs
+
 _require_scratch
+_require_no_large_scratch_dev
 
 # devzero blows away 512byte blocks, so make 512byte inodes (at least)
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
diff --git a/041 b/041
index 2800811..28dcb33 100755
--- a/041
+++ b/041
@@ -50,7 +50,7 @@ _supported_fs xfs
 _supported_os IRIX Linux
 
 _require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
 umount $SCRATCH_DEV 2>/dev/null
 
 _fill()
diff --git a/049 b/049
index c3065ad..149c4b5 100755
--- a/049
+++ b/049
@@ -60,9 +60,9 @@ _log()
     echo "--- $*" >> $seq.full
 }
 
-_require_nobigloopfs
 _require_nonexternal
 _require_scratch
+_require_no_large_scratch_dev
 _require_loop
 _require_ext2
 
diff --git a/083 b/083
index e0670b9..7a73f30 100755
--- a/083
+++ b/083
@@ -58,7 +58,7 @@ _supported_fs generic
 _supported_os IRIX Linux
 
 _require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
 
 rm -f $seq.full
 
diff --git a/092 b/092
index 429fa80..02ccc71 100755
--- a/092
+++ b/092
@@ -48,7 +48,7 @@ _cleanup()
 _supported_fs xfs
 _supported_os IRIX Linux
 _require_scratch
-_require_nobigloopfs
+_require_no_large_scratch_dev
 
 MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
 _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
diff --git a/148 b/148
index 76cbf37..7bb1722 100755
--- a/148
+++ b/148
@@ -66,8 +66,8 @@ _check_ag()
 _supported_fs xfs
 _supported_os IRIX Linux
 
-_require_nobigloopfs
 _require_scratch
+_require_no_large_scratch_dev
 
 DSIZE="-dsize=100m"
 
diff --git a/149 b/149
index 5131a45..193e6d7 100755
--- a/149
+++ b/149
@@ -100,8 +100,8 @@ EOF
 _supported_fs xfs
 _supported_os IRIX Linux
 
-_require_nobigloopfs
 _require_scratch
+_require_no_large_scratch_dev
 
 # sanity test - default + one root directory entry
 # Note: must do this proto/mkfs now for later inode size calcs
diff --git a/common b/common
index 7d13078..da86cd9 100644
--- a/common
+++ b/common
@@ -239,7 +239,7 @@ s/ .*//p
 	    ;;
 
 	--large-fs)
-	    export USE_BIG_LOOPFS=yes
+	    export LARGE_SCRATCH_DEV=yes
 	    xpand=false
 	    ;;
 
diff --git a/common.rc b/common.rc
index 602513a..9517904 100644
--- a/common.rc
+++ b/common.rc
@@ -310,7 +310,7 @@ _scratch_mkfs_xfs()
     cat $tmp_dir.mkfsstd
     rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
 
-    if [ "$USE_BIG_LOOPFS" = yes ]; then
+    if [ "$LARGE_SCRATCH_DEV" = yes ]; then
         [ -z "$RETAIN_AG_BYTES" ] && RETAIN_AG_BYTES=0
         ./tools/ag-wipe -q -r $RETAIN_AG_BYTES $SCRATCH_DEV
     fi
@@ -432,7 +432,7 @@ _scratch_xfs_repair()
         SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
     [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
         SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -r$SCRATCH_RTDEV"
-    [ "$USE_BIG_LOOPFS" = yes ] && SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
+    [ "$LARGE_SCRATCH_DEV" = yes ] && SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
     $XFS_REPAIR_PROG $SCRATCH_OPTIONS $* $SCRATCH_DEV
 }
 
@@ -821,9 +821,9 @@ _require_ext2()
 
 # this test requires that (large) loopback device files are not in use
 #
-_require_nobigloopfs()
+_require_no_large_scratch_dev()
 {
-    [ "$USE_BIG_LOOPFS" = yes ] && \
+    [ "$LARGE_SCRATCH_DEV" = yes ] && \
 	_notrun "Large filesystem testing in progress, skipped this test"
 }
 
@@ -1202,7 +1202,7 @@ _check_xfs_filesystem()
 
     [ "$FSTYP" != xfs ] && return 0
     testoption=""
-    [ "$USE_BIG_LOOPFS" = yes ] && testoption=-t
+    [ "$LARGE_SCRATCH_DEV" = yes ] && testoption=-t
 
     type=`_fs_type $device`
     ok=1
@@ -1241,7 +1241,7 @@ _check_xfs_filesystem()
         ok=0
     fi
     # repair doesn't scale massively at this stage, optionally skip it for now
-    [ "$USE_BIG_LOOPFS" = yes ] || \
+    [ "$LARGE_SCRATCH_DEV" = yes ] || \
     $XFS_REPAIR_PROG -n $extra_log_options $extra_rt_options $device >$tmp.repair 2>&1
     if [ $? -ne 0 ]
     then
diff --git a/setup b/setup
index 62798cc..5225951 100755
--- a/setup
+++ b/setup
@@ -23,7 +23,7 @@ fi
 
 [ "$USE_EXTERNAL"    = yes ] || USE_EXTERNAL=no
 [ "$USE_LBD_PATCH"   = yes ] || USE_LBD_PATCH=no
-[ "$USE_BIG_LOOPFS"  = yes ] || USE_BIG_LOOPFS=no
+[ "$LARGE_SCRATCH_DEV"  = yes ] || LARGE_SCRATCH_DEV=no
 [ "$USE_ATTR_SECURE" = yes ] || USE_ATTR_SECURE=no
 [ -z "$FSTYP" ] && FSTYP="xfs"
 
@@ -31,5 +31,6 @@ cat <<EOF
 TEST: DIR=$TEST_DIR DEV=$TEST_DEV rt=[$TEST_RTDEV] log=[$TEST_LOGDEV]
 TAPE: dev=[$TAPE_DEV] rmt=[$RMT_TAPE_DEV] rmtirix=[$RMT_TAPE_USER@$RMT_IRIXTAPE_DEV]
 SCRATCH: MNT=$SCRATCH_MNT DEV=$SCRATCH_DEV rt=[$SCRATCH_RTDEV] log=[$SCRATCH_LOGDEV]
-VARIABLES: external=$USE_EXTERNAL largeblk=$USE_LBD_PATCH fstyp=$FSTYP bigloopfs=$USE_BIG_LOOPFS attrsecure=$USE_ATTR_SECURE
+VARIABLES: external=$USE_EXTERNAL largeblk=$USE_LBD_PATCH fstyp=$FSTYP
+	   large_scratch_dev=$LARGE_SCRATCH_DEV attrsecure=$USE_ATTR_SECURE
 EOF
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> easily as loop devices. It really means we are testing a large scratch device
> and that we should enable the special filesystem filling and checking options
> that enable xfstests to be run sanely on large XFS filesystems.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   004       |    2 +-
>   015       |    2 +-
>   030       |    2 +-
>   031       |    2 +-
>   032       |    2 +-
>   033       |    4 ++--
>   041       |    2 +-
>   049       |    2 +-
>   083       |    2 +-
>   092       |    2 +-
>   148       |    2 +-
>   149       |    2 +-
>   common    |    2 +-
>   common.rc |   12 ++++++------
>   setup     |    5 +++--
>   15 files changed, 23 insertions(+), 22 deletions(-)
>

>   MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
>   _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
> diff --git a/148 b/148
> index 76cbf37..7bb1722 100755
> --- a/148
> +++ b/148
> @@ -66,8 +66,8 @@ _check_ag()
>   _supported_fs xfs
>   _supported_os IRIX Linux

#RCJ As long as you are making changes to this file, please remove Linux 
from the _supported_os as xfs_prepair64 is an IRIX specific binary.

>
> -_require_nobigloopfs
>   _require_scratch
> +_require_no_large_scratch_dev
>
>   DSIZE="-dsize=100m"
>
> diff --git a/149 b/149
> index 5131a45..193e6d7 100755
> --- a/149
> +++ b/149
> @@ -100,8 +100,8 @@ EOF
>   _supported_fs xfs
>   _supported_os IRIX Linux

#RCJ As long as you are making changes to this file, please remove Linux 
from the _supported_os as xfs_prepair is an IRIX specific binary.

>
> -_require_nobigloopfs
>   _require_scratch
> +_require_no_large_scratch_dev
>
>   # sanity test - default + one root directory entry
>   # Note: must do this proto/mkfs now for later inode size calcs

Otherwise it looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic

[Dave Chinner]

On Tue, Aug 28, 2012 at 09:01:35AM -0500, Rich Johnston wrote:
> On 07/26/2012 03:39 AM, Dave Chinner wrote:
> >From: Dave Chinner <dchinner@redhat.com>
> >
> >USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> >easily as loop devices. It really means we are testing a large scratch device
> >and that we should enable the special filesystem filling and checking options
> >that enable xfstests to be run sanely on large XFS filesystems.
> >
> >Signed-off-by: Dave Chinner <dchinner@redhat.com>
> >---
> >  004       |    2 +-
> >  015       |    2 +-
> >  030       |    2 +-
> >  031       |    2 +-
> >  032       |    2 +-
> >  033       |    4 ++--
> >  041       |    2 +-
> >  049       |    2 +-
> >  083       |    2 +-
> >  092       |    2 +-
> >  148       |    2 +-
> >  149       |    2 +-
> >  common    |    2 +-
> >  common.rc |   12 ++++++------
> >  setup     |    5 +++--
> >  15 files changed, 23 insertions(+), 22 deletions(-)
> >
> 
> >  MOUNT_OPTIONS="$MOUNT_OPTIONS -o inode64"
> >  _scratch_mkfs_xfs | _filter_mkfs 2>/dev/null
> >diff --git a/148 b/148
> >index 76cbf37..7bb1722 100755
> >--- a/148
> >+++ b/148
> >@@ -66,8 +66,8 @@ _check_ag()
> >  _supported_fs xfs
> >  _supported_os IRIX Linux
> 
> #RCJ As long as you are making changes to this file, please remove
> Linux from the _supported_os as xfs_prepair64 is an IRIX specific
> binary.

Such a change belongs in a separate patch, not intermingled with a
patch in a series that has nothing to do with xfs_prepair64....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 02/10] xfstests: rename USE_BIG_LOOPFS to be more generic

[Christoph Hellwig]

On Thu, Jul 26, 2012 at 06:39:41PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> USE_BIG_LOOPFS is really misnamed - it can be used on real devices just as
> easily as loop devices. It really means we are testing a large scratch device
> and that we should enable the special filesystem filling and checking options
> that enable xfstests to be run sanely on large XFS filesystems.

Looks fine, even if I don't like the new name too much either.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 03/10] xfstests: rename RETAIN_AG_BYTES

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Rename the $RETAIN_AG_BYTES variable to be more generic so that it
reflects the fact that it is designed to retain a certain amount of
extra free space above the default amount in the filesystem when
doing large scratch device testing.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common.rc |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/common.rc b/common.rc
index 9517904..b0af482 100644
--- a/common.rc
+++ b/common.rc
@@ -311,8 +311,8 @@ _scratch_mkfs_xfs()
     rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
 
     if [ "$LARGE_SCRATCH_DEV" = yes ]; then
-        [ -z "$RETAIN_AG_BYTES" ] && RETAIN_AG_BYTES=0
-        ./tools/ag-wipe -q -r $RETAIN_AG_BYTES $SCRATCH_DEV
+        [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+        ./tools/ag-wipe -q -r $SCRATCH_DEV_EMPTY_SPACE $SCRATCH_DEV
     fi
 
     return $mkfs_status
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Rename the $RETAIN_AG_BYTES variable to be more generic so that it
> reflects the fact that it is designed to retain a certain amount of
> extra free space above the default amount in the filesystem when
> doing large scratch device testing.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common.rc |    4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 03/10] xfstests: rename RETAIN_AG_BYTES

[Christoph Hellwig]

On Thu, Jul 26, 2012 at 06:39:42PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> Rename the $RETAIN_AG_BYTES variable to be more generic so that it
> reflects the fact that it is designed to retain a certain amount of
> extra free space above the default amount in the filesystem when
> doing large scratch device testing.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 04/10] xfstests: use preallocation for ag-wiper

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

To enable sane testing of large scale filesystems, the --large-fs
test option uses xfs_db magic to mark AGs full without doing any IO.
This leaves only a small amount of free space left in the filesystem
to stress the high AGs of the filesystem rather than the low AGs.

This method requires us to have special filesystem check options to
avoid free space checking in xfs_check, and we cannot current run
xfs_repair on such a filesystem at all. As it is, free space
checking on xfs_check does not scale, so we still need to avoid this
checking regardless of how we fill the filesystem.

We can acheive exactly the same fill behaviour by preallocating a
single large file in the filesystem immediately after creating it.
This is a filesystem independent manner of filling the filesystem,
and allows us to do large filesystem testing on more than just XFS.

Further, this preallocation method effectively adds a new "very
large file" test. It also enables us to run an unmodified xfs_repair
or filesystem specific fsck program to check the filesystem for
sanity, so we can now do full sanity checking of such large
filesystems.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common.rc |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 53 insertions(+), 5 deletions(-)

diff --git a/common.rc b/common.rc
index b0af482..7bb1133 100644
--- a/common.rc
+++ b/common.rc
@@ -276,6 +276,47 @@ _scratch_mkfs_options()
     echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV
 }
 
+
+_setup_large_xfs_fs()
+{
+	fs_size=$1
+	local tmp_dir=/tmp/
+
+	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+	[ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+
+	# calculate the size of the file we need to allocate.
+	# Default free space in the FS is 50GB, but you can specify more via
+	# SCRATCH_DEV_EMPTY_SPACE
+	file_size=$(($fs_size - 50*1024*1024*1024))
+	file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
+
+	# mount the filesystem, create the file, unmount it
+	_scratch_mount 2>&1 >$tmp_dir/mnt.err
+	local status=$?
+	if [ $status -ne 0 ]; then
+		echo "mount failed"
+		cat $tmp_dir/mnt.err >&2
+		rm -f $tmp_dir/mnt.err
+		return $status
+	fi
+	rm -f $tmp_dir/mnt.err
+
+	xfs_io -F -f \
+		-c "truncate $file_size" \
+		-c "falloc -k 0 $file_size" \
+		$SCRATCH_MNT/.use_space 2>&1 > /dev/null
+	status=$?
+	umount $SCRATCH_MNT
+	if [ $status -ne 0 ]; then
+		echo "large file prealloc failed"
+		cat $tmp_dir/mnt.err >&2
+		return $status
+	fi
+	return 0
+}
+
 _scratch_mkfs_xfs()
 {
     # extra mkfs options can be added by tests
@@ -305,16 +346,23 @@ _scratch_mkfs_xfs()
         mkfs_status=$?
     fi
 
+    if [ $mkfs_status -eq 0 -a "$LARGE_SCRATCH_DEV" = yes ]; then
+	# manually parse the mkfs output to get the fs size in bytes
+	local fs_size
+	fs_size=`cat $tmp_dir.mkfsstd | perl -ne '
+	    if (/^data\s+=\s+bsize=(\d+)\s+blocks=(\d+)/) {
+		my $size = $1 * $2;
+		print STDOUT "$size\n";
+	    }'`
+	_setup_large_xfs_fs $fs_size
+	mkfs_status=$?
+    fi
+
     # output stored mkfs output
     cat $tmp_dir.mkfserr >&2
     cat $tmp_dir.mkfsstd
     rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
 
-    if [ "$LARGE_SCRATCH_DEV" = yes ]; then
-        [ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
-        ./tools/ag-wipe -q -r $SCRATCH_DEV_EMPTY_SPACE $SCRATCH_DEV
-    fi
-
     return $mkfs_status
 }
 
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 04/10] xfstests: use preallocation for ag-wiper

[Rich Johnston]

[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> To enable sane testing of large scale filesystems, the --large-fs
> test option uses xfs_db magic to mark AGs full without doing any IO.
> This leaves only a small amount of free space left in the filesystem
> to stress the high AGs of the filesystem rather than the low AGs.
>
> This method requires us to have special filesystem check options to
> avoid free space checking in xfs_check, and we cannot current run
> xfs_repair on such a filesystem at all. As it is, free space
> checking on xfs_check does not scale, so we still need to avoid this
> checking regardless of how we fill the filesystem.
>
> We can acheive exactly the same fill behaviour by preallocating a
> single large file in the filesystem immediately after creating it.
> This is a filesystem independent manner of filling the filesystem,
> and allows us to do large filesystem testing on more than just XFS.
>
> Further, this preallocation method effectively adds a new "very
> large file" test. It also enables us to run an unmodified xfs_repair
> or filesystem specific fsck program to check the filesystem for
> sanity, so we can now do full sanity checking of such large
> filesystems.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common.rc |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 53 insertions(+), 5 deletions(-)
>
> diff --git a/common.rc b/common.rc
> index b0af482..7bb1133 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -276,6 +276,47 @@ _scratch_mkfs_options()
>      echo $SCRATCH_OPTIONS $MKFS_OPTIONS $* $SCRATCH_DEV
>  }
>
> +
> +_setup_large_xfs_fs()
> +{
> +	fs_size=$1
> +	local tmp_dir=/tmp/
> +
> +	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
> +	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
> +	[ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
> +
> +	# calculate the size of the file we need to allocate.
> +	# Default free space in the FS is 50GB, but you can specify more via
> +	# SCRATCH_DEV_EMPTY_SPACE
> +	file_size=$(($fs_size - 50*1024*1024*1024))
> +	file_size=$(($file_size - $SCRATCH_DEV_EMPTY_SPACE))
> +
> +	# mount the filesystem, create the file, unmount it
> +	_scratch_mount 2>&1 >$tmp_dir/mnt.err
> +	local status=$?
> +	if [ $status -ne 0 ]; then
> +		echo "mount failed"
> +		cat $tmp_dir/mnt.err >&2
> +		rm -f $tmp_dir/mnt.err
> +		return $status
> +	fi
> +	rm -f $tmp_dir/mnt.err
> +
> +	xfs_io -F -f \
> +		-c "truncate $file_size" \
> +		-c "falloc -k 0 $file_size" \
> +		$SCRATCH_MNT/.use_space 2>&1 > /dev/null

Looks like patch 10 (exclude largefs fill files from dump tests) does 
not ignore this file as it should when using xfsdump -e and chattr +d. 
The following tests which are all in the auto group fail when using the 
--large-fs option:
	019 026-028 046 047 050 056 059 060 062-066

I have attached the specific error output.

Regards
--Rich




[-- Attachment #2: patch10.err --]
[-- Type: text/plain, Size: 35601 bytes --]

./check 019 026-028 046 047 050 056 059 060 062-066
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 cxfsxe4 3.6.0-rc1-0.9-default
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdc1
MOUNT_OPTIONS -- /dev/sdc1 /xfs_scratch

019	 10s
026	 21s
027	 22s
028	 32s
046	 20s
047	 31s
050	 81s
056	 20s
059	 31s
060	 31s
062      10s
063	 22s
064	 73s
065	 51s
066	 10s
Ran: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Passed all 15 tests

./check --large-fs 019 026-028 046 047 050 056 059 060 062-066
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 cxfsxe4 3.6.0-rc1-0.9-default
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdc1
MOUNT_OPTIONS -- /dev/sdc1 /xfs_scratch

019 10s ... - output mismatch (see 019.out.bad)
--- 019.out	2012-08-15 15:01:39.000000000 -0500
+++ 019.out.bad	2012-08-28 08:38:08.000000000 -0500
@@ -9,6 +9,11 @@
  Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
 Device: <DEVICE> Inode: <INODE> Links: 3 
 
+ File: "./.use_space"
+ Size: 6312890368 Filetype: Regular File
+ Mode: (0600/-rw-------) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
  File: "./bigfile"
  Size: 2097152 Filetype: Regular File
  Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
026 21s ... - output mismatch (see 026.out.bad)
--- 026.out	2012-08-15 15:01:39.000000000 -0500
+++ 026.out.bad	2012-08-28 08:38:34.000000000 -0500
@@ -9,6 +9,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_026"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -28,12 +29,13 @@
 xfsrestore: searching media for directory dump
 xfsrestore: examining media file 0
 xfsrestore: reading directories
-xfsrestore: 3 directories and 38 entries processed
+xfsrestore: 3 directories and 39 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
 xfsrestore: Restore Status: SUCCESS
 Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
 Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
 Files DUMP_DIR/small and RESTORE_DIR/DUMP_SUBDIR/small are identical
 Files DUMP_DIR/sub/a and RESTORE_DIR/DUMP_SUBDIR/sub/a are identical
027 22s ... ... - output mismatch (see 027.out.bad)
--- 027.out	2012-08-15 15:01:39.000000000 -0500
+++ 027.out.bad	2012-08-28 08:39:00.000000000 -0500
@@ -19,7 +19,7 @@
 xfsrestore: media ID: ID
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 39 entries processed
+xfsrestore: 3 directories and 40 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
028 32s ... - output mismatch (see 028.out.bad)
--- 028.out	2012-08-15 15:01:39.000000000 -0500
+++ 028.out.bad	2012-08-28 08:39:36.000000000 -0500
@@ -7,6 +7,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.0"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -27,6 +28,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.1"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -46,6 +48,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.2"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -65,6 +68,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.3"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -84,6 +88,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.4"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
046 20s ... - output mismatch (see 046.out.bad)
--- 046.out	2012-08-15 15:01:39.000000000 -0500
+++ 046.out.bad	2012-08-28 08:40:00.000000000 -0500
@@ -9,6 +9,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_046"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -28,7 +29,7 @@
 xfsrestore: searching media for directory dump
 xfsrestore: examining media file 0
 xfsrestore: reading directories
-xfsrestore: 3 directories and 10 entries processed
+xfsrestore: 3 directories and 11 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
047 31s ... - output mismatch (see 047.out.bad)
--- 047.out	2012-08-15 15:01:39.000000000 -0500
+++ 047.out.bad	2012-08-28 08:40:37.000000000 -0500
@@ -7,6 +7,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.0"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -27,6 +28,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.1"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -46,6 +48,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.2"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -65,6 +68,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.3"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
@@ -84,6 +88,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "session.4"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: creating dump session media file 0 (media 0, file 0)
050 81s ... - output mismatch (see 050.out.bad)
--- 050.out	2012-08-15 15:01:39.000000000 -0500
+++ 050.out.bad	2012-08-28 08:42:03.000000000 -0500
@@ -8,26 +8,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] =OK= 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** unmount
@@ -40,26 +40,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] =OK= 100 500 00 [7 days] 10 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** unmount
@@ -72,26 +72,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** unmount
@@ -104,26 +104,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** unmount
@@ -136,26 +136,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [7 days] 9 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] =OK= 100 500 00 [7 days] 9 4 10 00 [7 days] 0 0 0 00 [--------]
 
 *** unmount
@@ -168,26 +168,26 @@
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 
 *** report no quota settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 
 *** report initial settings
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 1 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft inode limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 0 100 500 00 [--------] 3 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the soft block limit
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 4 4 10 00 [--------] 0 0 0 00 [--------]
 
 *** push past the hard inode limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 140 100 500 00 [--------] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** push past the hard block limit (expect EDQUOT)
-[ROOT] 0 0 0 00 [--------] 3 0 0 00 [--------] 0 0 0 00 [--------]
+[ROOT] 0 0 0 00 [--------] 4 0 0 00 [--------] 0 0 0 00 [--------]
 [NAME] 540 100 500 00 [--none--] 16 4 10 00 [--none--] 0 0 0 00 [--------]
 
 *** unmount
056 20s ... - output mismatch (see 056.out.bad)
--- 056.out	2012-08-15 15:01:39.000000000 -0500
+++ 056.out.bad	2012-08-28 08:42:28.000000000 -0500
@@ -9,6 +9,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_056"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -28,7 +29,7 @@
 xfsrestore: searching media for directory dump
 xfsrestore: examining media file 0
 xfsrestore: reading directories
-xfsrestore: 7 directories and 11 entries processed
+xfsrestore: 7 directories and 12 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
059 31s ... - output mismatch (see 059.out.bad)
--- 059.out	2012-08-15 15:01:39.000000000 -0500
+++ 059.out.bad	2012-08-28 08:43:01.000000000 -0500
@@ -12,6 +12,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_059"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM
 xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM
 xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM
@@ -34,6 +35,7 @@
 Comparing listing of dump directory with restore directory
 Files TMP.dump_dir and TMP.restore_dir are identical
 Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
 Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
 Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical
 Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical
060 31s ... - output mismatch (see 060.out.bad)
--- 060.out	2012-08-15 15:01:39.000000000 -0500
+++ 060.out.bad	2012-08-28 08:43:33.000000000 -0500
@@ -12,6 +12,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_060"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM
 xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM
 xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM
@@ -42,7 +43,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 41 entries processed
+xfsrestore: 3 directories and 42 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: NOTE: restore interrupted: SECS seconds elapsed: may resume later using -R option
@@ -77,6 +78,7 @@
 Comparing listing of dump directory with restore directory
 Files TMP.dump_dir and TMP.restore_dir are identical
 Comparing dump directory with restore directory
+Only in SCRATCH_MNT: .use_space
 Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical
 Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical
 Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical
062 10s ... - output mismatch (see 062.out.bad)
--- 062.out	2012-08-15 15:01:39.000000000 -0500
+++ 062.out.bad	2012-08-28 08:43:47.000000000 -0500
@@ -1,6 +1,7 @@
 QA output created by 062
 *** create test bed
 SCRATCH_MNT
+SCRATCH_MNT/.use_space
 SCRATCH_MNT/dev
 SCRATCH_MNT/dev/b
 SCRATCH_MNT/dev/c
@@ -492,6 +493,7 @@
 
 *** extend test bed
 SCRATCH_MNT
+SCRATCH_MNT/.use_space
 SCRATCH_MNT/descend
 SCRATCH_MNT/descend/and
 SCRATCH_MNT/descend/down
063 22s ... - output mismatch (see 063.out.bad)
--- 063.out	2012-08-15 15:01:39.000000000 -0500
+++ 063.out.bad	2012-08-28 08:44:13.000000000 -0500
@@ -19,6 +19,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_063"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -38,7 +39,7 @@
 xfsrestore: searching media for directory dump
 xfsrestore: examining media file 0
 xfsrestore: reading directories
-xfsrestore: 4 directories and 21 entries processed
+xfsrestore: 4 directories and 22 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
064 73s ... - output mismatch (see 064.out.bad)
--- 064.out	2012-08-15 15:01:39.000000000 -0500
+++ 064.out.bad	2012-08-28 08:45:31.000000000 -0500
@@ -54,6 +54,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_064"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -314,7 +315,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -395,7 +396,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -428,7 +429,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -461,7 +462,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -494,7 +495,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -527,7 +528,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -560,7 +561,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -593,7 +594,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -626,7 +627,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -659,7 +660,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -693,7 +694,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -774,7 +775,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -855,7 +856,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -936,7 +937,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1017,7 +1018,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1098,7 +1099,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1179,7 +1180,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1260,7 +1261,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1341,7 +1342,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -1422,7 +1423,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 55 entries processed
+xfsrestore: 2 directories and 56 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
065 51s ... - output mismatch (see 065.out.bad)
--- 065.out	2012-08-15 15:01:39.000000000 -0500
+++ 065.out.bad	2012-08-28 08:46:26.000000000 -0500
@@ -19,6 +19,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_065"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -257,7 +258,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 6 directories and 11 entries processed
+xfsrestore: 6 directories and 12 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -290,7 +291,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 6 entries processed
+xfsrestore: 2 directories and 7 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -317,7 +318,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 4 directories and 7 entries processed
+xfsrestore: 4 directories and 8 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -346,7 +347,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 11 entries processed
+xfsrestore: 3 directories and 12 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -380,7 +381,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 7 entries processed
+xfsrestore: 3 directories and 8 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -410,7 +411,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -441,7 +442,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
 xfsrestore: directory post-processing
 xfsrestore: reading non-directory files
 xfsrestore: table of contents display complete: SECS seconds elapsed
@@ -525,7 +526,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 6 directories and 11 entries processed
+xfsrestore: 6 directories and 12 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -562,7 +563,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 6 entries processed
+xfsrestore: 2 directories and 7 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -595,7 +596,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 4 directories and 7 entries processed
+xfsrestore: 4 directories and 8 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -628,7 +629,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 11 entries processed
+xfsrestore: 3 directories and 12 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -665,7 +666,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 3 directories and 7 entries processed
+xfsrestore: 3 directories and 8 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -698,7 +699,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
@@ -729,7 +730,7 @@
 xfsrestore: using online session inventory
 xfsrestore: searching media for directory dump
 xfsrestore: reading directories
-xfsrestore: 2 directories and 5 entries processed
+xfsrestore: 2 directories and 6 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
066 10s ... - output mismatch (see 066.out.bad)
--- 066.out	2012-08-15 15:01:39.000000000 -0500
+++ 066.out.bad	2012-08-28 08:46:41.000000000 -0500
@@ -12,6 +12,7 @@
 xfsdump: session id: ID
 xfsdump: session label: "stress_066"
 xfsdump: ino map <PHASES>
+xfsdump: NOTE: pruned 1 files: skip attribute set
 xfsdump: ino map construction complete
 xfsdump: estimated dump size: NUM bytes
 xfsdump: /var/xfsdump/inventory created
@@ -31,7 +32,7 @@
 xfsrestore: searching media for directory dump
 xfsrestore: examining media file 0
 xfsrestore: reading directories
-xfsrestore: 2 directories and 2 entries processed
+xfsrestore: 2 directories and 3 entries processed
 xfsrestore: directory post-processing
 xfsrestore: restoring non-directory files
 xfsrestore: restore complete: SECS seconds elapsed
Ran: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Failures: 019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
Failed 15 of 15 tests


[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 04/10] xfstests: use preallocation for ag-wiper

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 05/10] xfstests: use command line option for setting extra space

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Allow the extra free space to leave in large scratch filesystems to
be specified by a command line option rather than just via an
environment variable.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/common b/common
index da86cd9..78ac654 100644
--- a/common
+++ b/common
@@ -247,6 +247,11 @@ s/ .*//p
 	    usage
 	    ;;
 
+	--extra-space=*)
+	    export SCRATCH_DEV_EMPTY_SPACE=${r#*=}
+	    xpand=false
+	    ;;
+
 	*)
 	    start=$r
 	    end=$r
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 05/10] xfstests: use command line option for setting extra space

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Allow the extra free space to leave in large scratch filesystems to
> be specified by a command line option rather than just via an
> environment variable.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common |    5 +++++
>   1 file changed, 5 insertions(+)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 05/10] xfstests: use command line option for setting extra space

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Now that large filesystem testing does not play free space games to
fill the space without IO, we can enable xfs_repair when running in
this mode. xfs_repair has had it's scalability problems solved, too,
so this is a safe thing to do.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common.rc |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/common.rc b/common.rc
index 7bb1133..a9252e0 100644
--- a/common.rc
+++ b/common.rc
@@ -1238,12 +1238,12 @@ _check_xfs_filesystem()
     extra_mount_options=""
     device=$1
     if [ "$2" != "none" ]; then
-       	extra_log_options="-l$2"
+	extra_log_options="-l$2"
         extra_mount_options="-ologdev=$2"
     fi
 
     if [ "$3" != "none" ]; then
-       	extra_rt_options="-r$3"
+	extra_rt_options="-r$3"
         extra_mount_options=$extra_mount_options" -ortdev=$3"
     fi
     extra_mount_options=$extra_mount_options" $MOUNT_OPTIONS"
@@ -1288,8 +1288,7 @@ _check_xfs_filesystem()
 
         ok=0
     fi
-    # repair doesn't scale massively at this stage, optionally skip it for now
-    [ "$LARGE_SCRATCH_DEV" = yes ] || \
+
     $XFS_REPAIR_PROG -n $extra_log_options $extra_rt_options $device >$tmp.repair 2>&1
     if [ $? -ne 0 ]
     then
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Now that large filesystem testing does not play free space games to
> fill the space without IO, we can enable xfs_repair when running in
> this mode. xfs_repair has had it's scalability problems solved, too,
> so this is a safe thing to do.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common.rc |    7 +++----
>   1 file changed, 3 insertions(+), 4 deletions(-)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 06/10] xfstest: enable xfs_repair for large filesystem testing

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 07/10] xfstests: always us test option when checking large scratch device

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Some tests call _check_scratch_device directly and when using a
large filesystem this needs to run with a -t option to avoid
consuming large amounts of memory. Make this happen in all cases
that the scratch device is checked.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 017       |    7 ++-----
 common.rc |    2 ++
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/017 b/017
index 9ca0e72..0a3ede3 100755
--- a/017
+++ b/017
@@ -48,9 +48,6 @@ _supported_os Linux
 
 _require_scratch
 
-checkopts=""
-[ "$USE_BIG_LOOPFS" = yes ] && checkopts=-t
-
 echo "*** init FS"
 
 rm -f $seq.full
@@ -81,8 +78,8 @@ do
         echo ""                             >>$seq.full
         echo "*** XFS_CHECK ***"            >>$seq.full
         echo ""                             >>$seq.full
-        _scratch_xfs_check $checkopts       >>$seq.full 2>&1 \
-            || _fail "xfs_check $checkopts failed"
+        _scratch_xfs_check                  >>$seq.full 2>&1 \
+            || _fail "xfs_check failed"
         _scratch_mount -o remount,rw \
             || _fail "remount rw failed"
 done
diff --git a/common.rc b/common.rc
index a9252e0..df47e2a 100644
--- a/common.rc
+++ b/common.rc
@@ -470,6 +470,8 @@ _scratch_xfs_check()
     SCRATCH_OPTIONS=""
     [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
         SCRATCH_OPTIONS="-l $SCRATCH_LOGDEV"
+    [ "$LARGE_SCRATCH_DEV" = yes ] && \
+        SCRATCH_OPTIONS=$SCRATCH_OPTIONS" -t"
     $XFS_CHECK_PROG $SCRATCH_OPTIONS $* $SCRATCH_DEV
 }
 
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device

[Paulo Alcantara]

Hi Dave,

From: Dave Chinner <david@fromorbit.com>
Date: Thu, 26 Jul 2012 18:39:46 +1000

> From: Dave Chinner <dchinner@redhat.com>
> 
> Some tests call _check_scratch_device directly and when using a
> large filesystem this needs to run with a -t option to avoid
> consuming large amounts of memory. Make this happen in all cases
> that the scratch device is checked.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  017       |    7 ++-----
>  common.rc |    2 ++
>  2 files changed, 4 insertions(+), 5 deletions(-)

I guess you meant "xfstests: always use test option..." instead in commit
message.


		Paulo

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Some tests call _check_scratch_device directly and when using a
> large filesystem this needs to run with a -t option to avoid
> consuming large amounts of memory. Make this happen in all cases
> that the scratch device is checked.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   017       |    7 ++-----
>   common.rc |    2 ++
>   2 files changed, 4 insertions(+), 5 deletions(-)
>


Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 07/10] xfstests: always us test option when checking large scratch device

[Christoph Hellwig]

Looks good modulo the subject typo pointed our earlier.

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 08/10] xfstests: enable large fs testing on ext4

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Now that setting up large filesystem testing on sparse loopback
devices uses a generic method for filling the filesystem, extent
support to ext4 filesystems.

ext4 is slightly more complex to fill as it does not support files
larger than 16TB. Hence a slightly more complex method of using
multiple smaller files to fill the space is necessary.

WARNING: be prepeared for ext4 to take forever to prepare large
filesystems as allocation of large amounts of space (especially as
it approaches ENOSPC) is can take minutes rather than milliseconds

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common.rc |   90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)

diff --git a/common.rc b/common.rc
index df47e2a..1c3c605 100644
--- a/common.rc
+++ b/common.rc
@@ -366,6 +366,93 @@ _scratch_mkfs_xfs()
     return $mkfs_status
 }
 
+_setup_large_ext4_fs()
+{
+	fs_size=$1
+	local tmp_dir=/tmp/
+
+	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0
+	[ $SCRATCH_DEV_EMPTY_SPACE -ge $fs_size ] && return 0
+
+	# Default free space in the FS is 50GB, but you can specify more via
+	# SCRATCH_DEV_EMPTY_SPACE
+	space_to_consume=$(($fs_size - 50*1024*1024*1024 - $SCRATCH_DEV_EMPTY_SPACE))
+
+	# mount the filesystem and create 16TB - 4KB files until we consume
+	# all the necessary space.
+	_scratch_mount 2>&1 >$tmp_dir/mnt.err
+	local status=$?
+	if [ $status -ne 0 ]; then
+		echo "mount failed"
+		cat $tmp_dir/mnt.err >&2
+		rm -f $tmp_dir/mnt.err
+		return $status
+	fi
+	rm -f $tmp_dir/mnt.err
+
+	file_size=$((16*1024*1024*1024*1024 - 4096))
+	nfiles=0
+	while [ $space_to_consume -gt $file_size ]; do
+
+		xfs_io -F -f \
+			-c "truncate $file_size" \
+			-c "falloc -k 0 $file_size" \
+			$SCRATCH_MNT/.use_space.$nfiles 2>&1
+		status=$?
+		if [ $status -ne 0 ]; then
+			break;
+		fi
+
+		space_to_consume=$(( $space_to_consume - $file_size ))
+		nfiles=$(($nfiles + 1))
+	done
+
+	# consume the remaining space.
+	if [ $space_to_consume -gt 0 ]; then
+		xfs_io -F -f \
+			-c "truncate $space_to_consume" \
+			-c "falloc -k 0 $space_to_consume" \
+			$SCRATCH_MNT/.use_space.$nfiles 2>&1
+		status=$?
+	fi
+
+	umount $SCRATCH_MNT
+	if [ $status -ne 0 ]; then
+		echo "large file prealloc failed"
+		cat $tmp_dir/mnt.err >&2
+		return $status
+	fi
+	return 0
+}
+_scratch_mkfs_ext4()
+{
+	local tmp_dir=/tmp/
+
+	/sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV \
+			2>$tmp_dir.mkfserr 1>$tmp_dir.mkfsstd
+	local mkfs_status=$?
+
+	if [ $mkfs_status -eq 0 -a "$LARGE_SCRATCH_DEV" = yes ]; then
+		# manually parse the mkfs output to get the fs size in bytes
+		fs_size=`cat $tmp_dir.mkfsstd | awk ' \
+			/^Block size/ { split($2, a, "="); bs = a[2] ; } \
+			/ inodes, / { blks = $3 } \
+			/reserved for the super user/ { resv = $1 } \
+			END { fssize = bs * blks - resv; print fssize }'`
+
+		_setup_large_ext4_fs $fs_size
+		mkfs_status=$?
+	fi
+
+	# output stored mkfs output
+	cat $tmp_dir.mkfserr >&2
+	cat $tmp_dir.mkfsstd
+	rm -f $tmp_dir.mkfserr $tmp_dir.mkfsstd
+
+	return $mkfs_status
+}
+
 _scratch_mkfs()
 {
     case $FSTYP in
@@ -381,6 +468,9 @@ _scratch_mkfs()
     btrfs)
         $MKFS_BTRFS_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null
 	;;
+    ext4)
+	_scratch_mkfs_ext4 $*
+	;;
     *)
 	/sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV
 	;;
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 08/10] xfstests: enable large fs testing on ext4

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 08/10] xfstests: enable large fs testing on ext4

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Now that setting up large filesystem testing on sparse loopback
> devices uses a generic method for filling the filesystem, extent
> support to ext4 filesystems.
>
> ext4 is slightly more complex to fill as it does not support files
> larger than 16TB. Hence a slightly more complex method of using
> multiple smaller files to fill the space is necessary.
>
> WARNING: be prepeared for ext4 to take forever to prepare large
> filesystems as allocation of large amounts of space (especially as
> it approaches ENOSPC) is can take minutes rather than milliseconds
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common.rc |   90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 90 insertions(+)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 09/10] xfstests: disable tests that typically fail on large filesystems

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

Test 017 typically fails due to the OOM killer kicking in and
killing everything but xfs_db so panics the machine.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 017 |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/017 b/017
index 0a3ede3..b3a7f9c 100755
--- a/017
+++ b/017
@@ -48,6 +48,10 @@ _supported_os Linux
 
 _require_scratch
 
+# xfs_db will OOM kill the machine if you don't have huge amounts of RAM, so
+# don't run this on large filesystems.
+_require_no_large_scratch_dev
+
 echo "*** init FS"
 
 rm -f $seq.full
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> Test 017 typically fails due to the OOM killer kicking in and
> killing everything but xfs_db so panics the machine.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   017 |    4 ++++
>   1 file changed, 4 insertions(+)
>

Looks good.

Reviewed-by: Rich Johnston <rjohnston@sgi.com>

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 09/10] xfstests: disable tests that typically fail on large filesystems

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 10/10] xfstests: exclude largefs fill files from dump tests

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

The xfsdump/restore tests will see the space filling files and may
try to backthem up and restore them, consuming huge amounts of time
to do (espcially when diffing the results). Exclude the space
filling files by setting the no dump attributes on them and ensure
that xfsdump runs with the -e flag to exclude such files.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 common.config |    3 ++-
 common.dump   |    2 +-
 common.rc     |    1 +
 3 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/common.config b/common.config
index 7bed1c5..bb9af93 100644
--- a/common.config
+++ b/common.config
@@ -146,7 +146,8 @@ export XFS_GROWFS_PROG=`set_prog_path xfs_growfs`
 export XFS_IO_PROG="`set_prog_path xfs_io`"
 export XFS_PARALLEL_REPAIR_PROG="`set_prog_path xfs_prepair`"
 export XFS_PARALLEL_REPAIR64_PROG="`set_prog_path xfs_prepair64`"
-export XFSDUMP_PROG="`set_prog_path xfsdump`"
+export __XFSDUMP_PROG="`set_prog_path xfsdump`"
+export XFSDUMP_PROG="$__XFSDUMP_PROG -e"
 export XFSRESTORE_PROG="`set_prog_path xfsrestore`"
 export XFSINVUTIL_PROG="`set_prog_path xfsinvutil`"
 export DBENCH_PROG="`set_prog_path dbench`"
diff --git a/common.dump b/common.dump
index e89dedc..a33dc81 100644
--- a/common.dump
+++ b/common.dump
@@ -829,7 +829,7 @@ _dump_filter_main()
 {
   _filter_devchar |\
   sed \
-      -e "s#$XFSDUMP_PROG#xfsdump#"			\
+      -e "s#$__XFSDUMP_PROG#xfsdump#"			\
       -e "s#$XFSRESTORE_PROG#xfsrestore#"		\
       -e "s#$XFSINVUTIL_PROG#xfsinvutil#"		\
       -e "s/`hostname`/HOSTNAME/"			\
diff --git a/common.rc b/common.rc
index 1c3c605..1b23b66 100644
--- a/common.rc
+++ b/common.rc
@@ -306,6 +306,7 @@ _setup_large_xfs_fs()
 	xfs_io -F -f \
 		-c "truncate $file_size" \
 		-c "falloc -k 0 $file_size" \
+		-c "chattr +d" \
 		$SCRATCH_MNT/.use_space 2>&1 > /dev/null
 	status=$?
 	umount $SCRATCH_MNT
-- 
1.7.10

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 10/10] xfstests: exclude largefs fill files from dump tests

[Rich Johnston]

On 07/26/2012 03:39 AM, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
>
> The xfsdump/restore tests will see the space filling files and may
> try to backthem up and restore them, consuming huge amounts of time
> to do (espcially when diffing the results). Exclude the space
> filling files by setting the no dump attributes on them and ensure
> that xfsdump runs with the -e flag to exclude such files.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>   common.config |    3 ++-
>   common.dump   |    2 +-
>   common.rc     |    1 +
>   3 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/common.config b/common.config
> index 7bed1c5..bb9af93 100644
> --- a/common.config
> +++ b/common.config
> @@ -146,7 +146,8 @@ export XFS_GROWFS_PROG=`set_prog_path xfs_growfs`
>   export XFS_IO_PROG="`set_prog_path xfs_io`"
>   export XFS_PARALLEL_REPAIR_PROG="`set_prog_path xfs_prepair`"
>   export XFS_PARALLEL_REPAIR64_PROG="`set_prog_path xfs_prepair64`"
> -export XFSDUMP_PROG="`set_prog_path xfsdump`"
> +export __XFSDUMP_PROG="`set_prog_path xfsdump`"
> +export XFSDUMP_PROG="$__XFSDUMP_PROG -e"
>   export XFSRESTORE_PROG="`set_prog_path xfsrestore`"
>   export XFSINVUTIL_PROG="`set_prog_path xfsinvutil`"
>   export DBENCH_PROG="`set_prog_path dbench`"
> diff --git a/common.dump b/common.dump
> index e89dedc..a33dc81 100644
> --- a/common.dump
> +++ b/common.dump
> @@ -829,7 +829,7 @@ _dump_filter_main()
>   {
>     _filter_devchar |\
>     sed \
> -      -e "s#$XFSDUMP_PROG#xfsdump#"			\
> +      -e "s#$__XFSDUMP_PROG#xfsdump#"			\
>         -e "s#$XFSRESTORE_PROG#xfsrestore#"		\
>         -e "s#$XFSINVUTIL_PROG#xfsinvutil#"		\
>         -e "s/`hostname`/HOSTNAME/"			\
> diff --git a/common.rc b/common.rc
> index 1c3c605..1b23b66 100644
> --- a/common.rc
> +++ b/common.rc
> @@ -306,6 +306,7 @@ _setup_large_xfs_fs()
>   	xfs_io -F -f \
>   		-c "truncate $file_size" \
>   		-c "falloc -k 0 $file_size" \
> +		-c "chattr +d" \
>   		$SCRATCH_MNT/.use_space 2>&1 > /dev/null
>   	status=$?
>   	umount $SCRATCH_MNT
>

Ditto my comments from patch 4, this does not work as expected when 
using the --large-fs option for the following tests:

	019 026-028 046 047 050 056 059 060 062-066

Regards
--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 10/10] xfstests: exclude largefs fill files from dump tests

[Christoph Hellwig]

On Thu, Jul 26, 2012 at 06:39:49PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> The xfsdump/restore tests will see the space filling files and may
> try to backthem up and restore them, consuming huge amounts of time
> to do (espcially when diffing the results). Exclude the space
> filling files by setting the no dump attributes on them and ensure
> that xfsdump runs with the -e flag to exclude such files.

Looks reasonable,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 0/10] xfstests: rework large filesystem testing

[Dave Chinner]

ping?

On Thu, Jul 26, 2012 at 06:39:39PM +1000, Dave Chinner wrote:
> This is a resend of a series that has been sitting inmy local
> workarea for months. It reworks the large filesystem testing
> infrastructure for XFS to allow it to use xfs_repair by switching
> the filesystem filling to using preallocation rather than xfs_db
> hacks.
> 
> It also introduces support for ext4 testing, also via preallocation
> to fill the filesystem. btrfs isn't yet supported because I haven't
> worked out how to reliably get the size of the filesystem and
> calculate how much space is actually available in it.
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 
> -- 
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
> 
> 

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output

[Ben Myers]

Hey Rich,

On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> Patch "rework large filesystem testing" introduces a new option --large-fs
> which creates a new file $SCRATCH_MNT/.use_space.  If this 10 part patchset is
> applied, the following tests will fail:
> 	019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
> 
> This patch accounts for the following new output when testing xfs filesystems with
> the --large-fs option by creating new output file to compare against
> ($seq.largefs.out):
> 
> 1. The following four lines appear in test 019. 
> 	 File: "./.use_space"
> 	 Size: 6312890368 Filetype: Regular File
> 	Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> 	Device: <DEVICE> Inode: <INODE> Links: 1
> 
> 2. When the nodump attribute is set, the xfsdump -e option will cause the
>    following additional lines to appear.
> 	xfsdump: NOTE: pruned 1 files: skip attribute set
> 	Only in SCRATCH_MNT: .use_space
> 	SCRATCH_MNT/.use_space
> 
> 3. Number of files off by one.
> 	xfsrestore: # directories and (off by 1) entries processed
> 	[ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
> 
> This patch also modifies check and common.quota to use the new output file
> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> or  $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
> 
> Signed-off-by: Rich Johnston <rjohnston@sgi.com>

This looks pretty good to me.  .use_space changed some golden output and you've
fixed that up by adding additional output files for when this --large-fs option
is used.  Dave will probably want to take a look too.

Reviewed-by: Ben Myers <bpm@sgi.com>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output

[Dave Chinner]

On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> Patch "rework large filesystem testing" introduces a new option --large-fs
> which creates a new file $SCRATCH_MNT/.use_space.  If this 10 part patchset is
> applied, the following tests will fail:
> 	019 026 027 028 046 047 050 056 059 060 062 063 064 065 066

That's a lot more tests than I see failing. 

> This patch accounts for the following new output when testing xfs filesystems with
> the --large-fs option by creating new output file to compare against
> ($seq.largefs.out):

Creating new output files is the absolute last resort.  Indeed, what
happens when you get different output for tests that already select
an output file based on, say, platform or some other criteria? We
get a combinatorial explosion of golden output files, and that is
simply not manageable.

The usual thing to do is update the necessary filters or change the
way the tests run to avoid trivial output file differences e.g. use
a subdir rather than SCRATCH_MNT directly. Or, for example the
filters that munge different standard error messages from different
platforms to be the same...

> 1. The following four lines appear in test 019. 
> 	 File: "./.use_space"
> 	 Size: 6312890368 Filetype: Regular File
> 	Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> 	Device: <DEVICE> Inode: <INODE> Links: 1

This test doesn't really need to be run for large filesystems -
running it on large filesystems doesn't improve the coverage of or
our confidence in the code it is testing, so I'd just add a
_require_no_large_scratch_dev to it.

> 2. When the nodump attribute is set, the xfsdump -e option will cause the
>    following additional lines to appear.
> 	xfsdump: NOTE: pruned 1 files: skip attribute set
> 	Only in SCRATCH_MNT: .use_space
> 	SCRATCH_MNT/.use_space

Ok, those are the errors I haven't seen - not sure why. I'll have to
look into that.

However, this is definitely a case of updating the dump output
filter to remove these messages from the output stream.  The
alternative is to change the common dump code to use a subdirectory
rather than the root directory so it doesn't see these files at all.

> 3. Number of files off by one.
> 	xfsrestore: # directories and (off by 1) entries processed

That would be fixed by using a subdir for the dump tests. I don't
recommend that the number should be filtered, as having dump report
the correct number of files scanned is important.

> 	[ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]

Perhaps the usre/group of the use_space file needs to be changed so
it doesn't impact on the test results. Alternatively, a filter could
be written/modified to fix the number appropriately.

> This patch also modifies check and common.quota to use the new output file
> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> or  $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
> 
> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
> 
> ---
>  019.largefs.out |    5 +++
>  026.largefs.out |    4 ++-
>  027.largefs.out |    2 -
>  028.largefs.out |    5 +++
>  046.largefs.out |    3 +-
>  047.largefs.out |    5 +++
>  050.largefs.out |   72 ++++++++++++++++++++++++++++----------------------------
>  056.largefs.out |    3 +-
>  059.largefs.out |    2 +
>  060.largefs.out |    4 ++-
>  062.largefs.out |    2 +
>  063.largefs.out |    3 +-
>  064.largefs.out |   41 ++++++++++++++++---------------
>  065.largefs.out |   29 +++++++++++-----------
>  066.largefs.out |    3 +-
>  check           |   12 +++++++--
>  common.quota    |   20 ++++++++++-----
>  17 files changed, 128 insertions(+), 87 deletions(-)

FWIW, this patch is supposed to add these *.largefs.out files, right? The
patch, however:

> Index: b/019.largefs.out
> ===================================================================
> --- a/019.largefs.out
> +++ b/019.largefs.out
> @@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
>   Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
>  Device: <DEVICE> Inode: <INODE> Links: 3 
>  
> + File: "./.use_space"
> + Size: 6312890368 Filetype: Regular File
> + Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1
> +
>   File: "./bigfile"
>   Size: 2097152 Filetype: Regular File
>   Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)

... assumes they already exist...

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output

[Rich Johnston]

Thanks for the comments.

On 09/05/2012 05:26 PM, Dave Chinner wrote:
> On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
>> Patch "rework large filesystem testing" introduces a new option --large-fs
>> which creates a new file $SCRATCH_MNT/.use_space.  If this 10 part patchset is
>> applied, the following tests will fail:
>> 	019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
>
> That's a lot more tests than I see failing.

It is very repeatable for me.

>
>> This patch accounts for the following new output when testing xfs filesystems with
>> the --large-fs option by creating new output file to compare against
>> ($seq.largefs.out):
>
> Creating new output files is the absolute last resort.  Indeed, what
> happens when you get different output for tests that already select
> an output file based on, say, platform or some other criteria? We
> get a combinatorial explosion of golden output files, and that is
> simply not manageable.
>
> The usual thing to do is update the necessary filters or change the
> way the tests run to avoid trivial output file differences e.g. use
> a subdir rather than SCRATCH_MNT directly. Or, for example the
> filters that munge different standard error messages from different
> platforms to be the same...
>

OK good to know.

>> 1. The following four lines appear in test 019.
>> 	 File: "./.use_space"
>> 	 Size: 6312890368 Filetype: Regular File
>> 	Mode: (0600/-rw-------) Uid: (0) Gid: (0)
>> 	Device: <DEVICE> Inode: <INODE> Links: 1
>
> This test doesn't really need to be run for large filesystems -
> running it on large filesystems doesn't improve the coverage of or
> our confidence in the code it is testing, so I'd just add a
> _require_no_large_scratch_dev to it.
>

Works for me.

>> 2. When the nodump attribute is set, the xfsdump -e option will cause the
>>     following additional lines to appear.
>> 	xfsdump: NOTE: pruned 1 files: skip attribute set
>> 	Only in SCRATCH_MNT: .use_space
>> 	SCRATCH_MNT/.use_space
>
> Ok, those are the errors I haven't seen - not sure why. I'll have to
> look into that.
>
> However, this is definitely a case of updating the dump output
> filter to remove these messages from the output stream.  The
> alternative is to change the common dump code to use a subdirectory
> rather than the root directory so it doesn't see these files at all.
>

Good suggestion

>> 3. Number of files off by one.
>> 	xfsrestore: # directories and (off by 1) entries processed
>
> That would be fixed by using a subdir for the dump tests. I don't
> recommend that the number should be filtered, as having dump report
> the correct number of files scanned is important.

I agree.

>
>> 	[ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
>
> Perhaps the usre/group of the use_space file needs to be changed so
> it doesn't impact on the test results. Alternatively, a filter could
> be written/modified to fix the number appropriately.

Sounds reasonable.

>
>> This patch also modifies check and common.quota to use the new output file
>> $seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
>> or  $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
>>
>> Signed-off-by: Rich Johnston <rjohnston@sgi.com>
>>
>> ---
>>   019.largefs.out |    5 +++
>>   026.largefs.out |    4 ++-
>>   027.largefs.out |    2 -
>>   028.largefs.out |    5 +++
>>   046.largefs.out |    3 +-
>>   047.largefs.out |    5 +++
>>   050.largefs.out |   72 ++++++++++++++++++++++++++++----------------------------
>>   056.largefs.out |    3 +-
>>   059.largefs.out |    2 +
>>   060.largefs.out |    4 ++-
>>   062.largefs.out |    2 +
>>   063.largefs.out |    3 +-
>>   064.largefs.out |   41 ++++++++++++++++---------------
>>   065.largefs.out |   29 +++++++++++-----------
>>   066.largefs.out |    3 +-
>>   check           |   12 +++++++--
>>   common.quota    |   20 ++++++++++-----
>>   17 files changed, 128 insertions(+), 87 deletions(-)
>
> FWIW, this patch is supposed to add these *.largefs.out files, right? The
> patch, however:
>
>> Index: b/019.largefs.out
>> ===================================================================
>> --- a/019.largefs.out
>> +++ b/019.largefs.out
>> @@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
>>    Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
>>   Device: <DEVICE> Inode: <INODE> Links: 3
>>
>> + File: "./.use_space"
>> + Size: 6312890368 Filetype: Regular File
>> + Mode: (0600/-rw-------) Uid: (0) Gid: (0)
>> +Device: <DEVICE> Inode: <INODE> Links: 1
>> +
>>    File: "./bigfile"
>>    Size: 2097152 Filetype: Regular File
>>    Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
>
> ... assumes they already exist...
>

Yup my bad, I only posted the differences from the original *.out files.

May I make the suggested changes, or as this is your patchset do you 
want to make them?

Regards,
--Rich

> Cheers,
>
> Dave.
>


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 11/10] xfstests: rework large filesystem testing - add golden output

[Dave Chinner]

On Thu, Sep 06, 2012 at 07:57:18AM -0500, Rich Johnston wrote:
> Thanks for the comments.
> 
> On 09/05/2012 05:26 PM, Dave Chinner wrote:
> >On Fri, Aug 31, 2012 at 02:43:27PM -0500, rjohnston@sgi.com wrote:
> >>Patch "rework large filesystem testing" introduces a new option --large-fs
> >>which creates a new file $SCRATCH_MNT/.use_space.  If this 10 part patchset is
> >>applied, the following tests will fail:
> >>	019 026 027 028 046 047 050 056 059 060 062 063 064 065 066
> >
> >That's a lot more tests than I see failing.
> 
> It is very repeatable for me.
> 
> >
> >>This patch accounts for the following new output when testing xfs filesystems with
> >>the --large-fs option by creating new output file to compare against
> >>($seq.largefs.out):
> >
> >Creating new output files is the absolute last resort.  Indeed, what
> >happens when you get different output for tests that already select
> >an output file based on, say, platform or some other criteria? We
> >get a combinatorial explosion of golden output files, and that is
> >simply not manageable.
> >
> >The usual thing to do is update the necessary filters or change the
> >way the tests run to avoid trivial output file differences e.g. use
> >a subdir rather than SCRATCH_MNT directly. Or, for example the
> >filters that munge different standard error messages from different
> >platforms to be the same...
> >
> 
> OK good to know.
> 
> >>1. The following four lines appear in test 019.
> >>	 File: "./.use_space"
> >>	 Size: 6312890368 Filetype: Regular File
> >>	Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> >>	Device: <DEVICE> Inode: <INODE> Links: 1
> >
> >This test doesn't really need to be run for large filesystems -
> >running it on large filesystems doesn't improve the coverage of or
> >our confidence in the code it is testing, so I'd just add a
> >_require_no_large_scratch_dev to it.
> >
> 
> Works for me.
> 
> >>2. When the nodump attribute is set, the xfsdump -e option will cause the
> >>    following additional lines to appear.
> >>	xfsdump: NOTE: pruned 1 files: skip attribute set
> >>	Only in SCRATCH_MNT: .use_space
> >>	SCRATCH_MNT/.use_space
> >
> >Ok, those are the errors I haven't seen - not sure why. I'll have to
> >look into that.
> >
> >However, this is definitely a case of updating the dump output
> >filter to remove these messages from the output stream.  The
> >alternative is to change the common dump code to use a subdirectory
> >rather than the root directory so it doesn't see these files at all.
> >
> 
> Good suggestion
> 
> >>3. Number of files off by one.
> >>	xfsrestore: # directories and (off by 1) entries processed
> >
> >That would be fixed by using a subdir for the dump tests. I don't
> >recommend that the number should be filtered, as having dump report
> >the correct number of files scanned is important.
> 
> I agree.
> 
> >
> >>	[ROOT] 0 0 0 00 [--------] (off by 1) 0 0 00 [--------] 0 0 0 00 [--------]
> >
> >Perhaps the usre/group of the use_space file needs to be changed so
> >it doesn't impact on the test results. Alternatively, a filter could
> >be written/modified to fix the number appropriately.
> 
> Sounds reasonable.
> 
> >
> >>This patch also modifies check and common.quota to use the new output file
> >>$seq.largefs.out when the --large-fs option is used (LARGE_SCRATCH_DEV = yes)
> >>or  $seq.out when the --large-fs option is NOT used (LARGE_SCRATCH_DEV != yes).
> >>
> >>Signed-off-by: Rich Johnston <rjohnston@sgi.com>
> >>
> >>---
> >>  019.largefs.out |    5 +++
> >>  026.largefs.out |    4 ++-
> >>  027.largefs.out |    2 -
> >>  028.largefs.out |    5 +++
> >>  046.largefs.out |    3 +-
> >>  047.largefs.out |    5 +++
> >>  050.largefs.out |   72 ++++++++++++++++++++++++++++----------------------------
> >>  056.largefs.out |    3 +-
> >>  059.largefs.out |    2 +
> >>  060.largefs.out |    4 ++-
> >>  062.largefs.out |    2 +
> >>  063.largefs.out |    3 +-
> >>  064.largefs.out |   41 ++++++++++++++++---------------
> >>  065.largefs.out |   29 +++++++++++-----------
> >>  066.largefs.out |    3 +-
> >>  check           |   12 +++++++--
> >>  common.quota    |   20 ++++++++++-----
> >>  17 files changed, 128 insertions(+), 87 deletions(-)
> >
> >FWIW, this patch is supposed to add these *.largefs.out files, right? The
> >patch, however:
> >
> >>Index: b/019.largefs.out
> >>===================================================================
> >>--- a/019.largefs.out
> >>+++ b/019.largefs.out
> >>@@ -9,6 +9,11 @@ Wrote 2048.00Kb (value 0x2c)
> >>   Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> >>  Device: <DEVICE> Inode: <INODE> Links: 3
> >>
> >>+ File: "./.use_space"
> >>+ Size: 6312890368 Filetype: Regular File
> >>+ Mode: (0600/-rw-------) Uid: (0) Gid: (0)
> >>+Device: <DEVICE> Inode: <INODE> Links: 1
> >>+
> >>   File: "./bigfile"
> >>   Size: 2097152 Filetype: Regular File
> >>   Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> >
> >... assumes they already exist...
> >
> 
> Yup my bad, I only posted the differences from the original *.out files.
> 
> May I make the suggested changes, or as this is your patchset do you
> want to make them?

I've got a couple of other fixes to make to the series, so I'll work
these in. Thanks for reporting them and prototyping fixes for them.
:)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs