[PATCH] xfstests: several 274 fixups

[PATCH] xfstests: several 274 fixups

[Eric Sandeen]

This changes quite a few things about 274 to make it more robust
and useful.

* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion
  (this breaks e2fsck when lost+found/ goes missing)
* Don't cd into $SCRATCH_MNT
* Try harder to completely fill the fs
* Use a larger preallocated space, and write into all of it (hopefully
  avoid just accidentally succeeding by writing into fs reserved
  space that may be there)
* Save more output in $seq.full instead of /dev/null

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

diff --git a/274 b/274
index b658004..acf4543 100755
--- a/274
+++ b/274
@@ -1,7 +1,9 @@
 #! /bin/bash
 # FS QA Test No. 274
 #
-# preallocation test
+# preallocation test:
+# Preallocate space to a file, and fill the rest of the fs to 100%.
+# Then test a write into that preallocated space, which should succeed.
 #
 #-----------------------------------------------------------------------
 # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
@@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }
 
@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"
@@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
+# Create a 4k file
+dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "create file err"
+	echo "create file error"
 	status=1
 	exit
 fi
 
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
+# Allocate 4M past EOF on that file
+xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fallocate file err"
+	echo "fallocate file error"
 	status=1
 	exit
 fi
 
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
+# Now attempt a write into all of the preallocated space 
+dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
 if [ $? -ne 0 ]
 then
-	echo "fill prealloc range err"
+	echo "fill prealloc range error"
 	status=1
 	exit
 fi

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Dave Chinner]

On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
> This changes quite a few things about 274 to make it more robust
> and useful.
> 
> * More comments
> * Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
> * use _require_xfs_io_falloc to be sure system & fs support preallocation
> * Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
> * Do not remove all of the files in $SCRATCH_MNT/ on completion
>   (this breaks e2fsck when lost+found/ goes missing)

FWIW, can't e2fsck be fixed to handle this case?

......
>  _cleanup()
>  {
>  	cd /
> -	rm -f $SCRATCH_MNT/* $tmp.*
> +	rm -f $tmp.*
>  	_scratch_unmount
>  }
>  
> @@ -46,6 +48,7 @@ _cleanup()
>  _supported_fs generic
>  _supported_os IRIX Linux
>  _require_scratch
> +_require_xfs_io_falloc
>  
>  echo "------------------------------"
>  echo "preallocation test"
> @@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
>  _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
>  _scratch_mount
>  
> -rm -rf $SCRATCH_MNT/*
> -cd $SCRATCH_MNT
> -dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
> +# Create a 4k file
> +dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "create file err"
> +	echo "create file error"
>  	status=1
>  	exit
>  fi
>  
> -fallocate -n -o 4K -l 1M test >/dev/null 2>&1
> +# Allocate 4M past EOF on that file
> +xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "fallocate file err"
> +	echo "fallocate file error"
>  	status=1
>  	exit
>  fi

That whole create and falloc step can be done with one command:

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test

The output of xfs_io will tell us what failed if it does.

Also, failure handling is as simple as appending:

	|| _fail "failure string"

to the command. It handles setting status appropriately, tees the
failure string to $seq.full, and tells the user to go look at
$seq.full for why the test failed. hence that 16 lines of script can
be simply replaced with these 2 lines:

xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
	>>$seq.full 2>&1 || _fail "failed to create test file"

> -dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
> -dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
> +# Fill the rest of the fs completely
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
>  sync
> +# Last effort, use O_SYNC
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
> +# Save space usage info
> +echo "Post-fill space:" >> $seq.full
> +df $SCRATCH_MNT >>$seq.full 2>&1
>  
> -dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
> +# Now attempt a write into all of the preallocated space 
> +dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "fill prealloc range err"
> +	echo "fill prealloc range error"
>  	status=1
>  	exit
>  fi

I'd still like to see this write attempt to trigger nasty behaviours
like needing to allocate a metadata block for the extent list. I
suggested randholes, but perhaps this would be easier:

for i in `seq 1 2 1023`; do
	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"
done

which will write every second block and so only convert every second
block from unwritten to written and hence blow out the size of the
extent list and require extent map block allocation and potentially
trigger ENOSPC that way....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Eric Sandeen]

On 1/31/12 4:41 PM, Dave Chinner wrote:
> On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
>> This changes quite a few things about 274 to make it more robust
>> and useful.
>>
>> * More comments
>> * Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
>> * use _require_xfs_io_falloc to be sure system & fs support preallocation
>> * Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
>> * Do not remove all of the files in $SCRATCH_MNT/ on completion
>>   (this breaks e2fsck when lost+found/ goes missing)
> 
> FWIW, can't e2fsck be fixed to handle this case?

perhaps some day.

but it'd return "modified" if it adds lost+found ...

I suppose it could just be left missing unless otherwise needed.

> ......
>>  _cleanup()
>>  {
>>  	cd /
>> -	rm -f $SCRATCH_MNT/* $tmp.*
>> +	rm -f $tmp.*
>>  	_scratch_unmount
>>  }
>>  
>> @@ -46,6 +48,7 @@ _cleanup()
>>  _supported_fs generic
>>  _supported_os IRIX Linux
>>  _require_scratch
>> +_require_xfs_io_falloc
>>  
>>  echo "------------------------------"
>>  echo "preallocation test"
>> @@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
>>  _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
>>  _scratch_mount
>>  
>> -rm -rf $SCRATCH_MNT/*
>> -cd $SCRATCH_MNT
>> -dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
>> +# Create a 4k file
>> +dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "create file err"
>> +	echo "create file error"
>>  	status=1
>>  	exit
>>  fi
>>  
>> -fallocate -n -o 4K -l 1M test >/dev/null 2>&1
>> +# Allocate 4M past EOF on that file
>> +xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "fallocate file err"
>> +	echo "fallocate file error"
>>  	status=1
>>  	exit
>>  fi
> 
> That whole create and falloc step can be done with one command:
> 
> xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test
> 
> The output of xfs_io will tell us what failed if it does.
> 
> Also, failure handling is as simple as appending:
> 
> 	|| _fail "failure string"
> 
> to the command. It handles setting status appropriately, tees the
> failure string to $seq.full, and tells the user to go look at
> $seq.full for why the test failed. hence that 16 lines of script can
> be simply replaced with these 2 lines:
> 
> xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
> 	>>$seq.full 2>&1 || _fail "failed to create test file"

Ok, that's fine.

>> -dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
>> -dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
>> +# Fill the rest of the fs completely
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
>>  sync
>> +# Last effort, use O_SYNC
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
>> +# Save space usage info
>> +echo "Post-fill space:" >> $seq.full
>> +df $SCRATCH_MNT >>$seq.full 2>&1
>>  
>> -dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
>> +# Now attempt a write into all of the preallocated space 
>> +dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "fill prealloc range err"
>> +	echo "fill prealloc range error"
>>  	status=1
>>  	exit
>>  fi
> 
> I'd still like to see this write attempt to trigger nasty behaviours
> like needing to allocate a metadata block for the extent list. I
> suggested randholes, but perhaps this would be easier:
> 
> for i in `seq 1 2 1023`; do
> 	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
> 		>>$seq.full 2>&1 || _fail "failed to write test file"
> done
> 
> which will write every second block and so only convert every second
> block from unwritten to written and hence blow out the size of the
> extent list and require extent map block allocation and potentially
> trigger ENOSPC that way....

but will also only write half the actual preallocated space.  I guess it
depends on what we are trying to test - reservations for metadata or proper
accounting of prealloc'd space itself...?

(and really, what is the proper amount of preallocated metadata space; if
I subbed 2G for 4M, would anything survive the test above?)

-Eric

> Cheers,
> 
> Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Dave Chinner]

On Tue, Jan 31, 2012 at 08:27:56PM -0600, Eric Sandeen wrote:
> On 1/31/12 4:41 PM, Dave Chinner wrote:
> > On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
> >> -dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
> >> -dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
> >> +# Fill the rest of the fs completely
> >> +dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
> >> +dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
> >>  sync
> >> +# Last effort, use O_SYNC
> >> +dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
> >> +# Save space usage info
> >> +echo "Post-fill space:" >> $seq.full
> >> +df $SCRATCH_MNT >>$seq.full 2>&1
> >>  
> >> -dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
> >> +# Now attempt a write into all of the preallocated space 
> >> +dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
> >>  if [ $? -ne 0 ]
> >>  then
> >> -	echo "fill prealloc range err"
> >> +	echo "fill prealloc range error"
> >>  	status=1
> >>  	exit
> >>  fi
> > 
> > I'd still like to see this write attempt to trigger nasty behaviours
> > like needing to allocate a metadata block for the extent list. I
> > suggested randholes, but perhaps this would be easier:
> > 
> > for i in `seq 1 2 1023`; do
> > 	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
> > 		>>$seq.full 2>&1 || _fail "failed to write test file"
> > done
> > 
> > which will write every second block and so only convert every second
> > block from unwritten to written and hence blow out the size of the
> > extent list and require extent map block allocation and potentially
> > trigger ENOSPC that way....
> 
> but will also only write half the actual preallocated space.  I guess it
> depends on what we are trying to test - reservations for metadata or proper
> accounting of prealloc'd space itself...?

So then go back and fill in the unwritten blocks. i.e:

for i in `seq 1 2 1023`; do
	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to write test file"
done
sync
for i in `seq 2 2 1023`; do
	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
		>>$seq.full 2>&1 || _fail "failed to fill test file"
done
sync

> (and really, what is the proper amount of preallocated metadata space; if
> I subbed 2G for 4M, would anything survive the test above?)

Given that the test is only using a 1GB filesystem.... ;)

But even if it was 2GB, then I'm pretty sure that XFS would handle
it fine. Take a long to run, though, with all those individual 4k
IOs. My point was simply to determine if the filesystem will be able
to hold up the guarantees posix_fallocate() tries to give
applications, and so enough extents simple to force a single
metadata block allocation is all that is required. Having a 1000
extent state changes shoul dbe sufficient to cause that....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Eric Sandeen]

On 1/31/12 4:41 PM, Dave Chinner wrote:
> On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
>> This changes quite a few things about 274 to make it more robust
>> and useful.
>>
>> * More comments
>> * Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
>> * use _require_xfs_io_falloc to be sure system & fs support preallocation
>> * Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
>> * Do not remove all of the files in $SCRATCH_MNT/ on completion
>>   (this breaks e2fsck when lost+found/ goes missing)
> 
> FWIW, can't e2fsck be fixed to handle this case?
> 
> ......
>>  _cleanup()
>>  {
>>  	cd /
>> -	rm -f $SCRATCH_MNT/* $tmp.*
>> +	rm -f $tmp.*
>>  	_scratch_unmount
>>  }
>>  
>> @@ -46,6 +48,7 @@ _cleanup()
>>  _supported_fs generic
>>  _supported_os IRIX Linux
>>  _require_scratch
>> +_require_xfs_io_falloc
>>  
>>  echo "------------------------------"
>>  echo "preallocation test"
>> @@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
>>  _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
>>  _scratch_mount
>>  
>> -rm -rf $SCRATCH_MNT/*
>> -cd $SCRATCH_MNT
>> -dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
>> +# Create a 4k file
>> +dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "create file err"
>> +	echo "create file error"
>>  	status=1
>>  	exit
>>  fi
>>  
>> -fallocate -n -o 4K -l 1M test >/dev/null 2>&1
>> +# Allocate 4M past EOF on that file
>> +xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "fallocate file err"
>> +	echo "fallocate file error"
>>  	status=1
>>  	exit
>>  fi
> 
> That whole create and falloc step can be done with one command:
> 
> xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test
> 
> The output of xfs_io will tell us what failed if it does.

too bad that doesn't quite fail properly ;)

[root@inode xfstests-dev]# xfs_io -F -f -c "write 0 4k" -c "falloc -k 4k 4m" foobar
command "write" not found
[root@inode xfstests-dev]# echo $?
0

:(

I'll fix that up too I guess.

--Eric



> Also, failure handling is as simple as appending:
> 
> 	|| _fail "failure string"
> 
> to the command. It handles setting status appropriately, tees the
> failure string to $seq.full, and tells the user to go look at
> $seq.full for why the test failed. hence that 16 lines of script can
> be simply replaced with these 2 lines:
> 
> xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
> 	>>$seq.full 2>&1 || _fail "failed to create test file"
> 
>> -dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
>> -dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
>> +# Fill the rest of the fs completely
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
>>  sync
>> +# Last effort, use O_SYNC
>> +dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
>> +# Save space usage info
>> +echo "Post-fill space:" >> $seq.full
>> +df $SCRATCH_MNT >>$seq.full 2>&1
>>  
>> -dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
>> +# Now attempt a write into all of the preallocated space 
>> +dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
>>  if [ $? -ne 0 ]
>>  then
>> -	echo "fill prealloc range err"
>> +	echo "fill prealloc range error"
>>  	status=1
>>  	exit
>>  fi
> 
> I'd still like to see this write attempt to trigger nasty behaviours
> like needing to allocate a metadata block for the extent list. I
> suggested randholes, but perhaps this would be easier:
> 
> for i in `seq 1 2 1023`; do
> 	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
> 		>>$seq.full 2>&1 || _fail "failed to write test file"
> done
> 
> which will write every second block and so only convert every second
> block from unwritten to written and hence blow out the size of the
> extent list and require extent map block allocation and potentially
> trigger ENOSPC that way....
> 
> Cheers,
> 
> Dave.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Dave Chinner]

On Thu, Mar 08, 2012 at 03:55:45PM -0600, Eric Sandeen wrote:
> On 1/31/12 4:41 PM, Dave Chinner wrote:
> > On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
> > 
> > That whole create and falloc step can be done with one command:
> > 
> > xfs_io -F -c "write 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test
> > 
> > The output of xfs_io will tell us what failed if it does.
> 
> too bad that doesn't quite fail properly ;)

Just a small typo - the command is pwrite ;)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH V2] xfstests: several 274 fixups

[Eric Sandeen]

This changes quite a few things about 274 to make it more robust
and useful.

* More comments
* Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
* use _require_xfs_io_falloc to be sure system & fs support preallocation
* Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
* Do not remove all of the files in $SCRATCH_MNT/ on completion
  (this breaks e2fsck when lost+found/ goes missing)
* Don't cd into $SCRATCH_MNT
* Try harder to completely fill the fs
* Use a larger preallocated space, and write into all of it (hopefully
  avoid just accidentally succeeding by writing into fs reserved
  space that may be there)
* Save more output in $seq.full instead of /dev/null
* Fill preallocated space diabolically

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

V2: Several fixes suggested by Dave.
Also fills prealloc space diabolically.  This fails on ext4.

diff --git a/274 b/274
index b658004..bba5eb4 100755
--- a/274
+++ b/274
@@ -1,7 +1,9 @@
 #! /bin/bash
 # FS QA Test No. 274
 #
-# preallocation test
+# preallocation test:
+# Preallocate space to a file, and fill the rest of the fs to 100%.
+# Then test a write into that preallocated space, which should succeed.
 #
 #-----------------------------------------------------------------------
 # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
@@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	rm -f $SCRATCH_MNT/* $tmp.*
+	rm -f $tmp.*
 	_scratch_unmount
 }
 
@@ -46,6 +48,7 @@ _cleanup()
 _supported_fs generic
 _supported_os IRIX Linux
 _require_scratch
+_require_xfs_io_falloc
 
 echo "------------------------------"
 echo "preallocation test"
@@ -57,35 +60,42 @@ umount $SCRATCH_DEV 2>/dev/null
 _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
 _scratch_mount
 
-rm -rf $SCRATCH_MNT/*
-cd $SCRATCH_MNT
-dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "create file err"
-	status=1
-	exit
-fi
+# Create a 4k file and Allocate 4M past EOF on that file
+xfs_io -F -f -c "pwrite 0 4k" -c "falloc -k 4k 4m" $SCRATCH_MNT/test \
+	>>$seq.full 2>&1 || _fail "failed to create test file"
 
-fallocate -n -o 4K -l 1M test >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "fallocate file err"
-	status=1
-	exit
-fi
-
-dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
-dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
+# Fill the rest of the fs completely
+# Note, this will show ENOSPC errors in $seq.full, that's ok.
+echo "Fill fs with 1M IOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
+echo "Fill fs with 4K IOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
 sync
+# Last effort, use O_SYNC
+echo "Fill fs with 4K DIOs; EIO expected" >> $seq.full
+dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
+# Save space usage info
+echo "Post-fill space:" >> $seq.full
+df $SCRATCH_MNT >>$seq.full 2>&1
 
-dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
-if [ $? -ne 0 ]
-then
-	echo "fill prealloc range err"
-	status=1
-	exit
-fi
+# Now attempt a write into all of the preallocated space -
+# in a very nasty way, badly fragmenting it and then filling it in.
+echo "Fill in prealloc space; fragment at offsets:" >> $seq.full
+for i in `seq 1 2 1023`; do
+	echo -n "$i " >> $seq.full
+	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
+		>>$seq.full 2>/dev/null || _fail "failed to write to test file"
+done
+sync
+echo >> $seq.full
+echo "Fill in prealloc space; fill holes at offsets:" >> $seq.full
+for i in `seq 2 2 1023`; do
+	echo -n "$i " >> $seq.full
+	dd if=/dev/zero of=$SCRATCH_MNT/test seek=$i bs=4K count=1 conv=notrunc \
+		>>$seq.full 2>/dev/null || _fail "failed to fill test file"
+done
+sync
+echo >> $seq.full
 
 echo "done"
 exit


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH V2] xfstests: several 274 fixups

[Mark Tinguely]

On 03/08/12 16:23, Eric Sandeen wrote:
> This changes quite a few things about 274 to make it more robust
> and useful.
>
> * More comments
> * Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
> * use _require_xfs_io_falloc to be sure system&  fs support preallocation
> * Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
> * Do not remove all of the files in $SCRATCH_MNT/ on completion
>    (this breaks e2fsck when lost+found/ goes missing)
> * Don't cd into $SCRATCH_MNT
> * Try harder to completely fill the fs
> * Use a larger preallocated space, and write into all of it (hopefully
>    avoid just accidentally succeeding by writing into fs reserved
>    space that may be there)
> * Save more output in $seq.full instead of /dev/null
> * Fill preallocated space diabolically
>
> Signed-off-by: Eric Sandeen<sandeen@redhat.com>
> ---

Looks very nice.

Reviewed-by: Mark Tinguely <tinguely@sgi.com>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] xfstests: several 274 fixups

[Christoph Hellwig]

Do you plan to resend this one with the fixups pointed out by Dave?

On Mon, Jan 30, 2012 at 04:27:51PM -0600, Eric Sandeen wrote:
> This changes quite a few things about 274 to make it more robust
> and useful.
> 
> * More comments
> * Use xfs_io for falloc (not all systems have /usr/bin/fallocate)
> * use _require_xfs_io_falloc to be sure system & fs support preallocation
> * Do not remove all of the files in $SCRATCH_MNT/ post-mkfs
> * Do not remove all of the files in $SCRATCH_MNT/ on completion
>   (this breaks e2fsck when lost+found/ goes missing)
> * Don't cd into $SCRATCH_MNT
> * Try harder to completely fill the fs
> * Use a larger preallocated space, and write into all of it (hopefully
>   avoid just accidentally succeeding by writing into fs reserved
>   space that may be there)
> * Save more output in $seq.full instead of /dev/null
> 
> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> ---
> 
> diff --git a/274 b/274
> index b658004..acf4543 100755
> --- a/274
> +++ b/274
> @@ -1,7 +1,9 @@
>  #! /bin/bash
>  # FS QA Test No. 274
>  #
> -# preallocation test
> +# preallocation test:
> +# Preallocate space to a file, and fill the rest of the fs to 100%.
> +# Then test a write into that preallocated space, which should succeed.
>  #
>  #-----------------------------------------------------------------------
>  # Copyright (c) 2011-2012 Fujitsu, Inc.  All Rights Reserved.
> @@ -35,7 +37,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
>  _cleanup()
>  {
>  	cd /
> -	rm -f $SCRATCH_MNT/* $tmp.*
> +	rm -f $tmp.*
>  	_scratch_unmount
>  }
>  
> @@ -46,6 +48,7 @@ _cleanup()
>  _supported_fs generic
>  _supported_os IRIX Linux
>  _require_scratch
> +_require_xfs_io_falloc
>  
>  echo "------------------------------"
>  echo "preallocation test"
> @@ -57,32 +60,39 @@ umount $SCRATCH_DEV 2>/dev/null
>  _scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1
>  _scratch_mount
>  
> -rm -rf $SCRATCH_MNT/*
> -cd $SCRATCH_MNT
> -dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1
> +# Create a 4k file
> +dd if=/dev/zero of=$SCRATCH_MNT/test bs=4K count=1 >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "create file err"
> +	echo "create file error"
>  	status=1
>  	exit
>  fi
>  
> -fallocate -n -o 4K -l 1M test >/dev/null 2>&1
> +# Allocate 4M past EOF on that file
> +xfs_io -F -c "falloc -k 4k 4m" $SCRATCH_MNT/test >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "fallocate file err"
> +	echo "fallocate file error"
>  	status=1
>  	exit
>  fi
>  
> -dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1
> -dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1
> +# Fill the rest of the fs completely
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp1 bs=1M >>$seq.full 2>&1
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp2 bs=4K >>$seq.full 2>&1
>  sync
> +# Last effort, use O_SYNC
> +dd if=/dev/zero of=$SCRATCH_MNT/tmp3 bs=4K oflag=sync >>$seq.full 2>&1
> +# Save space usage info
> +echo "Post-fill space:" >> $seq.full
> +df $SCRATCH_MNT >>$seq.full 2>&1
>  
> -dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1
> +# Now attempt a write into all of the preallocated space 
> +dd if=/dev/zero of=$SCRATCH_MNT/test seek=1 bs=4K count=1024 conv=notrunc >>$seq.full 2>&1
>  if [ $? -ne 0 ]
>  then
> -	echo "fill prealloc range err"
> +	echo "fill prealloc range error"
>  	status=1
>  	exit
>  fi
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
---end quoted text---

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs