xfstests 258: Test xfs fs creation with fs size close to 4 TB

xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Boris Ranto]

mkfs.xfs failed to create xfs filesystems with 4 TB minus few bytes due
to round up error in mkfs.xfs code.

This test case is a regression test for the fs creation problem.

I've tested the test case with mkfs.xfs patch (in the form posted by
Eric Sandeen) and the test passed (and therefore the patch fixed the
issue for me).

Signed-off-by: Boris Ranto <branto@redhat.com>


diff --git a/258 b/258
new file mode 100755
index 0000000..b881626
--- /dev/null
+++ b/258
@@ -0,0 +1,60 @@
+#! /bin/bash
+# FS QA Test No. 258
+#
+# Test fs creation on 4 TB minus few bytes partition
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Red Hat.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=branto@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f "$testfile"
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+
+testfile=$TEST_DIR/258.image
+
+fourtb=4398046511104
+for sub in 1 512 1024 1025 2048 4096;
+do
+	echo "Trying to make $fourtb - $sub B long xfs fs image"
+	rm -f "$testfile"
+	dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$(($fourtb - $sub))
>/dev/null 2>&1 || echo "dd failed"
+	lofile=$(losetup -f)
+	losetup $lofile "$testfile"
+	"$MKFS_XFS_PROG" -b size=512 $lofile >/dev/null || echo "mkfs failed!"
+	losetup -d $lofile
+done
+
+status=0 ; exit
diff --git a/258.out b/258.out
new file mode 100644
index 0000000..22e9652
--- /dev/null
+++ b/258.out
@@ -0,0 +1,7 @@
+QA output created by 258
+Trying to make 4398046511104 - 1 B long xfs fs image
+Trying to make 4398046511104 - 512 B long xfs fs image
+Trying to make 4398046511104 - 1024 B long xfs fs image
+Trying to make 4398046511104 - 1025 B long xfs fs image
+Trying to make 4398046511104 - 2048 B long xfs fs image
+Trying to make 4398046511104 - 4096 B long xfs fs image
diff --git a/group b/group
index 84c45da..739f806 100644
--- a/group
+++ b/group
@@ -371,3 +371,4 @@ deprecated
 255 auto quick prealloc
 256 auto quick
 257 auto quick
+258 auto quick

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Alex Elder]

On Wed, 2011-09-21 at 15:52 +0200, Boris Ranto wrote:
> mkfs.xfs failed to create xfs filesystems with 4 TB minus few bytes due
> to round up error in mkfs.xfs code.
> 
> This test case is a regression test for the fs creation problem.
> 
> I've tested the test case with mkfs.xfs patch (in the form posted by
> Eric Sandeen) and the test passed (and therefore the patch fixed the
> issue for me).
> 
> Signed-off-by: Boris Ranto <branto@redhat.com>

This looks OK, but I'm a little concerned about the
shell's ability to handle > 32-bit values in its
arithmetic expressions (within $((...))).

Using ${fourtb} works for me, but I just don't know
whether it is written somewhere that bash always
supports 64-bit (or even arbitrary) precision values.

Do you know?

Same general concern goes for dd, but I am more inclined
to think it can handle large numbers.

Otherwise this looks good to me (though I haven't yet
tried it out).

Reviewed-by: Alex Elder <aelder@sgi.com>

. . .

> +fourtb=4398046511104
> +for sub in 1 512 1024 1025 2048 4096;
> +do
> +	echo "Trying to make $fourtb - $sub B long xfs fs image"
> +	rm -f "$testfile"
> +	dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$(($fourtb - $sub))
> >/dev/null 2>&1 || echo "dd failed"
> +	lofile=$(losetup -f)
> +	losetup $lofile "$testfile"
> +	"$MKFS_XFS_PROG" -b size=512 $lofile >/dev/null || echo "mkfs failed!"
> +	losetup -d $lofile
> +done
> +

. . .


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Arkadiusz Miśkiewicz]

On Wednesday 21 of September 2011, Alex Elder wrote:
> On Wed, 2011-09-21 at 15:52 +0200, Boris Ranto wrote:
> > mkfs.xfs failed to create xfs filesystems with 4 TB minus few bytes due
> > to round up error in mkfs.xfs code.
> > 
> > This test case is a regression test for the fs creation problem.
> > 
> > I've tested the test case with mkfs.xfs patch (in the form posted by
> > Eric Sandeen) and the test passed (and therefore the patch fixed the
> > issue for me).
> > 
> > Signed-off-by: Boris Ranto <branto@redhat.com>
> 
> This looks OK, but I'm a little concerned about the
> shell's ability to handle > 32-bit values in its
> arithmetic expressions (within $((...))).
> 
> Using ${fourtb} works for me, but I just don't know
> whether it is written somewhere that bash always
> supports 64-bit (or even arbitrary) precision values.

I can say only that bashizm sux. posix shell unfortunately makes problems 
here.

mksh (https://www.mirbsd.org/mksh.htm) supports 32bit arithmetic only (even on 
64bit arch).

single unix spec says "signed long", so you cannot rely on posix shell for > 
32bit values.

-- 
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Boris Ranto]

On Wed, 2011-09-21 at 11:54 -0500, Alex Elder wrote:
> On Wed, 2011-09-21 at 15:52 +0200, Boris Ranto wrote:
> > mkfs.xfs failed to create xfs filesystems with 4 TB minus few bytes due
> > to round up error in mkfs.xfs code.
> > 
> > This test case is a regression test for the fs creation problem.
> > 
> > I've tested the test case with mkfs.xfs patch (in the form posted by
> > Eric Sandeen) and the test passed (and therefore the patch fixed the
> > issue for me).
> > 
> > Signed-off-by: Boris Ranto <branto@redhat.com>
> 
> This looks OK, but I'm a little concerned about the
> shell's ability to handle > 32-bit values in its
> arithmetic expressions (within $((...))).
> 
> Using ${fourtb} works for me, but I just don't know
> whether it is written somewhere that bash always
> supports 64-bit (or even arbitrary) precision values.
> 
> Do you know?
> 
> Same general concern goes for dd, but I am more inclined
> to think it can handle large numbers.
> 
> Otherwise this looks good to me (though I haven't yet
> tried it out).
> 
> Reviewed-by: Alex Elder <aelder@sgi.com>
> 
> . . .
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs

I'm not sure whether bash guarantees at least 64-bit precision values in
its arithmetic operations.
Therefore I suppose the values can be computed in advance in this case
and the arithmetic operation can be simply left out:

Signed-off-by: Boris Ranto <branto@redhat.com>

diff --git a/258 b/258
new file mode 100755
index 0000000..75f96d2
--- /dev/null
+++ b/258
@@ -0,0 +1,59 @@
+#! /bin/bash
+# FS QA Test No. 258
+#
+# Test fs creation on 4 TB minus few bytes partition
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2011 Red Hat.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+# creator
+owner=branto@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1	# failure is the default!
+
+_cleanup()
+{
+    rm -f "$testfile"
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+
+testfile=$TEST_DIR/258.image
+
+for ddseek in 4398046511103 4398046510592 4398046510080 4398046510079
4398046509056 4398046507008;
+do
+	echo "Trying to make $ddseek B long xfs fs image"
+	rm -f "$testfile"
+	dd if=/dev/zero "of=$testfile" bs=1 count=0 seek=$ddseek >/dev/null
2>&1 || echo "dd failed"
+	lofile=$(losetup -f)
+	losetup $lofile "$testfile"
+	"$MKFS_XFS_PROG" -b size=512 $lofile >/dev/null || echo "mkfs failed!"
+	losetup -d $lofile
+done
+
+status=0 ; exit
diff --git a/258.out b/258.out
new file mode 100644
index 0000000..6f27c16
--- /dev/null
+++ b/258.out
@@ -0,0 +1,7 @@
+QA output created by 258
+Trying to make 4398046511103 B long xfs fs image
+Trying to make 4398046510592 B long xfs fs image
+Trying to make 4398046510080 B long xfs fs image
+Trying to make 4398046510079 B long xfs fs image
+Trying to make 4398046509056 B long xfs fs image
+Trying to make 4398046507008 B long xfs fs image
diff --git a/group b/group
index 84c45da..739f806 100644
--- a/group
+++ b/group
@@ -371,3 +371,4 @@ deprecated
 255 auto quick prealloc
 256 auto quick
 257 auto quick
+258 auto quick

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Alex Elder]

On Thu, 2011-09-22 at 10:30 +0200, Boris Ranto wrote:
> On Wed, 2011-09-21 at 11:54 -0500, Alex Elder wrote:
> > On Wed, 2011-09-21 at 15:52 +0200, Boris Ranto wrote:
> > > mkfs.xfs failed to create xfs filesystems with 4 TB minus few bytes due
> > > to round up error in mkfs.xfs code.
> > > 
> > > This test case is a regression test for the fs creation problem.
> > > 
> > > I've tested the test case with mkfs.xfs patch (in the form posted by
> > > Eric Sandeen) and the test passed (and therefore the patch fixed the
> > > issue for me).
> > > 
> > > Signed-off-by: Boris Ranto <branto@redhat.com>
> > 
> > This looks OK, but I'm a little concerned about the
> > shell's ability to handle > 32-bit values in its
> > arithmetic expressions (within $((...))).
> > 
> > Using ${fourtb} works for me, but I just don't know
> > whether it is written somewhere that bash always
> > supports 64-bit (or even arbitrary) precision values.
> > 
> > Do you know?
> > 
> > Same general concern goes for dd, but I am more inclined
> > to think it can handle large numbers.
> > 
> > Otherwise this looks good to me (though I haven't yet
> > tried it out).
> > 
> > Reviewed-by: Alex Elder <aelder@sgi.com>
> > 
> > . . .
> > 
> > _______________________________________________
> > xfs mailing list
> > xfs@oss.sgi.com
> > http://oss.sgi.com/mailman/listinfo/xfs
> 
> I'm not sure whether bash guarantees at least 64-bit precision values in
> its arithmetic operations.
> Therefore I suppose the values can be computed in advance in this case
> and the arithmetic operation can be simply left out:

This at least makes it so we only have to worry about
one program (dd) handling >32-bit values correctly.
Based on that alone I guess I prefer it.  However
there should be a comment that explains where the
numbers come from, i.e.:

# 4398046511103 = 2^42 - 1
# 4398046510592 = 2^42 - 512
# 4398046510080 = 2^42 - 1024
# 4398046510079 = 2^42 - 1025
# 4398046509056 = 2^42 - 2048
# 4398046507008 = 2^42 - 4096

If dd doesn't support numbers that big we aren't working
in an environment suitable for running xfstests.  So
at least from my perspective, this is good enough.

Reviewed-by: Alex Elder <aelder@sgi.com>


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Christoph Hellwig]

This one actually fails for me in a 32-bit userspace VM:

--- 259.out	2011-09-26 09:14:39.000000000 +0000
+++ 259.out.bad	2011-09-26 11:22:44.000000000 +0000
@@ -1,7 +1,157 @@
 QA output created by 259
 Trying to make 4398046511103 B long xfs fs image
+dd failed
+can't get size of data subvolume
+Usage: mkfs.xfs
+/* blocksize */		[-b log=n|size=num]

...

When removing the dd output redirection thus turn into a:

+dd: failed to truncate to 4398046509056 bytes in output file `/259.image': File too large

As it turns out testfile is assigned to early in the test, and we thus
create the images on the root filesystem.

---
From: Christoph Hellwig <hch@lst.de>
Subject: [PATCH] xfstests: fix test 259

Move the assignment of testfile after the sourcing of the common.* files to
make sure TEST_DIR is already defined - without this we end up creating
the file on the root filesystem, which may not support large enough files.

Also add a sync after the mkfs.xfs invocation, as losetup -d might fail
the loop device deletion with -EBUSY otherwise.

Signed-off-by: Christoph Hellwig <hch@lst.de>

Index: xfstests-dev/259
===================================================================
--- xfstests-dev.orig/259	2011-09-26 11:29:17.000000000 +0000
+++ xfstests-dev/259	2011-09-26 11:32:54.000000000 +0000
@@ -29,8 +29,6 @@ echo "QA output created by $seq"
 
 status=1	# failure is the default!
 
-testfile=$TEST_DIR/259.image
-
 _cleanup()
 {
     rm -f "$testfile"
@@ -45,6 +43,8 @@ trap "_cleanup ; exit \$status" 0 1 2 3
 _supported_fs xfs
 _supported_os Linux
 
+testfile=$TEST_DIR/259.image
+
 # Test various sizes slightly less than 4 TB
 # 4398046511103 = 2^42 - 1
 # 4398046510592 = 2^42 - 512
@@ -63,6 +63,7 @@ do
 	lofile=$(losetup -f)
 	losetup $lofile "$testfile"
 	"$MKFS_XFS_PROG" -b size=512 $lofile >/dev/null || echo "mkfs failed!"
+	sync
 	losetup -d $lofile
 done
 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Alex Elder]

On Mon, 2011-09-26 at 07:44 -0400, Christoph Hellwig wrote:
> This one actually fails for me in a 32-bit userspace VM:

This is my fault, and you busted me doing something I
should not have.

It doesn't matter in this case, but there is a window in
which "testfile" is undefined but gets referenced in
_cleanup() (which could get called due to interrupt).
I "trivially" moved the definition to fix that oversight
and as a result broke the test.

I won't do that any more without taking proper credit
when I commit.

Sorry Boris (and all).

Your fix looks fine, Christoph.

Reviewed-by: Alex Elder <aelder@sgi.com>


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfstests 258: Test xfs fs creation with fs size close to 4 TB

[Eric Sandeen]

On 9/26/11 6:44 AM, Christoph Hellwig wrote:
> This one actually fails for me in a 32-bit userspace VM:
> 
> --- 259.out	2011-09-26 09:14:39.000000000 +0000
> +++ 259.out.bad	2011-09-26 11:22:44.000000000 +0000
> @@ -1,7 +1,157 @@
>  QA output created by 259
>  Trying to make 4398046511103 B long xfs fs image
> +dd failed
> +can't get size of data subvolume
> +Usage: mkfs.xfs
> +/* blocksize */		[-b log=n|size=num]
> 
> ...
> 
> When removing the dd output redirection thus turn into a:
> 
> +dd: failed to truncate to 4398046509056 bytes in output file `/259.image': File too large
> 
> As it turns out testfile is assigned to early in the test, and we thus
> create the images on the root filesystem.

Oh hell.

Thanks for fixing :(

> ---
> From: Christoph Hellwig <hch@lst.de>
> Subject: [PATCH] xfstests: fix test 259
> 
> Move the assignment of testfile after the sourcing of the common.* files to
> make sure TEST_DIR is already defined - without this we end up creating
> the file on the root filesystem, which may not support large enough files.
> 
> Also add a sync after the mkfs.xfs invocation, as losetup -d might fail
> the loop device deletion with -EBUSY otherwise.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> 
> Index: xfstests-dev/259
> ===================================================================
> --- xfstests-dev.orig/259	2011-09-26 11:29:17.000000000 +0000
> +++ xfstests-dev/259	2011-09-26 11:32:54.000000000 +0000
> @@ -29,8 +29,6 @@ echo "QA output created by $seq"
>  
>  status=1	# failure is the default!
>  
> -testfile=$TEST_DIR/259.image
> -
>  _cleanup()
>  {
>      rm -f "$testfile"
> @@ -45,6 +43,8 @@ trap "_cleanup ; exit \$status" 0 1 2 3
>  _supported_fs xfs
>  _supported_os Linux
>  
> +testfile=$TEST_DIR/259.image
> +
>  # Test various sizes slightly less than 4 TB
>  # 4398046511103 = 2^42 - 1
>  # 4398046510592 = 2^42 - 512
> @@ -63,6 +63,7 @@ do
>  	lofile=$(losetup -f)
>  	losetup $lofile "$testfile"
>  	"$MKFS_XFS_PROG" -b size=512 $lofile >/dev/null || echo "mkfs failed!"
> +	sync
>  	losetup -d $lofile
>  done
>  

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs