[PATCH 1/3 v2] 285: Fix test for ext4 in some configurations

[PATCH 1/3 v2] 285: Fix test for ext4 in some configurations

[Jan Kara]

In some configurations (e.g. 1 KB block size), ext4 can decide it is
better to zero out several blocks rather than splitting unwritten
extent. This changes results SEEK_HOLE / SEEK_DATA returns and thus the
test fails. Fix the problem by disabling the feature for this test.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 tests/generic/285 | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tests/generic/285 b/tests/generic/285
index b700a15..8078b1c 100755
--- a/tests/generic/285
+++ b/tests/generic/285
@@ -46,6 +46,12 @@ BASE_TEST_FILE=$TEST_DIR/seek_sanity_testfile
 
 [ -x $here/src/seek_sanity_test ] || _notrun "seek_sanitfy_tester not built"
 
+# Disable extent zeroing for ext4 as that change where holes are created
+if [ "$FSTYP" = "ext4" ]; then
+	DEV=`basename $TEST_DEV`
+	echo 0 >/sys/fs/ext4/$DEV/extent_max_zeroout_kb
+fi
+
 _cleanup()
 {
 	eval "rm -f $BASE_TEST_FILE.*"
-- 
1.8.1.4

[PATCH 2/3 v2] 285: Fix indentation of do_pwrite

[Jan Kara]

Signed-off-by: Jan Kara <jack@suse.cz>
---
 src/seek_sanity_test.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
index eec6903..748eec2 100644
--- a/src/seek_sanity_test.c
+++ b/src/seek_sanity_test.c
@@ -120,19 +120,19 @@ static int do_sync_dirty_pages(int fd, off64_t offset, off64_t nbytes)
 
 static ssize_t do_pwrite(int fd, const void *buf, size_t count, off_t offset)
 {
-       ssize_t ret, written = 0;
-
-       while (count > written) {
-               ret = pwrite(fd, buf + written, count - written, offset + written);
-               if (ret < 0) {
-                       fprintf(stderr, "  ERROR %d: Failed to write %ld "
-                               "bytes\n", errno, (long)count);
-                       return ret;
-               }
-               written += ret;
-       }
-
-       return 0;
+	ssize_t ret, written = 0;
+
+	while (count > written) {
+		ret = pwrite(fd, buf + written, count - written, offset + written);
+		if (ret < 0) {
+			fprintf(stderr, "  ERROR %d: Failed to write %ld "
+				"bytes\n", errno, (long)count);
+			return ret;
+		}
+		written += ret;
+	}
+
+	return 0;
 }
 
 #define do_close(x)	do { if ((x) > -1) close(x); } while(0);
-- 
1.8.1.4

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/3 v2] 285: Fix indentation of do_pwrite

[Eric Sandeen]

On 5/30/13 4:21 PM, Jan Kara wrote:
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
>  src/seek_sanity_test.c | 26 +++++++++++++-------------
>  1 file changed, 13 insertions(+), 13 deletions(-)

mmm, whitespace.

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

> diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> index eec6903..748eec2 100644
> --- a/src/seek_sanity_test.c
> +++ b/src/seek_sanity_test.c
> @@ -120,19 +120,19 @@ static int do_sync_dirty_pages(int fd, off64_t offset, off64_t nbytes)
>  
>  static ssize_t do_pwrite(int fd, const void *buf, size_t count, off_t offset)
>  {
> -       ssize_t ret, written = 0;
> -
> -       while (count > written) {
> -               ret = pwrite(fd, buf + written, count - written, offset + written);
> -               if (ret < 0) {
> -                       fprintf(stderr, "  ERROR %d: Failed to write %ld "
> -                               "bytes\n", errno, (long)count);
> -                       return ret;
> -               }
> -               written += ret;
> -       }
> -
> -       return 0;
> +	ssize_t ret, written = 0;
> +
> +	while (count > written) {
> +		ret = pwrite(fd, buf + written, count - written, offset + written);
> +		if (ret < 0) {
> +			fprintf(stderr, "  ERROR %d: Failed to write %ld "
> +				"bytes\n", errno, (long)count);
> +			return ret;
> +		}
> +		written += ret;
> +	}
> +
> +	return 0;
>  }
>  
>  #define do_close(x)	do { if ((x) > -1) close(x); } while(0);
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/3 v2] 285: Fix indentation of do_pwrite

[Rich Johnston]

On 05/30/2013 04:21 PM, Jan Kara wrote:
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
>   src/seek_sanity_test.c | 26 +++++++++++++-------------
>   1 file changed, 13 insertions(+), 13 deletions(-)
>
Thanks for the patch, it has been committed:

commit 1515c26fd44ee2f84ce43ebcd9168130801dd9a0
Author: Jan Kara <jack@suse.cz>
Date:   Fri May 31 07:19:47 2013 -0500

     xfstests 285: Fix indentation of do_pwrite

--Rich

[PATCH 3/3 v2] 285: Test offsets over 4GB

[Jan Kara]

Test whether SEEK_HOLE and SEEK_DATA works correctly with offsets over
4GB, 8TB, and 16TB.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 src/seek_sanity_test.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
index 748eec2..cd3b1ee 100644
--- a/src/seek_sanity_test.c
+++ b/src/seek_sanity_test.c
@@ -18,6 +18,7 @@
  */
 
 #define _XOPEN_SOURCE 500
+#define _FILE_OFFSET_BITS 64
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/vfs.h>
@@ -125,6 +126,9 @@ static ssize_t do_pwrite(int fd, const void *buf, size_t count, off_t offset)
 	while (count > written) {
 		ret = pwrite(fd, buf + written, count - written, offset + written);
 		if (ret < 0) {
+			/* Don't warn about too large file. It's fs dependent. */
+			if (errno == EFBIG)
+				return ret;
 			fprintf(stderr, "  ERROR %d: Failed to write %ld "
 				"bytes\n", errno, (long)count);
 			return ret;
@@ -191,6 +195,77 @@ static int do_lseek(int testnum, int subtest, int fd, int filsz, int origin,
 	return ret;
 }
 
+static int huge_file_test(int fd, int testnum, off_t filsz)
+{
+	char *buf = NULL;
+	int bufsz = alloc_size * 16;	/* XFS seems to round allocated size */
+	off_t off = filsz - 2*bufsz;
+	int ret = -1;
+
+	buf = do_malloc(bufsz);
+	if (!buf)
+		goto out;
+	memset(buf, 'a', bufsz);
+
+	ret = do_pwrite(fd, buf, bufsz, 0);
+	if (ret)
+		goto out;
+	ret = do_pwrite(fd, buf, bufsz, off);
+	if (ret) {
+		/*
+		 * Report success. Filesystem just cannot handle so large
+		 * offsets and correctly reports it.
+		 */
+		if (errno == EFBIG) {
+			fprintf(stdout, "Test skipped as fs doesn't support so large files.\n");
+			ret = 0;
+		}
+		goto out;
+	}
+
+	/* offset at the beginning */
+	ret += do_lseek(testnum,  1, fd, filsz, SEEK_HOLE, 0, bufsz);
+	ret += do_lseek(testnum,  2, fd, filsz, SEEK_HOLE, 1, bufsz);
+	ret += do_lseek(testnum,  3, fd, filsz, SEEK_DATA, 0, 0);
+	ret += do_lseek(testnum,  4, fd, filsz, SEEK_DATA, 1, 1);
+
+	/* offset around eof */
+	ret += do_lseek(testnum,  5, fd, filsz, SEEK_HOLE, off, off + bufsz);
+	ret += do_lseek(testnum,  6, fd, filsz, SEEK_DATA, off, off);
+	ret += do_lseek(testnum,  7, fd, filsz, SEEK_DATA, off + 1, off + 1);
+	ret += do_lseek(testnum,  8, fd, filsz, SEEK_DATA, off - bufsz, off);
+
+out:
+	do_free(buf);
+	return ret;
+}
+
+/*
+ * Test huge file to check for overflows of block counts due to usage of
+ * 32-bit types.
+ */
+static int test12(int fd, int testnum)
+{
+	return huge_file_test(fd, testnum,
+				((long long)alloc_size << 32) + (1 << 20));
+}
+
+/*
+ * Test huge file to check for overflows of block counts due to usage of
+ * signed types
+ */
+static int test11(int fd, int testnum)
+{
+	return huge_file_test(fd, testnum,
+				((long long)alloc_size << 31) + (1 << 20));
+}
+
+/* Test an 8G file to check for offset overflows at 1 << 32 */
+static int test10(int fd, int testnum)
+{
+	return huge_file_test(fd, testnum, 8ULL << 30);
+}
+
 /*
  * test file with unwritten extents, have both dirty and
  * writeback pages in page cache.
@@ -577,6 +652,9 @@ struct testrec seek_tests[] = {
        {  7, test07, "Test file with unwritten extents, only have dirty pages" },
        {  8, test08, "Test file with unwritten extents, only have unwritten pages" },
        {  9, test09, "Test file with unwritten extents, have both dirty && unwritten pages" },
+       { 10, test10, "Test a huge file for offset overflow" },
+       { 11, test11, "Test a huge file for block number signed" },
+       { 12, test12, "Test a huge file for block number overflow" },
 };
 
 static int run_test(struct testrec *tr)
-- 
1.8.1.4

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 3/3 v2] 285: Test offsets over 4GB

[Eric Sandeen]

On 5/30/13 4:21 PM, Jan Kara wrote:
> Test whether SEEK_HOLE and SEEK_DATA works correctly with offsets over
> 4GB, 8TB, and 16TB.
> 
> Signed-off-by: Jan Kara <jack@suse.cz>

Looks fine

Reviewed-by: Eric Sandeen <sandeen@redhat.com>

fallocate makes ext3 fail for similar reasons, will send a similar patch
to fix that up.

> ---
>  src/seek_sanity_test.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 78 insertions(+)
> 
> diff --git a/src/seek_sanity_test.c b/src/seek_sanity_test.c
> index 748eec2..cd3b1ee 100644
> --- a/src/seek_sanity_test.c
> +++ b/src/seek_sanity_test.c
> @@ -18,6 +18,7 @@
>   */
>  
>  #define _XOPEN_SOURCE 500
> +#define _FILE_OFFSET_BITS 64
>  #include <sys/types.h>
>  #include <sys/stat.h>
>  #include <sys/vfs.h>
> @@ -125,6 +126,9 @@ static ssize_t do_pwrite(int fd, const void *buf, size_t count, off_t offset)
>  	while (count > written) {
>  		ret = pwrite(fd, buf + written, count - written, offset + written);
>  		if (ret < 0) {
> +			/* Don't warn about too large file. It's fs dependent. */
> +			if (errno == EFBIG)
> +				return ret;
>  			fprintf(stderr, "  ERROR %d: Failed to write %ld "
>  				"bytes\n", errno, (long)count);
>  			return ret;
> @@ -191,6 +195,77 @@ static int do_lseek(int testnum, int subtest, int fd, int filsz, int origin,
>  	return ret;
>  }
>  
> +static int huge_file_test(int fd, int testnum, off_t filsz)
> +{
> +	char *buf = NULL;
> +	int bufsz = alloc_size * 16;	/* XFS seems to round allocated size */
> +	off_t off = filsz - 2*bufsz;
> +	int ret = -1;
> +
> +	buf = do_malloc(bufsz);
> +	if (!buf)
> +		goto out;
> +	memset(buf, 'a', bufsz);
> +
> +	ret = do_pwrite(fd, buf, bufsz, 0);
> +	if (ret)
> +		goto out;
> +	ret = do_pwrite(fd, buf, bufsz, off);
> +	if (ret) {
> +		/*
> +		 * Report success. Filesystem just cannot handle so large
> +		 * offsets and correctly reports it.
> +		 */
> +		if (errno == EFBIG) {
> +			fprintf(stdout, "Test skipped as fs doesn't support so large files.\n");
> +			ret = 0;
> +		}
> +		goto out;
> +	}
> +
> +	/* offset at the beginning */
> +	ret += do_lseek(testnum,  1, fd, filsz, SEEK_HOLE, 0, bufsz);
> +	ret += do_lseek(testnum,  2, fd, filsz, SEEK_HOLE, 1, bufsz);
> +	ret += do_lseek(testnum,  3, fd, filsz, SEEK_DATA, 0, 0);
> +	ret += do_lseek(testnum,  4, fd, filsz, SEEK_DATA, 1, 1);
> +
> +	/* offset around eof */
> +	ret += do_lseek(testnum,  5, fd, filsz, SEEK_HOLE, off, off + bufsz);
> +	ret += do_lseek(testnum,  6, fd, filsz, SEEK_DATA, off, off);
> +	ret += do_lseek(testnum,  7, fd, filsz, SEEK_DATA, off + 1, off + 1);
> +	ret += do_lseek(testnum,  8, fd, filsz, SEEK_DATA, off - bufsz, off);
> +
> +out:
> +	do_free(buf);
> +	return ret;
> +}
> +
> +/*
> + * Test huge file to check for overflows of block counts due to usage of
> + * 32-bit types.
> + */
> +static int test12(int fd, int testnum)
> +{
> +	return huge_file_test(fd, testnum,
> +				((long long)alloc_size << 32) + (1 << 20));
> +}
> +
> +/*
> + * Test huge file to check for overflows of block counts due to usage of
> + * signed types
> + */
> +static int test11(int fd, int testnum)
> +{
> +	return huge_file_test(fd, testnum,
> +				((long long)alloc_size << 31) + (1 << 20));
> +}
> +
> +/* Test an 8G file to check for offset overflows at 1 << 32 */
> +static int test10(int fd, int testnum)
> +{
> +	return huge_file_test(fd, testnum, 8ULL << 30);
> +}
> +
>  /*
>   * test file with unwritten extents, have both dirty and
>   * writeback pages in page cache.
> @@ -577,6 +652,9 @@ struct testrec seek_tests[] = {
>         {  7, test07, "Test file with unwritten extents, only have dirty pages" },
>         {  8, test08, "Test file with unwritten extents, only have unwritten pages" },
>         {  9, test09, "Test file with unwritten extents, have both dirty && unwritten pages" },
> +       { 10, test10, "Test a huge file for offset overflow" },
> +       { 11, test11, "Test a huge file for block number signed" },
> +       { 12, test12, "Test a huge file for block number overflow" },
>  };
>  
>  static int run_test(struct testrec *tr)
> 

Re: [PATCH 3/3 v2] 285: Test offsets over 4GB

[Rich Johnston]

On 05/30/2013 04:21 PM, Jan Kara wrote:
> Test whether SEEK_HOLE and SEEK_DATA works correctly with offsets over
> 4GB, 8TB, and 16TB.
>
> Signed-off-by: Jan Kara <jack@suse.cz>

Thanks, this patch has been committed.

commit e2549c60dd44b4d00a82bbc4acc91e094544839f
Author: Jan Kara <jack@suse.cz>
Date:   Mon Jun 3 13:53:49 2013 -0500

     xfstests 285: Test offsets over 4GB

--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/3 v2] 285: Fix test for ext4 in some configurations

[Rich Johnston]

On 05/30/2013 04:21 PM, Jan Kara wrote:
> In some configurations (e.g. 1 KB block size), ext4 can decide it is
> better to zero out several blocks rather than splitting unwritten
> extent. This changes results SEEK_HOLE / SEEK_DATA returns and thus the
> test fails. Fix the problem by disabling the feature for this test.
>
> Reviewed-by: Eric Sandeen <sandeen@redhat.com>
> Signed-off-by: Jan Kara <jack@suse.cz>

Thanks, this patch has been committed.

commit ce9d1cdd618eaea256356c81cfd579039169b999
Author: Jan Kara <jack@suse.cz>
Date:   Mon Jun 3 13:40:36 2013 -0500

     xfstests 285: Fix test for ext4 in some configurations

--Rich

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs