qa 166 failure on f8 kernel

qa 166 failure on f8 kernel

[Eric Sandeen]

Posting this just in case it rings any bells, though I plan to
investigate...

[root@inode xfstests]# ./check  166
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6

166      - output mismatch (see 166.out.bad)
2,6c2,7
< 0: [0..31]: XX..YY AG (AA..BB) 32
< 1: [32..127]: XX..YY AG (AA..BB) 96 10000
< 2: [128..159]: XX..YY AG (AA..BB) 32
< 3: [160..223]: XX..YY AG (AA..BB) 64 10000
< 4: [224..255]: XX..YY AG (AA..BB) 32
---
> /mnt/sdb6/test_file:  XX..YY AG (AA..BB)
> 0: [0..7]: XX..YY AG (AA..BB) 8
> 1: [8..127]: XX..YY AG (AA..BB) 120 10000
> 2: [128..135]: XX..YY AG (AA..BB) 8
> 3: [136..247]: XX..YY AG (AA..BB) 112 10000
> 4: [248..255]: XX..YY AG (AA..BB) 8
Failures: 166
Failed 1 of 1 tests
[root@inode xfstests]# uname -a
Linux inode.lab.msp.redhat.com 2.6.23-0.71.rc2.fc8 #1 SMP Sat Aug 4
01:21:06 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

-Eric

Re: qa 166 failure on f8 kernel

[David Chinner]

On Tue, Aug 07, 2007 at 08:39:06PM -0500, Eric Sandeen wrote:
> Posting this just in case it rings any bells, though I plan to
> investigate...
> 
> [root@inode xfstests]# ./check  166
> FSTYP         -- xfs (non-debug)
> PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
> MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
> MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6
> 
> 166      - output mismatch (see 166.out.bad)
> 2,6c2,7
> < 0: [0..31]: XX..YY AG (AA..BB) 32
> < 1: [32..127]: XX..YY AG (AA..BB) 96 10000
> < 2: [128..159]: XX..YY AG (AA..BB) 32
> < 3: [160..223]: XX..YY AG (AA..BB) 64 10000
> < 4: [224..255]: XX..YY AG (AA..BB) 32
> ---
> > /mnt/sdb6/test_file:  XX..YY AG (AA..BB)
> > 0: [0..7]: XX..YY AG (AA..BB) 8
> > 1: [8..127]: XX..YY AG (AA..BB) 120 10000
> > 2: [128..135]: XX..YY AG (AA..BB) 8
> > 3: [136..247]: XX..YY AG (AA..BB) 112 10000
> > 4: [248..255]: XX..YY AG (AA..BB) 8

When you post the failure, someone will say, "Duh, that's obvious".

Well: Duh! That's obvious. ;)

You've got 3x4k written blocks in the file which is *correct*.
There's nothing wrong with the kernel code. It's just that the test
is expecting 3x16k extents to be marked written.

See the problem yet?

mmap dirties entire pages. page size differs between platforms -
ia64 = 16k, x86 = 4k - so the size of the extent allocated is
different. Guess what platform I wrote the test on and use as my
primary platform?

The output needs better filtering, methinks.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

David Chinner wrote:
> On Tue, Aug 07, 2007 at 08:39:06PM -0500, Eric Sandeen wrote:
>> Posting this just in case it rings any bells, though I plan to
>> investigate...
>>
>> [root@inode xfstests]# ./check  166
>> FSTYP         -- xfs (non-debug)
>> PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
>> MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
>> MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6
>>
>> 166      - output mismatch (see 166.out.bad)
>> 2,6c2,7
>> < 0: [0..31]: XX..YY AG (AA..BB) 32
>> < 1: [32..127]: XX..YY AG (AA..BB) 96 10000
>> < 2: [128..159]: XX..YY AG (AA..BB) 32
>> < 3: [160..223]: XX..YY AG (AA..BB) 64 10000
>> < 4: [224..255]: XX..YY AG (AA..BB) 32
>> ---
>>> /mnt/sdb6/test_file:  XX..YY AG (AA..BB)
>>> 0: [0..7]: XX..YY AG (AA..BB) 8
>>> 1: [8..127]: XX..YY AG (AA..BB) 120 10000
>>> 2: [128..135]: XX..YY AG (AA..BB) 8
>>> 3: [136..247]: XX..YY AG (AA..BB) 112 10000
>>> 4: [248..255]: XX..YY AG (AA..BB) 8
> 
> When you post the failure, someone will say, "Duh, that's obvious".

fine, fine ;-)

> Well: Duh! That's obvious. ;)
> 
> You've got 3x4k written blocks in the file which is *correct*.
> There's nothing wrong with the kernel code. It's just that the test
> is expecting 3x16k extents to be marked written.
> 
> See the problem yet?
> 
> mmap dirties entire pages. page size differs between platforms -
> ia64 = 16k, x86 = 4k - so the size of the extent allocated is
> different. Guess what platform I wrote the test on and use as my
> primary platform?
> 
> The output needs better filtering, methinks.

yeah, figured it had something to do w/ the ia64 weenies when I saw the
32 vs. 8, factor of 4 ... fine, fine, my bad.  :)

Thanks,

-Eric

> Cheers,
> 
> Dave.

Re: qa 166 failure on f8 kernel

[David Chinner]

On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
> yeah, figured it had something to do w/ the ia64 weenies when I saw the
> 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)

No, not your bad. Mine if anyones because I wrote the test.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

David Chinner wrote:

> On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
>   
>> yeah, figured it had something to do w/ the ia64 weenies when I saw the
>> 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)
>>     
>
> No, not your bad. Mine if anyones because I wrote the test.
>
> Cheers,
>
> Dave.
>   
Well, I was going to look at it more closely before I sent it off :)

What do you think of a patch like this, to munmap 16k chunks regardless
of page size:


--- src/unwritten_mmap.c.orig	2007-08-07 22:53:08.962031839 -0500
+++ src/unwritten_mmap.c	2007-08-07 23:02:05.939112618 -0500
@@ -12,6 +12,7 @@
  */
 int main(int argc, char **argv) {
 	unsigned long long o;
+	int minsize;
 	int fd, i;
 	struct xfs_flock64 space;
 	unsigned char *buf;
@@ -23,6 +24,13 @@
 
 	errno = 0;
 	o = strtoull(argv[1], NULL, 0);
+
+	minsize = 3*16384; /* 3 ia64 pages */
+	if (o < minsize) {
+		fprintf(stderr, "count must be at least %d\n", minsize);
+		exit(1);
+	}
+
 	if (errno) {
 		perror("strtoull");
 		exit(errno);
@@ -55,9 +63,9 @@
 			perror("mmap()");
 			exit(5);
 		} else {
+			memset(buf, 0, 16384);
+			memset(buf+o/2, 0, 16384);
+			memset(buf+o-16384, 0, 16384);
 			munmap(buf, o);
 		}

-Eric
 

RE: qa 166 failure on f8 kernel

[Alex Elder]

> From: xfs-bounce@oss.sgi.com 
> [mailto:xfs-bounce@oss.sgi.com]On Behalf Of
> David Chinner
> Sent: Tuesday, August 07, 2007 9:56 PM
> To: Eric Sandeen
> Cc: xfs-oss
> Subject: Re: qa 166 failure on f8 kernel
...
> mmap dirties entire pages. page size differs between platforms -
> ia64 = 16k, x86 = 4k - so the size of the extent allocated is
> different. Guess what platform I wrote the test on and use as my
> primary platform?
> 
> The output needs better filtering, methinks.

Could the test be changed so the output is pagesize independent
rather than filtering the output?  Or maybe the test should
use a buffer size that's the LCM of all supported page sizes.

I'm not actually familiar with the test, so forgive me if I'm
way off...

					-Alex

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

Eric Sandeen wrote:

> David Chinner wrote:
>
>   
>> On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
>>   
>>     
>>> yeah, figured it had something to do w/ the ia64 weenies when I saw the
>>> 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)
>>>     
>>>       
>> No, not your bad. Mine if anyones because I wrote the test.
>>
>> Cheers,
>>
>> Dave.
>>   
>>     
> Well, I was going to look at it more closely before I sent it off :)
>
> What do you think of a patch like this, to munmap 16k chunks regardless
> of page size:
>   
Er, bad patch-ninja-boy, don't hand-edit  :)



--- src/unwritten_mmap.c.orig	2007-08-07 22:53:08.962031839 -0500
+++ src/unwritten_mmap.c	2007-08-07 23:04:14.230540957 -0500
@@ -12,6 +12,7 @@
  */
 int main(int argc, char **argv) {
 	unsigned long long o;
+	int minsize;
 	int fd, i;
 	struct xfs_flock64 space;
 	unsigned char *buf;
@@ -23,6 +24,13 @@
 
 	errno = 0;
 	o = strtoull(argv[1], NULL, 0);
+
+	minsize = 3*16384; /* 3 ia64 pages */
+	if (o < minsize) {
+		fprintf(stderr, "count must be at least %d\n", minsize);
+		exit(1);
+	}
+
 	if (errno) {
 		perror("strtoull");
 		exit(errno);
@@ -55,9 +63,9 @@
 			perror("mmap()");
 			exit(5);
 		} else {
-			buf[o-1] = 0;
-			buf[o/2] = 0;
-			buf[0] = 0;
+			memset(buf, 0, 16384);
+			memset(buf+o/2, 0, 16384);
+			memset(buf+o-16384, 0, 16384);
 			munmap(buf, o);
 		}

 

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

Barry Naujok wrote:
> On Wed, 08 Aug 2007 11:39:06 +1000, Eric Sandeen <sandeen@sandeen.net>  
> wrote:
> 
>> Posting this just in case it rings any bells, though I plan to
>> investigate...
>>
>> [root@inode xfstests]# ./check  166
>> FSTYP         -- xfs (non-debug)
>> PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
>> MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
>> MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6
>>
>> 166      - output mismatch (see 166.out.bad)
>> 2,6c2,7
>> < 0: [0..31]: XX..YY AG (AA..BB) 32
>> < 1: [32..127]: XX..YY AG (AA..BB) 96 10000
>> < 2: [128..159]: XX..YY AG (AA..BB) 32
>> < 3: [160..223]: XX..YY AG (AA..BB) 64 10000
>> < 4: [224..255]: XX..YY AG (AA..BB) 32
>> ---
>>> /mnt/sdb6/test_file:  XX..YY AG (AA..BB)
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> I suspect this line is appearing in the output due to a number
> in the mountpoint. I think the _filter_blocks() is only
> extracting lines with numbers in it.

Bingo.  Yep, changing it to /mnt/foo, plus my change to the test,
passes.  thanks!

-Eric

Re: qa 166 failure on f8 kernel

[Barry Naujok]

On Wed, 08 Aug 2007 11:39:06 +1000, Eric Sandeen <sandeen@sandeen.net>  
wrote:

> Posting this just in case it rings any bells, though I plan to
> investigate...
>
> [root@inode xfstests]# ./check  166
> FSTYP         -- xfs (non-debug)
> PLATFORM      -- Linux/x86_64 inode 2.6.23-0.71.rc2.fc8
> MKFS_OPTIONS  -- -f -bsize=4096 /dev/sdb6
> MOUNT_OPTIONS -- /dev/sdb6 /mnt/sdb6
>
> 166      - output mismatch (see 166.out.bad)
> 2,6c2,7
> < 0: [0..31]: XX..YY AG (AA..BB) 32
> < 1: [32..127]: XX..YY AG (AA..BB) 96 10000
> < 2: [128..159]: XX..YY AG (AA..BB) 32
> < 3: [160..223]: XX..YY AG (AA..BB) 64 10000
> < 4: [224..255]: XX..YY AG (AA..BB) 32
> ---
>> /mnt/sdb6/test_file:  XX..YY AG (AA..BB)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I suspect this line is appearing in the output due to a number
in the mountpoint. I think the _filter_blocks() is only
extracting lines with numbers in it.

>> 0: [0..7]: XX..YY AG (AA..BB) 8
>> 1: [8..127]: XX..YY AG (AA..BB) 120 10000
>> 2: [128..135]: XX..YY AG (AA..BB) 8
>> 3: [136..247]: XX..YY AG (AA..BB) 112 10000
>> 4: [248..255]: XX..YY AG (AA..BB) 8
> Failures: 166
> Failed 1 of 1 tests
> [root@inode xfstests]# uname -a
> Linux inode.lab.msp.redhat.com 2.6.23-0.71.rc2.fc8 #1 SMP Sat Aug 4
> 01:21:06 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
>
> -Eric
>
>

Re: qa 166 failure on f8 kernel

[David Chinner]

On Tue, Aug 07, 2007 at 11:11:05PM -0500, Eric Sandeen wrote:
> David Chinner wrote:
> 
> > On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
> >   
> >> yeah, figured it had something to do w/ the ia64 weenies when I saw the
> >> 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)
> >>     
> >
> > No, not your bad. Mine if anyones because I wrote the test.
> >
> > Cheers,
> >
> > Dave.
> >   
> Well, I was going to look at it more closely before I sent it off :)
> 
> What do you think of a patch like this, to munmap 16k chunks regardless
> of page size:

That could be done, but will break on 64k page size machines ;)

Perhaps the filter needs to use `getconf PAGE_SIZE` to filter the
output down:

ia64:

% getconf PAGE_SIZE
16384

x86_64:

% getconf PAGE_SIZE
4096

So the written extent sizes are output as a multiple of
page size rather than basic blocks....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

David Chinner wrote:

> That could be done, but will break on 64k page size machines ;)
> 
> Perhaps the filter needs to use `getconf PAGE_SIZE` to filter the
> output down:
> 
> ia64:
> 
> % getconf PAGE_SIZE
> 16384
> 
> x86_64:
> 
> % getconf PAGE_SIZE
> 4096
> 
> So the written extent sizes are output as a multiple of
> page size rather than basic blocks....


yeah, computing all the numbers for the filter seemed nasty.

Maybe just a dumbed-down filter that expects

written
unwritten
written
unwritten
written

would be sufficient for this test.

--- 166.orig    2007-08-07 23:19:33.391317217 -0500
+++ 166 2007-08-07 23:21:11.111409407 -0500
@@ -31,9 +31,9 @@
 {
        $AWK_PROG '/[0-9]/ {
                if ($7)
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6, $7;
+                       print $7
                else
-                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6;
+                       print "no flags"
        }'
 }

--- 166.out.orig        2007-08-07 23:21:25.299244961 -0500
+++ 166.out     2007-08-07 23:21:45.706131763 -0500
@@ -1,6 +1,6 @@
 QA output created by 166
-0: [0..31]: XX..YY AG (AA..BB) 32
-1: [32..127]: XX..YY AG (AA..BB) 96 10000
-2: [128..159]: XX..YY AG (AA..BB) 32
-3: [160..223]: XX..YY AG (AA..BB) 64 10000
-4: [224..255]: XX..YY AG (AA..BB) 32
+no flags
+10000
+no flags
+10000
+no flags


-Eric

Re: qa 166 failure on f8 kernel

[Barry Naujok]

On Wed, 08 Aug 2007 14:28:48 +1000, Eric Sandeen <sandeen@sandeen.net>  
wrote:

> David Chinner wrote:
>
>> That could be done, but will break on 64k page size machines ;)
>>
>> Perhaps the filter needs to use `getconf PAGE_SIZE` to filter the
>> output down:
>>
>> ia64:
>>
>> % getconf PAGE_SIZE
>> 16384
>>
>> x86_64:
>>
>> % getconf PAGE_SIZE
>> 4096
>>
>> So the written extent sizes are output as a multiple of
>> page size rather than basic blocks....
>
>
> yeah, computing all the numbers for the filter seemed nasty.
>
> Maybe just a dumbed-down filter that expects
>
> written
> unwritten
> written
> unwritten
> written
>
> would be sufficient for this test.
>
> --- 166.orig    2007-08-07 23:19:33.391317217 -0500
> +++ 166 2007-08-07 23:21:11.111409407 -0500
> @@ -31,9 +31,9 @@
>  {
>         $AWK_PROG '/[0-9]/ {
>                 if ($7)
> -                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6, $7;
> +                       print $7
>                 else
> -                       print $1, $2, "XX..YY", "AG", "(AA..BB)", $6;
> +                       print "no flags"
>         }'
>  }
>
> --- 166.out.orig        2007-08-07 23:21:25.299244961 -0500
> +++ 166.out     2007-08-07 23:21:45.706131763 -0500
> @@ -1,6 +1,6 @@
>  QA output created by 166
> -0: [0..31]: XX..YY AG (AA..BB) 32
> -1: [32..127]: XX..YY AG (AA..BB) 96 10000
> -2: [128..159]: XX..YY AG (AA..BB) 32
> -3: [160..223]: XX..YY AG (AA..BB) 64 10000
> -4: [224..255]: XX..YY AG (AA..BB) 32
> +no flags
> +10000
> +no flags
> +10000
> +no flags
>
>
> -Eric
>

Instead of "no flags", how about "00000" so the test doesn't break
with xfsprogs 2.9.x and later.
>

Re: qa 166 failure on f8 kernel

[Eric Sandeen]

Barry Naujok wrote:

> Instead of "no flags", how about "00000" so the test doesn't break
> with xfsprogs 2.9.x and later.
> 
> 
[root@inode xfstests]# rpm -q xfsprogs
xfsprogs-2.9.3-1


I have that... but... whatever works :)

-Eric

RE: qa 166 failure on f8 kernel

[nscott]

>> From: xfs-bounce@oss.sgi.com
>> [mailto:xfs-bounce@oss.sgi.com]On Behalf Of
>> David Chinner
>> Sent: Tuesday, August 07, 2007 9:56 PM
>> To: Eric Sandeen
>> Cc: xfs-oss
>> Subject: Re: qa 166 failure on f8 kernel
> ...
>> mmap dirties entire pages. page size differs between platforms -
>> ia64 = 16k, x86 = 4k - so the size of the extent allocated is
>> different. Guess what platform I wrote the test on and use as my
>> primary platform?
>>
>> The output needs better filtering, methinks.
>
> Could the test be changed so the output is pagesize independent
> rather than filtering the output?  Or maybe the test should
> use a buffer size that's the LCM of all supported page sizes.
>

FWIW, xfs-cmds/xfstests/src/feature.c will tell you the page size (-s);
with that you can accurately filter the output and not lose any info.

cheers.

--
Nathan

Re: qa 166 failure on f8 kernel

[Christoph Hellwig]

On Wed, Aug 08, 2007 at 02:08:04PM +1000, David Chinner wrote:
> On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
> > yeah, figured it had something to do w/ the ia64 weenies when I saw the
> > 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)
> 
> No, not your bad. Mine if anyones because I wrote the test.

Ia64 supports 4k, 8k, 16k and 64k base pagesize.  Time to add a little
script to make the pagesize random for your kernel builds?

Re: qa 166 failure on f8 kernel

[David Chinner]

On Wed, Aug 08, 2007 at 02:13:19PM +0100, Christoph Hellwig wrote:
> On Wed, Aug 08, 2007 at 02:08:04PM +1000, David Chinner wrote:
> > On Tue, Aug 07, 2007 at 09:58:50PM -0500, Eric Sandeen wrote:
> > > yeah, figured it had something to do w/ the ia64 weenies when I saw the
> > > 32 vs. 8, factor of 4 ... fine, fine, my bad.  :)
> > 
> > No, not your bad. Mine if anyones because I wrote the test.
> 
> Ia64 supports 4k, 8k, 16k and 64k base pagesize.  Time to add a little
> script to make the pagesize random for your kernel builds?

Interesting idea. Definitely worth considering for the randomised
mkfs/mount option config QA runs we already do....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group