Re: [ANNOUNCEMENT] Linux POSIX file system test suite

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Szabolcs Szakacsits]

On Wed, 2 Apr 2008, Joel Becker wrote:
> On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote:
> > The test suite mostly checks POSIX compliance and works for FreeBSD, 
> > Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list 
> > of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, 
> > rmdir, symlink, truncate, unlink. There are currently 1950 regression 
> > tests.
> <snip> 
> > Availability:
> > 
> > 	http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz
> 
> 	Very interesting.  ocfs2, running as 'ext3' mode, gets:
>
> Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay.

That's not bad as a start and it doesn't necessarily mean that there is 
anything wrong with ocfs2. There are many cases when SuS says that the 
behavior can be implementation specific.

But we did find that following ext3 as close as possible will reduce the 
number of bug reports. We started from "574/1950 subtests failed, 70.56% 
okay."

Regards,
	    Szaka

--
NTFS-3G:  http://ntfs-3g.org

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Joel Becker]

On Thu, Apr 03, 2008 at 07:12:12AM +0300, Szabolcs Szakacsits wrote:
> > > 	http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz
> > 
> > 	Very interesting.  ocfs2, running as 'ext3' mode, gets:
> >
> > Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay.
> 
> That's not bad as a start and it doesn't necessarily mean that there is 
> anything wrong with ocfs2. There are many cases when SuS says that the 
> behavior can be implementation specific.

	Yeah, I haven't looked into the failures.  Some day when I have
free time :-)

Joel

-- 

"When I am working on a problem I never think about beauty. I
 only think about how to solve the problem. But when I have finished, if
 the solution is not beautiful, I know it is wrong."
         - Buckminster Fuller

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org
Phone: (650) 506-8127

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Amar S. Tumballi]

Hi,
 Thanks for the test tool. It really helps us to very our posix compliance
level.

GlusterFS running as 'ext3' gets:

Failed Test                       Stat Wstat Total Fail  Failed  List of
Failed
-------------------------------------------------
/tmp/pjd-fstest-20080402/tests/ch              171    5   2.92%  69 141 145
149
                                                                 153
Failed 1/184 test scripts, 99.46% okay. 5/1950 subtests failed, 99.74% okay.

Regards,
Amar
-----


> >
> >       Very interesting.  ocfs2, running as 'ext3' mode, gets:
> >
> > Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36%
> okay.
>
> That's not bad as a start and it doesn't necessarily mean that there is
> anything wrong with ocfs2. There are many cases when SuS says that the
> behavior can be implementation specific.
>
> But we did find that following ext3 as close as possible will reduce the
> number of bug reports. We started from "574/1950 subtests failed, 70.56%
> okay."
>
> Regards,
>             Szaka
>
> --
> NTFS-3G:  http://ntfs-3g.org
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
>
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> fuse-devel mailing list
> fuse-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/fuse-devel
>
>


-- 
Amar Tumballi
Gluster/GlusterFS Hacker
[bulde on #gluster/irc.gnu.org]
http://www.zresearch.com - Commoditizing Supercomputing and Superstorage!
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Szabolcs Szakacsits]

On Thu, 3 Apr 2008, Badari Pulavarty wrote:
> On Thu, 2008-04-03 at 00:29 +0300, Szabolcs Szakacsits wrote:
>
> > The default file system type is ext3 and it passes all tests.
> 
> Hmm.. I ran it against ext2, ext3, jfs, btrfs. I don't see all "pass"
> on ext3. What am I missing ?

Your unique, consistently failing test cases for all file systems suggest 
that you have a buggy private kernel or some other individual issue in your 
test environment.

You could use the -x debug shell option in the test scripts, rerun the 
failing ones and they will show why these test cases exactly fail.
 
> btrfs seems to have little more failures.

If you find the reason for the unexpected failures then the btrfs result 
will be quite good. Apparently it has only a few link, truncate, and unlink 
ctimes update problems. I think that's quite impressive in its state of 
development.

	Szaka


> ext2:
> ====
> 
> Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
> /root/posix/tests/chown/00.t                 171    4   2.34%  141 145 149 153
> /root/posix/tests/link/00.t                   82    6   7.32%  3 5-6 8-10
> /root/posix/tests/open/05.t                   12    2  16.67%  5 9
> /root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
>                                                                39 42
> /root/posix/tests/symlink/00.t                14    2  14.29%  2 5
> /root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
> /root/posix/tests/truncate/12.t                3    1  33.33%  2
> /root/posix/tests/truncate/13.t                4    2  50.00%  2-3
> Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay.
> 
> 
> ext3:
> ====
> 
> Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
> /root/posix/tests/link/00.t                   82    6   7.32%  3 5-6 8-10
> /root/posix/tests/open/05.t                   12    2  16.67%  5 9
> /root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
>                                                                39 42
> /root/posix/tests/symlink/00.t                14    2  14.29%  2 5
> /root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
> /root/posix/tests/truncate/12.t                3    1  33.33%  2
> /root/posix/tests/truncate/13.t                4    2  50.00%  2-3
> Failed 8/184 test scripts, 95.65% okay. 29/1950 subtests failed, 98.51% okay.
> 
> 
> jfs:
> ===
> 
> Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
> /root/posix/tests/chown/00.t                 171    4   2.34%  141 145 149 153
> /root/posix/tests/link/00.t                   82    6   7.32%  3 5-6 8-10
> /root/posix/tests/open/05.t                   12    2  16.67%  5 9
> /root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
>                                                                39 42
> /root/posix/tests/symlink/00.t                14    2  14.29%  2 5
> /root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
> /root/posix/tests/truncate/12.t                3    1  33.33%  2
> /root/posix/tests/truncate/13.t                4    2  50.00%  2-3
> Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay.
> 
> 
> btrfs:
> =====
> 
> Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
> /root/posix/tests/chown/00.t                 171    4   2.34%  141 145 149 153
> /root/posix/tests/link/00.t                   82    8   9.76%  3 5-6 8-10 56 63
> /root/posix/tests/open/05.t                   12    2  16.67%  5 9
> /root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
>                                                                39 42
> /root/posix/tests/symlink/00.t                14    2  14.29%  2 5
> /root/posix/tests/truncate/00.t               21    1   4.76%  15
> /root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
> /root/posix/tests/truncate/12.t                3    1  33.33%  2
> /root/posix/tests/truncate/13.t                4    2  50.00%  2-3
> /root/posix/tests/unlink/00.t                 55    3   5.45%  17 22 53
> Failed 11/184 test scripts, 94.02% okay. 39/1950 subtests failed, 98.00% okay.
>
>
>

--
NTFS-3G:  http://ntfs-3g.org



-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Register now and save $200. Hurry, offer ends at 11:59 p.m., 
Monday, April 7! Use priority code J8TLD2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Badari Pulavarty]

On Mon, 2008-04-07 at 02:34 +0300, Szabolcs Szakacsits wrote:
> On Thu, 3 Apr 2008, Badari Pulavarty wrote:
> > On Thu, 2008-04-03 at 00:29 +0300, Szabolcs Szakacsits wrote:
> >
> > > The default file system type is ext3 and it passes all tests.
> > 
> > Hmm.. I ran it against ext2, ext3, jfs, btrfs. I don't see all "pass"
> > on ext3. What am I missing ?
> 
> Your unique, consistently failing test cases for all file systems suggest 
> that you have a buggy private kernel or some other individual issue in your 
> test environment.
> 
> You could use the -x debug shell option in the test scripts, rerun the 
> failing ones and they will show why these test cases exactly fail.

I am running 2.6.25-rc8-mm1. What I noticed is, these failures happen
on ppc64. I ran the tests on x86-64 and ext2, ext3 passed all the tests.

> > btrfs seems to have little more failures.
> 
> If you find the reason for the unexpected failures then the btrfs result 
> will be quite good. Apparently it has only a few link, truncate, and unlink 
> ctimes update problems. I think that's quite impressive in its state of 
> development.

Yes. Indeed. btrfs has only these tests failing now (x86-64).


Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
/root/posix/tests/link/00.t                   82    2   2.44%  56 63
/root/posix/tests/truncate/00.t               21    1   4.76%  15
/root/posix/tests/unlink/00.t                 55    3   5.45%  17 22 53
Failed 3/184 test scripts, 98.37% okay. 6/1950 subtests failed, 99.69% okay.

Thanks,
Badari

ext4 not updating ctime on truncate ?

[Badari Pulavarty]

Hi,

I ran Linux POSIX filesystem test suite released by  Szabolcs Szakacsits
on ext4. I get one failure on ext4.

Failed Test                     Stat Wstat Total Fail  Failed  List of
Failed
-------------------------------------------------------------------------------
/root/posix/tests/truncate/00.t               21    1   4.76%  15

Looking at the test, its complaining that ext4 is not updating ctime
for truncate. Is this a known issue ?

# successful truncate(2) updates ctime.
expect 0 create ${n0} 0644
ctime1=`${fstest} stat ${n0} ctime`
sleep 1
expect 0 truncate ${n0} 123
ctime2=`${fstest} stat ${n0} ctime`
test_check $ctime1 -lt $ctime2


Thanks,
Badari

Re: ext4 not updating ctime on truncate ?

[Eric Sandeen]

Badari Pulavarty wrote:
> Hi,
> 
> I ran Linux POSIX filesystem test suite released by  Szabolcs Szakacsits
> on ext4. I get one failure on ext4.
> 
> Failed Test                     Stat Wstat Total Fail  Failed  List of
> Failed
> -------------------------------------------------------------------------------
> /root/posix/tests/truncate/00.t               21    1   4.76%  15
> 
> Looking at the test, its complaining that ext4 is not updating ctime
> for truncate. Is this a known issue ?

Solofo sent a patch for this just yesterday:

[PATCH] ext4: update ctime and mtime for truncate with extents.

-Eric

Re: [ANNOUNCEMENT] Linux POSIX file system test suite

[Szabolcs Szakacsits]

On Fri, 4 Apr 2008, David Chinner wrote:
> On Fri, Apr 04, 2008 at 10:33:30AM +1000, David Chinner wrote:
> > On Thu, Apr 03, 2008 at 12:29:47AM +0300, Szabolcs Szakacsits wrote:
> > >   The test must be run as root user and requires a few basic Perl modules.
> > 
> > And openssl, it appears.

Openssl is replaced with md5sum+cut in the CVS. 

It would be also nice to eliminate the Perl dependency ...

> > The current xfs-dev tree:
> > 
> > Failed Test                    Stat Wstat Total Fail  Failed  List of Failed
> > -------------------------------------------------------------------------------
> > /root/posix/tests/chown/00.t                171    2   1.17%  84 88
> > /root/posix/tests/symlink/02.t                7    2  28.57%  6-7
> > Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay.
> 
> Symlink tests 6 and 7:
> 
> expect 0 symlink ${name256} ${n0}
> expect 0 unlink ${n0}
> 
> Test 6 is failing with ENAMETOOLONG
> Test 7 is failing (correctly) with ENOENT because test 6 failed.
> 
> So there's only one failure here, and that is that that we're rejecting
> ${name256} as too long. I think that getname() is doing this. Seems sane
> to me to disallow symlinking to pathnames that can't be constructed,
> even if POSIX apparently allows it.

As Christoph noted, I also noticed XFS is unique in this behavior.

> Chown tests 84 and 88:
[...] 
> So, either result is valid. Hence i suggest that test 84 and test 88
> (same failure) are special cased to "ext3" behaviour.

Done in the CVS.

> That means XFS is not failing any tests at all.

I added the xfs target but left the symlink Test 6 fail because POSIX says

 "The string pointed to by path1 shall be treated only as a character 
  string and shall not be validated as a pathname" 

and 
 
 "the length of the path1 argument is longer than {SYMLINK_MAX}" 

where {SYMLINK_MAX} is typically not defined on Linux or it's {PATH_MAX}.

	Szaka

--
NTFS-3G:  http://ntfs-3g.org


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Register now and save $200. Hurry, offer ends at 11:59 p.m., 
Monday, April 7! Use priority code J8TLD2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone