Re: OpenPosix test case mmap_11-4 fails in ext4 filesystem

[Theodore Ts'o <tytso@mit.edu>]

On Wed, May 21, 2014 at 11:20:03PM +0200, chrubis@suse.cz wrote:
> 
> The test in question here (mmap_11-4) is a part of the Open Posix
> Testsuite that continues to live in LTP. The whole testsuite runs in
> about 30 minutes and covers most of the POSIX interfaces in ~1600
> testcases.

I'm pretty familiar with the PCTS (which is different from the Open
Posix Testsuite), epsecially as it relates to the tty/termios
interfaces.

At least with the PCTS, there are a large number of the tests which
are primarily issues which are implemented in the core VFS layer or in
the core TTY layer.  If you are a core tty implementor (as I was), the
PCTS was really useful for that.  If you are a serial device driver
implementor, however, not all of the tests were as useful.

I suspect the same would be true with file system related tests.
There will be those tests which are are really useful if you are doing
core VFS work, and then there are those tests which are useful if you
are working on a particular file system.  Separating out those tests
which are most useful for developers would probably be a good thing,
although obviously tere will always be a certain amount of overlap.

> I guess that we can filter filesystem related syscalls quite easily. The
> overlap would take more work though. In LTP we have mostly conformance
> testcases and some stress testcases. I'm not much familiar with xfstests
> and its coverage.

Xfstests has already taken some parts of LTP.  The xfstests sources
has a ltp directory with the following:

% ls ltp
total 376
40 aio-stress.c   8 doio.h	44 fsx.c	44 iogen.c    8 rwtest.sh*
84 doio.c	   68 fsstress.c   76 growfiles.c   4 Makefile

Furthermore, there are a number of xfstests which run programs like
fsstress and fsx using a variety of configuration options which have
historically been best at stress testing file systems. 

So it sounds like there's quite a lot of overlap, some of it caused by
xfstests grabbing bits and pieces of LTP, and part of it because
people have been adding both functional and stress tests to xfstests.

The question is what's the best way of dealing with the overlap.
Clearly xfstests has a lot more mindshare amonst file system
developers, and LTP does have that unfortunate reputation which it is
still trying to overcome.  :-/

Cheers,

						- Ted