Re: OpenPosix test case mmap_11-4 fails in ext4 filesystem

[Eric Sandeen <sandeen@redhat.com>]

On 5/21/14, 4:20 PM, chrubis@suse.cz wrote:
> Hi!
>> There is a pretty large amount of overlap between LTP and xfstests,
>> and xfstests are what most of the file system developers are using,
>> and we have developed a lot of automated test automation which means
>> running xfstests is very easy and convenient.  For example:
>>
>> 	https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/README
>>
>> The ability for me to build a kernel and then with a single command,
>> "kvm-xfstests smoke", do a quick verification in about 30 minutes, is
>> very convenient.
> 
> LTP is automated to a degree where you run single script and get a file
> with list of failed testcases later. We do not have any kind of kvm
> integration though.
> 
>> As I recall, ltp was integrated with autotest, and my experience with
>> autotest at multiple companies is if anything, worse than ltp's
>> reputation.  (I considered ltp to be mostly harmless, albeit not
>> particularly useful, whereas I considered autotest to be activetly
>> harmful to engineer productivity.)
> 
> The autotest integration is not a part of the LTP at all. I remember
> seeing it somewhere but I've never used it/looked at the code.
> 
> LTP has it's own script and testdriver to run testcases, but given that
> LTP tests are just binaries that are mostly self-contained it's not
> doing much more than starting a test, writing logfiles and killing
> lefover processes (if the tests fails to collect them itself). I will
> not pretend that it's clean and well designed code but at least it works
> fine (as a matter of fact I've started to work on redesigning/rewriting
> it from scratch some time ago).
> 
>> Anyway, it's already the case that most of the useful file-system
>> specific bits of LTP has been cherry picked into xfstests, and I
>> suspect it will be a lot easier to get a few additional LTP test cases
>> added into xfstests, than it will be to convince a large number of
>> file system developers that they should (a) try to figure out how to
>> integrate LTP into their test harnesses, and (b) how to avoid
>> duplicating tests which xfstests are already running.
> 
> Well I can personally help with (a).
> 
> The test in question here (mmap_11-4) is a part of the Open Posix
> Testsuite that continues to live in LTP. The whole testsuite runs in
> about 30 minutes and covers most of the POSIX interfaces in ~1600
> testcases.
> 
> Then there is a syscalls testsuite which covers, in addition to the
> POSIX specs, some of the Linux specific interfaces too. The runtime is
> about 15 minutes for ~1030 testcases.
> 
> I guess that we can filter filesystem related syscalls quite easily. The
> overlap would take more work though. In LTP we have mostly conformance
> testcases and some stress testcases. I'm not much familiar with xfstests
> and its coverage.
> 
> And we have a more tests that may be interesting to fs maintaners, there
> are aio testcases (which are likely covered by xfstests allready), some
> fs stress tests, ...
> 

FWIW, I don't think there's any real compelling reason to migrate existing
LTP tests into xfstests.  Maybe LTP folks just need to do a better job of
pitching LTP to filesystem developers, as we did with xfstests.  :)

-Eric