Questions about xfstests regarding porting it to test ext4 filesystems

Questions about xfstests regarding porting it to test ext4 filesystems

[Theodore Ts'o]

I've been playing around with xfstests (aka xfsqa) and have some
questions:

1)  What is the best mailing list to use for discussions about xfstests? 
I'm *not* subscribed to xfs@oss.sgi.com, since I'm concerned about
traffic levels... I'm on too many high-volume mailing lists already.

2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
that when testing UDF and NFS, the scratch volume is used (and $testdir
is set to the point at the scratch directory).   Is there some
fundamental reason why there must be an XFS volume mounted, even if the
fundamental intention is to test some other filesystem type, whether
it's UDF, NFS, or Ext4?

3) How much latitude/interest is there in modifying xfstests to be a bit
more filesystem independent?  I understand the primary purpose of
xfstests has to be to support XFS development, but looking at the
scripts, there is a *huge* amount of XFS-specific assumptions all over
the shell scripts.  As a result I'm still of two minds whether it will
be less work to start from scratch to develop a test suite for ext4 (and
from the beginning try to make it filesystem independent) or to try to
hack xfstests and try to make it more filesystem indpendent.  A lot of
this depends on the time/interest with the xfstests upstream in working
with me.  (And assuming we get the licensing issues dealt with --- hence
my interest and time spent in trying to clarify the licensing situation
with the fsx program.)   

If there isn't a whole lot of interest in trying to make xfstests more
portable, that's fine.  It may be that I'm better off starting from
scratch.  There does seem to be a lot of work and experience represented
in the xfstest suite, though, so I would like to try exploring the
possibility of expanding its scope to also support testing other
filesystems (such as ext4, btrfs, etc.)

(I'm not on the xfs mailing list, so please keep me cc'ed, thanks.)

						- Ted

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Questions about xfstests regarding porting it to test ext4 filesystems

[Eric Sandeen]

Theodore Ts'o wrote:
> I've been playing around with xfstests (aka xfsqa) and have some
> questions:
> 
> 1)  What is the best mailing list to use for discussions about xfstests? 
> I'm *not* subscribed to xfs@oss.sgi.com, since I'm concerned about
> traffic levels... I'm on too many high-volume mailing lists already.

This one is fine.  it's not that high-volume but as you've found you
don't have to subscribe, anyway.  :)

> 2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
> that when testing UDF and NFS, the scratch volume is used (and $testdir
> is set to the point at the scratch directory).   Is there some
> fundamental reason why there must be an XFS volume mounted, even if the
> fundamental intention is to test some other filesystem type, whether
> it's UDF, NFS, or Ext4?

Hm I'll have to dig, not certain.  I thought that TESTDIR would usually
be used for non-destructive testing, general IO etc, while SCRATCHDIR
would usually be used for any test that required lots of repeatability,
and could therefore be more fs-specific.  But that's handwaving.

> 3) How much latitude/interest is there in modifying xfstests to be a bit
> more filesystem independent?  

I talked about just this at LSF last month and there seemed to be a bit
of interest, it was the best plan we had coming out of the talk.

I also talked about a fledgling effort from RH,
http://git.fedoraproject.org/git/?p=tafs.git;a=summary

> I understand the primary purpose of
> xfstests has to be to support XFS development, but looking at the
> scripts, there is a *huge* amount of XFS-specific assumptions all over
> the shell scripts.  As a result I'm still of two minds whether it will
> be less work to start from scratch to develop a test suite for ext4 (and
> from the beginning try to make it filesystem independent) or to try to
> hack xfstests and try to make it more filesystem indpendent.

Yep, agreed.  I think it's worth evaluating whether the xfstests harness
is within reach of the goal, though.  But you may be right about how
much work it is to get there.

Really the value to other filesystems is in the tests (those which are
or could be made generic), more than the harness, I'd say.

> A lot of
> this depends on the time/interest with the xfstests upstream in working
> with me.  (And assuming we get the licensing issues dealt with --- hence
> my interest and time spent in trying to clarify the licensing situation
> with the fsx program.)   

and at least as importantly, the rest of the scripts, but we're making
progress on that.

> If there isn't a whole lot of interest in trying to make xfstests more
> portable, that's fine.  It may be that I'm better off starting from
> scratch.  There does seem to be a lot of work and experience represented
> in the xfstest suite, though, so I would like to try exploring the
> possibility of expanding its scope to also support testing other
> filesystems (such as ext4, btrfs, etc.)

There is interest, it's what we decided at least preliminarily at LSF,
and now it's the "find time and see if it's feasible" part I guess.

-Eric

> (I'm not on the xfs mailing list, so please keep me cc'ed, thanks.)
> 
> 						- Ted

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Questions about xfstests regarding porting it to test ext4 filesystems

[Timothy Shimmin]

Hi there,

On Fri, May 15, 2009 at 2:24 AM, Eric Sandeen <sandeen@sandeen.net> wrote:
>
> Theodore Ts'o wrote:
>
> > 2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
> > that when testing UDF and NFS, the scratch volume is used (and $testdir
> > is set to the point at the scratch directory).   Is there some
> > fundamental reason why there must be an XFS volume mounted, even if the
> > fundamental intention is to test some other filesystem type, whether
> > it's UDF, NFS, or Ext4?
>
> Hm I'll have to dig, not certain.  I thought that TESTDIR would usually
> be used for non-destructive testing, general IO etc, while SCRATCHDIR
> would usually be used for any test that required lots of repeatability,
> and could therefore be more fs-specific.  But that's handwaving.
>
Yeah, I'm not sure if there is a reason to have testdir as a
persistent xfs volume -
in particular I'm not sure on any constraints existing in the tests.
However, the initial TESTDIR intention was to do tests over an ageing filesystem
and one where obviously we didn't need to mkfs over it etc. It exists
and is checked
beforehand and it exists and is checked afterwards.

When doing UDF/IRIX testing we copied over a bunch of xfstests that were general
enough to use on UDF for us. From distant memory, I don't think we
really cared for
a distinction between scratchdir and testdir, and just wanted one
volume to be necessary
for testing (and hence why the main interest for udf was in scratchdir).
Later, someone else ported back the udf tests into the xfstests in an
effort to make
things more filesystem independent.


> > I understand the primary purpose of
> > xfstests has to be to support XFS development, but looking at the
> > scripts, there is a *huge* amount of XFS-specific assumptions all over
> > the shell scripts.  As a result I'm still of two minds whether it will
> > be less work to start from scratch to develop a test suite for ext4 (and
> > from the beginning try to make it filesystem independent) or to try to
> > hack xfstests and try to make it more filesystem indpendent.
>
Tricky ;-)

> Yep, agreed.  I think it's worth evaluating whether the xfstests harness
> is within reach of the goal, though.  But you may be right about how
> much work it is to get there.
>
> Really the value to other filesystems is in the tests (those which are
> or could be made generic), more than the harness, I'd say.
>
I'd agree.
>From memory, the general filesystem handling for mkfs'ing, mounting, checking
and other stuff (in the common file scripts) was a bit messy and error prone.
I think extracting out the useful/fs-generic tests might be more valuable too.
It would be nice to see a clear distinction between filesystem specific tests
and generic ones.

--Tim

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Questions about xfstests regarding porting it to test ext4 filesystems

[Dave Chinner]

On Thu, May 14, 2009 at 08:54:04AM -0400, Theodore Ts'o wrote:
> 
> I've been playing around with xfstests (aka xfsqa) and have some
> questions:
> 
> 1)  What is the best mailing list to use for discussions about xfstests? 
> I'm *not* subscribed to xfs@oss.sgi.com, since I'm concerned about
> traffic levels... I'm on too many high-volume mailing lists already.

Just send email here for the moment, no need to subscribe.

I think if we move down the road of making xfstests generic and it
works out for everyone, traffic would move to a more general list
(e.g fsdevel) as more ppl use it.

> 2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
> that when testing UDF and NFS, the scratch volume is used (and $testdir
> is set to the point at the scratch directory).   Is there some
> fundamental reason why there must be an XFS volume mounted, even if the
> fundamental intention is to test some other filesystem type, whether
> it's UDF, NFS, or Ext4?

It is persistent because it ages the filesystem. If you run xfsqa
repeatedly on the same machine (e.g. for months on end), the TESTDIR
gets aged and so we exercise aged filesystems as well as a new fs'
(scratch). There is no reason it really needs to be XFS - it could
be ext4, UDF, etc - as long as it is persistent.

> 3) How much latitude/interest is there in modifying xfstests to be a bit
> more filesystem independent?

Plenty, I think. While it has lots of XFS specific stuff, many of
the tests are generic and have no XFS dependence at all. And many
of the tests that rely on preallocation and block mapping could be
made generic quite easily now. ;)

> I understand the primary purpose of
> xfstests has to be to support XFS development, but looking at the
> scripts, there is a *huge* amount of XFS-specific assumptions all over
> the shell scripts.

A lot of them are relatively easy to fix, I think.

> As a result I'm still of two minds whether it will
> be less work to start from scratch to develop a test suite for ext4 (and
> from the beginning try to make it filesystem independent) or to try to
> hack xfstests and try to make it more filesystem indpendent.  A lot of
> this depends on the time/interest with the xfstests upstream in working
> with me.

I don't see any problem with that - I think xfstests is a better
place to start because of all the knowledge already encoded in
the test suite.

Note that xfsqa also requires some functionality to be implemented
in filesystems to enable more compelte test coverage - e.g.  a
shutdown mechanism so that the test suite can simulate crashes for
it's sync/recovery tests. Ideally, if we are going to make this
a generic test uite, we should also develop generic methods
of doing this sort of shutdown (e.g. via the blockdev) so the
tests will no longer be XFS specific....

> (And assuming we get the licensing issues dealt with --- hence
> my interest and time spent in trying to clarify the licensing situation
> with the fsx program.)   

*nod*

> If there isn't a whole lot of interest in trying to make xfstests more
> portable, that's fine.  It may be that I'm better off starting from
> scratch.  There does seem to be a lot of work and experience represented
> in the xfstest suite, though, so I would like to try exploring the
> possibility of expanding its scope to also support testing other
> filesystems (such as ext4, btrfs, etc.)

Agreed.

The other advantages I see of supporting multiple filesystems in the
one test suite is that we can implement new interfaces in the test
suite and know that they work across multiple filesystems before
integration. i.e. less arguing on mailing lists because working,
useful code can be demonstrated and tested easily.  It would also
mean that new filesystems have something that can be used to test
generic functionality prior to review and merge....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Questions about xfstests regarding porting it to test ext4 filesystems

[Theodore Tso]

On Fri, May 15, 2009 at 08:53:29AM +1000, Dave Chinner wrote:
> > 2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
> > that when testing UDF and NFS, the scratch volume is used (and $testdir
> > is set to the point at the scratch directory).   Is there some
> > fundamental reason why there must be an XFS volume mounted, even if the
> > fundamental intention is to test some other filesystem type, whether
> > it's UDF, NFS, or Ext4?
> 
> It is persistent because it ages the filesystem. If you run xfsqa
> repeatedly on the same machine (e.g. for months on end), the TESTDIR
> gets aged and so we exercise aged filesystems as well as a new fs'
> (scratch). There is no reason it really needs to be XFS - it could
> be ext4, UDF, etc - as long as it is persistent.

At the moment there is a test in common.rc that seems to force that
TEST_DEV is a mounted xfs filesystem:

candygram:/host/usr/projects/e2fsprogs/xfstests# ./check 001
common.rc: Error: $TEST_DEV (/dev/sdb) is not a MOUNTED xfs filesystem
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sdb      ext4    20642428    522392  19071460   3% /test

Sounds like removing that test would be the right thing to do?

> > 3) How much latitude/interest is there in modifying xfstests to be a bit
> > more filesystem independent?
> 
> Plenty, I think. While it has lots of XFS specific stuff, many of
> the tests are generic and have no XFS dependence at all. And many
> of the tests that rely on preallocation and block mapping could be
> made generic quite easily now. ;)

What would people think about taking xfs_io, stripping out the
xfs-specific bits, and creating an fs_testio program that would live
in the xfstests tree?  Then the tests that don't need any of the XFS
bits could simply use fs_testio and not need to build and install
xfsprogs.

Also, maybe we need to have an --disable-xfs option to configure?
Right now for some reason a full make on blows up, probably because
the xfsprogs/xfslibs-dev packages in Ubuntu 9.04 aren't new enough for
whatever xfstests requires:

make[2]: Entering directory `/usr/projects/e2fsprogs/xfstests/src'
/usr/bin/libtool --tag=CC --mode=link gcc loggen.c -o loggen -g -O2 -g -O2 -DDEBUG  -I../include -DVERSION=\"1.0.0\" -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall      
libtool: link: gcc loggen.c -o loggen -g -O2 -g -O2 -DDEBUG -I../include -DVERSION=\"1.0.0\" -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall 
loggen.c: In function 'loggen_unmount':
loggen.c:143: warning: implicit declaration of function 'xlog_assign_lsn'
/tmp/ccey5aXu.o: In function `loggen_empty':
/usr/projects/e2fsprogs/xfstests/src/loggen.c:211: undefined reference to `xlog_assign_lsn'
/usr/projects/e2fsprogs/xfstests/src/loggen.c:263: undefined reference to `xlog_assign_lsn'
/tmp/ccey5aXu.o: In function `loggen_unmount':
/usr/projects/e2fsprogs/xfstests/src/loggen.c:143: undefined reference to `xlog_assign_lsn'
/usr/projects/e2fsprogs/xfstests/src/loggen.c:160: undefined reference to `xlog_assign_lsn'
collect2: ld returned 1 exit status
distcc[9013] ERROR: compile loggen.c on localhost failed

							- Ted

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Questions about xfstests regarding porting it to test ext4 filesystems

[Eric Sandeen]

Theodore Tso wrote:
> On Fri, May 15, 2009 at 08:53:29AM +1000, Dave Chinner wrote:
>>> 2)  Why is the TESTDIR have to be a persistent xfs volume?   I noticed
>>> that when testing UDF and NFS, the scratch volume is used (and $testdir
>>> is set to the point at the scratch directory).   Is there some
>>> fundamental reason why there must be an XFS volume mounted, even if the
>>> fundamental intention is to test some other filesystem type, whether
>>> it's UDF, NFS, or Ext4?
>> It is persistent because it ages the filesystem. If you run xfsqa
>> repeatedly on the same machine (e.g. for months on end), the TESTDIR
>> gets aged and so we exercise aged filesystems as well as a new fs'
>> (scratch). There is no reason it really needs to be XFS - it could
>> be ext4, UDF, etc - as long as it is persistent.
> 
> At the moment there is a test in common.rc that seems to force that
> TEST_DEV is a mounted xfs filesystem:
> 
> candygram:/host/usr/projects/e2fsprogs/xfstests# ./check 001
> common.rc: Error: $TEST_DEV (/dev/sdb) is not a MOUNTED xfs filesystem
> Filesystem    Type   1K-blocks      Used Available Use% Mounted on
> /dev/sdb      ext4    20642428    522392  19071460   3% /test
> 
> Sounds like removing that test would be the right thing to do?

Probably so.  I would hope that tests that require xfs would then be
skipped gracefully ...

>>> 3) How much latitude/interest is there in modifying xfstests to be a bit
>>> more filesystem independent?
>> Plenty, I think. While it has lots of XFS specific stuff, many of
>> the tests are generic and have no XFS dependence at all. And many
>> of the tests that rely on preallocation and block mapping could be
>> made generic quite easily now. ;)
> 
> What would people think about taking xfs_io, stripping out the
> xfs-specific bits, and creating an fs_testio program that would live
> in the xfstests tree?  Then the tests that don't need any of the XFS
> bits could simply use fs_testio and not need to build and install
> xfsprogs.

xfs_io is fairly tangled in with xfsprogs right now I _think_ ... I
dont' see much reason to strip xfs specific stuff out of it, I'd just
make sure the generic functionality is done in a generic way, and keep
adding more.  IOW, don't remove "resvsp" which calls the xfs ioctl, but
do add fallocate.

Installing xfsprogs shouldn't be -too- onerous.... xfs users have been
installing e2fsprogs for years to get libuuid for example.  :)

> Also, maybe we need to have an --disable-xfs option to configure?
> Right now for some reason a full make on blows up, probably because
> the xfsprogs/xfslibs-dev packages in Ubuntu 9.04 aren't new enough for
> whatever xfstests requires:

that was due to some header moves as I said on irc.  --disable-xfs might
be reasonable, to skip building & running the non-xfs tests.

-Eric

> make[2]: Entering directory `/usr/projects/e2fsprogs/xfstests/src'
> /usr/bin/libtool --tag=CC --mode=link gcc loggen.c -o loggen -g -O2 -g -O2 -DDEBUG  -I../include -DVERSION=\"1.0.0\" -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall      
> libtool: link: gcc loggen.c -o loggen -g -O2 -g -O2 -DDEBUG -I../include -DVERSION=\"1.0.0\" -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall 
> loggen.c: In function 'loggen_unmount':
> loggen.c:143: warning: implicit declaration of function 'xlog_assign_lsn'
> /tmp/ccey5aXu.o: In function `loggen_empty':
> /usr/projects/e2fsprogs/xfstests/src/loggen.c:211: undefined reference to `xlog_assign_lsn'
> /usr/projects/e2fsprogs/xfstests/src/loggen.c:263: undefined reference to `xlog_assign_lsn'
> /tmp/ccey5aXu.o: In function `loggen_unmount':
> /usr/projects/e2fsprogs/xfstests/src/loggen.c:143: undefined reference to `xlog_assign_lsn'
> /usr/projects/e2fsprogs/xfstests/src/loggen.c:160: undefined reference to `xlog_assign_lsn'
> collect2: ld returned 1 exit status
> distcc[9013] ERROR: compile loggen.c on localhost failed
> 
> 							- Ted
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs