XFStest suite problem

XFStest suite problem

[alexander.tsvetkov]

Hello,

There is a problem found in XFS tests suite: attempt to run XFS tests 
with some tests excluded from run, for example, "./check -g auto -X 231" 
causes the unexpected exclusion of some other tests (for example 
generic/127 in this case). Looking into suite's created exclude list 
indicates that it was incorrectly built by including some garbage 
together with correct test numbers records, so /tmp/30535.xlist looks like:

"generic/#!
generic//bin/bash
generic/#
generic/FSQA
generic/Test
generic/No.
generic/231
generic/#
generic/#
generic/Run
generic/fsx
generic/with
generic/quotas
generic/enabled
generic/and
generic/verify
generic/accounted
generic/quotas
generic/in
generic/the
generic/end
generic/#
generic/#
generic/Derived
generic/from
generic/test
generic/127
generic/#-----------------------------------------------------------------------
generic/#
generic/Copyright
generic/(c)
generic/2006
generic/Silicon
generic/Graphics,
..."

I think the following code of "check" script incorrectly builds this 
exclude list:

"-X) xfile=$2; shift ;
for d in $SRC_GROUPS $FSTYP; do
[ -f $SRC_DIR/$d/$xfile ] || continue
for f in `cat $SRC_DIR/$d/$xfile`; do
echo $d/$f >> $tmp.xlist
done
done
;;"

Used XFStests suite version from 2013-05-22.

Thanks,
Alexander Tsvetkov

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFStest suite problem

[Eric Sandeen]

On 5/30/13 3:07 AM, alexander.tsvetkov@oracle.com wrote:
> I think the following code of "check" script incorrectly builds this exclude list:
> 
> "-X) xfile=$2; shift ;
> for d in $SRC_GROUPS $FSTYP; do
> [ -f $SRC_DIR/$d/$xfile ] || continue
> for f in `cat $SRC_DIR/$d/$xfile`; do
> echo $d/$f >> $tmp.xlist

Well, it is perhaps unexpected, but:

    -X file		exclude individual tests

I think it's designed to actually exclude tests contained in that file, not test numbers.  It's not very obvious.  :(

But from the commit log:

commit 0b1e8abd4a6dbce54bcc5d0467e58966e41e7eb8
Author: Dave Chinner <dchinner@redhat.com>
Date:   Fri Mar 15 12:28:05 2013 +0000

    xfstests: Reintroduce configurable test expunging.
    
    It is definitely handy to be able to disabdle certain tests (e.g. tests that are
    known to hang or crash the test machine on certain kernels), so re-introducing
    the capability of avoiding certain tests just by placing them in a file is
    useful.
    
    Introduce a command line option to specify the expunged file name. The file will
    exist in each tests/* sub-directory so that only the testname is required, and
    can be managed independently. The use of a command line parameter allows
    multiple expunge files to exist simultaneously in the one xfstests tree which
    simplifies management of a source repository used for multiple versions of a
    distro.
    
    Typical usage:
    
    $ cat tests/generic/3.0-stable-avoid
    280
    $ sudo ./check -X 3.0-stable-avoid generic/280
    FSTYP         -- xfs (debug)
    PLATFORM      -- Linux/x86_64 test-1 3.0.39-dgc+
    MKFS_OPTIONS  -- -f -bsize=4096 /dev/vdb
    MOUNT_OPTIONS -- /dev/vdb /mnt/scratch
    
    generic/280     [expunged]
    Passed all 0 tests
    $



_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFStest suite problem

[Dave Chinner]

On Thu, May 30, 2013 at 08:42:29AM -0500, Eric Sandeen wrote:
> On 5/30/13 3:07 AM, alexander.tsvetkov@oracle.com wrote:
> > I think the following code of "check" script incorrectly builds this exclude list:
> > 
> > "-X) xfile=$2; shift ;
> > for d in $SRC_GROUPS $FSTYP; do
> > [ -f $SRC_DIR/$d/$xfile ] || continue
> > for f in `cat $SRC_DIR/$d/$xfile`; do
> > echo $d/$f >> $tmp.xlist
> 
> Well, it is perhaps unexpected, but:
> 
>     -X file		exclude individual tests
> 
> I think it's designed to actually exclude tests contained in that
> file, not test numbers.  It's not very obvious.  :(

Right, it's a file full of test numbers to expunged. Two things can
be done here:

	1. improve the description
	2. change the `cat $file` to `egrep ^$SUPPORTED_TESTS $file`
	   so that it requires files in the same format as group
	   files.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs