public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
* xfstests tests not in the auto group; do we know why?
@ 2008-12-16  5:01 Eric Sandeen
  2008-12-19 21:44 ` Christoph Hellwig
  0 siblings, 1 reply; 6+ messages in thread
From: Eric Sandeen @ 2008-12-16  5:01 UTC (permalink / raw)
  To: xfs-oss

Of the tests that are not in the auto group, do we know why they are not?

022: # Test out a level 0 dump/restore to a tape of a subdir
023: # To test xfsdump/restore to tape using a directory with
024: # Test out incremental dumps
025: # Test dump/restore using -m option (min strategy)
036: # Test xfsdump/restore minrmt to a remote IRIX tape
037: # Test xfsdump/restore minrmt to a remote linux tape
038: # Test xfsdump/restore to a remote linux tape
039: # Test xfsdump/restore to a remote IRIX tape
043: # Test out xfsdump/restore but rmv inventory prior to restore.
055: # Test xfsdump/restore to a remote IRIX tape using RMT user
059: # place holder for IRIX 059 test for xfsdump/xfsrestore multi streams
060: # place holder for IRIX 060 test for xfsdump/xfsrestore multi streams
064: # test multilevel dump and restores with hardlinks
071: # Exercise IO at large file offsets.
080: # rwtest (iogen|doio)
085: # To test log replay by shutdown of file system
086: # To test log replay with version 2 logs
087: # like 086 but want to create more/different kinds of metadata
098: # simple attr tests for EAs:
101: # This tests mkfs_udf with -s [SIZE] option.
102: # This tests mkfs_udf/mkudffs and the device detection code
104: # XFS online growfs-while-allocating tests (data subvol variant)
106: # Exercise basic xfs_quota functionality (user/group/project quota)
107: # Project quota.
108: # Simple quota accounting test for direct/buffered/mmap IO.
109: # ENOSPC deadlock case from Asano Masahiro.
110: # Incorrect dir2 freetab warning case from Masanori Tsuda.
111: # Infinite xfs_bulkstat bad-inode loop case from Roger Willcocks.
113: # aio-stress
114: # Test some parent ptr stuff
115: # Test out xfs_repair_ipaths
116: # Test out resetting of sb_qflags when mounting with no quotas
after having mounted with quotas.
118: # To test out pv#940675 crash in xfs_trans_brelse + quotas
119: # Leaking reservation space in the GRH
133: # Concurrent I/O to same file to ensure no deadlocks
136: # Test the attr2 code


udf tests are probably not auto out of principle? :)
071 fails/hangs on some platforms IIRC
104 hangs ...
"parent" requires code not committed(?)
"tape" group requires... tape so not auto?

# auto - tests to be run as part of nightly qa

I'm not sure what that means; is this group always supposed to pass?  If
so there are filestreams tests that don't, for example.  Maybe "tests
that don't hang?"

I wonder if it'd be worth documenting this a bit, and have a group which
should always run & pass on the core architectures.  (and for those that
don't pass, do a bit of documentation on why they don't?)

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: xfstests tests not in the auto group; do we know why?
  2008-12-16  5:01 xfstests tests not in the auto group; do we know why? Eric Sandeen
@ 2008-12-19 21:44 ` Christoph Hellwig
  2008-12-20  5:10   ` Dave Chinner
                     ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Christoph Hellwig @ 2008-12-19 21:44 UTC (permalink / raw)
  To: Eric Sandeen; +Cc: xfs-oss

On Mon, Dec 15, 2008 at 11:01:10PM -0600, Eric Sandeen wrote:
> Of the tests that are not in the auto group, do we know why they are not?
> 
> 022: # Test out a level 0 dump/restore to a tape of a subdir
> 023: # To test xfsdump/restore to tape using a directory with
> 024: # Test out incremental dumps
> 025: # Test dump/restore using -m option (min strategy)
> 036: # Test xfsdump/restore minrmt to a remote IRIX tape
> 037: # Test xfsdump/restore minrmt to a remote linux tape
> 038: # Test xfsdump/restore to a remote linux tape
> 039: # Test xfsdump/restore to a remote IRIX tape
> 043: # Test out xfsdump/restore but rmv inventory prior to restore.
> 055: # Test xfsdump/restore to a remote IRIX tape using RMT user

all these won't run without a tape, but I don't see any reason not
to put them into the default group.

> 059: # place holder for IRIX 059 test for xfsdump/xfsrestore multi streams
> 060: # place holder for IRIX 060 test for xfsdump/xfsrestore multi streams

These obviously don't matter right now.  Just curious, does anyone know
what the multi-streams were and if there's any chance we might ever seen
them on Linux?

> 080: # rwtest (iogen|doio)

Doesn't run under Linux anyway.  Not sure why.

> 071: # Exercise IO at large file offsets.

Fails for me with a not really large enough FS..

> 064: # test multilevel dump and restores with hardlinks
> 085: # To test log replay by shutdown of file system
> 086: # To test log replay with version 2 logs
> 087: # like 086 but want to create more/different kinds of metadata
> 098: # simple attr tests for EAs:

All these are pretty quick and seem useful.

> 106: # Exercise basic xfs_quota functionality (user/group/project quota)
> 107: # Project quota.
> 108: # Simple quota accounting test for direct/buffered/mmap IO.

We should run all these.  Although 108 currently claims that my kernel
doesn't support project quotas for some reason.

> 109: # ENOSPC deadlock case from Asano Masahiro.
> 110: # Incorrect dir2 freetab warning case from Masanori Tsuda.

These take long time, but seems useful.

> 111: # Infinite xfs_bulkstat bad-inode loop case from Roger Willcocks.

This trips over an assert in xfs_imap_to_bp very quickly for me.
Another one on the todo list..

> 113: # aio-stress

Very quick one, should be default.  Also simply gets skipped without
libaio installed.

> 115: # Test out xfs_repair_ipaths

Well, claims to not run on Linux.  Probably needs parent pointers, too.

> 116: # Test out resetting of sb_qflags when mounting with no quotas
> after having mounted with quotas.
> 118: # To test out pv#940675 crash in xfs_trans_brelse + quotas
> 119: # Leaking reservation space in the GRH

All pretty quick ones, no reason to skip them AFAIK.

> 133: # Concurrent I/O to same file to ensure no deadlocks

Also a nice one.

> 136: # Test the attr2 code

Takes quite long, but seems useful.  And I need to update it for my
latest libxfs resync :)

> udf tests are probably not auto out of principle? :)
> 071 fails/hangs on some platforms IIRC

depends on the size of the filesystem I think.  Shouldn't hang.

> 104 hangs ...

Yeah, we should fix this eventually :) 

> "parent" requires code not committed(?)
> "tape" group requires... tape so not auto?
> 
> # auto - tests to be run as part of nightly qa
> 
> I'm not sure what that means; is this group always supposed to pass?  If
> so there are filestreams tests that don't, for example.  Maybe "tests
> that don't hang?"
> 
> I wonder if it'd be worth documenting this a bit, and have a group which
> should always run & pass on the core architectures.  (and for those that
> don't pass, do a bit of documentation on why they don't?)

I think that would be auto.  I'm all for a slight reshuffling of the
groups:

	auto - stuff that should succeed everywhere
	large - stuff that needs a large enough machine / fs to succeed
		(for whatever defintion of large)
	xfail - expected to fail
	xhang - expected to hang
		(should be empty normally, only for new testcases)

And we might want to document which ones will not be run for some fairly
standard conditions (OS mismatch and lack of tapes are the two thinks I
can think of right now).  Or we make the above check per-OS.  Or should
we just declare IRIX dead officially?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: xfstests tests not in the auto group; do we know why?
  2008-12-19 21:44 ` Christoph Hellwig
@ 2008-12-20  5:10   ` Dave Chinner
  2008-12-20  5:50     ` Eric Sandeen
  2008-12-20  5:52   ` Eric Sandeen
  2008-12-20  6:01   ` Eric Sandeen
  2 siblings, 1 reply; 6+ messages in thread
From: Dave Chinner @ 2008-12-20  5:10 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: Eric Sandeen, xfs-oss

On Fri, Dec 19, 2008 at 04:44:12PM -0500, Christoph Hellwig wrote:
> > 104 hangs ...
> 
> Yeah, we should fix this eventually :) 

Changing the locking in growfs to loop doing trylocks
will prevent the deadlock. Might take a long time to get the
lock though....


> > # auto - tests to be run as part of nightly qa
> > 
> > I'm not sure what that means; is this group always supposed to pass?  If
> > so there are filestreams tests that don't, for example.  Maybe "tests
> > that don't hang?"
> > 
> > I wonder if it'd be worth documenting this a bit, and have a group which
> > should always run & pass on the core architectures.  (and for those that
> > don't pass, do a bit of documentation on why they don't?)
> 
> I think that would be auto.  I'm all for a slight reshuffling of the
> groups:
> 
> 	auto - stuff that should succeed everywhere
> 	large - stuff that needs a large enough machine / fs to succeed
> 		(for whatever defintion of large)
> 	xfail - expected to fail
> 	xhang - expected to hang
> 		(should be empty normally, only for new testcases)

I'd say we should add another:

	fast - tests that complete in only a few seconds

so that we can run a quicker set of sanity checks while developing
stuff. The current auto run takes a couple of hours under UML, which
means a qa cycle doesn't keep up with the rate at which I want to
test new changes......

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: xfstests tests not in the auto group; do we know why?
  2008-12-20  5:10   ` Dave Chinner
@ 2008-12-20  5:50     ` Eric Sandeen
  0 siblings, 0 replies; 6+ messages in thread
From: Eric Sandeen @ 2008-12-20  5:50 UTC (permalink / raw)
  To: Christoph Hellwig, Eric Sandeen, xfs-oss

Dave Chinner wrote:

> I'd say we should add another:
> 
> 	fast - tests that complete in only a few seconds
> 
> so that we can run a quicker set of sanity checks while developing
> stuff. The current auto run takes a couple of hours under UML, which
> means a qa cycle doesn't keep up with the rate at which I want to
> test new changes......

Since the tests keep track of how long they ran last time, maybe we can
make that sort of auto-tuning...?

Perhaps rather than "fast" - "slow" might be better because first, it'd
be fewer to mark, and also I think because of how we invoke & select things,

# ./check -g auto -x slow

would do what you want.

So is there agreement that "auto" by itself should include all tests
which are expected to pass (or not run due to dependencies) reliably?

Thanks,
-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: xfstests tests not in the auto group; do we know why?
  2008-12-19 21:44 ` Christoph Hellwig
  2008-12-20  5:10   ` Dave Chinner
@ 2008-12-20  5:52   ` Eric Sandeen
  2008-12-20  6:01   ` Eric Sandeen
  2 siblings, 0 replies; 6+ messages in thread
From: Eric Sandeen @ 2008-12-20  5:52 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: xfs-oss

Christoph Hellwig wrote:
> On Mon, Dec 15, 2008 at 11:01:10PM -0600, Eric Sandeen wrote:
>> Of the tests that are not in the auto group, do we know why they are not?
>>
>> 022: # Test out a level 0 dump/restore to a tape of a subdir
>> 023: # To test xfsdump/restore to tape using a directory with
>> 024: # Test out incremental dumps
>> 025: # Test dump/restore using -m option (min strategy)
>> 036: # Test xfsdump/restore minrmt to a remote IRIX tape
>> 037: # Test xfsdump/restore minrmt to a remote linux tape
>> 038: # Test xfsdump/restore to a remote linux tape
>> 039: # Test xfsdump/restore to a remote IRIX tape
>> 043: # Test out xfsdump/restore but rmv inventory prior to restore.
>> 055: # Test xfsdump/restore to a remote IRIX tape using RMT user
> 
> all these won't run without a tape, but I don't see any reason not
> to put them into the default group.
> 
>> 059: # place holder for IRIX 059 test for xfsdump/xfsrestore multi streams
>> 060: # place holder for IRIX 060 test for xfsdump/xfsrestore multi streams
> 
> These obviously don't matter right now.  Just curious, does anyone know
> what the multi-streams were and if there's any chance we might ever seen
> them on Linux?
> 
>> 080: # rwtest (iogen|doio)
> 
> Doesn't run under Linux anyway.  Not sure why.
> 
>> 071: # Exercise IO at large file offsets.
> 
> Fails for me with a not really large enough FS..
> 
>> 064: # test multilevel dump and restores with hardlinks
>> 085: # To test log replay by shutdown of file system
>> 086: # To test log replay with version 2 logs
>> 087: # like 086 but want to create more/different kinds of metadata
>> 098: # simple attr tests for EAs:
> 
> All these are pretty quick and seem useful.
> 
>> 106: # Exercise basic xfs_quota functionality (user/group/project quota)
>> 107: # Project quota.
>> 108: # Simple quota accounting test for direct/buffered/mmap IO.
> 
> We should run all these.  Although 108 currently claims that my kernel
> doesn't support project quotas for some reason.
> 
>> 109: # ENOSPC deadlock case from Asano Masahiro.
>> 110: # Incorrect dir2 freetab warning case from Masanori Tsuda.
> 
> These take long time, but seems useful.
> 
>> 111: # Infinite xfs_bulkstat bad-inode loop case from Roger Willcocks.
> 
> This trips over an assert in xfs_imap_to_bp very quickly for me.
> Another one on the todo list..
> 
>> 113: # aio-stress
> 
> Very quick one, should be default.  Also simply gets skipped without
> libaio installed.
> 
>> 115: # Test out xfs_repair_ipaths
> 
> Well, claims to not run on Linux.  Probably needs parent pointers, too.
> 
>> 116: # Test out resetting of sb_qflags when mounting with no quotas
>> after having mounted with quotas.
>> 118: # To test out pv#940675 crash in xfs_trans_brelse + quotas
>> 119: # Leaking reservation space in the GRH
> 
> All pretty quick ones, no reason to skip them AFAIK.
> 
>> 133: # Concurrent I/O to same file to ensure no deadlocks
> 
> Also a nice one.
> 
>> 136: # Test the attr2 code
> 
> Takes quite long, but seems useful.  And I need to update it for my
> latest libxfs resync :)
> 
>> udf tests are probably not auto out of principle? :)
>> 071 fails/hangs on some platforms IIRC
> 
> depends on the size of the filesystem I think.  Shouldn't hang.
> 
>> 104 hangs ...
> 
> Yeah, we should fix this eventually :) 
> 
>> "parent" requires code not committed(?)
>> "tape" group requires... tape so not auto?
>>
>> # auto - tests to be run as part of nightly qa
>>
>> I'm not sure what that means; is this group always supposed to pass?  If
>> so there are filestreams tests that don't, for example.  Maybe "tests
>> that don't hang?"
>>
>> I wonder if it'd be worth documenting this a bit, and have a group which
>> should always run & pass on the core architectures.  (and for those that
>> don't pass, do a bit of documentation on why they don't?)
> 
> I think that would be auto.  I'm all for a slight reshuffling of the
> groups:
> 
> 	auto - stuff that should succeed everywhere
> 	large - stuff that needs a large enough machine / fs to succeed
> 		(for whatever defintion of large)

I'd rather have tests with any significant requirements just test for
those requirements, and [notrun] if they're not present...

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: xfstests tests not in the auto group; do we know why?
  2008-12-19 21:44 ` Christoph Hellwig
  2008-12-20  5:10   ` Dave Chinner
  2008-12-20  5:52   ` Eric Sandeen
@ 2008-12-20  6:01   ` Eric Sandeen
  2 siblings, 0 replies; 6+ messages in thread
From: Eric Sandeen @ 2008-12-20  6:01 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: xfs-oss

Christoph Hellwig wrote:

>> 080: # rwtest (iogen|doio)
> 
> Doesn't run under Linux anyway.  Not sure why.

Heh, that one is funny:

_supported_os IRIX
[ $HOSTOS == IRIX ] && _notrun "Not working on IRIX yet"

Eh?

Anyway, if we add "Linux" to _supported_os and un-comment:

#quiet=-q

it seems to pass fine on Linux.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2008-12-20  6:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-12-16  5:01 xfstests tests not in the auto group; do we know why? Eric Sandeen
2008-12-19 21:44 ` Christoph Hellwig
2008-12-20  5:10   ` Dave Chinner
2008-12-20  5:50     ` Eric Sandeen
2008-12-20  5:52   ` Eric Sandeen
2008-12-20  6:01   ` Eric Sandeen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox