From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Elder Subject: Re: [LSF/MM TOPIC] [ATTEND] xfstests: what do we need to do to make it better? Date: Wed, 04 Jan 2012 11:18:11 -0600 Message-ID: <1325697491.3346.18.camel@doink> References: <20120103234455.GU23662@dastard> Reply-To: elder@dreamhost.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com To: Dave Chinner Return-path: Received: from mail.hq.newdream.net ([66.33.206.127]:57472 "EHLO mail.hq.newdream.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750806Ab2ADRSO (ORCPT ); Wed, 4 Jan 2012 12:18:14 -0500 In-Reply-To: <20120103234455.GU23662@dastard> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Wed, 2012-01-04 at 10:44 +1100, Dave Chinner wrote: > Given that more people are using xfstests and developing tests, we > need to consider how to make it friendlier to hack on. The current > structure of the tree is difficult to work with, the way tests are > organised and numbered make it difficult to co-ordinate new tests > and results in patch conflicts, etc. Coordination of numbers is not a big deal, the test names/numbers can be easily fixed up at commit time. I also thought that the numbers--though meaningless on their own--also avoided having to decide where a particular test belongs. I.e., a test that exercises several categories of things (maybe preallocation, quota, and ENOSPC) won't be hidden in any sort of "enospc" test directory. I do think the growing number of tests is making it a bit unwieldy though, so I think some sort of reorganization is a good plan. > We also see problems arising from people not really understanding how > the xfstests harness is designed and how it really is supposed to > work, so an overview of the underlying principles of operation would > probably be helpful to a lot of people. It will also save > review and rework time if we can avoid having people make the same > mistakes the first time they submit tests.... This is very important. And the gist of it ought to be captured somewhere if it is not already. > I'd also like to discuss some potential infrastructure changes to > make it easier to add new tests without conflicts with others > developing new tests. Some of the ideas Christoph and I have > previously tossed around include: > > - break tests up into groups in their own subdirectories. > e.g. generic tests, xfs/ext4/btrfs specific tests, stress > tests, performance tests, large FS tests, etc > - change the way we define groups of tests so we don't have > a single registry of tests and their groups > - allow different naming of tests, such as desciptive text > names rather than just plain numbers > - allow duplicate test names in different groups Despite what I said above, I don't disagree with any of this. Perhaps the tests can be buried in one or more subdirectories, but each FSTYP defines its own groups file to drive testing. > I'm sure that other users of xfstests will have some ideas on how to > improve it for the way they run it, so I'd like to gather and > incorporate these ideas into any structural change we make to > xfstests. Should be a good discussion. It might be useful to have a proposal or two to work with as a starting point, or maybe an outline of the types of changes (naming, directory structure, etc.), to help keep things focused. -Alex > Cheers, > > Dave.