Re: [PATCH] ext4: add fallocate mode blocking for debugging purposes

Re: [PATCH] ext4: add fallocate mode blocking for debugging purposes

[Dave Chinner]

[added xfs@oss.sgi.com]

On Tue, Apr 15, 2014 at 07:30:39PM -0400, Theodore Ts'o wrote:
> On Tue, Apr 15, 2014 at 05:32:43PM -0500, Eric Sandeen wrote:
> > > I also had a sneaking suspicion that we might have a similar issue
> > > with the INSERT RANGE patches which are coming down the pike, and so
> > > having a general way of also being able INSERT RANGE if to be able to
> > > quickly determine whether a potential bug was caused by INSERT RANGE
> > > or some other pending changes might also be useful.
> > 
> > Also: I'd humbly suggest just not merging those until they pass stringent
> > tests like fsx & fsstress...
> > 
> > Adding a pre-emptive knob to turn them off post-merge when they turn
> > out to be broken sounds backwards to me...
> 
> Having learned from COLLAPSE RANGE, I agree.  The fact that we didn't
> have full testing during the whole development cycle was unfortunate.
> And we got lucky with the renameat patches, since I wasn't able to get
> tha testing done because the xfstests commits didn't get merged until
> *after* the they renameat commits got merged, and also because I
> didn't notice that the i386 system call wasn't wired up when I was
> doing my manual "just before I push to Linus" testing.

I asked for renameat2 tests long before inclusion occurred. The fact
is that we can't co-ordinate xfstests inclusion for a feature that
we don't even know is going to be included until someone sends Linus
a pull request....

> I plan on insisting that INSERT RANGE support being in the VFS, and be
> fully enabled, and that we have full INSERT RANGE testing into
> xfstests, during the development cycle.

There wasn't a problem with the timing of xfstests inclusion - the
problem was with the fact we didn't have sufficient QA coverage in
xfstests when the initial upstream kernel commits occurred.  This
time around, the difference will be that this time we'll have fsx
and fsstress coverage *before* kernel support is added, and I've
already asked for that:

http://oss.sgi.com/archives/xfs/2014-04/msg00121.html

> Some of the work that I've
> been doing with kvm-xfstests and why I created a github tytso/xfstests
> git tree is specifically to make sure things go much more smoothly
> this time around.

Ted, this looks and sounds like you're preparing to fork xfstests.
Why?  What's the problem with working upstream on test development
and refinement like everyone else does?

This thread is a demonstration of how avoiding upstream interaction
results in nasty hacks being proposed. If you asked the question on
the xfs mailing list of how to avoid various fsstress/fsx
operations, we woul dhave told you that using FSSTRESS_AVOID and
adding an equivalent FSX_AVOID to xfstests is all that is needed.
i.e. we already have partial infrastructure support for what you
need in xfstests and it would be about 30 minutes work to add
FSX_AVOID....

Is that fast enough for you?

Indeed, we could also use similar env vars to ensure various
_requires_* checks fail and to populate FSSTRESS_AVOID/FSX_AVOID
automatically and so tests that require this functionality are not
run.

IOWs, it's in your best interests to work with upstream to add the
functionality you require to xfstests. History tells us that forking
development into private repositories has never worked out well for
anyone, so I'd really, really like you to *at least try* to work
with upstream as your primary test development environment....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH] ext4: add fallocate mode blocking for debugging purposes

[Theodore Ts'o]

On Wed, Apr 16, 2014 at 10:06:35AM +1000, Dave Chinner wrote:
> > Some of the work that I've
> > been doing with kvm-xfstests and why I created a github tytso/xfstests
> > git tree is specifically to make sure things go much more smoothly
> > this time around.
> 
> Ted, this looks and sounds like you're preparing to fork xfstests.
> Why?  What's the problem with working upstream on test development
> and refinement like everyone else does?

I'd prefer not to fork xfstests.  However, I do want to get more ext4
developers using automated xfstests testing, so I can scale better.
In order to do that, I need to be able to make it really easy for
people to who aren't hard-core xfstests people to be able to use it.

One of the nice things about kvm-xfstests is that a *lot* easier for
people to figure out how to use it.  If I can lower the activation
energy required to get people to use xfstests, it saves me time in the
end.

The reason why I created the github repository is because if I'm going
to be shipping a KVM test appliance image that people can use in a
turn-key environment, I'd prefer that all of the sources, including
any local changes that I might need to make the tests run as smoothly
as possible, are available in a public repository.  (And at one point,
I did have up to 12 local changes, which is why I wanted it tracked in
a repo.)

Every single local change I made was either a test or commit that
hadn't been accepted into the upstream xfstests repository yet, or a
fix I wrote that I sent upstream.  And as soon as the fixes made it
into the upstream xfstests repository, I rebased them away.  At the
moment, there's only once commit in my xfstests github repository
which isn't upstream and it's the:

  check: add support for an external file containing tests to exclude

commit for which I've sent the V2 version to you.

So for the most part, I want to keep the repo as close to upstream as
possible, and ideally identical to upstream, and I've been working
towards that end.

> This thread is a demonstration of how avoiding upstream interaction
> results in nasty hacks being proposed. If you asked the question on
> the xfs mailing list of how to avoid various fsstress/fsx
> operations, we woul dhave told you that using FSSTRESS_AVOID and
> adding an equivalent FSX_AVOID to xfstests is all that is needed.
> i.e. we already have partial infrastructure support for what you
> need in xfstests and it would be about 30 minutes work to add
> FSX_AVOID....
> 
> Is that fast enough for you?
> 
> Indeed, we could also use similar env vars to ensure various
> _requires_* checks fail and to populate FSSTRESS_AVOID/FSX_AVOID
> automatically and so tests that require this functionality are not
> run.

Well, it took me about 1 minute to write the dozen line kernel patch.
I really didn't want to ask you to make changes to xfstests for me,
but if you're willing to make those changes, that would be great.  I
really didn't want to presume, though.  And if the answer is that I
need to spend the time making all of these changes --- I'll try, but
if I don't have time, I may end up taking the more expedient path.

> IOWs, it's in your best interests to work with upstream to add the
> functionality you require to xfstests. History tells us that forking
> development into private repositories has never worked out well for
> anyone, so I'd really, really like you to *at least try* to work
> with upstream as your primary test development environment....

As I said, every single patch which I put in my local xfstests tree I
also sent upstream.

That being said, I wasn't sure whether you were going to accept that
last change, since there was similar, but for me, not usable
functionality in the form of the -X option.  So if you weren't going
to accept a change to allow the excluded list of tests to be kept in a
single file outside of the tests/* subdirectory, I probably would have
carried it as a separate patch --- because it's something I need, and
the current -X functionality really isn't easy to maintain (you need
to have many more files, and they have to be dropped into the xfstests
tests/* subdirectory).

I know that you and I haven't seen eye to eye in the past.  For
example, the NO_HIDE_STALE out of tree patch which is running on
thousands and thousands numbers of machines inside Google, but which
the XFS folks have considered evil incarnate.  I will freely admit
that I'm much more of a pragmatist and much less of a purist on
certain matters.

So sure, I'm certainly going to _try_ to work with upstream xfstests.
I've done that to date.  But I'm certainly not going to presume that
you're going to like or accept all of the changes I might want to
propose.

Regards,

						- Ted

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs