Re: Submitting patches to xfstests based on OSDI '18 paper (CrashMonkey)

[Dave Chinner <david@fromorbit.com>]

On Sun, Oct 21, 2018 at 10:23:06PM -0500, Jayashree Mohan wrote:
> On Sun, Oct 21, 2018 at 9:49 PM Amir Goldstein <amir73il@gmail.com> wrote:
> 
> > Wait, why _scratch_shutdown and not using dm_flakey?
> > Fewer filesystems support _scratch_shutdown and _scratch_shutdown
> > could be buggy and not simulate crash accurately.
> 
> Thanks for this note. I was looking at previous xfstest tests
> (something not very old - generic/468 for eg), and it was using
> scratch_shutdown. Hence I used that as an example. I'll use
> flakey_drop_and_remount while writing up future patches.
> 
> Doesn't this mean the current tests in xfstest suite might miss bugs
> in current/future kernel versions, because some file systems don't
> support it?

scratch_shutdown is testing shutdown behaviour, whatever the cause.
Not all filesystems have a shutdown mechanism - it's a specific set
of operations that the filesystem performs in response to failure
triggers. Some filesystems just go read-only or have other response
mechanisms, which aren't compatible with expected shutdown
behaviour.

And because shutdowns can be caused by things other than IO errors
(which dm-flakey triggers) and so need to be tested /in addition/ to
other types of failures.

> There are many crash-consistency tests in xfstest suite
> that still use _scratch_shutdown.

Yes, because shutdowns can /simulate/ such failures.

> In fact generic/468 is one of the
> test cases that could not run on btrfs because it does not support
> _shutdown.

Yes, but that doesn't stop us from having the test. If btrfs were to
implement a shutdown mechanism, then they'd gain coverage from these
tests too.

> But then this is the exact test case required to reveal a
> bug in btrfs - where you lose allocated blocks beyond the eof on
> fallocate. Just wanted to bring this up, in case you did not notice it

Then add support for shutdowns to btrfs, and it will get test
coverage from these tests. That's not a problem with the test -
that's a filesystem functionality deficiency. i.e. fix the
filesystem, don't change the test.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com