Re: [patch] fs: improved handling of page and buffer IO errors

[Nick Piggin <npiggin@suse.de>]

On Wed, Oct 22, 2008 at 11:46:24AM -0700, Brad Boyer wrote:
> On Wed, Oct 22, 2008 at 12:31:13PM +0200, Nick Piggin wrote:
> > The problem I've had with testing is that it's hard to trigger a specific
> > path for a given error, because write IO especially can be quite non
> > deterministic, and the filesystem or kernel may give up at various points.
> > 
> > I agree, but I just don't know exactly how they can be turned into
> > standard tests. Some filesystems like XFS seem to completely shut down
> > quite easily on IO errors. Others like ext2 can't really unwind from
> > a failure in a multi-block operation (eg. allocating a block to an
> > inode) if an error is detected, and it just gets ignored.
> > 
> > I am testing, but mainly just random failure injections and seeing if
> > things go bug or go undetected etc.
> 
> Something that might be useful for this kind of testing is a block
> device that is just a map onto a real block device but allows the
> user to configure it to generate various errors. If we could set it
> to always error on either read or write of particular block ranges,
> or randomly choose blocks to error from a pattern it could easily
> trigger many of the error paths. There was something like this on the
> Sega Dreamcast developer kit. The special version of the system used
> by developers had this sort of thing implemented in hardware in the
> link to the optical drive and this made error testing much easier. It
> should be possible to do something similar with a software driver.

We have this failure injection stuff for the block layer, which is
what I've been using. Probably it doesn't quite cover the possible
failure modes of block devices, but it seems to be a reasonable start.

It could use some extending to distinguish reads vs writes, and data
vs metadata, as Dave pointed out.