Re: [patch] document flash/RAID dangers

[Theodore Tso <tytso@mit.edu>]

On Wed, Aug 26, 2009 at 01:25:36PM +0200, Pavel Machek wrote:
> > you just plain cannot count on writes that are in flight when a powerfail 
> > happens to do predictable things, let alone what you consider sane or  
> > proper.
> 
> From what I see, this kind of failure is rather harder to reproduce
> than the software problems. And at least SGI machines were designed to
> avoid this...
> 
> Anyway, I'd like to hear from ext3 people... what happens on read
> errors in journal? That's what you'd expect to see in situation above.

On a power failure, what normally happens is that the random garbage
gets written into the disk drive's last dying gasp, since the memory
starts going insane and sends garbage to the disk.  So the disk
successfully completes the write, but the sector contains garbage.
Since HDD's tend to be last thing to die, being less sensitive to
voltage drops than the memory or DMA controller, my experience is that
you don't get a read error after the system comes up, you just get
garbage written into the journal.

The ext3 journalling code waits until all of the journal code is
written, and only then writes the commit block.  On restart, we look
for the last valid commit block.  So if the power failure is before we
write the commit block, we replay the journal up until the previous
commit block.  If the power failure is while we are writing the commit
block, garbage will be written out instead of the commit block, and so
it falls back to the previous case.

We do not allow any updates to the filesystem metadata to take place
until the commit block has been written; therefore the filesystem
stays consistent.

If there the journal *does* develop read errors, then fsck will
require a manual fsck, and so the boot operation will get stopped so a
system administrator can provide manual intervention.  The best bet
for the sysadmin is to replay as much of the journal she can, and then
let fsck fix any resulting filesystem inconsistencies.  In practice,
though, I've not experienced or seen any reports of this happening
from a power failure; usually it happens if the laptop gets dropped or
the hard drive suffers or suffers some other kind of hardware failure.

    	       	       	  	       - Ted