Unexpected reset corrupted Reiser4 filesystem

Unexpected reset corrupted Reiser4 filesystem

[Andrew James Wade]

Hello,

One of my Reiser4 filesystems was corrupted by a power glitch.
fsck fixed the corruption, but my understanding is that an
unexpected reset should not have corrupted the filesystem. I
have an image of the corrupted filesystem, is it of any use to
anyone?

Details:
kernel: 2.6.12-rc4-mm2
fsck.reiser4: 1.0.4
I was installing oracle, when the power flickered. I was
unable to delete oracle's directory due to what was reported
as I/O errors. fsck revealed a corrupted filesystem (FSCK:
Node (13142228), item (77), unit (0): Points to the block
(12981542) which is in the tree already. The whole subtree
is skipped.) I have an image of the partition at this point,
and dd reported no errors while copying. The image is
unfortunately a bit large to upload (70 GB), but I am happy
to run diagnostic tools against it.

Andrew Wade

Re: Unexpected reset corrupted Reiser4 filesystem

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew James Wade wrote:
> Hello,
> 
> One of my Reiser4 filesystems was corrupted by a power glitch.

No filesystem can prepare for a power glitch, AFAIK.

> fsck fixed the corruption, but my understanding is that an
> unexpected reset should not have corrupted the filesystem. I

It's my understanding that an unexpected _should_ not have corrupted the
filesystem.  Generally, this means that if we have everything working
the way it's supposed to right up until someone hits the Reset switch,
there _should_ be zero corruption.

Caveats:
1.)  _should_ is not _is_.  Just because the developers can't crash it
doesn't mean it's invulnerable.
2.)  power flicker is different than the power/reset button.  It really
all depends on what your hardware actually does when power is cut, but
most hard drives do some sort of write caching, and some of them make it
impossible to turn that off.

To say more, I'd have to know about the physical mechanics involved, but
even if you could make the system absolutely invulnerable to power loss,
you're still going to lose data (not corrupt -- LOSE) in such an event,
unless you have a battery backup.  Even then, your drive is eventually
going to fail -- so use RAID.  And someone is eventually going to rm -rf
your RAID, or spill coffee on it, or hit it with a
tornado/earthquake/nuke, so have multiple sites and make backups.

That's what it all comes down to -- make backups.  The fact that you
have journalling/transactions/fsck/batteries/RAID is all just to make it
a little less catostrophic when stuff does fail.

> have an image of the corrupted filesystem, is it of any use to
> anyone?

Probably someone, not me.  I don't work here, but I'll bet money that
they are going to ask for something from debugfs.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQpQD7ngHNmZLgCUhAQLvSA//bPqBcYMs/NRS7mT//zlGF0WK2YlnNUCd
2hvzXxQhKcLMIvaTo9DJ9wzkaDIHot1DZUcuFIU3f4qmb7QIlMbkVoYE1AW34Wg/
/G06KTkHUi0iU+GJ9Dq65Oj4nK1SM4AtBHMpLqhpcW0FExsj9ScmewcSehFNSzDS
Frptzvk/iWtkip2rJxReF8rKr170Y2aR+1DNaCUiLLoESTgbF5WMmcJFyyuaa0/y
JbP8/kTz4mbLuZuIt6qKswHqUACUA98sK32zeylUOeccKih35FkiaZtegIwti6Op
QwfuCQNyhlODVxAUCghMxz6RtF59Zk4hfoWv5f1GHB7/MHgHGQmMeBy3G/Y/y8D/
P29LVyjhyStVmRyfYtDewMIBQz+nx1+StTqZ3jimfwlpMZxOxeGc0IbV9qiM4o6U
uYgtf9nrdeywn2HruijYO5xiTUcIbaBUGsvCCxNYrdzMYKEDsk3iTR/Jrxl/8GJl
d88NwSAdiU+uUcPkfdlcl+bHk3SB1yX7sjbItOtrUgZxFYsjtuFOusaOo5CkT84l
2lduqioG2u4zWirRk3d3QJEWETykhX4JnjnCjw/qAQ9+JOLc5xjUbgO2fB8d1toy
QKGGVTH1P3i8Ux5ISrtP3noNtA/zzGRlhYShzc50bq8c8+/CAiqnx9FZGv854Ttp
hpTuQ1Ht0FA=
=1k9p
-----END PGP SIGNATURE-----

Re: Unexpected reset corrupted Reiser4 filesystem

[John Dong]

[-- Attachment #1: Type: text/plain, Size: 3173 bytes --]

If thse were IDE drives, the IDE writeback cache is probably the bad boy -- 
on FreeBSD 5.x, Soft Updates is virtually broken on IDE drives because they 
simply haven't written all the data they promised the kernel that they had.

On 5/25/05, David Masover <ninja@slaphack.com> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Andrew James Wade wrote:
> > Hello,
> >
> > One of my Reiser4 filesystems was corrupted by a power glitch.
> 
> No filesystem can prepare for a power glitch, AFAIK.
> 
> > fsck fixed the corruption, but my understanding is that an
> > unexpected reset should not have corrupted the filesystem. I
> 
> It's my understanding that an unexpected _should_ not have corrupted the
> filesystem. Generally, this means that if we have everything working
> the way it's supposed to right up until someone hits the Reset switch,
> there _should_ be zero corruption.
> 
> Caveats:
> 1.) _should_ is not _is_. Just because the developers can't crash it
> doesn't mean it's invulnerable.
> 2.) power flicker is different than the power/reset button. It really
> all depends on what your hardware actually does when power is cut, but
> most hard drives do some sort of write caching, and some of them make it
> impossible to turn that off.
> 
> To say more, I'd have to know about the physical mechanics involved, but
> even if you could make the system absolutely invulnerable to power loss,
> you're still going to lose data (not corrupt -- LOSE) in such an event,
> unless you have a battery backup. Even then, your drive is eventually
> going to fail -- so use RAID. And someone is eventually going to rm -rf
> your RAID, or spill coffee on it, or hit it with a
> tornado/earthquake/nuke, so have multiple sites and make backups.
> 
> That's what it all comes down to -- make backups. The fact that you
> have journalling/transactions/fsck/batteries/RAID is all just to make it
> a little less catostrophic when stuff does fail.
> 
> > have an image of the corrupted filesystem, is it of any use to
> > anyone?
> 
> Probably someone, not me. I don't work here, but I'll bet money that
> they are going to ask for something from debugfs.
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> 
> iQIVAwUBQpQD7ngHNmZLgCUhAQLvSA//bPqBcYMs/NRS7mT//zlGF0WK2YlnNUCd
> 2hvzXxQhKcLMIvaTo9DJ9wzkaDIHot1DZUcuFIU3f4qmb7QIlMbkVoYE1AW34Wg/
> /G06KTkHUi0iU+GJ9Dq65Oj4nK1SM4AtBHMpLqhpcW0FExsj9ScmewcSehFNSzDS
> Frptzvk/iWtkip2rJxReF8rKr170Y2aR+1DNaCUiLLoESTgbF5WMmcJFyyuaa0/y
> JbP8/kTz4mbLuZuIt6qKswHqUACUA98sK32zeylUOeccKih35FkiaZtegIwti6Op
> QwfuCQNyhlODVxAUCghMxz6RtF59Zk4hfoWv5f1GHB7/MHgHGQmMeBy3G/Y/y8D/
> P29LVyjhyStVmRyfYtDewMIBQz+nx1+StTqZ3jimfwlpMZxOxeGc0IbV9qiM4o6U
> uYgtf9nrdeywn2HruijYO5xiTUcIbaBUGsvCCxNYrdzMYKEDsk3iTR/Jrxl/8GJl
> d88NwSAdiU+uUcPkfdlcl+bHk3SB1yX7sjbItOtrUgZxFYsjtuFOusaOo5CkT84l
> 2lduqioG2u4zWirRk3d3QJEWETykhX4JnjnCjw/qAQ9+JOLc5xjUbgO2fB8d1toy
> QKGGVTH1P3i8Ux5ISrtP3noNtA/zzGRlhYShzc50bq8c8+/CAiqnx9FZGv854Ttp
> hpTuQ1Ht0FA=
> =1k9p
> -----END PGP SIGNATURE-----
>

[-- Attachment #2: Type: text/html, Size: 3622 bytes --]

Re: Unexpected reset corrupted Reiser4 filesystem

[Andrew James Wade]

John Dong wrote:
> If thse were IDE drives, the IDE writeback cache is probably the bad boy -- 
> on FreeBSD 5.x, Soft Updates is virtually broken on IDE drives because they 
> simply haven't written all the data they promised the kernel that they had.
I do indeed have an IDE drive (Seagate Barracuda) with a writeback cache.
But I thought that write barriers were now working (by flushing the writeback
cache if the drive doesn't support anything fancier). However, I couldn't find
any updates on the write barrier work since March of last year.
(<http://lwn.net/Articles/77074/>). So the writeback cache may indeed be the
bad boy.

On May 25, 2005 12:49 am, David Masover wrote:
> That's what it all comes down to -- make backups.  The fact that you
> have journalling/transactions/fsck/batteries/RAID is all just to make it
> a little less catostrophic when stuff does fail.
Yup, time for me to make backups. Thanks for the nudge. Especially as I'm
running a bleeding-edge kernel. (I did have one eat some of my data).

Andrew

P.S. My internet connection's been flaky lately, so apologies for any
bounces. I check the mailing list archives for missed messages.