Murphy hits (Kernel 2.6, ext2, "check=strict"): corrupted filesystem

Murphy hits (Kernel 2.6, ext2, "check=strict"): corrupted filesystem

[Ulrich Windl]

Hello everybody,

I'd like to present a little story how to shredder your ext2 filesystem:

I was installing SuSE Linux 9.1 when the kernel froze rather late during 
installation. So I had to reset the PC. There is a minor bug in the forementioned 
distribution that prevents a filesystem check of the root filesystem after such a 
crash it seems. Anyway, the system booted without a fsck being run. I realized 
that and shut down (maybe it would have been better to hit the reset button 
immediately), but during the manual fsck, the system reported (among a lot of 
others) about 1700 inodes that share blocks. You can imagine what that means...

Why I'm writing this: If something can go wrong, eventually it will. For a true 
disaster you always need more than just one problem (1: Kernel freeze, 2: no fsck 
being run, 3: kernel happily mounts unclean filesystem for read-write).

In Kernel 2.2 there was a "check=strict" that caused some extra checks during 
mounting a filesystem. In 2.4 the option was ignored, and a warning was presented 
instead. In 2.6 it seems that "check=strict" prevents the filesystem from being 
mounted read-write (I ended up with an read-only root filesystem when having that 
option).

I'll have to spend a few more days to cleanup the mess in the root filesystem 
(which is everything except /home and /boot), but the lession to learn might be 
this: The kernel should not mount a unclean filesystem read-write. Maybe also if a 
filesystem code results in an attempt to access a partition outside its limits, 
the filesystem should trtigger a panic immediately instead of going on.

I think nobody really wants to read reports where Linux has shreddered a 
filesystem, do we?

Regards,
Ulrich

Re: Murphy hits (Kernel 2.6, ext2, "check=strict"): corrupted filesystem

[Pavel Machek]

Hi!

> I'd like to present a little story how to shredder your ext2 filesystem:
> 
> I was installing SuSE Linux 9.1 when the kernel froze rather late during 
> installation. So I had to reset the PC. There is a minor bug in the forementioned 

You call this "minor"?

> Why I'm writing this: If something can go wrong, eventually it will. For a true 
> disaster you always need more than just one problem (1: Kernel freeze, 2: no fsck 
> being run, 3: kernel happily mounts unclean filesystem for read-write).

3 is feature. It prints warning, but lets you mount it. I sometimes mount
broken fs's rw; it actually saved me once when I was hitting fsck bug.
It is also handy when quickly recovering scratch machine.

MS-DOS had no fsck... and survive. ext2 can survive with similar results
if you just dont fsck...

> I think nobody really wants to read reports where Linux has shreddered a 
> filesystem, do we?

I actually liked your report ;-).
				Pavel
-- 
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         

Re: Murphy hits (Kernel 2.6, ext2, "check=strict"): corrupted filesystem

[Ulrich Windl]

On 13 Jul 2004 at 22:48, Pavel Machek wrote:

> Hi!
> 
> > I'd like to present a little story how to shredder your ext2 filesystem:
> > 
> > I was installing SuSE Linux 9.1 when the kernel froze rather late during 
> > installation. So I had to reset the PC. There is a minor bug in the forementioned 
> 
> You call this "minor"?

...in the eyes of real-life (Linux distributor's) software support it is. Not in 
mine.

> 
> > Why I'm writing this: If something can go wrong, eventually it will. For a true 
> > disaster you always need more than just one problem (1: Kernel freeze, 2: no fsck 
> > being run, 3: kernel happily mounts unclean filesystem for read-write).
> 
> 3 is feature. It prints warning, but lets you mount it. I sometimes mount
> broken fs's rw; it actually saved me once when I was hitting fsck bug.
> It is also handy when quickly recovering scratch machine.
> 
> MS-DOS had no fsck... and survive. ext2 can survive with similar results
> if you just dont fsck...

The syslog was full with attempts to access the device outside its limits. Seems 
there was some  bad constellation after the kernel crash.

> 
> > I think nobody really wants to read reports where Linux has shreddered a 
> > filesystem, do we?
> 
> I actually liked your report ;-).

(after two days of work and reinstalling about 90% of the RPMs I have a working 
system again. The other stupid thing that doesn't belong to this list is: RPM v4 
fails to install a package if one of the packet's file targets exists as a 
directory (happened after the crash and fsck to me). RPM then seems unable to 
cleanup and leaves aditional files with a semicolon and hexadecimal number as 
suffix then. Repeated invocations create new files then...)

Regards,
Ulrich
(& thanks for replying)