2.4.19-pre7 / corruption on unwanted reboot

2.4.19-pre7 / corruption on unwanted reboot

[Dirk Mueller]

Hi, 

I've seen HEAVY file corruption on unwanted reboots (like pressing the reset 
button accidently) on reiserfs with this kernel on 3 machines now. 

The symptom is that it finds a LOT of files to "unlink" on journal replay, 
which I find suspicious as those machines are lightly loaded. 

I didn't follow the development too closely the last few weeks, but I 
believe that something turned worse in this respect lately. 

Note that reiserfsck doesn't find any error in the file system structure 
before and after the journal replay on reboot, 
still many files (especially those that were not touched for several hours 
before the reboot) contain complete garbage after the journal replay. 


Dirk

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Chris Mason]

On Sat, 2002-05-04 at 17:11, Dirk Mueller wrote:
> Hi, 
> 
> I've seen HEAVY file corruption on unwanted reboots (like pressing the reset 
> button accidently) on reiserfs with this kernel on 3 machines now. 
> 
> The symptom is that it finds a LOT of files to "unlink" on journal replay, 
> which I find suspicious as those machines are lightly loaded. 
> 
> I didn't follow the development too closely the last few weeks, but I 
> believe that something turned worse in this respect lately. 
> 
> Note that reiserfsck doesn't find any error in the file system structure 
> before and after the journal replay on reboot, 
> still many files (especially those that were not touched for several hours 
> before the reboot) contain complete garbage after the journal replay. 

Hmmm, not good at all.  Are these 3 systems IDE or scsi?  Do they run
additional patches on top of pre7?  What kernels < pre7 have you tried
that didn't show this problem?

-chris

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Oleg Drokin]

Hello!

On Sat, May 04, 2002 at 11:11:47PM +0200, Dirk Mueller wrote:

> I've seen HEAVY file corruption on unwanted reboots (like pressing the reset 
> button accidently) on reiserfs with this kernel on 3 machines now. 

This is bad. Does any earlier (but still relatively recent) kernel excibits
this same behavior?

> The symptom is that it finds a LOT of files to "unlink" on journal replay, 
> which I find suspicious as those machines are lightly loaded. 

How much is this "a LOT"? I think that even 10+ files that are unlinked
after the unclean reboot/crash is pretty normal. Programs tend to use
deleted files after the deletion, so this number highly depends
on kind of programs you usually run.

> I didn't follow the development too closely the last few weeks, but I 
> believe that something turned worse in this respect lately. 

In fact nothing unexpected went into 2.4.19-pre7, that was not available in
our pending patches directory on our ftp site.

> Note that reiserfsck doesn't find any error in the file system structure 
> before and after the journal replay on reboot, 
> still many files (especially those that were not touched for several hours 
> before the reboot) contain complete garbage after the journal replay. 

I sthere any other pattern for damaged files besides these were not touched for
several hours prior to reboot?
Do their mtimes confirm they were modified long ago?

Thanks for the report.

Bye,
    Oleg

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Hans Reiser]

Dirk Mueller wrote:

>Hi, 
>
>I've seen HEAVY file corruption on unwanted reboots (like pressing the reset 
>button accidently) on reiserfs with this kernel on 3 machines now. 
>
>The symptom is that it finds a LOT of files to "unlink" on journal replay, 
>which I find suspicious as those machines are lightly loaded. 
>
>I didn't follow the development too closely the last few weeks, but I 
>believe that something turned worse in this respect lately. 
>
>Note that reiserfsck doesn't find any error in the file system structure 
>before and after the journal replay on reboot, 
>still many files (especially those that were not touched for several hours 
>before the reboot) contain complete garbage after the journal replay. 
>
>
>Dirk
>
>
>  
>
Were these files being written to near the time of the reboot?

hans

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Dirk Mueller]

On Sam, 04 Mai 2002, Chris Mason wrote:

> Hmmm, not good at all.  Are these 3 systems IDE or scsi?  Do they run
> additional patches on top of pre7?  What kernels < pre7 have you tried
> that didn't show this problem?

All IDE. The kernel that didn't show this problem was 2.4.16 (plain). No 
additional patches on 2.4.19-pre7. 


Dirk

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Chris Mason]

On Mon, 2002-05-06 at 08:36, Dirk Mueller wrote:
> On Sam, 04 Mai 2002, Chris Mason wrote:

[ reiserfs corruption after a crash, 2.4.19pre7 ]

> 
> > Hmmm, not good at all.  Are these 3 systems IDE or scsi?  Do they run
> > additional patches on top of pre7?  What kernels < pre7 have you tried
> > that didn't show this problem?
> 
> All IDE. The kernel that didn't show this problem was 2.4.16 (plain). No 
> additional patches on 2.4.19-pre7. 

Please tell us everything about your IDE config.  Jens and I are already
trying to track down some odd reiserfs + ide problems on 2.4.19pre7, but
so far that was only with our barrier write patches applied.

-chris

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Dirk Mueller]

On Mon, 06 Mai 2002, Chris Mason wrote:

> Please tell us everything about your IDE config.  Jens and I are already
> trying to track down some odd reiserfs + ide problems on 2.4.19pre7, but
> so far that was only with our barrier write patches applied.

There is not much common. two of them are VIA 686 southbridge (KT133A, 
KT333), one is something older, a Pentium chipset. 

DMA 100 and DMA 66. We all use those Maxtor 80GB EIDE disks. 


Dirk

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Chris Mason]

On Mon, 2002-05-06 at 09:59, Dirk Mueller wrote:
> On Mon, 06 Mai 2002, Chris Mason wrote:
> 
> > Please tell us everything about your IDE config.  Jens and I are already
> > trying to track down some odd reiserfs + ide problems on 2.4.19pre7, but
> > so far that was only with our barrier write patches applied.
> 
> There is not much common. two of them are VIA 686 southbridge (KT133A, 
> KT333), one is something older, a Pentium chipset. 
> 
> DMA 100 and DMA 66. We all use those Maxtor 80GB EIDE disks. 

Any suggestions on how I might reproduce locally?

-chris

Re: 2.4.19-pre7 / corruption on unwanted reboot

[Dirk Mueller]

On Mon, 06 Mai 2002, Chris Mason wrote:

> Any suggestions on how I might reproduce locally?

not much. maybe try a lot of open, unlinked files when pressing reset and 
then check the md5sum's of all files..


Dirk