How to work around bad blocks?

How to work around bad blocks?

[Dirk Leber]

Hello List,
I am the unhappy owner of a "corrupt" ReiserFS. All the other partitions on 
the same disk work fine, just this one seems to have some bad blocks. HEre 
are some outputs:

> mount /dev/hda4 /mnt/data2/
mount: wrong fs type, bad option, bad superblock on /dev/hda4,
       or too many mounted file systems

> reiserfsck --check
[no problem]

> reiserfsck --fix-fixable /dev/hda4
reiserfsck 3.6.2 (2002)
Will check consistency of the filesystem on /dev/hda4
Will fix what can be fixed w/o --rebuild-tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes):Yes
###########
reiserfsck --fix-fixable started at Mon Mar 10 20:50:03 2003
###########

bread: Cannot read a block # 7503872.


> badblocks -b 4096
[detects bad blocks 7503858 - 7503871]

Obviously the build-tree option also fails. The "How Reiser handles Bad 
blocks-FAQ" can only be considered a bad joke, since you need a degree in 
computer science AND a mountable filesystem for fooling the FS in that way. 

What I would like to do is, detecting the bad blocks, freeing them (accepting 
there content) rebuilding the tree (or not) and than mounting it while 
accepting that some files will be corrupt. 

What can I do? 

Dirk 

Re: How to work around bad blocks?

[Anders Widman]

> Hello List,
> I am the unhappy owner of a "corrupt" ReiserFS. All the other partitions on
> the same disk work fine, just this one seems to have some bad blocks. HEre
> are some outputs:

<snip>

>> badblocks -b 4096
> [detects bad blocks 7503858 - 7503871]

> Obviously the build-tree option also fails. The "How Reiser handles Bad
> blocks-FAQ" can only be considered a bad joke, since you need a degree in
> computer science AND a mountable filesystem for fooling the FS in that way.

Agreed  that  handling  of  bad  blocks  in  most FS in Linux is quite
annoying and far from optimal.

> What I would like to do is, detecting the bad blocks, freeing them (accepting
> there content) rebuilding the tree (or not) and than mounting it while
> accepting that some files will be corrupt. 

> What can I do? 

Right  now  the easiest way it to get a new drive and use dd_rescue to
move  the data. dd_rescue can copy the contents from your bad drive to
a file and run fsck on the file and "mount -o loop".

//Anders

--------
PGP public key: https://tnonline.net/secure/pgp_key.txt

power loss -> corruption

[Mirar]

I've had this problem for a while. Every time the system gets a power
loss, all files that was recently written to gets garbage in them. 
(It seems to me like the wrong content gets in the wrong files.)

I assume it's not supposed to be that way.

Is this something known? 

It happens with 2.4.19 and 2.4.20 kernel versions. 
Is there anything I can do to track down this issue?

Since it's every time, or at least close to every time...

I don't power down by breaking that power that often, so it hasn't
disturbed me that much, but I lost an important file today so I guess
I'm extra aggravated.

/Mirar

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 13:18 schrieb Mirar:
> I've had this problem for a while. Every time the system gets a power
> loss, all files that was recently written to gets garbage in them.
> (It seems to me like the wrong content gets in the wrong files.)
>
> I assume it's not supposed to be that way.
>
> Is this something known?

Maybe, if you have some more details.

> It happens with 2.4.19 and 2.4.20 kernel versions.
> Is there anything I can do to track down this issue?

data-logging patches and mount option "data=ordered" solved similar problems 
for me.

Regards,
	Dieter

Re: power loss -> corruption

[Mirar]

> Maybe, if you have some more details.

How do I get the necessary details?

> data-logging patches and mount option "data=ordered" solved similar
> problems for me.

Which patches? I'll add "data=ordered" to see if it'll help.

/Mirar

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 14:06 schrieb Mirar:
> > Maybe, if you have some more details.
>
> How do I get the necessary details?

Maybe you can preset some corrupted file snippets?

-Dieter

Re: power loss -> corruption

[Mirar]

> Maybe you can preset some corrupted file snippets?

I'll see if I can dig up any good. 

But I can give examples, for instance .zsh-history with 65536 bytes of
illicit data inserted in the middle - 65536 bytes of which 63898 comes
from a gtk-gnutella file (downloads, I think) and the remaining 1638
is NUL characters. Both files were corrupted after restart, but
gtk-gnutella ate it's file so I can't check how it looked.

Lots of things like that. The numbers are so even so I would figure
that the wrong blocks get in the wrong files somehow.

/Mirar

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 15:19 schrieb Mirar:
> > Maybe you can preset some corrupted file snippets?
>
> I'll see if I can dig up any good.

OK, I think it is not needed ;-)

> But I can give examples, for instance .zsh-history with 65536 bytes of
> illicit data inserted in the middle - 65536 bytes of which 63898 comes
> from a gtk-gnutella file (downloads, I think) and the remaining 1638
> is NUL characters. Both files were corrupted after restart, but
> gtk-gnutella ate it's file so I can't check how it looked.
> Lots of things like that. The numbers are so even so I would figure
> that the wrong blocks get in the wrong files somehow.

This _is_ very well known for me.
Some KDE config files and my tcsh history file where broken without 
data-logging (data=journal/ordered).

ReiserFS 3.5/3.6 offer metadata journaling.
For data journaling you _need_ data-logging.

See Chris latest announcement "new data logging patches available".
ftp.suse.com/pub/people/mason/patches/data-logging/2.4.21

Regards,
	Dieter

Re: power loss -> corruption

[Mirar]

> This _is_ very well known for me.
> Some KDE config files and my tcsh history file where broken without 
> data-logging (data=journal/ordered).

My .zsh-history just got nuked with data=ordered. 

The kernel couldn't agree with my HPT controller and stopped all (!)
IDE activities when I did a 'hdparm -y /dev/hd?' (all sleep). (Note,
the .zsh-history was not on that controller.)

Not in the same way as previously (just a block), though. It was
completely filled with binary data.

No other active files seems to have been affected, though.

/Mirar

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 15. Mai 2003 10:40 schrieb Mirar:
> > This _is_ very well known for me.
> > Some KDE config files and my tcsh history file where broken without
> > data-logging (data=journal/ordered).
>
> My .zsh-history just got nuked with data=ordered.
>
> The kernel couldn't agree with my HPT controller and stopped all (!)
> IDE activities when I did a 'hdparm -y /dev/hd?' (all sleep). (Note,
> the .zsh-history was not on that controller.)

In short, DON'T DO THAT without "suspend kernel patches".
*nix is NOT designed for "disk power management" in the first place.
But it evolves...;-)

HPT ("small" IDE/ATA "RAID") controllers are not that good (supported) as they 
seems. Apart that they are "cheap".

> Not in the same way as previously (just a block), though. It was
> completely filled with binary data.

Sounds like another problem (hardware, see above).

> No other active files seems to have been affected, though.

Good.

-Dieter

Re: power loss -> corruption

[Mirar]

> In short, DON'T DO THAT without "suspend kernel patches". *nix is
> NOT designed for "disk power management" in the first place. But it
> evolves...;-)

Never had a trouble with it before. :)

I have them regularly spin down using hdparm -S, and never had a
problem before... This time I was tracking down a noise, which turned
out to be air in the waterpump.

> HPT ("small" IDE/ATA "RAID") controllers are not that good
> (supported) as they seems. Apart that they are "cheap".

Nope. It's always been a headache. Never behaved *this* bad though.
I only have my spare data disks (80+120Gb) on it though.

> > Not in the same way as previously (just a block), though. It was
> > completely filled with binary data.
> 
> Sounds like another problem (hardware, see above).

Not the same controller. The root/home disk is on the main IDE controller 
(whatever's on VIA KT266). So it seems like a sync issue of some sort.

/Mirar

Re: power loss -> corruption

[Hans Reiser]

You have run into the disadvantage of meta-data journaling, which is 
that files that you are writing to may be corrupted if you crash.  This 
disadvantage exists for performance reasons: using traditional 
journaling approaches such as V3 uses, if you journal the data you must 
write it twice.  V3 also offers you the option of performing ordered 
writes, in which, if I remember right, metadata is only updated after 
the data reaches disk.

V4 uses a completely different technology we call wandering logs, in 
which every filesystem operation is performed as a fully atomic 
transaction, and because of its new design, there is very little 
performance cost (in fact, reiser4 is higher performance than V3).  We 
hope V4 will be ready for prime time sometime this summer.

-- 
Hans

Re: power loss -> corruption

[Mirar]

Hans Reiser <reiser@namesys.com> writes:

> You have run into the disadvantage of meta-data journaling, which is
> that files that you are writing to may be corrupted if you crash.

Hmm, I guess the main problem here is the atomicity? 
I haven't studied traditional journaling very much. I somehow find it
odd that the wrong block can get in the wrong files... shouldn't at
least that be in the metadata?

(The only thing I know with journaling is an experiment of a low-level
database that I wrote, with atomic transactions and safe-tolerance.)

> This disadvantage exists for performance reasons: using traditional
> journaling approaches such as V3 uses, if you journal the data you
> must write it twice.  V3 also offers you the option of performing
> ordered writes, in which, if I remember right, metadata is only
> updated after the data reaches disk.

Any performance loss or data risk with using ordered?

> V4 uses a completely different technology we call wandering logs, in
> which every filesystem operation is performed as a fully atomic
> transaction, and because of its new design, there is very little
> performance cost (in fact, reiser4 is higher performance than V3).  We
> hope V4 will be ready for prime time sometime this summer.

I think V4 sounds excellent, and I hope it will be incorporated in
Linux 2.6/3.0. I assume there isn't any kernal patch for 2.4.20 around
so I can play with it? I'm sorry if it has been mentioned on the list,
I'm not up to date.

/Mirar

Re: power loss -> corruption

[Hans Reiser]

Mirar wrote:

>I think V4 sounds excellent, and I hope it will be incorporated in
>Linux 2.6/3.0. I assume there isn't any kernal patch for 2.4.20 around
>so I can play with it? I'm sorry if it has been mentioned on the list,
>I'm not up to date.
>
>/Mirar
>
>
>  
>
Please wait until July,  when we hope to have a beta-release.

-- 
Hans

Re: power loss -> corruption

[Hans Reiser]

Mirar wrote:

>Hans Reiser <reiser@namesys.com> writes:
>
>  
>
>>You have run into the disadvantage of meta-data journaling, which is
>>that files that you are writing to may be corrupted if you crash.
>>    
>>
>
>Hmm, I guess the main problem here is the atomicity? 
>I haven't studied traditional journaling very much. I somehow find it
>odd that the wrong block can get in the wrong files... shouldn't at
>least that be in the metadata?
>
Hmm, the right block got into the file, but the contents of that right 
block never reached disk because of the crash, does that make it easier 
to understand the problem?;-)


-- 
Hans

Re: power loss -> corruption

[Mirar]

> Hmm, the right block got into the file, but the contents of that right
> block never reached disk because of the crash, does that make it
> easier to understand the problem?;-)

Ah yes, that makes sense. :)

I would suggest that checksums could be used to see if the right
blocks hit the disk when recapping the journal, but since V4 is on the
way anyway...

/Mirar

RE: power loss -> corruption

[Bill Rees]

I have seen this as well. Recently accessed or edited files become corrupt
when the system kernel panics and halts as well. For instance, I will edit
/etc/inittab, a kernel panic occurs and when the system attempts to reboot
it fails because the inittab contains unreadable garbage.

Bill

-----Original Message-----
From: mirar@orchid.mirar.org [mailto:mirar@orchid.mirar.org]On Behalf Of
Mirar
Sent: Thursday, May 08, 2003 7:19 AM
To: reiserfs-list@namesys.com
Subject: power loss -> corruption


I've had this problem for a while. Every time the system gets a power
loss, all files that was recently written to gets garbage in them.
(It seems to me like the wrong content gets in the wrong files.)

I assume it's not supposed to be that way.

Is this something known?

It happens with 2.4.19 and 2.4.20 kernel versions.
Is there anything I can do to track down this issue?

Since it's every time, or at least close to every time...

I don't power down by breaking that power that often, so it hasn't
disturbed me that much, but I lost an important file today so I guess
I'm extra aggravated.

/Mirar

RE: power loss -> corruption

[berthiaume_wayne]

	"data=journal" will also work. You'll need to use the data logging
patches that Chris Mason has put together. 

-----Original Message-----
From: Mirar [mailto:mirar-yg@mirar.org]
Sent: Thursday, May 08, 2003 8:07 AM
To: Dieter Nützel
Cc: reiserfs-list@namesys.com
Subject: Re: power loss -> corruption


> Maybe, if you have some more details.

How do I get the necessary details?

> data-logging patches and mount option "data=ordered" solved similar
> problems for me.

Which patches? I'll add "data=ordered" to see if it'll help.

/Mirar

Re: power loss -> corruption

[Mirar]

berthiaume_wayne@emc.com writes:

> 	"data=journal" will also work. You'll need to use the data logging
> patches that Chris Mason has put together. 

Thanks.

I assume it's the ones I found here?
ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.20

Do I need all of them or just 03-data-logging?

Any reason why they haven't entered the official 2.4.21 (or have they)?

/Mirar

Re: power loss -> corruption

[Marc-Christian Petersen]

On Thursday 08 May 2003 14:32, Mirar wrote:

Hi Mirar,

> berthiaume_wayne@emc.com writes:
> > 	"data=journal" will also work. You'll need to use the data logging
> > patches that Chris Mason has put together.
>
> Thanks.
>
> I assume it's the ones I found here?
> ftp://ftp.suse.com/pub/people/mason/patches/data-logging/2.4.20
>
> Do I need all of them or just 03-data-logging?
>
> Any reason why they haven't entered the official 2.4.21 (or have they)?
2.4.21 isn't released yet. It's in -rc1 stage and there are still tons of 
outstanding bugs which have to be fixed (I am afraid they won't).

However, there are 2.4.21-rc1 patches. See:
ftp://namesys.com/pub/reiserfs-for-2.4/testing/data-logging-and-quota-2.4.21-rc1

--
ciao, Marc

Re: power loss -> corruption

[Anders Widman]

> "data=journal" will also work. You'll need to use the data logging
> patches that Chris Mason has put together. 

  When will data logging be included in stock kernels?

  //Anders

Re: power loss -> corruption

[Marc-Christian Petersen]

On Thursday 08 May 2003 14:34, Anders Widman wrote:

Hi Anders,

> > "data=journal" will also work. You'll need to use the data logging
> > patches that Chris Mason has put together.
>   When will data logging be included in stock kernels?
I am pretty sure, never in 2.4 timeframe. 2.5/2.6 will have it (not yet).

--
ciao, Marc

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 14:39 schrieb Marc-Christian Petersen:
> On Thursday 08 May 2003 14:34, Anders Widman wrote:
>
> Hi Anders,
>
> > > "data=journal" will also work. You'll need to use the data logging
> > > patches that Chris Mason has put together.
> >
> >   When will data logging be included in stock kernels?
>
> I am pretty sure, never in 2.4 timeframe. 2.5/2.6 will have it (not yet).

Why do you think?

It _is_ more than a usefull addition (fix) for 2.4 and 2.4.22-preX is a very 
good starting point. Chris et. al. are working more than a few days on it.

2.4 is more than a few months in production.
2.4 plus data-logging is in production as well.

Regards,
	Dieter

Re: power loss -> corruption

[Chris Mason]

On Thu, 2003-05-08 at 08:39, Marc-Christian Petersen wrote:
> On Thursday 08 May 2003 14:34, Anders Widman wrote:
> 
> Hi Anders,
> 
> > > "data=journal" will also work. You'll need to use the data logging
> > > patches that Chris Mason has put together.
> >   When will data logging be included in stock kernels?
> I am pretty sure, never in 2.4 timeframe. 2.5/2.6 will have it (not yet).

I expect to submit for 2.4.22-preX once it starts, the latest data
logging update removes the VFS changes that held up inclusion in the
past.

-chris

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 15:24 schrieb Chris Mason:
> On Thu, 2003-05-08 at 08:39, Marc-Christian Petersen wrote:
> > On Thursday 08 May 2003 14:34, Anders Widman wrote:
> >
> > Hi Anders,
> >
> > > > "data=journal" will also work. You'll need to use the data logging
> > > > patches that Chris Mason has put together.
> > >
> > >   When will data logging be included in stock kernels?
> >
> > I am pretty sure, never in 2.4 timeframe. 2.5/2.6 will have it (not yet).
>
> I expect to submit for 2.4.22-preX once it starts, the latest data
> logging update removes the VFS changes that held up inclusion in the
> past.

He, he, that's what I've expected...;-)

Good work, Chris!

I'll compiling,now.
Does it work on-top of -aa (-jam) or do I have to do some handwork as always?

Regards,
	Dieter

Re: power loss -> corruption

[Chris Mason]

On Thu, 2003-05-08 at 09:36, Dieter Nützel wrote:

> > I expect to submit for 2.4.22-preX once it starts, the latest data
> > logging update removes the VFS changes that held up inclusion in the
> > past.
> 
> He, he, that's what I've expected...;-)
> 
> Good work, Chris!
> 
> I'll compiling,now.
> Does it work on-top of -aa (-jam) or do I have to do some handwork as always?

You'll have to do some hand merging ;-)

-chris

Re: power loss -> corruption

[Dieter Nützel]

Am Donnerstag, 8. Mai 2003 14:34 schrieb Anders Widman:
> > "data=journal" will also work. You'll need to use the data logging
> > patches that Chris Mason has put together.
>
>   When will data logging be included in stock kernels?

See "Data-logging for 2.4.21-rc1+", "Data-logging for 2.4.21-rc1+ (again), 
"Data-logging and md as / fs bug".

Regards,
	Dieter

RE: power loss -> corruption

[Fong Vang]

Version 4 does sound impressive.  We rely heavily on ReiserFS so we are
keeping close watch of ReiserFS.  How long do you think it will remain in
Beta before the final release?

-----Original Message-----
From: Hans Reiser [mailto:reiser@namesys.com]
Sent: Thursday, May 08, 2003 12:29 PM
To: Mirar
Cc: Dieter Nützel; reiserfs-list@namesys.com
Subject: Re: power loss -> corruption


Mirar wrote:

>I think V4 sounds excellent, and I hope it will be incorporated in
>Linux 2.6/3.0. I assume there isn't any kernal patch for 2.4.20 around
>so I can play with it? I'm sorry if it has been mentioned on the list,
>I'm not up to date.
>
>/Mirar
>
>
>  
>
Please wait until July,  when we hope to have a beta-release.

-- 
Hans



This e-mail has been captured and archived by the ZANTAZ Digital Safe(tm)
service.  For more information, visit us at www.zantaz.com. 
IMPORTANT: This electronic mail message is intended only for the use of the
individual or entity to which it is addressed and may contain information
that is privileged, confidential or exempt from disclosure under applicable
law.  If the reader of this message is not the intended recipient, or the
employee or agent responsible for delivering this message to the intended
recipient, you are hereby notified that any dissemination, distribution or
copying of this communication is strictly prohibited.  If you have received
this communication in error, please notify the sender immediately by
telephone or directly reply to the original message(s) sent.  Thank you.

Re: power loss -> corruption

[Hans Reiser]

Fong Vang wrote:

>Version 4 does sound impressive.  We rely heavily on ReiserFS so we are
>keeping close watch of ReiserFS.  How long do you think it will remain in
>Beta before the final release?
>
>  
>
>  
>
I don't know....

-- 
Hans

Re: power loss -> corruption

[Anders Widman]

> Am Donnerstag, 15. Mai 2003 10:40 schrieb Mirar:
>> > This _is_ very well known for me.
>> > Some KDE config files and my tcsh history file where broken without
>> > data-logging (data=journal/ordered).
>>
>> My .zsh-history just got nuked with data=ordered.
>>
>> The kernel couldn't agree with my HPT controller and stopped all (!)
>> IDE activities when I did a 'hdparm -y /dev/hd?' (all sleep). (Note,
>> the .zsh-history was not on that controller.)

Hm..  Are  you sure you are using the right command? "-Y" is SLEEP and
"-y"  is STANDBY mode. With stand by mode the drive spins up by itself
when  data  is  requested.  However when you use sleep mode it needs a
reset to wake up again (or if the driver can issue a soft reset).

Even  if  the  drive spins back up there can be transfer errors or DMA
problems that follows. IF you want a reliable configuration you should
not use power-save mode. It is not very well supported on all hardware
in Linux :(


> In short, DON'T DO THAT without "suspend kernel patches".
> *nix is NOT designed for "disk power management" in the first place.
> But it evolves...;-)

> HPT ("small" IDE/ATA "RAID") controllers are not that good (supported)
as they
> seems. Apart that they are "cheap".

>> Not in the same way as previously (just a block), though. It was
>> completely filled with binary data.

> Sounds like another problem (hardware, see above).

>> No other active files seems to have been affected, though.

> Good.

> -Dieter






--------
PGP public key: https://tnonline.net/secure/pgp_key.txt