'let the hdd remap the bad blocks'

'let the hdd remap the bad blocks'

[Newsmail]

Hello Hans and Oleg,
maybe its an offtopic question, but Hans always talks about leaving the 
hard disk to remap the bad blocks by itself. could you explain it in some 
words, how all this works, what happens after, and since when it exists, or 
do you have any special URL explaining this?
thx in advance,
greg

Re: 'let the hdd remap the bad blocks'

[Oleg Drokin]

Hello!

   Basically uyou'd better search for this on HDD vendors sites.
   What's going on is simply can be described this way:
   You write some block to HDD, if HDD decides the block is bad for some reason
   and remapping is allowed (usually by tiurning on SMART), block is written to
   different on-platter location and drive adds one more entry to its
   remaped-blocks list. Next time you read this block, drive consults its
   remapped blocks list and if block is remapped, reads it from new location
   with correct content.
   Described mechanism works for writing.
   Actually I've seen something that looks like remapping on read, though 
   I have no meaningful explanation for that (except that they may have some
   extra redundant info stored when you write data to disk, so that if sector
   cannot be read, its content is restored with that redundant information and
   sector is then remapped.). And this process takes a lot of time.

Bye,
    Oleg
On Mon, Aug 19, 2002 at 03:58:30PM +0100, Newsmail wrote:
> Hello Hans and Oleg,
> maybe its an offtopic question, but Hans always talks about leaving the 
> hard disk to remap the bad blocks by itself. could you explain it in some 
> words, how all this works, what happens after, and since when it exists, or 
> do you have any special URL explaining this?
> thx in advance,
> greg
> 
> 
> 

Re: 'let the hdd remap the bad blocks'

[Matthias Andree]

Oleg Drokin <green@namesys.com> writes:

>    Basically uyou'd better search for this on HDD vendors sites.
>    What's going on is simply can be described this way:
>    You write some block to HDD, if HDD decides the block is bad for some reason
>    and remapping is allowed (usually by tiurning on SMART), block is written to
>    different on-platter location and drive adds one more entry to its
>    remaped-blocks list. Next time you read this block, drive consults its
>    remapped blocks list and if block is remapped, reads it from new location
>    with correct content.
>    Described mechanism works for writing.
>    Actually I've seen something that looks like remapping on read, though 
>    I have no meaningful explanation for that (except that they may have some
>    extra redundant info stored when you write data to disk, so that if sector
>    cannot be read, its content is restored with that redundant information and
>    sector is then remapped.). And this process takes a lot of time.

My Fujitsu MAH-3182MP drive (SCSI actually) had ARRE enabled as it
shipped, but ARWE disabled, for reasons I cannot tell, not even from the
data book (PDF). That's Automatic Remap on Read/Write Error. I'm not
sure what it really means, but if the drive really remaps on a read
error, it's going to leak a block at power loss while it is amidst a
block write the next time this block is read. So I switched that to do
ARWE. IDE users are not too lucky unless their vendor provides them with
a tool (and not many ship raw floppy images, many have some multi-MB
Windoze tools just to write some hundred kByte to a floppy disk...)

-- 
Matthias Andree

Re: 'let the hdd remap the bad blocks'

[Hans Reiser]

Oleg Drokin wrote:

>Hello!
>
>   Basically uyou'd better search for this on HDD vendors sites.
>   What's going on is simply can be described this way:
>   You write some block to HDD, if HDD decides the block is bad for some reason
>   and remapping is allowed (usually by tiurning on SMART), block is written to
>   different on-platter location and drive adds one more entry to its
>   remaped-blocks list. Next time you read this block, drive consults its
>   remapped blocks list and if block is remapped, reads it from new location
>   with correct content.
>   Described mechanism works for writing.
>   Actually I've seen something that looks like remapping on read, though 
>   I have no meaningful explanation for that (except that they may have some
>   extra redundant info stored when you write data to disk, so that if sector
>   cannot be read, its content is restored with that redundant information and
>   sector is then remapped.). And this process takes a lot of time.
>
>Bye,
>    Oleg
>On Mon, Aug 19, 2002 at 03:58:30PM +0100, Newsmail wrote:
>  
>
>>Hello Hans and Oleg,
>>maybe its an offtopic question, but Hans always talks about leaving the 
>>hard disk to remap the bad blocks by itself. could you explain it in some 
>>words, how all this works, what happens after, and since when it exists, or 
>>do you have any special URL explaining this?
>>thx in advance,
>>greg
>>
>>
>>
>>    
>>
>
>
>  
>
Just taking a guess, many hard drives have difficult and time-consuming 
procedures that they can go through to read a troublesome block.  These 
can take 20-30 seconds.  Probably if they have to go through these 
procedures, once they finally succeed the smart vendors remap the block.

Hans

Re: 'let the hdd remap the bad blocks'

[Matthias Andree]

Hans Reiser <reiser@namesys.com> writes:

> Just taking a guess, many hard drives have difficult and time-consuming
> procedures that they can go through to read a troublesome block.  These
> can take 20-30 seconds.  Probably if they have to go through these
> procedures, once they finally succeed the smart vendors remap the block.

They should try to rewrite and write verify the block before remapping
it, as there is only a finite amount of spares.

For SCSI drives, there's also Jörg Schilling's "sformat" tool that can
do the "badblocks" stuff directly in the drive rather than through all
the kernel buffers, and can also refresh or reassign bad blocks.

-- 
Matthias Andree

Re: 'let the hdd remap the bad blocks'

[Hans Reiser]

Matthias Andree wrote:

>Hans Reiser <reiser@namesys.com> writes:
>
>  
>
>>Just taking a guess, many hard drives have difficult and time-consuming
>>procedures that they can go through to read a troublesome block.  These
>>can take 20-30 seconds.  Probably if they have to go through these
>>procedures, once they finally succeed the smart vendors remap the block.
>>    
>>
>
>They should try to rewrite and write verify the block before remapping
>it, as there is only a finite amount of spares.
>
>For SCSI drives, there's also Jörg Schilling's "sformat" tool that can
>do the "badblocks" stuff directly in the drive rather than through all
>the kernel buffers, and can also refresh or reassign bad blocks.
>
>  
>

Vitaly, take a look at that.  Part of a good user interface is letting 
users know what tools are available.  Remember, most users will 
encounter a failing drive and/or fsck on a journaling fs as a rare and 
stressful event in their lives, so it is good to educate them with URLs 
and other references at the time they run fsck.

Re: 'let the hdd remap the bad blocks'

[Matthias Andree]

On Tue, 20 Aug 2002, Hans Reiser wrote:

> Vitaly, take a look at that.  Part of a good user interface is letting 
> users know what tools are available.  Remember, most users will 
> encounter a failing drive and/or fsck on a journaling fs as a rare and 
> stressful event in their lives, so it is good to educate them with URLs 
> and other references at the time they run fsck.

A propos URL, here we go:

ftp://ftp.fokus.gmd.de/pub/unix/sformat/

-- 
Matthias Andree

quota support ?

[Serge Kolodeznyh]

Hi, All !

Who know, will quota support and fs at all be working, if my kernel is
2.4.19 version, and make the patches for quota support from reiserfs site ?
The problem is that reiser's patches are for 2.4.19-pre3, not clean 2.4.19.

---
Serge Kolodeznyh
Paradigma AG
Network /system administrator
SVK42-RIPN / SVK33-RIPE

Re: quota support ?

[Oleg Drokin]

Hello!

On Tue, Aug 20, 2002 at 05:12:09PM +0400, Serge Kolodeznyh wrote:

> Who know, will quota support and fs at all be working, if my kernel is
> 2.4.19 version, and make the patches for quota support from reiserfs site ?
> The problem is that reiser's patches are for 2.4.19-pre3, not clean 2.4.19.

This is not a problem. Everything should work fine if applies correctly.

Bye,
    Oleg

Re: quota support ?

[Serge Kolodeznyh]

Last patch doesn't apllied :-(

[root@helios linux-2.4.19]# gzip -cd kinoded-8-2.4.19-pre3.diff.gz |
patch -p1
patching file fs/inode.c
Hunk #4 FAILED at 698.
Hunk #5 succeeded at 1119 (offset -1 lines).
Hunk #7 succeeded at 1278 (offset -1 lines).
1 out of 7 hunks FAILED -- saving rejects to file fs/inode.c.rej

inode.c.rej:

***************
*** 697,731 ****
   spin_unlock(&inode_lock);

   dispose_list(freeable);

   /*
-   * If we didn't freed enough clean inodes schedule
-   * a sync of the dirty inodes, we cannot do it
-   * from here or we're either synchronously dogslow
-   * or we deadlock with oom.
    */
   if (goal)
-   schedule_task(&unused_inodes_flush_task);
  }

  int shrink_icache_memory(int priority, int gfp_mask)
  {
   int count = 0;
-
-  /*
-   * Nasty deadlock avoidance..
-   *
-   * We may hold various FS locks, and we don't
-   * want to recurse into the FS that called us
-   * in clear_inode() and friends..
-   */
-  if (!(gfp_mask & __GFP_FS))
-   return 0;
-
   count = inodes_stat.nr_unused / priority;
-
   prune_icache(count);
-  kmem_cache_shrink(inode_cachep);
   return 0;
  }

--- 698,729 ----
   spin_unlock(&inode_lock);

   dispose_list(freeable);
+  kmem_cache_shrink(inode_cachep);

   /*
+   * If we didn't freed enough clean inodes
+   * start a sync now
    */
   if (goal)
+         try_to_sync_unused_inodes();
+ }
+
+ void prune_icache(int goal) {
+  if (!atomic_read(&kinoded_goal)) {
+   atomic_set(&kinoded_goal, goal) ;
+   wake_up_interruptible(&kinoded_wait);
+  }
+  /* otherwise, kinoded has already been woken up, move on
+  ** to other caches that might be shrinkable instead of spending time
+  ** waiting for kinoded to make progress
+  */
  }

  int shrink_icache_memory(int priority, int gfp_mask)
  {
   int count = 0;
   count = inodes_stat.nr_unused / priority;
   prune_icache(count);
   return 0;
  }



---
Serge Kolodeznyh
Paradigma AG
Network /system administrator
SVK42-RIPN / SVK33-RIPE

----- Original Message -----
From: "Oleg Drokin" <green@namesys.com>
To: "Serge Kolodeznyh" <mysubscribes@paradigma.ru>
Cc: <reiserfs-list@namesys.com>
Sent: Tuesday, August 20, 2002 5:15 PM
Subject: Re: [reiserfs-list] quota support ?


> Hello!
>
> On Tue, Aug 20, 2002 at 05:12:09PM +0400, Serge Kolodeznyh wrote:
>
> > Who know, will quota support and fs at all be working, if my kernel is
> > 2.4.19 version, and make the patches for quota support from reiserfs
site ?
> > The problem is that reiser's patches are for 2.4.19-pre3, not clean
2.4.19.
>
> This is not a problem. Everything should work fine if applies correctly.
>
> Bye,
>     Oleg
>

Re: quota support ?

[Chris Mason]

On Tue, 2002-08-20 at 10:24, Serge Kolodeznyh wrote:
> 
> Last patch doesn't apllied :-(
> 
> [root@helios linux-2.4.19]# gzip -cd kinoded-8-2.4.19-pre3.diff.gz |
> patch -p1
> patching file fs/inode.c
> Hunk #4 FAILED at 698.
> Hunk #5 succeeded at 1119 (offset -1 lines).
> Hunk #7 succeeded at 1278 (offset -1 lines).
> 1 out of 7 hunks FAILED -- saving rejects to file fs/inode.c.rej

I'll release an updated set of patches later this week.

-chris