write-refresh md array (feature request)

write-refresh md array (feature request)

[Henrique de Moraes Holschuh]

(please CC me on replies).

I've been in several situations where it would have been helpful to be able
to refresh weak sectors by rewriting the whole md raid component device,
without the need to increase array failure risk through a fail+remove+add
cycle for the component device.

How difficult would it be to implement a "refresh" as a Linux md driver
sync_action, pigging back on "check" ?

Are there any drawbacks to write-refreshing component devices?

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: write-refresh md array (feature request)

[Dan Williams]

On Tue, Sep 10, 2013 at 11:19 AM, Henrique de Moraes Holschuh
<hmh@hmh.eng.br> wrote:
> (please CC me on replies).
>
> I've been in several situations where it would have been helpful to be able
> to refresh weak sectors by rewriting the whole md raid component device,
> without the need to increase array failure risk through a fail+remove+add
> cycle for the component device.
>
> How difficult would it be to implement a "refresh" as a Linux md driver
> sync_action, pigging back on "check" ?
>
> Are there any drawbacks to write-refreshing component devices?
>

Why is "check" insufficient?  If it trips over any bad sectors it will
re-write them.

--
Dan

Re: write-refresh md array (feature request)

[Henrique de Moraes Holschuh]

On Tue, 10 Sep 2013, Dan Williams wrote:
> On Tue, Sep 10, 2013 at 11:19 AM, Henrique de Moraes Holschuh
> <hmh@hmh.eng.br> wrote:
> > (please CC me on replies).
> > I've been in several situations where it would have been helpful to be able
> > to refresh weak sectors by rewriting the whole md raid component device,
> > without the need to increase array failure risk through a fail+remove+add
> > cycle for the component device.
> >
> > How difficult would it be to implement a "refresh" as a Linux md driver
> > sync_action, pigging back on "check" ?
> >
> > Are there any drawbacks to write-refreshing component devices?
> >
> 
> Why is "check" insufficient?  If it trips over any bad sectors it will
> re-write them.

The idea is to rewrite the sector _before_ it goes bad.

Consumer SATA HDDs nowadays not only apparently fail to properly remap
sectors because they don't track anymore that some sectors in that subtrack
were reported uncorrect a number of times (and thus are "weak").  They also
appear to not write-refresh sectors that required ECC correction, except
maybe during an offline SMART test routine.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: write-refresh md array (feature request)

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]

On Wed, 11 Sep 2013 09:32:23 -0300 Henrique de Moraes Holschuh
<hmh@hmh.eng.br> wrote:

> On Tue, 10 Sep 2013, Dan Williams wrote:
> > On Tue, Sep 10, 2013 at 11:19 AM, Henrique de Moraes Holschuh
> > <hmh@hmh.eng.br> wrote:
> > > (please CC me on replies).
> > > I've been in several situations where it would have been helpful to be able
> > > to refresh weak sectors by rewriting the whole md raid component device,
> > > without the need to increase array failure risk through a fail+remove+add
> > > cycle for the component device.
> > >
> > > How difficult would it be to implement a "refresh" as a Linux md driver
> > > sync_action, pigging back on "check" ?
> > >
> > > Are there any drawbacks to write-refreshing component devices?
> > >
> > 
> > Why is "check" insufficient?  If it trips over any bad sectors it will
> > re-write them.
> 
> The idea is to rewrite the sector _before_ it goes bad.
> 
> Consumer SATA HDDs nowadays not only apparently fail to properly remap
> sectors because they don't track anymore that some sectors in that subtrack
> were reported uncorrect a number of times (and thus are "weak").  They also
> appear to not write-refresh sectors that required ECC correction, except
> maybe during an offline SMART test routine.
> 

My view on this is that it has nothing to do with RAID.  If it is valuable to
"write-refresh" a device in a RAID array, then it would equally make sense to
write-refresh a device that wasn't in a RAID array.

So any solution to this should try to address the whole problem.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: write-refresh md array (feature request)

[CoolCold]

Hello!
You can achieve this with a bit inderect way - there are sync_min and
sync_max params which can be used to operate on certain borders of
array, more info
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/md.txt?id=refs/tags/v3.11#n579
.

On Tue, Sep 10, 2013 at 10:19 PM, Henrique de Moraes Holschuh
<hmh@hmh.eng.br> wrote:
> (please CC me on replies).
>
> I've been in several situations where it would have been helpful to be able
> to refresh weak sectors by rewriting the whole md raid component device,
> without the need to increase array failure risk through a fail+remove+add
> cycle for the component device.
>
> How difficult would it be to implement a "refresh" as a Linux md driver
> sync_action, pigging back on "check" ?
>
> Are there any drawbacks to write-refreshing component devices?
>
> --
>   "One disk to rule them all, One disk to find them. One disk to bring
>   them all and in the darkness grind them. In the Land of Redmond
>   where the shadows lie." -- The Silicon Valley Tarot
>   Henrique Holschuh
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best regards,
[COOLCOLD-RIPN]

Re: write-refresh md array (feature request)

[Henrique de Moraes Holschuh]

On Wed, 11 Sep 2013, CoolCold wrote:
> You can achieve this with a bit inderect way - there are sync_min and
> sync_max params which can be used to operate on certain borders of array,
> more info
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/md.txt?id=refs/tags/v3.11#n579.

Doesn't do anything close to what I requested.  Neither check nor repair
will re-write sectors that appear to be good (and are actually returning the
correct data after ECC correction by the component device, but will
eventually fail if not rewritten to soon).

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: write-refresh md array (feature request)

[Dan Williams]

On Wed, Sep 11, 2013 at 5:37 AM, Henrique de Moraes Holschuh
<hmh@hmh.eng.br> wrote:
> On Wed, 11 Sep 2013, CoolCold wrote:
>> You can achieve this with a bit inderect way - there are sync_min and
>> sync_max params which can be used to operate on certain borders of array,
>> more info
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/md.txt?id=refs/tags/v3.11#n579.
>
> Doesn't do anything close to what I requested.  Neither check nor repair
> will re-write sectors that appear to be good (and are actually returning the
> correct data after ECC correction by the component device, but will
> eventually fail if not rewritten to soon).

Sounds like a modification of the "replace" code to allow replacing
the with self-same drive. Something like "want_refresh" would mark the
drive as a soft replacement to mark that slot to be re-written.  But
if the drive is expected to be "weak" you could just rotate in a spare
drive without degrading the array with the existing code.

Re: write-refresh md array (feature request)

[Henrique de Moraes Holschuh]

On Wed, 11 Sep 2013, Dan Williams wrote:
> On Wed, Sep 11, 2013 at 5:37 AM, Henrique de Moraes Holschuh
> <hmh@hmh.eng.br> wrote:
> > On Wed, 11 Sep 2013, CoolCold wrote:
> >> You can achieve this with a bit inderect way - there are sync_min and
> >> sync_max params which can be used to operate on certain borders of array,
> >> more info
> >> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/md.txt?id=refs/tags/v3.11#n579.
> >
> > Doesn't do anything close to what I requested.  Neither check nor repair
> > will re-write sectors that appear to be good (and are actually returning the
> > correct data after ECC correction by the component device, but will
> > eventually fail if not rewritten to soon).
> 
> Sounds like a modification of the "replace" code to allow replacing
> the with self-same drive. Something like "want_refresh" would mark the

That would do it, if the system considers that the component device is still
valid while it is being replaced in-place with itself :)

> drive as a soft replacement to mark that slot to be re-written.  But
> if the drive is expected to be "weak" you could just rotate in a spare
> drive without degrading the array with the existing code.

Well, that requires a spare device, which is often not the case in small
setups (which are the ones more likely to be using md raid with SATA,
instead of SAS).

Although I really don't know if SAS nearline devices are any better than the
current crop of untrustworthy crap that passes for SATA HDDs.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh