Re: Suggestion needed for fixing RAID6

[Neil Brown <neilb@suse.de>]

On Mon, 03 May 2010 12:21:08 +0200
MRK <mrk@shiftmail.org> wrote:

> On 05/03/2010 12:04 PM, MRK wrote:
> > On 05/03/2010 04:17 AM, Neil Brown wrote:
> >> On Sat, 1 May 2010 23:44:04 +0200
> >> "Janos Haar"<janos.haar@netcenter.hu>  wrote:
> >>
> >>> The general problem is, i have one single-degraded RAID6 + 2 
> >>> badblock disk
> >>> inside wich have bads in different location.
> >>> The big question is how to keep the integrity or how to do the 
> >>> rebuild by 2
> >>> step instead of one continous?
> >> Once you have the fix that has already been discussed in this thread, 
> >> the
> >> only other problem I can see with this situation is if attempts to 
> >> write good
> >> data over the read-errors results in a write-error which causes the 
> >> device to
> >> be evicted from the array.
> >>
> >> And I think you have reported getting write
> >> errors.
> >
> > His dmesg AFAIR has never reported any error of the kind "raid5:%s: 
> > read error NOT corrected!! " (the error message you get on failed 
> > rewrite AFAIU)
> > Up to now (after my patch) he only tried with MD above DM-COW and DM 
> > was dropping the drive on read error so I think MD didn't get any 
> > opportunity to rewrite.
> >
> > It is not clear to me what kind of error MD got from DM:
> >
> > Apr 29 09:50:29 Clarus-gl2k10-2 kernel: device-mapper: snapshots: 
> > Invalidating snapshot: Error reading/writing.
> > Apr 29 09:50:29 Clarus-gl2k10-2 kernel: ata8: EH complete
> > Apr 29 09:50:29 Clarus-gl2k10-2 kernel: raid5: Disk failure on dm-1, 
> > disabling device.
> >
> > I don't understand from what place the md_error() is called...
> > [CUT]
> 
> Oh and there is another issue I wanted to expose:
> 
> His last dmesg:
> http://download.netcenter.hu/bughunt/20100430/messages
> 
> Much after the line:
> Apr 29 09:50:29 Clarus-gl2k10-2 kernel: raid5: Disk failure on dm-1, 
> disabling device.
> 
> there are many lines like this:
> Apr 29 09:50:29 Clarus-gl2k10-2 kernel: raid5:md3: read error not 
> correctable (sector 1662189872 on dm-1).
> 
> How come MD still wants to read from a device it has disabled?
> looks like a problem to me...

There are often many IO requests in flight at the same time.  When one
returns with an error we might fail the device but there are still lots more
that have not yet completed.  As they complete we might write messages about
them - even after we have reported the device as 'failed'.  But we never
initiate an IO after the device has been marked 'faulty'.

NeilBrown



> MD also scrubs failed devices during check?
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html