Re: md failing mechanism

[Phil Turmel <philip@turmel.org>]

On 01/23/2016 05:33 AM, Dark Penguin wrote:
>>> I wonder what happens if the only drive in a degraded mirror fails?..
>>
>> Or any drive in a raid5 hits an error while replacing a failed disk.
>>
>> BOOM.
> 
> Oh... That sucks. I understand that BOOM would happen on a RAID-F, but
> kicking the last drive doesn't seem like a very rational thing to do -
> you would get a RAID with zero drives, which is not something anybody
> would want in any situation!.. I thought it would get into some kind of
> extreme panic mode so that you could at least save what you can...

It does.  It's called manual intervention.

> Actually, it would make sense to avoid "BOOM" as much as possible;
> instead of going into a "BOOM" mode, it should go into some kind of
> "recovery" mode, when the array immediately turns read-only and allows
> you to salvage as much data as you can from the faulty drive. It would
> be especially useful in RAID5; only one faulty sector or even some other
> stupid error without even faulty sectors should not cause total data
> loss!.. I don't think it would be very hard to implement, or contradict
> some important philosophy, but it would save data, jobs and lives. :)

Well, the root causes vary, and the actions needed vary accordingly.
There isn't enough information to proceed without operator assistance.
This list has helped numerous people recover their data.  ( But not
always. )

> Well, suppose "BOOM" happened on my RAID1, and both drives failed.
> Naturally, not at the same time, so data differs between the drives. So
> I have two faulty drives; is it possible (with some other recovery
> software?) to just mount each one read-only and copy data from it?.. I
> can live with having to copy as much data as I can from the first one
> and try to copy the rest from the older drive. I mean, I basically have
> the entire filesystem there! I should be able to just mount the
> filesystem on that drive, skipping the RAID header - just fdisk the
> drive, delete the RAID partition and create another one starting a few
> blocks later!.. Would that not work?.. That's actually why I chose
> RAID1; even if everything goes BOOM, I should be able to mount the
> drives separately!..

Yes, you can use dmsetup to gain access to any raid1's data area.
Mirrors using v0.90 and v1.0 metadata can be accessed directly.  (v.90
is strongly discouraged, though, as there are unavoidable ambiguities in
that old format that can blow up on you.)

raid10 and parity raids cannot be accessed like this.  You need
scrubbing and good drives, quick response to failures, and backups for
the final tiny probability of failure.

Phil