emergency recovery

emergency recovery

[jeff sacksteder]

I have a root raid-5 set that has failed 2 disks simultaneously. This
is catastrophically bad news. I'm not sure what happened, but the
disks spin up and appear to function independently without problem.
The superblocks are all out of sync if I look at them with mdadm
examine.

It is essential that I not make this worse than it is. Is the
information in the wiki current regarding recovery of  failed arrays?
Are there any other resources I should be looking at?

Re: emergency recovery

[Mario 'BitKoenig' Holbe]

jeff sacksteder <jsacksteder@gmail.com> wrote:
> It is essential that I not make this worse than it is. Is the

Well, this in mind the first thing you should do is taking raw images
from all the component drives before even thinking about any kind of
recovery. Just to make sure you have more than one try when something
goes wrong.


regards
   Mario
-- 
The social dynamics of the net are a direct consequence of the fact that
nobody has yet developed a Remote Strangulation Protocol.  -- Larry Wall

Re: emergency recovery

[john]

Mario 'BitKoenig' Holbe wrote:
> jeff sacksteder <jsacksteder@gmail.com> wrote:
>   
>> It is essential that I not make this worse than it is. Is the
>>     
>
> Well, this in mind the first thing you should do is taking raw images
> from all the component drives before even thinking about any kind of
> recovery. Just to make sure you have more than one try when something
> goes wrong.
>
>
> regards
>    Mario
>   
I have the same problem. Regarding raw images, I have one drive that has 
a periodic clicking and the other is fine (but stale by 1 event).
What's the best way to make a raw image of a temperamental drive?  I 
have an exact duplicate that I could image to.

thanks. 
john.

Re: emergency recovery

[Mario 'BitKoenig' Holbe]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

On Thu, Apr 10, 2008 at 09:08:37AM -0400, john wrote:
> What's the best way to make a raw image of a temperamental drive?  I  

Don't know. I'd first try a simple dd, you could also have a look at
ddrescue (http://www.garloff.de/kurt/linux/ddrescue/)
or gddrescue (http://savannah.gnu.org/download/ddrescue/)
With them it's probably easier to deal with erroneous drives.


regards
   Mario
-- 
<Sique> Huch? 802.1q? Was sucht das denn hier? Wie kommt das ans TAGgeslicht?

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 481 bytes --]

Re: emergency recovery

[john]

Mario 'BitKoenig' Holbe wrote:
> jeff sacksteder <jsacksteder@gmail.com> wrote:
>   
>> It is essential that I not make this worse than it is. Is the
>>     
>
> Well, this in mind the first thing you should do is taking raw images
> from all the component drives before even thinking about any kind of
> recovery.

Thanks.  I'll try to use a fresh drive with a ddrescue(ed) image on it 
with the corrupt drive. I may do the same with the stale(evms 
terminology?) drive.

Do I need to dd image all 6 dives somewhere to be safe?

 I want to force assemble 4 good +1stale +1 corrupt to a degraded raid5 
array with 5 good +1 corrupt.  I could avoid dd imaging the 4 good 
drives if I could mark these 4 drives read-only.  Is that possible or 
with the forced assembly does it need to write new md meta data to all 
the drives and therefore need to modify these drives?

  Tell me to start a new thread if this doesn't pertain to the OP's 
question.

Re: emergency recovery

[jeff sacksteder]

I have images of all 5 drives in my set. The set looks like this right now.

/dev/sda1 - active sync
/dev/sdb1 - faulty
/dev/sdc1 - faulty removed
/dev/sdd1 - active sync
/dev/sde1 - active sync

I am lead to believe that the way to proceed would be to
force-assemble the array with mdadm. Should I start with 4 drives, or
try to do all 5 at once? Which should I leave out?

Re: emergency recovery

[jeff sacksteder]

I should also note that I'm doing this from a liveCD and though I have
modprobe-ed the appropriate modules, I still don't see the node in
/dev for md0.

Re: emergency recovery

[Bill Davidsen]

jeff sacksteder wrote:
> I should also note that I'm doing this from a liveCD and though I have
> modprobe-ed the appropriate modules, I still don't see the node in
> /dev for md0.

You can try doing an assemble with force after backing up whatever you 
feel you should (or can). Assuming you get an array at that point, you 
can do an information only file system check (fsck -n) to see what the 
state of the f/s might be. You can also run the 'check' action to see 
how unhappy mdadm thinks things are.

By that time you will have information to use or report back to the list 
for more ideas.

-- 
Bill Davidsen <davidsen@tmr.com>
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark