Re: What just happened to my disks/RAID5 array?

[Johannes Truschnigg <johannes@truschnigg.info>]

[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]

Hi Phil,

first of all, thanks for replying and providing both technical and moral
support ;) As it turned out today, I won't be able to get my hands on
the box for at least another 12 hours, so I can only speculate what 
happened (at the physical/hardware level, that is) still.

On 09/13/2011 01:37 PM, Phil Turmel wrote:
> Simultaneous failure of that many devices strains credulity, so I
> doubt you've lost your array.  One possible variant of "2" would be a
> failed drive that draws enough current to drop the voltage to its
> sibling drives.

All the drives are located in seperate hot-swap trays with a full,
unoccupied 5.25" slot in between them. If my appartment wasn't set on
fire with half the drives roasting in it, I think bad cooling can be 
ruled out - the drives never went over 40°C even with all case fans 
turned off.

The controller seems alive still - lsdrv (output attached) lists the
kernel still having registered some of the component devices.

> Since some drives are still "alive", they'll have newer event counts
>  than the devices that went offline.  When you fix the root cause,
> you may need to use "--assemble --force" to get mdadm to restart your
> array.

I see - I don't have the interim storage capacity to dump the drives
before trying to do so - is there any advice you can offer to do this
assembly procedure in the safest way possible?

> The output of "lsdrv" [1] would be helpful in offering more specific
>  advice, along with "mdadm -D" of the array and "mdadm -E" of all of
>  its components (when you get them back).

I will provide the components' info asap.

Thanks very much for sharing your input and expertise!

-- 
with best regards:
- Johannes Truschnigg ( johannes@truschnigg.info )

www: http://johannes.truschnigg.info/
phone: +43 650 2 133337
xmpp: johannes@truschnigg.info

Please do not bother me with HTML-eMail or attachments. Thank you.

[-- Attachment #2: lsdrv.txt --]
[-- Type: text/plain, Size: 884 bytes --]

PCI [pata_amd] 00:06.0 IDE interface: nVidia Corporation MCP78S [GeForce 8200] IDE (rev a1)
 ├─scsi 0:0:0:0 ATA TRANSCEND {20090625_D40D51BB}
 │  └─sda: [8:0] Partitioned (dos) 1.87g
 │     └─sda1: [8:1] (ext2) 1.87g 'VIRTUE' {ff586bcd-b1fd-4c08-a0ea-08e2e1c7b8f9}
 │        ├─Mounted as /dev/root @ /
 │        └─Mounted as /dev/root @ /srv/web/virtue
 └─scsi 1:x:x:x [Empty]
PCI [ahci] 00:09.0 SATA controller: nVidia Corporation MCP78S [GeForce 8200] AHCI Controller (rev a2)
 ├─scsi 2:x:x:x [Empty]
 ├─scsi 3:x:x:x [Empty]
 └─scsi 7:x:x:x [Empty]
Other Block Devices
 ├─dm-0: [253:0] (ext4) 5.46t 'MAIN_STORAGE' {aff33f2a-1dac-47e5-a9ed-05e24d3bda15}
 │  ├─Mounted as /dev/mapper/VG_STORAGE-LV_MAIN @ /media/virtue_main
 │  └─Mounted as /dev/mapper/VG_STORAGE-LV_MAIN @ /srv/files
 ├─md0: [9:0] Empty/Unknown 5.46t


[-- Attachment #3: md0-examine.txt --]
[-- Type: text/plain, Size: 979 bytes --]

/dev/md0:
        Version : 1.2
  Creation Time : Tue Dec 21 10:25:32 2010
     Raid Level : raid5
     Array Size : 5860548608 (5589.05 GiB 6001.20 GB)
  Used Dev Size : 1465137152 (1397.26 GiB 1500.30 GB)
   Raid Devices : 5
  Total Devices : 3
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Tue Sep 13 10:15:49 2011
          State : active, FAILED
 Active Devices : 0
Working Devices : 0
 Failed Devices : 3
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       0        0        1      removed
       2       0        0        2      removed
       3       0        0        3      removed
       4       0        0        4      removed

       1       8       64        -      faulty spare
       2       8       48        -      faulty spare
       5       8       80        -      faulty spare