* Import/recover RAID6 created with SAS2108?
@ 2016-03-06 21:37 Dan Russell
2016-03-07 21:50 ` Andreas Klauer
0 siblings, 1 reply; 6+ messages in thread
From: Dan Russell @ 2016-03-06 21:37 UTC (permalink / raw)
To: linux-raid
I have a RAID6 array which was created on a Supermicro card built around the LSI 2108. Two drives failed, I replaced one and started a rebuild, then two more failed, making my array go offline.
It looks like the failure was due to a cabling or backplane problem, and I suspect the data on the drives is good. However, the Supermicro card is unable to read the foreign configs (it thinks there are 3 foreign configs, none of which are importable into a working array).
I would like to attempt recovery per the Wiki’s instructions (overlay files, force assembly, etc); can md read a RAID6 created by the 2108, or would I be wasting my time?
I have moved the drives from the hardware RAID card to a non-RAID SAS card and can see all drives. However, mdadm -E on these drives seems to show the RAID controller’s field of vision (how many defined arrays, total number of physical disks in the system), not any insight into this drive’s role as a member of an array. I can paste or gist the output of madam -E (304 lines) if it would help.
Thanks in advance-
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Import/recover RAID6 created with SAS2108?
2016-03-06 21:37 Import/recover RAID6 created with SAS2108? Dan Russell
@ 2016-03-07 21:50 ` Andreas Klauer
2016-03-09 3:20 ` Dan Russell
0 siblings, 1 reply; 6+ messages in thread
From: Andreas Klauer @ 2016-03-07 21:50 UTC (permalink / raw)
To: Dan Russell; +Cc: linux-raid
On Sun, Mar 06, 2016 at 04:37:42PM -0500, Dan Russell wrote:
> can md read a RAID6 created by the 2108, or would I be wasting my time?
It might be possible.
I don't use HW-RAID anywhere but if I was forced to, this would be
one of the first things to determine, how to read it with software
in case the card fails.
> I can paste or gist the output of madam -E (304 lines) if it would help.
I don't know if it would help, but paste everything you have anyhow.
Is the data encrypted? If not - search for known file type headers
(like a large jpeg or tar.gz, something larger than disks * chunksize),
then look at the data on the other disks at the same offsets
(grab a few megs of each disks), then try to deduce the layout
and structure from that.
Basically use known data to reverse-engineer the layout if you
currently know nothing about disk orders, chunk sizes, etc.
Regards
Andreas Klauer
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Import/recover RAID6 created with SAS2108?
2016-03-07 21:50 ` Andreas Klauer
@ 2016-03-09 3:20 ` Dan Russell
2016-03-09 19:27 ` Andreas Klauer
2016-03-10 7:02 ` NeilBrown
0 siblings, 2 replies; 6+ messages in thread
From: Dan Russell @ 2016-03-09 3:20 UTC (permalink / raw)
To: linux-raid
Thank you. I’ve made some progress. My hardware controller uses a DDF container, which I’m able to start and inspect with mdadm. Based on the mdadm release notes, I’ve updated from v3.2.5 to v3.4.
The data is not encrypted, and I believe I have the HDD order and chunk sizes correct (in part because the DDF container matches my independently-gathered notes).
The HDD order is sdi, sdj, sdag, sdl - sdz, sdaa - sdaf (when the array initially failed, I replaced the drive in slot 2 with a new drive and started a rebuild. The partially-rebuilt drive is sdk, the original “failed” drive is sdag).
When I did an Incremental on the container, I ended up with 3 inactive arrays (one 2-disk, one 20-disk, one 2-disk), which is comparable to what the hardware RAID controller told me about foreign configurations. So I tried to create a new 24-disk array with the same parameters as the old.
I am able to create an array and see the LVM label on md1. However fdisk and mdadm are reporting the array is 17.6TB in size, whereas it should be 66TB (24 3TB HDDs RAID6). This is the same whether I specify or leave off the —size option when creating the array.
The first mdadm —create to make the ddf container shows that the ctime for sdag and sdp is Jan 17; this is the last time I booted this server prior to the breakage. I’m wondering if there’s some way I can use the container metadata from either of those drives and ignore the rest?
Note that md0 is my root volume and is perfectly fine; it is only listed because I didn’t want to edit any command output.
# uname -r
3.13.0-49-generic
# mdadm -V
mdadm - v3.4 - 28th January 2016
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdh1[1] sdg1[0]
124967808 blocks super 1.2 [2/2] [UU]
# mdadm --create /dev/md127 -e ddf -l container -n 24 /dev/mapper/sdi /dev/mapper/sdj /dev/mapper/sdag /dev/mapper/sdl /dev/mapper/sdm /dev/mapper/sdn /dev/mapper/sdo /dev/mapper/sdp /dev/mapper/sdq /dev/mapper/sdr /dev/mapper/sds /dev/mapper/sdt /dev/mapper/sdu /dev/mapper/sdv /dev/mapper/sdw /dev/mapper/sdx /dev/mapper/sdy /dev/mapper/sdz /dev/mapper/sdaa /dev/mapper/sdab /dev/mapper/sdac /dev/mapper/sdad /dev/mapper/sdae /dev/mapper/sdaf
mdadm: /dev/mapper/sdj appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdag appears to be part of a raid array:
level=container devices=212 ctime=Sun Jan 17 17:03:35 2016
mdadm: /dev/mapper/sdl appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdm appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdn appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdo appears to be part of a raid array:
level=container devices=199 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdp appears to be part of a raid array:
level=container devices=212 ctime=Sun Jan 17 17:03:35 2016
mdadm: /dev/mapper/sdq appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdr appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sds appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdt appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdu appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdv appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdw appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdx appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdy appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdz appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdaa appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdab appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdac appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdad appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdae appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
mdadm: /dev/mapper/sdaf appears to be part of a raid array:
level=container devices=198 ctime=Fri Feb 26 13:38:07 2016
Continue creating array? y
mdadm: container /dev/md127 prepared.
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md127 : inactive dm-24[23](S) dm-23[22](S) dm-22[21](S) dm-21[20](S) dm-20[19](S) dm-19[18](S) dm-18[17](S) dm-17[16](S) dm-16[15](S) dm-15[14](S) dm-14[13](S) dm-13[12](S) dm-12[11](S) dm-11[10](S) dm-10[9](S) dm-9[8](S) dm-8[7](S) dm-7[6](S) dm-6[5](S) dm-5[4](S) dm-4[3](S) dm-25[2](S) dm-2[1](S) dm-1[0](S)
786432 blocks super external:ddf
md0 : active raid1 sdh1[1] sdg1[0]
124967808 blocks super 1.2 [2/2] [UU]
# mdadm --create /dev/md1 --assume-clean --level=6 --raid-devices=24 --chunk=64 --size=2929686528 /dev/mapper/sdi /dev/mapper/sdj /dev/mapper/sdag /dev/mapper/sdl /dev/mapper/sdm /dev/mapper/sdn /dev/mapper/sdo /dev/mapper/sdp /dev/mapper/sdq /dev/mapper/sdr /dev/mapper/sds /dev/mapper/sdt /dev/mapper/sdu /dev/mapper/sdv /dev/mapper/sdw /dev/mapper/sdx /dev/mapper/sdy /dev/mapper/sdz /dev/mapper/sdaa /dev/mapper/sdab /dev/mapper/sdac /dev/mapper/sdad /dev/mapper/sdae /dev/mapper/sdaf
mdadm: /dev/mapper/sdi appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdj appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdag appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdl appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdm appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdn appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdo appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdp appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdq appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdr appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sds appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdt appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdu appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdv appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdw appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdx appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdy appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdz appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdaa appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdab appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdac appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdad appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdae appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
mdadm: /dev/mapper/sdaf appears to be part of a raid array:
level=container devices=24 ctime=Tue Mar 8 21:18:40 2016
Continue creating array? y
mdadm: Creating array inside ddf container md127
mdadm: array /dev/md1 started.
# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md1 : active (auto-read-only) raid6 dm-24[23] dm-23[22] dm-22[21] dm-21[20] dm-20[19] dm-19[18] dm-18[17] dm-17[16] dm-16[15] dm-15[14] dm-14[13] dm-13[12] dm-12[11] dm-11[10] dm-10[9] dm-9[8] dm-8[7] dm-7[6] dm-6[5] dm-5[4] dm-4[3] dm-25[2] dm-2[1] dm-1[0]
17208463360 blocks super external:/md127/0 level 6, 64k chunk, algorithm 10 [24/24] [UUUUUUUUUUUUUUUUUUUUUUUU]
md127 : inactive dm-24[23](S) dm-23[22](S) dm-22[21](S) dm-21[20](S) dm-20[19](S) dm-19[18](S) dm-18[17](S) dm-17[16](S) dm-16[15](S) dm-15[14](S) dm-14[13](S) dm-13[12](S) dm-12[11](S) dm-11[10](S) dm-10[9](S) dm-9[8](S) dm-8[7](S) dm-7[6](S) dm-6[5](S) dm-5[4](S) dm-4[3](S) dm-25[2](S) dm-2[1](S) dm-1[0](S)
786432 blocks super external:ddf
md0 : active raid1 sdh1[1] sdg1[0]
124967808 blocks super 1.2 [2/2] [UU]
unused devices: <none>
# mdadm -E /dev/md127
/dev/md127:
Magic : de11de11
Version : 01.02.00
Controller GUID : 4C696E75:782D4D44:2064616D:2D73746F:72616765:2D6E3031
(Linux-MD dam-storage-n01)
Container GUID : 4C696E75:782D4D44:DEADBEEF:00000000:4410E200:C7DA314C
(Linux-MD 03/08/16 21:18:40)
Seq : 00000003
Redundant hdr : yes
Virtual Disks : 1
VD GUID[0] : 4C696E75:782D4D44:DEADBEEF:00000000:4410E257:2EF31542
(Linux-MD 03/08/16 21:20:07)
unit[0] : 1
state[0] : Optimal, Consistent
init state[0] : Fully Initialised
access[0] : Read/Write
Name[0] : 1
Raid Devices[0] : 24 (0@0K 1@0K 2@0K 3@0K 4@0K 5@0K 6@0K 7@0K 8@0K 9@0K 10@0K 11@0K 12@0K 13@0K 14@0K 15@0K 16@0K 17@0K 18@0K 19@0K 20@0K 21@0K 22@0K 23@0K)
Chunk Size[0] : 128 sectors
Raid Level[0] : RAID6
Device Size[0] : 782202880
Array Size[0] : 17208463360
Physical Disks : 1023
Number RefNo Size Device Type/State
0 1bfc5b4f 2930233816K /dev/dm-1 active/Online
1 44f3f4b1 2930233816K /dev/dm-2 active/Online
2 22f1c2e4 2930233816K /dev/dm-25 active/Online
3 ef32ccff 2930233816K /dev/dm-4 active/Online
4 c7e6f1fe 2930233816K /dev/dm-5 active/Online
5 00272b4d 2930233816K /dev/dm-6 active/Online
6 f961cba2 2930233816K /dev/dm-7 active/Online
7 40f01419 2930233816K /dev/dm-8 active/Online
8 75858e24 2930233816K /dev/dm-9 active/Online
9 ac398181 2930233816K /dev/dm-10 active/Online
10 b39d9cbb 2930233816K /dev/dm-11 active/Online
11 a71a4095 2930233816K /dev/dm-12 active/Online
12 f1bf38e9 2930233816K /dev/dm-13 active/Online
13 1a8973b2 2930233816K /dev/dm-14 active/Online
14 c107b1b5 2930233816K /dev/dm-15 active/Online
15 26b44a36 2930233816K /dev/dm-16 active/Online
16 7f376a5f 2930233816K /dev/dm-17 active/Online
17 22944f44 2930233816K /dev/dm-18 active/Online
18 8e356094 2930233816K /dev/dm-19 active/Online
19 0b454914 2930233816K /dev/dm-20 active/Online
20 71df5ccc 2930233816K /dev/dm-21 active/Online
21 763d65a1 2930233816K /dev/dm-22 active/Online
22 aacda00d 2930233816K /dev/dm-23 active/Online
23 9837ac03 2930233816K /dev/dm-24 active/Online
(picking a device at random)
# mdadm -E /dev/mapper/sdp
/dev/mapper/sdp:
Magic : de11de11
Version : 01.02.00
Controller GUID : 4C696E75:782D4D44:2064616D:2D73746F:72616765:2D6E3031
(Linux-MD dam-storage-n01)
Container GUID : 4C696E75:782D4D44:DEADBEEF:00000000:4410E200:C7DA314C
(Linux-MD 03/08/16 21:18:40)
Seq : 00000003
Redundant hdr : yes
Virtual Disks : 1
VD GUID[0] : 4C696E75:782D4D44:DEADBEEF:00000000:4410E257:2EF31542
(Linux-MD 03/08/16 21:20:07)
unit[0] : 1
state[0] : Optimal, Consistent
init state[0] : Fully Initialised
access[0] : Read/Write
Name[0] : 1
Raid Devices[0] : 24 (0@0K 1@0K 2@0K 3@0K 4@0K 5@0K 6@0K 7@0K 8@0K 9@0K 10@0K 11@0K 12@0K 13@0K 14@0K 15@0K 16@0K 17@0K 18@0K 19@0K 20@0K 21@0K 22@0K 23@0K)
Chunk Size[0] : 128 sectors
Raid Level[0] : RAID6
Device Size[0] : 782202880
Array Size[0] : 17208463360
Physical Disks : 1023
Number RefNo Size Device Type/State
0 1bfc5b4f 2930233816K active/Online
1 44f3f4b1 2930233816K active/Online
2 22f1c2e4 2930233816K active/Online
3 ef32ccff 2930233816K active/Online
4 c7e6f1fe 2930233816K active/Online
5 00272b4d 2930233816K active/Online
6 f961cba2 2930233816K active/Online
7 40f01419 2930233816K /dev/dm-8 active/Online
8 75858e24 2930233816K active/Online
9 ac398181 2930233816K active/Online
10 b39d9cbb 2930233816K active/Online
11 a71a4095 2930233816K active/Online
12 f1bf38e9 2930233816K active/Online
13 1a8973b2 2930233816K active/Online
14 c107b1b5 2930233816K active/Online
15 26b44a36 2930233816K active/Online
16 7f376a5f 2930233816K active/Online
17 22944f44 2930233816K active/Online
18 8e356094 2930233816K active/Online
19 0b454914 2930233816K active/Online
20 71df5ccc 2930233816K active/Online
21 763d65a1 2930233816K active/Online
22 aacda00d 2930233816K active/Online
23 9837ac03 2930233816K active/Online
# fdisk -l /dev/md1
Disk /dev/md1: 17621.5 GB, 17621466480640 bytes
2 heads, 4 sectors/track, -1 cylinders, total 34416926720 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 65536 bytes / 1441792 bytes
Disk identifier: 0x00000000
Disk /dev/md1 doesn't contain a valid partition table
# file -s /dev/md1
/dev/md1: LVM2 PV (Linux Logical Volume Manager), UUID: 1lz3cZ-j4Sj-ZTcH-GiEm-eXGi-qMSr-lIzwyS, size: 65999978102784
For reference an array of the same size, attached to the HW RAID controller
# fdisk -l /dev/sda
Disk /dev/sda: 66000.0 GB, 65999989637120 bytes
255 heads, 63 sectors/track, 8024041 cylinders, total 128906229760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000
Disk /dev/sda doesn't contain a valid partition table
> On Mar 7, 2016, at 4:50 PM, Andreas Klauer <Andreas.Klauer@metamorpher.de> wrote:
>
> On Sun, Mar 06, 2016 at 04:37:42PM -0500, Dan Russell wrote:
>> can md read a RAID6 created by the 2108, or would I be wasting my time?
>
> It might be possible.
>
> I don't use HW-RAID anywhere but if I was forced to, this would be
> one of the first things to determine, how to read it with software
> in case the card fails.
>
>> I can paste or gist the output of madam -E (304 lines) if it would help.
>
> I don't know if it would help, but paste everything you have anyhow.
>
> Is the data encrypted? If not - search for known file type headers
> (like a large jpeg or tar.gz, something larger than disks * chunksize),
> then look at the data on the other disks at the same offsets
> (grab a few megs of each disks), then try to deduce the layout
> and structure from that.
>
> Basically use known data to reverse-engineer the layout if you
> currently know nothing about disk orders, chunk sizes, etc.
>
> Regards
> Andreas Klauer
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Import/recover RAID6 created with SAS2108?
2016-03-09 3:20 ` Dan Russell
@ 2016-03-09 19:27 ` Andreas Klauer
2016-03-09 20:53 ` Dan Russell
2016-03-10 7:02 ` NeilBrown
1 sibling, 1 reply; 6+ messages in thread
From: Andreas Klauer @ 2016-03-09 19:27 UTC (permalink / raw)
To: Dan Russell; +Cc: linux-raid
On Tue, Mar 08, 2016 at 10:20:17PM -0500, Dan Russell wrote:
> The partially-rebuilt drive is sdk, the original “failed” drive is sdag
Best to leave both out if one has outdated and the other only half
the content...
> However fdisk and mdadm are reporting the array is 17.6TB in size, whereas it should be 66TB (24 3TB HDDs RAID6).
I reproduced your commands using tmpfs based loop devices and it gives me
the same problem. The RAID size is only 16 TiB. It seems to be hitting a
limit somewhere.
Your /dev/mapper/sdx are snapshot/overlays, I hope?
DDF metadata seems to be located at the end of the device, so you could try
your luck with mdadm 1.0 metadata instead; that gives me a RAID of a size
closer to home.
# mdadm --create /dev/md42 --assume-clean --metadata=1.0 --level=6 --raid-devices=24 --chunk=64 /dev/loop{0..23}
# fdisk -l /dev/md42
Disk /dev/md42: 60 TiB, 65999996846080 bytes, 128906243840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 1441792 bytes
It wastes some sectors at the end though, not sure if more or less
than what DDF uses for metadata. You might have to add some empty
space to your device mappings to get a full view.
Regards
Andreas Klauer
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Import/recover RAID6 created with SAS2108?
2016-03-09 19:27 ` Andreas Klauer
@ 2016-03-09 20:53 ` Dan Russell
0 siblings, 0 replies; 6+ messages in thread
From: Dan Russell @ 2016-03-09 20:53 UTC (permalink / raw)
To: linux-raid
On Mar 9, 2016, at 2:27 PM, Andreas Klauer <Andreas.Klauer@metamorpher.de> wrote:
>
> On Tue, Mar 08, 2016 at 10:20:17PM -0500, Dan Russell wrote:
>> The partially-rebuilt drive is sdk, the original “failed” drive is sdag
>
> Best to leave both out if one has outdated and the other only half
> the content…
I generally agree, but in my case the filesystem wasn’t mounted (commented out of fstab, movers dropped system, I booted system, RAID failed before I mounted the filesystem), so I’m OK with the risk.
>
>> However fdisk and mdadm are reporting the array is 17.6TB in size, whereas it should be 66TB (24 3TB HDDs RAID6).
>
> I reproduced your commands using tmpfs based loop devices and it gives me
> the same problem. The RAID size is only 16 TiB. It seems to be hitting a
> limit somewhere.
>
> Your /dev/mapper/sdx are snapshot/overlays, I hope?
Yes. I can’t recommend highly enough the overlay_setup approach on the Wiki.
> DDF metadata seems to be located at the end of the device, so you could try
> your luck with mdadm 1.0 metadata instead; that gives me a RAID of a size
> closer to home.
This got me closer, but the LVM2 label was missing. When I’d previously assembled the RAID in the container, I noticed algorithm 10; whereas with this approach it was 2. I switched it to 10 and my array is back. fsck (xfs_repair -n, really) says the FS is clean and random poking at files seems to back that up.
I have a backup, of course, but doing a disk-to-disk verify/recovery is going to be so much quicker.
Thank you so much for your help, Andreas and all the contributors to the “RAID_Recovery” and “Recovering_a_failed_software_RAID” Wiki pages.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Import/recover RAID6 created with SAS2108?
2016-03-09 3:20 ` Dan Russell
2016-03-09 19:27 ` Andreas Klauer
@ 2016-03-10 7:02 ` NeilBrown
1 sibling, 0 replies; 6+ messages in thread
From: NeilBrown @ 2016-03-10 7:02 UTC (permalink / raw)
To: Dan Russell, linux-raid
[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]
On Wed, Mar 09 2016, Dan Russell wrote:
> # mdadm --create /dev/md1 --assume-clean --level=6 --raid-devices=24
--chunk=64 --size=2929686528
22 * 2929686528 == 0xF01B45000
> # cat /proc/mdstat
> Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
> md1 : active (auto-read-only) raid6 dm-24[23] dm-23[22].....
> 17208463360 blocks super external:/md127/0 level 6, 64k chunk,....
17208463360 == 0x401B45000
So the size hasn't just be truncated to to some number of bits, but the
5th byte has changed from 0xF to 0x4.
That is very odd....
Ahhh.. I tried it myself, and used "mdadm -D" to look at the RAID6
array.
Used Dev Size : 782202880 (745.97 GiB 800.98 GB)
The requested per-device size was
2929686528 = 0xAE9F7800
the size given was
782202880 = 0x2E9F7800
so we lost the msbit there... Ahhhh.
diff --git a/super-ddf.c b/super-ddf.c
index faaf0a7ca9e0..0e00d17dd169 100644
--- a/super-ddf.c
+++ b/super-ddf.c
@@ -2688,10 +2688,10 @@ static int init_super_ddf_bvd(struct supertype *st,
free(vcl);
return 0;
}
- vc->blocks = cpu_to_be64(info->size * 2);
+ vc->blocks = cpu_to_be64(size * 2);
vc->array_blocks = cpu_to_be64(
calc_array_size(info->level, info->raid_disks, info->layout,
- info->chunk_size, info->size*2));
+ info->chunk_size, size*2));
memset(vc->pad1, 0xff, 8);
vc->spare_refs[0] = cpu_to_be32(0xffffffff);
vc->spare_refs[1] = cpu_to_be32(0xffffffff);
That was careless. "info" is a legacy structure which has a 32bit size
field. So a 64bit size if passed as a separate arg, but this function
used the wrong one :-(
I'll send off a patch.
Thanks for the report - and glad you could get at your data :-)
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-03-10 7:02 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-03-06 21:37 Import/recover RAID6 created with SAS2108? Dan Russell
2016-03-07 21:50 ` Andreas Klauer
2016-03-09 3:20 ` Dan Russell
2016-03-09 19:27 ` Andreas Klauer
2016-03-09 20:53 ` Dan Russell
2016-03-10 7:02 ` NeilBrown
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).