From: Louis-David Mitterrand <vindex+lists-linux-raid@apartia.org>
To: linux-raid@vger.kernel.org
Subject: Re: dm-crypt over raid6 unreadable after crash
Date: Thu, 7 Jul 2011 11:05:40 +0200 [thread overview]
Message-ID: <20110707090540.GA7288@apartia.fr> (raw)
In-Reply-To: <4E1494BB.9060101@turmel.org>
On Wed, Jul 06, 2011 at 01:00:43PM -0400, Phil Turmel wrote:
> > After a hardware crash I can no longer open a dm-crypt partition located
> > directly over a md-raid6 partition. I get this error:
> >
> > root@grml ~ # cryptsetup isLuks /dev/md1
> > Device /dev/md1 is not a valid LUKS device
> >
> > It seems the LUKS header has been shifted a few bytes forward, but looks
> > otherwise fine to specialists on the dm-crypt mailing list. Normally the
> > "LUKS" signature should be at 0x00000000
> >
> > Is there some way that the md layer could have shifted its contents?
> >
> > Here is a hexdum of /dev/md1 done with "hd /dev/md1 | head -n 40"
> >
> > 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> > *
> > 00100000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00 |LUKS....aes.....|
> > 00100010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> > 00100020 00 00 00 00 00 00 00 00 63 62 63 2d 65 73 73 69 |........cbc-essi|
> > 00100030 76 3a 73 68 61 32 35 36 00 00 00 00 00 00 00 00 |v:sha256........|
Hi Phil,
> The offset is precisely 1MB. This is the default data offset for
> metadata types 1.1 and 1.2 (nowadays). Metadata types 0.90 and 1.0
> have a zero offset (the metadata is at the end.)
>
> You don't say what your recovery efforts were, but I'd guess you did a
> "mdadm --create" somewhere in there, and didn't match the original
> parameters. Or you used an older version of mdadm than was used
> originally, and therefore got different defaults.
No I did a mdadm-startall with a grml livecd.
> Another possibility is that the original array was set up on a 1MB
> aligned partition, and the array is now using the whole device. This
> can happen with v0.90 metadata. If so, the original partition table
> is obviously zeroed out now.
>
> Please share more information about what you've done so far. Also
Nothing appart from assembling the array and failing to decrypt it with
cryptsetup.
> show us the output of "mdadm -D /dev/md1"
/dev/md1:
Version : 1.2
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Array Size : 841863168 (802.86 GiB 862.07 GB)
Used Dev Size : 140310528 (133.81 GiB 143.68 GB)
Raid Devices : 8
Total Devices : 8
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Thu Jul 7 09:44:49 2011
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : grml:1 (local to host grml)
UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Events : 8292
Number Major Minor RaidDevice State
0 8 130 0 active sync /dev/sdi2
1 8 50 1 active sync /dev/sdd2
2 8 34 2 active sync /dev/sdc2
3 8 82 3 active sync /dev/sdf2
4 8 66 4 active sync /dev/sde2
5 8 146 5 active sync /dev/sdj2
8 8 114 6 active sync /dev/sdh2
7 8 98 7 active sync /dev/sdg2
> and then "mdadm -E /dev/xxx" for each of its components.
/dev/sdc2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 0a10d6c3:8a6f1948:f1a546a4:32f10094
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : 16c1099b - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdd2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 5f8c46cb:614354cf:dd91f7c2:f1260b2e
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : 5e277b71 - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sde2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : ab27b114:ea95aa0a:9adb310b:c456ee56
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : 5405e2d7 - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdf2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 31bc8c85:7b754501:ea0b713e:2714810a
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : a24e44a0 - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdg2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 4db23afd:d4422390:e39d701e:7223cc9e
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : 1d24a95f - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 7
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdh2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 1f220bf2:1c86fc2b:0e99f2d2:8283497c
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : 3fcdb7b5 - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 6
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdi2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 6acb87a4:3ac53237:1f5fff58:3611a0b0
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : b7e3f3da - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAAAA ('A' == active, '.' == missing)
/dev/sdj2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1434a46a:f2b751cd:8604803c:b545de8c
Name : grml:1 (local to host grml)
Creation Time : Wed Oct 20 21:40:40 2010
Raid Level : raid6
Raid Devices : 8
Avail Dev Size : 280621372 (133.81 GiB 143.68 GB)
Array Size : 1683726336 (802.86 GiB 862.07 GB)
Used Dev Size : 280621056 (133.81 GiB 143.68 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 1515f5a7:b0c78638:8ca1d918:f1fa47d7
Internal Bitmap : 2 sectors from superblock
Update Time : Thu Jul 7 11:00:42 2011
Checksum : a3276c28 - correct
Events : 8292
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAAAAAAA ('A' == active, '.' == missing)
> The output of "lsdrv"[1] would also be useful for visualizing your setup.
PCI [ata_piix] 00:1f.2 IDE interface: Intel Corporation 82801IB (ICH9) 2 port SATA IDE Controller (rev 02)
├─scsi 0:0:0:0 HL-DT-ST DVD+-RW GH50N {K1LA7D41849}
│ └─sr0: [11:0] Partitioned (dos) 224.00m 'grml64-medium_2011.05'
│ └─Mounted as /dev/sr0 @ /live/image
└─scsi 1:x:x:x [Empty]
PCI [mpt2sas] 02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 02)
├─scsi 2:0:0:0 ATA WDC WD1002FAEX-0 {WD-WCATR1851552}
│ └─sdc: [8:32] Partitioned (dos) 931.51g
│ ├─sdc1: [8:33] MD raid1 (2/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ │ └─md0: [9:0] Partitioned (dos) 250.88m {d1d876e9-6905-4940-bf55-7cdb4b64484f}
│ ├─sdc2: [8:34] MD raid6 (2/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ │ └─md1: [9:1] Empty/Unknown 802.86g
│ └─sdc3: [8:35] MD raid6 (0/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
│ └─md2: [9:2] (crypto_LUKS) 4.67t {1d30a244-9d40-48e8-925a-1d6c93a45474}
│ └─dm-0: [253:0] (xfs) 4.67t {3cad63a0-a586-43e0-bf89-5be9066c884f}
│ └─Mounted as /dev/mapper/cmd2 @ /backup
├─scsi 2:0:1:0 ATA WDC WD1002FAEX-0 {WD-WCATR2968402}
│ └─sdd: [8:48] Partitioned (dos) 931.51g
│ ├─sdd1: [8:49] MD raid1 (3/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdd2: [8:50] MD raid6 (1/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdd3: [8:51] MD raid6 (5/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:2:0 ATA WDC WD1002FAEX-0 {WD-WCATR1851573}
│ └─sde: [8:64] Partitioned (dos) 931.51g
│ ├─sde1: [8:65] MD raid1 (7/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sde2: [8:66] MD raid6 (4/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sde3: [8:67] MD raid6 (6/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:3:0 ATA WDC WD1002FAEX-0 {WD-WCATR3005506}
│ └─sdf: [8:80] Partitioned (dos) 931.51g
│ ├─sdf1: [8:81] MD raid1 (0/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdf2: [8:82] MD raid6 (3/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdf3: [8:83] MD raid6 (4/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:4:0 ATA WDC WD1002FAEX-0 {WD-WCATR3007070}
│ └─sdg: [8:96] Partitioned (dos) 931.51g
│ ├─sdg1: [8:97] MD raid1 (6/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdg2: [8:98] MD raid6 (7/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdg3: [8:99] MD raid6 (3/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:5:0 ATA WDC WD1002FAEX-0 {WD-WCATR3004862}
│ └─sdh: [8:112] Partitioned (dos) 931.51g
│ ├─sdh1: [8:113] MD raid1 (4/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdh2: [8:114] MD raid6 (6/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdh3: [8:115] MD raid6 (1/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:6:0 ATA WDC WD1002FAEX-0 {WD-WCATR2969087}
│ └─sdi: [8:128] Partitioned (dos) 931.51g
│ ├─sdi1: [8:129] MD raid1 (1/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdi2: [8:130] MD raid6 (0/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdi3: [8:131] MD raid6 (7/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
├─scsi 2:0:7:0 ATA WDC WD1002FAEX-0 {WD-WCATR2984361}
│ └─sdj: [8:144] Partitioned (dos) 931.51g
│ ├─sdj1: [8:145] MD raid1 (5/8) 250.98m md0 clean in_sync {2871f814-ceb7-6a88-d8b7-8f6599226e41}
│ ├─sdj2: [8:146] MD raid6 (5/8) 133.81g md1 clean in_sync 'grml:1' {1434a46a-f2b7-51cd-8604-803cb545de8c}
│ └─sdj3: [8:147] MD raid6 (2/8) 797.36g md2 clean in_sync 'zenon:2' {5c037ba3-ca4b-f7b9-eb8f-b01608e1fd3b}
└─scsi 2:x:x:x [Empty]
USB [usb-storage] Bus 002 Device 004: ID 0624:0249 Avocent Corp. {20080519}
└─scsi 3:0:0:0 iDRAC LCDRIVE
└─sda: [8:0] Empty/Unknown 0.00k
USB [usb-storage] Bus 002 Device 004: ID 0624:0249 Avocent Corp. {20080519}
├─scsi 4:0:0:0 iDRAC Virtual CD
│ └─sr1: [11:1] Empty/Unknown 1.00g
└─scsi 4:0:0:1 iDRAC Virtual Floppy
└─sdb: [8:16] Empty/Unknown 0.00k
Other Block Devices
└─loop0: [7:0] (squashfs) 199.44m
└─Mounted as /dev/loop0 @ /grml64-medium.squashfs
> Regards,
Thanks,
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-07-07 9:05 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-07-06 16:12 dm-crypt over raid6 unreadable after crash Louis-David Mitterrand
2011-07-06 17:00 ` Phil Turmel
2011-07-07 9:05 ` Louis-David Mitterrand [this message]
2011-07-07 12:41 ` Phil Turmel
2011-07-07 13:09 ` Louis-David Mitterrand
2011-07-07 13:32 ` Phil Turmel
2011-07-08 5:40 ` Luca Berra
2011-07-10 12:03 ` Louis-David Mitterrand
2011-07-10 12:28 ` Phil Turmel
2011-07-10 23:44 ` Stan Hoeppner
2011-07-11 7:57 ` Louis-David Mitterrand
2011-07-11 11:49 ` Stan Hoeppner
2011-07-11 7:54 ` Louis-David Mitterrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110707090540.GA7288@apartia.fr \
--to=vindex+lists-linux-raid@apartia.org \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.